Standard error

The standard error is a metric of error for a Scalar estimator. Given an estimator θ^\hat{\theta}, it is defined as

SE(θ^)=var(θ^)\text{SE}(\hat{\theta})=\sqrt{ \text{var}(\hat{\theta}) }

It is a more convenient error metric than simple Variance because it has the same units as the estimator. Once a sample of NN elements is collected and a numerical estimate θ\theta is obtained, the estimated standard error is obtained by replacing θ^\hat{\theta} with θ\theta. A common related quantity is the standard error of the sample mean

SE(Xˉ)=σN\text{SE}(\bar{X})=\frac{\sigma}{\sqrt{ N }}

where σ\sigma is the standard deviation.

Delta method

Suppose that we are interested in a parameter which is a function of a scalar parameter θ\theta, namely

ψ=g(θ)\psi=g(\theta)

where gg is some continuous differentiable function. In this situation, we can apply the continuous mapping theorem.

The standard error of ψ\psi is approximately provided by the so-called delta method, which states

SE(ψ^)≃SE(θ^)∣dg(θ)dθ∣\text{SE}(\hat{\psi})\simeq\text{SE}(\hat{\theta})\left\lvert \frac{dg(\theta)}{d\theta} \right\rvert

The approximation improves as the sample size NN gets larger.

> We know that $\psi=g(\theta)$ and that its estimator is $\hat{\psi}=g(\hat{\theta})=g(T)$. We want to know $\hat{\psi}$. We'll use a [[Taylor series]] expansion in $t=\theta$ and truncate to first order: > $$g(T)\simeq g(\theta)+g'(\theta)(T-\theta)

We can reorder this to read

> and so > $$\hat{\psi}=g(T)\approx

(Finish this 10/10/2025)