Gaussian distribution

The Gaussian distribution or normal distribution is a real univariate continuous Probability distribution. For a Random variable XX, the Probability density function is

fX(x;μ,σ2)=12πσ2e−(x−μ)22σ2f_{X}(x;\mu,\sigma ^{2})=\frac{1}{\sqrt{ 2\pi \sigma^{2} }}e^{- \frac{(x-\mu)^{2}}{2\sigma ^{2}}}

where μ\mu is the Expected value e σ2\sigma ^{2} is the Variance.

A standard normal distribution is defined as a normal distribution with μ=0\mu=0 and σ2=1\sigma ^{2}=1:

fX(x;0,1)=12πe−x2/2f_{X}(x;0,1)=\frac{1}{\sqrt{ 2\pi }}e^{-x^{2}/2}

Moments

The central and raw Moment-generating function for the Gaussian are

MX(t)=E[et(X−μ)]=eσ2t2/2,MX∗(t)=etμMX(t)=etμ+t2σ2/2M_{X}(t)=\text{E}[e^{t(X-\mu)}]=e^{\sigma ^{2}t^{2}/2},\qquad M_{X}^{*}(t)=e^{t\mu}M_{X}(t)=e^{t\mu+t^{2}\sigma ^{2}/2}

For a standard normal distribution, they simplify to

MX(t)=et2/2=MX∗(t)M_{X}(t)=e^{t^{2}/2}=M_{X}^{*}(t) > Combine exponentials: > $$e^{t(x-\mu)}e^{-(x-\mu)^{2}/2\sigma ^{2}}=e^{t(x-\mu)-(x-\mu)^{2}/2\sigma ^{2}}

Use the following identity:

t(x-\mu)-\frac{(x-\mu)^{2}}{2\sigma ^{2}}&=t(x-\mu)- \frac{(x-\mu)^{2}}{2\sigma ^{2}}+ \frac{\sigma ^{2}t^{2}}{2}- \frac{\sigma ^{2}t^{2}}{2} \\ &=\left[- \frac{(x-\mu)^{2}}{2\sigma ^{2}}+ t(x-\mu)- \frac{\sigma ^{2}t^{2}}{2} \right]+ \frac{\sigma ^{2}t^{2}}{2} \\ \left( \text{extract } \frac{-1}{2\sigma ^{2}} \right)&=- \frac{1}{2\sigma ^{2}}[(x-\mu)^{2}- 2\sigma ^{2}t(x-\mu)+\sigma^{4}t^{2}]+ \frac{\sigma ^{2}t^{2}}{2} \\ (\text{recognize square})&=- \frac{1}{2\sigma ^{2}}[(x-\mu)-\sigma ^{2}t]^{2}+ \frac{\sigma ^{2}t^{2}}{2} \\ &=\frac{\sigma ^{2}t^{2}}{2}- \frac{(x-\mu+\sigma ^{2}t)^{2}}{2\sigma ^{2}} \end{align}
> Substitute in the integral: > $$M_{X}(t)=\frac{1}{\sqrt{ 2\pi }\sigma}e^{\sigma ^{2}t^{2}/2}\int_{-\infty}^{+\infty}e^{-(x-\mu-\sigma ^{2}t)^{2}/2\sigma ^{2}}dx

This is a Gaussian integral with a=1/2σ2a=1/2\sigma ^{2} and b=−μ−σ2tb=-\mu-\sigma ^{2}t. Gaussian integrals are equal to π/a\sqrt{ \pi/a } so

> The raw MGF follows immediately by > $$M_{X}^{*}(t)=e^{t\mu}M_{X}(t)=e^{t\mu+\sigma ^{2}t^{2}/2}

For a standard normal N(0,1)\mathcal{N}(0,1) we then have

Some [[Function moments|moments]] are: - Raw 0. $\mu_{0}^{*}=1$ 1. $\mu_{1}^{*}=\mu$ ([[Expected value]]) - Central 0. $\mu_{0}=1$ 1. $\mu_{1}=0$ 2. $\mu_{2}=\sigma ^{2}$ ([[Variance]]) 3. $\mu_{3}=0$ 4. $\mu_{4}=3\sigma^{4}$ - Coefficients 1. $\gamma_{1}=0$ ([[skewness]], it is symmetrical around the mean) 2. $\gamma_{2}=0$ ([[kurtosis]]) These moments have particular significance, as the Gaussian distribution is the gold standard of distributions. It is extremely common, well-understood and well-behaved, so other distributions and their moments are frequently compared to it to get an idea of how they behave. For kurtosis in particular, negative values can be seen as "flatter than Gaussian" and positive ones as "more peaked than Gaussian." ### As sum of normal variables The sum of $N$ [[iid]] normal variables $X_{i}\sim \mathcal{N}(\mu,\sigma ^{2})$ is itself a normal distribution:

Y=\sum_{i=1}^{N} X_{i}\sim \mathcal{N}(N\mu,N\sigma ^{2})

> [!quote]- Proof > The joint MGF of a sum of [[independent variables]] is the product of MGFs: > $$M_{Y}(t)=\prod_{i=1}^{N} M_{X_{i}}(t)=\prod_{i=1}^{N} e^{t\mu+t^{2}\sigma ^{2}/2}=e^{tN\mu+t^{2}N\sigma ^{2}/2}

But this is the MGF of a Gaussian with mean NμN\mu and variance Nσ2N\sigma ^{2}. Thus Y∼N(Nμ,Nσ2)Y\sim \mathcal{N}(N\mu,N\sigma ^{2}).