## Student’s t-Distribution

Where does “Student’s t-distribution” come from? Well, suppose that $X\sim N(0,1)$  is a standard normal random variable and $Z\sim\chi_{m}^{2}$ is a chi-squared random variable with $m$ degrees of freedom. In addition, suppose that $X$ and $Z$ are independent. Define a random variable $Y$ by

$\displaystyle Y:=\sqrt{\dfrac{Z}{m}}$

We call the distribution of the quotient random variable $\frac{X}{Y}$ the $t$-distribution with $m$ degrees of freedom. Its distribution is absolutely continuous with respect to the Lebesgue measure, and we calculate its probability density function (pdf) now. Recall that $Z$ has pdf

$\displaystyle\dfrac{e^{-\frac{z}{2}}z^{\frac{m}{2}-1}}{2^{\frac{m}{2}}\Gamma(\frac{m}{2})},\indent\forall z>0$

Using the calculus, we obtain the pdf of $Y$:

$\begin{array}{lcl}\displaystyle f_{Y}(y)=\dfrac{\partial}{\partial y}F_{Y}(y)=\dfrac{\partial}{\partial y}F_{Z}(my^{2})&=&\displaystyle2myf_{Z}(my^{2})\\[1 em]&=&2m\cdot y\cdot\dfrac{e^{-\frac{my^{2}}{2}}(my^{2})^{\frac{m}{2}-1}}{2^{\frac{m}{2}}\Gamma(\frac{m}{2})}\\[1 em]&=&\dfrac{m^{\frac{m}{2}}y^{m-1}e^{-\frac{my^{2}}{2}}}{2^{\frac{m}{2}-1}\Gamma(\frac{m}{2})}\end{array}$

for all $y>0$. Since $X$ and $Y$ are independent, their joint density function is simply the product of the marginal density functions. Hence,

$\begin{array}{lcl}\displaystyle f_{V}(v)=\int_{-\infty}^{\infty}\left|y\right|f_{X,Y}(vy,y)dy&=&\displaystyle\int_{0}^{\infty}\left|y\right|\left(\dfrac{1}{\sqrt{2\pi}}e^{-\frac{(vy)^{2}}{2}}\right)\left(\dfrac{m^{\frac{m}{2}}y^{m-1}e^{-\frac{my^{2}}{2}}}{2^{\frac{m}{2}-1}\Gamma(\frac{m}{2})}\right)\\[2 em]&=&\displaystyle\left(\dfrac{m}{2}\right)^{\frac{m}{2}}\dfrac{2}{\sqrt{2\pi}\Gamma(\frac{m}{2})}\int_{0}^{\infty}y^{m}e^{-\frac{(m+v^{2})y^{2}}{2}}dy\\[2 em]&=&\displaystyle\left(\dfrac{m}{2}\right)^{\frac{m}{2}}\dfrac{2}{\sqrt{2\pi}\Gamma(\frac{m}{2})}\int_{0}^{\infty}y^{m}e^{-\frac{(m+v^{2})y^{2}}{2}}dy\end{array}$

Making the change of variable $u=2^{-1}(m+v^{2})y^{2}$, we conclude that

$\begin{array}{lcl}\displaystyle f_{V}(v)&=&\displaystyle\dfrac{m^{\frac{m}{2}}}{2^{\frac{m-1}{2}}\sqrt{\pi}\Gamma(\frac{m}{2})}\cdot\dfrac{1}{(m+v^{2})}\int_{0}^{\infty}\left(\dfrac{2}{(m+v^{2})}u\right)^{\frac{m-1}{2}}e^{-u}du\\[2 em]&=&\displaystyle\dfrac{m^{\frac{m}{2}}}{\sqrt{\pi}\Gamma(\frac{m}{2})}\cdot\dfrac{\Gamma(\frac{m+1}{2})}{(m+v^{2})^{\frac{m+1}{2}}}\\[2 em]&=&\displaystyle\dfrac{\Gamma(\frac{m+1}{2})}{\sqrt{m\pi}\Gamma(\frac{m}{2})}\left(1+\dfrac{v^{2}}{m}\right)^{-\frac{m+1}{2}}\end{array}$

Statistics textbooks–particularly, bad ones–state that the normal distribution approximates the t-distribution for sufficiently large degrees of freedom; the quantity behind sufficiently varies from author to author. I’m more interested in mathematical proofs than taking an author’s word, so let’s examine for ourselves the limiting behavior of the t-distribution as $m\rightarrow\infty$.

Using the below version of Stirling’s formula,

$\displaystyle\Gamma(s)=\sqrt{2\pi}s^{s-\frac{1}{2}}e^{-s}\left(1+O(\left|s\right|^{-1})\right),\indent\forall\text{Re}(s)>0$

we see that

$\begin{array}{lcl}\displaystyle\lim_{m\rightarrow\infty}\dfrac{\Gamma(\frac{m+1}{2})}{\sqrt{m\pi}\Gamma(\frac{m}{2})}&=&\displaystyle\lim_{m\rightarrow\infty}\dfrac{\sqrt{2\pi}\left(\frac{m+1}{2}\right)^{\frac{m+1}{2}-\frac{1}{2}}e^{-\frac{m+1}{2}}}{\sqrt{2\pi}\cdot\sqrt{m\pi}\left(\frac{m}{2}\right)^{\frac{m}{2}-\frac{1}{2}}e^{-\frac{m}{2}}}\\[2 em]&=&\displaystyle\lim_{m\rightarrow\infty}\left(\dfrac{m+1}{m}\right)^{\frac{m}{2}}\cdot\dfrac{\sqrt{m}}{\sqrt{m\pi}\cdot\sqrt{2}\sqrt{e}}\\[2 em]&=&\displaystyle\dfrac{1}{\sqrt{2\pi e}}\cdot\left(\lim_{m\rightarrow\infty}\left(1+\dfrac{1}{m}\right)^{m}\right)^{\frac{1}{2}}\\[2 em]&=&\displaystyle\dfrac{1}{\sqrt{2\pi e}}\cdot\sqrt{e} \\[2 em]&=&\displaystyle\dfrac{1}{\sqrt{2\pi}}\end{array}$

where we use the sequence limit definition of $e$ and the continuity of the exponential function.

$\begin{array}{lcl}\displaystyle\left(1+\dfrac{v^{2}}{m}\right)^{-\frac{m+1}{2}}=\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{-\frac{m}{v^{2}}\cdot\frac{v^{2}}{m}\cdot\frac{m+1}{2}}=\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\end{array}$

Since $\frac{m+1}{m}\rightarrow 1$ as $m\rightarrow\infty$, we see that, for any $\varepsilon>0$,

$\displaystyle\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-(1-\varepsilon)\frac{v^{2}}{2}}\leq\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\leq\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{v^{2}}{2}}$

Letting $m\rightarrow\infty$, we see that

$\begin{array}{lcl}\displaystyle e^{-(1-\varepsilon)\frac{v^{2}}{2}}&\leq&\displaystyle\liminf_{m\rightarrow\infty}\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\\[2 em]&\leq&\displaystyle\limsup_{m\rightarrow\infty}\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\\[2 em]&\leq&\displaystyle e^{-\frac{v^{2}}{2}}\end{array}$

The preceding inequality holds for arbitrarily small $\varepsilon$, so we conclude that

$\displaystyle\lim_{m\rightarrow\infty}\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}=e^{-\frac{v^{2}}{2}}$

Putting these results together, for fixed $v\in\mathbb{R}$,

$\displaystyle\lim_{m\rightarrow\infty}\dfrac{\Gamma(\frac{m+1}{2})}{\sqrt{m\pi}\Gamma(\frac{m}{2})}\left(1+\dfrac{v^{2}}{m}\right)^{-\frac{m+1}{2}}=\dfrac{1}{\sqrt{2\pi}}e^{-\frac{v^{2}}{2}}$,

where the right-hand side is the density function of the standard normal distribution.

This entry was posted in math.PR, math.ST and tagged , , , , , . Bookmark the permalink.