Student’s t-Distribution

Where does “Student’s t-distribution” come from? Well, suppose that X\sim N(0,1)  is a standard normal random variable and Z\sim\chi_{m}^{2} is a chi-squared random variable with m degrees of freedom. In addition, suppose that X and Z are independent. Define a random variable Y by

\displaystyle Y:=\sqrt{\dfrac{Z}{m}}

We call the distribution of the quotient random variable \frac{X}{Y} the t-distribution with m degrees of freedom. Its distribution is absolutely continuous with respect to the Lebesgue measure, and we calculate its probability density function (pdf) now. Recall that Z has pdf

\displaystyle\dfrac{e^{-\frac{z}{2}}z^{\frac{m}{2}-1}}{2^{\frac{m}{2}}\Gamma(\frac{m}{2})},\indent\forall z>0

Using the calculus, we obtain the pdf of Y:

\begin{array}{lcl}\displaystyle f_{Y}(y)=\dfrac{\partial}{\partial y}F_{Y}(y)=\dfrac{\partial}{\partial y}F_{Z}(my^{2})&=&\displaystyle2myf_{Z}(my^{2})\\[1 em]&=&2m\cdot y\cdot\dfrac{e^{-\frac{my^{2}}{2}}(my^{2})^{\frac{m}{2}-1}}{2^{\frac{m}{2}}\Gamma(\frac{m}{2})}\\[1 em]&=&\dfrac{m^{\frac{m}{2}}y^{m-1}e^{-\frac{my^{2}}{2}}}{2^{\frac{m}{2}-1}\Gamma(\frac{m}{2})}\end{array}

for all y>0. Since X and Y are independent, their joint density function is simply the product of the marginal density functions. Hence,

\begin{array}{lcl}\displaystyle f_{V}(v)=\int_{-\infty}^{\infty}\left|y\right|f_{X,Y}(vy,y)dy&=&\displaystyle\int_{0}^{\infty}\left|y\right|\left(\dfrac{1}{\sqrt{2\pi}}e^{-\frac{(vy)^{2}}{2}}\right)\left(\dfrac{m^{\frac{m}{2}}y^{m-1}e^{-\frac{my^{2}}{2}}}{2^{\frac{m}{2}-1}\Gamma(\frac{m}{2})}\right)\\[2 em]&=&\displaystyle\left(\dfrac{m}{2}\right)^{\frac{m}{2}}\dfrac{2}{\sqrt{2\pi}\Gamma(\frac{m}{2})}\int_{0}^{\infty}y^{m}e^{-\frac{(m+v^{2})y^{2}}{2}}dy\\[2 em]&=&\displaystyle\left(\dfrac{m}{2}\right)^{\frac{m}{2}}\dfrac{2}{\sqrt{2\pi}\Gamma(\frac{m}{2})}\int_{0}^{\infty}y^{m}e^{-\frac{(m+v^{2})y^{2}}{2}}dy\end{array}

Making the change of variable u=2^{-1}(m+v^{2})y^{2}, we conclude that

\begin{array}{lcl}\displaystyle f_{V}(v)&=&\displaystyle\dfrac{m^{\frac{m}{2}}}{2^{\frac{m-1}{2}}\sqrt{\pi}\Gamma(\frac{m}{2})}\cdot\dfrac{1}{(m+v^{2})}\int_{0}^{\infty}\left(\dfrac{2}{(m+v^{2})}u\right)^{\frac{m-1}{2}}e^{-u}du\\[2 em]&=&\displaystyle\dfrac{m^{\frac{m}{2}}}{\sqrt{\pi}\Gamma(\frac{m}{2})}\cdot\dfrac{\Gamma(\frac{m+1}{2})}{(m+v^{2})^{\frac{m+1}{2}}}\\[2 em]&=&\displaystyle\dfrac{\Gamma(\frac{m+1}{2})}{\sqrt{m\pi}\Gamma(\frac{m}{2})}\left(1+\dfrac{v^{2}}{m}\right)^{-\frac{m+1}{2}}\end{array}

Statistics textbooks–particularly, bad ones–state that the normal distribution approximates the t-distribution for sufficiently large degrees of freedom; the quantity behind sufficiently varies from author to author. I’m more interested in mathematical proofs than taking an author’s word, so let’s examine for ourselves the limiting behavior of the t-distribution as m\rightarrow\infty.

Using the below version of Stirling’s formula,

\displaystyle\Gamma(s)=\sqrt{2\pi}s^{s-\frac{1}{2}}e^{-s}\left(1+O(\left|s\right|^{-1})\right),\indent\forall\text{Re}(s)>0

we see that

\begin{array}{lcl}\displaystyle\lim_{m\rightarrow\infty}\dfrac{\Gamma(\frac{m+1}{2})}{\sqrt{m\pi}\Gamma(\frac{m}{2})}&=&\displaystyle\lim_{m\rightarrow\infty}\dfrac{\sqrt{2\pi}\left(\frac{m+1}{2}\right)^{\frac{m+1}{2}-\frac{1}{2}}e^{-\frac{m+1}{2}}}{\sqrt{2\pi}\cdot\sqrt{m\pi}\left(\frac{m}{2}\right)^{\frac{m}{2}-\frac{1}{2}}e^{-\frac{m}{2}}}\\[2 em]&=&\displaystyle\lim_{m\rightarrow\infty}\left(\dfrac{m+1}{m}\right)^{\frac{m}{2}}\cdot\dfrac{\sqrt{m}}{\sqrt{m\pi}\cdot\sqrt{2}\sqrt{e}}\\[2 em]&=&\displaystyle\dfrac{1}{\sqrt{2\pi e}}\cdot\left(\lim_{m\rightarrow\infty}\left(1+\dfrac{1}{m}\right)^{m}\right)^{\frac{1}{2}}\\[2 em]&=&\displaystyle\dfrac{1}{\sqrt{2\pi e}}\cdot\sqrt{e} \\[2 em]&=&\displaystyle\dfrac{1}{\sqrt{2\pi}}\end{array}

where we use the sequence limit definition of e and the continuity of the exponential function.

\begin{array}{lcl}\displaystyle\left(1+\dfrac{v^{2}}{m}\right)^{-\frac{m+1}{2}}=\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{-\frac{m}{v^{2}}\cdot\frac{v^{2}}{m}\cdot\frac{m+1}{2}}=\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\end{array}

Since \frac{m+1}{m}\rightarrow 1 as m\rightarrow\infty, we see that, for any \varepsilon>0,

\displaystyle\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-(1-\varepsilon)\frac{v^{2}}{2}}\leq\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\leq\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{v^{2}}{2}}

Letting m\rightarrow\infty, we see that

\begin{array}{lcl}\displaystyle e^{-(1-\varepsilon)\frac{v^{2}}{2}}&\leq&\displaystyle\liminf_{m\rightarrow\infty}\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\\[2 em]&\leq&\displaystyle\limsup_{m\rightarrow\infty}\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}\\[2 em]&\leq&\displaystyle e^{-\frac{v^{2}}{2}}\end{array}

The preceding inequality holds for arbitrarily small \varepsilon, so we conclude that

\displaystyle\lim_{m\rightarrow\infty}\left[\left(1+\dfrac{1}{\frac{m}{v^{2}}}\right)^{\frac{m}{v^{2}}}\right]^{-\frac{(m+1)v^{2}}{2m}}=e^{-\frac{v^{2}}{2}}

Putting these results together, for fixed v\in\mathbb{R},

\displaystyle\lim_{m\rightarrow\infty}\dfrac{\Gamma(\frac{m+1}{2})}{\sqrt{m\pi}\Gamma(\frac{m}{2})}\left(1+\dfrac{v^{2}}{m}\right)^{-\frac{m+1}{2}}=\dfrac{1}{\sqrt{2\pi}}e^{-\frac{v^{2}}{2}},

where the right-hand side is the density function of the standard normal distribution.

Advertisements
This entry was posted in math.PR, math.ST and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s