As promised in my last post, we will use the result that the only solutions to the functional equation
are of the form to show that the only continuous distribution with the so-called “memorylessness” property is the exponential distribution. Even though the analogous result for discrete distributions does not make use of the results of my last post, it is instructive to begin by showing that the geometric distribution is the unique memoryless discrete distribution.
Recall that a random variable is said to be geometrically distributed with parameter , which we denote by , if its probability mass function is of the form
If we have a collection of independent Bernoulli trials with probability , the geometric distribution models the number of “failures” before the first “success” is achieved.
If , where , then for any integers ,
This identity is referred to as the “memorylessness” property of the Geometric distribution. If we have observed at least failures, then the probability that we observe more than failures before achieving the first success is just the probability that we observe at least more failures before achieving the first success. Set . Then
Suppose we have shown that , for all . Then
So is necessarily geometrically distributed with parameter .
A random variable is said to be exponentially distributed with parameter , which we denote by , if it has a probability density function of the form
The memorylessness property refers to the following result for :
If denotes the time until a mechanical device fails (e.g. a flashlight), then the memorylessness property tells us that the probability that the device lasts more than 2 years, given that it’s lasted more than 1 is year, is just the probability that it lasts for at least another year.
Now suppose that is an a.s. finite random variable with probability distribution such that singletons have probability zero, exists for all and
Define the survival function by . Since the cummulative distribution function is right-continuous, it follows that is also right-continuous, hence measurable. I claim that for all . Set . by right-continuity and
which is a contradiction. By the definition of conditional probability, we see that satisfies the functional equation for all . Hence satisfies the functional equation
By our uniqueness result, there exists such that . Since is monotonically nonincreasing and , we see that . Hence,
where . This is precisely the survival function of a random variable .