Sum of independent exponential random variables

For Chiara,

who once encouraged me

to boldly keep trying

Introduction

For the last four months, I have experienced the worst level of my illness: I have been completely unable to think for most of the time. So I could do nothing but hanging in there, waiting for a miracle, passing from one medication to the other, well aware that this state could have lasted for years, with no reasonable hope of receiving help from anyone. This has been the quality of my life for most of the last two decades. Then, some days ago, the miracle happened again and I found myself thinking about a theorem I was working on in July. And once more, with a great effort, my mind, which is not so young anymore, started her slow process of recovery. I concluded this proof last night. This is only a poor thing but since it is not present in my books of statistics, I have decided to write it down in my blog, for those who might be interested.

I can now come back to my awkward studies, which span from statistics to computational immunology, from analysis of genetic data to mathematical modelling of bacterial growth. Desperately searching for a cure.

The problem

Let $X_1, X_2, ..., X_m$ be independent random variables with an exponential distribution with pairwise distinct parameters $\lambda_1, \lambda_2, ..., \lambda_m$ , respectively. Our problem is: what is the expression of the distribution of the random variable $Y = X_1 + X_2 + ... + X_m$ ? I faced the problem for m = 2, 3, 4. Then, when I was quite sure of the expression of the general formula of $f_Y$ (the distribution of Y) I made my attempt to prove it inductively. But before starting, we need to mention two preliminary results that I won’t demonstrate since you can find these proofs in any book of statistics.

PROPOSITION 1. Let $X_1, X_2$ be independent random variables. The distribution of $Y = X_1 + X_2$ is given by:

where f_X is the distribution of the random vector [ $X_1, X_2$ ].

PROPOSITION 2. Let $X_1, X_2, X_3$ be independent random variables. The two random variables $Y _1 = X_1 + X_2 + ...+ X_n$ and $Y _2 = X_1 + X_2 + ...+ X_m$ (with n<m) are independent.

DEFINITION 1. For those who might be wondering how the exponential distribution of a random variable $X_i$ with a parameter $\lambda_i$ looks like, I remind that it is given by:

Guessing the solution

As mentioned, I solved the problem for m = 2, 3, 4 in order to understand what the general formula for $f_Y$ might have looked like.

PROPOSITION 3 (m = 2). Let $X_1, X_2$ be independent exponential random variables with distinct parameters $\lambda_1, \lambda_2$ , respectively. The law of $Y _1 = X_1 + X_2$ is given by:

Proof. We just have to substitute $f_{X_1}, f_{X_2}$ in Prop. 1. We obtain:

And the demonstration is complete ♦

PROPOSITION 4 (m = 3). Let $X_1, X_2, X_3$ be independent exponential random variables with pairwise distinct parameters $\lambda_1, \lambda_2, \lambda_3$ , respectively. The law of $Y _1 = X_1 + X_2 + X_3$ is given by:

Proof. If we define $Y _1 = X_1 + X_2$ and $Y _2 = X_3$ , then we can say – thanks to Prop. 2 – that $Y _1$ and $Y _2$ are independent. This means that – according to Prop. 1 – we have

The reader will now recognize that we know the expression of $f_{Y_1}$ because of Prop. 3. So, we have:

For the first integral we find:

For the second one we have:

Hence, we find:

And the thesis is proved ♦

PROPOSITION 5 (m = 4). Let $X_1, X_2, X_3, X_4$ be independent exponential random variables with pairwise distinct parameters $\lambda_1, \lambda_2, \lambda_3, \lambda_4$ , respectively. The law of $Y = X_1 + X_2 + X_3 + X_4$ is given by:

for y>0, while it is zero otherwise.

Proof. Let’s consider the two random variables $Y_1 = X_1 + X_2$ , $Y_2 = X_3 + X_4$ . Prop. 2 tells us that $Y_1, Y_2$ are independent. This means that – according to Prop. 1 – we can write:

The reader has likely already realized that we have the expressions of $f_{Y_2}$ and $f_{Y_2}$ , thanks to Prop. 3. So we have:

For the four integrals we can easily calculate what follows:

Adding these four integrals together we obtain:

And this proves the thesis ♦

We are now quite confident in saying that the expression of $f_Y$ for the generic value of m is given by:

for y>0, while being zero otherwise. But we aim at a rigorous proof of this expression.

Proof

In order to carry out our final demonstration, we need to prove a property that is linked to the matrix named after Vandermonde, that the reader who has followed me till this point will likely remember from his studies of linear algebra. The determinant of the Vandermonde matrix is given by:

PROPOSITION 6 (lemma). The following relationship is true:

Proof. In the following lines, we calculate the determinant of the matrix below, with respect to the second line. In the end, we will use the expression of the determinant of the Vandermonde matrix, mentioned above:

But this determinant has to be zero since the matrix has two identical lines, which proves the thesis ♦

PROPOSITION 7. Let $X_1, X_2, ... , X_m$ be independent exponential random variables with pairwise distinct parameters $\lambda_1, \lambda_2, ... , \lambda_m$ , respectively. The law of $Y = X_1 + X_2 + ... + X_m$ is given by:

for y > 0, while being zero otherwise.

Proof. We already know that the thesis is true for m = 2, 3, 4. We now admit that it is true for m-1 and we demonstrate that this implies that the thesis is true for m (proof by induction). Let’s define the random variables $Y_1 = X_1 + X_2 + ... + {X_{m-1}}$ and $Y_2 = X_m$ . These two random variables are independent (Prop. 2) so – according to Prop. 1 – we have: