Sum of independent exponential random variables

Sum of independent exponential random variables

For Chiara,

who once encouraged me

to boldly keep trying.

Introduction

For the last four months, I have experienced the worst level of my illness: I have been completely unable to think for most of the time. So I could do nothing but hanging in there, waiting for a miracle, passing from one medication to the other, well aware that this state could have lasted for years, with no reasonable hope of receiving help from anyone. This has been the quality of my life for most of the last two decades. Then, some days ago, the miracle happened again and I found myself thinking about a theorem I was working on in July. And once more, with a great effort, my mind, which is not so young anymore, started her slow process of recovery. I concluded this proof last night. This is only a poor thing but since it is not present in my books of statistics, I have decided to write it down in my blog, for those who might be interested.

I can now come back to my awkward studies, which span from statistics to computational immunology, from analysis of genetic data to mathematical modelling of bacterial growth. Desperately searching for a cure.

The problem

Let X_1, X_2, …, X_m be independent random variables with an exponential distribution with pairwise distinct parameters λ_1, λ_2, …, λ_m, respectively. Our problem is: what is the expression of the distribution of the random variable Y = X_1 + X_2 + …+ X_m? I faced the problem for m = 2, 3, 4. Then, when I was quite sure of the expression of the general formula of  f_Y (the distribution of Y) I made my attempt to prove it inductively. But before starting, we need to mention two preliminary results that I won’t demonstrate since you can find these proofs in any book of statistics.

PROPOSITION 1. Let X_1, X_2 be independent random variables. The distribution of Y = X_1 + X_2 is given by:

where f_X is the distribution of the random vector [X_1, X_2].

PROPOSITION 2. Let X_1, X_2, …, X_m be independent random variables. The two random variables Y _1 = X_1 + X_2 + …+ X_n and Y _2 = X_n+1 + X_n+2 + …+ X_m (with n<m) are independent.

DEFINITION 1. For those who might be wondering how the exponential distribution of a random variable X_i with a parameter λ_i looks like, I remind that it is given by:

Guessing the solution

As mentioned, I solved the problem for m = 2, 3, 4 in order to understand what the general formula for f_Y might have looked like.

PROPOSITION 3 (m = 2). Let X_1, X_2 be independent exponential random variables with distinct parameters λ_1, λ_2, respectively. The law of Y _1 = X_1 + X_2 is given by:

Proof. We just have to substitute f_X_1, f_X_2 in Prop. 1. We obtain:

And the demonstration is complete ♦

PROPOSITION 4 (m = 3). Let X_1, X_2, X_3  be independent exponential random variables with pairwise distinct parameters λ_1, λ_2, λ_3, respectively. The law of Y = X_1 + X_2 + X_3 is given by:

Proof. If we define Y_1 = X_1 + X_2 and Y_2 = X_3, then we can say – thanks to Prop. 2 – that Y_1 and Y_2 are independent. This means that – according to Prop. 1 – we have

The reader will now recognize that we know the expression of  f_Y_1 because of Prop. 3. So, we have: 

For the first integral we find:

For the second one we have:

Hence, we find:

And the thesis is proved

PROPOSITION 5 (m = 4). Let X_1, X_2, X_3, X_4 be independent exponential random variables with pairwise distinct parameters λ_1, λ_2, λ_3, λ_4, respectively. The law of Y = X_1 + X_2 + X_3 + X_4 is given by:

for y>0, while it is zero otherwise.

Proof. Let’s consider the two random variables Y_1 = X_1 +  X_2, Y_2 = X_3 + X_4. Prop. 2 tells us that Y_1, Y_2 are independent. This means that – according to Prop. 1 – we can write:

The reader has likely already realized that we have the expressions of f_Y_1 and f_Y_2, thanks to Prop. 3. So we have:

For the four integrals we can easily calculate what follows:

Adding these four integrals together we obtain:

And this proves the thesis

We are now quite confident in saying that the expression of f_Y for the generic value of m is given by:

for y>0, while being zero otherwise. But we aim at a rigorous proof of this expression.

Proof

In order to carry out our final demonstration, we need to prove a property that is linked to the matrix named after Vandermonde, that the reader who has followed me till this point will likely remember from his studies of linear algebra. The determinant of the Vandermonde matrix is given by:

PROPOSITION 6 (lemma). The following relationship is true:

Proof. In the following lines, we calculate the determinant of the matrix below, with respect to the second line. In the end, we will use the expression of the determinant of the Vandermonde matrix, mentioned above:

But this determinant has to be zero since the matrix has two identical lines, which proves the thesis

PROPOSITION 7. Let X_1, X_2, … , X_m be independent exponential random variables with pairwise distinct parameters λ_1, λ_2, … , λ_m, respectively. The law of Y = X_1 + X_2 + … + X_m is given by:

for y > 0, while being zero otherwise.

Proof. We already know that the thesis is true for m = 2, 3, 4. We now admit that it is true for m-1 and we demonstrate that this implies that the thesis is true for (proof by induction). Let’s define the random variables Y_1 = X_1 + X_2 + … + X_m-1 and Y_2 = X_m. These two random variables are independent (Prop. 2) so – according to Prop. 1 – we have:

Now, f_Y_1 is the thesis for m-1 while f_Y_2 is the exponential distribution with parameter λ_m. So we have:

For the sum we have:

The sum within brackets can be written as follows:

So far, we have found the following relationship:

The last sum in this expression can be written as follows:

In a more concise way:

But it is also true that:

Which means that we have found:

So, for the thesis to be true it suffices to prove that:

But, as can be easly seen, we have

Hence, we just need to prove that

But we can write:

Moreover, we have:

So, we just need to prove that:

We note now that:

Moreover, we have

Therefore we have found:

But we also have that:

Which means that:

So we just have to prove that:

But this expression has been demonstrated in Prop. 6 and this proof is concluded ♦

References. A paper on this same topic has been written by Markus Bibinger and it is available here

La retta e il punto

La retta e il punto

Avevi dieci anni in vacanza
con i tuoi tu di me non ti ricordi
ma ero lì in Egitto di passaggio
per gli altipiani di fossili, la terra
degli Afar.

Ti vidi e seppi di amare
la donna che saresti stata
e allora pregai la dea nera
i fianchi bruni che cullano
il mondo.

Siete separati da troppe stagioni
non si può –
mi disse – la legge non vuole.
Avete solo una donna nella vita
dalla culla alla tomba di volto in volto
sono sempre io, dovresti saperlo.
La troverai con un’altra voce, dimentica
il suo nome.

Ma io pregai fino a farmi del male
la sedussi con l’arte che non pensavo
di avere e le strappai il patto terribile
che sarei stato una statua pur di poterti
amare. Tutto questo viaggio
immobile qui l’ho fatto solo
per te, Chiara.

La dea ammonì che il tempo per me
non avrebbe ripreso e nel momento
in cui ti trovo ti perdo, resto indietro.
L’incontro dura l’intersezione
di una retta con un punto, tu devi andare
e io sorrido oltre il presagio liquido
del pianto.