# Probability generating functions¶

Probability generating functions are a useful tool for studying discrete random variables, taking values in \(n = 0, 1, 2 ...\). Each pmf has a unique pgf and vice versa. The moments of a random variable can be obtained straightforwardly from its pgf. Pgfs are useful when dealing with sums and random sums of independent random variables.

## Definition¶

**Definition (Generating function)** Given a sequence \(u_0, u_1
, ...\), its generating function is

for all values of \(s\) for which the converges absolutely.

The generating function is a general definition, that is not specific to
probability theory. When the terms of the sequence \(u_0, u_1, ...\) are the
values of a pmf, then the generating function is called a *probability
generating function*.

**Definition (Probability generating function)** Let \(X\) be a random variable
on \((\Omega, \mathcal{F}, \mathbb{P})\), which takes values on the non
-negative integers and let \(p_n = \mathbb{P}(X = n)\). Then the probability
generating function (or pgf) of \(X\) is defined as

for all values of \(s\) for which the sum converges absolutely.

## Uniqueness of PGFs and examples¶

One very useful result is that if two random variables have the same pgf , then they have the same pmf - and vice versa.

**Theorem (Uniqueness of pgfs)** Let \(X\) and \(Y\) be discrete random variables
with probability generating functions \(G_X\) and \(G_Y\). Then

One direction of this result follows by the definition of the pgf, whereas the other can be shown by taking the Taylor expansion of \(G_X\) and \(G_Y\), and observing that all coefficients are equal, which shows that the pmfs of \(X\) and \(Y\) are identical.

### Bernoulli¶

If \(X\) has the Bernoulli distribution with parameter \(p\), then

### Binomial distribution¶

If \(X\) has the binomial distribution with parameters \(p\) and \(n\), then

### Poisson distribution¶

If \(X\) has the binomial distribution with parameter \(\lambda\), then

### Geometric distribution¶

If \(X\) has the geometric distribution with parameter \(p\), then

### Negative binomial distribution¶

If \(X\) has the negative binomial distribution with parameters \(p\) and \(n\), then

## Moments¶

We are often interested in the moments, such as the mean, of a random variable as these summarise certain aspects of its pmf.

**Definition (Moment)** The \(k \geq 1\) moment of a random variable \(X\) is the
quantity \(\mathbb{E}(X^k)\).

We can easily obtain all moments of a random variable from its pgf, as stated by the following result.

**Theorem (Moments from pgf derivatives)** Let \(X\) be a random variable
with pgf \(G_X\). Then

where the \(G_X^{(k)}\) notation denotes the \(k^{th}\) derivative of \(G_X\).

The above result is useful in computing higher order moments of random variables, by first finding the pgf and taking derivatives.

## Sums of independent variables¶

**Theorem (Independence \(\implies\) \(G\) factorises)** If \(X\) and \(Y\) are
independent random variables, each taking values on the non-negative
integers, then

By extension, if \(X_1, X_2, ..., X_n\) are independent random variables, then their sum \(X = X_1 + X_2 + ... + X_n\) has pgf \(G_X(s) = G_1(s)G_2(s)...G_n(s )\). One very useful consequence of the above result is that we can easily find the pmf of a sum of random variables by simply taking the product of (known) pgfs and matching them against other (known) pgfs. For example, by inspecting the example pgfs above, we see that:

If \(X\) and \(Y\) are independent and binomially distributed with parameters \(p\) and \(n\) and \(p\) and \(m\), then \(X + Y\) is also binomially distributed with parameters \(p\) and \(n + m\).

If \(X\) and \(Y\) are Poisson distributed with parameters \(\lambda\) and \(\mu\), then \(X + Y\) is also Poisson distributed with parameter \(\lambda + \mu\).

If \(X\) and \(Y\) are negative binomially distributed with parameters \(p\) and \(n\) and \(p\) and \(m\) respectively, then \(X + Y\) is also negative binomially distributed with parameters \(p\) and \(n + m\).

If \(X_1, X_2, ..., X_n\) are independent and geometrically distributed , then \(X_1 + X_2 + ... + X_n\) is negative binomially distributed with parameters \(p\) and \(n\).

In some problems of interest, we may have a sum of \(N\) i.i.d. random variables, where \(N\) is itself a random variable. In this case, the following result, called the random sum formula, is very useful.

**Theorem (Random sum formula)** Let \(N\) and \(X_1, X_2, ...\) be random
variables taking values on the non-negative integers. If \(N\) has pgf \(G_N\)
and the \(X_n\) are independent and identically distributed, with pgf \(G_X\)
, then the pgf of the sum

has pgf

Using \(G_S\) we can easily determine the moments of a random sum. The random sum formula is especially useful when studying branching processes.