**Definition (Generating function)** Given a sequence $u_0, u_1
, ...$, its generating function is
$$\begin{align}
\sum^\infty_{n = 0} u_n s^n = u_0 + u_1 s + u_2 s^2 + ...
\end{align}$$
for all values of $s$ for which the converges absolutely.

The generating function is a general definition, that is not specific to probability theory. When the terms of the sequence $u_0, u_1, ...$ are the values of a pmf, then the generating function is called a *probability generating function*.

**Definition (Probability generating function)** Let $X$ be a random variable
on $(\Omega, \mathcal{F}, \mathbb{P})$, which takes values on the non
-negative integers and let $p_n = \mathbb{P}(X = n)$. Then the probability
generating function (or pgf) of $X$ is defined as
$$\begin{align}
G_X(s) = \sum^\infty_{n = 0} p_n s^n = p_0 + p_1 s + p_2 s^2 + ...
\end{align}$$
for all values of $s$ for which the sum converges absolutely.

## Uniqueness of PGFs and examples One very useful result is that if two random variables have the same pgf , then they have the same pmf - and vice versa.

**Theorem (Uniqueness of pgfs)** Let $X$ and $Y$ be discrete random variables
with probability generating functions $G_X$ and $G_Y$. Then
$$\begin{align}
G_X(s) = G_Y(s) \text{ for all } s \iff \mathbb{P}(X = k) = \mathbb{P}(Y = k
), \text{ for } k = 0, 1, 2, ...
\end{align}$$

One direction of this result follows by the definition of the pgf, whereas the other can be shown by taking the Taylor expansion of $G_X$ and $G_Y$, and observing that all coefficients are equal, which shows that the pmfs of $X$ and $Y$ are identical. ### Bernoulli If $X$ has the Bernoulli distribution with parameter $p$, then $$\begin{align} G_X(s) = q + ps, \text{ where } q = 1 - p. \end{align}$$ ### Binomial distribution If $X$ has the binomial distribution with parameters $p$ and $n$, then $$\begin{align} G_X(s) = (q + ps)^n, \text{ where } q = 1 - p. \end{align}$$ ### Poisson distribution If $X$ has the binomial distribution with parameter $\lambda$, then $$\begin{align} G_X(s) = e^{\lambda(s - 1)}. \end{align}$$ ### Geometric distribution If $X$ has the geometric distribution with parameter $p$, then $$\begin{align} G_X(s) = \frac{ps}{1 - qs}, \text{ where } q = 1 - p. \end{align}$$ ### Negative binomial distribution If $X$ has the negative binomial distribution with parameters $p$ and $n$, then $$\begin{align} G_X(s) = \left(\frac{ps}{1 - qs}\right)^n, \text{ where } q = 1 - p. \end{align}$$ ## Moments We are often interested in the moments, such as the mean, of a random variable as these summarise certain aspects of its pmf.

**Definition (Moment)** The $k \geq 1$ moment of a random variable $X$ is the
quantity $\mathbb{E}(X^k)$.

We can easily obtain all moments of a random variable from its pgf, as stated by the following result.

**Theorem (Moments from pgf derivatives)** Let $X$ be a random variable
with pgf $G_X$. Then
$$\begin{align}
G_X^{(k)}(1) = \mathbb{E}\left(X(X - 1)...(X - k + 1)\right),
\end{align}$$
where the $G_X^{(k)}$ notation denotes the $k^{th}$ derivative of $G_X$.

The above result is useful in computing higher order moments of random variables, by first finding the pgf and taking derivatives. ## Sums of independent variables

**Theorem (Independence $\implies$ $G$ factorises)** If $X$ and $Y$ are
independent random variables, each taking values on the non-negative
integers, then
$$\begin{align}
G_{X + Y}(s) = G_X(s) G_Y(s).
\end{align}$$

By extension, if $X_1, X_2, ..., X_n$ are independent random variables, then their sum $X = X_1 + X_2 + ... + X_n$ has pgf $G_X(s) = G_1(s)G_2(s)...G_n(s )$. One very useful consequence of the above result is that we can easily find the pmf of a sum of random variables by simply taking the product of (known) pgfs and matching them against other (known) pgfs. For example, by inspecting the example pgfs above, we see that: - If $X$ and $Y$ are independent and binomially distributed with parameters $p$ and $n$ and $p$ and $m$, then $X + Y$ is also binomially distributed with parameters $p$ and $n + m$. - If $X$ and $Y$ are Poisson distributed with parameters $\lambda$ and $\mu$, then $X + Y$ is also Poisson distributed with parameter $\lambda + \mu$. - If $X$ and $Y$ are negative binomially distributed with parameters $p$ and $n$ and $p$ and $m$ respectively, then $X + Y$ is also negative binomially distributed with parameters $p$ and $n + m$. - If $X_1, X_2, ..., X_n$ are independent and geometrically distributed , then $X_1 + X_2 + ... + X_n$ is negative binomially distributed with parameters $p$ and $n$. In some problems of interest, we may have a sum of $N$ i.i.d. random variables, where $N$ is itself a random variable. In this case, the following result, called the random sum formula, is very useful.

**Theorem (Random sum formula)** Let $N$ and $X_1, X_2, ...$ be random
variables taking values on the non-negative integers. If $N$ has pgf $G_N$
and the $X_n$ are independent and identically distributed, with pgf $G_X$
, then the pgf of the sum
$$\begin{align}
S = X_1 + X_2 + ... + X_N
\end{align}$$
has pgf
$$\begin{align}
G_S(s) = G_N(G_X(s)).
\end{align}$$

Using $G_S$ we can easily determine the moments of a random sum. The random sum formula is especially useful when studying branching processes.