{
"cells": [
{
"cell_type": "markdown",
"id": "0facf8cc-98aa-474e-85d5-c19029c003c4",
"metadata": {},
"source": [
"# Probability generating functions\n",
"\n",
"Probability generating functions are a useful tool for studying discrete random variables, taking values in $n = 0, 1, 2 ...$.\n",
"Each probability mass function has a unique probability generating function and vice versa.\n",
"The moments of a random variable can be obtained straightforwardly from its probability generating function.\n",
"Probability generating functions are useful when dealing with sums and random sums of independent random variables."
]
},
{
"cell_type": "markdown",
"id": "8b015c00-0b76-44fd-9282-ae6eac3d1b02",
"metadata": {},
"source": [
"## Definition\n",
"\n",
"\n",
":::{prf:definition} Generating function\n",
"\n",
"Given a sequence $u_0, u_1, ...$, its generating function is\n",
"\n",
"$$\\begin{align}\n",
"\\sum^\\infty_{n = 0} u_n s^n = u_0 + u_1 s + u_2 s^2 + ...\n",
"\\end{align}$$\n",
"\n",
"for all values of $s$ for which the converges absolutely.\n",
":::\n",
"\n",
"\n",
"The generating function is a general definition, that is not specific to probability theory.\n",
"When the terms of the sequence $u_0, u_1, ...$ are the values of a probability mass function, then the generating function is called a probability generating function.\n",
"\n",
":::{prf:definition} Probability generating function\n",
"\n",
"Let $X$ be a random variable on $(\\Omega, \\mathcal{F}, \\mathbb{P})$, which takes values on the non-negative integers and let $p_n = \\mathbb{P}(X = n)$. Then the probability generating function (or pgf) of $X$ is defined as\n",
"\n",
"$$\\begin{align}\n",
"G_X(s) = \\sum^\\infty_{n = 0} p_n s^n = p_0 + p_1 s + p_2 s^2 + ...\n",
"\\end{align}$$\n",
"\n",
"for all values of $s$ for which the sum converges absolutely.\n",
":::"
]
},
{
"cell_type": "markdown",
"id": "d483479b-54f6-4394-96d9-353f0c8d8b9d",
"metadata": {},
"source": [
"## Uniqueness of PGFs and examples\n",
"\n",
"One very useful result is that if two random variables have the same pgf, then they have the same pmf - and vice versa.\n",
"\n",
":::{prf:theorem} Uniqueness of pgfs\n",
"\n",
"Let $X$ and $Y$ be discrete random variables with probability generating functions $G_X$ and $G_Y$.\n",
"Then \n",
"\n",
"$$\\begin{align}\n",
"G_X(s) = G_Y(s) \\text{ for all } s \\iff \\mathbb{P}(X = k) = \\mathbb{P}(Y = k\n",
"), \\text{ for } k = 0, 1, 2, ...\n",
"\\end{align}$$\n",
"\n",
":::\n",
"\n",
"One direction of this result follows by the definition of the pgf, whereas the other can be shown by taking the Taylor expansion of $G_X$ and $G_Y$, and observing that all coefficients are equal, which shows that the pmfs of $X$ and $Y$ are identical.\n",
" \n",
" \n",
"### Bernoulli\n",
"\n",
"If $X$ has the Bernoulli distribution with parameter $p$, then\n",
"\n",
"$$\\begin{align}\n",
"G_X(s) = q + ps, \\text{ where } q = 1 - p.\n",
"\\end{align}$$\n",
"\n",
"\n",
"### Binomial distribution\n",
"\n",
"If $X$ has the binomial distribution with parameters $p$ and $n$, then\n",
"\n",
"$$\\begin{align}\n",
"G_X(s) = (q + ps)^n, \\text{ where } q = 1 - p.\n",
"\\end{align}$$\n",
"\n",
"### Poisson distribution\n",
"\n",
"If $X$ has the binomial distribution with parameter $\\lambda$, then\n",
"\n",
"$$\\begin{align}\n",
"G_X(s) = e^{\\lambda(s - 1)}.\n",
"\\end{align}$$\n",
"\n",
"\n",
"### Geometric distribution\n",
"\n",
"If $X$ has the geometric distribution with parameter $p$, then\n",
"\n",
"$$\\begin{align}\n",
"G_X(s) = \\frac{ps}{1 - qs}, \\text{ where } q = 1 - p.\n",
"\\end{align}$$\n",
"\n",
"\n",
"### Negative binomial distribution\n",
"\n",
"If $X$ has the negative binomial distribution with parameters $p$ and $n$, then\n",
"\n",
"$$\\begin{align}\n",
"G_X(s) = \\left(\\frac{ps}{1 - qs}\\right)^n, \\text{ where } q = 1 - p.\n",
"\\end{align}$$"
]
},
{
"cell_type": "markdown",
"id": "8984aaa3-3f3b-444e-9dcd-2f6b1a4fc107",
"metadata": {},
"source": [
"## Moments\n",
"\n",
"We are often interested in the moments, such as the mean, of a random variable as these summarise certain aspects of its pmf.\n",
"\n",
":::{prf:definition} Moment\n",
"\n",
"The $k \\geq 1$ moment of a random variable $X$ is the quantity $\\mathbb{E}(X^k)$.\n",
"\n",
":::\n",
"\n",
"\n",
"We can easily obtain all moments of a random variable from its pgf, as stated by the following result.\n",
"\n",
"\n",
":::{prf:theorem} Moments from pgf derivatives\n",
"\n",
"Let $X$ be a random variable with pgf $G_X$. Then\n",
" \n",
"$$\\begin{align}\n",
" G_X^{(k)}(1) = \\mathbb{E}\\left(X(X - 1)...(X - k + 1)\\right),\n",
"\\end{align}$$\n",
"\n",
"where the $G_X^{(k)}$ notation denotes the $k^{th}$ derivative of $G_X$.\n",
":::\n",
"\n",
"The above result is useful in computing higher order moments of random variables, by first finding the pgf and taking derivatives."
]
},
{
"cell_type": "markdown",
"id": "83ce274f-517d-40af-9b9a-8d89ab8595a5",
"metadata": {},
"source": [
"## Sums of independent variables\n",
"\n",
":::{prf:theorem} Independence $\\implies$ $G$ factorises\n",
"\n",
"If $X$ and $Y$ are independent random variables, each taking values on the non-negative integers, then\n",
" \n",
"$$\\begin{align}\n",
"G_{X + Y}(s) = G_X(s) G_Y(s).\n",
"\\end{align}$$\n",
"\n",
":::\n",
"\n",
"By extension, if $X_1, X_2, ..., X_n$ are independent random variables, then their sum $X = X_1 + X_2 + ... + X_n$ has pgf $G_X(s) = G_1(s)G_2(s)...G_n(s)$.\n",
"One very useful consequence of the above result is that we can easily find the pmf of a sum of random variables by simply taking the product of (known) pgfs and matching them against other (known) pgfs. For example, by inspecting the example pgfs above, we see that:\n",
"\n",
"- If $X$ and $Y$ are independent and binomially distributed with parameters $p$ and $n$ and $p$ and $m$, then $X + Y$ is also binomially distributed with parameters $p$ and $n + m$.\n",
"- If $X$ and $Y$ are Poisson distributed with parameters $\\lambda$ and $\\mu$, then $X + Y$ is also Poisson distributed with parameter $\\lambda + \\mu$.\n",
"- If $X$ and $Y$ are negative binomially distributed with parameters $p$ and $n$ and $p$ and $m$ respectively, then $X + Y$ is also negative binomially distributed with parameters $p$ and $n + m$.\n",
"- If $X_1, X_2, ..., X_n$ are independent and geometrically distributed, then $X_1 + X_2 + ... + X_n$ is negative binomially distributed with parameters $p$ and $n$.\n",
"\n",
"In some problems of interest, we may have a sum of $N$ i.i.d. random variables, where $N$ is itself a random variable. In this case, the following result, called the random sum formula, is very useful.\n",
"\n",
":::{prf:theorem} Random sum formula\n",
"\n",
"Let $N$ and $X_1, X_2, ...$ be random variables taking values on the non-negative integers. If $N$ has pgf $G_N$ and the $X_n$ are independent and identically distributed, with pgf $G_X$, then the pgf of the sum\n",
" \n",
"$$\\begin{align}\n",
"S = X_1 + X_2 + ... + X_N\n",
"\\end{align}$$\n",
" \n",
"has pgf\n",
"\n",
"$$\\begin{align}\n",
"G_S(s) = G_N(G_X(s)).\n",
"\\end{align}$$\n",
":::\n",
"\n",
"Using $G_S$ we can easily determine the moments of a random sum. The random sum formula is especially useful when studying branching processes."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "rw",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.14"
}
},
"nbformat": 4,
"nbformat_minor": 5
}