{ "cells": [ { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "# Annealed importance sampling\n", "\n", "\n", "Star\n", "Issue\n", "Watch\n", "Follow\n", "\n", "Simulating samples from distributions is a central problem in Statistics and Machine Learning, because it enables estimating important quantities such as integrals.\n", "For example, we are often interested in evaluating integrals of the form\n", "\n", "$$\\begin{equation}\n", "I = \\int p(x) f(x) dx,\n", "\\end{equation}$$\n", "\n", "where $p$ is a probability density and $f$ is a function of interest.\n", "If we have acccess to the cumulative density function $p$, we can easily draw samples from it using [inverse transform sampling](https://en.wikipedia.org/wiki/Inverse_transform_sampling).\n", "However, in most cases, the cumulative density function does not have an analytically tractable closed form, so sampling via the inverse transform does not apply.\n", "For example, non-conjugate Bayesian models typically involve intractable distributions of this kind.\n", "In such cases, we must resort to approximation methods, such as Monte Carlo.\n", "\n", "A standard Monte Carlo method for handling intractable integrals is importance sampling.\n", "Importance sampling gets around the intractability of $p$ by introducing another tractable distribution $q$ and drawing samples from $q$ insted of $p$.\n", "It then corrects for the bias in the samples, to account for the fact that these now come from $q$ rather than $p$, by weighing them appropriately, using so called importance weights.\n", "In this way, importance sampling yields an unbiased estimate of downstream integrals, while circumventing the intractability of $p$.\n", "\n", "Unfortunately, if $q$ and $p$ are very dissimilar, the importance sampling estimator has a large random error because, as we will see, the variance in the random weights is large.\n", "Annealed importance sampling{cite}`neal2001annealed` is a method which circumvents this issue by using an annealing procedure which produces samples with more equally distributed importance weights while remaining unbiased, and can greatly reduce the variance of the resulting estimates.\n", "\n", "## Importance sampling\n", "\n", "Suppose we wish to evaluate an integral of the form\n", "\n", "$$\\begin{equation}\n", "I = \\int p(x) f(x) dx.\n", "\\end{equation}$$\n", "\n", "If we could draw samples from $p$, we could estimate this integral by Monte Carlo, by computing\n", "\n", "$$\\begin{equation}\n", "I \\approx \\frac{1}{N} \\sum_{n=1}^N f(x_n) ~~\\text{ where }~~ x_n \\sim p.\n", "\\end{equation}$$\n", "\n", "However, in many applications of interest, we cannot draw samples from $p$ directly.\n", "Importance sampling gets around this by introducing another tractable distribution $q$, and rewriting this integral as \n", "\n", "$$\\begin{equation}\n", "I = \\int q(x) \\frac{p(x)}{q(x)} f(x) dx.\n", "\\end{equation}$$\n", "\n", "A technical, but important, requirement here is that $q$ should be non-zero whenever $p$ is non-zero, for the integral above to be well defined.\n", "Hereafter we assume this to be the case.\n", "Since the distribution $q$ is tractable, we can estimate the value of the integral by drawing samples from it and computing\n", "\n", "$$\\begin{equation}\n", "I \\approx \\frac{1}{N} \\sum_{n=1}^N \\underbrace{\\frac{p(x_n)}{q(x_n)}}_{w_n} f(x_n) ~~\\text{ where }~~ x_n \\sim q.\n", "\\end{equation}$$\n", "\n", "The ratios $w_n$ are called importance weights, since they weigh the contribution of each $f(x_n)$ term in the sum.\n", "Since $q$ proposes the samples to be used in the Monte Carlo sum, it is commonly referred to it as the proposal distribution.\n", "We also refer to $p$ as the target distribution, since this is the one we are ultimately interested in.\n", "This is called the importance sampling estimator, and is unbiased, meaning that in expectation it is equal to the original integral\n", "\n", "$$\\begin{equation}\n", "\\mathbb{E}\\left[ \\frac{1}{N} \\sum_{n=1}^N \\frac{p(x_n)}{q(x_n)} f(x_n) \\right] = \\frac{1}{N} \\sum_{n=1}^N \\mathbb{E}\\left[\\frac{p(x_n)}{q(x_n)} f(x_n) \\right] = \\frac{1}{N} \\sum_{n=1}^N I = I.\n", "\\end{equation}$$\n", "\n", "However, in practice we always end up using a finite number of samples, so the importance sampling estimate will not be exactly equal to $I$, but will be off by some random error, whose magnitude is captured by the variance of the estimator.\n", "In particular, if the variance of the estimator using a single sample is $V$, that is\n", "\n", "$$\\begin{equation}\n", "V = \\mathbb{V}\\left[\\sum_{n=1}^N \\frac{p(x_n)}{q(x_n)} f(x_n) \\right] \\geq 0,\n", "\\end{equation}$$\n", "\n", "then using $N$ samples reduces the variance by a factor of $N$\n", "\n", "$$\\begin{equation}\n", "\\mathbb{V}\\left[\\frac{1}{N}\\sum_{n=1}^N \\frac{p(x_n)}{q(x_n)} f(x_n) \\right] = \\frac{1}{N^2}\\mathbb{V}\\left[\\sum_{n=1}^N \\frac{p(x_n)}{q(x_n)} f(x_n) \\right] = \\frac{1}{N^2}\\sum_{n=1}^N \\mathbb{V}\\left[\\frac{p(x_n)}{q(x_n)} f(x_n) \\right] = \\frac{V}{N}.\n", "\\end{equation}$$\n", "\n", "Using more samples reduces the overall variance, but increases the computational cost.\n", "A common issue that arises with the importance sampling estimator is that its variance, $V$, can be very large.\n", "This means that lots of samples, and thus lots of computate, must be used to obtain a reasonable estimate." ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "remove-cell" ] }, "outputs": [], "source": [ "from typing import *\n", "\n", "import numpy as np\n", "import tensorflow as tf\n", "import tensorflow_probability as tfp\n", "\n", "tfd = tfp.distributions\n", "dtype = tf.float64\n", "\n", "import matplotlib\n", "import matplotlib.pyplot as plt\n", "from matplotlib_inline.backend_inline import set_matplotlib_formats\n", "\n", "set_matplotlib_formats('pdf', 'svg')\n", "\n", "matplotlib.rcParams['mathtext.fontset'] = 'stix'\n", "matplotlib.rcParams['font.family'] = 'STIXGeneral'" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Let's look at a simple example of Importance Sampling.\n", "Suppose $f(x) = x^3$ and let $p$ be a mixture of Gaussians\n", "\n", "$$\\begin{equation}\n", "p(x) = \\pi \\mathcal{N}(x; \\mu_1, \\sigma_1^2) + (1 - \\pi) \\mathcal{N}(x; \\mu_2, \\sigma_2^2).\n", "\\end{equation}$$\n", "\n", "Of course, we can in fact draw samples from a mixture of Gaussians directly, but let's pretend we can't.\n", "Now, define $q$ to be a Gaussian\n", "\n", "$$\\begin{equation}\n", "q(x) = \\mathcal{N}(x; \\mu_q, \\sigma_q^2).\n", "\\end{equation}$$" ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "remove-cell" ] }, "outputs": [], "source": [ "f = lambda x: x**3.\n", "\n", "x_plot = tf.cast(tf.linspace(-4., 4., 200), dtype=dtype)\n", "\n", "m1 = tf.convert_to_tensor(-2., dtype=dtype)\n", "s1 = tf.convert_to_tensor(0.4, dtype=dtype)\n", "\n", "m2 = tf.convert_to_tensor(2., dtype=dtype)\n", "s2 = tf.convert_to_tensor(0.4, dtype=dtype)\n", "\n", "mq = tf.convert_to_tensor(0., dtype=dtype)\n", "sq = tf.convert_to_tensor(0.8, dtype=dtype)\n", "\n", "mix_probs = tf.convert_to_tensor([0.5, 0.5], dtype=dtype)\n", "\n", "p1 = tfd.Normal(loc=m1, scale=s1)\n", "p2 = tfd.Normal(loc=m2, scale=s2)\n", "\n", "p = tfd.Mixture(cat=tfd.Categorical(probs=mix_probs), components=[p1, p2])\n", "q = tfd.Normal(loc=mq, scale=sq)\n", "\n", "p_plot = tf.math.exp(p.log_prob(x_plot))\n", "q_plot = tf.math.exp(q.log_prob(x_plot))\n", "f_plot = f(x_plot)" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "center-output", "remove-input" ] }, "outputs": [ { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNTY5LjYyODEyNSAyNDYuMTI2MjUgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic1ZpdjxXHEYbv51fMJUhm6O+PSxPHKL6K7ZUSKcmFRYDYWsCYOPbPz1PVM9Pdhz0LRE6kIC3aqTOnu7q66q23qvbxF8//9f2z5988fbL+7tvlcX969m6x6w/8vFzN+gM/v6x2fcrPy8Xw9GqJqW7JFesij7fjowtpsy7x2y3vjk//WJYXy+PPWeQdX3q6xLzlNZotGfmcNU3YbBfc7gJn01b3JfQro2Bf07U1X6Ih2m4FfdlBJO/vIs8hb3bfZHnCMX9Z3vK/WR8ZVuBD42P1Oa7ebt7ob89eLU9ulsdf2tWW9ebFYt3mvTHGFm/lmzd/X/6yPggP17+tN18tv79Z3qIHn65ir/7bs1crazz69uYPf376/PXzn767ffTq+9c/v1u/eLN+vXy9qO6LxZx+VLoJ7tXaxrKl/1Bt99uo7YrZ6qh2E9yrtstpC/cobE4lTVfy2NBXt8Vxwya4d0Nfykdt597f7sJBD4+935kqkfFRG4ZxwwsL1dXafZO2gF6OPPLdx1+6fbEHvz5cb36YVL70/rC5McTuuJDCGyFsIWaN53suZLvjSnRD6/LmU/X+2LELrm5pnd2cM6W6kH24d1t/ZVsn1+FdsMe2XXB925q3WnzNLoSU7902zRckSzySxZxhnVq2nOb7KQ0Q5xv6UW+obibbWk1NqbQd+pYP/vpA34lbrb6WYny2xzsXF43q1dZSrY/5/WUeXlnmwWf6AR4VAAGuOdr63gZvm57/ZeWaKdfZS7kIIMGmtHm3/vR8/dP6enXrV6sltMF18omNqSRHNJktpP1fxhOyiaU4H11dv3m6zvlsDIXsEi5SwpZsiVURJBubq/hJJVQDiwSR4o3cokjjVjwOAioTiNlxeKTW4IzOWutEnAquq1JAKAXPUaIjSrPJScVhKyZHX0RMxnTGqph0E5GDLH5zNgfbxHWTXRJ6hM2EgHeJmFuL2ByYjZspJWevUr/lYoJBPaxW0UAVIVEa55NNYuFaUyztbRwfxwtoXbcaXSkCBRY/jpwIcSJLmOKsvu3chtIOR00RDKolt7eDrBiKXVPZgrENTyTQHVrXsGZHPHNoPY2rG0atbgXMYk28IVJgkavAeoVTYTA+F6nfWABrrwXTJCxSVJywjouxrtVLVDche8eYMwFYt2yTd6pzcFsN7I6/WSAgEN9qEG6p2ij3YYG4QvBzzyLnpchNg7PySs61rYM1OVo0TtMr5tNNJdNyIXiZTQAlV2307AmHKZmNV8nXIRY8WeWiL5GGU2e/BReK0YNiT5+DB3YsZunijF/mWJDmjTX4gorrVpLBg5CXjfCwCqc4I7eCllXfd8aH5oU54MjZEUPyay2xHVUvAH+KiLE6btN2Zc3i0UWUx+zFRq83x9VgL3ERm/BxF62eqXg835iKNniiT2S3Jo+4tuerNsjVJDVYJVWEit/I3Qbis+jSNW1YOoh1LZHMNu2muUrj8AveN5iu2BDlpMQLfivXtPJNjxPZomK/RVPF0WQfjyWqiiN+6xxfJdCdt8U5FUskRDbiwLVmfhcp+ztf8LpVzOaBgqRih9uiflozLwTs094O+G20mByQSqCcHlN5aQIm4sqNJ8iBaYsQydw+F5rElgSyXATeiX1CBj/xp+pyyaqf46uoVYLErAXKNMI5Bx4N8kncZxNKViF+gIPgqziwwELWk3uMCWvgQkAU4I5rULHbwKISquBPyUS4LuJxDovxFa2wXgMagpgbAZisQFvE7uom2AAfB7uyiA3oGdQigqv4c1b6Byglq9JKzFQ8WHE32Krx4wouiCP7nStyB6I2+nBEbq/hMUhoZUdv45YS9lDwrpADxR+cD+uQBBSRXcaL9G1gKQWXWBFxQafqVZxQO1Y3HxLEIZuACHYyiRe6RuQ0SFYDWhXzYAVtJnN7T26IkgrPu2mLcDmGEPHHReq1+0DKCPhlna7dg1isERDvTqKHFJDy2HX2KC8VCqgo2Dv4nw8UCbYQuoe3Bl1a7g+wADhG3/YcjZQRRdwioZkEyDN4pT/CJhc9JMo6+Iqg7xBkxD2GCFbEGpKlmYQEklmjXEQw8EIE+6K4TBwG3xDSC/x5bnlEDY9agUBQUFaIKVUNW8Q7BE8nRPIFzE/Jq7gBmOZXlAAUiXi7A16uTUeSC/cAtE346At8nWMqKDc49borvIB0jAvP8OsrqYPE72a0JsERg5i9nti+v07uAE059pQLCDWiEL4az9yhcR/EfBE17ZxrAujGWqGWMTMRxRJmSVG55TF1ugCZYE+o9Zz3AkCGG+M4q2ToGMG0rHKSAZYSxNb8SXSJLUEanJfEWqZ0y0HRIIM5R2pWk/GM95pE3hsTOSfAe7kWv6d9H1UX4AlvM2WiCHgIzou/pYlPkDRxXh+xxc4+QnubzCEszU1cJQQQooD0ZWI2ASyLxDrikQeRGnFevDRMrCngmQJO5eRYmvAIIFxaEtpIyIIwPJPklZG9gfdb5hoaMivX0x1ZsHI5yU3MMKAVIISnnTxSHDcABVm4TRCIQxGrsRUwBOVTVKmBkqqtxbOJt6pFX2Z/rxuCwNAR1/iwi81nx/JWGPJdkuXb5ev1kwk7yTJUjJdW2Q41uBkTlumTjyDsA/kdCPtAOccEMfDFI0FcUMOdsidl8mBnyrCXKT8MvPDID3EmgGOCHXjemB86nxtS98DajuxQZna2Z4emRydhI4XoZGvMDp1t7ckhNlIlKokeR3LQqBu41kR8OqeaaNJJnnZKZeLMkvbcoPRuoEMTXeu8Z2Z3J8M5yGCeSYs0CDK+0lhL5yE2CBGpRhcnSD25K6Ujk+Sscrm+yDHcSVfbOTE+hYYTvjrkkoG47KjrNAtiJxANLpRnkux4CQJVwIaJ+TsJCStl/pl72jIgNpuaOicBx81gUsjBmZJ8k/tN/MbHqVCgACYnEzt2Zv6Oh0jWz6HnpCYHszOZ4qKCoMTFBzi/v5RzyVLh+XMdP9O3edtSINwW68xa4hMVU6q451lqWzwo458XNqDS5tdS7VyfwINBvSD5aTYxDkyyJkvPV+IpwskWUeVD4vRGWluy8cweOvmc6hlvRbeU3FTQeCEvwqj85H0DywR4yPOpaBIbiONUjg4U0ZokmcG6OJPBPc0mP9O+XgEP7G5MsiONa0nW+ZmvjVmWsgYPMFKJj1nWY2j4oSGbjzU+ASbdH/LQlGUHvrZn2aZg52Vjlh3o15hlB5o1djIGOjWl2c6a9jSb3UyOxpbKwIKmNAtOJUAop7NkiDPXmdLsSVPGNDswiV7+BPwr5MK/s/zpU42PSrOSRM050ZjT451zjXE0whpXGtaXb31oDnO80793ZSWj+rYpjNUZzMtpRIGZ6uqc27/TW6XkiRyPFiHwJy3Cm9alJP4CIQJJdXur9MF3ew8V56FCJc8ejc2f9sYjgWjIhvDQ/YOXe8dzf3x+bYF/6geQIyIEjoxX7B+se9cz9mbnruaPRzOV2pHvBGkRmIujfDZ/+3K1c40PHePN/N7F1y4+faePZBqq8lrIWe5D5rv99NO/nfb8hCO/uHbGn+dTvJ4fn33yzX1/7YM39250/cgveuf68edeBo/nmJQwbUPSBbyUslVy1j7Bo660g0SoF7TUy79BDFPfvJZ0cJUqya4JwO09ZpYm0fbWuVKTZOE4gqPD0k4SW1taNejPz5qSXXC7QFey9tIHKRCfx7X352N3VhFBV/B2F/Qj9FX7OS/s80wGuU+OQa7WDgIh9zf81+sN/wWAFIa9So/xgCz4yShoHflZMmFYnyGPGGZ1GidTggsMC30eJ2xywDChXXi5qVJVjDEog0OZoqSQ4xT75C/YWoEDmzpB2AGBz699v8UBHCdI1BupbkaHDsOwJg34pe78ETXben/N1mye/en2zean4LT5IPlom+Mz9sLmJvcR22n0P+7AR8quUrOF/CHbvpls++P0NH/2bkcTR8VZq5Ex0Qcu9PZTL+TtezNcNRmcqig/lNxLRdkFd802o8yLhULl4O4c4979hwDJ/DZ/CdBUlm5/G4oeOnfJdaWlnofGfmj4fG08LzNfuNawaZfcs2mFvNa7TWXuNM80Ao6g3W5onWNfTILzwG/OKemL3Ruh/ySmIkT3fzBs/fTuTfG14a+X4AdqpQhLcdH+e1FosFe7N1pO6l1IEeag+W5NMgc1WgVS4ZGWyRIrv0khqR1eKTYpIyjdshRUtvVR+riWikJmSNrt6ONaKSiSthi4S0OhW9JapZoxyR7Is89fQXxpk+6z06OpJNPc7PPeRelDUvlidMa14VSfe1pyJMXpjmvnLFMWBBxrG/r2AaWWfqQ8bcpDQzYOI403QbcsTdjWceoTSku+zsm16d8wc3Ry5P344wyRggUaFd3lVJBS1KCrbeO/PuajWpWDmzq1kiT8CkWkv+gaSYPYl9QarUN/iNJTyifJDVMrSAxCcs6XPR+pn7zg4DpNqkReqMiy9lMoqinYMyoLYnjyjVrNRak3pcGk8hRqax0N/REFGNeqWiP9ryStKxFDFNr8oE+OMDomztzPOk19RJ6CCa2A7RMboourDSG2Sc45bhF5DepE06hEepAwoLQX2X34IYV/iCXqfKYPP3zd4OkEzFw32yAtlCh/uTQPKKRPQAm2V8594hCCtF8oRtZ5giB/jFe0U3oxEyjYGIukuRy2UswmbWhO9TCV8CbhEy/a+TIu9z5GndsNLXor7TRHDVLnWlkapsY6p2bojXSBolxxL92298attMeqKdoRGLrgeCcXYffuPS8BSMK0uBs0M/o2kGIL9hXY5A4FkHX14oUmGGn0Vfw4Wq/hEOQBIJDGndhVGqBa0lcqlgxQFbKKAy2c7QW8w6I+TX+6OEj+T0v6vcQxSsl3htbLHBkUZTX0sQgOJVyt9HKAIAXaW51zigNWlfxk5XKlSDgE4eB4S5OMdc4hOWuRYelesDQNxjqnaTkWOuLk8bLSCVbKvXH5XdBrnSbpWt6ekn6QvvRw3As73V/vXMm46/WMK/VOAGTrVPCASJNEoslfSD7MvoOE450Vj6BovGCvL67Q3YGfn6zmkocPy/x6hU6Py4ysZvk3NoIDNQplbmRzdHJlYW0KZW5kb2JqCjEyIDAgb2JqCjM4MDQKZW5kb2JqCjEwIDAgb2JqClsgXQplbmRvYmoKMTcgMCBvYmoKPDwgL0xlbmd0aCAyOTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZG5ccVADENzVcESltce9cjjcSD3n/qRsh38T4gngM1cMkQnfxkmc5p86MV3oe/LVjZ6Lp+nUVRXqMQxyXPkvtJScmnPp78x9qhKIZuS5uLbyaZ41Exw1smGbfEBZpP1trpjR6kusb3F2FzxvixfZMzYpmtYd1IBqRbLQisoLQiKKdt71PhgXyY5hqZo3SowmNL/QPqBdnWtuiKbRSXloFFFx6h8BTUjXSgpPC+CHY0TKiid6LuvMtAdC3CtjZ5R6V+Hn+urOSd6vhu51zazwAFIB46V3ukd8UdXI09+vINz0+cqT0vtwVMvX88bOU+lCZBhNul1VOUpWZ7wUHYPIu9ovjqWZf5mtCJOD95fV7sdLbHYzuHyx/9Gy+cP+B1umgplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMTgyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2QS24EMQhE95yijmAwYHyejqJZdO6/TeGeGck2T5hvpSoGdPGJPZFp+FFp9G34k8h96JaIQnjyTkQpwpwpC5fEcKQaPAtpA65x7CUz8tB9SNnroTUxU+GF2VgdWg5bzB0w/oRBN90Nna85vpYft9iwDjQr9GEN747GmkF3ONY+Rm22n/RMovtLzGjSMc/81QslgzO44eg6vbDrUeASqvRIwc1bKGpS+aG3Yre85Pcfk/FAHQplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9MZW5ndGggNDAyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSTdLdMAjb5xRcoDPm1/g8r/NNF6/331bC6SJBsQOSgIiQJb7l15EIk6glv/UhrJK/A7zl+4SWFGDxT09j+DyuQWDHRFOsW8pxbL0Jvo+hPC70TJrGm6a6CDrBDRptnAWqF5WU8n2C5Tdy8MsS3UhfW0yV+Q5OB+dBTDzgo4bL7Cj1vWiXGDL9tFilhI66VAnrqRDgtHUmom5fpJs3Cun4tmEMZuMEzLHWRAfD1eKtI9R3jEmvuYF5T9rO6dhEPLeae94cQwSX6+QYsleLgstXCAwyfB66Adh7ruu8/9ctkOgcdDPkwjFBm2yH2J7Gha8R2xLJdtaM2uyOHE15TxzjihG6YJhiKT2WjhXO3zFZ32y2F/kaJz12Jo5aIlVuABHVbhBDO/wEdYcaQRhEu7zr9nn+zNjUNlZvhlrcPYNDC+6JQ3HMWGzZ+OEpms8mZt5o02K8QcbONvxiFHcm6EyiAjsDidiy4PIZtju4EUk5BsdFGwZRGKntFFKj0LtkqIAlGK0w8l8z9f/8A6GvncwKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvTGVuZ3RoIDI2OCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1kUluRDEIRPc+BReIZAZj+zw/amXRuf82D1pZWJQYiiocETLFlnxdCTWJnPKtw+8RXSm/o5IaW97Dj8tVcdLLxH2KmzzDbkrAsIvCtJmeoesWUA2e7CPLyYbJrd5mJJg2AwASOjxZNlEyPxxTIlz0rtZlaK3IxMlGbiiOiyai1Ywf6+ky4Gd9ItqetuRRGfY48QTzXdkgZSapYMCXdqRCvTN4aMOzNGq7psJB3iBlo4qxLbBre6OlexInVkZR63T46Ygzpgtp4gh+9dtsVGCo/To5MT4qWusvpLdOmnxNfcpqTGW7HM6DyPx0Z58VQQDXKYl11p1KFzvXLtNqU/5/+hk/4/UHJ55hhwplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggMTY0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDWQQQ5DIQhE955iLmDiIIKe5zc/XbT33xa0f+PgIyODMhoaxkClNnANsAleLLMf9C2elRCfQpmo0eAwVJ1HqbjKrsI7GABUSQnc17533ZL2nliiryB9m9hTruKaOpLkqclCTWANbnBiWrApWIblOyp5IuewP4ksK0K6wD35dBj3bs+K+XIWNc0zxmqksVDf8TTjNp6AFMXzGVd5l/sHi4s4FQplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9MZW5ndGggMjQ3IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC2RS24EMQhE9z4FF4hkPjb4PBNFWczcf5uHOwsLGiiqio4ImWJLvo6EhcSe8q0jZonukM/ooq6U9/DqtrhGD5tezGvo2p2oTp7klmVUw+RY79XIDqZJtZMqgdPZVCZ+elhtSniK5rwCzO1GAOfJ3OtOeCoYpeOHKgpjUrFNZHc2ycmbvMdBpaNguhhPJyqrsYp2XXyBU+cFXWzRYT9Ywz59W9EBGbU7aesbEUDKW8MqgcQxolfCc7TX+G1uh+8zHObO3ngBupQLEAuXSefUPQvHR5PhsO9teOnYep6KLmeCiB8/t6P8iqQyn2P+Ezb3zx+nd1dTCmVuZHN0cmVhbQplbmRvYmoKMjMgMCBvYmoKPDwgL0xlbmd0aCAyMzggL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVFJcgQhDLvzCj3BCzbwnkmlcpj8/xrZTHXl0EhYYKmNu0NgwsW3I/l96TDFDMHvUPFm78uOQCUR1FUVMR2voRqIDKhNxN5Q30iqVDKRcWB6O9s6ja/hcpnTPteCB5E9qCwy7jqNn4tW3ZrxnlcXYd7FBMdKSafz7C6xqUxtpGJxKyp91g4TaVIxVnOykka/gE1vpCLRTPei88XIulPsTqPy/2O6a0KlPkxlfhivKq3LizP0ClwDMe75fwxTUTQ+maqnPcDyewQbLKzN8+B8vQZ4Nhi4HiVXQ/m97it1hnrCh3TSn/H9B/r/XFsKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDkxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE2MOw7AMAhD95yCIwAJoN6nqjo091/LR2m64CfbWBABgdjPsA5qCie1QFKD2bph0pOErnzwp5XQD7hTdFlzqr42VRrrm+r1zr1wZ5LYcFfW4AqjeL0QYiKgCmVuZHN0cmVhbQplbmRvYmoKMjUgMCBvYmoKPDwgL0xlbmd0aCA1NjEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVNLjiUxCNvXKbhApPDJ7zw1as2i3/23YzvzpO4HRRLA2IzerVtsa+HLanWr2e2PP3Rzb/s8uZe83yfPsRzHqk/L9Gt72fvQixG6F164Vebw3yf9mM9ACUTq4FaXfZ+ouJFEBNFwfCPH+/hJeIWKvo7OfE7cdfg4icU7iRPHV+K/u+xxHaStgZvH8CjcbbLDrG5DINa0MdV74Q/Qla9QZSPi1nxeW+xf3kakeBIETQd9MOeAewYynDvA4CEd9IErch24WvRlC1O+Q6lilaO5O7qQjc6h0EOSROYdhqaa2sdPw8WD78AgA+NgDQwfABHFlHdeJMCXZSutkr+Dz5dc1D54SEYiNSo8T052iwny5FMn6EOzZ4GzWSH83OSBybP3OWQOa8JJ0LkHFIBmKCUXk8haRS0cCStyygIoainCk4F3nDrmiakmeMhFqgbyQB0JPgvYErqoyxaFGrzz1env8/ehevj6898bVNCg9sACeAzi3ISypbIjjQYHugrKHLLoba0bAbUOFrIfTQG9RVx9xxEW2asveNm3Ue+JOsFd2UOUAqlzP4gL70EeLU56yHPsRoERX5TXFQ8ykVbimMu+yF6gLAi8gaXP9eq+bdrHIXm82tkGFnJQoSUEtNy6lEdGGmAVL4pxODG0U42biT6aVE45UoQYQAMRNJs9HkmVW6jNgIySuYrsUkVaEg4ZUsfMVfdS/43d/Rku0TQuLph1xvAjcCfsC5fQf/4BC3Ld9AplbmRzdHJlYW0KZW5kb2JqCjI2IDAgb2JqCjw8IC9MZW5ndGggMjM2IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVQO5LFIAzrOYWOgI0N+DzZ2dkie//2SSSvQQo2+sTXRodNHp4Dcw/8WLPdMYfj/2G2YCsxe3IzkZW4miXZ5k040WC+D3Ji8dx0Pxvl75PtR4OWkhTQ42oiMUHrmRtFmFsynYwzszy5zOsgJ/EkVYSzwVDvG8WkyIlNzW+Rq/019euqpLK3gMJ3S9ZnGIcPMPSQQXXQoqSq03roMjk0vhDJUK2xOvbGoAvbDZLCCowKTD9wNaYQGfqmST9LVyOJLn9FjnkSrULkfJm9U5bm76d4BKwYUKJWBQ+4n/BqZvrBT8WbZX8/vBxYlAplbmRzdHJlYW0KZW5kb2JqCjI3IDAgb2JqCjw8IC9MZW5ndGggMTU0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1PSQ4DMQi75xV+QoBA4D1TVT1M/3+tmVSKsANmse7EhBSD+kak4SWj6cR3qBDuBnGiK2RjG3SBQlu4RhnWQiXiiaWdS3gIG5qkoxwxBVkIEdbZGzqxleCUPnCNdf5KGRdIgzHdxIrTknVnwTguiqece6noym5Js9x/4tRkYLW/gPVmnZyvfN5O2qb05uP3Hp/x/gEwpDMrCmVuZHN0cmVhbQplbmRvYmoKMjggMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtOTcwIC00NDMgMjAwMCAxMDIzIF0gL0xlbmd0aCAzNwovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjMyVDAyMlDI5TIzATNywAwLMxADJIdggSUzuNIA/9IKDQplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9MZW5ndGggMjg1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSuXXFMAzrNQVH4KljHuflpfjZvw1AOY0A8wRJl6qo2MSTuySnypcNUpXf4WsBP8AtViChLrYk7IinhINOeUYEUrdEqixrMC3awUKTiWT0oUzC53M1PsNLryUSaOKOyBP0wBpQYnsilyrREm2eQbYcZUkKkiaaQyLkhtKPDHdxDap1P1SPgiAcx/Y/9FDZUZjcQ1iIvdcUmLcKGuA9CRswYiGBZJqwGxazAr2pirlnSR3M4ZLVgCVYE+hJ7VXHuTNIdvsJh0OKbklKqegzMAKZtXpMbG6dZv4yV/SBVkcE40PnmxeZ1wJlmbO3mU5PWrxf2GTVReUFyXYvA6R2K8LJEucNDpGo6VcYdsX/ozd6f5TP+Bnff02QbOEKZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvTGVuZ3RoIDIyNSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UTtuQCEM2zmFL1CJfIBwnldVXXr/tXZeO9kicWKHNScm/OLjIs9E7olPG6Iehp9mtg2ZFzYDOQ9W4RmxFywQZtJ6LsEzrELEwlW99ja32LXH1hW461nkHrAaq5vjbs2gOKNgtduQr2zkUntfYp/uyFl/muwh2SH8vmGU4BnfI4p8JtM0K+npPKgI8nJNVjZb8MOs3O+xGmmSJxKzQ1d2mIA+q7PSf/A2ZkT3Rj+hCpkVe2P3q7Rbie0mIhk/WYdvF+gQSsshR9XIAgPqxrqbLPITcuM/iEJ9/QL+YlIzCmVuZHN0cmVhbQplbmRvYmoKMzEgMCBvYmoKPDwgL0xlbmd0aCAzNDIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVJJbsQwDLv7FfpAAGvxovekKHpI/38tKU+AXkaMLVMkNaN36TLkMp0Sq0vMLl/adKTE2PJbyLfK06yrhLtYZLV596p386HnJFEDJIabzpsYIY4bUnvPqhaLN0C6Av0pquxPmUGyiYMhrilXiuExyt3Mz7epXFOsLyG/pkO9riXQX1KdpwCXQsTzgVChIL84UBfs6q6+TZgGOnblKm0AmwMJGcvTxj+gmQBOspwynLXLNHIansEWxrjUlDFPdWoFcl94TABby8QB50K+VDMVAWIZzBGpRRUaL8BIYDbr925ZgUHRqLSVu0DyOt6dvBt82s/ZIQjPNvf+5AKVCrloYoxRw2DTkLIzaYTvLDjO+nbbGC8OiYsiHBa0Uz/1aVVuBjdEE5JpUq2eeO29OJCUIWT+Rwy2z3/FEE6dKK2DORkQV0GVDi3lI7a8fm54+/4DKHSDbAplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9MZW5ndGggMTQzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2Puw0DMQxDe0/BBQ4w9bNvnguCFMn+bagLkEoEJT1K7o6J2Di4Cj4DpfrgaHmwJj7DTvu138PYrhuYEosgDYlr1IltzWFFF0uTKxFeoE/kknnmjb9uegkuovo1BTw1E6F97eytGUZqWx0PuOK6WnZYK6a4mhCOCuft96HyzJeuVMT/iWu8xvMLQYcr9QplbmRzdHJlYW0KZW5kb2JqCjMzIDAgb2JqCjw8IC9MZW5ndGggMTM5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1POQ7DMAzb9Qp+IICuyPZ7UgQd0v+vpdxOlMQLiggofOGwUfB5oogvE56qFB9J26dHzAYqFC08a8FdkctxiVcgLbc9GNjoOZrpQC7qOAbJ+eu5ZDcy/5E1OGnAMnDQZNQvKmyOttpam2r0zCY4RZKhIL22IWe3te7MCUaWFv4vXPKW+wu1WSuQCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0xlbmd0aCAxMjUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLU5BDgQhCLv7in5gElBQfM9MNnvZ/1+3MCZGSiml3QWCJbhUoKFQETzaEtrAr0AP6HIo3zQ47qY+cLG1s9i9KgeyC+2Z85ilfnd5hEb8zZIT+AYnq6csBrm9sA2ahpKmsypN7TC8mgKGKH2GWlqxPXAi3+3bPn9vXSdUCmVuZHN0cmVhbQplbmRvYmoKMzUgMCBvYmoKPDwgL0xlbmd0aCAyNjUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVE7lgMxCOt9Co5gPgZ8nuzblyJ7/3YlkhRjGARIgLvLloPP/Ujklh9dChOn5W88z5TX0ga8TWynhKWYuURseSwLZrvYudPAeo8Fcms8V2YccYONBOIOz/NNik5jdw2CvMuIirehJsQriGiNFt8NCyU3x4Kn/R0pqm0oyamlNnYDQi6oMfDSAtFPZCPSJdoxtQ/MjP7RMrNDxVg9g2y5m8uAk0SoXrQgjGza4AaZEhJyTxkdVn1fBF/LQa0CoVaSoGZiGdtVCnaPtzl1cUmBAjpIL4xaKtTY3HHiv68ctLjBchouaxy9Mz70XSbjHJvHDFwK0G0cAaOf8znY9/Cv9Vy//0beZD0KZW5kc3RyZWFtCmVuZG9iagozNiAwIG9iago8PCAvTGVuZ3RoIDM0NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtkjtuBDEMQ3ufQhdYwPr4d54NghTJ/ds8erYZamRbIinlPtZtuL2OZW2rcezLm49p2d3+Wsxukdt+W5y08G3p0/xwPY75Dnu3LDefdQt4xYM+7sm27ZZZNsj3abFJx6KMRU21jb4F7+br/js3X7TIATOyvmyYQ4ZSx/R8LWAeK5vDXklGTxKOoA8Fc38COnkvO9wVqTKaUhRCkZCGOE/OEMDLj4JIAGJjwPzS7VbLxBqVgj0/KhwzZJFjQcS4FaWEusdpzycQgsGxS5CppwoWlNIKchvf49JyEtfEdSfhCBaK2TObKKjAKxiQbnKCrsL0wJXiVH2EnKiuMkeZxH691Um63+i3aZiJK0Q4/ERB11SvhX4JGNTplwUqK9Zdg8oFvwdhziRuRswT9RhX8cwPDV1O5a12cZ6P2hzrepilEbBCGXc8LItmoWXqWolnHd/tp33/A7hDgCcKZW5kc3RyZWFtCmVuZG9iagozNyAwIG9iago8PCAvTGVuZ3RoIDI3NiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1kUFuxUAIQ/c5hS9QacDAkPP8quqi//7bmon+IgIBY55JroUFBr6MiGpUB77tiqiTvpWpqMbfxbpRmeAiyglPQ27H67K7kW6wSkQ2LB1sns5CSNopeW/4vk98XeQ+2czFUtU2eNRCuhTS4Hg/WG73dGq0VdGErXXe5DS4N0yv/DHiYpio/btOZpr0hTb0HqEbFjG+ffUJ+8io1I0t5V6aIFJ+paJL5C0F11VcURMV06GFrsNDXtr1udvr+hWUaCX3PhnHn67Adng88XW5nE9m+rjFWfobz07qjW5k2kdBTvSq6SgbXaNciMi0X8YGU6MlSunIUo3JYVwxCWlwbUuxzzR1zHvIRynxoR3yn39b9mPQCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKPDwgL0xlbmd0aCAxNyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwzMjVQQMAUQ640AB2YA00KZW5kc3RyZWFtCmVuZG9iagozOSAwIG9iago8PCAvTGVuZ3RoIDE5MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kMsNwzAMQ++eggsUsGR97HlSFD20+19LOWkOCV/0iShpTnTIwGNBc8H5PKVpBDLxbRUjf5paQgLao0rFZ8nR1irNjjHqLZIMEkwmm/h1gcF0zzHVU5yFBYPhSQmQzIxhdxgHTHYvg2jCXapcJ2mwX6j0wyaPURljJOjLnCs4NfcqzLCzSNyodio3qgxNea/d9sCbTseVvUmG/qn3vX3WOQLndP7XOKIM6z4R5WiqG9TocuA66dHe7fUDAUlF8wplbmRzdHJlYW0KZW5kb2JqCjQwIDAgb2JqCjw8IC9MZW5ndGggMzIyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nCWSS24EIQxE95zCFxgJf4HzdBRlkdx/m2d6MVNu/ClXQc4pU0Lloy7hKrVKvnSUSuqUvxFJEPI7akpWyA6p6aK6pNzkGRpHinI9b7PFi8+wvW7kllS4eNLrm4xvk1ITP0tynRf9dIYo4PAq1tniMySmdiam+E4ynNYi5hc9LRQOzbu+LXtRe1pHWmTIKifdvy5NUcYcO8JyvShi2Ri1H74xpf3Q9IvPWPs1CFf6L/zSsrPQwSyC6rn4sKCYzYLGy3V2Y9NzrrRmdyuL4p+hzYBepN00tu3vg6R2Di/rii/po2QQXRchIpGtQKnEM9a2xCG7WpwTqyvGfV4xjVB3FH2nesglNbqvr+bt6+3mNgNPnEz0TLDfAYwdBXUTRITtI3luD/pqLl4A96w9dV5sxjdSdsnd8X1dz/gZ3/9H9XbhCmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0xlbmd0aCAyMjkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVG7ccUwDOs1BUbgX+I8L5dL4ezfBpQvhQ1YJCGAThEIrPmKHahd+NI1VH3jd4VN/Vm344LaoGyY6LB0WDTs8KQDzop74rPcyHbyayN0v1g6FbIUViyQIZwRZM2MsT8P1UJRYtA+Fz9Ly94Td2QT6SNp87NakZnYBOOjDDIDPok2HVIqvVG8ix5H0HwEnYJ5oLtvbPNzkSbaL/M5yWYIdtpUQupeHlTIlBdpnhWyYBw/jBPGVSQsr3EaodiznA5ecmpIiIPdoQUmHolGc0XB3WtyVQKNcfr/N571s77/AG3XURYKZW5kc3RyZWFtCmVuZG9iago0MiAwIG9iago8PCAvTGVuZ3RoIDI1NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNkUtuxDAMQ/c+BS8wgPWxLJ8nRdFFev9tKXWCdhM9R7FIKmtOTBy8DjwnfE18yCgkfTdY4B6uCVVYKjbMDa/ENcwSr8V2wY6G+AU9DjlQj5otx6pcQ9bqsy4oZBoI10ihOrZgn3qK1HCCxeGoAt6PgAt9JVwn+07LemAGN2cEp/Q1Tme4W+iPRPdDk75CkMS9sGuS5AFbOqNsqHa5hpYUzxklatTeFcHoLrkEpaizJGdKL4POJz+kI6HDrra7Y4ySvR7S+1aFMa7IO63NihNUqyTCvVS1tvFkctn/SOZ+U1qvJGoziSgztYzF10vqR/r2KtfzY+/xNT5/AHN0XpAKZW5kc3RyZWFtCmVuZG9iago0MyAwIG9iago8PCAvTGVuZ3RoIDI3OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UEtOBTEM288pfAGk5tv2PA8hNtx/i50H0ozipm5iu9bCgic+LJC70fw/7RGMWPgZ5Kd4aeSR1AFfgdeTlTCeMg3mG8kxN3WxNnohTiL4iVZsh9bw8bmzzWtNfT2+3h3bIYKlIPtmjtg4B13YDqPY11MF49ghGafDzTRkz0Tu8OsqEa02wT0IQ5YcokZ7O4rnwzMlmwfaZ2VdLrt06BOFU7wq1dsdFBTSyf2WqLMmh0JZT3wZjv/sXs/3E2SEF3MUSnajD9ms2ajuv1zaqfkad8vCmapgepDtGoZR8vuN0ZCmmGuqTQ2/uiFyb4hh9Ks3V5Y1JOSMPZ+sWLiDaSnnCWpUDVsy9VyynQb+jcjU1y96GGieCmVuZHN0cmVhbQplbmRvYmoKMTUgMCBvYmoKPDwgL1R5cGUgL0ZvbnQgL0Jhc2VGb250IC9DWlJCTFIrU1RJWEdlbmVyYWwtUmVndWxhciAvRmlyc3RDaGFyIDAKL0xhc3RDaGFyIDI1NSAvRm9udERlc2NyaXB0b3IgMTQgMCBSIC9TdWJ0eXBlIC9UeXBlMwovTmFtZSAvQ1pSQkxSK1NUSVhHZW5lcmFsLVJlZ3VsYXIgL0ZvbnRCQm94IFsgLTk3MCAtNDQzIDIwMDAgMTAyMyBdCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9DaGFyUHJvY3MgMTYgMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcKL0RpZmZlcmVuY2VzIFsgMzIgL3NwYWNlIDQwIC9wYXJlbmxlZnQgL3BhcmVucmlnaHQgNDQgL2NvbW1hIDQ2IC9wZXJpb2QgNDggL3plcm8gNTAKL3R3byAvdGhyZWUgL2ZvdXIgNTQgL3NpeCA4MCAvUCA4NCAvVCA5NyAvYSA5OSAvYyAxMDEgL2UgL2YgL2cgMTA1IC9pIDEwOAovbCAxMTAgL24gL28gL3AgMTE0IC9yIC9zIC90IC91IF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0NaUkJMUitTVElYR2VuZXJhbC1SZWd1bGFyIC9GbGFncyAzMgovRm9udEJCb3ggWyAtOTcwIC00NDMgMjAwMCAxMDIzIF0gL0FzY2VudCAxMDU1IC9EZXNjZW50IC00NTUgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDExMDkgPj4KZW5kb2JqCjEzIDAgb2JqClsgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAKMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMzMzIDQwOCA1MDAgNTAwCjc0NyA3NzggMTgwIDMzMyAzMzMgNTAwIDY4NSAyNTAgMzMzIDI1MCAyNzggNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMAo1MDAgNTAwIDI3OCAyNzggNjg1IDY4NSA2ODUgNDQ0IDkyMSA3MjIgNjY3IDY2NyA3MjIgNjExIDU1NiA3MjIgNzIyIDMzMyAzNzMKNzIyIDYxMSA4ODkgNzIyIDcyMiA1NTcgNzIyIDY2NyA1NTYgNjExIDcyMiA3MjIgOTQ0IDcyMiA3MjIgNjEyIDMzMyAyNzggMzMzCjQ2OSA1MDAgMzMzIDQ0NCA1MDAgNDQ0IDUwMCA0NDQgMzMzIDUwMCA1MDAgMjc4IDI3OCA1MDAgMjc4IDc3OCA1MDAgNTAwIDUwMAo1MDAgMzMzIDM4OSAyNzggNTAwIDUwMCA3MjIgNTAwIDUwMCA0NDQgNDgwIDIwMCA0ODAgNTQxIDI1MCA1MDAgMjUwIDMzMyA0MzQKNDQ0IDEwMDAgNTAwIDUwMCAzMzMgMTEwOSA1NTYgMzMzIDg4OSAyNTAgNjEyIDI1MCAyNTAgMzMzIDMzMyA0NDQgNDQ0IDUyMwo1MDAgMTAwMCAzMzMgOTgwIDM4OSAzMzMgNzIyIDI1MCA0NDQgNzIyIDI1MCAzMzAgNTAwIDUwMCA1MDAgNTAwIDIwMCA1MDAKMzMzIDc2MCAyNzYgNTAwIDYwMCAzMzMgNzYwIDMzMyA0MDAgNjg1IDMwMCAzMDAgMzMzIDUwMCA1OTIgMjUwIDMzMyAzMDAgMzEwCjUwMCA3NTAgNzUwIDc1MCA0NDQgNzIyIDcyMiA3MjIgNzIyIDcyMiA3MjIgODg5IDY2NyA2MTEgNjExIDYxMSA2MTEgMzMzIDMzMwozMzMgMzMzIDcyMiA3MjIgNzIyIDcyMiA3MjIgNzIyIDcyMiA2NDAgNzIyIDcyMiA3MjIgNzIyIDcyMiA3MjIgNTU2IDUwMCA0NDQKNDQ0IDQ0NCA0NDQgNDQ0IDQ0NCA2NjcgNDQ0IDQ0NCA0NDQgNDQ0IDQ0NCAyNzggMjc4IDI3OCAyNzggNTAwIDUwMCA1MDAgNTAwCjUwMCA1MDAgNTAwIDU2NCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9QIDE3IDAgUiAvVCAxOCAwIFIgL2EgMTkgMCBSIC9jIDIwIDAgUiAvY29tbWEgMjEgMCBSIC9lIDIyIDAgUgovZiAyMyAwIFIgL2ZvdXIgMjQgMCBSIC9nIDI1IDAgUiAvaSAyNiAwIFIgL2wgMjcgMCBSIC9uIDI5IDAgUiAvbyAzMCAwIFIKL3AgMzEgMCBSIC9wYXJlbmxlZnQgMzIgMCBSIC9wYXJlbnJpZ2h0IDMzIDAgUiAvcGVyaW9kIDM0IDAgUiAvciAzNSAwIFIKL3MgMzYgMCBSIC9zaXggMzcgMCBSIC9zcGFjZSAzOCAwIFIgL3QgMzkgMCBSIC90aHJlZSA0MCAwIFIgL3R3byA0MSAwIFIKL3UgNDIgMCBSIC96ZXJvIDQzIDAgUiA+PgplbmRvYmoKNDggMCBvYmoKPDwgL0xlbmd0aCAzNTAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVJBkhwxCLv3K/hAVxkwGL9ntlI5bP5/jYS9e2lkyyABbatkyKsz5bWRMm1K4u5Ln6k4lfx71Gej70dzSKiJ1pTILczIofJ5bJpkmFhl57tXx8/jaeemHNHE95ZEnQ8EQnLW0WwNRE0yRAM3uiU2o0pUM8Ml1kIVRkdV6widdZAnvK0Uj0TOJAON2Co+oTfGieiCDBG8of20QM6AE/bjAbcOnaBHvbF1iILVFiJ03DuCGdHISjvX2PnvdCJDDBoxwOiZLXK07mxdOZsEMnR+kLrJIqiNHeGB+VnTm+uAz/OS6zsdRcR18p0WBV7FJl5deZesMHURmiff2Jkw7SfVg3qHt8XS/CgrDb9KROB33svqdzgVHWiTi48N59Wu9gVzNNlw4k4JSOKg0bKL1RLEYuuY3QW7SUJ+Fh0XyNgH0HAdqN0IFsVR0Y1ywz1X/xnw/ce/n7/Pn//2kpCnCmVuZHN0cmVhbQplbmRvYmoKNDkgMCBvYmoKPDwgL0xlbmd0aCAzNzAgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPZJLcu0wCETnWgUbcJX46Lcev0plcLP/aQ7YeQNXtwWChtboIV2uFXIZNJZJhMo/baZDwrb8NN1THPZp5lOiu9jxSvOxCu/m70kYBbzDwb6JxDRxKmRpH+NBnRmBmdM0uug44mvLqWKEjS6x5VJiaxTeTYc+J6i5QrRPyULr5KnLnODYLx4ixeaRxYy6llTmzh47a+9BjUiyn5pJaJM01/Fp/zfz0kyADku65PKVhL+6TcbdLs0mMzVnVvCXKmxkZVbiqP40VVgwg4X43A+iMLvDTio7rDBk94K7MVmUksE2aYIf+tj1edyCmeKInxeznLHMcLaZGTjz5+vdvpvv9MLxuBiepQSfzrdeOemxY0tFD6KtF1L5RDFDfjpsmTHj7emo0cNdZUgWbTsjihMWE7856WRkVpwaewnpSnleTELMOqbdsyY2qAyiZRG9lKuD/YgioASR+8o5NVc+ipzgkIKSne/twYzmBr5+ATZjjqAKZW5kc3RyZWFtCmVuZG9iago1MCAwIG9iago8PCAvTGVuZ3RoIDMxOCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxVUkluxDAMu+cV/EABa7Ot90xRzCHz/2spOSjQkwhLISkqMQYGNPClY8O3w13wLVdD3fhcJokviYGbcBUMPCAfMAWvp7kFZpMgFbb8ADZ3HngfWGr3JaH/4BmQ3ZC2Rn2/AyrFuBaJlJ4omFB7vKgbATWpYgYJI4emoRho0RdUOe8ckEwIZBbvgNTerK8rrQFpB0Io3GTwjqbEAjKi5AliwvhE29Xp3bOSoDZdGNu1geV47DqldVkna7K7kn6O88L9nIKWjHUlOy7S0d9/R7ivN2PjDNf5HLSi460wbZbT5qR3pw9jNs6zmUrXV+dRSKdyVlG+bI2OxHte5oQmq9NtFJuo9glEKrJxarMVmhzt5TpO16Yq47zWRpDEJ9YubfqVsbg7a9T/scmbnQE1SdQbyVOjOu/r5xeLb3voCmVuZHN0cmVhbQplbmRvYmoKNTEgMCBvYmoKPDwgL0xlbmd0aCA0NTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLZNJjiQxCEX3PgUXCMkMns6TpVIvou+/rQfORQSI4fMZHBHS5bEpj3aJWHwqP9osXHwM+d+Mv50tb7PTsZm4bvF9xIMU0j7Np0k43h0FEH2W/LQgtiwGtOc3KhIPFcL8FtW4skd5lvhJj19E8nzt9LiiaeX5BFmvxNNXaX7gtfrlt2ZyW2gb1jPlQGpJPHE179lLiE3nSwalZd+Tjy6NmqYnc7oLKW/LpEFNrKS6J93CLMXH9e4uu8wQ2LO6SWJ7Cf2l+cjJZk20Q/oEshxnopHA4JVK0VVUrQY6KwYGoAw6hrEhgshKZUZPLslrpwbmU0syRlgWmnroC37VayYCkcOgS/aDAzPVlQG+TVkhprfB1I7QEiATnagBeLK8wP6ttEo+5XvgkbbHVFZdml7hWlYOJflyPwkyZeTOMp0V0AZ1Vv4+TUcVHemgYMWRM3qS42LoKjIOzlHjJU4BPBmozNGYH93kcXj1lUBkaS2KyQ+tTRAdYLEPJS5PUi2+p5mAZWEDuZO8npNV593i22aG5mtZYNyKbCcPV+d9XUrcfRzW73OxPH8GbzkGyyUagw7NdeTRwP37HD/tX/v9AwICr4EKZW5kc3RyZWFtCmVuZG9iago0NiAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0ZUWFdaQStTVElYR2VuZXJhbC1JdGFsaWMgL0ZpcnN0Q2hhciAwCi9MYXN0Q2hhciAyNTUgL0ZvbnREZXNjcmlwdG9yIDQ1IDAgUiAvU3VidHlwZSAvVHlwZTMKL05hbWUgL0ZUWFdaQStTVElYR2VuZXJhbC1JdGFsaWMgL0ZvbnRCQm94IFsgLTk3MCAtMzA1IDE0MjkgMTAyMyBdCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9DaGFyUHJvY3MgNDcgMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcgL0RpZmZlcmVuY2VzIFsgMTAyIC9mIDExMiAvcCAvcSAxMjAgL3ggXSA+PgovV2lkdGhzIDQ0IDAgUiA+PgplbmRvYmoKNDUgMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9Gb250TmFtZSAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRmxhZ3MgOTYKL0ZvbnRCQm94IFsgLTk3MCAtMzA1IDE0MjkgMTAyMyBdIC9Bc2NlbnQgMTA1NSAvRGVzY2VudCAtNDU1IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSA0MzkwOSAvU3RlbVYgMCAvTWF4V2lkdGggMTExNyA+PgplbmRvYmoKNDQgMCBvYmoKWyAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MAoyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAzMzMgNDIwIDUwMSA1MDAKNzU1IDc3OCAyMTQgMzMzIDMzMyA1MDAgNjc1IDI1MCAzMzMgMjUwIDI3OCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwCjUwMCA1MDAgMzMzIDMzMyA2NzUgNjc1IDY3NSA1MDAgOTIwIDYxMSA2MTEgNjY3IDcyMiA2MTEgNjExIDcyMiA3MjIgMzMzIDQ0NAo2NjcgNTU2IDgzMyA2NjcgNzIyIDYxMSA3MjIgNjExIDUwMCA1NTYgNzIyIDYxMSA4MzMgNjExIDU1NiA1NTYgMzg5IDI3OCAzODkKNDIyIDUwMCAzMzMgNTAxIDUwMCA0NDQgNTAwIDQ0NCAyNzggNTAwIDUwMCAyNzggMjc4IDQ0NCAyNzggNzIyIDUwMCA1MDAgNTA0CjUwMCAzODkgMzg5IDI3OCA1MDAgNDQ0IDY2NyA0NDQgNDQ0IDM4OSA0MDAgMjc1IDQwMCA1NDEgMjUwIDUwMCAyNTAgMzMzIDQ3Mgo1NTYgODg5IDUwMCA1MDAgMzMzIDExMTcgNTAwIDMzMyA5NDQgMjUwIDU1NiAyNTAgMjUwIDMzMyAzMzMgNTU2IDU1NiA1MjMKNTAwIDg4OSAzMzMgOTgwIDM4OSAzMzMgNjY3IDI1MCAzODkgNTU2IDI1MCAzODkgNTAwIDUwMCA1MDAgNTAwIDI3NSA1MDAgMzMzCjc2MCAyNzYgNTAwIDY3NSAzMzMgNzYwIDMzMyA0MDAgNjc1IDMwMCAzMDAgMzMzIDUwMCA1NTkgMjUwIDMzMyAzMDAgMzEwIDUwMAo3NTAgNzUwIDc1MCA1MDAgNjExIDYxMSA2MTEgNjExIDYxMSA2MTEgODg5IDY2NyA2MTEgNjExIDYxMSA2MTEgMzMzIDMzMyAzMzMKMzMzIDcyMiA2NjcgNzIyIDcyMiA3MjIgNzIyIDcyMiA2NzUgNzIyIDcyMiA3MjIgNzIyIDcyMiA1NTYgNjExIDUwMCA1MDEgNTAxCjUwMSA1MDEgNTAxIDUwMSA2NjcgNDQ0IDQ0NCA0NDQgNDQ0IDQ0NCAyNzggMjc4IDI3OCAyNzggNTAwIDUwMCA1MDAgNTAwIDUwMAo1MDAgNTAwIDY3NSA1MDAgNTAwIDUwMCA1MDAgNTAwIDQ0NCA1MDAgNDQ0IF0KZW5kb2JqCjQ3IDAgb2JqCjw8IC9mIDQ4IDAgUiAvcCA0OSAwIFIgL3EgNTAgMCBSIC94IDUxIDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSIC9GMiA0NiAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDAgL2NhIDEgPj4KL0EyIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4KL0EzIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDAuOCAvY2EgMC44ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1TVElYR2VuZXJhbC1taW51cyAyOCAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjUyIDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkKL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTIyMTc0NiswMScwMCcpID4+CmVuZG9iagp4cmVmCjAgNTMKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMTg0MjQgMDAwMDAgbiAKMDAwMDAxODE0NyAwMDAwMCBuIAowMDAwMDE4MTkwIDAwMDAwIG4gCjAwMDAwMTgzMzIgMDAwMDAgbiAKMDAwMDAxODM1MyAwMDAwMCBuIAowMDAwMDE4Mzc0IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MyAwMDAwMCBuIAowMDAwMDA0MjQzIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwNDIyMiAwMDAwMCBuIAowMDAwMDEzMzIxIDAwMDAwIG4gCjAwMDAwMTMxMDUgMDAwMDAgbiAKMDAwMDAxMjYwNiAwMDAwMCBuIAowMDAwMDE0MzY4IDAwMDAwIG4gCjAwMDAwMDQyNjMgMDAwMDAgbiAKMDAwMDAwNDYyNyAwMDAwMCBuIAowMDAwMDA0ODgyIDAwMDAwIG4gCjAwMDAwMDUzNTcgMDAwMDAgbiAKMDAwMDAwNTY5OCAwMDAwMCBuIAowMDAwMDA1OTM1IDAwMDAwIG4gCjAwMDAwMDYyNTUgMDAwMDAgbiAKMDAwMDAwNjU2NiAwMDAwMCBuIAowMDAwMDA2NzI5IDAwMDAwIG4gCjAwMDAwMDczNjMgMDAwMDAgbiAKMDAwMDAwNzY3MiAwMDAwMCBuIAowMDAwMDA3ODk5IDAwMDAwIG4gCjAwMDAwMDgwNjggMDAwMDAgbiAKMDAwMDAwODQyNiAwMDAwMCBuIAowMDAwMDA4NzI0IDAwMDAwIG4gCjAwMDAwMDkxMzkgMDAwMDAgbiAKMDAwMDAwOTM1NSAwMDAwMCBuIAowMDAwMDA5NTY3IDAwMDAwIG4gCjAwMDAwMDk3NjUgMDAwMDAgbiAKMDAwMDAxMDEwMyAwMDAwMCBuIAowMDAwMDEwNTIzIDAwMDAwIG4gCjAwMDAwMTA4NzIgMDAwMDAgbiAKMDAwMDAxMDk2MSAwMDAwMCBuIAowMDAwMDExMjI3IDAwMDAwIG4gCjAwMDAwMTE2MjIgMDAwMDAgbiAKMDAwMDAxMTkyNCAwMDAwMCBuIAowMDAwMDEyMjU0IDAwMDAwIG4gCjAwMDAwMTcwNDAgMDAwMDAgbiAKMDAwMDAxNjgyMSAwMDAwMCBuIAowMDAwMDE2NDc1IDAwMDAwIG4gCjAwMDAwMTgwODUgMDAwMDAgbiAKMDAwMDAxNDY5NCAwMDAwMCBuIAowMDAwMDE1MTE3IDAwMDAwIG4gCjAwMDAwMTU1NjAgMDAwMDAgbiAKMDAwMDAxNTk1MSAwMDAwMCBuIAowMDAwMDE4NDg0IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgNTMgL1Jvb3QgMSAwIFIgL0luZm8gNTIgMCBSID4+CnN0YXJ0eHJlZgoxODY0MQolJUVPRgo=", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T22:17:45.889719\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(8, 3))\n", "\n", "# Plot proposal\n", "plt.plot(\n", " x_plot,\n", " p_plot,\n", " color=\"tab:blue\",\n", " label=\"Target $p$\",\n", ")\n", "\n", "# Plot target\n", "plt.plot(\n", " x_plot,\n", " q_plot,\n", " color=\"tab:green\",\n", " label=\"Proposal $q$\",\n", ")\n", "\n", "plt.title(\"Target $p$, proposal $q$, function $f$\", fontsize=22)\n", "plt.xlabel(\"$x$\", fontsize=18)\n", "plt.ylabel(\"$p(x),~ q(x)$\", fontsize=18)\n", "\n", "plt.xticks(tf.linspace(-4., 4., 5), fontsize=18)\n", "plt.yticks(tf.linspace(0., 0.6, 3), fontsize=18)\n", "\n", "plt.xlim([-4., 4.])\n", "plt.ylim([0, 0.65])\n", "\n", "plt.legend(loc=\"upper left\", fontsize=16)\n", "\n", "twin_axis = plt.gca().twinx()\n", "\n", "plt.plot(\n", " x_plot,\n", " f_plot,\n", " color=\"tab:red\",\n", " label=\"$f(x)$\",\n", ")\n", "\n", "plt.xlabel(\"$x$\", fontsize=18)\n", "plt.ylabel(\"$f(x)$\", fontsize=18)\n", "\n", "plt.legend(loc=\"upper right\", fontsize=16)\n", "plt.yticks(tf.linspace(-60., 60., 3), fontsize=18)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Using importance sampling, we can obtain an estimate of this intergral." ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "remove-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Importance Sampling estimate I = -5.40\n" ] } ], "source": [ "# Set the random seed for repeatable results\n", "tf.random.set_seed(0)\n", "\n", "# How many samples to draw\n", "num_samples = 100\n", "\n", "# Draw samples\n", "samples = q.sample([num_samples])\n", "\n", "# Compute importance weights and importance weighted integral\n", "w = tf.math.exp(p.log_prob(samples) - q.log_prob(samples))\n", "I = tf.reduce_mean(w * f(samples))\n", "\n", "print(f\"Importance Sampling estimate I = {I:.2f}\")" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Note that the exact value of the integral is $I = 0$, which we can see because $p(x)$ is symmetric around $x = 0$ while $f(x) = x^3$ is antisymmetric.\n", "However the estimate was $I \\approx -5.46$ which seems a bit off.\n", "Repeating this experiment $100$ times we obtain:" ] }, { "cell_type": "code", "execution_count": 48, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "remove-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Importance Sampling over 100 trials I = 0.10 +/- 13.309.\n" ] } ], "source": [ "# How many samples to draw\n", "num_samples = 100\n", "num_trials = 100\n", " \n", "# Draw samples\n", "samples = q.sample([num_trials, num_samples])\n", "\n", "# Compute importance weights and importance weighted integral\n", "w = tf.math.exp(p.log_prob(samples) - q.log_prob(samples))\n", "I = tf.reduce_mean(w * f(samples), axis=1)\n", "\n", "mean_I = tf.reduce_mean(I)\n", "std_I = tf.math.reduce_std(I)\n", "\n", "print(\n", " f\"Importance Sampling over {num_trials} trials \"\n", " f\"I = {mean_I:.2f} +/- {2.*std_I:.3f}.\"\n", ")" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "So even though the estimator is unbiased, it has a large variance.\n", "Before we go further, let's compare the importance sampling estimate to a Monte Carlo estimate using samples directly from $p$.\n", "Note that in practice this is not possible, because $p$ is not in general available in closed form, but that's why we picked $p$ to be a mixture of Gaussians.\n", "The point of this additional Monte Carlo estimate is to give us an idea of how much random error we should expect if we had access to exact samples from $p$, which we cannot expect to beat by simply reducing the variance in the importance weights." ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "remove-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Monte Carlo with samples from p over 100 trials I = -0.07 +/- 1.848.\n" ] } ], "source": [ "# How many samples to draw\n", "num_samples = 100\n", "num_trials = 100\n", " \n", "# Draw samples\n", "px = q.sample([num_trials, num_samples])\n", "\n", "# Compute importance weights and importance weighted integral\n", "I = tf.reduce_mean(w * f(px), axis=1)\n", "\n", "mean_I = tf.reduce_mean(I)\n", "std_I = tf.math.reduce_std(I)\n", "\n", "print(\n", " f\"Monte Carlo with samples from p over {num_trials} trials \"\n", " f\"I = {mean_I:.2f} +/- {2.*std_I:.3f}.\"\n", ")" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "The Importance sampling estimator has seven times larger random error than the Monte Carlo estimator.\n", "To see why, this occurs, let's plot the raw samples drawn from $q$ (in green) as well as the samples weighted according to their importance weights (in blue)." ] }, { "cell_type": "code", "execution_count": 50, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "center-output", "remove-input" ] }, "outputs": [ { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNTY5LjYyODEyNSAyNDYuMTI2MjUgXSAvQ29udGVudHMgOSAwIFIgL0Fubm90cyAxMCAwIFIgPj4KZW5kb2JqCjkgMCBvYmoKPDwgL0xlbmd0aCAxMiAwIFIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnic7V1dj922EX3Xr9CjAyQ0vz8enaYxmrfUBlKg7UPg2E6C3SS2kzo/v2eG+uBoqV37NhdlgS7gZO9ciZpDDmcOJR7t4y9e/uuHFy//+vTz+U/Ppsf7pxfvJjP/iH+vZz3/iH/vZzM/xb/Xk8an2ynEoqLNxgZ8vGk/Wh+VsRG/3eDY9tP30/RqevwEjbzDSU+nkFSag1ZR0/doU3tldsPNYrAmqrI0wae0Bm7zzSyb8rion02Mytn57cv5m/mn+fETy9dVJtngizYl4kO0yQddsvaT+OYt4X56fvR8OHq6AwYdYnw0wbWAukZ55gLp63kEUB1/b6esVSjW6dg63TV24Q4Er+P07VSs0i77UFqnu8Yu5oHgdZy+nYx2ymeMihiUvrWLeiCAPa+B0HhVnE5RjMuJtYt7JIgdtwHRwocUixEjc2LtAh8JYsdtQHRJZUyuZAWYvrULfCSIHbcB0RdlY7H40ILpW7vAR4LYcRsQo1YJBSBlAaaxJlVSugN8t44EsQMGEJNVBkXOy/HqW7vdMRLEjtuAmB1+xwSTo7hbY1DOZxuMRN6aRwLZgQOQJSgdXA6CgjZWAbKPfSSQHTi3k9VRBY0f34Jsrc18bFoYcj72wACigbPEOkWwNlbEZzKlDljTRGseCWQHDkDaohy4pxHB2lgFyD72kUB24ACk1yqDf0YZrLvVOKNiRlMH8MI+EswOIMAMVlnioGIV2FiNLiokj6YO8Fv7SDA7gAAzOpVAYCTKzWisV87EBWQDvrWPBPIuHGBMAceBw4jlRGM1riCn5iVid/CteSSQHTxAmSPOclqw7d1o6PaMDwa4JPbWPhLIu3CAsWSl4baXI7lbFzRLvB7BjxeuHTy3k6MMAj5axNqwsUqUJ+gHgtkDBJhWq0KUVERsYy0eORULDyfRt+aRQHbgAKSzyiOZaBGxjTVrpYtGapXYW/NIIDtwANI7lYmTyoDdrQJkH/tIIDtwADJgSUGkVIbrbm2XHE0Tg65EenAAEj4kVPgobmg0VjGSexOjjmQHDkCmrAxxUnEX4MTa7ZCRIHbcBsRcVCQ+KpZbJ9Yu8JEgdty+nbw2SsNoxcj0rX3gA0HsuQ2IxqpAVFSMzIm1C3wkiB23AdE6VYiJipE5sXaBjwSx4zYguqDwjc7irsCJtQt8JIgdtwERnmRaToh7AifWLvCRIHbcBsSQseJFSRcr5RNrF/hIEDtuA2IsKhH9lCHZt3aBjwSx4zYgZqMs3SGWIdm3doGPBLHjNiAWqyIGxsuQ7Fu7wEeC2HH7dEvKg3A3YI+fuLrd5fVsaCONyvP76tr8+u4GGPrskzJLK9Pnk5nfT2/wXz1/ptECvtQuFIfC5AwoMf/24nb6/Pn0+Eszmzw/fzWheDliyybTzVA9P/9u+vv8yH8y/3N+/tX05+fTG/iBb2fayrP/9uJ2RhufPXv+l789ffnTy7ff3nx2+8NPv72bv/h5/nr6emLfJ4NYlY/b2XCv1wZTOF7otv1j3LZYN8gbPGy4122bKAzPHdabk3p3cr2gQ1xIosyGey/ocv6gy9m7lzvE5Rq59wdTAVf9oAv69oKHHqJ7XstFagM8OPQR5z7+0i6NPfr9k/n5j8LlY/R7JRhlZ0AyjvBe+ZB4q9k9A6I6Q8IXpL0Ctr3ibji9pLFo7wOuGU6uacGYc3vN3XB+zRKQVx66ppE430x0/mfUEly2c8moK3Jkct2lJ8fmFx6bonQypegSY65X2C/56B+P+JigSnElZ+2SWY85DHHCIabkYlxId5v55KSZR5/yF4glj+kP4MGUOxd4U/28snPclR9alTi92/krTu8GM/w91RZrQswoNwFh4OPygwuqpEPO1gVb5r8+neWOy3ZGBGdoqx2xO5N15eg5exSrmwmRkZ2PidNw8cYYy1uZNBzjDwG11lvAZLNHqCVUKDIjGkyKbC7KorPoUZSlFavR/Fwb/R/Qe0iVTtkQtPFsdSqhatK9OK+09cEmNkd8cNFEMpfkbKwbjhC/CCEPR2jzJ23V5E06mCBwksxJBQvH2BFrFbyztGAu9KggJL6kRYvwDtw9GurPWAqbk7LwGjQiOuUS9TGbi/IZV6J7YDhLR24DqS3ZiA6JWeWcrOUmHBY+saAD6WmowRVc3WCDNICZEcqcgkouhFTNGekmpGTnRFtcQ3bciLfoeYvunnNWJpjqBroBtc5myolAS3zHsR2Ro03QCEaD9IVejoyd6qL3ydLxCS7GOghRk7tonzNPdknXZgDZ0ORAFGL1RnHh2Rm68Ze8i/VpGYymDltMCsMdMsyAVKz31VwQVRpBwfaSLe3VWDZuYJAx+GT36JHMxyevHJx0lps32pZcj48qFO1xMg2iziUEjriE3nbwhtBiXBw6lQMAFBA9knEyRY7VOMgteymc1brAH1p365hqZOSgQnSYCEZjEGzSdfwKkrEvGB2Ur0BHc9eXqBK6DF2M04p3FSoO0RYhEGZcHDPGm7rfARMk0kDNKdEIIDbY7HB1IDJzQozgxGpF3wV+CpAwmRMgRzZTKgk+z+hlB8pS+Ek7sq7LNOrRI1oKQoDNFuFp6MRoEWVs8ojNgA6aMdoICIQBm1FQI2Y3qAMte7zha6EbEWrUvYikhGmr+XmMRbIMPiHZBXQVskqFgdiIJRUMHuLRJ48+ZzP1C2YSTe0IZxL7gWJjsvf0hA8hbYuubThNPR5dJLNLKAqMz2FWGq/pvgj6EytrHinMSMS3RrYgM+Jcu/qkHtPf6WX/L+It2/pwNyCIHPE3ymrBZs/+YSiCC4i2JQUWdtARu0CHWGY3SJ+Gg9XRngY0XpNgziXxqDgEoisFzEZ4gvSCY0Kx0m9H3AeBTTtcFpTVTB1Ek170iXPIu4Gqy9aD9QlLpjmha3Lk7ubHSB7pGIkSMdsODoAotOHJzEOJWcNmetSEiSzG3RHdTz5TEmyCxHkwbpr9hsMJQ5P5aEALkfpEBJ+Ds8jd9GGJ1MT+4fIasePWsEbGYTOSN/qB0uA+BxwuH2jylHXCcOJxuHxCE1lOL0x9TC+UFDNjrlKNMTySyBeoFrHI+Uw3hT2ilZNmnf6WHQRBT7Sd45AuXMaYxOjYXtNLHWPMe/QFLaLXdMQ5wOWkqGDFLNMX36kFzpi2dJfYn2IUZpjBHBbp0RWcjAi2MpsCHrCg38uWfGuXgXtFKtlRJmvMEwSu1Zw1ObmjFLMdByFRIujbWuCRfdCURy0XpQNzEOOAVJ7WUpNMWu5cGlA8tE4ck+owH26Jo6P6Ij1SLUflWW8CIvQsJcKmugEPgjRhiNZaWLhtFN/swE28qJweuUJTELi1zlYPHaaUSTqLmozEjCDVtGm+LeAoXghSF3RYyn3ikPG0L4KYjhXkwHvk74zsmwWV8B75G3O6psmNeKBAIUrpprygKQhirCHRxkZqHB+NdGJJyiIYEAKIIlPXfLjRJY/rFERCtIJceWDwKeNHUDHPHQFm5zfiRgGN0opABLXjh4DJ+swJBP9DQNAYUDrkMi5vPxBF7FmmZ/8hed1ujEznt0w+gLy2PHDJ3F7yL0FfMUqo+B4lAn3mEFxc40CtMF2ISUn6ujOntj41TGgdPSsZj+CvO7ER/HXnL0sYVVqz05Ql6JadoImWciDNkr9itY8zC4ZS8Nedpizh77wkJMtkCVFSj2VmuQPLaOdhQylaBtuQh2WOB0kSloTAVaGp+0vqXvbcoSihVCBE19zN2Yb2aaGsIhluubu6gh7CAFvibm3uBhOjHRXoI0kNLQqWphLkJOO1FK+Gy/JGMZm1oDvCsuGmTa7wC4OBNhuqymaHNJLJN0GQaXOSRaaPRjLehopsub5u5CoYvUy3okVpsChJGkNs3NGOxT6tYdzeDndbpl2nyHcHpm2xlkhYS8SjnxgLkBq/2LcShgUcAoGED4duwArR40Mxkpnbkii/GN5W13YzaHKh8u3ksDhN91roAocSjBgotFAIkrE7rBM9xsweKDvtNsEaCMOwVqXEma9hVA1pb5iTYBU7RWpJe0OGWsKyk56WtDfspmHtDY0RxGmnMStrR7HGAGHNmyQh25hNS952/tKS9oamCF64s5GWtO+koyXtDYdYq5GXnICypkcmYjPyAhx3njmxR6mp9Xmv/YEWZrrmnKZAt6S9LYALaee6jUjD2tesz4ITMYw/vEytd/63uypze1dlEndV9jv//aPnw9H/deXmdUANo9y8DrxhlJvXgTeQcvM6AIdSbl4J4kjKzStBHEm5eSWIIyk3rwTxMuWmR0UvWD3pA3RhHwnmSOrNK0G8TL1Jt28dFsLM/5o2hH0kmJfpN40DhQX7rc9M9jaEfSSYlyk44aPBKqtYib41DwTyQg0n1mnWalsfbDTYG/NIIC/TcGIhGLRZHiztLezWkSBepuDEqlZn/F4OyBvzSCAv02/Sbc7lQZloYDGOBPAS5SbfDwiWH9ocGljNI0G8SLfJt+C940cYsoXVOhLES1Sby+2bnO8CX80jQbxIs8njRQN2B/lqHQniZYpNsoKh1of7sonVPBDIC/Wa9DzJ2xLLHeybeSSQl+k16TGYK8aVA/bGPBLIy/SaIaiYQNjSAXtjHgnkZXpNeuqQMXb2gL0xjwTyMr0mbRLECtny44KmDWEfCeZIis0rQRxJsXkliCMpNq8DcSjF5pUgjqTYvBLEkRSbV4I4kmLzShBHUmxeCeJIis0rQRxJsXkliCMpNq8E8aqKTdoVuas15V7Hrm6zfSv59OxMkHc86qFXoK/H7OedtKTJ30Vlatjr10KCiWVHma21yzm7IKxglRVWIRQYAgmhnlctllPeW62N83YRhD36dlGKpZi1JqXQKt96u8irnI1ae22zW754vei6lo8vzxr4lb+gpxShlORom3/9Yl60XWGXdC1u/rJKxnROOMe7uB2zQflUnn1sbWvjIRg/y+MOpx2+fccfswoh0eZ82uL1QPfdfDz6N+KaHwH51RnG3ySKn+THFx89cj+cffHzvRc6h/xq1+c9fuJpmm5/oQBTtf59gilm0qTSrLHrnc+gTGOhm38kI6Ofxky7Lx1rlVOxxaTVEPU6Z6Zq4W25W0vVkmgfLO0lb5q2tE1yaZo82D+/qE7uhpuJ1pI10e3WGFg8urW9fF6vjlbIsDt4sxh2CHurO85D/7yg1Pf59JC+Xn7D2fr9HY39PRvGkbHpyfJsiqHAs3VHkxeWm8ViNcj8YuGTWsPyVySErN40ovo26RmWJ2/tN/rktKtgcVzk6Hq2zNpcEK5RFyeTHkmpSV0afVrj+LYGrkWepImuQQtEfshidvu7h708a/ndMnWsdaUUTXSxnRxeTOUAfKnkFJNeG3579sXPwrFzAOI6LKONTdZZpuD9Bf5Ysuf1T4kcC/09ItktbBC1RobNZlnDxqSyBVI9qTF8fNhs7T8QNt/UPgx3B/F0dH84i4fXYnS+F59+/egg+k6cLyPnNMD+H+zdYG9fsxAtcoRZuFhScTf05PyBFIy8B9zbrpy//+KLqP+YN19Ul0l1U18FsPq8W86dJrl1NA+++ODsdRQW/42+7ajdcs9FS1bg6A9cNJ69+CAEUh7w2cgwpEASUzg1fHd7N8AaWRqRWVLWtvNugMFeMZBdqfnT0ZtvkCpJ1kH8Vy1Z1QdzqtJCZltiIWKe2hKDpb0z4HUsR0oknSUCMeM3mDzL/xKpEnPxbk6kpsXl6BFUxoAa4idzRhgHm1loUVjCRROZ1Jc4gGwZ+TNh4s+kjbI6mjV1L+8nKJHlCyxE2l9EQO81SC55brfRlNGJAcSAxcPN6wJIeaL1Whi2VwBQg+AhpcrHdlU/bbKl9034gwiNikAi1USVm20qNBLIoKPr44xWwG8J8gJfKPKdAsMOvA/iILHXhpTIs5TM28jAdd2ltkngSXuZfHaMtgC6I03gTG8+cDnGur1kF7Abx3sTchWK7/pz6hCS+lZF+KYfpz1wzmW9yLk3ERjZc7GLVtwboIVvnpOHA7XjXmsE3WSPvvAbc1vVFecaW9XVGsOMgSjVjELP0dVqhByGJCWMzyw02mSPXtNiVKiEMNEwtN6Hqh7a9EBkL56DSEh/6FUPhfY3sDu7yoeiAhGQq/ZpVyu7Qm9iibHqdHb1sSdVVnC8N6JVE5PEG6vzzBKnRh7svUK0liphbeS+JJPP/E4LshdF7/mgCCTRHfUIy3IQjhjPQBFIj7kjvwZCqHhMwHod04elXo0ol94+4VwILIVshLaY4vATy1OW/O7aWXrNhDa2vmJ1l8NSVkoF4VXlQJvw1ZDmruhcdby0N4ZeiwH3E+YA5msVGpMKKxjaym7oDa6JFVYIPSBH/1IGpS0npergEOYYIEwUtFJoG5FxVSNNH5AISA5I/VpQxlkPVbCYTUhUmVcKyVlWYK1FCAER23tBrWVRIf2P3vRZFsGaa8Prymf3hbBHskzc32sjdOurNJYq4NLLSngze3QuVSxDY0zLyNXgV0Y8VUu7El4t22q1aXpf0lYP2pVw9bJdClOsh+Na2Bu6IdA2vxj21XC17F7ebJYdyN50A/fQT9uK+IHCO39g4Z0QPSTALxy121i4JCw0qdzBIu7rddcqnmZld4lLyTQc+OyrE2rerGY2nnMk000zv58Q+baZludM/wZuCxiXCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKNDk3OQplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagoxNyAwIG9iago8PCAvTGVuZ3RoIDM5NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kktuBCEMRPd9Cl9gJPwFzjNRlEXn/ts806NIrbbBuKgqk1kyJFxe6hJbpWbJl15RW6Jcfq+IONl9hZvkYKVb0lNiuGROeV9eJjW2uIYUFct14vsym8/O4ISV6OLEWFS0SnIPURCy1hNjdKWzPltDElparId2ZUJycXbBeU7igttB2yWRm3vgHUvMiA8DVmEqVs2ciMBABRW6fU9x+PhKcS9xdKMHXu6TPjSNRg76Gi0m2O0IVpmOE5sVlc6Unqkyjcsd0IPFafHIY7GteWLzekw3uLwSkVv2EQ8M2uFimGPWgW2c7fVANB9uahuC6hQueNn5q7HH5Z30vET3Jqn8JIo7qwm3QSjSDlBEIonjFyP2tr8Z+twyod5Dwv4TPT72KKDOGBTZXm3GacEwiwl0S0XGxgU7uuDrUOPOdpXwjLQT3MWWRIyisewMNHkw1fhx3qRhYMce2ziZKzsM35N1Hl6sKqgsegMl3RutsAffWSCzEBc8rwctYNvZ/f/k7+vn+v4D3jSTYQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMTgyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2QS24EMQhE95yijmAwYHyejqJZdO6/TeGeGck2T5hvpSoGdPGJPZFp+FFp9G34k8h96JaIQnjyTkQpwpwpC5fEcKQaPAtpA65x7CUz8tB9SNnroTUxU+GF2VgdWg5bzB0w/oRBN90Nna85vpYft9iwDjQr9GEN747GmkF3ONY+Rm22n/RMovtLzGjSMc/81QslgzO44eg6vbDrUeASqvRIwc1bKGpS+aG3Yre85Pcfk/FAHQplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9MZW5ndGggMzExIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE2STW7FMAiE9z4FF6hk/vF5UlVdpPffdiDvqd2EUQzDB/Yxo01OH7zpqFCE0Cevt/x5lCnd62yn0KSqRGSqRBSla1VAsVN5kZ9DZYc8+yTx1TiUqOU0OD2trhW6R93L6xBvIdcgMSHLTcaCHOV45ejml5JEj040YmditBMYX4vdSE8Qi8GiLQExeAfeydS4aBMK1sJvQIaiZoMKGTPltXzmRp8+/lOzARGbXNm9AbQ4hejdvNpsExfqRZ44M4zafQJuaAaiR9eo9d8DX7NG7l6Gid2yFWr8bNIKOCVptnN2XU+Ae1AwNY2yvdhHDbPn+a+G3lEdxuSzhP3E3Zto1ffmmMsLJwztc0vRswSqs/nFW3Wm4x3gTr1kYi8W76CVFGXxkKW+tzpqeN4v617f6+sXoQh4fgplbmRzdHJlYW0KZW5kb2JqCjIwIDAgb2JqCjw8IC9MZW5ndGggNDAyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSTdLdMAjb5xRcoDPm1/g8r/NNF6/331bC6SJBsQOSgIiQJb7l15EIk6glv/UhrJK/A7zl+4SWFGDxT09j+DyuQWDHRFOsW8pxbL0Jvo+hPC70TJrGm6a6CDrBDRptnAWqF5WU8n2C5Tdy8MsS3UhfW0yV+Q5OB+dBTDzgo4bL7Cj1vWiXGDL9tFilhI66VAnrqRDgtHUmom5fpJs3Cun4tmEMZuMEzLHWRAfD1eKtI9R3jEmvuYF5T9rO6dhEPLeae94cQwSX6+QYsleLgstXCAwyfB66Adh7ruu8/9ctkOgcdDPkwjFBm2yH2J7Gha8R2xLJdtaM2uyOHE15TxzjihG6YJhiKT2WjhXO3zFZ32y2F/kaJz12Jo5aIlVuABHVbhBDO/wEdYcaQRhEu7zr9nn+zNjUNlZvhlrcPYNDC+6JQ3HMWGzZ+OEpms8mZt5o02K8QcbONvxiFHcm6EyiAjsDidiy4PIZtju4EUk5BsdFGwZRGKntFFKj0LtkqIAlGK0w8l8z9f/8A6GvncwKZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iago8PCAvTGVuZ3RoIDI2OCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1kUluRDEIRPc+BReIZAZj+zw/amXRuf82D1pZWJQYiiocETLFlnxdCTWJnPKtw+8RXSm/o5IaW97Dj8tVcdLLxH2KmzzDbkrAsIvCtJmeoesWUA2e7CPLyYbJrd5mJJg2AwASOjxZNlEyPxxTIlz0rtZlaK3IxMlGbiiOiyai1Ywf6+ky4Gd9ItqetuRRGfY48QTzXdkgZSapYMCXdqRCvTN4aMOzNGq7psJB3iBlo4qxLbBre6OlexInVkZR63T46Ygzpgtp4gh+9dtsVGCo/To5MT4qWusvpLdOmnxNfcpqTGW7HM6DyPx0Z58VQQDXKYl11p1KFzvXLtNqU/5/+hk/4/UHJ55hhwplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9MZW5ndGggMTY0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDWQQQ5DIQhE955iLmDiIIKe5zc/XbT33xa0f+PgIyODMhoaxkClNnANsAleLLMf9C2elRCfQpmo0eAwVJ1HqbjKrsI7GABUSQnc17533ZL2nliiryB9m9hTruKaOpLkqclCTWANbnBiWrApWIblOyp5IuewP4ksK0K6wD35dBj3bs+K+XIWNc0zxmqksVDf8TTjNp6AFMXzGVd5l/sHi4s4FQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMzE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROXIDMQzr9xX8gGfEU9R7nPGkcP7fBuDaFSGKBwDmWrLEtjyOxFGpdvnRizBM/i6P4NcbYMmjPiA3wWpRF6tkhakxPC/NIlC8dcluyYUsxvThHg1lsJXIEuyU2OKe0i3eTKu1BBK6t0SBHjAjG3SQr5KIFJIJ3fi5afmX4HGM3FKLoyFLgzWWUoZucCsLsRMTMbdvZNhYBlHbJnIjkI8DDpTY2ZhXsEvP2AVx6Cxk34MUu250QM5KqJC9zVJvkJLIMTGQS7IPbOF7nwlwP6f6PsP7+h1Vuu6TQIHbR6mj0O2AlcEVsEIN9QCZzXG427wn4gcHJ9JRAI9pPrjT9YCfuBCO6ksnWsw9gBRKFf6pQyHYH9JWSON9uBdLcthyuxE4zE8EdO2xnze+b3ZCvoKeEPf6Bx4/eEwKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtkUtuBDEIRPc+BReIZD42+DwTRVnM3H+bhzsLCxooqoqOCJliS76OhIXEnvKtI2aJ7pDP6KKulPfw6ra4Rg+bXsxr6NqdqE6e5JZlVMPkWO/VyA6mSbWTKoHT2VQmfnpYbUp4iua8AsztRgDnydzrTngqGKXjhyoKY1KxTWR3NsnJm7zHQaWjYLoYTycqq7GKdl18gVPnBV1s0WE/WMM+fVvRARm1O2nrGxFAylvDKoHEMaJXwnO01/htbofvMxzmzt54AbqUCxALl0nn1D0Lx0eT4bDvbXjp2Hqeii5ngogfP7ej/IqkMp9j/hM2988fp3dXUwplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggMjM4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSXIEIQy78wo9wQs28J5JpXKY/P8a2Ux15dBIWGCpjbtDYMLFtyP5fekwxQzB71DxZu/LjkAlEdRVFTEdr6EaiAyoTcTeUN9IqlQykXFgejvbOo2v4XKZ0z7XggeRPagsMu46jZ+LVt2a8Z5XF2HexQTHSkmn8+wusalMbaRicSsqfdYOE2lSMVZzspJGv4BNb6Qi0Uz3ovPFyLpT7E6j8v9jumtCpT5MZX4Yryqty4sz9ApcAzHu+X8MU1E0Ppmqpz3A8nsEGyyszfPgfL0GeDYYuB4lV0P5ve4rdYZ6wod00p/x/Qf6/1xbCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0xlbmd0aCAyOTMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVJLbgUhDNtzCl+gEvkQwnleVXUxvf+2Nq/qhngIsRNn1pyYCMOHBTIa1Y1PG29o+Bk5HesUnhGnsXoj+jAGYq8bX8P4UugZlobku2e4JbITXhtZjaBSLufrCKdUIYqSk5lj5JrMJGuCyukFP+sdUwpCphtr2CIzuzK/bF1gQ1GO4iROvsVrb0fAWabJbPWNbNXnRSz52Ch2cdkhZaOozmzxGjb7I3lh35PFnIHAPMHZbU0saRnzQZFz4NI0BbbApL5ZkfL4FpFYapNCi0zZ73juKES+xEVzJ7HmSmWc3QQ3YxwxzdWQwmvQWIGiI+RXcBlZ8m5pH5u3tIbb447+EMWEcupGO5AR+/ofKG4iOUJt7o1+6Id4/f8Pz/geX78Qp2k6CmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0xlbmd0aCA5MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNjDsOwDAIQ/ecgiMACaDep6o6NPdfy0dpuuAn21gQAYHYz7AOagontUBSg9m6YdKThK588KeV0A+4U3RZc6q+NlUa65vq9c69cGeS2HBX1uAKo3i9EGIioAplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9MZW5ndGggNTYxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVTS44lMQjb1ym4QKTwye88NWrNot/9t2M786TuB0USwNiM3q1bbGvhy2p1q9ntjz90c2/7PLmXvN8nz7Ecx6pPy/Rre9n70IsRuhdeuFXm8N8n/ZjPQAlE6uBWl32fqLiRRATRcHwjx/v4SXiFir6OznxO3HX4OInFO4kTx1fiv7vscR2krYGbx/Ao3G2yw6xuQyDWtDHVe+EP0JWvUGUj4tZ8XlvsX95GpHgSBE0HfTDngHsGMpw7wOAhHfSBK3IduFr0ZQtTvkOpYpWjuTu6kI3OodBDkkTmHYammtrHT8PFg+/AIAPjYA0MHwARxZR3XiTAl2UrrZK/g8+XXNQ+eEhGIjUqPE9OdosJ8uRTJ+hDs2eBs1kh/Nzkgcmz9zlkDmvCSdC5BxSAZiglF5PIWkUtHAkrcsoCKGopwpOBd5w65ompJnjIRaoG8kAdCT4L2BK6qMsWhRq889Xp7/P3oXr4+vPfG1TQoPbAAngM4tyEsqWyI40GB7oKyhyy6G2tGwG1DhayH00BvUVcfccRFtmrL3jZt1HviTrBXdlDlAKpcz+IC+9BHi1Oeshz7EaBEV+U1xUPMpFW4pjLvsheoCwIvIGlz/Xqvm3axyF5vNrZBhZyUKElBLTcupRHRhpgFS+KcTgxtFONm4k+mlROOVKEGEADETSbPR5JlVuozYCMkrmK7FJFWhIOGVLHzFX3Uv+N3f0ZLtE0Li6YdcbwI3An7AuX0H/+AQty3fQKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDI4NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtUrtxxTAM6zUFR+Cf0jzO5V7h7N8GpF5hA7IFEoQUzMR08Pguym30I6sp09/SCuA7KAFi7CSbTI00yEzIgp5ldsixdqOsAWHt72DGXWFYMRnUzkmaZ/BZ6pepwQILqTjZabUc1NpKUuiYCQe3Nv6AdYd3GFpKShsQVB1DUtUG5UTbVR376KV3nvN935kgPxTRgtSrb3dOhXGy33cakMDzDjlO2JS8KbELnp/VVhRDYK2OpgPwkkOk111qAJ9BENv0Ty+QjVowIApBQRA653HHzc3fca2y2eYbGxu5Ylqo3HkCRcSBA0INT0TOF3EcO4f1fucaNET+rGY30GYI1Ps0ApCIvje4UqvikN6rMoHei/Kuz/r9BzplbO0KZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvTGVuZ3RoIDIzNiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UDuSxSAM6zmFjoCNDfg82dnZInv/9kkkr0EKNvrE10aHTR6eA3MP/Fiz3TGH4/9htmArMXtyM5GVuJol2eZNONFgvg9yYvHcdD8b5e+T7UeDlpIU0ONqIjFB65kbRZhbMp2MM7M8uczrICfxJFWEs8FQ7xvFpMiJTc1vkav9NfXrqqSyt4DCd0vWZxiHDzD0kEF10KKkqtN66DI5NL4QyVCtsTr2xqAL2w2SwgqMCkw/cDWmEBn6pkk/S1cjiS5/RY55Eq1C5HyZvVOW5u+neASsGFCiVgUPuJ/wamb6wU/Fm2V/P7wcWJQKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvTGVuZ3RoIDE1NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtT0kOAzEIu+cVfkKAQOA9U1U9TP9/rZlUirADZrHuxIQUg/pGpOElo+nEd6gQ7gZxoitkYxt0gUJbuEYZ1kIl4omlnUt4CBuapKMcMQVZCBHW2Rs6sZXglD5wjXX+ShkXSIMx3cSK05J1Z8E4LoqnnHup6MpuSbPcf+LUZGC1v4D1Zp2cr3zeTtqm9Obj9x6f8f4BMKQzKwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9MZW5ndGggNDIxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1TO3IFMQjr9xQcwebv8ySTSfFy/zYSzkuz0oIxCHBVy5Kd+FSFeC753A/pkp8nooCvwR0kHbJLcqmoSe4lpvLx5G7xLakK18BZNIPoScSRmSUiUiwdp1ysGRoHnnMkysRXS3gMwqP7WhYRXuS24/B4HkS7eBhuO+LWYl70gB0WSpKIcWVhHkiyJxIV4b839QTqByAVCHVq5z+OXtuj18wZZk49OG8B4TCHS+QAAmjGsVOMA7NCBNpDDdrnT5XWupZYo0pxoR0WobBao9EHqgrYVEfPLtwWrARp/fxhjwdzY8UksMxJRygKYT94ly7UvjljRVNGL8lE5RtGrTuRLcKt6Bhn2Gw+5cXht5kVyGG+hmDgfQuuW//Hw1i0gf3HosQeQPMvMQpH46l7NKA1kx574ah6Yxic246efcSJTIywplJkRDqyfrPD5YNgRbyFKDbBPd69Rt/03M02jn5uNG5bshBuj05eK87Q61rC7onAJt+YzBpWK0cAUZsesmJ5JMhXi9qkuOx7+jHPpXAlHkrNBGnGKxvh97m9nu/n6xekBKoQCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtOTcwIC00NDMgMjAwMCAxMDIzIF0gL0xlbmd0aCAzNwovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjMyVDAyMlDI5TIzATNywAwLMxADJIdggSUzuNIA/9IKDQplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9MZW5ndGggMjg1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSuXXFMAzrNQVH4KljHuflpfjZvw1AOY0A8wRJl6qo2MSTuySnypcNUpXf4WsBP8AtViChLrYk7IinhINOeUYEUrdEqixrMC3awUKTiWT0oUzC53M1PsNLryUSaOKOyBP0wBpQYnsilyrREm2eQbYcZUkKkiaaQyLkhtKPDHdxDap1P1SPgiAcx/Y/9FDZUZjcQ1iIvdcUmLcKGuA9CRswYiGBZJqwGxazAr2pirlnSR3M4ZLVgCVYE+hJ7VXHuTNIdvsJh0OKbklKqegzMAKZtXpMbG6dZv4yV/SBVkcE40PnmxeZ1wJlmbO3mU5PWrxf2GTVReUFyXYvA6R2K8LJEucNDpGo6VcYdsX/ozd6f5TP+Bnff02QbOEKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvTGVuZ3RoIDIyNSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UTtuQCEM2zmFL1CJfIBwnldVXXr/tXZeO9kicWKHNScm/OLjIs9E7olPG6Iehp9mtg2ZFzYDOQ9W4RmxFywQZtJ6LsEzrELEwlW99ja32LXH1hW461nkHrAaq5vjbs2gOKNgtduQr2zkUntfYp/uyFl/muwh2SH8vmGU4BnfI4p8JtM0K+npPKgI8nJNVjZb8MOs3O+xGmmSJxKzQ1d2mIA+q7PSf/A2ZkT3Rj+hCpkVe2P3q7Rbie0mIhk/WYdvF+gQSsshR9XIAgPqxrqbLPITcuM/iEJ9/QL+YlIzCmVuZHN0cmVhbQplbmRvYmoKMzYgMCBvYmoKPDwgL0xlbmd0aCAxNDYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY9RDsMwCEP/cwofAYfQhPN0mvbR3f93Jq0mRfiJCGPCDAaSqp4DxzzwYis0fBu5pNdWRkEMcOoluoPp8MDZuh0YE50uhy2524IYQ3Ob5N1NqimuW8/G+XRiIVaCXsr6UaqY3OtFq/b3tJ2xKEX+0L+lfG4BmTiVMeXj3pVUQWfdUKexwt03Xu3T3j8cmTFkCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKPDwgL0xlbmd0aCAzNDIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVJJbsQwDLv7FfpAAGvxovekKHpI/38tKU+AXkaMLVMkNaN36TLkMp0Sq0vMLl/adKTE2PJbyLfK06yrhLtYZLV596p386HnJFEDJIabzpsYIY4bUnvPqhaLN0C6Av0pquxPmUGyiYMhrilXiuExyt3Mz7epXFOsLyG/pkO9riXQX1KdpwCXQsTzgVChIL84UBfs6q6+TZgGOnblKm0AmwMJGcvTxj+gmQBOspwynLXLNHIansEWxrjUlDFPdWoFcl94TABby8QB50K+VDMVAWIZzBGpRRUaL8BIYDbr925ZgUHRqLSVu0DyOt6dvBt82s/ZIQjPNvf+5AKVCrloYoxRw2DTkLIzaYTvLDjO+nbbGC8OiYsiHBa0Uz/1aVVuBjdEE5JpUq2eeO29OJCUIWT+Rwy2z3/FEE6dKK2DORkQV0GVDi3lI7a8fm54+/4DKHSDbAplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjw8IC9MZW5ndGggMTQzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2Puw0DMQxDe0/BBQ4w9bNvnguCFMn+bagLkEoEJT1K7o6J2Di4Cj4DpfrgaHmwJj7DTvu138PYrhuYEosgDYlr1IltzWFFF0uTKxFeoE/kknnmjb9uegkuovo1BTw1E6F97eytGUZqWx0PuOK6WnZYK6a4mhCOCuft96HyzJeuVMT/iWu8xvMLQYcr9QplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9MZW5ndGggMTM5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1POQ7DMAzb9Qp+IICuyPZ7UgQd0v+vpdxOlMQLiggofOGwUfB5oogvE56qFB9J26dHzAYqFC08a8FdkctxiVcgLbc9GNjoOZrpQC7qOAbJ+eu5ZDcy/5E1OGnAMnDQZNQvKmyOttpam2r0zCY4RZKhIL22IWe3te7MCUaWFv4vXPKW+wu1WSuQCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0xlbmd0aCAxMjUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLU5BDgQhCLv7in5gElBQfM9MNnvZ/1+3MCZGSiml3QWCJbhUoKFQETzaEtrAr0AP6HIo3zQ47qY+cLG1s9i9KgeyC+2Z85ilfnd5hEb8zZIT+AYnq6csBrm9sA2ahpKmsypN7TC8mgKGKH2GWlqxPXAi3+3bPn9vXSdUCmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0xlbmd0aCAyNjUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVE7lgMxCOt9Co5gPgZ8nuzblyJ7/3YlkhRjGARIgLvLloPP/Ujklh9dChOn5W88z5TX0ga8TWynhKWYuURseSwLZrvYudPAeo8Fcms8V2YccYONBOIOz/NNik5jdw2CvMuIirehJsQriGiNFt8NCyU3x4Kn/R0pqm0oyamlNnYDQi6oMfDSAtFPZCPSJdoxtQ/MjP7RMrNDxVg9g2y5m8uAk0SoXrQgjGza4AaZEhJyTxkdVn1fBF/LQa0CoVaSoGZiGdtVCnaPtzl1cUmBAjpIL4xaKtTY3HHiv68ctLjBchouaxy9Mz70XSbjHJvHDFwK0G0cAaOf8znY9/Cv9Vy//0beZD0KZW5kc3RyZWFtCmVuZG9iago0MiAwIG9iago8PCAvTGVuZ3RoIDM0NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtkjtuBDEMQ3ufQhdYwPr4d54NghTJ/ds8erYZamRbIinlPtZtuL2OZW2rcezLm49p2d3+Wsxukdt+W5y08G3p0/xwPY75Dnu3LDefdQt4xYM+7sm27ZZZNsj3abFJx6KMRU21jb4F7+br/js3X7TIATOyvmyYQ4ZSx/R8LWAeK5vDXklGTxKOoA8Fc38COnkvO9wVqTKaUhRCkZCGOE/OEMDLj4JIAGJjwPzS7VbLxBqVgj0/KhwzZJFjQcS4FaWEusdpzycQgsGxS5CppwoWlNIKchvf49JyEtfEdSfhCBaK2TObKKjAKxiQbnKCrsL0wJXiVH2EnKiuMkeZxH691Um63+i3aZiJK0Q4/ERB11SvhX4JGNTplwUqK9Zdg8oFvwdhziRuRswT9RhX8cwPDV1O5a12cZ6P2hzrepilEbBCGXc8LItmoWXqWolnHd/tp33/A7hDgCcKZW5kc3RyZWFtCmVuZG9iago0MyAwIG9iago8PCAvTGVuZ3RoIDI3NiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1kUFuxUAIQ/c5hS9QacDAkPP8quqi//7bmon+IgIBY55JroUFBr6MiGpUB77tiqiTvpWpqMbfxbpRmeAiyglPQ27H67K7kW6wSkQ2LB1sns5CSNopeW/4vk98XeQ+2czFUtU2eNRCuhTS4Hg/WG73dGq0VdGErXXe5DS4N0yv/DHiYpio/btOZpr0hTb0HqEbFjG+ffUJ+8io1I0t5V6aIFJ+paJL5C0F11VcURMV06GFrsNDXtr1udvr+hWUaCX3PhnHn67Adng88XW5nE9m+rjFWfobz07qjW5k2kdBTvSq6SgbXaNciMi0X8YGU6MlSunIUo3JYVwxCWlwbUuxzzR1zHvIRynxoR3yn39b9mPQCmVuZHN0cmVhbQplbmRvYmoKNDQgMCBvYmoKPDwgL0xlbmd0aCAxNyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwzMjVQQMAUQ640AB2YA00KZW5kc3RyZWFtCmVuZG9iago0NSAwIG9iago8PCAvTGVuZ3RoIDE5MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kMsNwzAMQ++eggsUsGR97HlSFD20+19LOWkOCV/0iShpTnTIwGNBc8H5PKVpBDLxbRUjf5paQgLao0rFZ8nR1irNjjHqLZIMEkwmm/h1gcF0zzHVU5yFBYPhSQmQzIxhdxgHTHYvg2jCXapcJ2mwX6j0wyaPURljJOjLnCs4NfcqzLCzSNyodio3qgxNea/d9sCbTseVvUmG/qn3vX3WOQLndP7XOKIM6z4R5WiqG9TocuA66dHe7fUDAUlF8wplbmRzdHJlYW0KZW5kb2JqCjQ2IDAgb2JqCjw8IC9MZW5ndGggMjI5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRu3HFMAzrNQVG4F/iPC+XS+Hs3waUL4UNWCQhgE4RCKz5ih2oXfjSNVR943eFTf1Zt+OC2qBsmOiwdFg07PCkA86Ke+Kz3Mh28msjdL9YOhWyFFYskCGcEWTNjLE/D9VCUWLQPhc/S8veE3dkE+kjafOzWpGZ2ATjowwyAz6JNh1SKr1RvIseR9B8BJ2CeaC7b2zzc5Em2i/zOclmCHbaVELqXh5UyJQXaZ4VsmAcP4wTxlUkLK9xGqHYs5wOXnJqSIiD3aEFJh6JRnNFwd1rclUCjXH6/zee9bO+/wBt11EWCmVuZHN0cmVhbQplbmRvYmoKNDcgMCBvYmoKPDwgL0xlbmd0aCAyNTcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTZFLbsQwDEP3PgUvMID1sSyfJ0XRRXr/bSl1gnYTPUexSCprTkwcvA48J3xNfMgoJH03WOAerglVWCo2zA2vxDXMEq/FdsGOhvgFPQ45UI+aLceqXEPW6rMuKGQaCNdIoTq2YJ96itRwgsXhqALej4ALfSVcJ/tOy3pgBjdnBKf0NU5nuFvoj0T3Q5O+QpDEvbBrkuQBWzqjbKh2uYaWFM8ZJWrU3hXB6C65BKWosyRnSi+Dzic/pCOhw662u2OMkr0e0vtWhTGuyDutzYoTVKskwr1UtbbxZHLZ/0jmflNaryRqM4koM7WMxddL6kf69irX82Pv8TU+fwBzdF6QCmVuZHN0cmVhbQplbmRvYmoKNDggMCBvYmoKPDwgL0xlbmd0aCAyNzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVBLTgUxDNvPKXwBpObb9jwPITbcf4udB9KM4qZuYrvWwoInPiyQu9H8P+0RjFj4GeSneGnkkdQBX4HXk5UwnjIN5hvJMTd1sTZ6IU4i+IlWbIfW8PG5s81rTX09vt4d2yGCpSD7Zo7YOAdd2A6j2NdTBePYIRmnw800ZM9E7vDrKhGtNsE9CEOWHKJGezuK58MzJZsH2mdlXS67dOgThVO8KtXbHRQU0sn9lqizJodCWU98GY7/7F7P9xNkhBdzFEp2ow/ZrNmo7r9c2qn5GnfLwpmqYHqQ7RqGUfL7jdGQpphrqk0Nv7ohcm+IYfSrN1eWNSTkjD2frFi4g2kp5wlqVA1bMvVcsp0G/o3I1NcvehhongplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQ1pSQkxSK1NUSVhHZW5lcmFsLVJlZ3VsYXIgL0ZpcnN0Q2hhciAwCi9MYXN0Q2hhciAyNTUgL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMKL05hbWUgL0NaUkJMUitTVElYR2VuZXJhbC1SZWd1bGFyIC9Gb250QkJveCBbIC05NzAgLTQ0MyAyMDAwIDEwMjMgXQovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA0MCAvcGFyZW5sZWZ0IC9wYXJlbnJpZ2h0IDQ0IC9jb21tYSA0NiAvcGVyaW9kIDQ4IC96ZXJvIC9vbmUKL3R3byA1MiAvZm91ciAvZml2ZSAvc2l4IDgzIC9TIC9UIDg3IC9XIDk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOAovbCAvbSAvbiAvbyAvcCAxMTQgL3IgL3MgL3QgL3UgXQo+PgovV2lkdGhzIDEzIDAgUiA+PgplbmRvYmoKMTQgMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9Gb250TmFtZSAvQ1pSQkxSK1NUSVhHZW5lcmFsLVJlZ3VsYXIgL0ZsYWdzIDMyCi9Gb250QkJveCBbIC05NzAgLTQ0MyAyMDAwIDEwMjMgXSAvQXNjZW50IDEwNTUgL0Rlc2NlbnQgLTQ1NSAvQ2FwSGVpZ2h0IDAKL1hIZWlnaHQgMCAvSXRhbGljQW5nbGUgMCAvU3RlbVYgMCAvTWF4V2lkdGggMTEwOSA+PgplbmRvYmoKMTMgMCBvYmoKWyAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MAoyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAzMzMgNDA4IDUwMCA1MDAKNzQ3IDc3OCAxODAgMzMzIDMzMyA1MDAgNjg1IDI1MCAzMzMgMjUwIDI3OCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwCjUwMCA1MDAgMjc4IDI3OCA2ODUgNjg1IDY4NSA0NDQgOTIxIDcyMiA2NjcgNjY3IDcyMiA2MTEgNTU2IDcyMiA3MjIgMzMzIDM3Mwo3MjIgNjExIDg4OSA3MjIgNzIyIDU1NyA3MjIgNjY3IDU1NiA2MTEgNzIyIDcyMiA5NDQgNzIyIDcyMiA2MTIgMzMzIDI3OCAzMzMKNDY5IDUwMCAzMzMgNDQ0IDUwMCA0NDQgNTAwIDQ0NCAzMzMgNTAwIDUwMCAyNzggMjc4IDUwMCAyNzggNzc4IDUwMCA1MDAgNTAwCjUwMCAzMzMgMzg5IDI3OCA1MDAgNTAwIDcyMiA1MDAgNTAwIDQ0NCA0ODAgMjAwIDQ4MCA1NDEgMjUwIDUwMCAyNTAgMzMzIDQzNAo0NDQgMTAwMCA1MDAgNTAwIDMzMyAxMTA5IDU1NiAzMzMgODg5IDI1MCA2MTIgMjUwIDI1MCAzMzMgMzMzIDQ0NCA0NDQgNTIzCjUwMCAxMDAwIDMzMyA5ODAgMzg5IDMzMyA3MjIgMjUwIDQ0NCA3MjIgMjUwIDMzMCA1MDAgNTAwIDUwMCA1MDAgMjAwIDUwMAozMzMgNzYwIDI3NiA1MDAgNjAwIDMzMyA3NjAgMzMzIDQwMCA2ODUgMzAwIDMwMCAzMzMgNTAwIDU5MiAyNTAgMzMzIDMwMCAzMTAKNTAwIDc1MCA3NTAgNzUwIDQ0NCA3MjIgNzIyIDcyMiA3MjIgNzIyIDcyMiA4ODkgNjY3IDYxMSA2MTEgNjExIDYxMSAzMzMgMzMzCjMzMyAzMzMgNzIyIDcyMiA3MjIgNzIyIDcyMiA3MjIgNzIyIDY0MCA3MjIgNzIyIDcyMiA3MjIgNzIyIDcyMiA1NTYgNTAwIDQ0NAo0NDQgNDQ0IDQ0NCA0NDQgNDQ0IDY2NyA0NDQgNDQ0IDQ0NCA0NDQgNDQ0IDI3OCAyNzggMjc4IDI3OCA1MDAgNTAwIDUwMCA1MDAKNTAwIDUwMCA1MDAgNTY0IDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgXQplbmRvYmoKMTYgMCBvYmoKPDwgL1MgMTcgMCBSIC9UIDE4IDAgUiAvVyAxOSAwIFIgL2EgMjAgMCBSIC9jIDIxIDAgUiAvY29tbWEgMjIgMCBSCi9kIDIzIDAgUiAvZSAyNCAwIFIgL2YgMjUgMCBSIC9maXZlIDI2IDAgUiAvZm91ciAyNyAwIFIgL2cgMjggMCBSIC9oIDI5IDAgUgovaSAzMCAwIFIgL2wgMzEgMCBSIC9tIDMyIDAgUiAvbiAzNCAwIFIgL28gMzUgMCBSIC9vbmUgMzYgMCBSIC9wIDM3IDAgUgovcGFyZW5sZWZ0IDM4IDAgUiAvcGFyZW5yaWdodCAzOSAwIFIgL3BlcmlvZCA0MCAwIFIgL3IgNDEgMCBSIC9zIDQyIDAgUgovc2l4IDQzIDAgUiAvc3BhY2UgNDQgMCBSIC90IDQ1IDAgUiAvdHdvIDQ2IDAgUiAvdSA0NyAwIFIgL3plcm8gNDggMCBSID4+CmVuZG9iago1MyAwIG9iago8PCAvTGVuZ3RoIDM1MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UkGSHDEIu/cr+EBXGTAYv2e2Ujls/n+NhL17aWTLIAFtq2TIqzPltZEybUri7kufqTiV/HvUZ6PvR3NIqInWlMgtzMih8nlsmmSYWGXnu1fHz+Np56Yc0cT3lkSdDwRCctbRbA1ETTJEAze6JTajSlQzwyXWQhVGR1XrCJ11kCe8rRSPRM4kA43YKj6hN8aJ6IIMEbyh/bRAzoAT9uMBtw6doEe9sXWIgtUWInTcO4IZ0chKO9fY+e90IkMMGjHA6JktcrTubF05mwQydH6QuskiqI0d4YH5WdOb64DP85LrOx1FxHXynRYFXsUmXl15l6wwdRGaJ9/YmTDtJ9WDeoe3xdL8KCsNv0pE4Hfey+p3OBUdaJOLjw3n1a72BXM02XDiTglI4qDRsovVEsRi65jdBbtJQn4WHRfI2AfQcB2o3QgWxVHRjXLDPVf/GfD9x7+fv8+f//aSkKcKZW5kc3RyZWFtCmVuZG9iago1NCAwIG9iago8PCAvTGVuZ3RoIDM3MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kkty7TAIROdaBRtwlfjotx6/SmVws/9pDth5A1e3BYKG1ughXa4Vchk0lkmEyj9tpkPCtvw03VMc9mnmU6K72PFK87EK7+bvSRgFvMPBvonENHEqZGkf40GdGYGZ0zS66Djia8upYoSNLrHlUmJrFN5Nhz4nqLlCtE/JQuvkqcuc4NgvHiLF5pHFjLqWVObOHjtr70GNSLKfmklokzTX8Wn/N/PSTIAOS7rk8pWEv7pNxt0uzSYzNWdW8JcqbGRlVuKo/jRVWDCDhfjcD6Iwu8NOKjusMGT3grsxWZSSwTZpgh/62PV53IKZ4oifF7OcscxwtpkZOPPn692+m+/0wvG4GJ6lBJ/Ot1456bFjS0UPoq0XUvlEMUN+OmyZMePt6ajRw11lSBZtOyOKExYTvznpZGRWnBp7CelKeV5MQsw6pt2zJjaoDKJlEb2Uq4P9iCKgBJH7yjk1Vz6KnOCQgpKd7+3BjOYGvn4BNmOOoAplbmRzdHJlYW0KZW5kb2JqCjU1IDAgb2JqCjw8IC9MZW5ndGggMzE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nFVSSW7EMAy75xX8QAFrs633TFHMIfP/ayk5KNCTCEshKSoxBgY08KVjw7fDXfAtV0Pd+FwmiS+JgZtwFQw8IB8wBa+nuQVmkyAVtvwANnceeB9Yavclof/gGZDdkLZGfb8DKsW4FomUniiYUHu8qBsBNaliBgkjh6ahGGjRF1Q57xyQTAhkFu+A1N6sryutAWkHQijcZPCOpsQCMqLkCWLC+ETb1ends5KgNl0Y27WB5XjsOqV1WSdrsruSfo7zwv2cgpaMdSU7LtLR339HuK83Y+MM1/kctKLjrTBtltPmpHenD2M2zrOZStdX51FIp3JWUb5sjY7Ee17mhCar020Um6j2CUQqsnFqsxWaHO3lOk7XpirjvNZGkMQn1i5t+pWxuDtr1P+xyZudATVJ1BvJU6M67+vnF4tve+gKZW5kc3RyZWFtCmVuZG9iago1NiAwIG9iago8PCAvTGVuZ3RoIDQ1MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtk0mOJDEIRfc+BRcIyQyezpOlUi+i77+tB85FBIjh8xkcEdLlsSmPdolYfCo/2ixcfAz534y/nS1vs9Oxmbhu8X3EgxTSPs2nSTjeHQUQfZb8tCC2LAa05zcqEg8VwvwW1biyR3mW+EmPX0TyfO30uKJp5fkEWa/E01dpfuC1+uW3ZnJbaBvWM+VAakk8cTXv2UuITedLBqVl35OPLo2apidzugspb8ukQU2spLon3cIsxcf17i67zBDYs7pJYnsJ/aX5yMlmTbRD+gSyHGeikcDglUrRVVStBjorBgagDDqGsSGCyEplRk8uyWunBuZTSzJGWBaaeugLftVrJgKRw6BL9oMDM9WVAb5NWSGmt8HUjtASIBOdqAF4srzA/q20Sj7le+CRtsdUVl2aXuFaVg4l+XI/CTJl5M4ynRXQBnVW/j5NRxUd6aBgxZEzepLjYugqMg7OUeMlTgE8GajM0Zgf3eRxePWVQGRpLYrJD61NEB1gsQ8lLk9SLb6nmYBlYQO5k7yek1Xn3eLbZobma1lg3IpsJw9X531dStx9HNbvc7E8fwZvOQbLJRqDDs115NHA/fscP+1f+/0DAgKvgQplbmRzdHJlYW0KZW5kb2JqCjUxIDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRmlyc3RDaGFyIDAKL0xhc3RDaGFyIDI1NSAvRm9udERlc2NyaXB0b3IgNTAgMCBSIC9TdWJ0eXBlIC9UeXBlMwovTmFtZSAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRm9udEJCb3ggWyAtOTcwIC0zMDUgMTQyOSAxMDIzIF0KL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0gL0NoYXJQcm9jcyA1MiAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZyAvRGlmZmVyZW5jZXMgWyAxMDIgL2YgMTEyIC9wIC9xIDEyMCAveCBdID4+Ci9XaWR0aHMgNDkgMCBSID4+CmVuZG9iago1MCAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9GVFhXWkErU1RJWEdlbmVyYWwtSXRhbGljIC9GbGFncyA5NgovRm9udEJCb3ggWyAtOTcwIC0zMDUgMTQyOSAxMDIzIF0gL0FzY2VudCAxMDU1IC9EZXNjZW50IC00NTUgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDQzOTA5IC9TdGVtViAwIC9NYXhXaWR0aCAxMTE3ID4+CmVuZG9iago0OSAwIG9iagpbIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwCjI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDMzMyA0MjAgNTAxIDUwMAo3NTUgNzc4IDIxNCAzMzMgMzMzIDUwMCA2NzUgMjUwIDMzMyAyNTAgMjc4IDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAKNTAwIDUwMCAzMzMgMzMzIDY3NSA2NzUgNjc1IDUwMCA5MjAgNjExIDYxMSA2NjcgNzIyIDYxMSA2MTEgNzIyIDcyMiAzMzMgNDQ0CjY2NyA1NTYgODMzIDY2NyA3MjIgNjExIDcyMiA2MTEgNTAwIDU1NiA3MjIgNjExIDgzMyA2MTEgNTU2IDU1NiAzODkgMjc4IDM4OQo0MjIgNTAwIDMzMyA1MDEgNTAwIDQ0NCA1MDAgNDQ0IDI3OCA1MDAgNTAwIDI3OCAyNzggNDQ0IDI3OCA3MjIgNTAwIDUwMCA1MDQKNTAwIDM4OSAzODkgMjc4IDUwMCA0NDQgNjY3IDQ0NCA0NDQgMzg5IDQwMCAyNzUgNDAwIDU0MSAyNTAgNTAwIDI1MCAzMzMgNDcyCjU1NiA4ODkgNTAwIDUwMCAzMzMgMTExNyA1MDAgMzMzIDk0NCAyNTAgNTU2IDI1MCAyNTAgMzMzIDMzMyA1NTYgNTU2IDUyMwo1MDAgODg5IDMzMyA5ODAgMzg5IDMzMyA2NjcgMjUwIDM4OSA1NTYgMjUwIDM4OSA1MDAgNTAwIDUwMCA1MDAgMjc1IDUwMCAzMzMKNzYwIDI3NiA1MDAgNjc1IDMzMyA3NjAgMzMzIDQwMCA2NzUgMzAwIDMwMCAzMzMgNTAwIDU1OSAyNTAgMzMzIDMwMCAzMTAgNTAwCjc1MCA3NTAgNzUwIDUwMCA2MTEgNjExIDYxMSA2MTEgNjExIDYxMSA4ODkgNjY3IDYxMSA2MTEgNjExIDYxMSAzMzMgMzMzIDMzMwozMzMgNzIyIDY2NyA3MjIgNzIyIDcyMiA3MjIgNzIyIDY3NSA3MjIgNzIyIDcyMiA3MjIgNzIyIDU1NiA2MTEgNTAwIDUwMSA1MDEKNTAxIDUwMSA1MDEgNTAxIDY2NyA0NDQgNDQ0IDQ0NCA0NDQgNDQ0IDI3OCAyNzggMjc4IDI3OCA1MDAgNTAwIDUwMCA1MDAgNTAwCjUwMCA1MDAgNjc1IDUwMCA1MDAgNTAwIDUwMCA1MDAgNDQ0IDUwMCA0NDQgXQplbmRvYmoKNTIgMCBvYmoKPDwgL2YgNTMgMCBSIC9wIDU0IDAgUiAvcSA1NSAwIFIgL3ggNTYgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNSAwIFIgL0YyIDUxIDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAvY2EgMSA+PgovQTIgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMC40IC9jYSAwLjQgPj4KL0EzIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4KL0E0IDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDAuOCAvY2EgMC44ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1TVElYR2VuZXJhbC1taW51cyAzMyAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjU3IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkKL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTIyMTgxMCswMScwMCcpID4+CmVuZG9iagp4cmVmCjAgNTgKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMjE2MTAgMDAwMDAgbiAKMDAwMDAyMTI5MCAwMDAwMCBuIAowMDAwMDIxMzMzIDAwMDAwIG4gCjAwMDAwMjE1MTggMDAwMDAgbiAKMDAwMDAyMTUzOSAwMDAwMCBuIAowMDAwMDIxNTYwIDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MyAwMDAwMCBuIAowMDAwMDA1NDE4IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwNTM5NyAwMDAwMCBuIAowMDAwMDE2NDEzIDAwMDAwIG4gCjAwMDAwMTYxOTcgMDAwMDAgbiAKMDAwMDAxNTY5NyAwMDAwMCBuIAowMDAwMDE3NDYwIDAwMDAwIG4gCjAwMDAwMDU0MzggMDAwMDAgbiAKMDAwMDAwNTkwNSAwMDAwMCBuIAowMDAwMDA2MTYwIDAwMDAwIG4gCjAwMDAwMDY1NDQgMDAwMDAgbiAKMDAwMDAwNzAxOSAwMDAwMCBuIAowMDAwMDA3MzYwIDAwMDAwIG4gCjAwMDAwMDc1OTcgMDAwMDAgbiAKMDAwMDAwNzk4NSAwMDAwMCBuIAowMDAwMDA4MzA1IDAwMDAwIG4gCjAwMDAwMDg2MTYgMDAwMDAgbiAKMDAwMDAwODk4MiAwMDAwMCBuIAowMDAwMDA5MTQ1IDAwMDAwIG4gCjAwMDAwMDk3NzkgMDAwMDAgbiAKMDAwMDAxMDEzNiAwMDAwMCBuIAowMDAwMDEwNDQ1IDAwMDAwIG4gCjAwMDAwMTA2NzIgMDAwMDAgbiAKMDAwMDAxMTE2NiAwMDAwMCBuIAowMDAwMDExMzM1IDAwMDAwIG4gCjAwMDAwMTE2OTMgMDAwMDAgbiAKMDAwMDAxMTk5MSAwMDAwMCBuIAowMDAwMDEyMjEwIDAwMDAwIG4gCjAwMDAwMTI2MjUgMDAwMDAgbiAKMDAwMDAxMjg0MSAwMDAwMCBuIAowMDAwMDEzMDUzIDAwMDAwIG4gCjAwMDAwMTMyNTEgMDAwMDAgbiAKMDAwMDAxMzU4OSAwMDAwMCBuIAowMDAwMDE0MDA5IDAwMDAwIG4gCjAwMDAwMTQzNTggMDAwMDAgbiAKMDAwMDAxNDQ0NyAwMDAwMCBuIAowMDAwMDE0NzEzIDAwMDAwIG4gCjAwMDAwMTUwMTUgMDAwMDAgbiAKMDAwMDAxNTM0NSAwMDAwMCBuIAowMDAwMDIwMTgzIDAwMDAwIG4gCjAwMDAwMTk5NjQgMDAwMDAgbiAKMDAwMDAxOTYxOCAwMDAwMCBuIAowMDAwMDIxMjI4IDAwMDAwIG4gCjAwMDAwMTc4MzcgMDAwMDAgbiAKMDAwMDAxODI2MCAwMDAwMCBuIAowMDAwMDE4NzAzIDAwMDAwIG4gCjAwMDAwMTkwOTQgMDAwMDAgbiAKMDAwMDAyMTY3MCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDU4IC9Sb290IDEgMCBSIC9JbmZvIDU3IDAgUiA+PgpzdGFydHhyZWYKMjE4MjcKJSVFT0YK", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T22:18:09.995047\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Set the random seed for repeatable results\n", "tf.random.set_seed(0)\n", "\n", "# How many samples to draw\n", "num_samples = 100\n", "\n", "# Draw samples and compute importance weights\n", "samples = q.sample([num_samples])\n", "w = tf.math.exp(p.log_prob(samples) - q.log_prob(samples))\n", "\n", "plt.figure(figsize=(8, 3))\n", "\n", "# Plot proposal\n", "plt.plot(x_plot, p_plot, color=\"tab:blue\")\n", "\n", "# Plot target\n", "plt.plot(x_plot, q_plot, color=\"tab:green\")\n", "\n", "# Plot raw samples\n", "plt.hist(\n", " samples,\n", " density=True,\n", " color=\"tab:green\",\n", " alpha=0.4,\n", " bins=tf.linspace(-4., 4., 40),\n", " zorder=1,\n", " label=\"Samples from $q$\"\n", ")\n", "\n", "# Plot samples weighted by their importance weights\n", "plt.hist(\n", " samples,\n", " weights=w,\n", " density=True,\n", " color=\"tab:blue\",\n", " alpha=0.4,\n", " bins=tf.linspace(-4., 4., 40),\n", " zorder=2,\n", " label=\"Weighted samples from $q$\"\n", ")\n", "\n", "plt.title(\"Target $p$, proposal $q$, function $f$\", fontsize=22)\n", "plt.xlabel(\"$x$\", fontsize=18)\n", "plt.ylabel(\"$p(x),~ q(x)$\", fontsize=18)\n", "\n", "plt.xticks(tf.linspace(-4., 4., 5), fontsize=18)\n", "plt.yticks(tf.linspace(0., 1.0, 3), fontsize=18)\n", "\n", "plt.xlim([-4., 4.])\n", "plt.ylim([0, 1.1])\n", "\n", "plt.legend(loc=\"upper left\", fontsize=16)\n", "\n", "twin_axis = plt.gca().twinx()\n", "\n", "# Plot function\n", "plt.plot(x_plot, f_plot, color=\"tab:red\", label=\"$f(x)$\")\n", "\n", "plt.xlabel(\"$x$\", fontsize=18)\n", "plt.ylabel(\"$f(x)$\", fontsize=18)\n", "\n", "plt.legend(loc=\"upper right\", fontsize=16)\n", "plt.yticks(tf.linspace(-60., 60., 3), fontsize=18)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "First off, this plot illustrates how importance sampling works.\n", "Although $q$ proposes many samples in the middle, these are down-weighted by the importance weights, since $p$ has small density there.\n", "Note that the samples in the middle are still there in the blue histogram, but they have just been down-weighted by the importance weights.\n", "Conversely, whenever $q$ proposes a sample at a point where $p$ has high density, the importance weight becomes much larger.\n", "In this way, importance sampling moulds the empirical distribution of samples from $q$ (in green) to resemble the target distribution $p$ (in blue).\n", "\n", "However, the samples which receive a high importance are relatively infrequent, which means that sometimes we may get sample with a large importance weight in one of the two Gaussian modes, but not in the other mode.\n", "Since this sample has a large importance weight, it greatly affects the overall estimate, introducing lots of random error.\n", "Looking at a histogram of the importance weights we see that most weights are very small, and it's only a few large weights which dominate the value of the integral (note the $x$-axis has a log-scale)." ] }, { "cell_type": "code", "execution_count": 51, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "center-output", "remove-input" ] }, "outputs": [ { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNTA2LjAxODc1IDI0Ni43NTEyNSBdIC9Db250ZW50cyA5IDAgUiAvQW5ub3RzIDEwIDAgUiA+PgplbmRvYmoKOSAwIG9iago8PCAvTGVuZ3RoIDEyIDAgUiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzNWsluHDcQvfdX8JgcRLG482jHtmDfFAtIgCAHYTKSZWixRnaE/H0ee2FV92hsA4mMNiDb/fqRrMelWFXq41fbv682219PXqpf3nfH/LR56Eh9xM+lMuojfh4VqRP8XHZGHRl10wUTtaGcQn28Fo/WR50C2QDUzJ4+dN1Fd/wCnTygx5MuWO37JviPRZe+ZJ0ZuBaApaxDhVqbhoy92qHXS9gIe3WGxRijIl2yOpkS4zROe/ZZJ/TQvYTQx+4ef/fiSIWgLRihZGuyclaTiZCwuelenkEUug8UKnL2V3f8hhRldXbR/UQ/q7OPXUEvwH8y/RMlXcjbbIOjpKL2iVtZHft2oWe+Puvu0bUhG9GDmf4LnTY7m+scJu1sDIMpCl0cvT97+/vJ9na7O78+urm6/fKgXt2p0+6062egI1e0dcm6SToDB7WTJZ0qxbhiUxVfjKOZemN8zOn/Uu+/V31tXvqBv0+9NUGn6LOhSb5ADuvPGLJysAF8/AH63bPpx0a3pS5308/IQf02eJ0qh4KL4Qfot8+l37nqPkrxTb9ADup32P+u55DJ/gfop+fS77HbHUakdvwFclh/sQPHxmzcf9Rf5tbvizeT+GZ0Io2+TWajGTlotA955Nhg6bmNJml0hCWwNGab+YoRgOmvMpgcsBdwTbI3EkAjRdxvxntTJpIAmBThoazncy0AJhUdYwi2Xa8CaCSsdXLRBz+RBNBIZLz22DMpt2tEIEzD/KTorGeaQAQN84sOXGQaI0yzGCJlYpESEbSsC45rKkxjhGnYEalgC4ibkBFBi5qKg+dnGiNMC6VeEYbnViJMS0aXlIuYEIEIGtoWA7/ENEaYlnF3OVxLqdEEwrRC2OohCaUCEbSAFSyBFTAgSAWuMEUSfTHSaNZgKq2JvP0lwjQLcwtmqo0pEaa5pHEqAk+HRJjmQ40NHbMYYFJ9hz2f2pxJRNAwgHOOdUqEaZF0yYn4OmVAkGAGzkb2zGJE0LDEPkMa0xhhWnXGWEAxtQJpNIfzaJw3uc2GRJhG1bUH3t0CECQ4WRwg3rUSYRpOI3xXZp0SETSE6eRjIaYxwjScRkQliR2RRAQNGyaFzGdAIkyrt4GPiV2MRJiG0xiimFsBCFK9WRzub2YxwjScRWdNIp41gTCtYATMkNApEEEL2ngLn840RgQNrs4Yx35IIo0Gt6QRzQv/LRGm4TQ6eAmeXIkwDacR28ryhEiEaTiO3hDxdScRpgVc7tj1vHMlImhJE0XhcSXCNBzIEAtu70YTiKDBEgs/HJnGiKBhoeH6PdsmEKbBv5aa+/L0CoSd6SLNgCeNfSZLBArHKWkIXOZhyts+4gi6FKwwwrJEY8RyM4SZOHlI13C1GKRsw5tPs8jmbva0O9Td5/HFXm/n/QsELIVKLuTC9OJ21vHmEG176IXqX3gdxufHQRACuBKNsbjH3be6uDpk9OXMtg+zp4NKFwYdv7HTIjwuY9W9AgYOn6xfPBFcx7GFj9qMNYRh6XmxzTDwH6qGo3+qs3dPDlmjodmYA/DVQQmO/Nujhq+NWuJi1B54Mo9MrY35jlFpJva+q10c1c4QHPoh68FRjvSVLl7ffLraXW3Or9XFbnv/ZXu7+WfWp5qvl8caeAVvDzetdlv1m7pVxy9cX0fSZGFwjrh/8ODj+Cd19XzCXVsXbFE7ro71taZDrdSi1VSBCs2xlHp1ZgQ509ZhgJCzRiDcZkL6ytepWpGuhYwa2vd+FZflJEwgk459retTtqdkSA1SjnDkTRsj02mUckdojeIWUmpCkzX5YFLbkQJBstYvk2g2QWsUt5Ay5FehljdtE8fI5NFEswlan7ilkpp6wWWjYcyTNoFkmmvtn9enak9ETRXxr6U4RZ3XEqGIu2audILWKG4hpaauDjmA8VOaei2RyW1Ivav1JEslNa31GlEmksOmjZFac5irHZEVKlvogLIMr1Ay5eYiBUJw+DOpA7BCXQsVtXSQdAw1DZl0CWTUtRS6Pl17MmrloWgDt56aaxQImYJQd651hNYnbqmkFjhMTdWTbcmHQHo3L9qs1e0vRdRaS/1lpEOa32QxMkWLotl6A8ilklqrcdoU6z1vR0ZGPyhardYzLnUgO6T+1wO2NJ8vkEnZntb1KVvqqKWuWmQ25JtvFAhMjHOtI7JCZQsdYz0t5pLbbhTIuEKi1XrXbKGj/v6wfimSslgzRnpXKNrMXaNV7wYz+u9J5t/HLIs/Ye/LlO798pOWm6c+aQFtr5C0/BJGcsaGh/oyvc3DdzDUT9+lLMtkbM6hFmMRRQYr6ilICL5VT1F3F+rq5tPd7vP57WarHrdXlx8+P8gqy2n3L8TyWn0KZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iagoxNzYzCmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9MZW5ndGggMjkwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWSu3HEMAxEc1aBEvAnUI88ngvk/lMveDo7kPCG+OwKVIoQkyhe0Zsylb5kDUo2/ayIyd/rVJwgjRhMKrSdtKicrOla7YTjbtoxbxGfw6aI6RyoTW2UkCwENeQ3wJy8KV3GR/ocD8AKhPyfkuItDumHxCgSD499JS9Uc2CA5whtcg0owWHuE6+llofupZgzpDJRSHmfHtTwdBtOkOF3jKqTEcwqdFvoh9oPOWYYduIWZJXjAt9ljUw0sshk4BkXjmlD9yG1WdFkH0K9bnwP+nX64dmEp0+hOjqjqZuOC90fZ6D7UPkD0FAu7IsUrR7PBnBjirUaJvAJ1zKrAahzD3jy1Hszrgj5UKbCnnEPIoG99/lHrr+/5V6v9f0Ly3dvTQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMTYyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVQS3JDMQjb+xQ6gvkY4/Mkk8nCvf+2gr5uLA0gIWxmmJDkY7IQoXjLKDrxM7p+C+QQY0InMqCBk7CN15B54OxTsa1BRKtOtpaUulge1GSI4xDUOZKJMGcN4Z0h3EqZHeOO/0APY/MO5ZJwgXq0WMug3VSVVgaV3esaGYAdsgp1m22K1LB4iy1Y1oDvOqm89e8feO7r+YU7vuPzC6LJNwcKZW5kc3RyZWFtCmVuZG9iagoxOSAwIG9iago8PCAvTGVuZ3RoIDQwMiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1Uk3S3TAI2+cUXKAz5tf4PK/zTRev999WwukiQbEDkoCIkCW+5deRCJOoJb/1IaySvwO85fuElhRg8U9PY/g8rkFgx0RTrFvKcWy9Cb6PoTwu9Eyaxpumugg6wQ0abZwFqheVlPJ9guU3cvDLEt1IX1tMlfkOTgfnQUw84KOGy+wo9b1olxgy/bRYpYSOulQJ66kQ4LR1JqJuX6SbNwrp+LZhDGbjBMyx1kQHw9XirSPUd4xJr7mBeU/azunYRDy3mnveHEMEl+vkGLJXi4LLVwgMMnweugHYe67rvP/XLZDoHHQz5MIxQZtsh9iexoWvEdsSyXbWjNrsjhxNeU8c44oRumCYYik9lo4Vzt8xWd9sthf5Gic9diaOWiJVbgAR1W4QQzv8BHWHGkEYRLu86/Z5/szY1DZWb4Za3D2DQwvuiUNxzFhs2fjhKZrPJmbeaNNivEHGzjb8YhR3JuhMogI7A4nYsuDyGbY7uBFJOQbHRRsGURip7RRSo9C7ZKiAJRitMPJfM/X//AOhr53MCmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0xlbmd0aCAyNjggL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNZFJbkQxCET3PgUXiGQGY/s8P2pl0bn/Ng9aWViUGIoqHBEyxZZ8XQk1iZzyrcPvEV0pv6OSGlvew4/LVXHSy8R9ips8w25KwLCLwrSZnqHrFlANnuwjy8mGya3eZiSYNgMAEjo8WTZRMj8cUyJc9K7WZWityMTJRm4ojosmotWMH+vpMuBnfSLanrbkURn2OPEE813ZIGUmqWDAl3akQr0zeGjDszRqu6bCQd4gZaOKsS2wa3ujpXsSJ1ZGUet0+OmIM6YLaeIIfvXbbFRgqP06OTE+KlrrL6S3Tpp8TX3KakxluxzOg8j8dGefFUEA1ymJddadShc71y7TalP+f/oZP+P1ByeeYYcKZW5kc3RyZWFtCmVuZG9iagoyMSAwIG9iago8PCAvTGVuZ3RoIDI0NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtkUtuBDEIRPc+BReIZD42+DwTRVnM3H+bhzsLCxooqoqOCJliS76OhIXEnvKtI2aJ7pDP6KKulPfw6ra4Rg+bXsxr6NqdqE6e5JZlVMPkWO/VyA6mSbWTKoHT2VQmfnpYbUp4iua8AsztRgDnydzrTngqGKXjhyoKY1KxTWR3NsnJm7zHQaWjYLoYTycqq7GKdl18gVPnBV1s0WE/WMM+fVvRARm1O2nrGxFAylvDKoHEMaJXwnO01/htbofvMxzmzt54AbqUCxALl0nn1D0Lx0eT4bDvbXjp2Hqeii5ngogfP7ej/IqkMp9j/hM2988fp3dXUwplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9MZW5ndGggMjM4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSXIEIQy78wo9wQs28J5JpXKY/P8a2Ux15dBIWGCpjbtDYMLFtyP5fekwxQzB71DxZu/LjkAlEdRVFTEdr6EaiAyoTcTeUN9IqlQykXFgejvbOo2v4XKZ0z7XggeRPagsMu46jZ+LVt2a8Z5XF2HexQTHSkmn8+wusalMbaRicSsqfdYOE2lSMVZzspJGv4BNb6Qi0Uz3ovPFyLpT7E6j8v9jumtCpT5MZX4Yryqty4sz9ApcAzHu+X8MU1E0Ppmqpz3A8nsEGyyszfPgfL0GeDYYuB4lV0P5ve4rdYZ6wod00p/x/Qf6/1xbCmVuZHN0cmVhbQplbmRvYmoKMjMgMCBvYmoKPDwgL0xlbmd0aCAyOTMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVJLbgUhDNtzCl+gEvkQwnleVXUxvf+2Nq/qhngIsRNn1pyYCMOHBTIa1Y1PG29o+Bk5HesUnhGnsXoj+jAGYq8bX8P4UugZlobku2e4JbITXhtZjaBSLufrCKdUIYqSk5lj5JrMJGuCyukFP+sdUwpCphtr2CIzuzK/bF1gQ1GO4iROvsVrb0fAWabJbPWNbNXnRSz52Ch2cdkhZaOozmzxGjb7I3lh35PFnIHAPMHZbU0saRnzQZFz4NI0BbbApL5ZkfL4FpFYapNCi0zZ73juKES+xEVzJ7HmSmWc3QQ3YxwxzdWQwmvQWIGiI+RXcBlZ8m5pH5u3tIbb447+EMWEcupGO5AR+/ofKG4iOUJt7o1+6Id4/f8Pz/geX78Qp2k6CmVuZHN0cmVhbQplbmRvYmoKMjQgMCBvYmoKPDwgL0xlbmd0aCA5MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNjDsOwDAIQ/ecgiMACaDep6o6NPdfy0dpuuAn21gQAYHYz7AOagontUBSg9m6YdKThK588KeV0A+4U3RZc6q+NlUa65vq9c69cGeS2HBX1uAKo3i9EGIioAplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggNTYxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVTS44lMQjb1ym4QKTwye88NWrNot/9t2M786TuB0USwNiM3q1bbGvhy2p1q9ntjz90c2/7PLmXvN8nz7Ecx6pPy/Rre9n70IsRuhdeuFXm8N8n/ZjPQAlE6uBWl32fqLiRRATRcHwjx/v4SXiFir6OznxO3HX4OInFO4kTx1fiv7vscR2krYGbx/Ao3G2yw6xuQyDWtDHVe+EP0JWvUGUj4tZ8XlvsX95GpHgSBE0HfTDngHsGMpw7wOAhHfSBK3IduFr0ZQtTvkOpYpWjuTu6kI3OodBDkkTmHYammtrHT8PFg+/AIAPjYA0MHwARxZR3XiTAl2UrrZK/g8+XXNQ+eEhGIjUqPE9OdosJ8uRTJ+hDs2eBs1kh/Nzkgcmz9zlkDmvCSdC5BxSAZiglF5PIWkUtHAkrcsoCKGopwpOBd5w65ompJnjIRaoG8kAdCT4L2BK6qMsWhRq889Xp7/P3oXr4+vPfG1TQoPbAAngM4tyEsqWyI40GB7oKyhyy6G2tGwG1DhayH00BvUVcfccRFtmrL3jZt1HviTrBXdlDlAKpcz+IC+9BHi1Oeshz7EaBEV+U1xUPMpFW4pjLvsheoCwIvIGlz/Xqvm3axyF5vNrZBhZyUKElBLTcupRHRhpgFS+KcTgxtFONm4k+mlROOVKEGEADETSbPR5JlVuozYCMkrmK7FJFWhIOGVLHzFX3Uv+N3f0ZLtE0Li6YdcbwI3An7AuX0H/+AQty3fQKZW5kc3RyZWFtCmVuZG9iagoyNiAwIG9iago8PCAvTGVuZ3RoIDI4NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtUrtxxTAM6zUFR+Cf0jzO5V7h7N8GpF5hA7IFEoQUzMR08Pguym30I6sp09/SCuA7KAFi7CSbTI00yEzIgp5ldsixdqOsAWHt72DGXWFYMRnUzkmaZ/BZ6pepwQILqTjZabUc1NpKUuiYCQe3Nv6AdYd3GFpKShsQVB1DUtUG5UTbVR376KV3nvN935kgPxTRgtSrb3dOhXGy33cakMDzDjlO2JS8KbELnp/VVhRDYK2OpgPwkkOk111qAJ9BENv0Ty+QjVowIApBQRA653HHzc3fca2y2eYbGxu5Ylqo3HkCRcSBA0INT0TOF3EcO4f1fucaNET+rGY30GYI1Ps0ApCIvje4UqvikN6rMoHei/Kuz/r9BzplbO0KZW5kc3RyZWFtCmVuZG9iagoyNyAwIG9iago8PCAvTGVuZ3RoIDIzNiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UDuSxSAM6zmFjoCNDfg82dnZInv/9kkkr0EKNvrE10aHTR6eA3MP/Fiz3TGH4/9htmArMXtyM5GVuJol2eZNONFgvg9yYvHcdD8b5e+T7UeDlpIU0ONqIjFB65kbRZhbMp2MM7M8uczrICfxJFWEs8FQ7xvFpMiJTc1vkav9NfXrqqSyt4DCd0vWZxiHDzD0kEF10KKkqtN66DI5NL4QyVCtsTr2xqAL2w2SwgqMCkw/cDWmEBn6pkk/S1cjiS5/RY55Eq1C5HyZvVOW5u+neASsGFCiVgUPuJ/wamb6wU/Fm2V/P7wcWJQKZW5kc3RyZWFtCmVuZG9iagoyOCAwIG9iago8PCAvTGVuZ3RoIDE1NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtT0kOAzEIu+cVfkKAQOA9U1U9TP9/rZlUirADZrHuxIQUg/pGpOElo+nEd6gQ7gZxoitkYxt0gUJbuEYZ1kIl4omlnUt4CBuapKMcMQVZCBHW2Rs6sZXglD5wjXX+ShkXSIMx3cSK05J1Z8E4LoqnnHup6MpuSbPcf+LUZGC1v4D1Zp2cr3zeTtqm9Obj9x6f8f4BMKQzKwplbmRzdHJlYW0KZW5kb2JqCjI5IDAgb2JqCjw8IC9MZW5ndGggNDIxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1TO3IFMQjr9xQcwebv8ySTSfFy/zYSzkuz0oIxCHBVy5Kd+FSFeC753A/pkp8nooCvwR0kHbJLcqmoSe4lpvLx5G7xLakK18BZNIPoScSRmSUiUiwdp1ysGRoHnnMkysRXS3gMwqP7WhYRXuS24/B4HkS7eBhuO+LWYl70gB0WSpKIcWVhHkiyJxIV4b839QTqByAVCHVq5z+OXtuj18wZZk49OG8B4TCHS+QAAmjGsVOMA7NCBNpDDdrnT5XWupZYo0pxoR0WobBao9EHqgrYVEfPLtwWrARp/fxhjwdzY8UksMxJRygKYT94ly7UvjljRVNGL8lE5RtGrTuRLcKt6Bhn2Gw+5cXht5kVyGG+hmDgfQuuW//Hw1i0gf3HosQeQPMvMQpH46l7NKA1kx574ah6Yxic246efcSJTIywplJkRDqyfrPD5YNgRbyFKDbBPd69Rt/03M02jn5uNG5bshBuj05eK87Q61rC7onAJt+YzBpWK0cAUZsesmJ5JMhXi9qkuOx7+jHPpXAlHkrNBGnGKxvh97m9nu/n6xekBKoQCmVuZHN0cmVhbQplbmRvYmoKMzAgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtOTcwIC00NDMgMjAwMCAxMDIzIF0gL0xlbmd0aCAzNwovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjMyVDAyMlDI5TIzATNywAwLMxADJIdggSUzuNIA/9IKDQplbmRzdHJlYW0KZW5kb2JqCjMxIDAgb2JqCjw8IC9MZW5ndGggMjg1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSuXXFMAzrNQVH4KljHuflpfjZvw1AOY0A8wRJl6qo2MSTuySnypcNUpXf4WsBP8AtViChLrYk7IinhINOeUYEUrdEqixrMC3awUKTiWT0oUzC53M1PsNLryUSaOKOyBP0wBpQYnsilyrREm2eQbYcZUkKkiaaQyLkhtKPDHdxDap1P1SPgiAcx/Y/9FDZUZjcQ1iIvdcUmLcKGuA9CRswYiGBZJqwGxazAr2pirlnSR3M4ZLVgCVYE+hJ7VXHuTNIdvsJh0OKbklKqegzMAKZtXpMbG6dZv4yV/SBVkcE40PnmxeZ1wJlmbO3mU5PWrxf2GTVReUFyXYvA6R2K8LJEucNDpGo6VcYdsX/ozd6f5TP+Bnff02QbOEKZW5kc3RyZWFtCmVuZG9iagozMiAwIG9iago8PCAvTGVuZ3RoIDIyNSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UTtuQCEM2zmFL1CJfIBwnldVXXr/tXZeO9kicWKHNScm/OLjIs9E7olPG6Iehp9mtg2ZFzYDOQ9W4RmxFywQZtJ6LsEzrELEwlW99ja32LXH1hW461nkHrAaq5vjbs2gOKNgtduQr2zkUntfYp/uyFl/muwh2SH8vmGU4BnfI4p8JtM0K+npPKgI8nJNVjZb8MOs3O+xGmmSJxKzQ1d2mIA+q7PSf/A2ZkT3Rj+hCpkVe2P3q7Rbie0mIhk/WYdvF+gQSsshR9XIAgPqxrqbLPITcuM/iEJ9/QL+YlIzCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL0xlbmd0aCAxNDYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY9RDsMwCEP/cwofAYfQhPN0mvbR3f93Jq0mRfiJCGPCDAaSqp4DxzzwYis0fBu5pNdWRkEMcOoluoPp8MDZuh0YE50uhy2524IYQ3Ob5N1NqimuW8/G+XRiIVaCXsr6UaqY3OtFq/b3tJ2xKEX+0L+lfG4BmTiVMeXj3pVUQWfdUKexwt03Xu3T3j8cmTFkCmVuZHN0cmVhbQplbmRvYmoKMzQgMCBvYmoKPDwgL0xlbmd0aCAzNDIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVJJbsQwDLv7FfpAAGvxovekKHpI/38tKU+AXkaMLVMkNaN36TLkMp0Sq0vMLl/adKTE2PJbyLfK06yrhLtYZLV596p386HnJFEDJIabzpsYIY4bUnvPqhaLN0C6Av0pquxPmUGyiYMhrilXiuExyt3Mz7epXFOsLyG/pkO9riXQX1KdpwCXQsTzgVChIL84UBfs6q6+TZgGOnblKm0AmwMJGcvTxj+gmQBOspwynLXLNHIansEWxrjUlDFPdWoFcl94TABby8QB50K+VDMVAWIZzBGpRRUaL8BIYDbr925ZgUHRqLSVu0DyOt6dvBt82s/ZIQjPNvf+5AKVCrloYoxRw2DTkLIzaYTvLDjO+nbbGC8OiYsiHBa0Uz/1aVVuBjdEE5JpUq2eeO29OJCUIWT+Rwy2z3/FEE6dKK2DORkQV0GVDi3lI7a8fm54+/4DKHSDbAplbmRzdHJlYW0KZW5kb2JqCjM1IDAgb2JqCjw8IC9MZW5ndGggMjg0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWRy3HEMAxD766CDewMvxJVjzOZHDb9XwNwk+xJz6QIEXCpioqnPNyWZLfkUvmwizi178vLX/h8oZ1D7E1cEmYElfAAtMp9RRRwo5Zslv0CbqAJXEmJU2KF1zF3xDp53JfFJuAVU1lHahRRmEUN0zhcC1XC3pIlgSFsFL2o4EcSK1j7+PHMOTHQNhSuktkS4HRKpeFObazF7zc9LPUfF6ahDWu4wjWt6SiTzUNpf4Uy8TGo+53k8/oa7+YbqZIiAtWhg11gN4Puek5sqzVkG9JG73So49AR6RLDjDMpnsmZIWUH/jdmsLjBP3MFjQqydLG1ZDfryIwh4T+WjUrN43sRAjvAdsD2eGXQ5/VjDa0/QzfMff4AU55thQplbmRzdHJlYW0KZW5kb2JqCjM2IDAgb2JqCjw8IC9MZW5ndGggMjY1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRO5YDMQjrfQqOYD4GfJ7s25cie/92JZIUYxgESIC7y5aDz/1I5JYfXQoTp+VvPM+U19IGvE1sp4SlmLlEbHksC2a72LnTwHqPBXJrPFdmHHGDjQTiDs/zTYpOY3cNgrzLiIq3oSbEK4hojRbfDQslN8eCp/0dKaptKMmppTZ2A0IuqDHw0gLRT2Qj0iXaMbUPzIz+0TKzQ8VYPYNsuZvLgJNEqF60IIxs2uAGmRISck8ZHVZ9XwRfy0GtAqFWkqBmYhnbVQp2j7c5dXFJgQI6SC+MWirU2Nxx4r+vHLS4wXIaLmscvTM+9F0m4xybxwxcCtBtHAGjn/M52Pfwr/Vcv/9G3mQ9CmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKPDwgL0xlbmd0aCAzNDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLZI7bgQxDEN7n0IXWMD6+HeeDYIUyf3bPHq2GWpkWyIp5T7Wbbi9jmVtq3Hsy5uPadnd/lrMbpHbfluctPBt6dP8cD2O+Q57tyw3n3ULeMWDPu7Jtu2WWTbI92mxSceijEVNtY2+Be/m6/47N1+0yAEzsr5smEOGUsf0fC1gHiubw15JRk8SjqAPBXN/Ajp5LzvcFakymlIUQpGQhjhPzhDAy4+CSABiY8D80u1Wy8QalYI9PyocM2SRY0HEuBWlhLrHac8nEILBsUuQqacKFpTSCnIb3+PSchLXxHUn4QgWitkzmyiowCsYkG5ygq7C9MCV4lR9hJyorjJHmcR+vdVJut/ot2mYiStEOPxEQddUr4V+CRjU6ZcFKivWXYPKBb8HYc4kbkbME/UYV/HMDw1dTuWtdnGej9oc63qYpRGwQhl3PCyLZqFl6lqJZx3f7ad9/wO4Q4AnCmVuZHN0cmVhbQplbmRvYmoKMzggMCBvYmoKPDwgL0xlbmd0aCAxNyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwzMjVQQMAUQ640AB2YA00KZW5kc3RyZWFtCmVuZG9iagozOSAwIG9iago8PCAvTGVuZ3RoIDE5MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kMsNwzAMQ++eggsUsGR97HlSFD20+19LOWkOCV/0iShpTnTIwGNBc8H5PKVpBDLxbRUjf5paQgLao0rFZ8nR1irNjjHqLZIMEkwmm/h1gcF0zzHVU5yFBYPhSQmQzIxhdxgHTHYvg2jCXapcJ2mwX6j0wyaPURljJOjLnCs4NfcqzLCzSNyodio3qgxNea/d9sCbTseVvUmG/qn3vX3WOQLndP7XOKIM6z4R5WiqG9TocuA66dHe7fUDAUlF8wplbmRzdHJlYW0KZW5kb2JqCjQwIDAgb2JqCjw8IC9MZW5ndGggMzIyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nCWSS24EIQxE95zCFxgJf4HzdBRlkdx/m2d6MVNu/ClXQc4pU0Lloy7hKrVKvnSUSuqUvxFJEPI7akpWyA6p6aK6pNzkGRpHinI9b7PFi8+wvW7kllS4eNLrm4xvk1ITP0tynRf9dIYo4PAq1tniMySmdiam+E4ynNYi5hc9LRQOzbu+LXtRe1pHWmTIKifdvy5NUcYcO8JyvShi2Ri1H74xpf3Q9IvPWPs1CFf6L/zSsrPQwSyC6rn4sKCYzYLGy3V2Y9NzrrRmdyuL4p+hzYBepN00tu3vg6R2Di/rii/po2QQXRchIpGtQKnEM9a2xCG7WpwTqyvGfV4xjVB3FH2nesglNbqvr+bt6+3mNgNPnEz0TLDfAYwdBXUTRITtI3luD/pqLl4A96w9dV5sxjdSdsnd8X1dz/gZ3/9H9XbhCmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0xlbmd0aCAyMjkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVG7ccUwDOs1BUbgX+I8L5dL4ezfBpQvhQ1YJCGAThEIrPmKHahd+NI1VH3jd4VN/Vm344LaoGyY6LB0WDTs8KQDzop74rPcyHbyayN0v1g6FbIUViyQIZwRZM2MsT8P1UJRYtA+Fz9Ly94Td2QT6SNp87NakZnYBOOjDDIDPok2HVIqvVG8ix5H0HwEnYJ5oLtvbPNzkSbaL/M5yWYIdtpUQupeHlTIlBdpnhWyYBw/jBPGVSQsr3EaodiznA5ecmpIiIPdoQUmHolGc0XB3WtyVQKNcfr/N571s77/AG3XURYKZW5kc3RyZWFtCmVuZG9iago0MiAwIG9iago8PCAvTGVuZ3RoIDI1NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNkUtuxDAMQ/c+BS8wgPWxLJ8nRdFFev9tKXWCdhM9R7FIKmtOTBy8DjwnfE18yCgkfTdY4B6uCVVYKjbMDa/ENcwSr8V2wY6G+AU9DjlQj5otx6pcQ9bqsy4oZBoI10ihOrZgn3qK1HCCxeGoAt6PgAt9JVwn+07LemAGN2cEp/Q1Tme4W+iPRPdDk75CkMS9sGuS5AFbOqNsqHa5hpYUzxklatTeFcHoLrkEpaizJGdKL4POJz+kI6HDrra7Y4ySvR7S+1aFMa7IO63NihNUqyTCvVS1tvFkctn/SOZ+U1qvJGoziSgztYzF10vqR/r2KtfzY+/xNT5/AHN0XpAKZW5kc3RyZWFtCmVuZG9iago0MyAwIG9iago8PCAvTGVuZ3RoIDMzOCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxVUjluBDEM6+cV+sACum2/Z4MgRfL/NqRmF0GKgTiyJVKUl7uouMnDQvqkZKl82PWGPzeKku+rN5FJr5b0RnRElefVvSQtpdskFTcqJLbhpKxkigvpkDw+TLnXxOeVO14ZRVTJteXRPADLAwTL5Eh2iB9kA5zebOgN3QNyc4Q408fBdnd2f2Ug0cT1SFDrSmETjANwtlSN9AQJvAAvh8JdSMIHZzg8utn4ATr9h2Y42/dtg1gWGxSms8xqj0c2NH5H9TmBM4eZBa8Q4R2FPWcyM0PfKOSMXAHfo5sISlgX6JY4CwW30RvnpnyJw8rR3v4WTzSak8r+0KjPtNkrd8r6NGqe5XAh/NM1XBOVLhLFPjNTNL2Dm8mT0g31VFpwN2LTapoP9c3XsPfEVCobxKeHeTlFa72465xRU3gqVFfrvYpBM8P7lX5fX9fnL5aYhV8KZW5kc3RyZWFtCmVuZG9iago0NCAwIG9iago8PCAvTGVuZ3RoIDMxOCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNkktuBCEMRPecwhcYyT9oOM9EURad+2/zDD1SFj0uqLIp29NVRcVSXm6X5NUlu8qXtQ/8PSi63C1HgPg6ny/JHESXd8uYklTJMEl1brvETBgn++VJtueS10pxMl6WKbYG4ApUdikqEo1C24qpHvBu13M1rCIPrNIqTFSBuSS0CvUT+yqmEPaCzgzzWW/6gOnFPNXMQnh/x3dbfpDpOhKjiyfLomRRoyoVJ+t2AOToz93cRae4AmYZceM9rLlRtA+6p5uomXm5R5om55c8bgFeMhzy2lpiVnUG26Ho3TqTduXAsF32zLfB3OtizMj/ob04W2QzCpt+si7UPrf1WiHMgKmWK+phCmEBRUyYmu+ofXid0N6tVhwrNkKF2ZjY034iu2IThegmLt9uIk+EKbSdfv5sd/tp339JN3oOCmVuZHN0cmVhbQplbmRvYmoKNDUgMCBvYmoKPDwgL0xlbmd0aCAyNzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVBLTgUxDNvPKXwBpObb9jwPITbcf4udB9KM4qZuYrvWwoInPiyQu9H8P+0RjFj4GeSneGnkkdQBX4HXk5UwnjIN5hvJMTd1sTZ6IU4i+IlWbIfW8PG5s81rTX09vt4d2yGCpSD7Zo7YOAdd2A6j2NdTBePYIRmnw800ZM9E7vDrKhGtNsE9CEOWHKJGezuK58MzJZsH2mdlXS67dOgThVO8KtXbHRQU0sn9lqizJodCWU98GY7/7F7P9xNkhBdzFEp2ow/ZrNmo7r9c2qn5GnfLwpmqYHqQ7RqGUfL7jdGQpphrqk0Nv7ohcm+IYfSrN1eWNSTkjD2frFi4g2kp5wlqVA1bMvVcsp0G/o3I1NcvehhongplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQ1pSQkxSK1NUSVhHZW5lcmFsLVJlZ3VsYXIgL0ZpcnN0Q2hhciAwCi9MYXN0Q2hhciAyNTUgL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMKL05hbWUgL0NaUkJMUitTVElYR2VuZXJhbC1SZWd1bGFyIC9Gb250QkJveCBbIC05NzAgLTQ0MyAyMDAwIDEwMjMgXQovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIDY5IC9FIDczIC9JIDk3IC9hIDk5IC9jCjEwMSAvZSAvZiAvZyAvaCAvaSAxMDggL2wgL20gL24gL28gL3AgL3EgL3IgL3MgL3QgL3UgMTE5IC93IDEyMSAveSBdCj4+Ci9XaWR0aHMgMTMgMCBSID4+CmVuZG9iagoxNCAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9DWlJCTFIrU1RJWEdlbmVyYWwtUmVndWxhciAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTk3MCAtNDQzIDIwMDAgMTAyMyBdIC9Bc2NlbnQgMTA1NSAvRGVzY2VudCAtNDU1IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMTA5ID4+CmVuZG9iagoxMyAwIG9iagpbIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwCjI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDMzMyA0MDggNTAwIDUwMAo3NDcgNzc4IDE4MCAzMzMgMzMzIDUwMCA2ODUgMjUwIDMzMyAyNTAgMjc4IDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAKNTAwIDUwMCAyNzggMjc4IDY4NSA2ODUgNjg1IDQ0NCA5MjEgNzIyIDY2NyA2NjcgNzIyIDYxMSA1NTYgNzIyIDcyMiAzMzMgMzczCjcyMiA2MTEgODg5IDcyMiA3MjIgNTU3IDcyMiA2NjcgNTU2IDYxMSA3MjIgNzIyIDk0NCA3MjIgNzIyIDYxMiAzMzMgMjc4IDMzMwo0NjkgNTAwIDMzMyA0NDQgNTAwIDQ0NCA1MDAgNDQ0IDMzMyA1MDAgNTAwIDI3OCAyNzggNTAwIDI3OCA3NzggNTAwIDUwMCA1MDAKNTAwIDMzMyAzODkgMjc4IDUwMCA1MDAgNzIyIDUwMCA1MDAgNDQ0IDQ4MCAyMDAgNDgwIDU0MSAyNTAgNTAwIDI1MCAzMzMgNDM0CjQ0NCAxMDAwIDUwMCA1MDAgMzMzIDExMDkgNTU2IDMzMyA4ODkgMjUwIDYxMiAyNTAgMjUwIDMzMyAzMzMgNDQ0IDQ0NCA1MjMKNTAwIDEwMDAgMzMzIDk4MCAzODkgMzMzIDcyMiAyNTAgNDQ0IDcyMiAyNTAgMzMwIDUwMCA1MDAgNTAwIDUwMCAyMDAgNTAwCjMzMyA3NjAgMjc2IDUwMCA2MDAgMzMzIDc2MCAzMzMgNDAwIDY4NSAzMDAgMzAwIDMzMyA1MDAgNTkyIDI1MCAzMzMgMzAwIDMxMAo1MDAgNzUwIDc1MCA3NTAgNDQ0IDcyMiA3MjIgNzIyIDcyMiA3MjIgNzIyIDg4OSA2NjcgNjExIDYxMSA2MTEgNjExIDMzMyAzMzMKMzMzIDMzMyA3MjIgNzIyIDcyMiA3MjIgNzIyIDcyMiA3MjIgNjQwIDcyMiA3MjIgNzIyIDcyMiA3MjIgNzIyIDU1NiA1MDAgNDQ0CjQ0NCA0NDQgNDQ0IDQ0NCA0NDQgNjY3IDQ0NCA0NDQgNDQ0IDQ0NCA0NDQgMjc4IDI3OCAyNzggMjc4IDUwMCA1MDAgNTAwIDUwMAo1MDAgNTAwIDUwMCA1NjQgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCBdCmVuZG9iagoxNiAwIG9iago8PCAvRSAxNyAwIFIgL0kgMTggMCBSIC9hIDE5IDAgUiAvYyAyMCAwIFIgL2UgMjEgMCBSIC9mIDIyIDAgUgovZml2ZSAyMyAwIFIgL2ZvdXIgMjQgMCBSIC9nIDI1IDAgUiAvaCAyNiAwIFIgL2kgMjcgMCBSIC9sIDI4IDAgUiAvbSAyOSAwIFIKL24gMzEgMCBSIC9vIDMyIDAgUiAvb25lIDMzIDAgUiAvcCAzNCAwIFIgL3EgMzUgMCBSIC9yIDM2IDAgUiAvcyAzNyAwIFIKL3NwYWNlIDM4IDAgUiAvdCAzOSAwIFIgL3RocmVlIDQwIDAgUiAvdHdvIDQxIDAgUiAvdSA0MiAwIFIgL3cgNDMgMCBSCi95IDQ0IDAgUiAvemVybyA0NSAwIFIgPj4KZW5kb2JqCjUwIDAgb2JqCjw8IC9MZW5ndGggMzg5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSR44bQQy89yvqAwKaqcN7ZCz2IP//6iqODAggxdCsMGttTFjhZRsrDzINf2zkDKQv/B0ZhZ34jJrGwYXKgluiuOp58R51Lvwk1pwI8ydGstNZBuo6ogq1D2L1zkrEcVRtxF0d06Y6nR2+PwmBO4uowtTZi9lGnWqcxT1F3uGmsuWETcAronfZ4fUk2uZGBopxng4zJ49NHlM3FrGxkc5CkbFEeAXHT8sjuIrvEQTSlc2BYqAyXU/S5PZneAVLesQJ2CiT24XZgt2DrTO2ig3+st+yyO/r5vepuK5wgmBepxs85/zv4FtucClpPTp5yowl1mwSBhW5FH85R8g3zgR/cU1uSnEq8EhSCgkJNWUavwYln05MbOi01Ly3jTCJ3faaRWeCINmFSQYJKnfjQFwyCCmcBsle8VRmnJNlxu5jopToiqzSTs02RYi4zResOEERrCH3HZGh4kaKTtONnKUbvTjakSxx1mPunX05bn2x5Hwq/bVH4f9X/x6/4+cfkgCTiQplbmRzdHJlYW0KZW5kb2JqCjQ4IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRmlyc3RDaGFyIDAKL0xhc3RDaGFyIDI1NSAvRm9udERlc2NyaXB0b3IgNDcgMCBSIC9TdWJ0eXBlIC9UeXBlMwovTmFtZSAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRm9udEJCb3ggWyAtOTcwIC0zMDUgMTQyOSAxMDIzIF0KL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0gL0NoYXJQcm9jcyA0OSAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZyAvRGlmZmVyZW5jZXMgWyAxMTkgL3cgXSA+PiAvV2lkdGhzIDQ2IDAgUiA+PgplbmRvYmoKNDcgMCBvYmoKPDwgL1R5cGUgL0ZvbnREZXNjcmlwdG9yIC9Gb250TmFtZSAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRmxhZ3MgOTYKL0ZvbnRCQm94IFsgLTk3MCAtMzA1IDE0MjkgMTAyMyBdIC9Bc2NlbnQgMTA1NSAvRGVzY2VudCAtNDU1IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSA0MzkwOSAvU3RlbVYgMCAvTWF4V2lkdGggMTExNyA+PgplbmRvYmoKNDYgMCBvYmoKWyAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MAoyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAzMzMgNDIwIDUwMSA1MDAKNzU1IDc3OCAyMTQgMzMzIDMzMyA1MDAgNjc1IDI1MCAzMzMgMjUwIDI3OCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwCjUwMCA1MDAgMzMzIDMzMyA2NzUgNjc1IDY3NSA1MDAgOTIwIDYxMSA2MTEgNjY3IDcyMiA2MTEgNjExIDcyMiA3MjIgMzMzIDQ0NAo2NjcgNTU2IDgzMyA2NjcgNzIyIDYxMSA3MjIgNjExIDUwMCA1NTYgNzIyIDYxMSA4MzMgNjExIDU1NiA1NTYgMzg5IDI3OCAzODkKNDIyIDUwMCAzMzMgNTAxIDUwMCA0NDQgNTAwIDQ0NCAyNzggNTAwIDUwMCAyNzggMjc4IDQ0NCAyNzggNzIyIDUwMCA1MDAgNTA0CjUwMCAzODkgMzg5IDI3OCA1MDAgNDQ0IDY2NyA0NDQgNDQ0IDM4OSA0MDAgMjc1IDQwMCA1NDEgMjUwIDUwMCAyNTAgMzMzIDQ3Mgo1NTYgODg5IDUwMCA1MDAgMzMzIDExMTcgNTAwIDMzMyA5NDQgMjUwIDU1NiAyNTAgMjUwIDMzMyAzMzMgNTU2IDU1NiA1MjMKNTAwIDg4OSAzMzMgOTgwIDM4OSAzMzMgNjY3IDI1MCAzODkgNTU2IDI1MCAzODkgNTAwIDUwMCA1MDAgNTAwIDI3NSA1MDAgMzMzCjc2MCAyNzYgNTAwIDY3NSAzMzMgNzYwIDMzMyA0MDAgNjc1IDMwMCAzMDAgMzMzIDUwMCA1NTkgMjUwIDMzMyAzMDAgMzEwIDUwMAo3NTAgNzUwIDc1MCA1MDAgNjExIDYxMSA2MTEgNjExIDYxMSA2MTEgODg5IDY2NyA2MTEgNjExIDYxMSA2MTEgMzMzIDMzMyAzMzMKMzMzIDcyMiA2NjcgNzIyIDcyMiA3MjIgNzIyIDcyMiA2NzUgNzIyIDcyMiA3MjIgNzIyIDcyMiA1NTYgNjExIDUwMCA1MDEgNTAxCjUwMSA1MDEgNTAxIDUwMSA2NjcgNDQ0IDQ0NCA0NDQgNDQ0IDQ0NCAyNzggMjc4IDI3OCAyNzggNTAwIDUwMCA1MDAgNTAwIDUwMAo1MDAgNTAwIDY3NSA1MDAgNTAwIDUwMCA1MDAgNTAwIDQ0NCA1MDAgNDQ0IF0KZW5kb2JqCjQ5IDAgb2JqCjw8IC93IDUwIDAgUiA+PgplbmRvYmoKMyAwIG9iago8PCAvRjEgMTUgMCBSIC9GMiA0OCAwIFIgPj4KZW5kb2JqCjQgMCBvYmoKPDwgL0ExIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDAgL2NhIDEgPj4KL0EyIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4KL0EzIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDAuNCAvY2EgMC40ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1TVElYR2VuZXJhbC1taW51cyAzMCAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjUxIDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkKL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTIyMTgxMSswMScwMCcpID4+CmVuZG9iagp4cmVmCjAgNTIKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMTYzMzkgMDAwMDAgbiAKMDAwMDAxNjA2MiAwMDAwMCBuIAowMDAwMDE2MTA1IDAwMDAwIG4gCjAwMDAwMTYyNDcgMDAwMDAgbiAKMDAwMDAxNjI2OCAwMDAwMCBuIAowMDAwMDE2Mjg5IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MiAwMDAwMCBuIAowMDAwMDAyMjAxIDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwMjE4MCAwMDAwMCBuIAowMDAwMDEyNjA1IDAwMDAwIG4gCjAwMDAwMTIzODkgMDAwMDAgbiAKMDAwMDAxMTkyNiAwMDAwMCBuIAowMDAwMDEzNjUyIDAwMDAwIG4gCjAwMDAwMDIyMjEgMDAwMDAgbiAKMDAwMDAwMjU4NCAwMDAwMCBuIAowMDAwMDAyODE5IDAwMDAwIG4gCjAwMDAwMDMyOTQgMDAwMDAgbiAKMDAwMDAwMzYzNSAwMDAwMCBuIAowMDAwMDAzOTU1IDAwMDAwIG4gCjAwMDAwMDQyNjYgMDAwMDAgbiAKMDAwMDAwNDYzMiAwMDAwMCBuIAowMDAwMDA0Nzk1IDAwMDAwIG4gCjAwMDAwMDU0MjkgMDAwMDAgbiAKMDAwMDAwNTc4NiAwMDAwMCBuIAowMDAwMDA2MDk1IDAwMDAwIG4gCjAwMDAwMDYzMjIgMDAwMDAgbiAKMDAwMDAwNjgxNiAwMDAwMCBuIAowMDAwMDA2OTg1IDAwMDAwIG4gCjAwMDAwMDczNDMgMDAwMDAgbiAKMDAwMDAwNzY0MSAwMDAwMCBuIAowMDAwMDA3ODYwIDAwMDAwIG4gCjAwMDAwMDgyNzUgMDAwMDAgbiAKMDAwMDAwODYzMiAwMDAwMCBuIAowMDAwMDA4OTcwIDAwMDAwIG4gCjAwMDAwMDkzOTAgMDAwMDAgbiAKMDAwMDAwOTQ3OSAwMDAwMCBuIAowMDAwMDA5NzQ1IDAwMDAwIG4gCjAwMDAwMTAxNDAgMDAwMDAgbiAKMDAwMDAxMDQ0MiAwMDAwMCBuIAowMDAwMDEwNzcyIDAwMDAwIG4gCjAwMDAwMTExODMgMDAwMDAgbiAKMDAwMDAxMTU3NCAwMDAwMCBuIAowMDAwMDE0OTg1IDAwMDAwIG4gCjAwMDAwMTQ3NjYgMDAwMDAgbiAKMDAwMDAxNDQzNyAwMDAwMCBuIAowMDAwMDE2MDMwIDAwMDAwIG4gCjAwMDAwMTM5NzUgMDAwMDAgbiAKMDAwMDAxNjM5OSAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDUyIC9Sb290IDEgMCBSIC9JbmZvIDUxIDAgUiA+PgpzdGFydHhyZWYKMTY1NTYKJSVFT0YK", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T22:18:11.121636\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot distributions and samples\n", "plt.figure(figsize=(8, 3))\n", "\n", "# Plot importance weights\n", "plt.hist(\n", " w,\n", " density=False,\n", " color=\"tab:blue\",\n", " alpha=0.4,\n", " bins=10**tf.linspace(-5., 1.1, 20),\n", " zorder=2,\n", " label=\"Importance weights\",\n", ")\n", "\n", "plt.title(\"Empirical frequency of importance weights\", fontsize=22)\n", "\n", "plt.xlabel(\"Importance weight $w$\", fontsize=18)\n", "plt.ylabel(\"Empirical frequency\", fontsize=18)\n", "\n", "plt.xticks(fontsize=18)\n", "plt.yticks([0, 5, 10], fontsize=18)\n", "\n", "plt.xscale(\"log\")\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Going a little further, note that the expectation of the value of an importance weight is always equal to one since\n", "\n", "$$\\begin{equation}\n", "\\int q(x) \\frac{p(x)}{q(x)} dx = 1 \\implies \\mathbb{E}_{x \\sim q(x)}\\left[\\frac{p(x)}{q(x)} \\right] = 1,\n", "\\end{equation}$$\n", "\n", "This means that if $q$ proposes samples with small importance weights most of the time, it must also propose some samples with very large importance weights to make up for the small importance weights: the integrand cannot always be smaller than $1$ or always larger than $1$, otherwise it would not integrate to $1$.\n", "The quantity which affects the amount of random error in an importance sampling estimator is the variability of the importance weights around this mean, that is the variance of the importance weights.\n", "It is reasonable to expect that the more dissimilar $q$ and $p$ are, the larger the variance will be.\n", "In partricular, we can show that the variance of the importance weights can be lower bounded by a quantity that scales exponentially with the KL divergence.\n", "\n", ":::{prf:lemma} Lower bound to importance weight variance\n", "\n", "Given distributions $p$ and $q$, it holds that\n", " \n", "$$\\begin{equation}\n", "\\mathbb{V}_{x \\sim q}\\left[\\frac{p(x)}{q(x)}\\right] \\geq e^K - 1, \\text{ where } K = \\max\\left(D_{KL}(p || q),~ 2 D_{KL}(q || p)\\right) \n", "\\end{equation}$$\n", " \n", "where $D_{KL}$ denotes the Kullback-Leibler diverence, measured in nats.\n", " \n", ":::\n", "\n", "\n", ":::{dropdown} Derivation (Lower bound to importance weight variance)\n", " \n", "The variance in the importance weights can be written as\n", "\n", "$$\\begin{align}\n", "\\mathbb{V}_{x \\sim q}\\left[\\frac{p(x)}{q(x)}\\right] &= \\mathbb{E}_{x \\sim q}\\left[\\left(\\frac{p(x)}{q(x)}\\right)^2\\right] - \\mathbb{E}_{x \\sim q}\\left[\\frac{p(x)}{q(x)}\\right]^2, \\\\\n", "&= \\mathbb{E}_{x \\sim q}\\left[\\left(\\frac{p(x)}{q(x)}\\right)^2\\right] - 1, \\\\\n", "&= \\mathbb{E}_{x \\sim p}\\left[\\frac{p(x)}{q(x)}\\right] - 1, \\\\\n", "\\end{align}$$\n", "\n", "By applying Jensen's inequality once, we can get a lower bound to the expectation above, to obtain\n", "\n", "$$\\begin{align}\n", "\\mathbb{V}_{x \\sim q}\\left[\\frac{p(x)}{q(x)}\\right] &= \\exp\\left(\\log\\left(\\mathbb{E}_{x \\sim q}\\left[\\frac{p(x)^2}{q(x)^2}\\right] \\right)\\right) - 1 \\\\\n", "&\\geq \\exp\\left(\\mathbb{E}_{x \\sim p}\\left[2 \\log \\frac{p(x)}{q(x)} \\right]\\right) - 1 \\\\\n", "&= e^{2 D_{KL}(q || p)} - 1,\n", "\\end{align}$$\n", " \n", "and similarly\n", "\n", "$$\\begin{align}\n", "\\mathbb{V}_{x \\sim q}\\left[\\frac{p(x)}{q(x)}\\right] &= \\exp\\left(\\log\\left( \\mathbb{E}_{x \\sim p}\\left[\\frac{p(x)}{q(x)}\\right] \\right)\\right) - 1 \\\\\n", "&\\geq \\exp\\left(\\mathbb{E}_{x \\sim p}\\left[\\log \\frac{p(x)}{q(x)} \\right]\\right) - 1 \\\\\n", "&= e^{D_{KL}(p || q)} - 1,\n", "\\end{align}$$\n", "\n", "which is the lower bound in the lemma.\n", " \n", ":::\n", "\n", "Note that when $q = p$, this lower bound becomes $0$.\n", "This is in agreement with the fact that when $q = p$, all importance weights are equal to $1$ and their variance is $0$.\n", "As $q$ becomes more and more dissimilar to $p$, $D_{KL}(q || p)$ increases and so does the lower bound.\n", "Therefore the variance also increases and in fact it increases at least exponentially with $D_{KL}(q || p)$.\n", "Of course, this is only a lower bound, and the actual variance can be even larger than this.\n", "On the flipside, this lower bound tells us that if we want to reduce the variance in the importance weights we must draw our samples from a proposal distribution which is as similar as possible with the target distribution, in the sense of having a small KL divergence.\n", "\n", "This is where Annealed Importance Sampling (AIS) becomes useful.\n", "AIS is an importance sampling method which uses an annealing procedure based on Markov Chain Monte Carlo (MCMC), to produce samples whose distribution is closer to $p$.\n", "To achieve this, instead of using samples from $q$ directly in the importance sampling estimator, AIS gradually transforms the samples, such that the distribution of the transformed samples is closer to $p$.\n", "This reduces the variability in the importance samples and thus the random error in the importance sampling estimator.\n", "\n", "## Importance-weighted MCMC\n", "\n", "Motivated by the above intuition, given some initial samples from $q$, we want to transform them in a way such that the distribution of the transformed samples is as close as possible to $p$.\n", "This would reduce the variance of the importance weights and thus the error in our estimator.\n", "Markov Chain Monte Carlo (MCMC) is a standard class of methods geared towards solving this type of problem: an MCMC algorithm begins from an arbitrarily initialised distribution, and proceeds to transform this distribution acording to a randomised rule, such that the resulting distribution is closer to a target distribution.\n", "So we could, in principle, use MCMC within an importance-weighted estimator to reduce its variance.\n", "The following algorithm is based on this intuition.\n", "\n", "\n", ":::{prf:definition} Importance weighted MCMC algorithm\n", "\n", "Given a proposal density $q$, a target density $p$ and a sequence of transition kernels $T_1(x, x'), \\dots, T_K(x, x')$ be a sequence of transition kernels such that $T_k$ leaves $p$ invariant.\n", "Sampling $x_0 \\sim q(x)$ followed by\n", " \n", "$$\\begin{equation}\n", "x_k \\sim T_k(x_{k-1}, x_k) ~\\text{ for }~ k = 1, \\dots, K,\n", "\\end{equation}$$\n", " \n", "and return the sample $x_K$ with an appropriately chosen importance weight $w$ such that the resulting estimator is unbiased.\n", " \n", ":::\n", "\n", "Note that the only requirement on the transition kernels $T_k(x, x')$ is that they leave $p$ invariant, and do not need to result in an ergodic Markov Chain, which is a usual requirement in standard MCMC.\n", "The distribution which results after drawing an intial sample drawn from the proposal $q$ and applying the transition kernels $T_1(x, x'), \\dots, T_K(x, x')$ can be written as\n", "\n", "$$\\begin{align}\n", "q_K(x_K) = \\int q(x_0) T_1(x_0, x_1) \\dots T_K(x_{K-1}, x_K) dx_0 \\dots dx_{K-1},\n", "\\end{align}$$\n", "\n", "and gets closer to $p$ as we increase $K$.\n", "Note that this algorithm does not specify how to select $w$.\n", "However, while we could in principle draw $x_K \\sim q_K$ and return this sample together with the importance weight\n", "\n", "$$\\begin{align}\n", "w = \\frac{p(x_K)}{q_K(x_K)},\n", "\\end{align}$$\n", "\n", "each of the nested integrals above is intractable, which means we cannot compute $q_K$ in closed form, and by extension we cannot compute $w$ in closed form either.\n", "One way to get around this issue, is to consider the joint distribution of $(x_0, x_1, \\dots, x_K)$ under the Markov Chain, which has density\n", "\n", "$$\\begin{align}\n", "h(x_0, x_1, \\dots, x_K) = q(x_0) T_1(x_0, x_1) \\dots T_K(x_{K-1}, x_K).\n", "\\end{align}$$\n", "\n", "We can then define the reverse transition kernels $\\tilde{T}_k$ as\n", "\n", "$$\\begin{align}\n", "\\tilde{T}_k(x, x') = T_k(x', x)\\frac{p(x')}{p(x)}.\n", "\\end{align}$$\n", "\n", "Because $T_k$ leaves $p$ invariant, it holds that\n", "\n", "$$\\begin{align}\n", "\\int \\tilde{T}_k(x, x') dx' = \\int T_k(x', x)\\frac{p(x')}{p(x)} dx' = \\frac{1}{p(x)}\\int T_k(x', x)p(x') dx' = 1,\n", "\\end{align}$$\n", "\n", "so the reverse kernels integrate to $1$ and are also valid transition kernels.\n", "We can now consider a reversed Markov chain which starts with the target distribution $p$ as its intial distribution and applies the reverse kernels $\\tilde{T}_k$.\n", "This Markov chain has corresponding joint distribution\n", "\n", "$$\\begin{align}\n", "\\tilde{h}(x_0, x_1, \\dots, x_K) = p(x_K) \\tilde{T}_K(x_K, x_{K-1}) \\dots \\tilde{T}_1(x_1, x_0).\n", "\\end{align}$$\n", "\n", "Now, consider performing importance sampling in this augmented space, with $h$ as the proposal and $\\tilde{h}$ as the target distributions.\n", "Specifically, we draw draw $(x_{n, 0}, \\dots, x_{n, K}) \\sim h$ for $n = 1, \\dots, N$ and compute the importance weight\n", "\n", "$$\\begin{align}\n", "w_n = \\frac{\\tilde{h}(x_{n, 0}, \\dots, x_{n, K})}{h(x_{n, 0}, \\dots, x_{n, K})}.\n", "\\end{align}$$\n", "\n", "The importance weights $w_n$ ensure that for any function $g$ of the augmented sample, the importance weighted estimator\n", "\n", "$$\\begin{align}\n", "\\int g(x_0, \\dots, x_K) \\tilde{h}(x_0, \\dots, x_K) dx_0 \\dots dx_K \\approx \\frac{1}{N}\\sum_{n=1}^N w_n g(x_{n, 0}, \\dots, x_{n, K}),\n", "\\end{align}$$\n", "\n", "is unbiased.\n", "Therefore, if we set $g(x_{n, 0}, \\dots, x_{n, K}) = f(x_{n, K})$, we obtain the estimator\n", "\n", "$$\\begin{align}\n", "\\int f(x_K) p(x_K) dx_K \\approx \\frac{1}{N}\\sum_{n=1}^N w_n g(x_{n, 0}, \\dots, x_{n, K}),\n", "\\end{align}$$\n", "\n", "which is also unbiased.\n", "Crucially, the importance weights $w_n$ can now be computed in closed form.\n", "Using the definition of the reverse transition kernels we have\n", "\n", "$$\\begin{align}\n", "p(x_K) \\tilde{T}_K(x_K, x_{K-1}) \\dots \\tilde{T}_1(x_1, x_0) = p(x_0) T_1(x_0, x_1) \\dots T_K(x_{K-1}, x_K).\n", "\\end{align}$$\n", "\n", "and all terms coming from the transition kernels cancel in the importance weight ratio, yielding\n", "\n", "$$\\begin{align}\n", "w_n = \\frac{\\tilde{h}(x_{n, 0}, \\dots, x_{n, K})}{h(x_{n, 0}, \\dots, x_{n, K})} = \\frac{p(x_0)}{q(x_0)}.\n", "\\end{align}$$\n", "\n", "By performing importance sampling in this augmented space, we have got around the issue of intractable importance weights, by cancelling out a load of terms.\n", "However, unfortunately these importance weight are exactly the same as the importance weights of the standard importance sampling estimator, so this algorithm does not improve on the variance of the standard estimator at all! However, it is possible to modify this algorithm to obtain better importance weights, while still taking advantage of the cancellation of the transition kernels." ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Annealed Importance Sampling\n", "\n", "Given a sequnece $0 = \\beta_0 \\leq \\dots \\leq \\beta_K = 1$, AIS introduces a sequence of distributions with densities\n", " \n", "$$\\begin{equation}\n", "\\pi_k(x) = p(x)^{\\beta_k} q(x)^{(1 - \\beta_k)}.\n", "\\end{equation}$$\n", "\n", "These distributions interpolate between $q$ and $p$ as we vary $\\beta$.\n", "AIS then proceeds in a similar way to the importance weighted MCMC algorithm we highlighted above, except that it requires that each $T_k$ leaves $\\pi_k$, instead of $p$, invariant.\n", "\n", ":::{prf:definition} Annealed Importance Sampling\n", "\n", "Given a target density $p$, a proposal density $q$ and a sequence $0 = \\beta_0 \\leq \\dots \\leq \\beta_K = 1$, define\n", " \n", "$$\\begin{equation}\n", "\\pi_k(x) = p(x)^{\\beta_n} q(x)^{(1 - \\beta_k)},\n", "\\end{equation}$$\n", " \n", "and let $T_1(x, x'), \\dots, T_K(x, x')$ be a sequence of transition kernels such that $T_k$ leaves $\\pi_k$ invariant.\n", "Annealed Impoprtance Sampling amounts to drawing $x_0 \\sim \\pi_0(x)$ followed by\n", " \n", "$$\\begin{equation}\n", "x_k \\sim T_k(x_{k-1}, x_k) ~\\text{ for }~ k = 1, \\dots, K,\n", "\\end{equation}$$\n", " \n", "and return the sample $x_K$ together with the importance weight\n", " \n", "$$\\begin{equation}\n", "w = \\frac{\\pi_1(x_1)}{\\pi_0(x_1)} \\dots \\frac{\\pi_K(x_K)}{\\pi_{K-1}(x_K)}.\n", "\\end{equation}$$\n", " \n", ":::\n", "\n", "Note that drawing samples according to this algorithm and setting $g(x_{n, 0}, \\dots, x_{n, K}) = f(x_K)$ still results in an unbiased estimator of $I$.\n", "This is because the augmented sample $(x_{n, 0}, \\dots, x_{n, K})$ is distributed acccording to $h$ whose marginal, after integrating out all but the last variable, is $q$.\n", "Let's implement this and see how it performs.\n", "\n", "### Implementation\n", "\n", "To implement this procedure, we need to specify the transition kernels, and the sequence of annealing parameters $\\beta_k$.\n", "For the transition kernel, we will use a standard [Metropolis-Hastings](https://en.wikipedia.org/wiki/Metropolis-Hastings_algorithm) transition kernel, which itself uses a Gaussian distribution as its proposal distribution (not to be confused with the initial proposal distribution $p$).\n", "We'll leave the $\\beta_k$ to be specified by the user." ] }, { "cell_type": "code", "execution_count": 52, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "class TransitionKernel:\n", " \n", " def __init__(self):\n", " pass\n", " \n", " def __call__(self, x: tf.Tensor):\n", " pass\n", "\n", " \n", "class GaussianTransitionKernel(TransitionKernel):\n", " \n", " def __init__(self, scale: tf.Tensor):\n", " \n", " self.scale = scale\n", " \n", " def __call__(self, x: tf.Tensor, distribution: tf.Tensor):\n", " \n", " # Create forward proposal distribution and propose next point\n", " forward = tfd.Normal(loc=x, scale=self.scale)\n", " next_x = forward.sample()\n", " \n", " # Create reverse proposal distribution\n", " reverse = tfd.Normal(loc=next_x, scale=self.scale)\n", " \n", " # Compute acceptance probability\n", " log_prob_1 = forward.log_prob(next_x) + distribution(next_x)\n", " log_prob_2 = reverse.log_prob(x) + distribution(x)\n", " \n", " log_prob_ratio = log_prob_1 - log_prob_2\n", " p = tf.math.exp(tf.reduce_min([0., log_prob_ratio]))\n", " \n", " # Accept reject step\n", " accept = tf.random.categorical(\n", " [[tf.math.log(1. - p), tf.math.log(p)]],\n", " num_samples=1,\n", " dtype=tf.int32\n", " )[0, 0]\n", " \n", " x_accept = tf.convert_to_tensor([x, next_x])[accept]\n", " \n", " return x_accept, accept" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "We can then put together an `AnnealedImportanceSampler`, which accepts an initial and a target distribution, a transition kernel and a list containing a schedule for $\\beta_k$, to perform AIS." ] }, { "cell_type": "code", "execution_count": 53, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "class AnnealedImportanceSampler:\n", " \n", " def __init__(\n", " self,\n", " initial_distribution: tfd.Distribution,\n", " target_distribution: tfd.Distribution,\n", " transition_kernel: TransitionKernel,\n", " betas: List[float],\n", " ):\n", " \n", " self.initial_distribution = initial_distribution\n", " self.target_distribution = target_distribution\n", " self.transition_kernel = transition_kernel\n", " \n", " self.betas = betas\n", " self.num_steps = betas.shape[0]\n", " \n", " def __call__(self, num_samples: int) -> List[tf.Tensor]:\n", " \n", " # Draw samples from intial distribution\n", " x0 = self.initial_distribution.sample([num_samples])\n", " \n", " # Run AIS chain on the initial samples\n", " samples_and_log_weights = tf.map_fn(self.run_chain, x0)\n", " \n", " return x0, samples_and_log_weights\n", "\n", " @tf.function(jit_compile=True)\n", " def run_chain(self, x: tf.Tensor, *args) -> tf.Tensor:\n", " \n", " # Initialise chain history and current distribution\n", " chain_history = [x]\n", " annealed_log_prob = self.initial_distribution.log_prob\n", " \n", " # Initialise log importance weight\n", " log_w = - annealed_log_prob(x)\n", " \n", " for i in tf.range(self.num_steps):\n", " \n", " # Create next annealed distribution\n", " next_annealed_log_prob = self.log_geometric_mixture(\n", " beta=betas[i],\n", " )\n", " log_w = log_w + next_annealed_log_prob(x)\n", " \n", " # Propose next point\n", " x, accept = self.transition_kernel(\n", " x=x,\n", " distribution=next_annealed_log_prob,\n", " )\n", " \n", " log_w = log_w - next_annealed_log_prob(x)\n", " \n", " annealed_log_prob = next_annealed_log_prob\n", " chain_history.append(x)\n", " \n", " log_w = log_w + self.target_distribution.log_prob(x)\n", " \n", " return tf.convert_to_tensor([x, log_w], dtype=tf.float64)\n", " \n", " \n", " def log_geometric_mixture(self, beta: tf.Tensor) -> Callable:\n", " \n", " def _log_geometric_mixture(x: tf.Tensor) -> tf.Tensor:\n", " \n", " log_prob_1 = self.initial_distribution.log_prob(x)\n", " log_prob_2 = self.target_distribution.log_prob(x)\n", " \n", " return ((1. - beta) * log_prob_1 + beta * log_prob_2)\n", " \n", " return _log_geometric_mixture" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "### Toy experiment\n", "\n", "Now we have an AIS sampler which can be used with arbitrary annealing parameters $\\beta_k$.\n", "For this example, let's set\n", "\n", "$$\\begin{equation}\n", "\\beta_k = \\frac{1}{1 + e^{\\gamma_k}},\n", "\\end{equation}$$\n", "\n", "where we start from large and negative values to large and positive values.\n", "This gives $\\beta_k$ which gradually interpolate from $0$ to $1$." ] }, { "cell_type": "code", "execution_count": 59, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "hide-input" ] }, "outputs": [], "source": [ "dtype = tf.float64\n", "\n", "transition_scale = tf.convert_to_tensor(0.3, dtype=dtype)\n", "\n", "# Intialise transition kernel\n", "transition_kernel = GaussianTransitionKernel(\n", " scale=transition_scale\n", ")\n", "\n", "# Initialise betas\n", "betas = tf.nn.sigmoid(10. * (tf.cast(tf.linspace(1e-3, 1., 1000), dtype=dtype) - 0.5))\n", "\n", "# Initialise AIS sampler\n", "sampler = AnnealedImportanceSampler(\n", " initial_distribution=q,\n", " target_distribution=p,\n", " transition_kernel=transition_kernel,\n", " betas=betas,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "Let's draw some samples on the same problem we considered earlier, and visualise them as before." ] }, { "cell_type": "code", "execution_count": 60, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "remove-cell" ] }, "outputs": [], "source": [ "# Set the random seed for repeatable results\n", "tf.random.set_seed(0)\n", "\n", "# How many samples to draw\n", "num_samples = 100\n", "\n", "# Draw samples and compute importance weights\n", "x0, samples_and_log_weights = sampler(num_samples=num_samples)\n", "\n", "samples = samples_and_log_weights[:, 0]\n", "log_w = samples_and_log_weights[:, 1]\n", "\n", "w = tf.math.exp(log_w)" ] }, { "cell_type": "code", "execution_count": 61, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "hide-input", "center-output" ] }, "outputs": [ { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNTY5LjYyODEyNSAyNDUuMTU3NSBdIC9Db250ZW50cyA5IDAgUiAvQW5ub3RzIDEwIDAgUiA+PgplbmRvYmoKOSAwIG9iago8PCAvTGVuZ3RoIDEyIDAgUiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzVXVuP3saRfeev4KMMxK2+Xx7lzUZw3hwJSIBkHwxlrNiYcSwrWefn76lqfmQ1pzmjUCbSK0COWB/ZrNNdXReyDvPyt3f/+/27uz+8/mr+rzfTy+3o3cfJzD/g7/tZzz/g7y+zmV/j7/tJ4+hhCrGoaLOxAYf38tD6oExIAVItD/42Td9NL19hiI+45PUUkkpz0CrqiKswovbKbIL7RWBNVMUuErpECnjMD3M7lPdR+dnEqJydf76b/zj/OL98Zfm+yiQbfNGmRBxEm3zQJWs/Nb/8TKhfH589786eHoHBdBgfTXASUFfYXrlA+mYeAVRH34cpaxWKdTpKpbvCLtyB4HWUfpiKVdplH4pUuivsYh4IXkfph8lop3zGqjSL0pd2UQ8EsKc1EBqvitMpNutyIO3iHgliR21AtNAhxWKalTmQdoGPBLGjNiC6pDI2V7INmL60C3wkiB21AdEXZWOxOJBg+tIu8JEgdtQGxKhVQgBIuQEjpEmVlB4B36QjQeyAAcRklUGQ8+169aXd6RgJYkdtQMwO/8YGa1dxk8agnM82mBa5FI8EsgMHIEtQOrgcmhRUSBuQfewjgezAeZisjipo/PESpJSK/ShGGHI/9sAAooGylHU2xiqksM9kSl0wMYQUjwSyAwcgbVEOuadpjFVIG5B97COB7MABSK9VRv4ZW2PdpMYZFTOG2oFv5CPB7AACzGCVpRy0qQKF1OiiQvIYagdfykeC2QEEmNGphASmRbkKjfXKmbiAFOClfCSQj+EAYwo4DzlMU04IqXEFPjUvFruBl+KRQHbwAGWOuMrpJtvehIYez/hggKvFLuUjgXwMBxhLVhpq+3YlN+mCZrHXPfjxzLWD52Fy5EGQj5amNhTSFuUB+oFg9gABptWqUEraWKyQFg+fisLDteileCSQHTgA6azycCa6sVghzVrpouFaW+xSPBLIDhyA9E5lyklbg92kDcg+9pFAduAAZEBJQUlpa66bVJYcYohBK5EeHICEDgkRPjYPNIS0WcltiFFXsgMHIFNWhnLS5inAgbQ7ISNB7KgNiLmoSPloU24dSLvAR4LYUfth8tooDaFtVqYv7QMfCGJPbUA0VgVKRZuVOZB2gY8EsaM2IFqnCmWizcocSLvAR4LYURsQXVD4RefmqcCBtAt8JIgdtQERmmQqJ5pnAgfSLvCRIHbUBsSQUfEipDeV8oG0C3wkiB21ATEWlSj9bE2yL+0CHwliR21AzEZZekLcmmRf2gU+EsSO2oBYrIpYGN+aZF/aBT4SxI7ahy0pz8I9Ayxk7Yo1IRYc+OXfOQJY8sak6JMWwA7Onndn/6cbUy4CNUpjykXwRmlMuQjeOI0pFwEcqTHlKojnGlPkCygxxOe+l7oK5LnWlAZkH/tIIM81p8iHo2KIz31mehXIc+0pOSnswfq0SQwhxSOBPNegsrxHNHkHvpGPBPNck0rzVljC/9y3xVfBPNem0rwVlvA/923xVTAHalS5COLJRpUD4EOmAyfbVJ4FPhLEc00qwiTlCGMa6rkGlQPgY67imeaUZ2GPBPBUY8pzsEcCeKYp5VnUIwE81ZDyHOyRAJ5rRnkW90AQTzaiPAt8JIjn2lCeBT4SxHNNKCK4yxGGDPknW1AOgI+5iucaUJoum37zzUggzzWgNFWhGGPUavFkE4ohVW2xsNEWvpSPBPNcIwrChcpaLzAFfCkfCObJZhTZAyaG+NzWsKtAnmtHMRqFhnb1SaoYo5GPBPNcS0qzltsQo67lyaaULfTLEYZMCEZqSrkK4kBNKVdBHKgp5SqIAzWlXAXxmqaUl69c/QbL+9nQt11Unn+pqs3vH3+VhY59UmYZZfpqMvMv0wf8V89faoyAH7ULxSE4OYM0mf/17mH66u308ndmNnl++92EAOYogzaZHn/q+e1fpz/PL/wX8//Mb38//ffb6QP0wK8zfV1m+9e7hxljfPnm7dd/en33493P395/+fD9j//8OP/27/M30zcT6z4Z2Gr7qp0FT2ptsIXjSbXtr6O2RZRrH/Kw4Em1bSIzPFZYr0rqTcnbDWFpqk2cWfDkDV3On3Q7+/h2O7u8We7TxlSQtX7SDb284W6GiIi13KQOwItDh7j25e/sMtiLf30xv/2hUXlv/V41yXZnQTLO8F75kPjrR08siOosCd+QmgSsvOMmOLylsRjvE+4ZDu5pkThnec9NcHzPEuBXnrunaXF+mOj6L2kkqGznkhFX2pXJ9cNR7dr8xGtTlE6mFF1izPUO2y1f/OUFnxNUKa5keNVkbufslhipVjElF+NCejzMFwfDvPgN/wBb8tj+AB5MeXSDD1XPi5XjqfzUqMTu3c6/Z/dusMPh3pWpUcamADPwcfmTKOrokLN1wZb5D6/n9iNgckcEZ6jNjrI7k3XN83P2hapKWEZ2PiZ2wwVhzHDPhNZQjA8CYq23gMliD1NLiFAkhjUg6LG4KIvJopdHlmpXo2ufECBg9uAqnbIhaMOvto1TKWtPT9a80tYHm1gcceCiiSQuydlYu6xgvzAhD0UipoK+H8bdOdggUJLESQULxVgRaxW0s1R+F3pZEBLf0mJEaIfcPRqaz1gKi5Oy0BppRHTKJZpjFhflM+5Ej4xwlY48BlxbshETErPKOVnLQzhUPrFgAqlLyOAOrvYUwQ1gZ4Qyp6CSC/VDasZluJuQqNWIvroWsuNBvMXMW0z3nLMywVQ1MA2IdTaTTwRayne4ZwfzgOI5aBijgfvCLEfGTnHR+2Tp/AQVY12EqEldjM+eJ7uk6zCAbGhzwApRvpFdeFaGnpMl72KlcENo6rLFpLDcAQWuBaRiva/iAqvSMAqWl2ypNWPp08AiY/FJ7jEjmc9PXjko6SwPj6K41PYVOP9QtKe+DiyiziUEtriE2XbQhtBiXVA/16YepICYkYyLyXKsxkluaZxwVusCfaj21jFVy8hBhegc9VlgEWzSdf0KnLEvWB2Er0Bn89SXqBKmDFOMy4p3FSpO0RYmEGbcHDvGm/qKHxsk0kLNKdEKwDZY7HB3IDJzgo3gwirF3AVus0rYzAmQI4vJlQSfZ8yyQ8pS+MU6vK7LtOrRw1oKTIDFFuZp6MJoYWUs8rDNgAmasdowCJgBixFQI3Y3Ugcqe7zhe2EaYWo0vbCkhG2rmSRs4SyDT3B2AVMFr1JhwDZiSQWLB3v0yWPOWUzzgp1EWztCmcR6INiY7D29oYNJ26LrGE7TjEcXSewSggLjc9iVxmt6voL5RGXNK4UdCfvW8BYkhp1rV19QY/s7vfT+wt6yrV8cCDAiR/kbebVgs2f9sBTBBVjb4gILK+gou8CEWM5u4D4NG6ujFgYMXp1gziXxqjgYoisFmU2jCdwLzgnFtno7yn1g2NTNsqCsYpog2vTNnDgHvxsouqwzWF87ZNoTujpHnm7mNnu4YzhK2KxcHABBTQLfWJalxK5hMfXyYSM36+4o3U8+kxMURuI8Mm7a/YbNCUuT+WxAC5HmpDE+xw8kPR0slppYP9xew3bczazhcVgM5415IDe47QGH2wfaPOW2YdjxONw+YYjcbi9sfWwvhBQzY69SjDG8kvAXiBaoxJr9TI+IPayVnWbd/pYVRIKeqIFj5y5cxprEyI+UF/dS1xj7HnNBRfTNHbEPcNRBiIBFz2al++JntsAZ0+ruEutTjMIOM9jDjXt0BRfDgm3rTQEPWDDvZXW+dcqQe0UK2bF11tgnMFyr2Wuyc0coZjlOgqOE0ctY4OF9MJSnrjIZOrAHsQ5w5ekWapJJy9NLgxSPHl4ix6Q4zKdbytERfeEeKZYj8tyegcL0LDlCEd2AB0aasES3WFh4bATf7JCb+CZyevgKTUbgbnG2auiwpUzSuYnJcMwwUk0N8zKAI3jBSF3QYQn3iU3G08c6KNOxTXLgPfx3hvfNTSrhPfw39nR1k2vigQAFK6XH802aAiNGDYkx1qTG8dlwJ5a+rtpkQDAgskxd/eGaLnncp8ASom2SKw8MPmX8aVIxzxOBzM6viRs/cMZkIqEptb07WZ/ZgeB/YBC0BuQOOYy3jx8oRexJpjefmbyubB0kr0c8nk9IXmUeuHhu3+ZfTfpKH6/NwOtpzhyMi2McUitsF8qk2vR1y5xkfBKZ0G31bJvxNPnrltg0+euWvyxmVNOaLU1ZjG75PFmiUg5Jc5u/otrHlQVL2eSvW5qymL/zbUKybJYQ29Rj2Vlul2XIfShSCpnBiuRh2eOhTRIWh8BRQcT9xXUvrWYISggVMNGb72ZvQx1aCKtwhqvvrqpghrDAlnI36buRiVHLOuaoTQ0tApamEOTajNeSvRoOy2uKyVkLpiMsX4GRzhV6YTEwpkhVWezgRjLp1iTI1J5k4emjaTNekYqsvr62MxWsXqZH0U1osAhJGkts3F6OYp9qGLeNw9OWqcWUnkzu7otaIqGWiHs9sRZIavwiX0MYCjgYAjEedtOACtHjoJg2M7clkX8x/K0nOc1IkwuFb9cui9P0rIVusAvBsIFChUJoM3aHOtFjzewuZafmE9RAWIZbVErs+URGJZJ2kTk1WcWWIsmkXSRDMmHZkh6ZtIvsRmTtIo1pEqctjbll7QjWWCDUvKlNyNbMRiZvW/4ik3aRpjR54ZaNyKR9Szpk0i5yiFs08m1OQF7TwxOxGH4BijvPObFHqKnxeYv9gQozXX2OCNAyaZcBcEnaOW7D0lD7mttb5UQZxq8epm501PWpyiyfqkzNU5WNjto/e96d/Z9mbV4EahTW5kXwRmFtXgRvHNbmRQBHYm1eBfEkazPS65jqiyVrU4hHAnmStUlh0lnrd9iFeCSQJ1mbVBuWyKWzZG0K8UggT7I26aWDXyo7wdoU4pFAnmVtIplFGlbrQMnaFPKRYJ5kbVLNQeVK2cGX8pFgnmVtoiCHaZpHrE0hHwnmSdampWLA13dvYgghHgjkSLzNqyAOxNu8CuJJ3mZAmR9cfQ0khhDikUAOxNy8CuIwzM2rAA7D3LwK4DDMzasADsPcvArgQMzNiyCOxNy8CuJAzM2rIJ5kbloiZwSbd8iFeCSQA3E3r4J4jruZjAooDd3u4+FSPBLIs9xNZN5aU9G/424K+UgwT3I3qTs5hvAIvpSPBPMsdxNmisBY9ug38UAgzzI3DRypttq12KV4JJBnmZuOer4Tv3VsmJtCPhLMk8xN7opPec/cFOKRQJ5jbkZP3VGp9lttQ0jxSCAH4m5eBXEg7uZVEAfibl4FcSDu5lUQL+VuUn/kxttsux67DE5u4rd1sOnNETVvf9Z+pL2yt3O26w5G0qTvwjc1rPX7hl1GzJpcyZjW2uXKjWKGHKGhmL36+s388duHn+7vPkqq2ctXnibHzLRaldj6mo6mKWbiBMKBh+TM8uAwKaahCSk9QsPOof5bJ+XcwlfPjUjMuLdQyLJK9bx30yrlNkl7e261ChO1JlJ7r7iRpc41PnXVaRO9q6pvgvuJMv1qdJs0Bqb0rcMvxzcdMAoJhKr3N8kKaBtWIN/P2zuyxK+m577G3/7Cm+eXR1/kf6KTFxuIXvXNphhs2WpXhu1qk9wvEiSOKiwSvkgKeN/s+M5GsJ0lIdgwb3QdXxBH00ZPxHmReXlfL+w+Z1PJKSa9MABf/Mg/LGTFF9/zkVfep0I44+20fxz9cHjFtwtZEHlkyTl6ckH8w/3RFXP9YTn6yEdRWetKKZpi/DMDP1T6o3088k8NyEMF7o5GPtSlVfm7o0n++eiHvzeKHQNo7sOMzCjYnIs3eZrdPz9m9/8gDH27anqS5V8NHRvNtIa+Sm6GblJZTb9eJAT/vqGv4wsmb/xEN/t06NwHw93EfBIRdZ0YH4haLeblJlinJUQiJW+zsh3/+5NyG/x4Tv549/37v/3j7q/zweRI6nu0KhezREW49k3Qo1gHYpVxX663XYp1/2MEUf86XyOoKhMTotKzbzpvkmOliQIbzbNk9KN5svhv9HKiNskTNy1ZIVt65qbxiIweAnWD89WwSGKFNE4/lZjTnpN+80ca27mkrG2Hrz0Y7TvfHI+j/4scbC12PrRhl13ogzlkzsBFLLaAitdb1LyWat8M/FT6JqIzFlsMPVmFyHMDWSKmWC7eUYNgzrgdPbzKWFBD2cmcYcbBZm5+L0yrIfdfv6lPjyzpDVFKCBcz8VWsjua22RfOeIncUs7kkI0cTlzz5JLXtUtx5fnQhQE5Ab+yFxRuapTS+uZhV1o2DWioFY6H2ZjW1PdI3wDwO2IQOY5EneyVArQyg4i0gIle+gkFqdoS5AV+w5J2ymZMjLs1Om20Z22IHTq3NGYbGTgT9QQtmfhwyWfHaAugO+JpzcRGdznGyo3dSMXGwaaMz5W8u3GCaUKIflnbH1ZOryECnct6odiuxByS52IX/q43QOvo8Qg5D4esjmdNkGxJHn3hj5hJJgz7GlsZrxrLjIUoVYzAwNYleRsOS5IS1mdueLMkj177+hpnY25QHosp8fVjhxtHg+TFsxE1dAxKjgu9cGZ1NuYFWQUsIFc+ysYgdYUCT4yVO7ExQj0xZQJ9SKVleBLtFnVSZtqJoGx6r2CtpdIKBQWTqMuZvzNA8qLo2wtkgUSEohlhqgTMEesZyALhLlJkan7DrKCvAdL24Q5BQZSkwOlcCExPE+RHbHHoWXJkGubGZyTqvzZ2+XZbuFEUySuhstFVzY2MaKgKKjpXbiW1KtCnCqB+wh7Afq3kT2LGBEPdxYa+AZeY9QLTA3LML3lQ6gEolZsEM8cCYaNglAI7DsZV3iodwBEQRYvmFfVNYY5KSbARTd07VCQkZ5kVcwtCMIgoq3IpWZgh/0/L76Uw1hwb3tccaCuOPZxl4vm+DUIPIYqQVFKNdvRHiD0mlyKWoTXGTlkF/pZaTlUiq+GbZK1VxdBbQVs1kHVw1VIWwmTrYV8JewNX2Ay/CLZauEo2Le9XyQZkG1rA3c3TWgw/E3jnTwy8E6yHSNGFrXZdC5caCW0qt5M0T1i6+a2nXdmtbsmZhl0V9N1BQSfq3zXP2ZdgYph/HZR/chiZ50z/B/iHlQMKZW5kc3RyZWFtCmVuZG9iagoxMiAwIG9iago1Mjg4CmVuZG9iagoxMCAwIG9iagpbIF0KZW5kb2JqCjE3IDAgb2JqCjw8IC9MZW5ndGggMjE0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1QS47FIAzbcwofAYeQwHk6Gs2ic//tc6j6JIRNyMdxmqGDU1f2RKTjh61ox3/zWb/3Qe4iuYr4TqhwKs0WrjYZGHob4f2AChQWiY1Jx65sNbGKexCmkrtx75fNASpDzBMrQHekHYiuoiI69I0hUOyMZrKkcO0SZholuFqRftq+8OgfKE0Dg8jQHkpdS2uDlA2seVSzUHz448fdRrwsKJVkMcvqELXvQGhfFV4twmEbsawGlY9HzmPo3f6aq6vNlLlDGuaojcuBit3f38r8/QCrYk0iCmVuZHN0cmVhbQplbmRvYmoKMTggMCBvYmoKPDwgL0xlbmd0aCAxNjIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVBLckMxCNv7FDqC+Rjj8ySTycK9/7aCvm4sDSAhbGaYkORjshCheMsoOvEzun4L5BBjQicyoIGTsI3XkHng7FOxrUFEq062lpS6WB7UZIjjENQ5kokwZw3hnSHcSpkd447/QA9j8w7lknCBerRYy6DdVJVWBpXd6xoZgB2yCnWbbYrUsHiLLVjWgO86qbz17x947uv5hTu+4/MLosk3BwplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9MZW5ndGggMzk0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2SS24EIQxE930KX2Ak/AXOM1GURef+2zzTo0ittsG4qCqTWTIkXF7qElulZsmXXlFbolx+r4g42X2Fm+RgpVvSU2K4ZE55X14mNba4hhQVy3Xi+zKbz87ghJXo4sRYVLRKcg9RELLWE2N0pbM+W0MSWlqsh3ZlQnJxdsF5TuKC20HbJZGbe+AdS8yIDwNWYSpWzZyIwEAFFbp9T3H4+EpxL3F0owde7pM+NI1GDvoaLSbY7QhWmY4TmxWVzpSeqTKNyx3Qg8Vp8chjsa15YvN6TDe4vBKRW/YRDwza4WKYY9aBbZzt9UA0H25qG4LqFC542fmrscflnfS8RPcmqfwkijurCbdBKNIOUEQiieMXI/a2vxn63DKh3kPC/hM9PvYooM4YFNlebcZpwTCLCXRLRcbGBTu64OtQ4852lfCMtBPcxZZEjKKx7Aw0eTDV+HHepGFgxx7bOJkrOwzfk3UeXqwqqCx6AyXdG62wB99ZILMQFzyvBy1g29n9/+Tv6+f6/gPeNJNhCmVuZHN0cmVhbQplbmRvYmoKMjAgMCBvYmoKPDwgL0xlbmd0aCAzMTEgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTZJNbsUwCIT3PgUXqGT+8XlSVV2k9992IO+p3YRRDMMH9jGjTU4fvOmoUITQJ6+3/HmUKd3rbKfQpKpEZKpEFKVrVUCxU3mRn0Nlhzz7JPHVOJSo5TQ4Pa2uFbpH3cvrEG8h1yAxIctNxoIc5Xjl6OaXkkSPTjRiZ2K0Exhfi91ITxCLwaItATF4B97J1LhoEwrWwm9AhqJmgwoZM+W1fOZGnz7+U7MBEZtc2b0BtDiF6N282mwTF+pFnjgzjNp9Am5oBqJH16j13wNfs0buXoaJ3bIVavxs0go4JWm2c3ZdT4B7UDA1jbK92EcNs+f5r4beUR3G5LOE/cTdm2jV9+aYywsnDO1zS9GzBKqz+cVbdabjHeBOvWRiLxbvoJUUZfGQpb63Omp43i/rXt/r6xehCHh+CmVuZHN0cmVhbQplbmRvYmoKMjEgMCBvYmoKPDwgL0xlbmd0aCA0MDIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVJN0t0wCNvnFFygM+bX+Dyv800Xr/ffVsLpIkGxA5KAiJAlvuXXkQiTqCW/9SGskr8DvOX7hJYUYPFPT2P4PK5BYMdEU6xbynFsvQm+j6E8LvRMmsabproIOsENGm2cBaoXlZTyfYLlN3LwyxLdSF9bTJX5Dk4H50FMPOCjhsvsKPW9aJcYMv20WKWEjrpUCeupEOC0dSaibl+kmzcK6fi2YQxm4wTMsdZEB8PV4q0j1HeMSa+5gXlP2s7p2EQ8t5p73hxDBJfr5BiyV4uCy1cIDDJ8HroB2Huu67z/1y2Q6Bx0M+TCMUGbbIfYnsaFrxHbEsl21oza7I4cTXlPHOOKEbpgmGIpPZaOFc7fMVnfbLYX+RonPXYmjloiVW4AEdVuEEM7/AR1hxpBGES7vOv2ef7M2NQ2Vm+GWtw9g0ML7olDccxYbNn44SmazyZm3mjTYrxBxs42/GIUdyboTKICOwOJ2LLg8hm2O7gRSTkGx0UbBlEYqe0UUqPQu2SogCUYrTDyXzP1//wDoa+dzAplbmRzdHJlYW0KZW5kb2JqCjIyIDAgb2JqCjw8IC9MZW5ndGggMTY0IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDWQQQ5DIQhE955iLmDiIIKe5zc/XbT33xa0f+PgIyODMhoaxkClNnANsAleLLMf9C2elRCfQpmo0eAwVJ1HqbjKrsI7GABUSQnc17533ZL2nliiryB9m9hTruKaOpLkqclCTWANbnBiWrApWIblOyp5IuewP4ksK0K6wD35dBj3bs+K+XIWNc0zxmqksVDf8TTjNp6AFMXzGVd5l/sHi4s4FQplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMzE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROXIDMQzr9xX8gGfEU9R7nPGkcP7fBuDaFSGKBwDmWrLEtjyOxFGpdvnRizBM/i6P4NcbYMmjPiA3wWpRF6tkhakxPC/NIlC8dcluyYUsxvThHg1lsJXIEuyU2OKe0i3eTKu1BBK6t0SBHjAjG3SQr5KIFJIJ3fi5afmX4HGM3FKLoyFLgzWWUoZucCsLsRMTMbdvZNhYBlHbJnIjkI8DDpTY2ZhXsEvP2AVx6Cxk34MUu250QM5KqJC9zVJvkJLIMTGQS7IPbOF7nwlwP6f6PsP7+h1Vuu6TQIHbR6mj0O2AlcEVsEIN9QCZzXG427wn4gcHJ9JRAI9pPrjT9YCfuBCO6ksnWsw9gBRKFf6pQyHYH9JWSON9uBdLcthyuxE4zE8EdO2xnze+b3ZCvoKeEPf6Bx4/eEwKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtkUtuBDEIRPc+BReIZD42+DwTRVnM3H+bhzsLCxooqoqOCJliS76OhIXEnvKtI2aJ7pDP6KKulPfw6ra4Rg+bXsxr6NqdqE6e5JZlVMPkWO/VyA6mSbWTKoHT2VQmfnpYbUp4iua8AsztRgDnydzrTngqGKXjhyoKY1KxTWR3NsnJm7zHQaWjYLoYTycqq7GKdl18gVPnBV1s0WE/WMM+fVvRARm1O2nrGxFAylvDKoHEMaJXwnO01/htbofvMxzmzt54AbqUCxALl0nn1D0Lx0eT4bDvbXjp2Hqeii5ngogfP7ej/IqkMp9j/hM2988fp3dXUwplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggMjM4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSXIEIQy78wo9wQs28J5JpXKY/P8a2Ux15dBIWGCpjbtDYMLFtyP5fekwxQzB71DxZu/LjkAlEdRVFTEdr6EaiAyoTcTeUN9IqlQykXFgejvbOo2v4XKZ0z7XggeRPagsMu46jZ+LVt2a8Z5XF2HexQTHSkmn8+wusalMbaRicSsqfdYOE2lSMVZzspJGv4BNb6Qi0Uz3ovPFyLpT7E6j8v9jumtCpT5MZX4Yryqty4sz9ApcAzHu+X8MU1E0Ppmqpz3A8nsEGyyszfPgfL0GeDYYuB4lV0P5ve4rdYZ6wod00p/x/Qf6/1xbCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0xlbmd0aCAyOTMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVJLbgUhDNtzCl+gEvkQwnleVXUxvf+2Nq/qhngIsRNn1pyYCMOHBTIa1Y1PG29o+Bk5HesUnhGnsXoj+jAGYq8bX8P4UugZlobku2e4JbITXhtZjaBSLufrCKdUIYqSk5lj5JrMJGuCyukFP+sdUwpCphtr2CIzuzK/bF1gQ1GO4iROvsVrb0fAWabJbPWNbNXnRSz52Ch2cdkhZaOozmzxGjb7I3lh35PFnIHAPMHZbU0saRnzQZFz4NI0BbbApL5ZkfL4FpFYapNCi0zZ73juKES+xEVzJ7HmSmWc3QQ3YxwxzdWQwmvQWIGiI+RXcBlZ8m5pH5u3tIbb447+EMWEcupGO5AR+/ofKG4iOUJt7o1+6Id4/f8Pz/geX78Qp2k6CmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0xlbmd0aCA5MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNjDsOwDAIQ/ecgiMACaDep6o6NPdfy0dpuuAn21gQAYHYz7AOagontUBSg9m6YdKThK588KeV0A+4U3RZc6q+NlUa65vq9c69cGeS2HBX1uAKo3i9EGIioAplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9MZW5ndGggNTYxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVTS44lMQjb1ym4QKTwye88NWrNot/9t2M786TuB0USwNiM3q1bbGvhy2p1q9ntjz90c2/7PLmXvN8nz7Ecx6pPy/Rre9n70IsRuhdeuFXm8N8n/ZjPQAlE6uBWl32fqLiRRATRcHwjx/v4SXiFir6OznxO3HX4OInFO4kTx1fiv7vscR2krYGbx/Ao3G2yw6xuQyDWtDHVe+EP0JWvUGUj4tZ8XlvsX95GpHgSBE0HfTDngHsGMpw7wOAhHfSBK3IduFr0ZQtTvkOpYpWjuTu6kI3OodBDkkTmHYammtrHT8PFg+/AIAPjYA0MHwARxZR3XiTAl2UrrZK/g8+XXNQ+eEhGIjUqPE9OdosJ8uRTJ+hDs2eBs1kh/Nzkgcmz9zlkDmvCSdC5BxSAZiglF5PIWkUtHAkrcsoCKGopwpOBd5w65ompJnjIRaoG8kAdCT4L2BK6qMsWhRq889Xp7/P3oXr4+vPfG1TQoPbAAngM4tyEsqWyI40GB7oKyhyy6G2tGwG1DhayH00BvUVcfccRFtmrL3jZt1HviTrBXdlDlAKpcz+IC+9BHi1Oeshz7EaBEV+U1xUPMpFW4pjLvsheoCwIvIGlz/Xqvm3axyF5vNrZBhZyUKElBLTcupRHRhpgFS+KcTgxtFONm4k+mlROOVKEGEADETSbPR5JlVuozYCMkrmK7FJFWhIOGVLHzFX3Uv+N3f0ZLtE0Li6YdcbwI3An7AuX0H/+AQty3fQKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDI4NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtUrtxxTAM6zUFR+Cf0jzO5V7h7N8GpF5hA7IFEoQUzMR08Pguym30I6sp09/SCuA7KAFi7CSbTI00yEzIgp5ldsixdqOsAWHt72DGXWFYMRnUzkmaZ/BZ6pepwQILqTjZabUc1NpKUuiYCQe3Nv6AdYd3GFpKShsQVB1DUtUG5UTbVR376KV3nvN935kgPxTRgtSrb3dOhXGy33cakMDzDjlO2JS8KbELnp/VVhRDYK2OpgPwkkOk111qAJ9BENv0Ty+QjVowIApBQRA653HHzc3fca2y2eYbGxu5Ylqo3HkCRcSBA0INT0TOF3EcO4f1fucaNET+rGY30GYI1Ps0ApCIvje4UqvikN6rMoHei/Kuz/r9BzplbO0KZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvTGVuZ3RoIDIzNiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UDuSxSAM6zmFjoCNDfg82dnZInv/9kkkr0EKNvrE10aHTR6eA3MP/Fiz3TGH4/9htmArMXtyM5GVuJol2eZNONFgvg9yYvHcdD8b5e+T7UeDlpIU0ONqIjFB65kbRZhbMp2MM7M8uczrICfxJFWEs8FQ7xvFpMiJTc1vkav9NfXrqqSyt4DCd0vWZxiHDzD0kEF10KKkqtN66DI5NL4QyVCtsTr2xqAL2w2SwgqMCkw/cDWmEBn6pkk/S1cjiS5/RY55Eq1C5HyZvVOW5u+neASsGFCiVgUPuJ/wamb6wU/Fm2V/P7wcWJQKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvTGVuZ3RoIDE1NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtT0kOAzEIu+cVfkKAQOA9U1U9TP9/rZlUirADZrHuxIQUg/pGpOElo+nEd6gQ7gZxoitkYxt0gUJbuEYZ1kIl4omlnUt4CBuapKMcMQVZCBHW2Rs6sZXglD5wjXX+ShkXSIMx3cSK05J1Z8E4LoqnnHup6MpuSbPcf+LUZGC1v4D1Zp2cr3zeTtqm9Obj9x6f8f4BMKQzKwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9MZW5ndGggNDIxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1TO3IFMQjr9xQcwebv8ySTSfFy/zYSzkuz0oIxCHBVy5Kd+FSFeC753A/pkp8nooCvwR0kHbJLcqmoSe4lpvLx5G7xLakK18BZNIPoScSRmSUiUiwdp1ysGRoHnnMkysRXS3gMwqP7WhYRXuS24/B4HkS7eBhuO+LWYl70gB0WSpKIcWVhHkiyJxIV4b839QTqByAVCHVq5z+OXtuj18wZZk49OG8B4TCHS+QAAmjGsVOMA7NCBNpDDdrnT5XWupZYo0pxoR0WobBao9EHqgrYVEfPLtwWrARp/fxhjwdzY8UksMxJRygKYT94ly7UvjljRVNGL8lE5RtGrTuRLcKt6Bhn2Gw+5cXht5kVyGG+hmDgfQuuW//Hw1i0gf3HosQeQPMvMQpH46l7NKA1kx574ah6Yxic246efcSJTIywplJkRDqyfrPD5YNgRbyFKDbBPd69Rt/03M02jn5uNG5bshBuj05eK87Q61rC7onAJt+YzBpWK0cAUZsesmJ5JMhXi9qkuOx7+jHPpXAlHkrNBGnGKxvh97m9nu/n6xekBKoQCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtOTcwIC00NDMgMjAwMCAxMDIzIF0gL0xlbmd0aCAzNwovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjMyVDAyMlDI5TIzATNywAwLMxADJIdggSUzuNIA/9IKDQplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9MZW5ndGggMjg1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSuXXFMAzrNQVH4KljHuflpfjZvw1AOY0A8wRJl6qo2MSTuySnypcNUpXf4WsBP8AtViChLrYk7IinhINOeUYEUrdEqixrMC3awUKTiWT0oUzC53M1PsNLryUSaOKOyBP0wBpQYnsilyrREm2eQbYcZUkKkiaaQyLkhtKPDHdxDap1P1SPgiAcx/Y/9FDZUZjcQ1iIvdcUmLcKGuA9CRswYiGBZJqwGxazAr2pirlnSR3M4ZLVgCVYE+hJ7VXHuTNIdvsJh0OKbklKqegzMAKZtXpMbG6dZv4yV/SBVkcE40PnmxeZ1wJlmbO3mU5PWrxf2GTVReUFyXYvA6R2K8LJEucNDpGo6VcYdsX/ozd6f5TP+Bnff02QbOEKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvTGVuZ3RoIDIyNSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UTtuQCEM2zmFL1CJfIBwnldVXXr/tXZeO9kicWKHNScm/OLjIs9E7olPG6Iehp9mtg2ZFzYDOQ9W4RmxFywQZtJ6LsEzrELEwlW99ja32LXH1hW461nkHrAaq5vjbs2gOKNgtduQr2zkUntfYp/uyFl/muwh2SH8vmGU4BnfI4p8JtM0K+npPKgI8nJNVjZb8MOs3O+xGmmSJxKzQ1d2mIA+q7PSf/A2ZkT3Rj+hCpkVe2P3q7Rbie0mIhk/WYdvF+gQSsshR9XIAgPqxrqbLPITcuM/iEJ9/QL+YlIzCmVuZHN0cmVhbQplbmRvYmoKMzYgMCBvYmoKPDwgL0xlbmd0aCAxNDYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY9RDsMwCEP/cwofAYfQhPN0mvbR3f93Jq0mRfiJCGPCDAaSqp4DxzzwYis0fBu5pNdWRkEMcOoluoPp8MDZuh0YE50uhy2524IYQ3Ob5N1NqimuW8/G+XRiIVaCXsr6UaqY3OtFq/b3tJ2xKEX+0L+lfG4BmTiVMeXj3pVUQWfdUKexwt03Xu3T3j8cmTFkCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKPDwgL0xlbmd0aCAzNDIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVJJbsQwDLv7FfpAAGvxovekKHpI/38tKU+AXkaMLVMkNaN36TLkMp0Sq0vMLl/adKTE2PJbyLfK06yrhLtYZLV596p386HnJFEDJIabzpsYIY4bUnvPqhaLN0C6Av0pquxPmUGyiYMhrilXiuExyt3Mz7epXFOsLyG/pkO9riXQX1KdpwCXQsTzgVChIL84UBfs6q6+TZgGOnblKm0AmwMJGcvTxj+gmQBOspwynLXLNHIansEWxrjUlDFPdWoFcl94TABby8QB50K+VDMVAWIZzBGpRRUaL8BIYDbr925ZgUHRqLSVu0DyOt6dvBt82s/ZIQjPNvf+5AKVCrloYoxRw2DTkLIzaYTvLDjO+nbbGC8OiYsiHBa0Uz/1aVVuBjdEE5JpUq2eeO29OJCUIWT+Rwy2z3/FEE6dKK2DORkQV0GVDi3lI7a8fm54+/4DKHSDbAplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjw8IC9MZW5ndGggMTQzIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD2Puw0DMQxDe0/BBQ4w9bNvnguCFMn+bagLkEoEJT1K7o6J2Di4Cj4DpfrgaHmwJj7DTvu138PYrhuYEosgDYlr1IltzWFFF0uTKxFeoE/kknnmjb9uegkuovo1BTw1E6F97eytGUZqWx0PuOK6WnZYK6a4mhCOCuft96HyzJeuVMT/iWu8xvMLQYcr9QplbmRzdHJlYW0KZW5kb2JqCjM5IDAgb2JqCjw8IC9MZW5ndGggMTM5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nC1POQ7DMAzb9Qp+IICuyPZ7UgQd0v+vpdxOlMQLiggofOGwUfB5oogvE56qFB9J26dHzAYqFC08a8FdkctxiVcgLbc9GNjoOZrpQC7qOAbJ+eu5ZDcy/5E1OGnAMnDQZNQvKmyOttpam2r0zCY4RZKhIL22IWe3te7MCUaWFv4vXPKW+wu1WSuQCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0xlbmd0aCAxMjUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLU5BDgQhCLv7in5gElBQfM9MNnvZ/1+3MCZGSiml3QWCJbhUoKFQETzaEtrAr0AP6HIo3zQ47qY+cLG1s9i9KgeyC+2Z85ilfnd5hEb8zZIT+AYnq6csBrm9sA2ahpKmsypN7TC8mgKGKH2GWlqxPXAi3+3bPn9vXSdUCmVuZHN0cmVhbQplbmRvYmoKNDEgMCBvYmoKPDwgL0xlbmd0aCAyNjUgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVE7lgMxCOt9Co5gPgZ8nuzblyJ7/3YlkhRjGARIgLvLloPP/Ujklh9dChOn5W88z5TX0ga8TWynhKWYuURseSwLZrvYudPAeo8Fcms8V2YccYONBOIOz/NNik5jdw2CvMuIirehJsQriGiNFt8NCyU3x4Kn/R0pqm0oyamlNnYDQi6oMfDSAtFPZCPSJdoxtQ/MjP7RMrNDxVg9g2y5m8uAk0SoXrQgjGza4AaZEhJyTxkdVn1fBF/LQa0CoVaSoGZiGdtVCnaPtzl1cUmBAjpIL4xaKtTY3HHiv68ctLjBchouaxy9Mz70XSbjHJvHDFwK0G0cAaOf8znY9/Cv9Vy//0beZD0KZW5kc3RyZWFtCmVuZG9iago0MiAwIG9iago8PCAvTGVuZ3RoIDM0NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtkjtuBDEMQ3ufQhdYwPr4d54NghTJ/ds8erYZamRbIinlPtZtuL2OZW2rcezLm49p2d3+Wsxukdt+W5y08G3p0/xwPY75Dnu3LDefdQt4xYM+7sm27ZZZNsj3abFJx6KMRU21jb4F7+br/js3X7TIATOyvmyYQ4ZSx/R8LWAeK5vDXklGTxKOoA8Fc38COnkvO9wVqTKaUhRCkZCGOE/OEMDLj4JIAGJjwPzS7VbLxBqVgj0/KhwzZJFjQcS4FaWEusdpzycQgsGxS5CppwoWlNIKchvf49JyEtfEdSfhCBaK2TObKKjAKxiQbnKCrsL0wJXiVH2EnKiuMkeZxH691Um63+i3aZiJK0Q4/ERB11SvhX4JGNTplwUqK9Zdg8oFvwdhziRuRswT9RhX8cwPDV1O5a12cZ6P2hzrepilEbBCGXc8LItmoWXqWolnHd/tp33/A7hDgCcKZW5kc3RyZWFtCmVuZG9iago0MyAwIG9iago8PCAvTGVuZ3RoIDI3NiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1kUFuxUAIQ/c5hS9QacDAkPP8quqi//7bmon+IgIBY55JroUFBr6MiGpUB77tiqiTvpWpqMbfxbpRmeAiyglPQ27H67K7kW6wSkQ2LB1sns5CSNopeW/4vk98XeQ+2czFUtU2eNRCuhTS4Hg/WG73dGq0VdGErXXe5DS4N0yv/DHiYpio/btOZpr0hTb0HqEbFjG+ffUJ+8io1I0t5V6aIFJ+paJL5C0F11VcURMV06GFrsNDXtr1udvr+hWUaCX3PhnHn67Adng88XW5nE9m+rjFWfobz07qjW5k2kdBTvSq6SgbXaNciMi0X8YGU6MlSunIUo3JYVwxCWlwbUuxzzR1zHvIRynxoR3yn39b9mPQCmVuZHN0cmVhbQplbmRvYmoKNDQgMCBvYmoKPDwgL0xlbmd0aCAxNyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwzMjVQQMAUQ640AB2YA00KZW5kc3RyZWFtCmVuZG9iago0NSAwIG9iago8PCAvTGVuZ3RoIDE5MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kMsNwzAMQ++eggsUsGR97HlSFD20+19LOWkOCV/0iShpTnTIwGNBc8H5PKVpBDLxbRUjf5paQgLao0rFZ8nR1irNjjHqLZIMEkwmm/h1gcF0zzHVU5yFBYPhSQmQzIxhdxgHTHYvg2jCXapcJ2mwX6j0wyaPURljJOjLnCs4NfcqzLCzSNyodio3qgxNea/d9sCbTseVvUmG/qn3vX3WOQLndP7XOKIM6z4R5WiqG9TocuA66dHe7fUDAUlF8wplbmRzdHJlYW0KZW5kb2JqCjQ2IDAgb2JqCjw8IC9MZW5ndGggMjI5IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRu3HFMAzrNQVG4F/iPC+XS+Hs3waUL4UNWCQhgE4RCKz5ih2oXfjSNVR943eFTf1Zt+OC2qBsmOiwdFg07PCkA86Ke+Kz3Mh28msjdL9YOhWyFFYskCGcEWTNjLE/D9VCUWLQPhc/S8veE3dkE+kjafOzWpGZ2ATjowwyAz6JNh1SKr1RvIseR9B8BJ2CeaC7b2zzc5Em2i/zOclmCHbaVELqXh5UyJQXaZ4VsmAcP4wTxlUkLK9xGqHYs5wOXnJqSIiD3aEFJh6JRnNFwd1rclUCjXH6/zee9bO+/wBt11EWCmVuZHN0cmVhbQplbmRvYmoKNDcgMCBvYmoKPDwgL0xlbmd0aCAyNzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVBLTgUxDNvPKXwBpObb9jwPITbcf4udB9KM4qZuYrvWwoInPiyQu9H8P+0RjFj4GeSneGnkkdQBX4HXk5UwnjIN5hvJMTd1sTZ6IU4i+IlWbIfW8PG5s81rTX09vt4d2yGCpSD7Zo7YOAdd2A6j2NdTBePYIRmnw800ZM9E7vDrKhGtNsE9CEOWHKJGezuK58MzJZsH2mdlXS67dOgThVO8KtXbHRQU0sn9lqizJodCWU98GY7/7F7P9xNkhBdzFEp2ow/ZrNmo7r9c2qn5GnfLwpmqYHqQ7RqGUfL7jdGQpphrqk0Nv7ohcm+IYfSrN1eWNSTkjD2frFi4g2kp5wlqVA1bMvVcsp0G/o3I1NcvehhongplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQ1pSQkxSK1NUSVhHZW5lcmFsLVJlZ3VsYXIgL0ZpcnN0Q2hhciAwCi9MYXN0Q2hhciAyNTUgL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMKL05hbWUgL0NaUkJMUitTVElYR2VuZXJhbC1SZWd1bGFyIC9Gb250QkJveCBbIC05NzAgLTQ0MyAyMDAwIDEwMjMgXQovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA0MCAvcGFyZW5sZWZ0IC9wYXJlbnJpZ2h0IDQ0IC9jb21tYSA0NiAvcGVyaW9kIDQ4IC96ZXJvIC9vbmUKL3R3byA1MiAvZm91ciAvZml2ZSAvc2l4IDY1IC9BIDczIC9JIDgzIC9TIDg3IC9XIDk3IC9hIDEwMCAvZCAvZSAvZiAvZyAvaAovaSAxMDggL2wgL20gL24gL28gL3AgMTE0IC9yIC9zIC90IF0KPj4KL1dpZHRocyAxMyAwIFIgPj4KZW5kb2JqCjE0IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0NaUkJMUitTVElYR2VuZXJhbC1SZWd1bGFyIC9GbGFncyAzMgovRm9udEJCb3ggWyAtOTcwIC00NDMgMjAwMCAxMDIzIF0gL0FzY2VudCAxMDU1IC9EZXNjZW50IC00NTUgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDAgL1N0ZW1WIDAgL01heFdpZHRoIDExMDkgPj4KZW5kb2JqCjEzIDAgb2JqClsgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAKMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMzMzIDQwOCA1MDAgNTAwCjc0NyA3NzggMTgwIDMzMyAzMzMgNTAwIDY4NSAyNTAgMzMzIDI1MCAyNzggNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMAo1MDAgNTAwIDI3OCAyNzggNjg1IDY4NSA2ODUgNDQ0IDkyMSA3MjIgNjY3IDY2NyA3MjIgNjExIDU1NiA3MjIgNzIyIDMzMyAzNzMKNzIyIDYxMSA4ODkgNzIyIDcyMiA1NTcgNzIyIDY2NyA1NTYgNjExIDcyMiA3MjIgOTQ0IDcyMiA3MjIgNjEyIDMzMyAyNzggMzMzCjQ2OSA1MDAgMzMzIDQ0NCA1MDAgNDQ0IDUwMCA0NDQgMzMzIDUwMCA1MDAgMjc4IDI3OCA1MDAgMjc4IDc3OCA1MDAgNTAwIDUwMAo1MDAgMzMzIDM4OSAyNzggNTAwIDUwMCA3MjIgNTAwIDUwMCA0NDQgNDgwIDIwMCA0ODAgNTQxIDI1MCA1MDAgMjUwIDMzMyA0MzQKNDQ0IDEwMDAgNTAwIDUwMCAzMzMgMTEwOSA1NTYgMzMzIDg4OSAyNTAgNjEyIDI1MCAyNTAgMzMzIDMzMyA0NDQgNDQ0IDUyMwo1MDAgMTAwMCAzMzMgOTgwIDM4OSAzMzMgNzIyIDI1MCA0NDQgNzIyIDI1MCAzMzAgNTAwIDUwMCA1MDAgNTAwIDIwMCA1MDAKMzMzIDc2MCAyNzYgNTAwIDYwMCAzMzMgNzYwIDMzMyA0MDAgNjg1IDMwMCAzMDAgMzMzIDUwMCA1OTIgMjUwIDMzMyAzMDAgMzEwCjUwMCA3NTAgNzUwIDc1MCA0NDQgNzIyIDcyMiA3MjIgNzIyIDcyMiA3MjIgODg5IDY2NyA2MTEgNjExIDYxMSA2MTEgMzMzIDMzMwozMzMgMzMzIDcyMiA3MjIgNzIyIDcyMiA3MjIgNzIyIDcyMiA2NDAgNzIyIDcyMiA3MjIgNzIyIDcyMiA3MjIgNTU2IDUwMCA0NDQKNDQ0IDQ0NCA0NDQgNDQ0IDQ0NCA2NjcgNDQ0IDQ0NCA0NDQgNDQ0IDQ0NCAyNzggMjc4IDI3OCAyNzggNTAwIDUwMCA1MDAgNTAwCjUwMCA1MDAgNTAwIDU2NCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIF0KZW5kb2JqCjE2IDAgb2JqCjw8IC9BIDE3IDAgUiAvSSAxOCAwIFIgL1MgMTkgMCBSIC9XIDIwIDAgUiAvYSAyMSAwIFIgL2NvbW1hIDIyIDAgUgovZCAyMyAwIFIgL2UgMjQgMCBSIC9mIDI1IDAgUiAvZml2ZSAyNiAwIFIgL2ZvdXIgMjcgMCBSIC9nIDI4IDAgUiAvaCAyOSAwIFIKL2kgMzAgMCBSIC9sIDMxIDAgUiAvbSAzMiAwIFIgL24gMzQgMCBSIC9vIDM1IDAgUiAvb25lIDM2IDAgUiAvcCAzNyAwIFIKL3BhcmVubGVmdCAzOCAwIFIgL3BhcmVucmlnaHQgMzkgMCBSIC9wZXJpb2QgNDAgMCBSIC9yIDQxIDAgUiAvcyA0MiAwIFIKL3NpeCA0MyAwIFIgL3NwYWNlIDQ0IDAgUiAvdCA0NSAwIFIgL3R3byA0NiAwIFIgL3plcm8gNDcgMCBSID4+CmVuZG9iago1MiAwIG9iago8PCAvTGVuZ3RoIDM1MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UkGSHDEIu/cr+EBXGTAYv2e2Ujls/n+NhL17aWTLIAFtq2TIqzPltZEybUri7kufqTiV/HvUZ6PvR3NIqInWlMgtzMih8nlsmmSYWGXnu1fHz+Np56Yc0cT3lkSdDwRCctbRbA1ETTJEAze6JTajSlQzwyXWQhVGR1XrCJ11kCe8rRSPRM4kA43YKj6hN8aJ6IIMEbyh/bRAzoAT9uMBtw6doEe9sXWIgtUWInTcO4IZ0chKO9fY+e90IkMMGjHA6JktcrTubF05mwQydH6QuskiqI0d4YH5WdOb64DP85LrOx1FxHXynRYFXsUmXl15l6wwdRGaJ9/YmTDtJ9WDeoe3xdL8KCsNv0pE4Hfey+p3OBUdaJOLjw3n1a72BXM02XDiTglI4qDRsovVEsRi65jdBbtJQn4WHRfI2AfQcB2o3QgWxVHRjXLDPVf/GfD9x7+fv8+f//aSkKcKZW5kc3RyZWFtCmVuZG9iago1MyAwIG9iago8PCAvTGVuZ3RoIDM3MCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kkty7TAIROdaBRtwlfjotx6/SmVws/9pDth5A1e3BYKG1ughXa4Vchk0lkmEyj9tpkPCtvw03VMc9mnmU6K72PFK87EK7+bvSRgFvMPBvonENHEqZGkf40GdGYGZ0zS66Djia8upYoSNLrHlUmJrFN5Nhz4nqLlCtE/JQuvkqcuc4NgvHiLF5pHFjLqWVObOHjtr70GNSLKfmklokzTX8Wn/N/PSTIAOS7rk8pWEv7pNxt0uzSYzNWdW8JcqbGRlVuKo/jRVWDCDhfjcD6Iwu8NOKjusMGT3grsxWZSSwTZpgh/62PV53IKZ4oifF7OcscxwtpkZOPPn692+m+/0wvG4GJ6lBJ/Ot1456bFjS0UPoq0XUvlEMUN+OmyZMePt6ajRw11lSBZtOyOKExYTvznpZGRWnBp7CelKeV5MQsw6pt2zJjaoDKJlEb2Uq4P9iCKgBJH7yjk1Vz6KnOCQgpKd7+3BjOYGvn4BNmOOoAplbmRzdHJlYW0KZW5kb2JqCjU0IDAgb2JqCjw8IC9MZW5ndGggMzE4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nFVSSW7EMAy75xX8QAFrs633TFHMIfP/ayk5KNCTCEshKSoxBgY08KVjw7fDXfAtV0Pd+FwmiS+JgZtwFQw8IB8wBa+nuQVmkyAVtvwANnceeB9Yavclof/gGZDdkLZGfb8DKsW4FomUniiYUHu8qBsBNaliBgkjh6ahGGjRF1Q57xyQTAhkFu+A1N6sryutAWkHQijcZPCOpsQCMqLkCWLC+ETb1ends5KgNl0Y27WB5XjsOqV1WSdrsruSfo7zwv2cgpaMdSU7LtLR339HuK83Y+MM1/kctKLjrTBtltPmpHenD2M2zrOZStdX51FIp3JWUb5sjY7Ee17mhCar020Um6j2CUQqsnFqsxWaHO3lOk7XpirjvNZGkMQn1i5t+pWxuDtr1P+xyZudATVJ1BvJU6M67+vnF4tve+gKZW5kc3RyZWFtCmVuZG9iago1NSAwIG9iago8PCAvTGVuZ3RoIDQ1MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtk0mOJDEIRfc+BRcIyQyezpOlUi+i77+tB85FBIjh8xkcEdLlsSmPdolYfCo/2ixcfAz534y/nS1vs9Oxmbhu8X3EgxTSPs2nSTjeHQUQfZb8tCC2LAa05zcqEg8VwvwW1biyR3mW+EmPX0TyfO30uKJp5fkEWa/E01dpfuC1+uW3ZnJbaBvWM+VAakk8cTXv2UuITedLBqVl35OPLo2apidzugspb8ukQU2spLon3cIsxcf17i67zBDYs7pJYnsJ/aX5yMlmTbRD+gSyHGeikcDglUrRVVStBjorBgagDDqGsSGCyEplRk8uyWunBuZTSzJGWBaaeugLftVrJgKRw6BL9oMDM9WVAb5NWSGmt8HUjtASIBOdqAF4srzA/q20Sj7le+CRtsdUVl2aXuFaVg4l+XI/CTJl5M4ynRXQBnVW/j5NRxUd6aBgxZEzepLjYugqMg7OUeMlTgE8GajM0Zgf3eRxePWVQGRpLYrJD61NEB1gsQ8lLk9SLb6nmYBlYQO5k7yek1Xn3eLbZobma1lg3IpsJw9X531dStx9HNbvc7E8fwZvOQbLJRqDDs115NHA/fscP+1f+/0DAgKvgQplbmRzdHJlYW0KZW5kb2JqCjUwIDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRmlyc3RDaGFyIDAKL0xhc3RDaGFyIDI1NSAvRm9udERlc2NyaXB0b3IgNDkgMCBSIC9TdWJ0eXBlIC9UeXBlMwovTmFtZSAvRlRYV1pBK1NUSVhHZW5lcmFsLUl0YWxpYyAvRm9udEJCb3ggWyAtOTcwIC0zMDUgMTQyOSAxMDIzIF0KL0ZvbnRNYXRyaXggWyAwLjAwMSAwIDAgMC4wMDEgMCAwIF0gL0NoYXJQcm9jcyA1MSAwIFIKL0VuY29kaW5nIDw8IC9UeXBlIC9FbmNvZGluZyAvRGlmZmVyZW5jZXMgWyAxMDIgL2YgMTEyIC9wIC9xIDEyMCAveCBdID4+Ci9XaWR0aHMgNDggMCBSID4+CmVuZG9iago0OSAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9GVFhXWkErU1RJWEdlbmVyYWwtSXRhbGljIC9GbGFncyA5NgovRm9udEJCb3ggWyAtOTcwIC0zMDUgMTQyOSAxMDIzIF0gL0FzY2VudCAxMDU1IC9EZXNjZW50IC00NTUgL0NhcEhlaWdodCAwCi9YSGVpZ2h0IDAgL0l0YWxpY0FuZ2xlIDQzOTA5IC9TdGVtViAwIC9NYXhXaWR0aCAxMTE3ID4+CmVuZG9iago0OCAwIG9iagpbIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwCjI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDMzMyA0MjAgNTAxIDUwMAo3NTUgNzc4IDIxNCAzMzMgMzMzIDUwMCA2NzUgMjUwIDMzMyAyNTAgMjc4IDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAKNTAwIDUwMCAzMzMgMzMzIDY3NSA2NzUgNjc1IDUwMCA5MjAgNjExIDYxMSA2NjcgNzIyIDYxMSA2MTEgNzIyIDcyMiAzMzMgNDQ0CjY2NyA1NTYgODMzIDY2NyA3MjIgNjExIDcyMiA2MTEgNTAwIDU1NiA3MjIgNjExIDgzMyA2MTEgNTU2IDU1NiAzODkgMjc4IDM4OQo0MjIgNTAwIDMzMyA1MDEgNTAwIDQ0NCA1MDAgNDQ0IDI3OCA1MDAgNTAwIDI3OCAyNzggNDQ0IDI3OCA3MjIgNTAwIDUwMCA1MDQKNTAwIDM4OSAzODkgMjc4IDUwMCA0NDQgNjY3IDQ0NCA0NDQgMzg5IDQwMCAyNzUgNDAwIDU0MSAyNTAgNTAwIDI1MCAzMzMgNDcyCjU1NiA4ODkgNTAwIDUwMCAzMzMgMTExNyA1MDAgMzMzIDk0NCAyNTAgNTU2IDI1MCAyNTAgMzMzIDMzMyA1NTYgNTU2IDUyMwo1MDAgODg5IDMzMyA5ODAgMzg5IDMzMyA2NjcgMjUwIDM4OSA1NTYgMjUwIDM4OSA1MDAgNTAwIDUwMCA1MDAgMjc1IDUwMCAzMzMKNzYwIDI3NiA1MDAgNjc1IDMzMyA3NjAgMzMzIDQwMCA2NzUgMzAwIDMwMCAzMzMgNTAwIDU1OSAyNTAgMzMzIDMwMCAzMTAgNTAwCjc1MCA3NTAgNzUwIDUwMCA2MTEgNjExIDYxMSA2MTEgNjExIDYxMSA4ODkgNjY3IDYxMSA2MTEgNjExIDYxMSAzMzMgMzMzIDMzMwozMzMgNzIyIDY2NyA3MjIgNzIyIDcyMiA3MjIgNzIyIDY3NSA3MjIgNzIyIDcyMiA3MjIgNzIyIDU1NiA2MTEgNTAwIDUwMSA1MDEKNTAxIDUwMSA1MDEgNTAxIDY2NyA0NDQgNDQ0IDQ0NCA0NDQgNDQ0IDI3OCAyNzggMjc4IDI3OCA1MDAgNTAwIDUwMCA1MDAgNTAwCjUwMCA1MDAgNjc1IDUwMCA1MDAgNTAwIDUwMCA1MDAgNDQ0IDUwMCA0NDQgXQplbmRvYmoKNTEgMCBvYmoKPDwgL2YgNTIgMCBSIC9wIDUzIDAgUiAvcSA1NCAwIFIgL3ggNTUgMCBSID4+CmVuZG9iagozIDAgb2JqCjw8IC9GMSAxNSAwIFIgL0YyIDUwIDAgUiA+PgplbmRvYmoKNCAwIG9iago8PCAvQTEgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMCAvY2EgMSA+PgovQTIgPDwgL1R5cGUgL0V4dEdTdGF0ZSAvQ0EgMC40IC9jYSAwLjQgPj4KL0EzIDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDEgL2NhIDEgPj4KL0E0IDw8IC9UeXBlIC9FeHRHU3RhdGUgL0NBIDAuOCAvY2EgMC44ID4+ID4+CmVuZG9iago1IDAgb2JqCjw8ID4+CmVuZG9iago2IDAgb2JqCjw8ID4+CmVuZG9iago3IDAgb2JqCjw8IC9GMS1TVElYR2VuZXJhbC1taW51cyAzMyAwIFIgPj4KZW5kb2JqCjIgMCBvYmoKPDwgL1R5cGUgL1BhZ2VzIC9LaWRzIFsgMTEgMCBSIF0gL0NvdW50IDEgPj4KZW5kb2JqCjU2IDAgb2JqCjw8IC9DcmVhdG9yIChNYXRwbG90bGliIHYzLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZykKL1Byb2R1Y2VyIChNYXRwbG90bGliIHBkZiBiYWNrZW5kIHYzLjguMCkKL0NyZWF0aW9uRGF0ZSAoRDoyMDIzMTAxMTIyMjAzMiswMScwMCcpID4+CmVuZG9iagp4cmVmCjAgNTcKMDAwMDAwMDAwMCA2NTUzNSBmIAowMDAwMDAwMDE2IDAwMDAwIG4gCjAwMDAwMjE1MDggMDAwMDAgbiAKMDAwMDAyMTE4OCAwMDAwMCBuIAowMDAwMDIxMjMxIDAwMDAwIG4gCjAwMDAwMjE0MTYgMDAwMDAgbiAKMDAwMDAyMTQzNyAwMDAwMCBuIAowMDAwMDIxNDU4IDAwMDAwIG4gCjAwMDAwMDAwNjUgMDAwMDAgbiAKMDAwMDAwMDM0MiAwMDAwMCBuIAowMDAwMDA1NzI2IDAwMDAwIG4gCjAwMDAwMDAyMDggMDAwMDAgbiAKMDAwMDAwNTcwNSAwMDAwMCBuIAowMDAwMDE2MzIxIDAwMDAwIG4gCjAwMDAwMTYxMDUgMDAwMDAgbiAKMDAwMDAxNTYwMSAwMDAwMCBuIAowMDAwMDE3MzY4IDAwMDAwIG4gCjAwMDAwMDU3NDYgMDAwMDAgbiAKMDAwMDAwNjAzMyAwMDAwMCBuIAowMDAwMDA2MjY4IDAwMDAwIG4gCjAwMDAwMDY3MzUgMDAwMDAgbiAKMDAwMDAwNzExOSAwMDAwMCBuIAowMDAwMDA3NTk0IDAwMDAwIG4gCjAwMDAwMDc4MzEgMDAwMDAgbiAKMDAwMDAwODIxOSAwMDAwMCBuIAowMDAwMDA4NTM5IDAwMDAwIG4gCjAwMDAwMDg4NTAgMDAwMDAgbiAKMDAwMDAwOTIxNiAwMDAwMCBuIAowMDAwMDA5Mzc5IDAwMDAwIG4gCjAwMDAwMTAwMTMgMDAwMDAgbiAKMDAwMDAxMDM3MCAwMDAwMCBuIAowMDAwMDEwNjc5IDAwMDAwIG4gCjAwMDAwMTA5MDYgMDAwMDAgbiAKMDAwMDAxMTQwMCAwMDAwMCBuIAowMDAwMDExNTY5IDAwMDAwIG4gCjAwMDAwMTE5MjcgMDAwMDAgbiAKMDAwMDAxMjIyNSAwMDAwMCBuIAowMDAwMDEyNDQ0IDAwMDAwIG4gCjAwMDAwMTI4NTkgMDAwMDAgbiAKMDAwMDAxMzA3NSAwMDAwMCBuIAowMDAwMDEzMjg3IDAwMDAwIG4gCjAwMDAwMTM0ODUgMDAwMDAgbiAKMDAwMDAxMzgyMyAwMDAwMCBuIAowMDAwMDE0MjQzIDAwMDAwIG4gCjAwMDAwMTQ1OTIgMDAwMDAgbiAKMDAwMDAxNDY4MSAwMDAwMCBuIAowMDAwMDE0OTQ3IDAwMDAwIG4gCjAwMDAwMTUyNDkgMDAwMDAgbiAKMDAwMDAyMDA4MSAwMDAwMCBuIAowMDAwMDE5ODYyIDAwMDAwIG4gCjAwMDAwMTk1MTYgMDAwMDAgbiAKMDAwMDAyMTEyNiAwMDAwMCBuIAowMDAwMDE3NzM1IDAwMDAwIG4gCjAwMDAwMTgxNTggMDAwMDAgbiAKMDAwMDAxODYwMSAwMDAwMCBuIAowMDAwMDE4OTkyIDAwMDAwIG4gCjAwMDAwMjE1NjggMDAwMDAgbiAKdHJhaWxlcgo8PCAvU2l6ZSA1NyAvUm9vdCAxIDAgUiAvSW5mbyA1NiAwIFIgPj4Kc3RhcnR4cmVmCjIxNzI1CiUlRU9GCg==", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T22:20:32.291769\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.figure(figsize=(8, 3))\n", "\n", "# Plot proposal\n", "plt.plot(x_plot, p_plot, color=\"tab:blue\")\n", "\n", "# Plot target\n", "plt.plot(x_plot, q_plot, color=\"tab:green\")\n", "\n", "# Plot initial samples\n", "plt.hist(\n", " x0,\n", " density=True,\n", " color=\"tab:green\",\n", " alpha=0.4,\n", " bins=tf.linspace(-4., 4., 40),\n", " zorder=1,\n", " label=\"Initial samples from $q$\",\n", ")\n", "\n", "# Plot samples after samples\n", "plt.hist(\n", " samples,\n", " density=True,\n", " color=\"tab:purple\",\n", " alpha=0.4,\n", " bins=tf.linspace(-4., 4., 40),\n", " zorder=1,\n", " label=\"AIS samples\",\n", ")\n", "\n", "# Plot samples weighted by their importance weights\n", "plt.hist(\n", " samples,\n", " weights=w,\n", " density=True,\n", " color=\"tab:blue\",\n", " alpha=0.4,\n", " bins=tf.linspace(-4., 4., 40),\n", " zorder=2,\n", " label=\"Weighted AIS samples\",\n", ")\n", "\n", "plt.title(\"AIS samples\", fontsize=22)\n", "plt.xlabel(\"$x$\", fontsize=18)\n", "plt.ylabel(\"$p(x),~ q(x)$\", fontsize=18)\n", "\n", "plt.xticks(np.linspace(-4., 4., 5), fontsize=18)\n", "plt.yticks(np.linspace(0., 1.0, 3), fontsize=18)\n", "\n", "plt.xlim([-4., 4.])\n", "plt.ylim([0, 1.1])\n", "\n", "plt.legend(loc=\"upper left\", fontsize=16)\n", "\n", "twin_axis = plt.gca().twinx()\n", "\n", "plt.plot(x_plot, f_plot, color=\"tab:red\", label=\"$f(x)$\")\n", "\n", "plt.xlabel(\"$x$\", fontsize=18)\n", "plt.ylabel(\"$f(x)$\", fontsize=18)\n", "\n", "plt.legend(loc=\"upper right\", fontsize=16)\n", "plt.yticks(tf.linspace(-60., 60., 3), fontsize=18)\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "We observe that the distribution of the AIS samples (purple bars) is similar to the target distribution $p$ (blue line).\n", "Further, weighing the AIS samples by their importance weights (blue bars) does not appear to significantly change the resulting distribution.\n", "This would suggest that moost of the importance weights are close to $1$, which we can verify by plotting them on a histogram." ] }, { "cell_type": "code", "execution_count": 62, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "center-output", "remove-input" ] }, "outputs": [ { "data": { "application/pdf": "JVBERi0xLjQKJazcIKu6CjEgMCBvYmoKPDwgL1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDIgMCBSID4+CmVuZG9iago4IDAgb2JqCjw8IC9Gb250IDMgMCBSIC9YT2JqZWN0IDcgMCBSIC9FeHRHU3RhdGUgNCAwIFIgL1BhdHRlcm4gNSAwIFIKL1NoYWRpbmcgNiAwIFIgL1Byb2NTZXQgWyAvUERGIC9UZXh0IC9JbWFnZUIgL0ltYWdlQyAvSW1hZ2VJIF0gPj4KZW5kb2JqCjExIDAgb2JqCjw8IC9UeXBlIC9QYWdlIC9QYXJlbnQgMiAwIFIgL1Jlc291cmNlcyA4IDAgUgovTWVkaWFCb3ggWyAwIDAgNTA2LjAxODc1IDI0Ni43NTEyNSBdIC9Db250ZW50cyA5IDAgUiAvQW5ub3RzIDEwIDAgUiA+PgplbmRvYmoKOSAwIG9iago8PCAvTGVuZ3RoIDEyIDAgUiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzNWk2PFDcQvfev8JEc8Lr87SOED8GNsFIiRTmgZVgWzS6wS7LKv89zz4yr2p6BSAmokZbdfv3artflssvlPnuy+evqYvPL88fq59fTGV9d3E2kPuDnUhn1AT/3itRz/FxORj006noKJmpDOYV6uRWX1kedAtkA1Cyu3k/Tu+nsERq5Q4vPp2C1nx/BHxZN+pJ1ZmArAEtZhwq1Zxqyb9XuWr2EjbBXZ1iMPioyJauTKTEe+mnXPuuEFqbHEHo/fcb/szhSIWgLRijZOeWsJhOh4OJ6enwOTWg9UKjI+dvp7Bkpyur83fSAflLnH6aCRoA/MPMVJV3I22yDo6Si9omfsjrOz4WZ+fR8+oymDdmIFszhT8i02dlcX2HSzsawM0WhiYevz1/89nxzs7l9s314fXXz55168lG9ml5N8wuYyBvtfDTWHaQL5KR4slbnmeN8iFV+MY4W+o3xMaf/S7//t/rr42Xu+N/ptwa9Z+tyOugXyGn9xWhfOTGH7H6Afvfd9Kego4wwBk6qtyFrqhTjEv0A8fZ7iXfeaRsCkT+oF8hJ+c4GzFfguJxd+gH66Xvp94Tf6D61wS+Q0/pL2nNcif4/6i9L60fx5iC+GZ3qrJ6yp2Y0IyeN9jHsOKaY8l8H7TeNJml0hCUhYT3xkdqLFoCZFzKYHHGrzt4tFgXApKgpBEOlkRhgUkEMYyw0TrtulOK0SZjA29AXQCOR8ZgLjTdtYZQI08hp70w2udEEImgFzwbvC9MYYRriy8FPuTlYIoJWdI4FGQXTGGGaczoVxKptNIEIWkLgEwVeBwXCtGjgt7rqN5pAmIb8IVLkuBIAkzJpExNuNZZABC3piFSG2DKBMK1gSCMkfXOoRAQt6WCCfB0CaTRrSHubko+8LDIiaF5nxAyJ1ZMRptkM/9kSm0clwjRkHnUedE2pRAQNQW5ccLxmCYRpNUMLmRJLEAjT4L8IX/PYlYigBU2ekudOBSJoBR70xEIZYBIGQ0DywD0ywKQS4L2CxaaxBNJoDtlKzsWX5gGJMI2idq7k0kJPIkyreV3xgddDBgQJS04wJhGzGGEa5tiUMhnuUiCChuwfU29yTGOEaRgJCDXrxVLNCNOig+9C5HcrEaaluhqEzANSIkzLDm882tiCRSJMQzYYHBYWliAQQQtwTPQ8biXSaN4gtyTveVqQiKB5+C/WDKTRGBE0LDTYyBhBY4RpjuBBawJ3KhCmIT2iaK1tXpAI0wLhneMltdCTiKAhj4zJRrZNIEyDB4O1iedciQha1iaY7LhTgTAtETzoWGe7FpSAxcuWJDIMRngq7fcEZZ84E96cyCvSLtFYphUv5gwBAV0Qg3lOpHcZxvUuLfRIR1wpKRub9nc+LTKRj4ur21PNfdnfGFp7M99A9lkIkwC5cLhxs2j44hRtc+qGmm94HfbX9ztBWItKNMZiuXHfauLqlNGXC9veL65OKu0MOntmD06473PLodyQkZ2LasORZDjun/ARY40CFwHY2WbX8e+qpo9/qPOXR7uEfo2RKjrdI0e3YOnwFHIeS9/q136t432ZRHTcCidf6djCoxhm3+jYLzr+PNUmHtbGkPj53VYl1ZnhK008vf50dXt18War3m5u7q6+/L1oUS1d5uEGrwg5MSL6dqN+VTfq7JGbCz+abF0HIna4uPBx/y9NNURDxnY+2KJuuZw1F4dOPaW6pw4lo9CWu1yXMddy5+0IdI/sC1Wv1IpUdTbXDQCyI0yE9mDzAAwy16eqs7luRDwAk2Mz+gjSC12hrs7ouifKOmasq9RkjMggdYXKOqtrnRJbqoR9ODtoRAatK1TWWQ1lIQKxWMeajhEZtK5QWWd13QYjhGKA3U3HiAxaV6iss7ruyRFDAbsu9tCIDFpXqKyzutYHEEPBtZ3X9hgyaF2hss5qbKhrvc5jSm/53YiMWtenrLe6llcQQ9jomWb1EWTQukJlndW7ilB02HWyjhEZtK5QWWd1rU4hhmym2GaHI8igdYXKOqtrDSzP9crCHhqRQesKlXVWQ1m2c3HJso4RGbSuUFlndS02IoZMcYF1jMigdYXKOqtr4RMxZJzJLY8fkVHr+pT1VtcyrNcBm2pqMXQEGbSuUFlndS0JZyAmePbZiAxaV6issxrK8Dtkbw8F6+0xZNC6QmWd1bXwHjVyXWPa3usIMmhdn7LB7HoKYIBQ8i3dFQime2zLlnIbtkZ5nZh6LIFIisFHji1GiOphRKeZwTUK7OTUIxAEVCj2cCiylQhW6nra0qlu4AoF9nIg0CKucNs2DwqknuRmRNxCNYNrFNjJqadTiK36SJs4BELGalPiUjWDaxTYyaknYQgubxxXriQCG0Neim7Y+uT1WurJHCLLYcZg943I8AZWqKyz+nr4Oml7DBm0NmVWvdyZMn+duvzaNnSHU6E7wNlO0+v+A9nrYx/Igta3NXxXKzl8QHS0LTPbvPuqluZXeClPjFLU5O3uqZqiWXHWg7j8+lmP+vhOPXrxWl1df/p4++XNzcVG3W+uLt9/uZOnQK+mfwBKZAwTCmVuZHN0cmVhbQplbmRvYmoKMTIgMCBvYmoKMTkwOAplbmRvYmoKMTAgMCBvYmoKWyBdCmVuZG9iagoxNyAwIG9iago8PCAvTGVuZ3RoIDIxNCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9UEuOxSAM23MKHwGHkMB5OhrNonP/7XOo+iSETcjHcZqhg1NX9kSk44etaMd/81m/90HuIrmK+E6ocCrNFq42GRh6G+H9gAoUFomNSceubDWxinsQppK7ce+XzQEqQ8wTK0B3pB2IrqIiOvSNIVDsjGaypHDtEmYaJbhakX7avvDoHyhNA4PI0B5KXUtrg5QNrHlUs1B8+OPH3Ua8LCiVZDHL6hC170BoXxVeLcJhG7GsBpWPR85j6N3+mqurzZS5QxrmqI3LgYrd39/K/P0Aq2JNIgplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9MZW5ndGggMjkwIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nEWSu3HEMAxEc1aBEvAnUI88ngvk/lMveDo7kPCG+OwKVIoQkyhe0Zsylb5kDUo2/ayIyd/rVJwgjRhMKrSdtKicrOla7YTjbtoxbxGfw6aI6RyoTW2UkCwENeQ3wJy8KV3GR/ocD8AKhPyfkuItDumHxCgSD499JS9Uc2CA5whtcg0owWHuE6+llofupZgzpDJRSHmfHtTwdBtOkOF3jKqTEcwqdFvoh9oPOWYYduIWZJXjAt9ljUw0sshk4BkXjmlD9yG1WdFkH0K9bnwP+nX64dmEp0+hOjqjqZuOC90fZ6D7UPkD0FAu7IsUrR7PBnBjirUaJvAJ1zKrAahzD3jy1Hszrgj5UKbCnnEPIoG99/lHrr+/5V6v9f0Ly3dvTQplbmRzdHJlYW0KZW5kb2JqCjE5IDAgb2JqCjw8IC9MZW5ndGggMTYyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVQS3JDMQjb+xQ6gvkY4/Mkk8nCvf+2gr5uLA0gIWxmmJDkY7IQoXjLKDrxM7p+C+QQY0InMqCBk7CN15B54OxTsa1BRKtOtpaUulge1GSI4xDUOZKJMGcN4Z0h3EqZHeOO/0APY/MO5ZJwgXq0WMug3VSVVgaV3esaGYAdsgp1m22K1LB4iy1Y1oDvOqm89e8feO7r+YU7vuPzC6LJNwcKZW5kc3RyZWFtCmVuZG9iagoyMCAwIG9iago8PCAvTGVuZ3RoIDM5NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kktuBCEMRPd9Cl9gJPwFzjNRlEXn/ts806NIrbbBuKgqk1kyJFxe6hJbpWbJl15RW6Jcfq+IONl9hZvkYKVb0lNiuGROeV9eJjW2uIYUFct14vsym8/O4ISV6OLEWFS0SnIPURCy1hNjdKWzPltDElparId2ZUJycXbBeU7igttB2yWRm3vgHUvMiA8DVmEqVs2ciMBABRW6fU9x+PhKcS9xdKMHXu6TPjSNRg76Gi0m2O0IVpmOE5sVlc6Unqkyjcsd0IPFafHIY7GteWLzekw3uLwSkVv2EQ8M2uFimGPWgW2c7fVANB9uahuC6hQueNn5q7HH5Z30vET3Jqn8JIo7qwm3QSjSDlBEIonjFyP2tr8Z+twyod5Dwv4TPT72KKDOGBTZXm3GacEwiwl0S0XGxgU7uuDrUOPOdpXwjLQT3MWWRIyisewMNHkw1fhx3qRhYMce2ziZKzsM35N1Hl6sKqgsegMl3RutsAffWSCzEBc8rwctYNvZ/f/k7+vn+v4D3jSTYQplbmRzdHJlYW0KZW5kb2JqCjIxIDAgb2JqCjw8IC9MZW5ndGggNDAyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSTdLdMAjb5xRcoDPm1/g8r/NNF6/331bC6SJBsQOSgIiQJb7l15EIk6glv/UhrJK/A7zl+4SWFGDxT09j+DyuQWDHRFOsW8pxbL0Jvo+hPC70TJrGm6a6CDrBDRptnAWqF5WU8n2C5Tdy8MsS3UhfW0yV+Q5OB+dBTDzgo4bL7Cj1vWiXGDL9tFilhI66VAnrqRDgtHUmom5fpJs3Cun4tmEMZuMEzLHWRAfD1eKtI9R3jEmvuYF5T9rO6dhEPLeae94cQwSX6+QYsleLgstXCAwyfB66Adh7ruu8/9ctkOgcdDPkwjFBm2yH2J7Gha8R2xLJdtaM2uyOHE15TxzjihG6YJhiKT2WjhXO3zFZ32y2F/kaJz12Jo5aIlVuABHVbhBDO/wEdYcaQRhEu7zr9nn+zNjUNlZvhlrcPYNDC+6JQ3HMWGzZ+OEpms8mZt5o02K8QcbONvxiFHcm6EyiAjsDidiy4PIZtju4EUk5BsdFGwZRGKntFFKj0LtkqIAlGK0w8l8z9f/8A6GvncwKZW5kc3RyZWFtCmVuZG9iagoyMiAwIG9iago8PCAvTGVuZ3RoIDI2OCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1kUluRDEIRPc+BReIZAZj+zw/amXRuf82D1pZWJQYiiocETLFlnxdCTWJnPKtw+8RXSm/o5IaW97Dj8tVcdLLxH2KmzzDbkrAsIvCtJmeoesWUA2e7CPLyYbJrd5mJJg2AwASOjxZNlEyPxxTIlz0rtZlaK3IxMlGbiiOiyai1Ywf6+ky4Gd9ItqetuRRGfY48QTzXdkgZSapYMCXdqRCvTN4aMOzNGq7psJB3iBlo4qxLbBre6OlexInVkZR63T46Ygzpgtp4gh+9dtsVGCo/To5MT4qWusvpLdOmnxNfcpqTGW7HM6DyPx0Z58VQQDXKYl11p1KFzvXLtNqU/5/+hk/4/UHJ55hhwplbmRzdHJlYW0KZW5kb2JqCjIzIDAgb2JqCjw8IC9MZW5ndGggMzE1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVROXIDMQzr9xX8gGfEU9R7nPGkcP7fBuDaFSGKBwDmWrLEtjyOxFGpdvnRizBM/i6P4NcbYMmjPiA3wWpRF6tkhakxPC/NIlC8dcluyYUsxvThHg1lsJXIEuyU2OKe0i3eTKu1BBK6t0SBHjAjG3SQr5KIFJIJ3fi5afmX4HGM3FKLoyFLgzWWUoZucCsLsRMTMbdvZNhYBlHbJnIjkI8DDpTY2ZhXsEvP2AVx6Cxk34MUu250QM5KqJC9zVJvkJLIMTGQS7IPbOF7nwlwP6f6PsP7+h1Vuu6TQIHbR6mj0O2AlcEVsEIN9QCZzXG427wn4gcHJ9JRAI9pPrjT9YCfuBCO6ksnWsw9gBRKFf6pQyHYH9JWSON9uBdLcthyuxE4zE8EdO2xnze+b3ZCvoKeEPf6Bx4/eEwKZW5kc3RyZWFtCmVuZG9iagoyNCAwIG9iago8PCAvTGVuZ3RoIDI0NyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtkUtuBDEIRPc+BReIZD42+DwTRVnM3H+bhzsLCxooqoqOCJliS76OhIXEnvKtI2aJ7pDP6KKulPfw6ra4Rg+bXsxr6NqdqE6e5JZlVMPkWO/VyA6mSbWTKoHT2VQmfnpYbUp4iua8AsztRgDnydzrTngqGKXjhyoKY1KxTWR3NsnJm7zHQaWjYLoYTycqq7GKdl18gVPnBV1s0WE/WMM+fVvRARm1O2nrGxFAylvDKoHEMaJXwnO01/htbofvMxzmzt54AbqUCxALl0nn1D0Lx0eT4bDvbXjp2Hqeii5ngogfP7ej/IqkMp9j/hM2988fp3dXUwplbmRzdHJlYW0KZW5kb2JqCjI1IDAgb2JqCjw8IC9MZW5ndGggMjM4IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nE1RSXIEIQy78wo9wQs28J5JpXKY/P8a2Ux15dBIWGCpjbtDYMLFtyP5fekwxQzB71DxZu/LjkAlEdRVFTEdr6EaiAyoTcTeUN9IqlQykXFgejvbOo2v4XKZ0z7XggeRPagsMu46jZ+LVt2a8Z5XF2HexQTHSkmn8+wusalMbaRicSsqfdYOE2lSMVZzspJGv4BNb6Qi0Uz3ovPFyLpT7E6j8v9jumtCpT5MZX4Yryqty4sz9ApcAzHu+X8MU1E0Ppmqpz3A8nsEGyyszfPgfL0GeDYYuB4lV0P5ve4rdYZ6wod00p/x/Qf6/1xbCmVuZHN0cmVhbQplbmRvYmoKMjYgMCBvYmoKPDwgL0xlbmd0aCAyOTMgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPVJLbgUhDNtzCl+gEvkQwnleVXUxvf+2Nq/qhngIsRNn1pyYCMOHBTIa1Y1PG29o+Bk5HesUnhGnsXoj+jAGYq8bX8P4UugZlobku2e4JbITXhtZjaBSLufrCKdUIYqSk5lj5JrMJGuCyukFP+sdUwpCphtr2CIzuzK/bF1gQ1GO4iROvsVrb0fAWabJbPWNbNXnRSz52Ch2cdkhZaOozmzxGjb7I3lh35PFnIHAPMHZbU0saRnzQZFz4NI0BbbApL5ZkfL4FpFYapNCi0zZ73juKES+xEVzJ7HmSmWc3QQ3YxwxzdWQwmvQWIGiI+RXcBlZ8m5pH5u3tIbb447+EMWEcupGO5AR+/ofKG4iOUJt7o1+6Id4/f8Pz/geX78Qp2k6CmVuZHN0cmVhbQplbmRvYmoKMjcgMCBvYmoKPDwgL0xlbmd0aCA5MSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNjDsOwDAIQ/ecgiMACaDep6o6NPdfy0dpuuAn21gQAYHYz7AOagontUBSg9m6YdKThK588KeV0A+4U3RZc6q+NlUa65vq9c69cGeS2HBX1uAKo3i9EGIioAplbmRzdHJlYW0KZW5kb2JqCjI4IDAgb2JqCjw8IC9MZW5ndGggNTYxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVTS44lMQjb1ym4QKTwye88NWrNot/9t2M786TuB0USwNiM3q1bbGvhy2p1q9ntjz90c2/7PLmXvN8nz7Ecx6pPy/Rre9n70IsRuhdeuFXm8N8n/ZjPQAlE6uBWl32fqLiRRATRcHwjx/v4SXiFir6OznxO3HX4OInFO4kTx1fiv7vscR2krYGbx/Ao3G2yw6xuQyDWtDHVe+EP0JWvUGUj4tZ8XlvsX95GpHgSBE0HfTDngHsGMpw7wOAhHfSBK3IduFr0ZQtTvkOpYpWjuTu6kI3OodBDkkTmHYammtrHT8PFg+/AIAPjYA0MHwARxZR3XiTAl2UrrZK/g8+XXNQ+eEhGIjUqPE9OdosJ8uRTJ+hDs2eBs1kh/Nzkgcmz9zlkDmvCSdC5BxSAZiglF5PIWkUtHAkrcsoCKGopwpOBd5w65ompJnjIRaoG8kAdCT4L2BK6qMsWhRq889Xp7/P3oXr4+vPfG1TQoPbAAngM4tyEsqWyI40GB7oKyhyy6G2tGwG1DhayH00BvUVcfccRFtmrL3jZt1HviTrBXdlDlAKpcz+IC+9BHi1Oeshz7EaBEV+U1xUPMpFW4pjLvsheoCwIvIGlz/Xqvm3axyF5vNrZBhZyUKElBLTcupRHRhpgFS+KcTgxtFONm4k+mlROOVKEGEADETSbPR5JlVuozYCMkrmK7FJFWhIOGVLHzFX3Uv+N3f0ZLtE0Li6YdcbwI3An7AuX0H/+AQty3fQKZW5kc3RyZWFtCmVuZG9iagoyOSAwIG9iago8PCAvTGVuZ3RoIDI4NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtUrtxxTAM6zUFR+Cf0jzO5V7h7N8GpF5hA7IFEoQUzMR08Pguym30I6sp09/SCuA7KAFi7CSbTI00yEzIgp5ldsixdqOsAWHt72DGXWFYMRnUzkmaZ/BZ6pepwQILqTjZabUc1NpKUuiYCQe3Nv6AdYd3GFpKShsQVB1DUtUG5UTbVR376KV3nvN935kgPxTRgtSrb3dOhXGy33cakMDzDjlO2JS8KbELnp/VVhRDYK2OpgPwkkOk111qAJ9BENv0Ty+QjVowIApBQRA653HHzc3fca2y2eYbGxu5Ylqo3HkCRcSBA0INT0TOF3EcO4f1fucaNET+rGY30GYI1Ps0ApCIvje4UqvikN6rMoHei/Kuz/r9BzplbO0KZW5kc3RyZWFtCmVuZG9iagozMCAwIG9iago8PCAvTGVuZ3RoIDIzNiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UDuSxSAM6zmFjoCNDfg82dnZInv/9kkkr0EKNvrE10aHTR6eA3MP/Fiz3TGH4/9htmArMXtyM5GVuJol2eZNONFgvg9yYvHcdD8b5e+T7UeDlpIU0ONqIjFB65kbRZhbMp2MM7M8uczrICfxJFWEs8FQ7xvFpMiJTc1vkav9NfXrqqSyt4DCd0vWZxiHDzD0kEF10KKkqtN66DI5NL4QyVCtsTr2xqAL2w2SwgqMCkw/cDWmEBn6pkk/S1cjiS5/RY55Eq1C5HyZvVOW5u+neASsGFCiVgUPuJ/wamb6wU/Fm2V/P7wcWJQKZW5kc3RyZWFtCmVuZG9iagozMSAwIG9iago8PCAvTGVuZ3RoIDE1NCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwtT0kOAzEIu+cVfkKAQOA9U1U9TP9/rZlUirADZrHuxIQUg/pGpOElo+nEd6gQ7gZxoitkYxt0gUJbuEYZ1kIl4omlnUt4CBuapKMcMQVZCBHW2Rs6sZXglD5wjXX+ShkXSIMx3cSK05J1Z8E4LoqnnHup6MpuSbPcf+LUZGC1v4D1Zp2cr3zeTtqm9Obj9x6f8f4BMKQzKwplbmRzdHJlYW0KZW5kb2JqCjMyIDAgb2JqCjw8IC9MZW5ndGggNDIxIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nD1TO3IFMQjr9xQcwebv8ySTSfFy/zYSzkuz0oIxCHBVy5Kd+FSFeC753A/pkp8nooCvwR0kHbJLcqmoSe4lpvLx5G7xLakK18BZNIPoScSRmSUiUiwdp1ysGRoHnnMkysRXS3gMwqP7WhYRXuS24/B4HkS7eBhuO+LWYl70gB0WSpKIcWVhHkiyJxIV4b839QTqByAVCHVq5z+OXtuj18wZZk49OG8B4TCHS+QAAmjGsVOMA7NCBNpDDdrnT5XWupZYo0pxoR0WobBao9EHqgrYVEfPLtwWrARp/fxhjwdzY8UksMxJRygKYT94ly7UvjljRVNGL8lE5RtGrTuRLcKt6Bhn2Gw+5cXht5kVyGG+hmDgfQuuW//Hw1i0gf3HosQeQPMvMQpH46l7NKA1kx574ah6Yxic246efcSJTIywplJkRDqyfrPD5YNgRbyFKDbBPd69Rt/03M02jn5uNG5bshBuj05eK87Q61rC7onAJt+YzBpWK0cAUZsesmJ5JMhXi9qkuOx7+jHPpXAlHkrNBGnGKxvh97m9nu/n6xekBKoQCmVuZHN0cmVhbQplbmRvYmoKMzMgMCBvYmoKPDwgL1R5cGUgL1hPYmplY3QgL1N1YnR5cGUgL0Zvcm0gL0JCb3ggWyAtOTcwIC00NDMgMjAwMCAxMDIzIF0gL0xlbmd0aCAzNwovRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJzjMjMyVDAyMlDI5TIzATNywAwLMxADJIdggSUzuNIA/9IKDQplbmRzdHJlYW0KZW5kb2JqCjM0IDAgb2JqCjw8IC9MZW5ndGggMjg1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVSuXXFMAzrNQVH4KljHuflpfjZvw1AOY0A8wRJl6qo2MSTuySnypcNUpXf4WsBP8AtViChLrYk7IinhINOeUYEUrdEqixrMC3awUKTiWT0oUzC53M1PsNLryUSaOKOyBP0wBpQYnsilyrREm2eQbYcZUkKkiaaQyLkhtKPDHdxDap1P1SPgiAcx/Y/9FDZUZjcQ1iIvdcUmLcKGuA9CRswYiGBZJqwGxazAr2pirlnSR3M4ZLVgCVYE+hJ7VXHuTNIdvsJh0OKbklKqegzMAKZtXpMbG6dZv4yV/SBVkcE40PnmxeZ1wJlmbO3mU5PWrxf2GTVReUFyXYvA6R2K8LJEucNDpGo6VcYdsX/ozd6f5TP+Bnff02QbOEKZW5kc3RyZWFtCmVuZG9iagozNSAwIG9iago8PCAvTGVuZ3RoIDIyNSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UTtuQCEM2zmFL1CJfIBwnldVXXr/tXZeO9kicWKHNScm/OLjIs9E7olPG6Iehp9mtg2ZFzYDOQ9W4RmxFywQZtJ6LsEzrELEwlW99ja32LXH1hW461nkHrAaq5vjbs2gOKNgtduQr2zkUntfYp/uyFl/muwh2SH8vmGU4BnfI4p8JtM0K+npPKgI8nJNVjZb8MOs3O+xGmmSJxKzQ1d2mIA+q7PSf/A2ZkT3Rj+hCpkVe2P3q7Rbie0mIhk/WYdvF+gQSsshR9XIAgPqxrqbLPITcuM/iEJ9/QL+YlIzCmVuZHN0cmVhbQplbmRvYmoKMzYgMCBvYmoKPDwgL0xlbmd0aCAxNDYgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicPY9RDsMwCEP/cwofAYfQhPN0mvbR3f93Jq0mRfiJCGPCDAaSqp4DxzzwYis0fBu5pNdWRkEMcOoluoPp8MDZuh0YE50uhy2524IYQ3Ob5N1NqimuW8/G+XRiIVaCXsr6UaqY3OtFq/b3tJ2xKEX+0L+lfG4BmTiVMeXj3pVUQWfdUKexwt03Xu3T3j8cmTFkCmVuZHN0cmVhbQplbmRvYmoKMzcgMCBvYmoKPDwgL0xlbmd0aCAzNDIgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicTVJJbsQwDLv7FfpAAGvxovekKHpI/38tKU+AXkaMLVMkNaN36TLkMp0Sq0vMLl/adKTE2PJbyLfK06yrhLtYZLV596p386HnJFEDJIabzpsYIY4bUnvPqhaLN0C6Av0pquxPmUGyiYMhrilXiuExyt3Mz7epXFOsLyG/pkO9riXQX1KdpwCXQsTzgVChIL84UBfs6q6+TZgGOnblKm0AmwMJGcvTxj+gmQBOspwynLXLNHIansEWxrjUlDFPdWoFcl94TABby8QB50K+VDMVAWIZzBGpRRUaL8BIYDbr925ZgUHRqLSVu0DyOt6dvBt82s/ZIQjPNvf+5AKVCrloYoxRw2DTkLIzaYTvLDjO+nbbGC8OiYsiHBa0Uz/1aVVuBjdEE5JpUq2eeO29OJCUIWT+Rwy2z3/FEE6dKK2DORkQV0GVDi3lI7a8fm54+/4DKHSDbAplbmRzdHJlYW0KZW5kb2JqCjM4IDAgb2JqCjw8IC9MZW5ndGggMjY1IC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nDVRO5YDMQjrfQqOYD4GfJ7s25cie/92JZIUYxgESIC7y5aDz/1I5JYfXQoTp+VvPM+U19IGvE1sp4SlmLlEbHksC2a72LnTwHqPBXJrPFdmHHGDjQTiDs/zTYpOY3cNgrzLiIq3oSbEK4hojRbfDQslN8eCp/0dKaptKMmppTZ2A0IuqDHw0gLRT2Qj0iXaMbUPzIz+0TKzQ8VYPYNsuZvLgJNEqF60IIxs2uAGmRISck8ZHVZ9XwRfy0GtAqFWkqBmYhnbVQp2j7c5dXFJgQI6SC+MWirU2Nxx4r+vHLS4wXIaLmscvTM+9F0m4xybxwxcCtBtHAGjn/M52Pfwr/Vcv/9G3mQ9CmVuZHN0cmVhbQplbmRvYmoKMzkgMCBvYmoKPDwgL0xlbmd0aCAzNDcgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicLZI7bgQxDEN7n0IXWMD6+HeeDYIUyf3bPHq2GWpkWyIp5T7Wbbi9jmVtq3Hsy5uPadnd/lrMbpHbfluctPBt6dP8cD2O+Q57tyw3n3ULeMWDPu7Jtu2WWTbI92mxSceijEVNtY2+Be/m6/47N1+0yAEzsr5smEOGUsf0fC1gHiubw15JRk8SjqAPBXN/Ajp5LzvcFakymlIUQpGQhjhPzhDAy4+CSABiY8D80u1Wy8QalYI9PyocM2SRY0HEuBWlhLrHac8nEILBsUuQqacKFpTSCnIb3+PSchLXxHUn4QgWitkzmyiowCsYkG5ygq7C9MCV4lR9hJyorjJHmcR+vdVJut/ot2mYiStEOPxEQddUr4V+CRjU6ZcFKivWXYPKBb8HYc4kbkbME/UYV/HMDw1dTuWtdnGej9oc63qYpRGwQhl3PCyLZqFl6lqJZx3f7ad9/wO4Q4AnCmVuZHN0cmVhbQplbmRvYmoKNDAgMCBvYmoKPDwgL0xlbmd0aCAxNyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJwzMjVQQMAUQ640AB2YA00KZW5kc3RyZWFtCmVuZG9iago0MSAwIG9iago8PCAvTGVuZ3RoIDE5MyAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw9kMsNwzAMQ++eggsUsGR97HlSFD20+19LOWkOCV/0iShpTnTIwGNBc8H5PKVpBDLxbRUjf5paQgLao0rFZ8nR1irNjjHqLZIMEkwmm/h1gcF0zzHVU5yFBYPhSQmQzIxhdxgHTHYvg2jCXapcJ2mwX6j0wyaPURljJOjLnCs4NfcqzLCzSNyodio3qgxNea/d9sCbTseVvUmG/qn3vX3WOQLndP7XOKIM6z4R5WiqG9TocuA66dHe7fUDAUlF8wplbmRzdHJlYW0KZW5kb2JqCjQyIDAgb2JqCjw8IC9MZW5ndGggMzIyIC9GaWx0ZXIgL0ZsYXRlRGVjb2RlID4+CnN0cmVhbQp4nCWSS24EIQxE95zCFxgJf4HzdBRlkdx/m2d6MVNu/ClXQc4pU0Lloy7hKrVKvnSUSuqUvxFJEPI7akpWyA6p6aK6pNzkGRpHinI9b7PFi8+wvW7kllS4eNLrm4xvk1ITP0tynRf9dIYo4PAq1tniMySmdiam+E4ynNYi5hc9LRQOzbu+LXtRe1pHWmTIKifdvy5NUcYcO8JyvShi2Ri1H74xpf3Q9IvPWPs1CFf6L/zSsrPQwSyC6rn4sKCYzYLGy3V2Y9NzrrRmdyuL4p+hzYBepN00tu3vg6R2Di/rii/po2QQXRchIpGtQKnEM9a2xCG7WpwTqyvGfV4xjVB3FH2nesglNbqvr+bt6+3mNgNPnEz0TLDfAYwdBXUTRITtI3luD/pqLl4A96w9dV5sxjdSdsnd8X1dz/gZ3/9H9XbhCmVuZHN0cmVhbQplbmRvYmoKNDMgMCBvYmoKPDwgL0xlbmd0aCAyMjkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVG7ccUwDOs1BUbgX+I8L5dL4ezfBpQvhQ1YJCGAThEIrPmKHahd+NI1VH3jd4VN/Vm344LaoGyY6LB0WDTs8KQDzop74rPcyHbyayN0v1g6FbIUViyQIZwRZM2MsT8P1UJRYtA+Fz9Ly94Td2QT6SNp87NakZnYBOOjDDIDPok2HVIqvVG8ix5H0HwEnYJ5oLtvbPNzkSbaL/M5yWYIdtpUQupeHlTIlBdpnhWyYBw/jBPGVSQsr3EaodiznA5ecmpIiIPdoQUmHolGc0XB3WtyVQKNcfr/N571s77/AG3XURYKZW5kc3RyZWFtCmVuZG9iago0NCAwIG9iago8PCAvTGVuZ3RoIDMzOCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxVUjluBDEM6+cV+sACum2/Z4MgRfL/NqRmF0GKgTiyJVKUl7uouMnDQvqkZKl82PWGPzeKku+rN5FJr5b0RnRElefVvSQtpdskFTcqJLbhpKxkigvpkDw+TLnXxOeVO14ZRVTJteXRPADLAwTL5Eh2iB9kA5zebOgN3QNyc4Q408fBdnd2f2Ug0cT1SFDrSmETjANwtlSN9AQJvAAvh8JdSMIHZzg8utn4ATr9h2Y42/dtg1gWGxSms8xqj0c2NH5H9TmBM4eZBa8Q4R2FPWcyM0PfKOSMXAHfo5sISlgX6JY4CwW30RvnpnyJw8rR3v4WTzSak8r+0KjPtNkrd8r6NGqe5XAh/NM1XBOVLhLFPjNTNL2Dm8mT0g31VFpwN2LTapoP9c3XsPfEVCobxKeHeTlFa72465xRU3gqVFfrvYpBM8P7lX5fX9fnL5aYhV8KZW5kc3RyZWFtCmVuZG9iago0NSAwIG9iago8PCAvTGVuZ3RoIDMxOCAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJxNkktuBCEMRPecwhcYyT9oOM9EURad+2/zDD1SFj0uqLIp29NVRcVSXm6X5NUlu8qXtQ/8PSi63C1HgPg6ny/JHESXd8uYklTJMEl1brvETBgn++VJtueS10pxMl6WKbYG4ApUdikqEo1C24qpHvBu13M1rCIPrNIqTFSBuSS0CvUT+yqmEPaCzgzzWW/6gOnFPNXMQnh/x3dbfpDpOhKjiyfLomRRoyoVJ+t2AOToz93cRae4AmYZceM9rLlRtA+6p5uomXm5R5om55c8bgFeMhzy2lpiVnUG26Ho3TqTduXAsF32zLfB3OtizMj/ob04W2QzCpt+si7UPrf1WiHMgKmWK+phCmEBRUyYmu+ofXid0N6tVhwrNkKF2ZjY034iu2IThegmLt9uIk+EKbSdfv5sd/tp339JN3oOCmVuZHN0cmVhbQplbmRvYmoKNDYgMCBvYmoKPDwgL0xlbmd0aCAyNzkgL0ZpbHRlciAvRmxhdGVEZWNvZGUgPj4Kc3RyZWFtCnicNVBLTgUxDNvPKXwBpObb9jwPITbcf4udB9KM4qZuYrvWwoInPiyQu9H8P+0RjFj4GeSneGnkkdQBX4HXk5UwnjIN5hvJMTd1sTZ6IU4i+IlWbIfW8PG5s81rTX09vt4d2yGCpSD7Zo7YOAdd2A6j2NdTBePYIRmnw800ZM9E7vDrKhGtNsE9CEOWHKJGezuK58MzJZsH2mdlXS67dOgThVO8KtXbHRQU0sn9lqizJodCWU98GY7/7F7P9xNkhBdzFEp2ow/ZrNmo7r9c2qn5GnfLwpmqYHqQ7RqGUfL7jdGQpphrqk0Nv7ohcm+IYfSrN1eWNSTkjD2frFi4g2kp5wlqVA1bMvVcsp0G/o3I1NcvehhongplbmRzdHJlYW0KZW5kb2JqCjE1IDAgb2JqCjw8IC9UeXBlIC9Gb250IC9CYXNlRm9udCAvQ1pSQkxSK1NUSVhHZW5lcmFsLVJlZ3VsYXIgL0ZpcnN0Q2hhciAwCi9MYXN0Q2hhciAyNTUgL0ZvbnREZXNjcmlwdG9yIDE0IDAgUiAvU3VidHlwZSAvVHlwZTMKL05hbWUgL0NaUkJMUitTVElYR2VuZXJhbC1SZWd1bGFyIC9Gb250QkJveCBbIC05NzAgLTQ0MyAyMDAwIDEwMjMgXQovRm9udE1hdHJpeCBbIDAuMDAxIDAgMCAwLjAwMSAwIDAgXSAvQ2hhclByb2NzIDE2IDAgUgovRW5jb2RpbmcgPDwgL1R5cGUgL0VuY29kaW5nCi9EaWZmZXJlbmNlcyBbIDMyIC9zcGFjZSA0OCAvemVybyAvb25lIC90d28gL3RocmVlIC9mb3VyIC9maXZlIDY1IC9BIDY5IC9FIDczIC9JIDgzIC9TCjk3IC9hIDk5IC9jIC9kIC9lIC9mIC9nIC9oIC9pIDEwOCAvbCAvbSAvbiAvbyAvcCAxMTQgL3IgL3MgL3QgMTE5IC93IDEyMSAveQpdCj4+Ci9XaWR0aHMgMTMgMCBSID4+CmVuZG9iagoxNCAwIG9iago8PCAvVHlwZSAvRm9udERlc2NyaXB0b3IgL0ZvbnROYW1lIC9DWlJCTFIrU1RJWEdlbmVyYWwtUmVndWxhciAvRmxhZ3MgMzIKL0ZvbnRCQm94IFsgLTk3MCAtNDQzIDIwMDAgMTAyMyBdIC9Bc2NlbnQgMTA1NSAvRGVzY2VudCAtNDU1IC9DYXBIZWlnaHQgMAovWEhlaWdodCAwIC9JdGFsaWNBbmdsZSAwIC9TdGVtViAwIC9NYXhXaWR0aCAxMTA5ID4+CmVuZG9iagoxMyAwIG9iagpbIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwCjI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDMzMyA0MDggNTAwIDUwMAo3NDcgNzc4IDE4MCAzMzMgMzMzIDUwMCA2ODUgMjUwIDMzMyAyNTAgMjc4IDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAKNTAwIDUwMCAyNzggMjc4IDY4NSA2ODUgNjg1IDQ0NCA5MjEgNzIyIDY2NyA2NjcgNzIyIDYxMSA1NTYgNzIyIDcyMiAzMzMgMzczCjcyMiA2MTEgODg5IDcyMiA3MjIgNTU3IDcyMiA2NjcgNTU2IDYxMSA3MjIgNzIyIDk0NCA3MjIgNzIyIDYxMiAzMzMgMjc4IDMzMwo0NjkgNTAwIDMzMyA0NDQgNTAwIDQ0NCA1MDAgNDQ0IDMzMyA1MDAgNTAwIDI3OCAyNzggNTAwIDI3OCA3NzggNTAwIDUwMCA1MDAKNTAwIDMzMyAzODkgMjc4IDUwMCA1MDAgNzIyIDUwMCA1MDAgNDQ0IDQ4MCAyMDAgNDgwIDU0MSAyNTAgNTAwIDI1MCAzMzMgNDM0CjQ0NCAxMDAwIDUwMCA1MDAgMzMzIDExMDkgNTU2IDMzMyA4ODkgMjUwIDYxMiAyNTAgMjUwIDMzMyAzMzMgNDQ0IDQ0NCA1MjMKNTAwIDEwMDAgMzMzIDk4MCAzODkgMzMzIDcyMiAyNTAgNDQ0IDcyMiAyNTAgMzMwIDUwMCA1MDAgNTAwIDUwMCAyMDAgNTAwCjMzMyA3NjAgMjc2IDUwMCA2MDAgMzMzIDc2MCAzMzMgNDAwIDY4NSAzMDAgMzAwIDMzMyA1MDAgNTkyIDI1MCAzMzMgMzAwIDMxMAo1MDAgNzUwIDc1MCA3NTAgNDQ0IDcyMiA3MjIgNzIyIDcyMiA3MjIgNzIyIDg4OSA2NjcgNjExIDYxMSA2MTEgNjExIDMzMyAzMzMKMzMzIDMzMyA3MjIgNzIyIDcyMiA3MjIgNzIyIDcyMiA3MjIgNjQwIDcyMiA3MjIgNzIyIDcyMiA3MjIgNzIyIDU1NiA1MDAgNDQ0CjQ0NCA0NDQgNDQ0IDQ0NCA0NDQgNjY3IDQ0NCA0NDQgNDQ0IDQ0NCA0NDQgMjc4IDI3OCAyNzggMjc4IDUwMCA1MDAgNTAwIDUwMAo1MDAgNTAwIDUwMCA1NjQgNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMCBdCmVuZG9iagoxNiAwIG9iago8PCAvQSAxNyAwIFIgL0UgMTggMCBSIC9JIDE5IDAgUiAvUyAyMCAwIFIgL2EgMjEgMCBSIC9jIDIyIDAgUiAvZCAyMyAwIFIKL2UgMjQgMCBSIC9mIDI1IDAgUiAvZml2ZSAyNiAwIFIgL2ZvdXIgMjcgMCBSIC9nIDI4IDAgUiAvaCAyOSAwIFIgL2kgMzAgMCBSCi9sIDMxIDAgUiAvbSAzMiAwIFIgL24gMzQgMCBSIC9vIDM1IDAgUiAvb25lIDM2IDAgUiAvcCAzNyAwIFIgL3IgMzggMCBSCi9zIDM5IDAgUiAvc3BhY2UgNDAgMCBSIC90IDQxIDAgUiAvdGhyZWUgNDIgMCBSIC90d28gNDMgMCBSIC93IDQ0IDAgUgoveSA0NSAwIFIgL3plcm8gNDYgMCBSID4+CmVuZG9iago1MSAwIG9iago8PCAvTGVuZ3RoIDM4OSAvRmlsdGVyIC9GbGF0ZURlY29kZSA+PgpzdHJlYW0KeJw1UkeOG0EMvPcr6gMCmqnDe2Qs9iD//+oqjgwIIMXQrDBrbUxY4WUbKw8yDX9s5AykL/wdGYWd+IyaxsGFyoJborjqefEedS78JNacCPMnRrLTWQbqOqIKtQ9i9c5KxHFUbcRdHdOmOp0dvj8JgTuLqMLU2YvZRp1qnMU9Rd7hprLlhE3AK6J32eH1JNrmRgaKcZ4OMyePTR5TNxaxsZHOQpGxRHgFx0/LI7iK7xEE0pXNgWKgMl1P0uT2Z3gFS3rECdgok9uF2YLdg60ztooN/rLfssjv6+b3qbiucIJgXqcbPOf87+BbbnApaT06ecqMJdZsEgYVuRR/OUfIN84Ef3FNbkpxKvBIUgoJCTVlGr8GJZ9OTGzotNS8t40wid32mkVngiDZhUkGCSp340BcMggpnAbJXvFUZpyTZcbuY6KU6Iqs0k7NNkWIuM0XrDhBEawh9x2RoeJGik7TjZylG7042pEscdZj7p19OW59seR8Kv21R+H/V/8ev+PnH5IAk4kKZW5kc3RyZWFtCmVuZG9iago0OSAwIG9iago8PCAvVHlwZSAvRm9udCAvQmFzZUZvbnQgL0ZUWFdaQStTVElYR2VuZXJhbC1JdGFsaWMgL0ZpcnN0Q2hhciAwCi9MYXN0Q2hhciAyNTUgL0ZvbnREZXNjcmlwdG9yIDQ4IDAgUiAvU3VidHlwZSAvVHlwZTMKL05hbWUgL0ZUWFdaQStTVElYR2VuZXJhbC1JdGFsaWMgL0ZvbnRCQm94IFsgLTk3MCAtMzA1IDE0MjkgMTAyMyBdCi9Gb250TWF0cml4IFsgMC4wMDEgMCAwIDAuMDAxIDAgMCBdIC9DaGFyUHJvY3MgNTAgMCBSCi9FbmNvZGluZyA8PCAvVHlwZSAvRW5jb2RpbmcgL0RpZmZlcmVuY2VzIFsgMTE5IC93IF0gPj4gL1dpZHRocyA0NyAwIFIgPj4KZW5kb2JqCjQ4IDAgb2JqCjw8IC9UeXBlIC9Gb250RGVzY3JpcHRvciAvRm9udE5hbWUgL0ZUWFdaQStTVElYR2VuZXJhbC1JdGFsaWMgL0ZsYWdzIDk2Ci9Gb250QkJveCBbIC05NzAgLTMwNSAxNDI5IDEwMjMgXSAvQXNjZW50IDEwNTUgL0Rlc2NlbnQgLTQ1NSAvQ2FwSGVpZ2h0IDAKL1hIZWlnaHQgMCAvSXRhbGljQW5nbGUgNDM5MDkgL1N0ZW1WIDAgL01heFdpZHRoIDExMTcgPj4KZW5kb2JqCjQ3IDAgb2JqClsgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAKMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMjUwIDI1MCAyNTAgMzMzIDQyMCA1MDEgNTAwCjc1NSA3NzggMjE0IDMzMyAzMzMgNTAwIDY3NSAyNTAgMzMzIDI1MCAyNzggNTAwIDUwMCA1MDAgNTAwIDUwMCA1MDAgNTAwIDUwMAo1MDAgNTAwIDMzMyAzMzMgNjc1IDY3NSA2NzUgNTAwIDkyMCA2MTEgNjExIDY2NyA3MjIgNjExIDYxMSA3MjIgNzIyIDMzMyA0NDQKNjY3IDU1NiA4MzMgNjY3IDcyMiA2MTEgNzIyIDYxMSA1MDAgNTU2IDcyMiA2MTEgODMzIDYxMSA1NTYgNTU2IDM4OSAyNzggMzg5CjQyMiA1MDAgMzMzIDUwMSA1MDAgNDQ0IDUwMCA0NDQgMjc4IDUwMCA1MDAgMjc4IDI3OCA0NDQgMjc4IDcyMiA1MDAgNTAwIDUwNAo1MDAgMzg5IDM4OSAyNzggNTAwIDQ0NCA2NjcgNDQ0IDQ0NCAzODkgNDAwIDI3NSA0MDAgNTQxIDI1MCA1MDAgMjUwIDMzMyA0NzIKNTU2IDg4OSA1MDAgNTAwIDMzMyAxMTE3IDUwMCAzMzMgOTQ0IDI1MCA1NTYgMjUwIDI1MCAzMzMgMzMzIDU1NiA1NTYgNTIzCjUwMCA4ODkgMzMzIDk4MCAzODkgMzMzIDY2NyAyNTAgMzg5IDU1NiAyNTAgMzg5IDUwMCA1MDAgNTAwIDUwMCAyNzUgNTAwIDMzMwo3NjAgMjc2IDUwMCA2NzUgMzMzIDc2MCAzMzMgNDAwIDY3NSAzMDAgMzAwIDMzMyA1MDAgNTU5IDI1MCAzMzMgMzAwIDMxMCA1MDAKNzUwIDc1MCA3NTAgNTAwIDYxMSA2MTEgNjExIDYxMSA2MTEgNjExIDg4OSA2NjcgNjExIDYxMSA2MTEgNjExIDMzMyAzMzMgMzMzCjMzMyA3MjIgNjY3IDcyMiA3MjIgNzIyIDcyMiA3MjIgNjc1IDcyMiA3MjIgNzIyIDcyMiA3MjIgNTU2IDYxMSA1MDAgNTAxIDUwMQo1MDEgNTAxIDUwMSA1MDEgNjY3IDQ0NCA0NDQgNDQ0IDQ0NCA0NDQgMjc4IDI3OCAyNzggMjc4IDUwMCA1MDAgNTAwIDUwMCA1MDAKNTAwIDUwMCA2NzUgNTAwIDUwMCA1MDAgNTAwIDUwMCA0NDQgNTAwIDQ0NCBdCmVuZG9iago1MCAwIG9iago8PCAvdyA1MSAwIFIgPj4KZW5kb2JqCjMgMCBvYmoKPDwgL0YxIDE1IDAgUiAvRjIgNDkgMCBSID4+CmVuZG9iago0IDAgb2JqCjw8IC9BMSA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAwIC9jYSAxID4+Ci9BMiA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAxIC9jYSAxID4+Ci9BMyA8PCAvVHlwZSAvRXh0R1N0YXRlIC9DQSAwLjQgL2NhIDAuNCA+PiA+PgplbmRvYmoKNSAwIG9iago8PCA+PgplbmRvYmoKNiAwIG9iago8PCA+PgplbmRvYmoKNyAwIG9iago8PCAvRjEtU1RJWEdlbmVyYWwtbWludXMgMzMgMCBSID4+CmVuZG9iagoyIDAgb2JqCjw8IC9UeXBlIC9QYWdlcyAvS2lkcyBbIDExIDAgUiBdIC9Db3VudCAxID4+CmVuZG9iago1MiAwIG9iago8PCAvQ3JlYXRvciAoTWF0cGxvdGxpYiB2My44LjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcpCi9Qcm9kdWNlciAoTWF0cGxvdGxpYiBwZGYgYmFja2VuZCB2My44LjApCi9DcmVhdGlvbkRhdGUgKEQ6MjAyMzEwMTEyMjIwMzMrMDEnMDAnKSA+PgplbmRvYmoKeHJlZgowIDUzCjAwMDAwMDAwMDAgNjU1MzUgZiAKMDAwMDAwMDAxNiAwMDAwMCBuIAowMDAwMDE2OTU4IDAwMDAwIG4gCjAwMDAwMTY2ODEgMDAwMDAgbiAKMDAwMDAxNjcyNCAwMDAwMCBuIAowMDAwMDE2ODY2IDAwMDAwIG4gCjAwMDAwMTY4ODcgMDAwMDAgbiAKMDAwMDAxNjkwOCAwMDAwMCBuIAowMDAwMDAwMDY1IDAwMDAwIG4gCjAwMDAwMDAzNDIgMDAwMDAgbiAKMDAwMDAwMjM0NiAwMDAwMCBuIAowMDAwMDAwMjA4IDAwMDAwIG4gCjAwMDAwMDIzMjUgMDAwMDAgbiAKMDAwMDAxMzIxNCAwMDAwMCBuIAowMDAwMDEyOTk4IDAwMDAwIG4gCjAwMDAwMTI1MjYgMDAwMDAgbiAKMDAwMDAxNDI2MSAwMDAwMCBuIAowMDAwMDAyMzY2IDAwMDAwIG4gCjAwMDAwMDI2NTMgMDAwMDAgbiAKMDAwMDAwMzAxNiAwMDAwMCBuIAowMDAwMDAzMjUxIDAwMDAwIG4gCjAwMDAwMDM3MTggMDAwMDAgbiAKMDAwMDAwNDE5MyAwMDAwMCBuIAowMDAwMDA0NTM0IDAwMDAwIG4gCjAwMDAwMDQ5MjIgMDAwMDAgbiAKMDAwMDAwNTI0MiAwMDAwMCBuIAowMDAwMDA1NTUzIDAwMDAwIG4gCjAwMDAwMDU5MTkgMDAwMDAgbiAKMDAwMDAwNjA4MiAwMDAwMCBuIAowMDAwMDA2NzE2IDAwMDAwIG4gCjAwMDAwMDcwNzMgMDAwMDAgbiAKMDAwMDAwNzM4MiAwMDAwMCBuIAowMDAwMDA3NjA5IDAwMDAwIG4gCjAwMDAwMDgxMDMgMDAwMDAgbiAKMDAwMDAwODI3MiAwMDAwMCBuIAowMDAwMDA4NjMwIDAwMDAwIG4gCjAwMDAwMDg5MjggMDAwMDAgbiAKMDAwMDAwOTE0NyAwMDAwMCBuIAowMDAwMDA5NTYyIDAwMDAwIG4gCjAwMDAwMDk5MDAgMDAwMDAgbiAKMDAwMDAxMDMyMCAwMDAwMCBuIAowMDAwMDEwNDA5IDAwMDAwIG4gCjAwMDAwMTA2NzUgMDAwMDAgbiAKMDAwMDAxMTA3MCAwMDAwMCBuIAowMDAwMDExMzcyIDAwMDAwIG4gCjAwMDAwMTE3ODMgMDAwMDAgbiAKMDAwMDAxMjE3NCAwMDAwMCBuIAowMDAwMDE1NjA0IDAwMDAwIG4gCjAwMDAwMTUzODUgMDAwMDAgbiAKMDAwMDAxNTA1NiAwMDAwMCBuIAowMDAwMDE2NjQ5IDAwMDAwIG4gCjAwMDAwMTQ1OTQgMDAwMDAgbiAKMDAwMDAxNzAxOCAwMDAwMCBuIAp0cmFpbGVyCjw8IC9TaXplIDUzIC9Sb290IDEgMCBSIC9JbmZvIDUyIDAgUiA+PgpzdGFydHhyZWYKMTcxNzUKJSVFT0YK", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2023-10-11T22:20:32.838807\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.8.0, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Plot distributions and samples\n", "plt.figure(figsize=(8, 3))\n", "\n", "# Plot importance weights\n", "plt.hist(\n", " tf.math.exp(log_w),\n", " density=False,\n", " color=\"tab:blue\",\n", " alpha=0.4,\n", " bins=10**tf.linspace(-5., 1.0, 30),\n", " zorder=2,\n", ")\n", "\n", "plt.title(\"Empirical density of AIS importance weights\", fontsize=22)\n", "\n", "plt.xlabel(\"Importance weight $w$\", fontsize=18)\n", "plt.ylabel(\"Empirical density\", fontsize=18)\n", "\n", "plt.xticks(fontsize=18)\n", "plt.yticks([0, 20, 40], fontsize=18)\n", "\n", "plt.xscale(\"log\")\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "As expected, most of the AIS importance weights are close to $1$.\n", "Further, they have a clearly smaller variance than standard importance sampling.\n", "This has a significant impact on the variance of AIS estimates.\n", "In the running example integral we've been using thus far, AIS reduces the variance of our Monte Carlo estimate by a factor of $\\approx 6$ over standard importance sampling, and gets close to the variance of an estimator that uses direct i.i.d. samples from $p$:" ] }, { "cell_type": "code", "execution_count": 63, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "remove-input" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Annealed Importance Sampling over 100 trials I = 0.02 +/- 2.198.\n" ] } ], "source": [ "# How many samples to draw\n", "num_samples = 100\n", "num_trials = 100\n", " \n", "# Draw samples\n", "samples_and_log_weights = [sampler(num_samples=num_samples)[1] for _ in range(num_trials)]\n", "samples_and_log_weights = tf.stack(samples_and_log_weights, axis=0)\n", "\n", "samples = samples_and_log_weights[:, :, 0]\n", "log_weights = samples_and_log_weights[:, :, 1]\n", "\n", "w = tf.math.exp(log_weights)\n", "\n", "# Compute importance weights and importance weighted integral\n", "I = tf.reduce_mean(w * f(samples), axis=1)\n", "\n", "mean_I = tf.reduce_mean(I)\n", "std_I = tf.math.reduce_std(I)\n", "\n", "print(\n", " f\"Annealed Importance Sampling over {num_trials} trials \"\n", " f\"I = {mean_I:.2f} +/- {2.*std_I:.3f}.\"\n", ")" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Conclusion\n", "\n", "In importance sampling, using a proposal distribution that is significantly different from the target distribution results in a large variance of the resulting importance weights.\n", "Large variance in the importance weights typically induces large variance in downstream estimates obtained using these weights.\n", "AIS is a method which helps address this issue.\n", "AIS introduces a sequence of annealed distributions, which interpolate between the target and proposal distribution, iteratively transforming samples from the proposal distribution using a sequence of Markov transition kernels which preserve the interpolating (annealed) distributions.\n", "In this way, AIS typically reduces the variance in the importance weights, which in turn reduces the variance in downstream Monte Carlo estimates.\n", "For more details, Radford Neal's original paper introducing AIS{cite}`neal2001annealed` is a classic worth reading.\n", "\n", "## References\n", "\n", ":::{bibliography}\n", ":filter: docname in docnames\n", "\n", ":::" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.4" } }, "nbformat": 4, "nbformat_minor": 4 }