{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Modules Imported!\n" ] } ], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import scipy as sp\n", "import scipy.stats as st\n", "import scipy.linalg as la\n", "from math import sqrt\n", "print(\"Modules Imported!\")" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# plot helper function for plotting a dataset along with the boundaries of classifiers\n", "def classPlot(thetas):\n", " # thetas = list of vectors theta, where each theta has three elements\n", " colors = ['k','g','m','y']\n", " fig, ax = plt.subplots()\n", " ax.scatter(X0[:,0],X0[:,1],color='b',alpha=.5) # X0: x-coordinates for class 0 datapoints\n", " ax.scatter(X1[:,0],X1[:,1],color='r',alpha=.5) # X1: x-coordinates for class 1 datapoints\n", " ylim = ax.get_ylim()\n", " x0_range = np.array(ax.get_xlim())\n", " for i in range(len(thetas)):\n", " theta = thetas[i]\n", " x1_range = [(-theta[2] - theta[0]*a)/theta[1] for a in x0_range]\n", " ax.plot(x0_range,x1_range,colors[i])\n", " ax.set_ylim(ylim);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Binary Classification with Gaussian Class-Conditionals\n", "(25 pts) Build and plot a classiffier assuming that the parameters of the generative model are known. First, we will generate the data for each class by drawing random samples from a multivariate normal distribution using the following parameters:\n", "\n", "m0 = [0,0] # mean of class 0\n", "m1 = [2,2] # mean of class 1\n", "K = [[1,0],[0,1]] # Covariance matrix for both classes\n", "N = 250 # number of data points\n", "pi = 1/2 # the probability that a data point belongs to class 1\n", "\n", "Next, assuming that the parameters for the generative model are known, we build a binary classifier using Gaussian Class-Conditionals. To be consistent with following problems, formulate your classifier as:\n", "\n", "$$\\theta_0 x_0 + \\theta_1 x_1 + \\theta_2 \\gtrless 0.$$\n", "\n", "We then plot the data points and the line indicating the decision boundary in one plot, using the plot helper function classPlot defined above. The input for this function is a list of vectors representing classifiers, each of the form theta=[$\\theta_0$,$\\theta_1$,$\\theta_2$]." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl8E+X2/z9PSzeWLkDZKbKIQtOFTVm1CAgqClTRr1cE\nLKJUEVCqXjatV6teROUCAgIioOiVy6+Cooggu7SyllBkB6kUgaqlrG2lPL8/DmNDyTJJJplJet6v\nV15p0snMmZnkPOc52yOklGAYhmH8hwC9BWAYhmG0hRU7wzCMn8GKnWEYxs9gxc4wDONnsGJnGIbx\nM1ixMwzD+Bms2BmGYfwMVuwMwzB+Bit2hmEYP6OKHgetXbu2vOmmmzTZV1lZGfLy8vDnn3+iatWq\naNq0KUJDQzXZN8MwjJHYsWPH71LKaEfb6aLYb7rpJmzfvl3TfS5duhSpqak4fPgwMjIyMGbMGAQG\nBmp6DIZhGD0RQhxXs53fuGIeeugh5Obmok+fPkhLS0P37t1x5MgRvcViGIbxOn6j2AGgbt26+PLL\nL7Fw4UKYzWYkJCRg9uzZ4EZnDMNUJvxKsQOAEAKDBw/Gnj170LlzZ6SmpqJPnz44ceKE3qIxDMN4\nBb9T7AqNGzfGqlWrMHPmTGzevBkmkwmffPIJW+8Mw/g9fqvYAbLeU1NTYTabERcXh8GDByM5ORln\nzpzRWzSGYRiP4deKXaF58+ZYv349pkyZgpUrVyI2NhaZmZl6i8UwDOMRKoViB4DAwECMHTsWO3bs\nQJMmTfDggw9i0KBBKCws1Fs0hmEYTak0il0hNjYWWVlZeO211/DFF1/AZDLhu+++01sshmEYzah0\nih0AgoKC8MorryA7OxuRkZG45557MGLECJw/f15v0RjGu5jNQHo6kJJCz2az3hIxGlApFbtCu3bt\nsGPHDrz44ouYM2cOEhISsHHjRr3FYhjvYDYDU6YAhYVAo0b0PGUKK3c/oFIrdgAIDQ3F5MmTsWnT\nJgQEBCApKQkvvPACLl++rLdoDONZMjOBqCh6BASU/82JBT5PpVfsCl26dMHu3buRmpqK999/H23b\ntsW2bdv0FothPEdeHhARcf17ERH0PuPTsGK3oFq1avjggw/w/fff48KFC+jUqRMmTZqE0tJSvUVj\nnIH9xuqIiQGKiq5/r6iI3md8GlbsVujVqxf27NmDQYMG4Y033sDtt9+OPXv26C0Wowb2G6snOZmu\nT2EhcPVq+d/JyXpLxrgJK3YbREZGYsGCBVi2bBlOnjyJdu3a4e2330ZZWZneojH2YL+xeuLjgbQ0\nuj4nTtBzWhr9j2c8Po1mil0IESiE2CWEWKHVPo1Av379sHfvXvTr1w/jxo1D165dcfDgQb3FYmzB\nfmPniI8n5T1/Pj0DPOPxA7S02EcD2Kfh/gxD7dq1sWTJEnz22Wc4cOAAEhMTMX36dFy9elVv0ZiK\nsN/YPYw64+G4iVNootiFEI0A3Adgnhb7MyJCCDz66KPIzc1FUlISRo0ahV69euH4cVULmjDegv3G\n7mHEGQ/HTZxGK4t9KoCXAPi9CdugQQN88803mDt3LrZu3Yq4uDh8/PHH3A7YKNjyG8fH6y2Zb2DE\nGY9RZxEGxm3FLoToC+CMlHKHg+2eEkJsF0JsLygocPewuiKEwJNPPgmz2Yy2bdsiJSUFDzzwAH77\n7Te9RWOAG/3GrNTVY8QZjxFnEQZHC4u9C4AHhBC/APgvgLuEEJ9W3EhKOUdK2V5K2T462uEi2z5B\n06ZNsXbtWkydOhVr1qyByWTCkiVL9BaLYVzH2RmPN3zfRpxFGByhpQtBCJEEIE1K2dfedu3bt5fb\nt2/X7LhGYP/+/RgyZAi2bt2KRx55BB988AFq1aqlt1gM4zkU33dUFFnQRUVk3Wvt+vLWcXwAIcQO\nKWV7R9txHrtG3Hrrrfjxxx+RkZGBzMxMmEwmrFjhV5mfDHM9Wvq+7Vn+HDdxGk0tdrX4o8Vuye7d\nuzF48GCYzWakpKTg/fffR3h4uN5iMYy2pKRQlkqAhX149Sop3/nz1e+HLXLVsMWuIwkJCdi6dSvG\njx+PBQsWIC4uDmvXrtVbLIbRlpgY4PBhYP16YPlyej582HnfN2e9aA4rdg8REhKCjIwMbNmyBaGh\noejRowdGjRqFS5cu6S0aw2iDyQRkZwNnzwI1atBzdja97wyc9aI5rNg9zO23345du3Zh9OjRmD59\nOhITE5GVlaW3WAzjPrm5QKdOQGQkcP48PXfqRO87A2e9aA4rdi9QtWpVTJ06FWvXrkVpaSm6du2K\ncePGoaSkRG/RPAZXgFcC8vKA5s2BpCSgXz96bt7ceUvbiLnzPg4rdi/SvXt3mM1mPPHEE3j77bfR\noUMH5OTk6C2W5nAFeCVBK0ubs140hxW7lwkPD8e8efOwYsUKFBQUoEOHDnjjjTdw5coVvUXTDI6F\nVRK0tLS5WlhTWLHrxH333Yfc3FwMHDgQkyZNQufOnbF//369xdIEjoWpwB98VWxpGxZW7DpSq1Yt\nfPbZZ1iyZAmOHj2KNm3a4P333/f5dsAcC3OAP/mq2NI2JKzYDcDAgQORm5uLXr164YUXXkD37t1x\n7NgxvcVyGcPEwoxqFbOvivEwrNgNQr169bB8+XJ8/PHHyMnJQVxcHObMmeOT7YANMUM3slXMvirG\nw1TRWwCmHCEEhg4dirvuugspKSl4+umn8eWXX2LevHlo2LCh3uI5RXy8zrNyS6sYKH/OzNTfXRAT\nQwONIhPAvipGU9hiNyAxMTH4/vvvMWPGDGzcuBEmkwmLFy/2SetdN4xsFRvGV8X4K6zYDUpAQACe\nffZZ5OTkoHXr1hg0aBAeeugh+PoiJV7DyBFcQ/iqGH+Guzv6AGVlZXjvvfcwceJEREREYM6cOejf\nv7/eYhkbo3QMNJvJ/ZOXR4NKcrLxFLgvyMgA4O6OfkVgYCBefPFF7NixA40aNcKAAQMwZMgQnD17\nVm/RjIsRrGIjB3AVfEFGxmk4eOpDmEwmZGdnIyMjAxkZGfjhhx8wf/583H333XqLZkz0juAaOYCr\n4AsyMk7DFruPERwcjNdeew3Z2dkIDw9H79698cwzz+DChQt6i8ZUxBMBXK1z840cZGZchhW7j9K+\nfXvs2LEDY8eOxezZs5GQkIBNmzbpLRZjidYBXE+4TYwcZGZchhW7DxMWFoYpU6Zgw4YNAIA777wT\naWlpKC4u1lkyBoD2aY2eqFjl1Eu/hBW7H9CtWzfs3r0bTz/9NN599120bdsWnHVkALQO4HrCbWKE\nIDOjOW4HT4UQoQA2Agi5tr+lUspX3d0v4xzVq1fHrFmz0L9/fwwbNgwdO3bEhAkTMHHiRAQFBekt\nXuVFywCupypW9Q4yM5rjdh67EEIAqCalvCCECAKwGcBoKWW2rc9wHrtnOXv2LEaNGoVPPvkEbdq0\nwaJFi2Bydh1KxniYzcD48UBBAVBSAoSEANHRwJtvXq+YOS/db/FaHrsklJSMoGsPrn3XkcjISCxa\ntAiZmZk4ceIE2rVrh8mTJ6OsrExv0Rh3EaL874sXgYMHgVdeKc+Q4bx0BhpVngohAgHsANACwAdS\nypftbc8Wu/coKCjAiBEjkJmZic6dO2PhwoVo0aKF3mL5FkaxgNPTy10xp04BWVmk6CMigMRE+l/V\nqrSwdH4+uWkiIoCGDYGbb6bPMz6NVytPpZRlUspEAI0A3CaEuGHeL4R4SgixXQixnfudeI/o6Ggs\nXboUn376KX7++WckJCRg5syZPr+Yh9cwkgVsGTzdvx8IDaXX586VZ8isXw/s2QNcvgyEh9Pznj2A\nlmvrGqXPvVHkMCCaZsVIKc8CWAegj5X/zZFStpdSto+OjtbysIwDhBB47LHHsGfPHnTr1g3PPvss\nevfujV9//VVv0YyPkRbFsMw5LyoixV5cXK7sIyKAs2dJzrAwctWcPk3ZLlu3AiNGuK8EjTLQGUUO\ng+K2YhdCRAshIq/9HQagFwD/WLzTz2jUqBFWrlyJDz/8EFlZWTCZTFi4cCG3A7aHKymGnrIkLXPO\nw8NJuRcXA61a0f+LioDISMpH//134NdfyWKXkqz6DRuAoCDnlaDl+YwaBfzxB7B7N/D11/RcVub9\ngc5IA64B0cJirw9gnRDCDGAbgNVSyhUa7JfxAEIIPPXUUzCbzUhISMDQoUPRv39/nDp1Sm/RjImz\nlZmetCQtc86jokhhm0yUGaMo/KQkeu/CBVLwYWGkzCMiaDA4cMA5JVjxfPLygB9/JOXurKtHywGP\nWyHYRYusGLOUso2UMl5KaZJS/ksLwRjP0qxZM6xfvx7vvfceVq1aBZPJhKVLl+otlvFwtjLT05ak\nsnj0smXAJ59QUNSysOiZZ4AqVSiI2qoVULcucOUKUK8euW6UQUqtEqx4Plev0kBx/jwFbsPC6H1H\nnUa1HvC4FYJduPK0EhMQEIDnn38eu3btQtOmTTFw4ED84x//wJ9//qm3aMbB2cpMb1qSipKfP5+e\nlUKjtDSgTh1yx4SFAc2akbK39MerVYIVzyckhBT5xYs0Y7h8mZR9ZKT9/Wg94HErBLtw214GrVq1\nQlZWFt5++2289tprWL9+PebNm4d7771Xb9GMgTOVmZ5ez1Rt6uWtt9JAdPky0LQpkJtL7ytpkYWF\nwLBhjndd8Xzq1yeFXlpKfvuICKB5c6BlS/ty5+WRpW6JOwOeMoBZCjxsGBdiXYMtdgYAUKVKFUyc\nOBFbt25FrVq1cN9992H48OE4d+6c3qL5Fp60JNW4M5RtQkKAHj3ovV27gLg44M47gb/+sjrrsLXr\ng6YK59OgAbl2unQB7r8fSEig2YCt81P86jt3AqtWUZaOgrsDnrUZCwOAl8ZjrFBSUoL09HRMnjwZ\njRs3xscff4zu3bvrLZbv4KmCJssCJQXltVJ8pGYbZ3edXOF8TCaaATg6P2W0KCujvPvDh8k/3707\nDRB6LFXo46gtUGJXjIcwSrGiK4SEhOCtt97CAw88gCFDhuCuu+7C6NGj8dZbbyEsLExv8YyPp5pq\nqXFnuOjysPsxa+fz0EOO5c3MJKWem0uB2xYtqCL2hx+ARx5hpe5B2BXjAfyldqJTp07YtWsXnnvu\nOfznP/9BmzZt8NNPP+ktVuXEbAaOHgWWLqXqUsWlUdGd4Up6Zno6ntuVgk6r0lHnlFnVx1SRl0d+\n/tBQCuLWqAHccgtl6NSty0rdg7Bi9wD+VDtRrVo1TJs2DWvWrMGlS5fQuXNnTJgwAaWlpXqLVnlQ\nLIUGDcifffYs5ZIfOnSj/94ZH7+FBdLgtkYIOFeI9humoPZvZm1CAzEx1IkyNLT8veJiyrvnfHOP\nwordA/hj7USPHj2wZ88eDBkyBG+++SY6dOiA3bt36y1W5UCxFFq2BDp3ptTCK1fIrVHRneFMeqaF\nBVK3fgASk6JwNTwKjX7K1Ga9jeRk8qkXFZWnRhYXU1Myy6kA93zRHPaxewBPZ7zpRUREBObPn48B\nAwZg+PDh6NChA9LT0/HSSy+hShUvfpW8EcAwUpDE0gFerx49rl4lxW1NJrU+/gqO9bp1gbq9I4AT\neeif7oa8lteuZUtqO1BaSpZ68+bXZ9Eos4aoqOv9lux/dwu22D2Av9dO3H///di7dy8GDBiACRMm\noGvXrjhw4IB3Du6NAIbRgiSeqrL0xH4rXrsGDcivfscd5fnulkp71ixKhfziC2DOHHIxXbnim35L\nA8GK3QNUhmUka9WqhS+++AL//e9/cejQISQmJuI///mP59sBeyOAoVeQxJZLwlOWgif2q1y7khJg\n40Z6nDlD/6uYb242AytWUG/5wEC61r/8Amzfrm2b4UoIK3YPUVlqJx555BHk5uaiR48eGDNmDHr0\n6IFffvnFcwf0RgBDjyCJvVmCs5aC2QykpgJt2lCl6YgR1mcbFfdbWko9ZqZOdd3XnZdHfvSsrPKe\n8FICa9bcuL/MTBpQAgPJFx8cTIVVRUWOe88wdmHFzrhN/fr18fXXX+Ojjz7Cjh07EBcXh3nz5nmm\nHbA3mj/p0WDK0SxBraVgNgMTJlBKpKIsN2ygtVJtKff0dGDMGOr/EhLinvspJoasbSXF8eLF8tWc\nRo26fn95eUD16qT4//qLngEaYBz1nmHswoqd0QQhBFJSUrBnzx506NABw4cPR9++fXHy5EltD+SN\nAIYeQRKtZgmZmeT6CA8n67tqVfq7oMC+K0kr91NyMrX0lZI6QP7yC7llGjcmuSwHi5gYUuB169Ig\nVFJC7zdqRDMNxmVYsTOa0qRJE6xZswbTpk3DunXrYDKZ8Pnnn2tnvXsjgKFHkESrWUJeHg1Cp09T\nGf8vv1AwsqTE/iCh1cASHw/06kUtffPzaQZw002kuOvUuX6wSE6mTJkrV+h/MTE0CNWsSX53Tn90\nGU53ZDQnICAAzz33HHr37o0hQ4bgH//4B7788kvMnDkTtWvXdv8AnirZ9+QxHKVPJieTNQuQQi0q\nstqB0SHBwcCff5LfOjSUXBzHj1OKpL1BQssc3dRUOpfz54HatWlQKS4mn7/lYBEfD7z5JjBzJpCd\nTYNB48Y0GBUUkNIvLub0Rxdgi53xGC1btsTmzZvx1ltvYdmyZYiNjcVXX32lt1iew1ZWi5r0SVdm\nCdaOJwSV7peVkSUM0N+lpbarT9PTyS++fj1w8KD77idrPeE7daLBpeJgER8PzJ5Nx1+4kGYaVauW\nK/W9ezn90QW4uyPjFcxmMwYPHozdu3dj6NChmDp1KiIqTv19GctCG0uLW+kZrljDp08D+/aRv7lO\nHWDaNNcsUVvHO3eOfNY7dtAAAVClZ+PGwL/+dWOXxq++Kt/HkSPUsKtpU/Jxu1uUZe+aWNtvejqw\nZAkpdSHovcuXaebRvDkFjis5ars7smJnvEZpaSlef/11vPXWW2jQoAHmz5+Pnj176i2WNlWm9vre\nKhWeBQXAli2kqEJCyJpt3941N4Ot4+3eTT3SK75fUgJcunS9kl2/HoiNvX6RDBUtfp3CmWubkkKD\nS3ExWfkABWELCoCHH9ZOJh9GrWJnVwzjNYKDg/H6669jy5YtqFatGnr16oWRI0fi4sWL+gmlVZWp\nveCjEhjdt688DbCk5MZgohbHi4y0ntEjxI1ZL3/9RQFOazKrQU2PF2cKOmJi6B4UF5OlLiVdt6Ag\n/ynb9hJuK3YhRGMhxDohxM9CiL1CiNFaCMb4L7fddht27tyJ559/HjNnzkRCQgJ+/PFHfYTRKs3P\nXlaLkj555gxZ6kozrFtvdb3wydbxEhOt++pLSm4cCKKjyRq2JrMjPNF2ITmZgr4mEw2ABQWk3CdN\n4sCpk2hhsV8BMFZK2RpARwDPCiFaa7Bfxo8JCwvDe++9h3Xr1qGsrAzdunXDyy+/jOLi4us39HTn\nP63S/OzlvjsTTLTE3rk7Ol5FK9naQNCoEVnDruTre6LtgnKdbr6ZfOoPPwx88om6RT2Y69Dcxy6E\nWA5ghpRyta1t2MfOAPjb/3r+yBGk/fwz5uzcidjYWCxatAht27Z1PvjmCs4sJefIX7x0KTBjBrk3\nGjYERo68Xik5cz5qtnXGf21rfw88oG6Zu4qkpNDAEGBhGyodJ70V5PRmB06DdPvUJXgqhLgJwEYA\nJimlzVWQWbEz1hTNd7m5GLZnD84UFmLixIkYf+UKgs6dc3r9TnflsKpsHW3nzH7UKAgX1y61exyA\nuilmZ5OLo2NH4JlnXFNQ7srnLloPfO4ey0t4XbELIaoD2AAgQ0p5w3xMCPEUgKcAICYmpt3x48c1\nOS7jo9hQDIWhoXguPx+LFy9Gu1q1sGjgQLSuW7d8G09YhY4sbTvy/q3IUlOp/WxpKf34W7WiYiFX\nFZ0ti9hsJj+6PWVlzzq3TG90NGOwpxT1VnaO7oeW8uk9iFng1awYIUQQgP8HYLE1pQ4AUso5Usr2\nUsr20dHRWhyW8WVs+LajCgrw6aefYunSpTh++TLazp2Ld7dsQZnSDljrZlxmMym7hATg0Ufp+auv\nbvTl2/PFm83A6tVkBYeHU3B0yxYKkLraEdKaT/zwYeDYMccBS1v+7xkz1PnFPVVQpSWOYiNaxgB8\ncEk0t1sKCCEEgI8A7JNSvue+SEylwEEJ+4MPPoiuNWvi6cGDkbZ6NZYfOICPk5LQXErny+ztYakA\ngPLnzMzrlZQ9eTMzgVq16D0hynOwt2yh91NSnHcFWGsxsHcvZYxYylpQQF0TmzUrP0aFlZH+3kd+\nPi14UfF9RUEpVvqyZZS907ZtuVK0dk280drBFo5aINi6Bq5mIPnYkmhaWOxdADwO4C4hRM61x70a\n7JfxZ1R0UKzbvTu+XLECC/v3h/nUKSR8/jlmx8RAxsVpJ4daa8yevHl55B6xzL++eJF6njRo8LfV\nWzhhCmaOMKtL8LFmETdtStkiCqdOUeDzzJnrLWulp7klRUXkZrKVkmlppQN0Dlu2UKWsrWuiJ46+\nP1q2XvbBJdHcVuxSys1SSiGljJdSJl57fKuFcIwfo3IqLxISMPjLL7Hn8GF0TkpCakYG+vTpgxNK\nuby7qFUA9uSNiaG8606dyFo/d45++E2aUFVnQABOl0Zh6+EoNN2VqT7tu2LaYmLi9bLu308WdZ06\n17sbpLxeER08SFWmQtjuB6PMXEpLaZGLY8coNXPbNjrWkSPA0aPG6bjo6PujpTLW2+3kAtxSgPEZ\npJSYPXs20tLSEBQUhOnTp2PQoEEQQrieAKFFkM3aPr75BujRA6hfHwCwbcUpRP62D1HFJ7En7jHs\nuzUZB0LinYu/VTzO0qW0MHSXLtQfBigPLo8ZQxckJ4eUtMlE1r6tfjApKZTTnp1NTcOUwqXSUqBz\nZ6qaveUWaktQUEDbTppk7Bxzg6Qoagn3imH8liNHjmDo0KHYvHkz+vfvj1GjPsTHH9dxXTdroQAq\n7uPUKXKJREUBp04hb0kWqgQJlIRG4HS9RIQWF2JzxzTs/CveuQQfy+McPUrulZtvLv9/xWwNtRkd\n6enAypX0d1gYcOEC8NtvpNirVaMsn5MnaWYSGkoXWUoqIPIVZekHil6tYud+7IxdjPhbaN68Odav\nX4+pU6diwoQJ+P77WCQlfYhmzWiabSvWZxMtgoAV96FY1wCwbx+CggSulkn8Ht0axWEkYPOcTPx+\nj5PHtTyOpV/cVg93tUHE5GTg009pkQspqbS/dm3KdV+7lix1pc+Nsg9lVSZXr523C4yU2Y6lL8zg\nLhVX4SZgjE080Q5EKwIDAzF27Fjs2LEDYWFN8O23DyIzcxAuX6bgn8djfY5aHVj6ZU+eRI1GEdhf\nszN+D6xLLnAZgWp/5LkXf1Pj+3UmhtCzJ/nhz50rb3sQGkqzgoIC+luhuJh6zbh6kZcuBR5/nNr0\nHjlCfn9Pfrk80QLBwLDFzthEbSagnsTGxiI1NQurV7+Fbdtexy+/rMMDD3yEWrX6aJONZs2qPHgQ\neP116o4YHU0NtqxZfxbWdfXCQphKo7BvH+nVusFFuKlXDOq5ex0dzTacWZnpmWesxxtGjqTzLSqi\n94uL6dGihWtZJmYz7U+I6xfUiI313JdLy/RHH4AVO2MTX/ktDBwYhOPHX0HTpvfhhx+GYPHie3Dz\nzU9h4cIpAGq4vmNr0/fx40mxW67yk5tLwUlLpWQ5IAQHA/n5qNusGereYaEwUzXMx7eFYtVbDk7D\nhllXno62ff11stzDwihI+9NPVJBlNjunjDMzywdFy7z//PzrZwVa4kouuhH9kCphxc7YxFfqMsr1\nUTsEBW3HkSOvIivrHTz22GosWLAAd1QsylGLtSlLQQFdlPr1r1dKJ05QsBS4cUBQAo2lpbSdPeXq\nCZyJIdja9qGHKHVz5kxgzRoqvOrcmQYtZ33VeXnlg6Jy/ZQ2vd27q9uHszi7pqyP++RZsTM20Wp9\nZW9Qro9CAfwbP/74AIYMGYKkpCSMGTMGGRkZCFOUiFqsTVlKSiiwaE8pWRsQmjf3boMsZyxNtdvH\nx1Or4fvuu360B5xzocTE0HXMzaXXSpaNJxfUcGbmAviGH9IOrNgZmzj7WzASXbp0we7du/HSSy/h\n/fffx8qVK7Fo0SJ0CAlRr/SsTVlCQihbROkbb00p6enDctbSdHZ7Lc5NsRhMJprBWObFu/vlsjdI\nOTNz8RU/pA1YsTN2UfNbMKorslq1avjggw/Qv39/pKSkoFPHjhhXuzYm1ayJ4Dp1bAc9FaxNWaKj\nURRaB7/8UQMhx/NRs6wANaKCEPamhVLS04flrKXp7PZanJulxRASQjMdLb409gYp5ZzUfkl9xQ9p\nA053ZNzCyCmRCr169cKezz/HoIgIvHHmDG7/9Vfs+fNPcgWUldlOebOSTnhw6Jv4d40MnKzWEpfq\nNceOZg/j9ZafwNzSogJTz94iznYidNS1smJKp1bn5sxaqGqxldI4a5bzX1If7A9jCVvsjFv4iisy\ncs0aLKhXDwPq18dThw6h3d69+FfDhnjx118RqAQ9rVFhyvJZOnCxOfBT+/L3LhZWOF89fVjOWpq2\ntleCotasX6P652y5T77+mrpaOvMl9WU/JFixM26ixhXpjqvG4WfV7vxaJka/4mJ06dABqQcPYtyJ\nE1j+xx9YmJCAlhqeLwDvtbSteP4mE/WTVwRzFPG2FSGvVs32iO2KhW3vPmnly7M1SEnpWj91PdsS\nuwm7Yhi3cFTYqNZVY23W7/CzzviBYmKogrK4GLWvXMGSVq3wWUwMDpSWInHePEyfPh1XlcU83Dhf\nr2Lt/L/6ilZKUtuJ0Fb1akmJdotL2LtPWvrybLlPOnY00E3zDmyxM27hKCVSjavGVsyralUHn3XG\nD6QIGhsL5OdDFBTg0chI3DlhAp5ctgyjRo3CsmXLMH/+fDRp0sTl8/Uqts4/N9e5tEprlqmWwUN7\n90l5rYUvz5b7BDDQTfMOrNgZt3DkinTkujCbaQGgM2eorfitt1Kq9O+/00I+9eoBkZHl719nNDqT\nkmYpaGjo35kYDeLj8c3w4fjoo4/w/PPPIy4uDlOnTsUTTzwBWhzMufP1Ksr5791LfdPPnwdq1KCW\nvO66N7QK1vlkAAAgAElEQVQcwRzdJy3TCm25Twxz07wDt+2thHgzPdFe11hFd2zfTqnhJSWUHt6y\nJXDgAHWNbdqUCjyLi6knldIJNz3dwc5dKAT65Zdf8MQTT2D9+vXoe8cdmNO+PeoXFtpfNFrPPM/0\ndGDrViAri4KdISG0glNxMS1rl5jo3kLOWp2fvftU8W/L115eKNoX4H7sjFW8vbi8veNlZtLfu3eT\nPgoLo+fTp8lKr1KFWoqEhlL8Swha6+FvWT1wMlevXsX0cePwz3ffRdWgIMy691483KgRdSBs3JhG\nH8sgpcYX0ildajYDvXqRP7lqVeDKlfJHaCgwfHj5tnoqS3v3CfDuF9LHUavYOXhayfB291J7nWWV\nFOpWrcqXCw0JoZXZrl4F2rendiRhYdRmpbS0wu9dTdtaJwkICMDosDDsevRRtKhVC49kZuL/vvkG\nfxw8COzcWR4EeP11yoHX8EI6HUeMj6cLFhZGA05QEA0+QUHUP90SjaomHXUrtimnrfvkgXvIsI+9\n0qFFpbSzM3Rbbk8lPle3bvnqa2fOkLVuMpFPHaD/KwbnDfuxsXO3vAh5ebi1aVP82LQpJv/4I9LX\nrcOGKlUw98oV9FWU+F9/kSKyXL3ITeXpUk1As2Y0EkZGXv9+1arXv9YgC8Stvlj2Ugd9OK3QqGhi\nsQsh5gshzgghcrXYH+M53E3X81R2WnQ0kJBAVvqUKeSGcaboz9KSHDECmDDBDRmvXaQqAQEY360b\ntrVsiTpVquD+Y8cwbPlynCspIYGVdUEVnLiQ1ixfZ4tGAVCv9HPnyqc5Z89Sk7IWLTSvmvT6WhUu\nTQ8YQDtXzAIAfTTaF+NB3K2U1vLHbWsW/tBDzs3OzWZqk75yJXlLvv6aMv5KS12UscJFSoiIwNbG\njTG+TRss2L0bcbNmYW1gILk8XLiQtgbH4GCVg66lwsvNBZ59liz2336j5//8B5g+XXP3hksDj6v4\nQq8KA6OJK0ZKuVEIcZMW+2LU4aqrIT6e6ldmzKB1DRo2JKNPr6Z3tmbhNmfnVk585sx4HDlCaz5E\nRNB5FRdTBmDfvs7LaEY8NldNQ4MNmWgi8tC4RRvUrpOPjGbN8ECbNhicmYke69ZhSNd78MjO6qh+\n6gSuNIxBw5HD0FLFhbTlciktJf2lyGs1w9CaP+ToUWDatOsvmAcUoFf7YvlKrwqDwsFTH8QdY8Zs\npmSOhATg0Ufp+auv1OsBa66cI0dIt3h8xmzjxH9fa0aNGhRDFIKq4atUIWNVQa0CUg5xICQeOx9I\nx4I75uOFsNk4OCQDiIrC7VJi1zPPoH+3x7Bw80r03/o/jKo+HPMapuONr+JVnbsty7ekRMVMRc2U\nyUPWrlf7Ynl1euB/eE2xCyGeEkJsF0JsL6jom2Scwh13iLuulIo/7kOHKI26QQNtZ8xW3as2hO95\n7nrha9emhJWyMucVkK3r81lu/N/dCA8/koGjRZ+iefO1EKIUOTldsXr1OJSWlqi6jvbiHA6bHqpR\neB5yhns1gcVQvRt8D69lxUgp5wCYA1Aeu7eO6w9U9D7k5Nz4Y1JrzLjrSqlYeZmfT604WrYETp0C\n9u8Hjh8HNm6k+pjEROfrWmxlX0w9l4ea8TcKHx+Zh/PnyVoPDaXYYY0atHqbsyvRqbk+ypKdDRt2\nR716Zhw5MhanTr2NzZnzcXe9jgDa2D1pt4o61fhDPLhIhNcSWAzVu8H34HRHg2NNyR07RtlsLS1a\nEqo1ZhS9UFpK6YVKh9a2bdXLpPywMzOBTZuoeOivv2iN57IySqH+6y9yz1SrZj0lzl6MwJZ7dXde\nDLoX3ajUmnWPQYsTlCpZVESp3SYTkJHhvBJSqzfLl+wMR5+Gz6Hp5aN4/9xPeOrECpxcdQb/PHwY\nVV56yaoA8TBjcrVM/LYxD8dlDE52TEbXtHh1sqpReD6+SAQAg/Vu8D00UexCiM8BJAGoLYQ4AeBV\nKeVHWuy7smNNyZlMlAwRHe28MZOcTKmAhw+TVRsURNlyv/6qfrF5y8GmQQM6/oED5AK5cIG2CQ8n\nn3d+Pvnx1TT9sixcsmZwro1MRvfCG5VavbRhyIBrwWRXut7GxJBS37uXXnf7PRMhQSZMqNUeWyK/\nxaTsbHx17BgWRUbi1hkzrF68elFRqHd/I7QpKgQKpwBIA6Ay+u1I4Wlh7erdLgHg/HY30Cor5lEt\n9sPciDUl17w5cPEiKUZnjZn4eMqEOXOGrPaICLLWlaVA1ezDcrBp3RrYsoX2df48WetCkJJXlgO1\n5wIGbkx4sGVwRiXGA8nWlVo8XGsRXnGAUbre5uY61pvXGkUi8mwezgQ3wr1JAUhr/SD+t/dWpH7z\nDdrMno03mzfH6NGjERAQoO7k1eBI4blr7bpVicQYAXbFGBxbSi4x0fW2H6WlQO/eFFdTuHpVvQvW\ncrBRqka//ZbkioykmUD16tQiIDAQWLWKMj7S00kpOnIB2zU4NbTiXO16W7FRZJSIQbeGhYi+mXYw\nMDYW3SIi8NSmTXjhhRewbNkyLFiwAE2bNvXeIsnuXCd3Bx8jWPuVHE53NCCWGSGnT1M6oZYpZu4m\nHFT8fN26QNeuQJMmpOSFoALIP/8E/viDXD23366+EMdb2RfuZNRZZq90n5aM6EC6Mad/u4rslYU4\nsr4UvZOW4F//+hg5OTmIi4vDnDlzIBs3Nn62hzsXhguLDAErdoNR8XcRHEyKsqREOyXnbj6ytc8H\nBgKTJlFAt1kzstwDA6nH+p13AvXrlxuBQjg+vsO0Pw3QLKPu2kh0qjQKB344gUJEYVePNBwMTcCh\nQ0OxZMkedOzYEU8//TTu/eEH5OfneykZ3EXcuTBe7zvAWIPb9uqAvZmqxi3GXZJBq8+npNDgVNHl\nc+IEMGaM/rN1a91kjx6lGERpqfNy2bt3r7xyFbNmzcJLL72E4MBAzOjZE/+IiIBo0sR4rgp32iHb\nu+nz53tW7kqA2ra97GP3Mq5mhBjJBav28zExlAJ58mR5ELVBA7LqjZDwUDHGGBJCqZshITTTcDZm\naO/e5eYGoKDgWfTpcze2bRuKQV9+iczkZMx+9llER0d75gRdxZ3gqz+kWvoBrNi9jDJTLS2lIh4l\nj3zWLHr40+/CZAIWLaLUx/Bw8rvn5RnL62A5wKSn071wNWZo696FhJQP5rGxN6NBg43YsuU9fP31\nRGzaFIs5c+agf//+mp6X27g68nJhkSHwex+70Tp/5uVRDvSWLRRY/PNPCo5+8QWwdKmX+3F4mNxc\nWs4uMpJSISMj6XWuk82dPX0Plf0vXkxVvadOlf/PmdmSrXsn5fVu51q1AtG164t48skdaNSoEQYM\nGIAhQ4bg7Nmz2p6YHvjDwhlGUxou4NeK3YgBeqUlwNWrlEt+5QpZdMHBtCgP4Pu/C4W8PMq5T0oC\n+vWj5+bNnV/Uw5P30HL/SrFVVla5cndmtmRLpyn1ApZERADFxSb89NNPePXVV7F48WKYTCZ8//33\n2pyYnngj8u0pjKg0XMCvXTFG7PyZnAx8+ml5jjdAZfgNG9J7mZm+91uwhRZuJU/fQ2vFVkJQu4WQ\nEOe9CNY8GPauQ1BQENLT09G3b18MHjwYvXv3RmpqKiZPnozq1au7f4JGxoj57kZUGi7g1xa7ETt/\nxscDbdqUT9MvXgRq1qQ2s9HR/tWV1FonyPXracaidoZreQ9Pn6bPb9gALF+ujRFluX+l2CoiggK+\nrs6WKs7kTSbH7rX27dtj586dGDt2LGbPno2EhARs2rTJ/RM0Kka1jI2oNFzArxW7ETt/ms3kY69a\nlQKK1aqRS0b5fvtikNQWlq4Js5l867Gx9L7a37FyD0+fJmv68mVyWwUHa6MHrBVbJSYCjz3m2szJ\nmr5S2hQ4cq+FhoZiypQp2LBhAwDgzjvvRFpaGoqLi907SU/jik/aqPnuRlQaLuDXrhgjBugzM6mA\nJyyMLM/SUpryh4SQa0bLIKkRZrqKayI9nSpTbc1wLWW1LMoKDqZ+LGfO0DUC6P1OnZzrb1MR5Xg5\nOdQt02Qi/7+73xF7bQqSk8vPUdFf1mTv1q0bdu/ejRdffBHvvvsuvv32WyxatAjt2ztMX/Y+rvaV\n8VZer7MYUWm4gF9b7EYM0Cszvdat6TsUG0sumCpVtJXNaDNdezNcS1mDgmjAW7+e/lZyy4uKaBAM\nCyOlXq+e63rA8njx8XQPcnPpfXe/I7bOMyfHuftRvXp1zJo1C9999x3OnTuHjh074tVXX8Vff/3l\nmmDW0CL7w1XL26iWsRGVhgv4tcUOGKMQxhLLQFrduvRQXmspp9FiQPYCiJayrl9PLiqAWgErmTQX\nLlD7Xy3y+ytem5YtaXC1Vd3rzMzH1nmePWt/xmKL3r17Izc3F6NGjcK//vUvfP3111i0aBFMJpPT\n533DSWnRwdFVy9vIlrHRlIYL+LXFbkS8ladutBiQvfO2lLWoiDomKi1/FbkjI7W7bs5cG2dnPrbO\nMzLS9fsRGRmJRYsWITMzEydOnEC7du0wefJklJWVqTtha2jl43bV8vYTy9io+L3FbjS8tTCMXhWs\ntnzlMTG2+5xbyhoRQQFS4Hplryyxp8V1c+baODvzsXV/MzPdvx8DBgxA165dMWLECLz88stYvnw5\nFi5ciBYtWqjfiYJWPm53LG8/sIyNCit2HfDG91mPma7l7F7xlQPAHXeUZ4dYM8osZb3lFmq1AJAy\nVyxeRYnbu25qXSYmExWD/fUXuWAaNqQYh7Vro+g/ZT3XoiJyFVkq6IrYklOL+xEdHY2lS5fis88+\nw8iRI5GQkIB33nkHI0aMKF/MQw1ajfy8hJ0h4e6OFTBCJolWePtcLLsbrl9fbnmHhZGv3F6XSnuW\nvjW5bS1p56ghoTL4lJWRB6CggAahSZOAhx66Ua4RI4DNmymvPSyMgrZlZRTQ/eQT7TtiOkN+fj6e\nfPJJfPfdd+jZsyfmz5+Pxo0bqxfG1Q6OjG6o7e7Iit0C/q47h9lMjcuys0nRXb5MCrx+fSogUoKg\n585RSwGturdau0/r15Nyv/nm8u2sDSTOtEU2m2l92KwsSkWtUoUGm5o1gQ4d6FhatlJ2BSkl5s6d\nixdeeAGBgYGYNm0aBg8eDCGE4w/7kxVTSeC2vS6gRyaJr/22LPO/f/653J0BkNL++mvqufLbb5R7\nHhVF658C5Z0O09Pp82fPUlBR8Z+rPW9r9+mvv+j4lordmsvYGdeyUnNw6BANWiUlJH94uPM9bzyF\nEAJPPfUUevbsiSeeeAJDhw5FZmYmPvzwQ9SrV8/+h33Ax+1rvw+joElWjBCijxDigBDisBDin1rs\nUw+8nUlitFxzR1jKW1hIivr8ebLEq1YlJf3HH8Cvv5Jyv3SJ/q5dm7Y/epReHzxIf589S8+HDjl3\n3tbuU3Q0uVUsseYydiaJQzlOvXr0uPVWoEULcsUYIeUawN+56M3eeAPrkpLw3pAhWPXNNzDddBOW\nPvywcb9MKvC134eRcFuxCyECAXwA4B4ArQE8KoRo7e5+9cDbNRNGraq2haW8586RHzwkBPj9d/r/\nlSvkhxaCXDPNm9O1O3yYPtOwIb2n+KsjI+k5P9+587Z2nxo2JF+5o3RIZ9JNlePceiu1gbh8ubyl\ngSFaKVfQfAGHDuH5jRuxq08fNK1ZEwP/9z/8o18//OmjPWd87fdhJLSw2G8DcFhKeVRKWQrgvwD6\nabBfr+PtXuhGyzV3hKW8ERHkd5aSlB5ADc2qViVF2K8f0LcvMGAA0LYtuV+U9rVKrjpQnq/ubt/z\nKlUoAOooLdqZ9GnlOCEhQMeO9N6ff9L5eCXu4qgyVNF8JSWUSrR2LXDpElqdOoWsJ5/E69274395\neTDdey++/fZbDwurPb72+zASWvjYGwL41eL1CQC3a7Bfr+PtzC1fWy3JUt5WrcjyLiggZX7pUvmy\ncq1alX/G8nyUzyu56mFhNCgoyt7ZvufW7pO1zBZrn1dzTyse5557vOjjVVMZmpdH05TsbBohpaQR\n7uhRVDlzBhPvuAP3NW+OwUuX4r777sOTTz6Jd999F+FKVNvg+Nrvw0h4LXgqhHgKwFMAEGPgO+PN\neJIrueZ6BJNsNczq0AHYto1cL6WlQPfupKiDg8mSVs6nW7fygOmxY+R/z88n/3xpKblRjhwBhg6l\n7dScmyfuk61rq0uwTk0kPyYGWLmSlHpYWPlIGRZGSff16qFN1arY/uyzSC8rw+TJk7F69Wp8/PHH\n6N69uw4n5RxG7jpgdNxOdxRCdAKQLqXsfe31OACQUr5l6zNGTXfUA2cUtR7pmBWPefgwsHcv0LSp\n9WyWiudTowate6pkz1SrRvsoKiLjsnp18rWHhND+mzXzzrmpzYOvWC3rNYs9JQVo1AinCwKwb981\nd1X4VcRHnUDNZfPLTyI5mfIvw8LIT3TiRHnF1Z13XncRs7KyMGTIEBw6dAijR4/GW2+9hbCwMC+c\njOtwVsz1eC2PXQhRBcBBAD0A5APYBuAfUsq9tj7Div1G1HyBncnB1gp3jmk2A48/ThY9LQVHj7Iy\nUvD33FO+7cqV9Gz5nqfOTW0e/KFDpNSTknSoa0hPR8GhQmzOjfq7d44oKsSfMgotPkkvP35qKrBz\nZ3kAo04d4Phx8rv373/DF+nixYsYN24cpk+fjltuuQULFy7E7beXe05ZkRobtYrd7eCplPIKgJEA\nVgHYB2CJPaXO3IjatC49gknuHDMzkyz1iAhS7mFhpKB+/530jiUlJTe+56lzs5ZtoeTBW7J/P8UR\nNm6kR0mJF7MykpORn1uIKFGIsNCrCCsuRKQsxMHY5OuPn5pKfRjuuIMe9evT60WLrK4UUq1aNUyb\nNg0//PADLl++jM6dO2PChAkoLS3l9EI/QpM8dinlt1LKllLK5lLKDC32WZlQm9alRwtrd46Zl0fu\nF8sFgEJDyWJXFs1QUBYbceU4zqImD/70aXIZBQdTQdLly1SBWlzspayM+Hj8v6ZpuBoRhYhzJ3A5\nLApft0zDyvx4LF5skSTjYpfEu+66C2azGUOHDsWbb76JDh06YObM3Zxe6Cdw5akBUFsNqWUwSe2U\n251jxsSQlZubS6+V1MaoKFKkSoZMURG9FuL69zwVKLOWbdGwIRVMKcffuZMSTqKiymcbAAWALd1F\nniQwMR4fHYpHfj41ISvcR/GIBg0qJsm4FuGNiIjARx99hP79+2P48OGYO7cD7rwzHV27voSAgCrX\ntnFyIGNfjiHgfuwGQK1VXNE4Ky2lVMOpU51bAMeZKbcag9BWunVyMuW6m0yk1AsKKCPvzTfpYbnP\nN98EMjLUGZ7uLvyjJg++pIR864GBZK1LSY8//vBeYZLJRLOEs2cpnfTqVWrTUKeOttb0/fffj717\n96JVqwFYt24C5s/vit9/PwDAyVkT+3IMAzcBMwCuZLu4kyGjZRDWkRxaG3BaZQY5kku5RqWl+Dsr\nJTiYipNmzXJdfmdIT6f2CydPAnv2UIZR9erUoiEpSbumagpmMzBy5BfYtu0ZXLlyCZ07v42YmOfw\n4osB6q6tHtH9SgY3AfMhXCmMcqdhmZbrCDuSQ+s8cK0atTmSS3FBRUVRTFIZQFJTXZfdWfLyqDdN\ny5b0+vLl61eW0joGER8PzJjxCBYuvANLlgzHxo1j0L79MoSHfwzgJnUCG3GB6koIu2IMQnw8GTXz\n51tNZrgBd7JVtAzCejtTx1vH88TKbc66kCzvk9KvRlnow1PtLuLjgXffrY+8vK/x0Ucf4cCBHYiL\ni8O8efPgcHZv1AWqKyGs2H0Ud35DWvbEUSOHuz5xZ4+nFc4OtvZwxf1seZ/q1AFiY8nPr8xYPJlP\nL4RASkoK9uzZgw4dOmD48OHo27cvTp48qU5gbzRbYmzCit1HUfMbsqVQtbRGHcmhdTzN3vG0HEC0\nxpVOhRXvU8uWtGrTsmXuDzRqadKkCdasWYNp06Zh3bp1MJlM+Pzzz61b77xAtWHg4KkBURtwtLed\nJ4KMtpassyeHJ+JpS5cCM2ZQv5mGDYGRI0npGXn1q2sdAmC5LKnWwU9Pc/DgQQwZMgTZ2dkYOHAg\nZs6cidrKKiqMV+DgqY+ipqmfsp095a9FkNHR4tSWctlak3TxYsq7bt0aqFuX/ueOT9xspp4uCQnl\nQc2vvqIWBd5e/coZ/KFTYcuWLbF582ZMmTIFr7zyCjZs2IC5c+figQce0Fs0pgLsivEyalts25uy\nq3FvaBFktJTlwAEK2oWH09/2XAmW8jVoQApsyxaq5gTcU2i2rk92trF7d/uL+zkwMBAvv/wytm/f\njgYNGqBfv3544oknUFQx8MHoCit2L6KVQlaj/LUIMlrKoiyOYZluZ2+tUEWmVq0o4CcErZHqrkKz\ndX2kNHZChr+5n+Pi4vDTTz9h4sSJ+OSTTxAXF4c1a9boLRZzDVbsXkQrhaxG+WthIVrKYtmd0VLZ\n21srFKC1Qjt1otcnT7qv0Gxdn44djW8Ra5llYwSCg4Px+uuvY8uWLahWrRp69eqFkSNH4uLFi3qL\nVulhxe5FtFLIapS/FhaipSy33ELrnJ47R3+rWStUoV496t3+2GPuKzRb1+eZZ4xtERs5Y8ddbrvt\nNuzcuRPPP/88Zs6ciYSEBPz44496i1Wp4awYL6I2Q8RRYNSbC26oyYqx9hlPyqdF1pA30WOBFL3Y\nsGEDhg4diuPHj+PFF1/Ea6+9hlBlgVvGbby20IYrVFbFruUP3ChKyxZ6y2ckZVrZWqicP38eaWlp\nmDNnDmJjY7Fo0SK0bdtWb7H8AlbsBkVvhedpjHJ+RlKm/pDD7grfffcdhg0bhjNnzmDixIkYP348\ngoKC7H/IKF8gg+K1FZQY5/C3AJolRuraqsdqU7aorC1U+vTpg9zcXDzyyCNIT09Hp06d8PPPP9v+\ngJG+QD4OK3bGJs4G/Fwpm/cURlKm/pLD7gpRUVH49NNPsXTpUhw/fhxt27bFu+++i7Kyshs3NtIX\nyMdhxc5YxRXjyUhWspGUqb/lsLvCgw8+iL179+Kee+5BWloaunfvjiNHjly/kZG+QD4OtxRgrOJK\nSwJnyuY97Up1pce9J9G6L70vUqdOHWRmZuLTTz/Fc889h4SEBEyZMgVPP/00hBD+0XfBILhlsQsh\nBgoh9gohrgohHDr0GefRK//ZFeNJrZXsLVeqP8czfBUhBB5//HHk5uaic+fOSE1NRZ8+fXDixAlj\nTbN8HHddMbkAkgFs1EAWpgJ6xpJc8VGrdTmwK5Vp1KgRVq1ahZkzZ2Lz5s0wmUz4ZPduyLFjK7fP\nSiPccsVIKfcBNAoz2mPPHaI8e8qVoSwNB1yfBz5smP3PKS4HxdUydeqN8qldQY0z3/wbIQRSU1Nx\n9913Y+jQoRg8eDAy+/fHhx9+iDp16ugtnk+jSR67EGI9gDQppark9Mqcx+4MtvKfzWbqsujp4hs1\nFbDW/u+oOMhWjnlpKbX2Vapc8/OBZs30LzBiPE9ZWRmmTp2KCRMmoEaNGvjwww+RzC6YG9Asj10I\nsUYIkWvl0c9JgZ4SQmwXQmwvKChw5qOVFlvukLNnvePKsOejtucmcuRqseZKPXoU+PXX8v1t3gxk\nZQGrVgEbN1IbA3bX+C+BgYEYO3YsduzYgSZNmuDBBx/EoEGDUFhYqLdoPolDxS6l7CmlNFl5LHfm\nQFLKOVLK9lLK9tHR0a5LXImwFUuKjNQ/K8ye8nYUeLXmi2/YEGjenP4+c4Y6QQYGApcv0yMrizpL\ncuabfxMbG4usrCy89tpr+OKLL2AymfDdd9/pLZbPwXnsBsZWMDIxUf/im5wceixfDqxfD5w6Va68\n1XaftJwNlJaWDwb79wNhYUCVKmSph4VRH/icHD/OfPPn9o9OEhQUhFdeeQXZ2dmIiorCPffcg6ef\nfhrnz5/XWzSfwd10xwFCiBMAOgH4RgixShuxGAVr7hC9s8LMZuDYMVLW4eHlFvXhw+W+dmflsxwM\nioqo1W9JCVntUtLjjz/8NPONS+mt0q5dO2zfvh0vvfQS5s6di4SEBGzcyAl4anBLsUspv5RSNpJS\nhkgp60ope2slGGMbvSsZMzMBk4mUbXExWdNCAHv3lgdQnZXPcjAIDwfKyoCaNYH69akHvBBAr15+\nGjjl/E+bhIaG4t///jc2bdqEgIAAJCUl4YUXXsDly5f1Fs3QcOWpj6JnJWNeHvnDw8OBffvIwo6I\nIF2kyOSsfMpgMGsWuXXOnCHjtV07GjgKC4HUVM+cj+6ozf+sxHTp0gW7d+/Gyy+/jPfffx8rV67E\nokWL0KFDB71FMyTsY2ecRnGb1K0LJCUB/fqR3z8x0f19X7wI9OkDPPwwULUq8MMP5H/36zRHI3Us\nMzDVqlXDjBkzsHr1aly4cAGdOnXCpEmTUFpaqrdohoMtdh/BSMU6joqXrMkKOJa/YkFW/frl+e5+\nq9QB16vBKik9e/bEnj17MGbMGLzxxhtYsWIFFi1ahLi4OL1FMwxssfsARout2fOhW5N1/HhgwgTH\n8lfa5n56B018kMjISCxYsADLli3DyZMn0a5dO7z99tvW2wFXQthi9wFc6bToaWz50K3JqtSjtWtX\n/p6yreU+KnVzP27/6BL9+vVDly5dkJqainHjxmH58uVYuHAhWrZsqbdousIWuw/gS5asNVlLSuhh\niTX59U7jZHyT2rVrY8mSJfjss89w4MABJCYmYvr06bh69areoukGK3YfwJdia9ZkDQmhhyXW5GeP\nBOMqQgg8+uijyM3NRVJSEkaNGoVevXrh+PHjeoumC6zYfQBfsmStyRodDdSpo05+7qHOuEODBg3w\nzTffYO7cudi6dSvi4uIwf/58aNHs0JfQpLujs3B3R+cxUlaMI1zNimEYLfnll1/wxBNPYP369ejb\nty/mzJmD+vXr6y2WW6jt7siKvRLiS4MEw7jD1atXMX36dPzzn/9E1apVMWvWLDz88MN6i+UymrXt\nZeoqERQAAAkWSURBVPwLo6VOMownCQgIwOjRo5GTk4Obb74ZjzzyCP7v//4Pf/zxh96ieRRW7JUM\nd9uScBNCxhe55ZZbsHnzZmRkZCAzMxMmkwkrVqzQWyyPwYq9kuFO6iRb+4wvU6VKFYwfPx7btm1D\nnTp1cP/992PYsGE4d+6c3qJpDiv2SoY7qZPchJDxBxISErB161aMHz8eCxYsQFxcHNauXau3WJrC\nir2S4U7qpC8VSjGMPUJCQpCRkYEtW7YgNDQUPXr0wKhRo3Dp0iW9RdMEVuyVDHeKgHypUIph1HD7\n7bdj165dGD16NKZPn47ExERkZWXpLZbbsGKvhLhaBORLhVIMo5aqVati6tSpWLt2LUpLS9G1a1eM\nGzcOJRX7YPgQrNgZ1XDJP+PPdO/eHWazGSkpKXj77bfRvn175OTk6C2WS3CBEsMwTAW+/fZbPPnk\nkygoKMCrr76Kf/7zn6hSRf9muFyg5Edw7jjDeJd7770Xubm5GDhwICZNmoTOnTtj//79eoulGrcU\nuxDiHSHEfiGEWQjxpRAiUivBGIJzxxlPw4aDdWrWrInPPvsMS5YswdGjR9GmTRu8//77PtEO2F2L\nfTUAk5QyHsBBAOPcF4mxhHPHXYOVlTrYcHDMwIEDsXfvXtx999144YUX0L17dxw7dkxvsezilmKX\nUn4vpbxy7WU2gEb2tmech3PHnYeVlXrYcFBH3bp1sWzZMixYsAA5OTmIi4vDnDlzDNsOWEsfewqA\nlRrujwHnjrsCKyv1sOGgHiEEhgwZgj179qBjx454+umnce+99yI/P19v0W7AoWIXQqwRQuRaefSz\n2GYCgCsAFtvZz1NCiO1CiO0FyiKYjEM4d9x5WFmphw0H54mJicH333+PGTNmYOPGjTCZTFi8eLGh\nrHeHil1K2VNKabLyWA4AQoihAPoCeEzaOTMp5RwpZXspZfvo6GjNTsDf8XTuuD/6ollZqYcNB9cI\nCAjAs88+i5ycHLRu3RqDBg3CQw89BKMYrW7lsQsh+gB4D8CdUkrVZ8R57MZA8UVHRZFFW1REP2pP\nFR15a4EPb5+Xr8MLr7hHWVkZ3nvvPUycOBERERGYM2cO+vfv75FjeWUFJSHEYQAhAJSu9dlSyhGO\nPseK3Rikp5PCi4oqf095nZ6u7bH8dRBhGIXc3FwMHjwYu3btwuOPP45p06YhMlLbDHC1it2tUiop\nZQt3Ps/oS14eZY1Y4ilftGVAEyh/zsz0jMKNj2dFzngXk8mEn376CRkZGXjjjTewdu1azJ8/H3ff\nfbfXZeHK00qMN33RHNBkKgNBQUFIT09HdnY2wsPD0bt3bzzzzDO4cOGCV+VgxV6J8WbgjAOaTGWi\nffv22LlzJ9LS0jB79mwkJCRg06ZNXjs+K/ZKjDe7NXL2BVPZCA0NxTvvvIMNGzYAAO68806kpaWh\nuLjY48fm7o6MS7gSnOSAJlNZuXDhAl588UXMnj0bS5cuxYMPPujSfrySFeMqrNh9G04nZBjX2Llz\nJ9q0aQMhhEuf90pWDFO5UCzu5cuB4GCgTZvykn3AcxkuDOMvtG3b1ivHYR87owrLxlpS0iMrCzh1\niv7PGS4MYxxYsTOqsMxDj4wEhABCQwFl7QHOcGEY48CKnVGFZR56q1ZAcTFZ7WfPcoYLwxgNVuyM\nKizz0OvWBTp3Jqsd4EWtGcZocPCUUUVyMvnYAbLcg4OBW25hhc4wRoQtdkYV3ixmYhjGPdhiZ1TD\njbUYxjdgi51hGMbPYMXOMAzjZ7BiZxiG8TNYsTMMw/gZrNgZhmH8DF26OwohCgAct7NJbQC/e0kc\nd2FZtcdX5ARYVk/gK3IC3pe1iZQy2tFGuih2RwghtqtpTWkEWFbt8RU5AZbVE/iKnIBxZWVXDMMw\njJ/Bip1hGMbPMKpin6O3AE7AsmqPr8gJsKyewFfkBAwqqyF97AzDMIzrGNViZxiGYVzEsIpdCPG6\nEMIshMgRQnwvhGigt0y2EEK8I4TYf03eL4UQkXrLZA0hxEAhxF4hxFUhhOEi+QAghOgjhDgghDgs\nhPin3vLYQggxXwhxRgiRq7cs9hBCNBZCrBNC/Hzt3o/WWyZbCCFChRBbhRC7r8n6mt4y2UMIESiE\n2CWEWKG3LBUxrGIH8I6UMl5KmQhgBYBX9BbIDqsBmKSU8QAOAhinszy2yAWQDGCj3oJYQwgRCOAD\nAPcAaA3gUSFEa32lsskCAH30FkIFVwCMlVK2BtARwLMGvqYlAO6SUiYASATQRwjRUWeZ7DEawD69\nhbCGYRW7lPKcxctqAAwbDJBSfi+lvHLtZTaARnrKYwsp5T4p5QG95bDDbQAOSymPSilLAfwXQD+d\nZbKKlHIjgD/1lsMRUsrfpJQ7r/19HqSIGuorlXUkceHay6BrD0P+7oUQjQDcB2Ce3rJYw7CKHQCE\nEBlCiF8BPAZjW+yWpABYqbcQPkpDAL9avD4BgyohX0QIcROANgB+0lcS21xzb+QAOANgtZTSqLJO\nBfASgKt6C2INXRW7EGKNECLXyqMfAEgpJ0gpGwNYDGCkkWW9ts0E0NR3sZHlZCofQojqAP4fgDEV\nZsOGQkpZds392gjAbUIIk94yVUQI0RfAGSnlDr1lsYWuKyhJKXuq3HQxgG8BvOpBceziSFYhxFAA\nfQH0kDrmkDpxTY1IPoDGFq8bXXuPcQMhRBBIqS+WUmbqLY8apJRnhRDrQHEMowWouwB4QAhxL4BQ\nAOFCiE+llIN0lutvDOuKEULcbPGyH4D9esniCCFEH9C07AEp5SW95fFhtgG4WQjRVAgRDOD/AHyl\ns0w+jRBCAPgIwD4p5Xt6y2MPIUS0klEmhAgD0AsG/N1LKcdJKRtJKW8CfUfXGkmpAwZW7ADevuZC\nMAO4GxSBNiozANQAsPpaeuZsvQWyhhBigBDiBIBOAL4RQqzSWyZLrgWgRwJYBQryLZFS7tVXKusI\nIT4HkAXgFiHECSHEML1lskEXAI8DuOvadzPnmqVpROoDWHftN78N5GM3XCqhL8CVpwzDMH6GkS12\nhmEYxgVYsTMMw/gZrNgZhmH8DFbsDMMwfgYrdoZhGD+DFTvDMIyfwYqdYRjGz2DFzjAM42f8f0om\nJfG7db8gAAAAAElFTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "m0 = np.array([0,0])\n", "m1 = np.array([2,2])\n", "K = np.array([[1,0],[0,1]])\n", "\n", "N = 250\n", "pi = 1/2\n", "N1 = st.binom.rvs(N,pi)\n", "N0 = N - N1\n", "X0 = ...\n", "X1 = ...\n", "..." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Maximum Likelihood\n", "(25 pts)Similar to the previous part, we are going to build a binary classifier over the same dataset but this time we will be using ML to estimate the parameters of the generative model from the data. i.e. the values of pi, m0, m1 and sigma are all computed from the data.\n", "\n", "we then plot the data points and the ML decision line along with the true decision line (obtained in the previous part) in one plot, using the same plot helper function classPlot." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# solution" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Gradient Descent for Logistic Regression\n", "(25 pts) Let us assume a logistic regression model. First, we modify the existing data points from the previous parts by concatanting a 1 to each data point (so each data point is of the form x = [x0,x1,1]).\n", "\n", "We then use gradient descent to find the optimized values for theta with 1000 iterations with a step size equal to $0.001$ to build the classifier.\n", "\n", "Similar to the previous part we plot this classifier generated using Gradient descent along with the other two classifiers in a single plot." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# solution" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Stochastic Gradient Descent for Logistic Regression\n", "(25 pts) In this part, we replace the Gardient Descent from the previous part with a Stochastic Gradient Descent using a step size of $t^{-3/4}$ and 1000 iterations to obtain the optimised value of theta. Plot the classifier so obtained along with the previously computed classfiers using the helper function classPlot." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# solution" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.2" } }, "nbformat": 4, "nbformat_minor": 2 }