{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "provenance": [] }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" } }, "cells": [ { "cell_type": "markdown", "source": [ "# CS640 Homework 1: ROC Analysis and Ethics\n", "\n", "In this assignment, you will answer some questions regarding ROC analysis.\n", "\n", "## Collaboration\n", "You are allowed to work in a team of at most **three** on the coding part (**Q1**), but you must run the experiments and answer written questions independently.\n", "\n", "## Instructions\n", "\n", "### General Instructions\n", "In an ipython notebook, to run code in a cell or to render [Markdown](https://en.wikipedia.org/wiki/Markdown)+[LaTeX](https://en.wikipedia.org/wiki/LaTeX) press `Ctrl+Enter` or `[>|]`(like \"play\") button above. To edit any code or text cell (double) click on its content. To change cell type, choose \"Markdown\" or \"Code\" in the drop-down menu above.\n", "\n", "Most of the written questions are followed by a cell for you enter your answers. If not, please insert one. Your answers and the questions should **not** be in the same cell.\n", "\n", "### Instructions on Math\n", "Some questions require you to enter math expressions. To enter your solutions, put down your derivations into the corresponding cells below using LaTeX. Show all steps when proving statements. If you are not familiar with LaTeX, you should look at some tutorials and at the examples listed below between \\$..\\$.\n", "\n", "Alternatively, you can scan your work from paper and insert the image(s) in a text cell.\n", "\n", "## Submission\n", "Once you are ready, save the notebook as PDF file (File -> Print -> Save as PDF) and submit to Gradescope. Make sure all outputs are visible." ], "metadata": { "id": "wWhk6f79x6MH" } }, { "cell_type": "markdown", "source": [ "## Q0: Name(s)\n", "\n", "Please write your name in the next cell. If you are collaborating with someone, please list their names as well." ], "metadata": { "id": "yH_FqNM805X5" } }, { "cell_type": "markdown", "source": [], "metadata": { "id": "ow6uTaD708Du" } }, { "cell_type": "markdown", "source": [ "## Q1: Confusion Matrix\n", "\n", "In the next cell, write code to manually compute the confusion matrix for a **binary** classification task and run the cell to test your code.\n", "\n", "The function `confusion_matrix` from sklearn is used to verify your answers, but you should **not** use it in your implemention." ], "metadata": { "id": "2OUZOOoA08ks" } }, { "cell_type": "code", "source": [ "import numpy as np\n", "from sklearn.metrics import confusion_matrix\n", "\n", "def compute_confusion_matrix(YTrue, YPredict):\n", " \"\"\"\n", " Computes the confusion matrix as a numpy matrix. For convention, the\n", " vertical axis represent YTrue while the horizontal axis represent YPredict.\n", " \"\"\"\n", " cm = np.zeros((2, 2))\n", "\n", " ############################ start of your code ############################\n", "\n", " ############################# end of your code #############################\n", "\n", " return cm\n", "\n", "is_correct = True\n", "for _ in range(10):\n", " rng = np.random.default_rng()\n", " YTrue = rng.choice([0, 1], size = 100)\n", " YPredict = rng.choice([0, 1], size = 100)\n", " if (compute_confusion_matrix(YTrue, YPredict) != confusion_matrix(YTrue, YPredict)).any():\n", " is_correct = False\n", " break\n", "print(is_correct)" ], "metadata": { "id": "V-6CofDE2Ziu" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "## Q2: Metrics" ], "metadata": { "id": "ZqzxSFz29AYa" } }, { "cell_type": "markdown", "source": [ "### Q2.1\n", "Suppose that we are given the confusion matrices and the ROC curves of two models **A** and **B**.\n", "\n", "![Screenshot 2023-09-02 210029.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAngAAAGCCAYAAABzdr7YAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAACrCSURBVHhe7d1PSF1n4sbx18BMf1raoehCKoh1o+3G1CnUoi5qGUyhm0SwFHVtmhQcO003RUsNWSWQVULsctRFBJPZjS4qFLV1wLFxZzbGCgYXSmHKKNNh9Jfn9by3x+O5ejVXve/b7wcO954/99zr9Zx7nvP+Oado5xkDAACAYJyLHgEAABAIAh4AAEBg9lTRFhUVRc8AAADgi2SLu30BL7kAgF3sHwCAQpR2fKKKFgAAIDAEPAAAgMAQ8AAAAAJDwAMAAAgMAQ8AACAwBDwAAIDAEPAAAAACQ8ADAAAIDAEPAAAgMAQ8AChgW1tb5vLly/ZK9WnDzMxMtGR+ufe9fv26Hdf7XLhwwWxsbNjxw2g5vVbreR6PHz8258+fP/bfqc+hz31S3xNQqAh4AOCBgYEBeyui+DA9PW2amppOJbw0Njaa8fFxU1paGk052N27d83q6mo0BuC0EfAAwFMKXQp+k5OT0RQA2EXAAwDPqaRMVaEqyVO16ueff26rb/XcVZGqutRV66rKU1Wfccmq4C+//DKasyutinZkZCSzvAaNu/X09/ebwcFB884772Tey1WXuuXjn89xVbJumW+//Taac7D436fhoFJNzYsvm/w+9Jk+/vjjPcvE15f8rpLzgUJAwAMATyloKNxVVFSY4uJiO02h6qWXXrJVuPfu3bPTFX4UQNbX1+30O3fumNra2kwo0Xp6e3vt883NTbuM1qF1ZaN1Dg0NZdapR40/ePDAvq9KFru7u833339vampqbLjr6OiwpY5aXoM+98WLFzOhUSHrww8/NNeuXcusU+tbWFiw89O4sKXvwX32xcVFc/Xq1dTQpWmffPKJXcZ9jra2NtPT02M/h/suXn311cx8VYVrffp8br4+e9p8oFAQ8ADAUwo/CmEtLS3RFGPq6upMe3t7NLYbmsbGxkxfX1+m/Zyr2lUgUyian583s7OzNri4oHjlyhXT2tpqnycpCCkoxdepR7XRU4hLo3mi9TruuZs3OjpqGhoazKVLl+y41qn3OMjKyop58uTJns+uQPno0SP7dyapOlvr1zKOvr+1tTUbKPV9LC8vm+rq6mju7vel9ek1h80HCgUBDwA8oCrPeJWghps3b9qSqHiQKS8vN2VlZdGYMXNzc3aaSuziFGoUVFxgUbCqrKyM5hpTUlJiqqqqorG99J4KRPH3OYhKvVTKpc8Z76Sh55q2tLRkl3n69Klpbm7OBDXReyi0ZqO/T99Frp9FgdGFRletq44qjvtMnZ2dqVXIh80HCgUBDwA8kNaLNtdSo4mJCRuA4uEwHmoUsJIUslQNmU0ySOYiLaRqmrigmaT30HsdREFUgTQXKnk8d+6cfW+VbCqsKnzGKQAODw/b0lGtV8sqDDpu/tdff506HygEBDwACJyqWl1bufigqlEFqHh1o6OSqYMuc+KqNI8iLaRqUGDKVmKo99B7HcSVRB5Gf5Oqpb/66iv7vgcFZFU1u8+nMKcgqk4kjuZvb29nnQ+cNQIeAARMoSktjCmMqIpRwUjLqA2e2rM52UrUxJXc5RrwVBqoTguut6+j5/oM+ixumampqT3L6D0O6mSR7e9L4/6md999N5qy67DLzCjMKZymlXTKYfOBs0DAA4CAqb1YvJeoqOOF2u91dXXZkrP6+nrbBu/27duZcKULFatqN41KvbROVUu6dbqwlq2qUh0qFK5u3boVTdntJKJpunSKqHOIgqami9Z9WNVn2mfX67TOZImaKyVUiZtbVlW2Kn1TiFRITHut66iidouHzQcKBQEPAAKnKlAFPdcOTx0udKkUTROVnikgiWtT9vPPP9vLnGSjdSogunXqdWqzp+niwpqmK0Spc4JCkZ5reQ2qLtU01/FCwfH+/fs2fGq+1q0erwd1stBn12VZ9N7us+t1+mzJHr1a9saNG+bHH3/MLKsAqTZ4eg+FTfc5FQLd59T3pUu36Pty8/XZ0+YDhaJoRw0IItpQY6MAYtg/AACFKO34RAkeAABAYAh4AAAAgSHgAQAABIaABwAAEBgCHgAAQGAIeAAAAIEh4AEAAASGgAcAABAYAh4AAEBgzm1vb0dPAQAAEIKip0+f7vzhD3/I3JePWzEB6bR/AABQiJL5reiXX37Z2djYMK+88or5v//7PwIekAUnQMDxaN9RbREnScDJSDs+nfvd735nXn75ZfPvf/87mgQAAACf2U4WxcXF5pdffrETAAAA4Dcb8FzxOQAAAPzHZVIAAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAkPAAwAACAwBD4C3ZmZm7GWe3KBxAM/n8ePH5qOPPjK6yxX8RcAD4CWFuatXr5rFxUV7i57p6Wk7roMTgONRqOvp6TE//fRTNAW+IuAB8M7W1pYZGhoybW1tpqamxk5rbGy046Ojo3YcwNHopKmsrMxMTExEU+AzAh4A72xubprl5WXT0tISTdmlcR2kqFoCjkb7TXNzsxkeHrYD/EfAA+Cd9fV1Wy2r0oaktbU1Ox9A7lQCrluWdnR0RFPgOwIeAABAYAh4AAAAgSHgAQAABIaAB8A7anun696ltbUrLy9PbZsHAL8lBDwA3ikpKTFVVVVmcnIymrJL42osXlpaGk0BgN8mAh4A7xQXF5uuri7T39+fuXuFHsfGxkx7e7sdB4DfMgIeAC+ppE53r2hqarLVtXq8c+dO5sLHAPBbVrSji0k98/TpU1NRUWGvLQVgP4UI9g/g6LTv6BpregSQf2nHJ0rwAAAAAkPAAwAACAwBDwAAIDAEPAAAgMAQ8AAAAAJDwAMAAAiMDXjqWnvuHFkPAAAgBDbVbW1tmd///vd2AgAAAPx27r///a/517/+ZV588cVoEgAAAHx2bn193bz88svmhRdeiCYBAADAZ0X/+9//dlz7O24jAwAA4J/krcoy96IV7rV5svh+/cb/Dzge7TvcixY4OWnHJ7rOAgAABIaABwAAEBgCHgAAQGAIeAAAAIEh4AEAAASGgAcAABAYAh4AAEBgCHgAAACBIeABAAAE5tQD3sbGhrlw4YI5f/68efz4cTT1aEZGRszMzEw0lh/uc+V7vaHb2toyly9ftlfR1qDnmgbkm34v9LvhtrXr169Hc37F9ggcnfYlHVeT3HExvj9tbm5Gc1HoTj3gLS4umldeecW8/vrrZnR0NJqaO/3I37x5MxrDWdKBs7e311RUVNhbpLgd/9atW/YRyBcdaHp6esy1a9fstra+vm5PxuIhj+0RODoFu/7+/mjsV9qfvvjiC9PY2Lhnf/r00085afLEqQY8bRRDQ0PmjTfeMB988IH9gdYPN/w0Pz9vZmdnTXt7ux0vLi62B9jp6eljl84CacbHx+2jShOktLTU9PX17fkNYXsEcqfjsUrkOjs7oyl7ZduflpeXzcrKip2GwnaqAU8bhTaYlpYW89Zbb5m1tTVbopeULBbWc03Tj3ltba1ZWFgwTU1N9uzdbaTJ4mXNi1fPuOXcOjWkVfEgd9rRGxoaTGVlZTTF2OevvfaamZubi6YAz6+jo8OGPAU7R9tfHNsjkBsdD11Y08lPa2trNAchOdWApx/Z8vJyG9L0w6sf48nJyWjuLm1s7733nunq6rLFwhr0XD/wep0CYV1dnT0r1xl8LtzG7KpuNGg9Y2Njqe0OkJulpSX7nerMLknzgJOiEz7VBui3wYU+tkcgN9pH7t27t++kKa6+vt4eo0ejplQ6jt6+fdtUVVXtOYlC4Tq1gKeNY2pqytbna4PSBtbc3GxDVrz6RCFQG9WlS5eiKeln70ehksMnT55kiprFBUx++AF/uJL4srIyO+6qbAHklwuBohqvkpISewKlaWknUSg8pxbwkvX5oh9nlei56hMXArOdhR9XTU2NmZiYsI8Kk2+++abdWAcHB6MlAPjAHXRUCq8SfJX2074OyD/tV+q1Xl1dnan5EtdkCoXv1AKeqmLVdk7VrK4NnM7CFbwU6hTuHG1Q+eTO+vWeev+LFy/aHkHd3d3REgB8k6xCApA/2q+StWlXrlyxx1HX6QmF7VQCntL+d999Z4aHhzNnAm5QWzqV7MV75eS72tSVHqrdnd4zrUs4jk5BfHV1dU84d/Id0oHDsD0C+ZWsTVPNl9rg0bTJD6cS8BSs1GNWPWeTVKKmalqdLWhD0gaV7Uf6IPENTq/VOpy03nWuRy+OTzt6Mpy79o5p/2vguJK94kWl8Nq3XXhjewTyixMmv514wNPGod5ub7/9dmrPG3WcUMcLXQJFFy9VGz39SD948CBawth52e584UJh/HpYem28fZ1++DWukjzRZ1JvIFUZHydMYperItN3qe/Qfa+6hI3aOwL5kva7cPfuXfvoOlqwPQL5k7bP6aLhOmGic5MfTjzguZIy/chm6zih6+KpLZ5K+vRD/M0339hQ6NrqXb161dy/f9/O09DW1mbX587oP/vsMxvi1KZPy6s0T9XBjgKkxvUazVcxs3rwDgwM2BIAd4VuHI3+nzqAir5TDaL/B5BP2u/1G6C72LjfBZ2cPXz4MNO7nu0RyJ+0Y7EKUnQsPe4VLXC6inbUKC2if2BsFHnG9+s3/n/A8Wjf2d7eto8A8i/t+HRqvWgBAABwOgh4AAAAgSHgAQAABIaABwAAEBgCHgAAQGAIeAAAAIEpyICnCxbrQoq65g4AAACOhhI8AACAwBDwAAAAAnOmAU83ENfVl90wPT0dzdlP1bXxZZP3ptUty3Trsvgy8Srew+YDAAAEYycmMXpiNjc3d7q7u+2g57K4uLjzLLTtPAt5O+vr6zutra32ueixrq7OLuMMDAzYZbSsW5+mOfHXHDb/tOj7ZWBgYGBgYGDI95B0JiV4Kysr5smTJ6a3t9feIFx0Y+MffvjBNDY22vG4yclJ09bWZpdxWlpazNramnkW8MyzAGeWl5dNdXV1NNfY9Tx69Mi+5rD5p+nZd87g6cD/j4HheIPoXrRp8xgYGJ5/SHMmAW9ubs5WkZaVlUVTDtbX12cHcdW6TU1NdlxKS0ttYOvs7LTVsKqOjUvOV+ADAAAI1Zm1wauqqjIlJSXR2MHi7e/GxsbM4uLivvZ6CoDDw8NmcHDQrlfLKgw68fkvvvjivvkAgOx0+aqnT5/mPGh5AGfnzAKeqkxzKUlTadzQ0JAZGBiwxZAHVat2dHRkiisV5vr7+83IyEg09/D5AIB0//nPf8yrr76a86DlAZydMwl4Kr1z7ecO49rPqc1dnNrlHURhTqFwaWkpmrLXYfMBAAB8dSYBr76+3jQ0NJjbt29n2supOP/999/fV6Km6lYFQpXiuWVVZavSt4WFBRsS3Z0v4q/VJVRUnatgeNh8AACAkJxJwFPP2Xv37pmKiopMezl1uFCpmoY4LXvjxg1bihdvW6c2eHV1dXa6OlEovCkEar6G2tpac+3aNdu54rD5AAAAISnaUYO0iIJPbBR5xvfrN/5/+C1Txwm1rZN//vOf9jHNH//4R/sYX177ji6TokcA+Zd2fDqzThYAAD+5EJeUbTqA00fAAwAcWTLMEe6AwkLAQ96obWSykwyQL8n7Uadd1Fzj8XtOq3MV12M7OS7UEe78k7Y/JS9dlrY/5XL1CxQGAh7yQsFOPZuBk6CDke5eo85VamfiDkS63aELeXrUuGi+llMnKnXcIuSdHMKdf7LtT59++umh+5PuCMX+5AcCHp6LO8PTTg+cFF33UtetdL3e1bteB5/Z2VkzPz9vp+lRverV697d4/rKlSv2UXe/AbAr2/70j3/849D9SSV57E9+IODh2NwZnn4EdF3B1tbWaA6QX/H7UTu6tFJ5eXk0tnvQ0gFLl0Vy9Hx8fDxzIAPwfPvT3//+d/YnTxDwcGw6q9P1DHUAjf8IAKdBpQi6I44OTDrZWF1dNdXV1ba5QLzNENVJ+fHCCy/sudfsYYOWhz9y3Z9og+cPAh4A7+gApAuX6444lZWVto2QSpJdUwG1F6INXn7pJC5+r9nDBk76/OH2p7fffvvQ/Yk2eP4g4AHwzq1bt2z7OzURcO2DRO2K4nfDcW3wVMoMIJ3bn/785z8fuj+pJI/9yQ8EPABe0eV4dB/p+/fvm5qammjqLlUpxbl7WS8tLUVTAMTF9yfdwjOO/clvBDwA3sgW7tyBB0Du2J/CRsAD4IWDSu5UrdTc3GympqZseyLHtSVKlkQAv3XsT+Ej4AEoeOrJl+1g5KiHnw4+Dx48sOM6MH3xxRf2ueYB2PU8+5M6W7A/+YGAB6CgqceeevgtLCzYNkLukg1u0MFK1Gvz4cOHttRB01XNJJpGj05g1/PuTwp87E9+KHqWxnei5/afGBtFnvH9+o3/H3A82ne2t7ftI4D8Szs+UYIHAAAQGAIeAABAYAh4AAAAgSHgAQAABIaABwAAEBgCHgAAQGAIeAAAAIEh4AEAAASGgAcAABAYAh4AAEBg9t2qDAAAAH5J3qpsT8ADAACA/6iiBQAACAwBDwAAIDAEPAAAgMAQ8AAAAAJDwAMAAAgMAQ8AACAwBDwAAIDAEPAAAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAkPAAwAACAwBDwAAIDAEPAAAgMAQ8AAAAAJDwAMAAAgMAQ8AACAwBDwAAIDAEPAAAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAkPAA+C9mZkZU1RUZEZGRqIpu7a2tszly5ftPA3Xr1+P5vwq22sBwGcEPABeU4gbHh42ra2t5sKFC9HUXSsrK2Z2djYa2w1zGxsb0diu2tpa+9qhoaF98wDAVwQ8AF6bn583g4ODprGx0ZSWlkZTd83NzZmFhQXT3d1tQ9zExIRZXFyM5u7Sa/RazRsfH4+mAoDfCHgAvDY5OWl2dnZMdXV1NGWXSvampqbs866uLhviRMsntbS02Ectr9cBgO8IeAC8pSpVVbtKVVWVfXRc9axK7lQN60JcWjVtWVmZqaurM8vLy2ZzczOaCgD+IuAB8Nb6+rpZW1uz4UwhLc5Vzyr4lZSUZNrapVXT6rXl5eWp8wDARwQ8AN5TOIsHvHj1bHNzsykuLs60tZNkNa0CYLIEEAB8RsAD4C2V4KmULinee7azszNzmZT+/n47La2aFgBCQsAD4C3Xdi7JVc9mQ49ZAKEj4AHwntrhqTRP4tWzuj6eetjGh4GBATsv3mNWHSvUwSKtLR8A+IiAB8BblZWVpqGhwZbWuYDnqmcV1t566y07Lc71ptUyWlZcZ41kWz4A8BUBD4C31HmioqLCPlcJnLjq2WxhzfWm1TJaVlxbPtfjFgB8R8AD4DWVyKkDxdLSkh3v6OiwVbFqY5e8s4VomuZpGS0rLhy6HrcA4DsCHgCv1dfX21uRHbdnrGuzp1K95L1sAcBXBDwAXlOJmy6FctyLFLs2e7qdWVqJHwD4qGhH9RQAAAAIBiV4AAAAgSHgAQAABIaABwAAEBgCHgAAQGAIeAAAAIEh4AEAAASGgAcAABAYAh4AAEBgCHgAAACBIeABAAAEhoAHAAAQGAIeAABAYAh4AAAAgSHgAQAABIaABwAAEBgCHgAAQGAIeAAAAIEh4AEAAASGgAcAABAYAh4AAEBgCHgAAACBIeABAAAEhoAHAAAQGAIeAABAYAh4AAAAgSnaeSZ6boqKiqJnAAAA8EUszln7Al5yAQC72D8AAIUo7fhEFS0AAEBgCHgAAACBIeABAAAEhoAHAAAQGAIeAABAYAh4AAAAgSHgAQAABIaABwAAEBgCHgAAQGAIeABQwLa2tszly5ftlerThpmZmWjJ/HLve/36dTuu97lw4YLZ2Niw44fRcnqt1vM8Hj9+bM6fP3/sv1OfQ5/7pL4noFAR8ADAAwMDA/ZWRPFhenraNDU1nUp4aWxsNOPj46a0tDSacrC7d++a1dXVaAzAaSPgAYCnFLoU/CYnJ6MpALCLgAcAnlNJmapCVZKnatXPP//cVt/quasiVXWpq9ZVlaeqPuOSVcFffvllNGdXWhXtyMhIZnkNGnfr6e/vN4ODg+add97JvJerLnXLxz+f46pk3TLffvttNOdg8b9Pw0GlmpoXXzb5fegzffzxx3uWia8v+V0l5wOFgIAHAJ5S0FC4q6ioMMXFxXaaQtVLL71kq3Dv3btnpyv8KICsr6/b6Xfu3DG1tbWZUKL19Pb22uebm5t2Ga1D68pG6xwaGsqsU48af/DggX1flSx2d3eb77//3tTU1Nhw19HRYUsdtbwGfe6LFy9mQqNC1ocffmiuXbuWWafWt7CwYOencWFL34P77IuLi+bq1aupoUvTPvnkE7uM+xxtbW2mp6fHfg73Xbz66quZ+aoK1/r0+dx8ffa0+UChIOABgKcUfhTCWlpaoinG1NXVmfb29mhsNzSNjY2Zvr6+TPs5V7WrQKZQND8/b2ZnZ21wcUHxypUrprW11T5PUhBSUIqvU49qo6cQl0bzROt13HM3b3R01DQ0NJhLly7Zca1T73GQlZUV8+TJkz2fXYHy0aNH9u9MUnW21q9lHH1/a2trNlDq+1heXjbV1dXR3N3vS+vTaw6bDxQKAh4AeEBVnvEqQQ03b960JVHxIFNeXm7KysqiMWPm5ubsNJXYxSnUKKi4wKJgVVlZGc01pqSkxFRVVUVje+k9FYji73MQlXqplEufM95JQ881bWlpyS7z9OlT09zcnAlqovdQaM1Gf5++i1w/iwKjC42uWlcdVRz3mTo7O1OrkA+bDxQKAh4AeCCtF22upUYTExM2AMXDYTzUKGAlKWSpGjKbZJDMRVpI1TRxQTNJ76H3OoiCqAJpLlTyeO7cOfveKtlUWFX4jFMAHB4etqWjWq+WVRh03Pyvv/46dT5QCAh4ABA4VbW6tnLxQVWjClDx6kZHJVMHXebEVWkeRVpI1aDAlK3EUO+h9zqIK4k8jP4mVUt/9dVX9n0PCsiqanafT2FOQVSdSBzN397ezjofOGsEPAAImEJTWhhTGFEVo4KRllEbPLVnc7KVqIkrucs14Kk0UJ0WXG9fR8/1GfRZ3DJTU1N7ltF7HNTJItvfl8b9Te+++240Zddhl5lRmFM4TSvplMPmA2eBgAcAAVN7sXgvUVHHC7Xf6+rqsiVn9fX1tg3e7du3M+FKFypW1W4alXppnaqWdOt0YS1bVaU6VChc3bp1K5qy20lE03TpFFHnEAVNTRet+7Cqz7TPrtdpnckSNVdKqBI3t6yqbFX6phCpkJj2WtdRRe0WD5sPFAoCHgAETlWgCnquHZ46XOhSKZomKj1TQBLXpuznn3+2lznJRutUQHTr1OvUZk/TxYU1TVeIUucEhSI91/IaVF2qaa7jhYLj/fv3bfjUfK1bPV4P6mShz67Lsui93WfX6/TZkj16teyNGzfMjz/+mFlWAVJt8PQeCpvucyoEus+p70uXbtH35ebrs6fNBwpF0Y4aEES0ocZGAcSwfwAAClHa8YkSPAAAgMAQ8AAAAAJDwAMAAAgMAQ8AACAwBDwAAIDAEPAAAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAsOtyoAcaf8AAKAQJfMbAQ/IEfsHcDzad7a3tzlJAk5I2vGJKloAAIDAEPAAAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAkPAAwAACAwBDwAAIDAEPAAAgMAQ8AB4a2Zmxl7B3Q0aB/B8Hj9+bD766COzsbERTYGPCHgAvKQwd/XqVbO4uGhv0TM9PW3HdXACcDwKdT09Peann36KpsBXBDwA3tna2jJDQ0Omra3N1NTU2GmNjY12fHR01I4DOBqdNJWVlZmJiYloCnxGwAPgnc3NTbO8vGxaWlqiKbs0roMUVUvA0Wi/aW5uNsPDw3aA/wh4ALyzvr5uq2VV2pC0trZm5wPInUrAt7e3TUdHRzQFviPgAQAABIaABwAAEBgCHgAAQGAIeAC8o7Z3uu5dWlu78vLy1LZ5APBbQsAD4J2SkhJTVVVlJicnoym7NK7G4qWlpdEUAPhtIuAB8E5xcbHp6uoy/f39mbtX6HFsbMy0t7fbcQD4LSPgAfCSSup094qmpiZbXavHO3fuZC58DAC/ZUU7uphURD+SsVEAMewfwPFo39E11vQIIP/Sjk+U4AEAAASGgAcAABAYAh4AAEBgCHgAAACBIeABAAAEhoAHAAAQGAIeAABAYAh4AAAAgSHgAQAABIaABwAAEJh9tyoDAACAX5K3KuNetKeI79dv/P+A49G+w71ogZOTdnyiihYAACAwBDwAAIDAEPAAAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAkPAAwAACAwBDwAAIDCnHvA2NjbMhQsXzPnz583jx4+jqUczMjJiZmZmorH8cJ8r3+sN3dbWlrl8+bK9irYGPdc04CRdv37d/g4kJbdH7dPatwHspeOvjsNuX9E+leSOiwctg8J16gFvcXHRvPLKK+b11183o6Oj0dTcaaO8efNmNIazpINpb2+vqaiosLdI2dzctNNv3bplH4GToGDX398fjf3KbY+ibVHbZGNjo+no6CDkATHaH3p6esy1a9fsfrK+vm4LN+IBTsto39E+pGXcbbAIef441YCnH+ChoSHzxhtvmA8++MBuUPzw+mt+ft7Mzs6a9vZ2O15cXGwPsNPT08cunQWycaVznZ2d0ZS9tD0uLy+bGzdu2G1Rrly5Yh91Yglg1/j4uH1U6ZyUlpaavr6+Pcdkt4zbh0TPv/vuO2q6PHGqAW9lZcUGgpaWFvPWW2+ZtbW11B9ebWDxYmFXzaKNqra21iwsLJimpiZ7JuF+9JPVNZoXry50y7l1Utz8/HQwbWhoMJWVldEUY5+/9tprZm5uLpoCPD/tvzp50Dank4fW1tZozq8mJydtaYMOVo6e60Cl6QB2qWRO+0V8X9G+Fbe0tLRvfyopKTFVVVX7lkVhOtWAp4N+eXm5DWkKAgoH+lGO04/3e++9Z7q6ujLFwnquDVKvUyCsq6uzpUQ648iFOzi4qkQNWs/Y2FhqOx7kRj8A+k5daUmc5gH5om3s3r17+w5Kjvbx1dVVU11dbffp5MkhgOy0j6h2TcfatP0rid93P5xawNMP8NTUVOaMQD/Yzc3NNmTFq/MUAhX8Ll26FE1JP9s4CpUcPnnyJFOVKC5gsqEC/lObO5UquOpbdyKn3xva4AHpXM1WWVmZHY+XjOtkKdmMyu1n8MOpBbxkey3R2bVK9Fx1nguB2UqFjqumpsZMTEzYR4XJN9980xY1Dw4ORksACMHAwIANdI5rP+TaEwH4lSsZ18mQasT+9Kc/ZQpcXDOqu3fv2nHRc/Ylf5xawFNVrNrOqZrVVZ/orEHBS6FO4c7RmUM+ubMUvafe/+LFi/ZMpLu7O1oCQAiSvx2uzRAl9cDB6uvrba3WaHR1CxWI3L9/39ayuWO25nPc9MepBDwV8arnzfDwcKbqxA1qS6eSPVWjOvn+MXalh2p3p/dMu8QCjk4HU7V7iodzJ98hHTiIC3IA8kch79GjR5njtULgjz/+yO+7J04l4ClYqahXRb5JKlFTNa3OGlRcrOrZbKHhIPFQqNdqHY7aDOjMI97b0/XoxfHpgJoM5669Y9r/Gjgprk1vsjbAtRnigAT8KnmVCUnuK+qslOykdNCxHIXnxAOeNiD1znn77bf3BCxHHSfUEFqNOXWxRbXRU2h48OBBtISx87Ld+cKFwnhjUL023r5OQUTjKskTfabbt2/bKuPjhEnsckX6+i71HbrvVZew0ZkfcJp0MNIByv12aHv84osv7HPNA7Ar7Tir9nUqpXP7imuD59rc6fiqYKgOkPy+++HEA54rKdNBP1vHCV0XT23xdHagDeebb76xodDV+1+9etW2BdA8DW1tbXZ97gzks88+syFObfq0vErzVB3sKEBqXK/RfFXn6GxfDbJ1QNCZC45O/08FOtF3qkH+8pe/2EfgNOlk8eHDh7YUz+3nomnH7YEPhEjHUR1TdVcod5xVYYcCn9tXksvo+KrLqOR6eTKcvaJniX33/iPP6J8YG0We8f36jf8fcDzad7a3t+0jgPxLOz6dWi9aAAAAnA4CHgAAQGAIeAAAAIEh4AEAAASGgAcAABAYAh4AAEBgCjLg6YKKutiiLl4MAACAo6EEDwAAIDAEPAAAgMCcacDTfe109WU3TE9PR3P2U3VtfNnkvWl1yzLduiy+TLyK97D5AAAAwdiJSYyemM3NzZ3u7m476LksLi7uPAttO89C3s76+vpOa2urfS56rKurs8s4AwMDdhkt69anaU78NYfNPy36fhkYGBgYGBgY8j0knUkJ3srKinny5Inp7e21N6wX3dj4hx9+MI2NjXY8bnJy0rS1tdllnJaWFrO2tmaeBTzzLMCZ5eVlU11dHc01dj2PHj2yrzls/ml69p0zeDrw/2NgON4guhdt2jwGBobnH9KcScCbm5uzVaRlZWXRlIP19fXZQVy1blNTkx2X0tJSG9g6OzttNayqY+OS8xX4AAAAQnVmbfCqqqpMSUlJNHawePu7sbExs7i4uK+9ngLg8PCwGRwctOvVsgqDTnz+iy++uG8+AABAKM4s4KnKNJeSNJXGDQ0NmYGBAVsMeVC1akdHR6a4UmGuv7/fjIyMRHMPnw8AABCCMwl4Kr1z7ecO49rPqc1dnNrlHURhTqFwaWkpmrLXYfMBAAB8dSYBr76+3jQ0NJjbt29n2svp7hXvv//+vhI1VbcqEKoUzy2rKluVvi0sLNiQ6O58EX+tLqGi6lwFw8PmAwAAhORMAp56zt67d89UVFRk2supw4VK1TTEadkbN27YUrx42zq1waurq7PT1YlC4U0hUPM11NbWmmvXrtnOFYfNBwAACEnRjhqkRRR8YqPIM75fv/H/A45H+44uk6JHAPmXdnw6s04WAAAAOBkEPAAAgMAQ8JA3ahuZ7CQD5Is6Ruke1KqK0HDYdSzZHoHc5LKvuP0veQ1aFC4CHvJCPw7q2QycBPWE7+npsR2j1M5EvefVmz5byGN7BHKTy76iK1joqhe6cgX8QcDDc9GOr9u/6TZwwEkZHx+3j7rckahnvO5Oo5Cn8OewPQK5Ocq+8uDBA3sXKPiFgIdj0w9Eb2+vvVSNiu9bW1ujOUB+6fJJCnkKdo62uzi2RyA3R9lXNP+vf/2r+dvf/mYvTQZ/EPBwbO56hskDL3DSVGqn61p2dXVltj22RyA3ue4rCoKqmm1qarLXjoVfCHgAvOGqlXRhdHFVtgDyb35+3pbyXblyJZoCnxDwAHjDlTyoo4Xa4L333nu2CglAfqmUXJ2YtJ9RIu4nAh4AL7l7Wo+OjkZTAOTL3bt37a08uZ2nvwh4AAAgQ6V36qGuy6e4606qDZ4uk9Lc3HzoNShRGAh4AAqeDihqe6c2eM7m5qZtH1RdXR1NAZAPqpJVBww1hXDD4uKi7UU7NTVlq21R+Ah4AApee3u7mZ2dtdfjclSFJHS0AID9CHgACl5NTY25f/++uXnzZqbKaHV11Tx8+JAG4ACQomhHZa8R/WjGRpFnfL9+4/8HHI/2ne3tbfsIIP/Sjk+U4AEAAASGgAcAABAYAh4AAEBgCHgAAACBIeABAAAEhoAHAAAQGAIeAABAYAh4AAAAgSHgAQAABIaABwAAEJh9tyoDAACAX5K3KtsT8AAAAOA/qmgBAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAkPAAwAACAwBDwAAIDAEPAAAgMAQ8AAAAAJDwAMAAAgMAQ8AACAwBDwAAIDAEPAAAAACQ8ADAAAIDAEPAAAgMAQ8AACAwBDwAAAAAkPAAwAACAwBDwAAIDAEPAAAgMAQ8AAAAAJDwAMAAAgMAQ8AACAwBDwAAIDAEPAAeG9mZsYUFRWZkZERO379+nU7nm3Q8k7ytQAQAgIeAK9tbW2Z4eFh09raai5cuBBNPVhTU1Mm5NXW1trXDg0NmfX1dTsNAHxHwAPgtfn5eTM4OGgaGxtNaWlpNHXXwMCA2dnZyQybm5umu7vbzpucnLSPeo1eOzExYcbHx+00APAdAQ+A1xTUFN6qq6ujKdkVFxeb5uZm+3x1ddWW/klLS4t9nJ6ezkwDAJ8R8AB4a2NjI1PVWlVVZR8PovA2NTVlnyvoKfBJWVmZqaurM8vLy7aUDwB8R8AD4C21mVtbW7PhTCEtqb+/f0/nipKSEludq6rbjo6OaKndgFdeXm6raRcXF6OpAOAvAh4A7ymcpQW8bBT84j1pFfxyKQEEAF8Q8AB4SyV4CwsL0dh+yU4WGtTjVnQpFVXxAkCICHgAvOXazh2FLqWiy6KoapfLogAIFQEPgPeOEtZ0KRS1tYtTxwp1sMjWlg8AfEPAA+CtyspK09DQYKtp0wJespOFhs7OTjuvra3N1NTU2Oeus8ZR2/IBQKEi4AHwli5zUlFRYZ+rBC5XapvX19cXjf3alk8dLdThAgB8R8AD4DVdpFglc0tLS9EUY8NbsnNFfIiHO3HhULcwc9fGAwCfEfAAeK2+vt7efkyXPTlOr1h38eOj3MsWAAodAQ+A11TipnZ1x71I8crKipmdnTVdXV20vwMQjKId1VcAAAAgGJTgAQAABIaABwAAEBRj/h/ABTdoh38cZwAAAABJRU5ErkJggg==)\n", "\n", "For each of the two models,\n", "\n", "1. compute true positive rate (TPR), false negative rate (FNR), false positive rate (FPR), and true negative rate (TNR); and\n", "2. compute accuracy, precision, recall and F-1 score.\n", "\n", "You can do it either by hand or by code (do **not** use existing functions). Either way, show details of your calculations in a new cell using variables TP, FN, FP, and TN. Round your answers to four digits if you choose to use decimals." ], "metadata": { "id": "hKBkTdeZ_6xz" } }, { "cell_type": "markdown", "source": [ "### Q2.2\n", "\n", "The two ROC curves are labeled as 1 and 2 in the following figure.\n", "\n", "![Screenshot 2023-09-02 183858.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAa0AAAGcCAYAAABqcP8/AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAFX9SURBVHhe7d0HvI7l/wfwD2c4E+dwbHLsvQsR7UR7/4pSfmgPlShNTQ3t5UdLkyRJOymEZGTL3vMMZ+///bmv+/SXnJzx7Ofzfr2eF+77IR3Peb7PdV3fUanIAhERET9Q2flRRETE5yloiYiI3/DZoJWRkYFFixZh27ZtzhUREQl2Phm0cnNzMWfOHDz33HPYtGmTc1VERIKdzwUtrrDee+89PProo1i+fLlzVURExMeCVk5Ojh2wfvzxR5x00klITEx07oiIiPhY0AoNDUX37t1x22234aKLLkJCQoJzR0RExMeCVkhICDp37oxu3bohLCzMuSoiImL4ZHFxYWEhli1bhhdeeAHXXnstTj75ZOeO64wfPx4fffSR8ysREfGuGtajAZohAVXqrcakzz4zl48QtEFrxIgRiI+PxyWXXOJcERERd9uwAViyBNi5E1i92vy4ezfvhFoBKQShOXvRJvE7LNh0v/38IwV10Grbti2GDBniXBEREXf77jvgwQeBlStZ3gQUFJgHRSEFeSGbUL9+PpYu74pq1ULMjcP4ZJ2WiIgEplNPBd56Cxg4EKhenYsUgEsnPgrDdiM3vxAZWQVYuDDD+R1/p6AlIiIeE2Itnlq0AC6+GKiVAISFFKKSdT08fAeKKmdbP6uEjIwIfP99mv38I/ls0GImYWRkpJ0GLyIi/o+rquRk4PnngaceL8DNJ/6OBrGp1p0DKCg4iOxsK4BZESwrq8heafH5R/LJoFW5cmV07NgRr7/+Onr37u1cFRERf5VtLaLmzwduvRVYtw54/aY/MGzFLYjK2Gzd3Wm97xegevUQREczLBXi0KECrFmTZf/ew2l7UERE3CY/H9i3D5g2DXj5ZaBb5wKMfbgQTS7qDLzyCmp1CUez5mE4++xq6NevGho3DkNR0W7r9+Rh7tx/nmspaImIiFtkZgK//QY88wzw88/WKuu/Wbj5pOVIyNtl9go7d8azrzXFN9+0xPTpTVGjRgiuuioKYWFvoH79cKxerZWWiIi4GVPY9+wBvv4amDQJiIoC7rk5AydmfIuQCa+bfHcuwSwdO0YiMTHEPstasiTT+nUYatVahcmTG+P66//Zyk9BS0REXIa1V7//Drz2GvDjj8B55wEjRwKJoduB2bOBTp2AHj2YLuj8DiMjoxBbt+aiVasIa6VVCS1aRKB16wjn7v9T0BIREZdgZuBXXwHvvgukpsJaKQHnnmtWWoiNBa65BrjiClOgdYQ1a7JRv37YUQuKD6egJSIiFcYdv1deAX74AWDS9wMPAO0SM0yvpq1bYUUk+wwL8fHO7/i7pUsz0KVLtPOrkiloiYhIufH86ssvTcBKSjKdLriYio/IBL7/3rS/MM0F/9WSJVno2pVLsn+noCUiIuWyeTMnZpigdeKJwF13ASec4NxkoJo7F2jUCGje3LlYsqVLM62VloKWiIi4wU8/AY89BuzdC1x5pVlh1avn3CSeW7FX01VXATU4dqRkKSkF2L8/Hy1b/jPx4kgKWiIiUmoHDgBvvglwHGG3bsANN5gzLKasIy8P+OMPs0/IQMUswRLOsA7HJIxGjcKdbhj/TkFLRERKZdkyYNQoE5cuu8wsopo0cW6yWHjOHGD0aGDLFudi6SxfnolOnSKdX/07BS0REflXrL36/HPgxReBpk1NKvtJJ5ks9r+w5QUD1vnnA82aORdLZ/nyLCtoRZnV2jEoaImISIm2bweeeML0DuSgdwasNm2AsDDnCcU6dDB7hVx+Va3qXCydZcu40jp2EgYpaImIyFEtXQrcf79JtrjtNuD004G4OCtwHB45fvnFTHDk2dV//gNEH7vW6nD79uUjM7MQiYl/75BREgUtERH5m/R04IMPgEcfNbkUjzxiui8d0XnJBKzLLzdNBimydOdSh1u1KgvNmlWx/uxS7A1aFLREROQvbHR7333ASy8BN90EDB8O1Kx5xOqKWIPFKuKnnwbOOsu5WHYrVmShbdtI689X0BIRkVJitvqKFcDtt5uff/ghcOqpTir74bgVSMxz53KMaYT/iGilt3JlFtq0ibDH8JeGgpaISJBjo1s2ueVU4TPPBF59FWjc2Ll5OI4TYS9B/gYGr759j5KRUXqFhUXWH5dtBS2ttERE5BgYg3bsMNmBXFmNHQtcd51z80jMe+doER5wcW4+f11B27blIjQUqFMntFTp7qSgJSIShFJSTEf2O++EHTjY4YI7fiVav95MdOzTxxRpVani3Cg/rrI4NyssrPShSEFLRCTIcFLI228DTz1laoEff9wkW/wrprIz752NBstYh1WSVauy0bx56TMHSUFLRCRIZGQAixcDr78OrF1rEv8Yg0qUmWlaMrHhYGKiyX9noZaLrF6dZQUtM6m4tBS0RESCAFPZP/7YBKxatcw5Vteuzs2jYcBiK/d33jEzSFyM4/V37cpD48ZVFLRERMRgkt+aNSY7kP1sWVLF2qtjLpi2bTOHXtwWPGoqYcVs2pSD6tVDrL9HSKmTMEhBS0QkQB06ZIYHT55sMgVZg8UzrKjStPljd4v+/YFrrgESEpyLrrN2bTaOO66K9Z8pWxhS0BIRCUC7dpnV1dSpZqE0ZAjQufNRWjEdLicH2LDB1GEddxxw2mluCVjEoNW4cTgiIsqwzLIoaImIBBCOteJ2ILPTN24EBgwwTStq13aeUBLWXXG8yDPPADt3Ohfdx6y0wrXSEhEJVswOZO9aJltwd++//zVnWNWqOU8oSUGBKRjmwKyWLYG6dZ0b7sHO7jk5RahTJwyhoVppiYgEHe7ocXXFLUFuA3KsVdu2pawBZiZEgwYmwvEMi6Py3ejPP7NRq1YoqlatXKYkDFLQEhHxY8wO5KDGceNM7dXQocBFF7E1kvOEY1m40IoEVijg3PxBg8xcLDdj0KpXLwwxMaXsknsYBS0RET/F8yu2A2RWIJtUjBpluiyVumHFr7+aldWCBSZwHXMf0TXWr89B3boMWmUPQQpaIiJ+6o03gJEjgQsuMJPuGzYsQ9N1nmFxD/Huu82ofA/JySnEli25aNAgHNHRWmmJiAQ8JlzceCPw1VfAm2+a4cHVqzs3S6tbN9N0cODAUhZuucbWrXmIja1sFxWXZwyXgpaIiB/57TdTPhURAUyZUoraqyOtWwdkZ5vfdOmlLunWXhabNmUjPj7U7oZRHgpaIiJ+gFnp771n4gyTLZ591sSbUmffMWODW4IcmPXNN2Y8cWnHBbvQxo25qFGDmYMKWiIiAYfBirOvHnvMdLfg3KvBg8sQrIotXWoOwNjW/fTTKzRxuLwYNzdvzkHNmqGoVk1BS0QkoHAXj9uBHGOVlGQSLzgdpFwLJDa+ffRRk9bOn3vBgQP5dnf32rVDy9wJo5iCloiID2Kz21mzzHT71q2B++8vQ+1VMfYS3L/fjBlhp4uTT3bZAMfy2LIlx07C4PZgeSloiYj4EHZjZ+s/bgMyO3DYMOCWW8rRpILLtLlzTXohq459AFPdY2NDFLRERAJBVpbZDnz5ZRbgAjffbEaJlGs3j91yP/0UCLUCBAu4fABXWiwojo8vfwKIgpaIiA/gduB335n+gcyRYB1Wx47lSLgoxnYZbPF+7bVuGy9SFllZhXaj3Li48mcOkoKWiIgXMbZw9tW0aWZgY69ewK23mlaAZcYtwR07TARs394ELc7W9wHbt+faxcRs3xQSUt5IrKAlIuI1PL/64w9gwgRg9WrT6JadlWrWdJ5QFky6mDcPmDjRREEfw6BVpUplu7t7RShoiYh4AQPWDz8Ar7xifs6+tUzuK1f5FAuFFy82c0n4B8TFOTd8x/bteVbQqqSgJSLib1gw/P77Jk+iUyfg+uvN7Kty48EX22OwpxMnPx5zTLFnsUnurl259igSFhZXhIKWiIgH8ciJtVdcGF1xheluUb++c7OseCDGfUVmCDJrg8s1HznDOtyBAwXIzi6yYmmYvUVYEQpaIiIewrFVzArk8RPrr049tYLNKTjAkaNFVq3y2W1B2r07z/6xbt2KrbJIQUtExAOYHcjpwn37mvorJvdVSHF/J7Z8r1vXueibuDVI9eqVpR390SloiYi4EWdfPf888PnnZnXFI6cGDZybFdG8uUk1HDLEIyPyK2LXLrPS4oj9ilLQEhFxEzaluOsu093izjvNoqjCE+1XrDDphpz6yIDloRH55ZWUlI+UlAL7PCsqquIhR0FLRMQNfv3VbAOyPy0DF7MDKzwNZNEiE6jmzDFzPmJinBu+a+/efGRnF6JRo7Dyd/c4jIKWiIiLzZgBPPUUcN55Jk8iMdEF8xYZsDjAkW2ZTjyxAv2dPGvv3jzk5BShYcOKn2eRgpaIiIukpZlmt2+9ZTqzDx9uulu4JL4wzfCZZ0xae2Skc9H37dnDoMWVloKWiIhPYLnUpk3AQw8Bv/wCjBljzq/Ya69C+Afn5pofub/Yrx8QFeXc9H3p6f/fJDc+vuLp7qSgJSJSAcyJWLYMGDvW9KnlcOCuXZ2bFcE/mIVdjz9uCoj9ECcVp6UVoEmTKs6VilPQEhEpB+ZBcCAwe9RyS7BNG+DJJ00mukswUL34omnP5ON1WCVh0Dp0qFBBS0TEm7hbt3cvMGWKaap+9tkm4aLM04X/DZdtl19uCrtc+gd7BoP6/v15SE8vQGKia86zSEFLRKQM+Ga8YQPwxhtmHD6T+S691LlZUezvdPCgWcL17g1ceKFPDHAsD6a5M909IqIy6tSpeFFxMQUtEZEyWLIEeO01YOtWk3BxyinOjYpiwOIZ1ocfAjt3Ohf9FwuKmYTRooXrtgZJQUtEpBQ4FJiThTmwkUdMTzwBtGvn3KwozsNavtzMK9m/3y+Kho8lObnA3h5s3jzCueIaCloiIsfA86t33jExpU8f05LJpSOreEjGIq9WrcxwLT9NvDhcSkq+FbS00hIR8ah160xNL6eADBwIXHmlC7pbFOMKa9s2kyHYq5c5IAuAgJWbW4Q9e/IRHl7JpedZpKAlIlICBiqeX3FrkB0uOP/KZTi+mONFnn7aHJBFRPjsPKyyOnSowIrFudbCMQKhoa5tN6WgJSJyBNb1MjOQ24HNmpmGtxyL77J2f0xB/P1306CQ2YEMWAGEBcXbt+eidWvXt5tS0BIROUxysjm/mjXLnF9xS7BRIxf3p+UfxqaExx8P3HCDiw/IvI8FxQxabdq4PhgraImIOHi89NxzpqH6uecC/fubsVUuDVhLl5of2fqdActP67BKkpdXhJ07c1GlSiWXDH08koKWiIiFXZOefRZITTX5EFxlubw3LbcEhw41h2WMhH7Y6eJYMjMLsXFjjn2eFRbmymhvKGiJSNBjwTD7B3LRw8GNbHjr8mOmxYvNvH2OyGejwgDFoLVhQw7at3fP+BQFLREJWpz68d13pi9tt24mQ5CJFxWeMHw0nId1221m8nBsrHMx8GRlFeLPP7PRsaN7RqgoaIlIUGItL7MDX3rJnF9xAVStmvWm6K53xRYtgCuuMPP3A1R+fhG2bOF5VmWXTSo+koKWiASdlBRg/Hjg1VeBESNMX1rW97ocky5GjgQ2bjQVyeHueSP3FSwqXrUqy1plRbot+CtoiUjQYHnUgQNmYCMzBJnafvLJblpdsZfg7bebguEATLg4mpycIqxcmY0uXaKdK66noCUiQYHt/fbsAe65xzRUf/ttN+dDsFP7TTeZ5AvmzQcBjiNZsyYLnTq5JwmDFLREJOAxSHGnjmVRTLRgpiBre12OrTQyMkxPQRZ5XXZZ0Kyy+KGAWYNRUZXRqJH7tkEVtEQkoLHuato0c3Z1wQXA6NHODVdjKiJ7CX76qalSDjIFBUVYvDgTJ5zgvq1BUtASkYDF7cC33gKef94Eq8GDnRuuxua3q1YBkyYBa9e6oSrZ95mglaGgJSJSHps2AW++aYYBM0uwXz/nhjtwlbVlC9CypalODoDxImWVkVGI9etz0LWrewO2gpaIBBQuepi4xwnDPMvihOEuXZybrsZgdfAgEBkJnHWWSbqoV8+5GVxWrcpGXFxlK167ozL7/yloiUjAYA7E7Nkmlb1WLTNShH1pXdrwthiTLdj/iVuCHJHPLcEALhw+lrlz09GrV4zzK/dR0BKRgMAFD3Mgpk4F2rc3TW/dNlORWYJMR+SESP6HueIKcr/8wqDl/vZUCloi4vd27QLefReYM8dMF774Yg+URjHHm1uBt94K1K/vXAxOSUn52LIlB126uD8BRUFLRPza5s0mYLFT0qWXAgMGuHmXbs0aIDTUtIJnwArSM6zD/fZbJho3Dkd8fIhzxX0UtETEb61fb86vsrLMduApp5hm6m7zxx/AnXea9Ha2gg/CLMGjmT07DSef7JnO9QpaIuJ3eKTEHAhmCHLKB+uvOnVyU9PbYsuWATfeCJxwgsnykL/88MMh9O2roCUi8g9MY//pJ1N7xcxAjhThj2yi7lbs0M7WTBy6FWAj8iti+/ZcpKQUoEMH9/UbPJyCloj4DW4DfvaZ6XDBcfj/+Q9Qp45z092aNjUrrSDpJVhav/6aYXd1Z89BT1DQEhG/wIDFzuzMMh861GQIui2lvRi3BO+7z6S1c+8xSLq1l8WcOWk45RTPTWJW0BIRn8c6Xg5tnD4dePJJ4Oyz3ZxwQWyrwZZMXFlFRDgX5UgKWiIiR2DC3u+/A//7H9C9u4cGADM18frrTZZHEDbALQ1OKY6ODkGTJp6byKygJSI+KzMTuOgiID3ddLto2NB603L3uxbHGxOLvgYOBOLj3dQHyv/Nm5eOnj2jEBLiua+PgpaI+Bw2vWUNL+dftW5t2vu5HTtc8Azr44+BrVudi/JvTL/BWI/GdAUtER/CgYVcXQQzprSz6e3w4cA55wBjxzo33ImrKxYMjxtnApfOsI4pN7cIS5eaoY8KWiJBiAHrgQeAr792LgShtDSTbMGvw5AhZjy+27cDiamJnDrcubNpzVS7tnNDSrJgQQZatIjwSOumwyloiXgZd6U4P5C1R2+84VwMQswq587cBx8Ao0YBV1xhOiW5FdMSOc+EiRaXX27qsNRLsFR+/TUd3bpFW/9Gnj3vU9AS8SIGLJ7dcFDhjz8Gb93q7t3A++8DCxcCI0aYKcNubclEDFjcCpwxwyzxmEPv9jz6wMDR+myS261blIKWSDBhwgF7sDZubApnuTsVTHiUxFXm5MnAzp2mLKp3bw+ktDNgsQ6LOfSLFgEpKc4NKY1du/KQlJSHli0jPJo5SApaIl7E7S+2Iho9OviG3jJuMPeBAYtHSuxy0bGjB3oIEjvu7thhtgXvuMPk0kupLVqUgXbtIj3WuulwCloi4nHZ2WYrkHOwOJpq0CCgWTPnpjtxP3b7diAyEjjzTLMX2aiRc1NKi1uDbdtGWl9GBS0RCXDMe+D5Hc+wOI6quEu723EvcuVK4KmngG3bzCpLK6wyS0srxIYN2WjTJhIREZ7dGiQFLRHxGHa2mDkT+PBDoF07szXqsbixejXw8MOmBssje5CBad26bLt1U926YR4/zyIFLRHxCCboTZsGzJoFnHaah8eKELNeOF6EW4L16zsXpayWLMlEs2ZVULWqdwK/gpaIuB1XWEy4mDMHuOQSM1aELf08gkkX1KKFCViqw6qQ4qAVG+ud8KGgJSJuxaQLThlmSRQzBJn/wBH5HrFiBfDYY+YgjduCHl3aBR5OKc7MLEDjxuHWl1NBSySoVasGPP000LevcyEAcJHDYyT2n737buD44z1QNFyMAev2280X1u2tNYLDihVZqF07DAkJYV5rfK+gJeIjmPrNjuaB0hUjOdmsrLjSGjPGpLR7NP/hzz/NPiQPzxS0XGL58iw0bBiOuDjvJbIoaImIy7HLBUdRMaWdzW/5o8exFxQHONaqpXlYLpCRUYj167ORmFjFClrWJywvUdASEZdiWyo2u2U7pvvvh/UG59zwBLbY+OQT08yQdVh8KGC5xNq12XbGYJ06oZ7pvF8CBS0RcZl588yOHAuG77rLNJ7wGHYe5lx+ZnxoO9DlOFq/Zs1Q+zzLmxS0RKTCWALF+is2vL3vPmDYMA/HDTYv5F/g1FOB226D9e7q3BBXYDORNWuyUatWqP3wJgUtEakQJlqwaJjzwO6912wNeixDkNGSKYpMZ+csLGYLaoCjy23ZkoPMzEIcd1wVrzTJPZyCloiUG4uGp0wxQYvx4rzzPDipnsGKZ1isWM7MNHuRbp9pEpzWr89BbGwI6tXz/rargpaIlAtHUHHS8C+/ANdfD5xxhgdXWMUB65VXgM8+M2OPxS3YGJ/9BqtWraygJSL+af9+E7BYv8tarD59PHyGxQ4XzPpg4RerljVexG1SUvKxc2cu6tQJsxMxvE1BS0RKjZ+6OTvxo4+ADRuAa64xXS48llXOFVZSkulywaLhkSOB445zboo7bNiQY3dz53mWL1DQEpFSYc7Dxo1mDtauXWZwY+fOzk1P4F+Aae0TJwIHDpgCsMaNnZviLhs35lir6Ep2v0FfoKAlIsfEBQ7HUXHSMEeMsNtFhw7OTU8wOdfAiy8CmzebDBBxO3bB2L49z1rYsqjY++dZpKAlIv8qLw9YutQscLgNyC3Btm2dm57CoJWaarID77lHKywPYVf39PQCNGlSBVWq+EZnEQUtESkRV1iLFgEvv2x24667Dmje3LnpKSwcZt+gLl1MwNIZlsds3ZprfWgpsudn+QoFLRE5KiZdMJ19/HgzP5FZgh5f4HBLkFXLxXVYyhL0mJycIruomCssX0nCIAUtETmq778HXnjBNL5lWyaPD/zlIRrT2YlzW8Sj9u/Pw4ED+dbnhHCvTSk+GgUtEfmHzz83AevCC810j4QE54YnrVsHdOsGXHutOl14wY4deTh0qBCtW3uy6/GxKWiJyN98+CHw2mvA8OHAJZcA1as7NzytVy/glls0It8LuDXM86y0tAK0auWpvlylo6AlIn956y0TtNip/ayzgOho54ancEuQbZnYI4rDGwNljLOfSUrKx65defbAx+rVvTel+GgUtETExtXVl18CY8cCPXt6sI9gMSZdMDuQFcwa3OhVe/bk2Y+OHSN97p9CQUtE7AS9n38GnngCaN/eC3kPzK3/4gvTxJDtmapWdW6IN+zdm2c/2rf3rfMsUtASCXJPPw38/jvw6KNAs2bWm4I33hUYJTlBkgMcmaaolZbXHDpUgA0bclG/frjPdME4nIKWSJDKyQEef9wcIz34INC0qYdjBbtcrF8PLFli6rCiokyWoAKWVx08mG/XZ3Xp4ntbg6SgJRJkGCvYKJ0rrO3bgVGjzArLo8yQJuCxx4AJEzQPy0fwtbF/f77dvqlLF+tDhA9S0BIJIowV7NDOtkx795rduJYtnZuexIFc7L7LMcdMvmjY0Lkh3pSVVWhPKWaDXGYO+iIFLZEgwckeW7YAb75pVlo33gi0auXc9BT+JbgVWLu2OcMaPVrNb31IamoB1qzJRo8eMc4V36OgJRIEmJz355+mU/uhQ6ZwuHVr56anMGBxS3DGDDN5mAkXClg+w2wb52PDhmx07+7pAr3SU9ASCXDFs7BYOMyG6ddf78WA9frrwLffmr1J8Sm5uUX21mB4eCW0aOGbW4OkoCUSwDgLa8UKYNIk83NuCXrlDIuRk0XDHN44ZgzQpIlzQ3xFZmYhFi/OwEknxTpXfJOClkiAYpxYvhx44w0gJMQcIXk8S5C4Fcj2GqedpoDlw1JS8rFsWZYVtHz3PIsUtEQCEHfjWP706qtAjPUexCxBr8QKrq4mTzaBi3VYClg+KT+/COvW5SAszLe3BklBSyTA8EB98WLgpZeAunWBO+7w0uzEDRuA++83xWD8S4nP4nTin39OsxbDsdaq3LeLuxW0RALMggXAs8+aacO33grUr+/c8LSVK00PQY485nJPfBbPs375Jd0OWr5OQUskgHA8Plszde9uki5YDuU1HTuaM6zjjnMuiC/iInjZskzr80UImjf3rdlZR6OgJRIgfvwRGDcO6N8fGDLES6Oo1q4FZs4EsrOBxESgQQPnhvgqBq2vvz6EM8+Mtc+0fJ2ClkgAYOkTx4tceikwaJCXpg0zYHF65LZtOsPyIwUFRXbQ6tevmnPFtyloifi5r74yWYJXXQVcfLEXj4++/hro0sUs9Tw+QVLKiwkY9eqFoWlT//g3U9AS8WPciWOT9MGDgXPO8cJ4/MMxat50k0lV9MpQLimPadNSMGBANYSG+sdIGL2yRPwUB/2yNdO11wJnnumlgMWGhswSZBPchASzL6mA5TeKiorw2WcpOP98b+wnl49eXSJ+iCusd94BrrsOOOMMU7frcSwcZh3W22+bLrzid776KhXt2kVai+Nw54rvU9AS8TMMWFxhcUuQKyyOpPK4zZuBBx4wdVhst1GnjnND/Am3Bi++uLpfDYtW0BLxE2zN9OWXwEcfAVdfDfTrB4SFOTc9hVmB/Iswnf2RR0y2oAY4+iV2df/yyzRccIH/bA2SgpaIH8jNNWntn35qMgQHDABCQ52bnsKxx2zNNH++aRnftKkKh/3YjBmp6N07GrVqefqTT8UoaIn4uJwc4IcfgM8+M8HqvPO8ELC4uuIZFuf0c/Qxa7HEb3HBPGNGsvVa8q+tQVLQEvFhHNo4e7YZ9nvWWcCFF5oxIx7H7MCffgIOHAAefNCsssRvHTiQj99/z8Tpp/t+r8EjKWiJ+CgGrJ9/Nqntp58OXHSR9Q3r6e9YfiTnXyTWenO77DLgoYe8NJRLXOnLL1PRp08M4uM9vWSvOAUtER/E1n1z5pgVFmcnMmB5fBuHAWvrVmDWLLPSqlYNaN7cuSn+irOzmOp+5pksKHYu+hEFLREfw6QLNr/1asCiLVuA8eOBefPMaksCwqZNudizJx/dukX5/Oyso1HQEvEhzHdgluD06aYGiwHLa5YtA5KSzIwTr7SMF3f49ttUe6R+tWreOBytOAUtER/COixmCbKP4AUXOBe9pVUrnWEFmOJhjzzPioryz7d/BS0RH8FgNWUKcMklJq3dK5jW/t135jyrdWtlCQaYFSuy7K3mVq0i/KZB7pEUtER8AIuGGbDY6eLss52LnsbCYbZmYuBi0JKAM3t2Grp2jbanFPsrBS0RL5s6lT3ggGHDTPNbr2GWYP365jBNndoDDmuzuNJiAkZMjP/+++qVKeJFDFiffw4MHw707etc9BYGq9tvN30FJeAsXpyJ+PgQu6O7v24NkoKWiJcUB6yhQ4FevbyU1s55WGvWmMIwJl7Uq+elv4i42y+/pNlnWTVq+GFx1mEUtES8gEkXDFhDhgA9e3qpNRPPsB5+2GwLqg4roG3enGPXZrVtG+nX51mkoCXiYQxWTLzgPKwTT/TCeBHatMmMFomPN6mK7HYhAWvRogzUqROKhg3D/f64UkFLxIO4qPnkE2DgQOCkk4Bwbw2M5UiRm24yAxyZ1q7Ei4DFRND58zPQuHE46tb1rzEkR6NXqoiHsNPFe+8Bl19u2jN5JWCxl+C6dWY/8oQTgCZNFLAC3NKlmU7ZXaTfFhQfTq9WETfjGwbHi7z7LnD++aYOyysThzkD68knzTkWa7GYcKGki4DHrMFatUKtxbW3lvWupaAl4kb5+czaMgGLqyvOw/LKGRbnYE2caHoJ8ixLnS6CwsGDpjaLW4MNGihoici/4ET6hQuByZOB7t2BK64AqlRxbnoKV1j8iyQkmMa3Y8eql2AQWb48C1FRldCiRUTALKoVtETcgHFi8WLg/feBtm1N4kVkpHPTUwoLgZ07zVKPM/tr14b17uXclECXl1dkr7KqVw+1Ftae/rTkPgpaIi7GgLV0qVlhNW4MXHstEBPj3PSU4jOs114DPvqI+0TODQkWW7bkYvfuPOtzShW/Lyg+nIKWiAvxDGv5cuCdd0xzCfYTrFrVuelJ/IvMnw+sXQuMGGH+MhI0+Jll1SpTMM6C4kCioCXiItyNW7ECmDTJHCHdcANQvbpz09OYxt6+PfD446Y9kwSVQ4cKsH59tt1rsHHjwNkaJAUtERcwn2zNblxcnKnbZbMJj+OWIFdYrMNi0GrZ0rkhwYQBKympAF26RCEiIrDKGhS0RFyAu3DPPGNWVrfealZaHsfC4cceA37/3bkgwSgnp8h6PWbb2YIdOkQ5VwOHgpZIBa1fDzz4IFCjBnDHHSZJzyvYI4pZgl6bIim+YM+ePGzalGtnDLKoONAoaIlUABul33svUKcOcNddQN26zg1v4HYgJw+rDiuobdmSg3378tC9e7RzJbAoaImU05YtJkYwMW/kSC8l6LEdE2diMc++d2/TS1CCVmpqAdasybZT3Js1C6wEjGIKWiLlsGOHaeHHrUCusBo0cG54EgPWo48Cc+cCubnORQlmu3blYd26bPTsGY3IyMB8e1fQEimjPXtMJjmzBG++GWjY0LnhSdu3A88+azI/2CMqIsK5IcGKCRhcZaWnF6JXL09Xs3uOgpZIGbCxxNNPmx6CrMNKTPRSo3SmJ3J4I/8STGv3yuhj8SVsjvvHH1no2jXK76cT/xsFLZFSOnQIeP5502yCnS68MjuRB2ncm+TK6tRTzV9CAUss+/fnYfnyTJx2mjdasHiOgpZIKWRlAa++aiZ7DB1qFjceD1gsHH7iCbMtuGuXmSKpgCWWtLQCzJ2bYb0uI9CkSWCMICmJgpbIMRQUAG+9ZfIeGLDYtd3jAYurq6eeAjIyzJagV3PrxdekpBRg3rx0nH12NetzTGAP9qxcVFRkfVMWlPgoZEM1y789j/cOd7Tn8HHk80R8HV+ybJL+22/Af/8LdOrkpTMspicyTZFVzBwvoonD4sjNLbKnE/PHvn0DNwGjWOVx48YhISHBftSoUQPR0dGoVq3aX9cGDhyIP/74A5MnT0aHDh0QHx//1z0+4uLiMGvWLOTn59uP2bNno127dv94Hn992223WR8Ydyh4iV/g2dX06cCPPwKDB5skPY/i98nevWbOCT88MuujeXPnpoiRkVGIb745hAsuqOZcCWyV77nnHiQlJeHgwYNWtF6M4cOHY+LEifY1Pj744AO0bt3afvJ5551nfeL87a97fPz888+45pprMJe1Io6WLVti/vz5f3vewoUL7VXbdOtdIDU11XmmiG9i2dO33wJffQVceimsT7DODU9hwOIAx1deMWdYPM8SOQJfJuyAsWhRBgYM8NZIAc+q8M58p06dcMcdd9hB699WUM2aNUOfPn2wfft2e7Ul4qsYsDjsd8YMoH9/oF8/54YnZWYCX3wBLFli+kRxmqTIETideMqUZOuDVRzi4oIjKafCQYurp1zruzwlJcX6AuY5V/8pJCQEMTEx9tlWVlbWX2dlIr6EAWvBAuDTT83q6oILnBueFhoK9Oxpki/atHEuivzd/v351oIhHRdeGByrLKpw0OKqaYn1abBFixYIZwpuCbgluGnTJsTGxqJmzZqo7PH0K5F/x89cixebxAsmXFxyiXPDk/btA9atM9XL/EswVVGkBFOnJqNdu0i0ahU8HVHKFDkYoL766it88sknfz3eeuste9V08sknO89iEeYhfPPNN397HhM5GLS6deuGOmyJLeJDmHTBMfnvvWeapA8cCISFOTc9ZfduYPx4M2JE5BiY5j59egoGDarhXAkOZQpae/fuxbx58/Ddd9/99WDA+t///mevtIqlp6fbiRi8P2PGDDzyyCOYMGECevbsiVNPPRWRkZHOM0W8j0exnDpsvUTtTu3XXQdEeWN23syZwMqVXjpEE3/z7bepiImpbDfHDSZlClpdu3bFo48+ageg4sfDDz+MukcUOtazvvN5nfdfe+01O1GDiRgMcEp3F1/DIY5sz8QGuNdfb3rQekVCgmls6GTripQkO7sQr7yyH8OGeWNEtne5/WCJZ1jnnnuuXePFLcNV1kdaJmOI+ILiCfUxMcBtt3lhTD7/Aps3mzosZn20auXcECnZ/PkZduA65ZRY50rw8Eg2BBMvLrC+IbktyMDFbUYRb+MR0iOPmN6zd9/thc5IbH7L7MDffze9okRKacKEAxg+PMHeHgw2Hvk/ZqYgC455nrVo0SL7nEsFxuJNSUnAk0+azPLRo4FGjZwbnsIPbuzAS1xdqfGtlBJbNrGje7AUEx/pr6BVqVIlO6tv8ODBOPHEE52rBmusWBh85ZVX/uP86nB8Xtu2ba1PrXejwRGjXCOsj7MMWjzf6t69+7+mx4u4E3vOMkmPNVlcYbE7ksdFRwMdO5opkjzDUgmIlNJ77x1E//5stWd94gpClYqCNDNixIgRdoAdMmSIc0WCAXfh2BWJx0i33uqFnAduCcbGAjVqmOjJvUmtsqSUVq7Msj5o7cQrrzREkyZVnKuBhW3/WBrFEqmj0cc7CSpvvmlGjBQP/PUoJl2MGwe8+67Zn+RqSwFLyoDFxD17RqFePU8XEfoOBS0JGowVzHnwykys7duBZ54xVcynnQZUDezpsuJ6GzfmYOHCDFxySRyqVAnet24FLQkK7CXIESPFM7E8vsDhWTDbbPAMixGTGSAiZcDuF127cpUVHtTj1BS0JOB98w3w2WfAoEHA8cd7OF4wS5DVy/yPdusGtGunLUEps1278uyZWeecU91apAf360dBSwLavHlmVP655wInneTheMHmt0xTHDUK2LDB/MeVJSjl8MUXZpV13HHhQf8S0neQBCQ2mOD51euvA6zgGDAA8GiVBRMt+B/n1OFHHzVdeEXKgeNHuDXYq1d00Ka5H05BSwIOA9bq1aZ2l7MTr7zStGnyKI4WYaRkfr3mYUk5sSDpyy9T0aFDFNq1i0JoaBAfZjkUtCSgMGCxvIOp7ezUPmwY24g5N92N7zBpacCBAyadvWtXc4YlUk4HDuTjhx/S0KlTFOrWDd4098MpaEnAYMzYtQv48EOTWc4GuA0bOjc9Yf9+4I03gEmTnAsi5cfX87ffHrJrsrp0ibQW71plkYKWBAweI02fbgIXR4x49BiJ7zBffAF8/TVw9tnORZHy41kW67JataqCRo3U9q6YgpYEhPR0k9rOYY7XXgt06ODc8BQGLQ43ZbZg+/bORZHy4ctp9uw0RERUwgknRFsvLb1VF9NXQvxedjbw/ffAzz8DV1wB65vcueEJnG+yY4f1nWR9KzHjQwFLXGDPnjwsW5aJpk2rIDExMHsMlpeClvg1nl2x08WsWcDFFwN9+zo3PGHnTpMdyNx6ERcpLCzCvHnpyMsrwoknxiAqSm/Th9NXQ/wWt1C4uvrkE6B/f+CMM5wbnpCSAvzvf+YArUkT56JIxbH7xYoVWXYX92bNtMo6koKW+K2FC01q+8knm44XHsXmb3FxwL33Kq1dXKagoAiLFmUgNbUAJ50Uo7Oso9BXRPzSH3+YRhM9egCXXebB9kzbtpmsj2rVgGuuMYXDwdy9VFxq584867WdZa+wmjfXKutoFLTE77B4+J57TO3u4MGmiNgjGLCee44tCoDMTBO41EtQXIRnWFxl7d6dh9NPr4qICL22jkZfFfErbJr+0ENmgcMpH9WrOzfcjfOwmM7OVMWmTYEwdScQ19q5MxfLl2eiQ4dIO2tQjk5BS/wGFzdPPWUm1d9xB5CQ4NzwhPh4oE8fM5CrY0cFLXGpnByusjKxdWuePX4kLExbziVR0BK/8cILQG4ucMstQP36zkV3Y1o7twXZS7BfPwUscQtuCS5enGF9LopBgwbqfvFvFLTEL7ClH8+yuCXIzu0eyX1g4fCLLwJPP20KiNnxQgFLXCw7uxALFpizrIsvjtOM0GNQ0BKf99FHwNy5wE03AS1aWC9aT7xqeXj20kvAn38CQ4aYcfkibrB3bz5mzz6Ec8+thrg4RaxjUdASn8Z+gp9+ao6S2CHJY8l6TEk8/3yT9cEtQX38FTfIyirEjz8eslZbRfYqS45NQUt8Ertd/PorMGECcMEFQPfuHoobHMhVUADExpr/KDvvqg5L3CQpqQCff56Kq6+uYb2+9TorDQUt8TmMGytWmMSL4483A4AjIpyb7sIoefCgGZE/bpxzUcR9mDH44YdJaNgwHKedZn1IklJR0BKfwtixZYtpz5SYCAwa5KFarJwc08SQEyQ1D0s8gJ3cZ8xIwdChNZwrUhoKWuJT9u0DPv4YqFLF5D/Uq+fccDcu73iOxeSLTp2ciyLuwYzBceP2oH//aujQwVMtXQKDgpb4jEOHgK++Mol7XGF5ZPIwxx1zW5ABi70EFbDEA5ji/ssv6Rg8WKusslLQEp+QkQF8+y2wdKlpgOuR2LF/P/DyyyafXsRDDh7Mx/PP78WYMXVRp47q/spKQUu8LisL+Okn82DSxYknOjfciT2h3n2XM81NtbKIhzD5onLlSrj0UqW4l4eClngV2zItWADMmAH06gWcfrpzw92YeJGcbFIUWYcl4gFr12bjiy9SMXJkHVVSlJOClngNy6E4F+vtt4HWrYHzzrNekO5+RXLSMCMlBzjedZepwxLxACZfjB+/F717x6BHj2jnqpSVgpZ4RXFqO8uiGjQABg40PWndis1v2UuQVct5eR6cayIC/PBDmt1fcNCgeOeKlIeClnhFWpoJWDExwA03ADVrOjfchQGL6exMTWSlsvZmxIO2bs3Fu+8exNChCXYxsZSfgpZ4xSuvmIxBjhnhSsvt2KGd80yGDwe6dAFCQ50bIu73wQdJ1ssvHCedFKN2TRWkoCUexxXW6tXAbbeZIcBuxRUWU9s5xJF7kAxYGi8iHsR6rHXrsvGf/8QhNlZvuRWlr6B4FMeMzJoFjB4NtGzpXHQXJl2wDouZHiwgZvJFuLZmxHP27cvHa6/tx4knxqBNm0itslxAQUs85rvvgP/9Dxg5EtY3sHPRXfbsMXuQXGn17m26tot42GefJSMhIRSnnBKLqCi93bqCvoriEUuWAPffDwwbZiZ+uB3bMp16qpkc2a2bVljicQsXZuC33zJxwQXVkZgYrtwfF1HQErfbtg245x7gP/8B+vd385ESc+nZ/JYrqz59gK5ddYYlHpecXICJEw+gWbMq6NgxEqGhiliuoqAlbsUMwUceMa2Z2ASXKe5uww4XnBr5/PMmeDFYKUtQvGDmzBRERlbGgAEcoa/XoCspaIlbPfOMKRq+/nqTwOc2bBHPIVwTJ5ptQY/N5Rf5u99+Mx3czzyzKtq3j9S2oIvpO1vcIj/fxBDmQfBYqW5d54a7sDUTK5Rfe03jRcRrDhzIxwcfMPkiTK2a3ERBS1yOHZI+/xyYPx8YOhRo0cK54Q7Z2aZNPAMWp0ayDkvEC/LyivD992koKirCZZdVR40a2hZ0BwUtcSkGrDlzgOnTgXPOcXM/Wm4Jsgbr66+dCyLeUVBQhEWLMjBvXhpOPjkWHTtqGrG7KGiJy7Br+4oVwOTJZtoHx4xwbL5bcP/xgw9M4kWjRs5FEe/YtSvPHjlSrZqpyRL3UdASl2Fq+5QpQGIicOWVbm6izrRERkgenDGtXcRLMjIK7cSLrKxCDBwYbwWuEOeOuIOClrjEgQPAl19aLyjrFcWAVa+ec8PV2EeQSRfVqgFPPKGAJV6Vm1uEBQvSMX9+hl1E3KpVhHNH3EVBSyosNdX0E+RK69JLgebNnRuuxrEizA5cu9bUYVWt6twQ8TzWsK9fn40pU5LRtm0E+vTRtqAnKGhJhXCX7ocfWJsCDBjgxmxz9hJkwOKo48xME7REvCg1tQDTp6cgPLwyLrywOkK0K+gRClpSbjk5ZgjwF1+YnrTseuE2TLxghBwxwvQSVPGweFFOThG++SYVO3bk4vrrE1CnjlqFeYq+86VcmCnIXTom8LVvD5x9tpta/O3ebVLbOSny9tuB449XaybxKm4Lzp2bjpkzU3HllfFo00bnWJ6koCXlkpQEvPuuiSWcreiW4yUGLE6MnDkTSE83k4fV/Fa8bN26LDzzzB6cemosevVyZzNNORoFLSkzNqGYNMn8nB0vatUyP3ep4oDF7A4FK/ERGRkFeOqpvejQIRIXXRSnoY5eoKAlZcKtEa6w1q0zPQUbNnRuuFpEBNC2LTB4MNCjhxurlEVKb9y4vdZLsxJuv702qldX5oU3KGhJmcyYwWms5nipSRPnoisx4aK4DovDtxSwxEdMmLAfq1dn495766JuXa38vUVBS0pt0SJg9Gjg1luBdu2ci67ECmX2EvzwQxO8OHxLAUt8wOefp2DKlBSMHVsPjRppCrY3KWhJqRSPGLnzTuC006wXjqtfOczs+N//TP48I6LOsMRHsNvFk0/uwR131EKLFsoU9DYFLTkm1vHedZfp2n711UC4Oz5osmC4dm1g1ChToazJeeID1q7Nxu23b8d119W0G+GqPND79E8gx3Tvvab57ciRbgpYxNz5a64xZ1hqLSA+YM+ePNxzz05cdFF1XH55HCIi9HbpC/SvICViATGzzjdsAB57DIiMdG64CjtcfPIJ8Omn5tf8GKsVlviAQ4cKMHbsbrRrF4EhQ2qialV9kPIVClpyVAxYPF5ipiCTL1w+ZoTTItlO45FH3Jg3L1J2mZmFeOGFfdaHtMp2i6aEBHVg8SUKWvIPrMViA9wXXjBlUm3amEWQS3GVtXChqVI+4QTnooh3paUVYMKEA0hKysewYTWtz1PKFPQ1ClryD1u3Am+9BVx4oekpyDpfl2EfQaazc+n28ssKWOIz2LX9gw+SsHlzjp14oUxB36SgJX/DGYvvv2927C65xMXbgikpwMSJJiqSS6OhSPklJxfg00+T7WxBTh9u397VB7jiKgpa8pfkZDMunzt3l13m4unDxXVY06aZImIRH8GtQBYPr1qVjcsvj0e3btHOHfFFClpiYxN1Th9euRI47zw3TB/mEm7BAuCpp7QlKD6DAWvGjFSsWJGFSy6JQ48eCli+TkFL7FZ/8+cDs2cDJ53k4trefftMW/iWLYHHHwe6d1dau/iEgwfzrYV/ivVBzQSsnj09H7CK59K9+qp5MKF2/XqTDCVHp6Al+PNP4JtvgA4dgLPOcmE91t69ZkuQ440ZGVu0UOGweB07vOzenWcFiGSsW5eNyy7zTsBiYGLAev55YMkSc9TL/p4sW+R1Ba6jU9AKcuwp+OWXQM2awEUXAfHxzo2KKg5Ya9aYdwk+RLyML0NmB7755gFs356Lq66KxwkneGdLkEm0c+aYksUnngCefBK4+WazMfHtt8xmdJ4of6OgFcQOHjSjRpjUd+mlppOSy7CwKzbWFHqdeKK6tYvXMWD98Ucmnnpqj7XwL8K119ZAp05Rzl3P47fI8ccDI0YACQlm15zJT4mJJm+JCVHyTwpaQYrfEPw0t3gxcP75QLNmzo2K4sdEPrh0u+oqoFcvpbaLT5gzJw1jxuxC/frhGDq0Jlq39u7rMjTUBC3OOi3GXXQmRXEX3W19Pv2cglYQ4nYE986nTjXFw507OzcqihmCPEn++mvz3VejhgKW+ISPP07G+PF7cdZZVTF8eE00bux7EYFnWFu2AJs2mTpJblTIPyloBSEeN733HtC3L9Cvn4s+0bH2ii2ZuElft675GCniZewj+PTTe+209ltuqY1rrqmB2rV9c1Ybz5dZxshtQmbx6vPe0SloBRl2UXrlFXN+xdlYHA7sEtzT4JYgx4swZ15ZguJl7G5xyy3bsG1bLsaMqYOTT46xVi+++brcs8d85uM5M7+FeK6lypCjU9AKIsxWYkIfV1p33OHiFk3165s2Gr17K+lCvO7rrw/hhhu2oXHjKhg5sjZatYqwFv++GQW4ScERQNu3AzfeaM64tFFRMgWtIMJMQY6v4qiRuDjnYkWkpZn5JcyZ53h8bsLr9Fi8KCWlAM89t89Oab/pplrWSqsWGjQI99lVC9PaX3rJ1EpyOnjr1tqkOBYFrSDBcqlbbjHDHF2SKZiVBUyeDDz4oMnXFfGyJUsyce21W7B8eab1Oq+HCy+shurVQ3w2YDFX6eOPmYZvxsqxaYzG+R+bvkRBgNmCQ4aY+HLaaS7aK+fhGFMQ2XtGvQTFizhleNKkAxg1aid69YrGs882sNPZQ0J8+1CI51gffmg2KxiwuFnBbUE+OBaIfUDlnyoVWZyfB5URI0agbdu21pu59W4ewHJygFtvBapWBZ5+2rlYEYyA3L/QR0LxIr5rsUCYfQPHj9+H9PQC3HdfHRx/vBre+rukpCR069YNm5j7fxR65wlgrPHl9GHum7NFTIVxS5DDtlhMIuIFDFbZ2YXYsCEHL764D48+utta6EfhzTePU8AKEgpaAYoLounTga++MvvlFT7cZUr7u+8CY8ea9EMRD8vJKbI+L+Vg6tQU6zW9y/75gw/Wxa231kKtWkq3CxYKWgGK7ZnefttkJDVp4lysCLadZl4uq5J79nQuirgftwE3bsyxBzWOG7cH06YlY+DAGnj55UZe7R0o3qGgFYA44oCLoosvBk4+uYI1HxxnzKE/3bqZIi82vxXxAG4DcnTIlCnJVoDah3feOWi9nqtar+1Eux2Tim+Dk4JWgCmurG/c2GQgRVdkm5/l+e+8Y1ZZDFxduzo3RNyHrZeYYPHBB0nW56QDmDo1GccfH4XJkxvj8svjEBOjt61gpn/9AMIRI+xXy8QLNqdgV6VyY5k+9xd/+slEQk2kEzdLTy+0a624onr33YOYNy/D+pwUbX0Ia4wrr6yBuDidW4mCVsBgoSJHjaxYAZx7rgtmYzH1kAkX7CvD7p0sIhFxg9TUAixYkIGJEw/gww+TsGlTjj2Ycdy4+vbKKi5OLSLk/yloBYilS4G5c4EzzzS1vuWOMZmZpriLUe+224A+fdSaSdyCLZdmz07DG28cwKefJiMtrQB9+8Zi1Kg6uPjiONSoEapzK/kHBa0AsGEDMGsW0KoVcMYZFZjDw3lYU6YA8+aZwMUmuJqPIC7GYDVrVqpdZzVzZqqdKMTEihtvTMA551RTsJJ/paDl53jcxBk8bFAxYEAFzrF4hsWUQy7X2KVdHS/EhVgUfPBgPj75JBmPPbYbP/yQZr1WQ3HRRdXtsfenn14V8fE6s5Jj0zuTH2O9Lzu3r1tnJhBz2mm5MXuDwYophxxlrDMscZEdO3IxYcIB3HHHDixenIk2bSIxaFA8rroqHr16xSjBQspEQctP8ZPrsmXAb7+ZgNWhQwUXR5w2zJTDU04BolSwKRW3alU2nnxyD66/3gxiPO+8avjvf2tYL7M4uyi4WjUlWEjZKWj5Kc7f+fxzU/PLkfnlOnpijvw33wC//GICVa1aQGSkc1OkfObOTceIEdsxevROu7xv9Og6VuCqaQWt6mjRIgLR0XrbkfLTq8cPMROdwxwZZ664opwj8zlahEVdnL2vQCUVxBqrzz9PtV6Pm/HCC/uQmFgFTz9d3w5WJ54YYw9iDA9XdoVUnIKWn+E5FldY3Bq85hqgWjXnRlkx8YJph0xr79jRuShSNjyveu21/TjrrPV4//2DOP/86vY8q+uuq4mWLSOUCSgup6DlR7jVwnosZqVzCnFionOjPNjniWOM2ZxQSRdSRuwJyO2/K67YZL0mM/HEEw3s8SCXXlodDRuGawtQ3EavLD/CxdFLLwFXXQX07VuOCcSMejzD4nA1Zm1wW7DCM0skWGRlFeLHH9Nw2WWbcPnlm+3JwJMmJeKNN47DSSfF2KPtQ0MraWUlbqWg5Sd4BMVBjs2aAYMHOxfLIj8f+Phj4LrrgF27nIsiJWO7SY4F2b8/325ce/75GzFmzE67EPibb5rh0UfroUWLKnaQUqAST1HQ8gPsK8jO7TyCuvde52JZ/fwzcPfdwEcfAb17OxdF/onDFpOTC7BsWSbGjt2NCy/caDevveeeOpg7txWGDKmJ2rW1pSzeoaDlBxhvOMqKR1BlzhRkL0EWdZ16qglYbH4rcoSCgiK7vdKWLbmYPj3F+nyzHddeuwVRUZUxYUIjvPXWcTjttFg1ShGv00vQx3Gg41NPmSn3LCAuk7Q0k9a+b5/5tQKWHIGzq1j4y9oqbgGyvop9Ac88syp+/bWVXWPVurVKIsR3KGj5MPYVfOghUzzM7kplwrZMDFhvvcXWBGa1JWLJzy+yPsfkY/nyLHuEPacCP/vsXmRkFNgd1idNOg6XXRZvr7JEfI1elT6KMWf8eDMq/447nItlwYj3ww/Agw+a8SI6KQ96XFWtWZONr75KtYcsvvrqPms1lWG3VJow4Tjcf389e44VswJFfJWClg9i1tb06cCSJcBdd1n/SGX5V8rIAPLygJYtgUcfNXVYjHwSlLiq2r49Fz/9lIb330/Ce+8dxDffHLJeY0UYPLgGHn+8Pq68Mt5OrNB5lfgDvUx90MKFZgrxyJFA8+bOxdI4eNBEuz/+MIGrRQsNcAxSHKj4228Z+OCDZEyadBBffJFqTwTu2jXK3gIcObIOevaMQUyM3gLEv+gV62M4ZoRjrdi5nXkTpf70y4D1/vvAjz/C7vWkM6ygxE4VU6YkY9y4vZg2LcX+df36YfZq6r776toTgdkHUMRfKWj5EDbCfeMNoE4doH//MnZuZ7sMViCfd55p/a4VVtDYvTsPX3+discf34OJEw/Y9VUc+8EU9ZtvTsB//1vTXmFpVSWBQK9iH8HdvE8/NWVVzBQscyPc2rWBq68GTj8diI52Lkqg4vbfggUZeOGFvdaqao99ZsWOXL17x1gvgxq4447a9jTgunVVBCyBRUHLR/z0k8lMv/hik0NRqpaAycnA7NnA8uVA9epAo0YKWAEsL6/I3u6bNOkA7r13Jz74IAlZWUV2oLr88ngMH56Ac8+tbndXV0tJCVQKWj6AeRNffmnOsE480Uy9PyYOcGSHi8mTTSNcCVgHDuRbq/AUjBixw26rtHVrrrOiisc119Swx4F07hxlN6xVZYMEOgUtL2PvWiZexMeb7PRSL5T4G/k4/3ygVSvnogSK7GzTUZ0rqmHDtmLu3DR06RKJm24yZ1QMVN26Rdvbf+ysLhIsFLS8iI1wv/iCb1Amf4LT7kuNw7RuvBE44wwzwlgCgplNtQcXXbTJ3gasVy8Md95Z21pl1call8ajR48Ye15VRIS+dSU46ZXvRXPnAitWAOecA7Rta/1jHOtfg9Htl1/MPCzOwqpbV6PyA8DGjdl45ZX96NfvT9x11w67ee2oUbXx9NMN7O2/Hj2i7UDF7D9t/0mwU9DyktWrgZkzgS5dgNNOK8Xw4OL0wptvBvbvdy6Kv2Ka+rvvJlkrqo32UMUNG7LtFdXUqU3tVVWvXjH21l9sbIjaKokcRkHLC9h0/cMPzfYgj6SOGbBYKMyJww88YBoSHn+8c0P8SXJyvt2gduDAzdYHlfXWP2kqrroqHj/+2BLjxjWwrlVFXFyI3ahWgUrk6BS0PIyBiuntGzcCQ4cCNWo4N0rCRoTcEzrrLNPxgnOx1CTO5/FzRvGMqu++O4Qbb9yGPn3W4403Dlj/hLGYM6el9c+ZaHeoYCFwWFgl/bOKlIK+TTyI8YclVd99B/znP0DHjs6NkhRvCTLScTnWo4dzQ3wRgxQ7qXM8/c8/p2HMmF0455wNeOqpPWjTJgIzZjTFrFnNcN11NZGQoCbGIuWhoOUh/OS9ebMZcdWmDXDuuc6NkmRlAVOmmDbv8+Y5F8XXsIt6amoBdu3Kw9y5GXjmmb24+urNeOSR3ahZMxRvvnkcvv++BW6+uRYSE0tTgCci/0ZBy0NYCzx1qulrO2iQc/HfMFPj2WeBiROBU05xLoovyM0tsgt+N27MwezZaXj11f249dZteOyx3YiIqGT9WA8//NDCTqzgCktEXKdSkcX5eVAZMWIE2rZtiyFDhjhX3Ie7fJzHOGkSMHo00Lmzc+No2HyQLTHYh+f3382TddjhdTk5RXYiBYPVli251meKbKxcmYX09AI7Jf3kk2PtAYoiUjFJSUno1q0bNrG05yj0buhm/EjARROzBQcOPEbA4rhiVhvzH4utmbp2VcDyIgaqHTty8dtvmdY/SwreeecgXnppn91SiZ8pbryxJj7+uIk9m0oBS8Qz9I7oZrt3Ay++aM6x2PWiRNw//OQTk3ixZYv6CXoJm9Ju3pyDn39Ot+dSsZaK035/+indTkW/9tqaePnlhvbWH7tTMOtPRDxHQcuNmC341lscIwFcd51zsSQ7dwIrVwJXXAH06qV5WB7EMyp2T+cYerZOeu+9JLueinOpatYMwaBB8XjiiXq45ZZa9lZgdLS+bUS8RWdabjzT+vpr4PXXgYcfLkV6O4c4JiUB9etrvIgHZGQU2uPn//wzB6tXZ9mBKyen0C7qbd060nptRKB58yp2RwoR8ZxjnWkpaLkpaC1dCjz4IKxP56an7VFxHtaaNWZUcZMmzkVxl+TkArtd0po12Vi/PtteCVeuXMnukt6+fYQ9h6pJkyr2NqCIeIeCVgncGbQ4Nn/4cDMba+RI5+KRis+w1q41E4c7dXJuiCsx22/VqiysWJGNbdtyUKlSJXt7j8W9iYnhVpCKsH/U2ZSIb1D2oBewtIqDhAcPdi4czdatwPbtpstF06bORakodqVgoe+sWal23dRDD+3CzJmpdmp6hw5RGDCgmj2OfujQmujXrxpatKiigCXiR7TScvFKa8YM08jinnuAdu2ci0fDKmM+6tUDYmKci1IeDFSbN+di8eIM65Fpd1CvXz8MDRqEo1Ej82DHdHaoUIAS8W3aHiyBO4LWsmXAo48Cw4aZvrahR7aXS08324EJCcBxxzkXpbyY3ffrryZQcSXVtGkVNGvGcykTpLgFWLVqiCb7ivgRbQ96iPV1xnPPwfpiA927HyVgsZfgZ58BjzxinixllpZWaNdLjR272x6Y+Pjje7BvXz7696+K++6rixtuSMBll8Whb99YO6kiPj5UAUskwChoucjkyUBcHHDppbA+3TsXi7GPE+eRvPMOrKUd0Lq1c0OOhedTn36ajFtu2Y4zz+Roj32IjKyMUaPq2jOobropwT6n6tAhUtN9RYKAtgddsD341VcmEfDGG0vovMQvMbMFWTzMZZgKh0vErumbNuXi++8P4YcfDtlBq1WrCHuSb58+MahRIxQREZXtxrQalCgSeLQ96Gbr15tGuGzEzsSLfwSs77+3PhpYb65chqnTxVFlZRXil1/Scf/9u6wv0TpceOFGu0PFNdfUsFZZTfHaa43snzdvHmEHLaasK2CJBCcFrQrIyTE7fi1bmgLiyEjnRrEvvzSDs9jindT81saFZ1paAT76KMkKRlvQocNq3H33Djuz75VXGmLZstZ44YWGOO+86qhXL8xeWfGetv1ERO+i5cR+ttOmmb6CPMeqW9e5UezHH814YnZtP+0052JwYXDidl92diHS0wvx+++ZeO65vVYw2oDGjVdi+vQU9O0bg2+/bY4FC1rhgQfqolu3aKWli0iJdKZVjjMttv/hMGEmX7Bz+4ABzg0qHo3PZcHs2UE3wJE1UxzpwS2/lJR8uxPFggUZ1iPdDvQ8mzrllFj7fIoJFSIih9OZlhvs2GESL2rXNq2a/sIBjjNnAvv2mWVGEAQs/m9yJcV2SRzpwQD19tsHMXr0TgwevNX6OiXbxb0vvtjIiuEt8OST9XHWWVUVsESkXPTOUUYZGWbnLyICuOoqk19hY+Ew67A4PGvJkoCeh8WVZlJSvvVJKMfuQjFjRqo9cv7ee3fi6af32oMTzzmnGqZNa4r330/EjTcm2CnpqpkSkYrS9mAZtgeZeMEdv1mzgAsvPGIhxXYYY8cCV14J6x3bjMwPIDyTOnjQjJvfsyfPDlhsnbRrV649vqNbtyj7PIpFvayVEhEpD7VxKkFZgxZXFxybP2GCyRYcOtQcXf1lzx7T3r1Fi6OkEfofnksxSO3dm2c98rF9e67d04+Pgwfz0LFjNDp1ivyrqFdExBUUtEpQ1qDF3rbvvmsGDN9xh5nViNRUWMsN0/S2Vi3zRD9VWFhk/e8UWP9/efaDW3xcVXEGVUpKgd0SqU2bCOtrFmH/WKWKVlMi4noKWiUoS9Bi20BOIWa51X//64y+YocLnmGtWwcMHHiMlu6+iasprqC2bs3Fli05dh8/JlUw84/Fu8cdF25v97EjRe3aYSozExG3U9AqQWmDFvMpFi8GXnrJnGNdfLFzg8kWH31k5uizgPgfDQd9D/+ld+/OtV4MfOTYASs7u8i+ThzdwRHz7JbOgKUMPxHxNAWtEpQ2aHEi/mOPmbwKzsj6KzZt28avrhmT78MBi1t7Gzbk4M8/s61Hjt2Jglt74eGV7ISJxo3DrYcJUtwCVNcJEfEmBa0SlCZosU74/feBRYuAUaOA4+IOmYnDNWqYcywfxK29LVtysXp1FtauzbZWVvnWiqmSPVeKj+LhiPyRW34MXiIivkJBqwTHClr8qvz8s5mRNWYMcHxrpw5r/nzgppt85gyLbZLYCX3lymzrkWWvqFgPlZAQYj3CrOAUjjp1Qq1HGGrVClM6uoj4NAWtEhwraHFbcNAg06ZpyKBchMyzIthbb5meTTzDio11nulZ/Nfilh+n9i5blmUHqszMQnv11KBBmL3NxwBVo0aIfUZVrVqoEihExG8oaJXgWEHrvvtMM1zWC1eLLTS57n/+aeZhRUc7z/IMrqbYbLb4wWy/Zs1MwgQTJ1gnVb16iP3gFqAazoqIv1LQKsG/Ba2pU4HXXwfefycftXcuAU44wVQX5+d7ZB4W/0VWrcrCwoUZWLSIj0y0aFHFro9q0ybS+nmEHaC41RcTE4IqVRSkRCQwKGiVoKSgtWGDac/09ltFOC3zC7Pk4pTH4493nuEebIk0f366/fj113TUqBGGjh0j0blzlPWItDP7OFeKaehMnlCWn4gEIgWtEpQUtFgn3KYNMLLD1wgddp1ZcvXvD4SGOs9wDRbyclrvL7+kYe7cdHt11bVrFLp3j0aPHtGoVy/cDk7c6tMARBEJFgpaJTha0GIB8a+/AhMnApHhBcC33wL9+llfpYpHjPT0Avz0U7r1SMOcOel2Tz8GKA5B7NMn1j6j4jZfJeu/xf+cgpSIBKNjBS3llTmYyf7uO4W4r8cPiKicC4SEAGefXaboYY69zBDE1NR8fP11KsaM2YmTT16PxMSVePPN/Xb6+auvNsT69W0xZUoT3HxzLbvpbFRUZbt1EjP9FLBERI5OQcvCvrcvjC/EDZ0WoOlTw1DpO2uFdQxcn+blFdnp5kxBZ0ukb789hCef3IPLLtuE1q1X2zOmYmNDMXZsXWzb1h4zZjTDXXfVwfHHR9vnUyIiUjZBvz145ZVD8PzzQNq6XRgx/2LUfGqkKc7iSusw/Crl5pox8nywIzr79/3xR6b1yLZWTtn2hN7u3WNwwgnmbIqrJxERKT2daZWAQatVq7aoWXMIpk0Dxvx3N1pFbAW6dPkrrZ3BicMP2a+PqynWRzE48bFjRx7i4kKsp0fZCRR8VK/u2mQNEZFgo6BVAgatELTCvg2X4owLonHRFeHWasqsoJKT8+0gxZlS7OPHYMVpvSzcZRp6x45RaN8+EgkJClIiIq4UkEGrsLAQ27dvt1Y865GZmYl69epZQaQ9IiIinGcc2+gRj+HrD6ujT/02OPOWpgirW93u4cfgxOm8/JG1Ue3acfBhpD1Tiu2RRETEfQIuaDFgbdy4ETNnzsSqVauQlZWFqKgoXHXVVejduzdCS1lP9dCIX/DO+N1o1grIiMrB2s0HERmZg2bNwu0AxeGHWkmJiHhWRkYGxo0bFzhBKyUlBRMnTkReXh4GDRqE2rVrY9q0aZg/fz5Gjhxpr7pK49NPkzF39i4UYBsKsdW6st16HLIeQblbKiLiM2rUqIGHH37Y+dXf+V3QWrduHSZNmoQePXrg7LPPtrcEDxw4gPvvvx8XXHABzjzzTLtAV0REAo/f5WRzvzM6OhoNGzZEFY4TtsTGxiIuLg5r1qyBn8VgEREpA78LWjzD4kqKK6wjV1SprBIWEZGApepXERHxGwpaIiLiN/wuaPH8iudW6enpdvr74erUqeP8TEREApHfBS2mQiYnJ9u1WtnZ2fY1Zg/u2rUL7dq1U+agiEgA87ugVbduXbRp0wa//fYbNmzYYAeuqVOn2vVZCloiIv4tJycHb7zxBubMmeNc+Tu/C1qRkZG44oor0KJFC1x99dVo3LgxFixYgGuvvdbeOhQREf/Eo5+PP/4YTz75pN0Z42j8tmFufn6+3RWDf/2QkBCEh4drlSUi4qe4a3b33XfjzTffRLVq1fD222+jf//+zt3/57fZg+wxyFUX+w6yyFgBS0TEPzFgjR49Glu2bMHChQvRp08f584/+W3QEhGRwMBmEePHj8cXX3xhdzf6NwpaIiLiNxS0RETEbyhoiYiI31DQEhERv6GgJSIifkNBS0RE/IaCloiI+IyqVavi3HPPRWJionPl7/y2I4aIiAQfrbRERMRvKGiJiIjfUNASERG/oaAlIiJ+Q0FLRET8hoKWiIj4DQUtES+ZOnUqunXrhkaNGv3t0axZM9x5551ITU3F6tWrcdVVV/3jOXy89NJLOHTokP1nbd++HZdffrk9yfvI5/Xr1w9//vmn/TwRf6c6LREveeedd/Drr7/itttuQ926df8aZMofOeSUw01XrFiBSZMmoVevXnbwKbZp0yaMHDkSF154Ia6++mrs378f1113nR3sTjrpJOdZQEpKCt577z3s3LkTDz74IOrUqePcEfFPWmmJeFFYWJjdAYDjxYsf/DUnchcHsZCQEPvXhz+nc+fOGDZsGA4ePGivyIpFR0f/7XkNGzbEBRdcYAfABQsWOM8S8V8KWiJ+jEErPT0dJW2YVK5cGbGxsXbQO3DgAPLy8pw7Iv5JQUvED2VkZGDbtm32qoyP4lXZkQoKCuxgxedzC5IrOxF/pqAl4kVpaWnYsGED1q1bh7Vr19oPBqPDV0Q5OTn2mVTxfT6++eYbfPfdd2jSpIkdtIrx9x7+vCVLlmDmzJmIiIjACSec4DxLxH8pEUPES5iIMW7cOISHh9sP4oqpS5cuGDNmjJ008ccff9gJFAxA1atXt59DtWvXxvXXX49TTz3VDkibN2+2EzH27duHmJgYe7swOTkZe/fuxeDBg+0/r1atWs7vFvFfCloiXsKgtXjxYowaNQr16tX7xxZfYWGhHbT4PAYnjmsoSXHQeuCBB3DKKafY24I///yzHayYMn/NNdfYSRoi/k7bgyIBiBmHPXv2xPDhw/HJJ59g7ty5yM/Pd+6K+C8FLZEAxTT3M844A3369MG0adPs2i4Rf6egJRKguN2YkJCAAQMG2GnxkydPts+4RPxZyEMW5+ci4kE8TmZyRIsWLexkipKwxoqtnWrWrOlcOTqms3fq1AlxcXHOFetTaeXK9q9ZaMwg1rRp079lG4r4GyViiIiI39D2oIiI+A0FLRER8RPA/wFB+evCgnjzTQAAAABJRU5ErkJggg==)\n", "\n", "1. Which one represents a better model? Briefly explain.\n", "2. Based on your results in Q2.1, which ROC curve is more likely to be associated with model **A** and which is more likely to be associated with model **B**? Briefly explain." ], "metadata": { "id": "6zFvugVNBIKM" } }, { "cell_type": "markdown", "source": [], "metadata": { "id": "dzbw4MD-nakF" } }, { "cell_type": "markdown", "source": [ "## Q3: Threshold\n", "\n", "Now suppose that we have a state-of-the-art model to predict earthquakes (negative vs positive) and by default the threshold value is set to 0.5. If we don't want to trigger too many false alarms, how should we tune the threshold value (i.e., should we increase it or decrease it)? Briefly explain." ], "metadata": { "id": "HpHXoQjIDNNV" } }, { "cell_type": "markdown", "source": [], "metadata": { "id": "1YCiUJIEnbLn" } }, { "cell_type": "markdown", "source": [ "## Q4: AI and Ethics\n", "\n", "For each the following three questions, provide an answer with **no more than three** sentences." ], "metadata": { "id": "FL0q1xyHv8B0" } }, { "cell_type": "markdown", "source": [ "### Q4.1\n", "\n", "Watch Joy Buolamwini's TED talk on [How I'm fighting bias in algorithms](https://www.ted.com/talks/joy_buolamwini_how_i_m_fighting_bias_in_algorithms). Joy mentions that judges use machine-generated risk scores to determine how long an individual is going to spend in prison. The state of Wisconsin uses the AI system Northpointe for making sentencing and parole decisions. Research this issue on the internet and provide one link to material you have read." ], "metadata": { "id": "SIzhe35JwS7b" } }, { "cell_type": "markdown", "source": [], "metadata": { "id": "JWQe9y3RwVtV" } }, { "cell_type": "markdown", "source": [ "### Q4.2\n", "\n", "Use some the bias terminology of Suresh and Guttag, 2021 (see lecture slides), to discuss whether you think the Northpointe system is a responsible use of AI." ], "metadata": { "id": "y8agoGL8wV5W" } }, { "cell_type": "markdown", "source": [], "metadata": { "id": "q81oP1N8waO1" } }, { "cell_type": "markdown", "source": [ "### Q4.3\n", "\n", "Research at BU and MIT by Canetti et al. ([arxiv.org/pdf/1810.02003.pdf](http://arxiv.org/pdf/1810.02003.pdf)) considers the scenario that decisions whether a prisoner should stay or leave prison can be deferred. In the publicly-available Northpointe data, decisions about Caucasian prisoners are deferred for 9% of prisoners, while decisions about African-Americans are deferred for 20% of the prisoners. Canetti et al. propose \"equalizing methods\" to change the AI system so that deferral decisions would be made at about the same rate for both racial groups. Briefly discuss what you think about adding an equalizing method to Northpointe's system. Would this yield a fair AI system? You may refer to Suresh and Guttag's terminology." ], "metadata": { "id": "Tpf2wWh6wadr" } }, { "cell_type": "markdown", "source": [], "metadata": { "id": "r_7l8dcBwtaf" } } ] }