2-Synthetic-Oversampling-NearMiss-Undersampling-EasyEsemble.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[],"collapsed_sections":["NSLqkKdd6hDg","hmzYDND-6nGa","xBv8NyF3_t8a"],"toc_visible":true,"authorship_tag":"ABX9TyOmN/DEpOHmpZnq/D9w9tIK"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["# Imports"],"metadata":{"id":"NSLqkKdd6hDg"}},{"cell_type":"code","execution_count":null,"metadata":{"id":"aIRiAkY95wG0"},"outputs":[],"source":["import pandas as pd\n","import numpy as np\n","\n","from sklearn.svm import SVC \n","from sklearn.ensemble import RandomForestClassifier\n","from sklearn.naive_bayes import GaussianNB\n","from sklearn.model_selection import train_test_split\n","from sklearn.preprocessing import MinMaxScaler\n","from imblearn.under_sampling import NearMiss\n","from imblearn.over_sampling import SMOTE\n","from imblearn.ensemble import EasyEnsembleClassifier\n","from imblearn.pipeline import make_pipeline\n","from imblearn.metrics import classification_report_imbalanced"]},{"cell_type":"markdown","source":["# Check the data"],"metadata":{"id":"hmzYDND-6nGa"}},{"cell_type":"code","source":["data_df = pd.read_csv(\"/content/carclaims.csv\")\n","data_df"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"hN_7K0nV6lnt","executionInfo":{"status":"ok","timestamp":1680579465419,"user_tz":-180,"elapsed":803,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}},"outputId":"a22dc2b3-f7bf-42bd-acdb-52f554ce88b7"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["      Month  WeekOfMonth  DayOfWeek     Make AccidentArea DayOfWeekClaimed  \\\n","0       Dec            5  Wednesday    Honda        Urban          Tuesday   \n","1       Jan            3  Wednesday    Honda        Urban           Monday   \n","2       Oct            5     Friday    Honda        Urban         Thursday   \n","3       Jun            2   Saturday   Toyota        Rural           Friday   \n","4       Jan            5     Monday    Honda        Urban          Tuesday   \n","...     ...          ...        ...      ...          ...              ...   \n","15415   Nov            4     Friday   Toyota        Urban          Tuesday   \n","15416   Nov            5   Thursday  Pontiac        Urban           Friday   \n","15417   Nov            5   Thursday   Toyota        Rural           Friday   \n","15418   Dec            1     Monday   Toyota        Urban         Thursday   \n","15419   Dec            2  Wednesday   Toyota        Urban         Thursday   \n","\n","      MonthClaimed  WeekOfMonthClaimed     Sex MaritalStatus  ...  \\\n","0              Jan                   1  Female        Single  ...   \n","1              Jan                   4    Male        Single  ...   \n","2              Nov                   2    Male       Married  ...   \n","3              Jul                   1    Male       Married  ...   \n","4              Feb                   2  Female        Single  ...   \n","...            ...                 ...     ...           ...  ...   \n","15415          Nov                   5    Male       Married  ...   \n","15416          Dec                   1    Male       Married  ...   \n","15417          Dec                   1    Male        Single  ...   \n","15418          Dec                   2  Female       Married  ...   \n","15419          Dec                   3    Male        Single  ...   \n","\n","       AgeOfPolicyHolder PoliceReportFiled WitnessPresent AgentType  \\\n","0               26 to 30                No             No  External   \n","1               31 to 35               Yes             No  External   \n","2               41 to 50                No             No  External   \n","3               51 to 65               Yes             No  External   \n","4               31 to 35                No             No  External   \n","...                  ...               ...            ...       ...   \n","15415           31 to 35                No             No  External   \n","15416           31 to 35                No             No  External   \n","15417           26 to 30                No             No  External   \n","15418           31 to 35                No             No  External   \n","15419           26 to 30                No             No  External   \n","\n","      NumberOfSuppliments  AddressChange-Claim  NumberOfCars  Year  \\\n","0                    none               1 year        3 to 4  1994   \n","1                    none            no change     1 vehicle  1994   \n","2                    none            no change     1 vehicle  1994   \n","3             more than 5            no change     1 vehicle  1994   \n","4                    none            no change     1 vehicle  1994   \n","...                   ...                  ...           ...   ...   \n","15415                none            no change     1 vehicle  1996   \n","15416         more than 5            no change        3 to 4  1996   \n","15417              1 to 2            no change     1 vehicle  1996   \n","15418         more than 5            no change     1 vehicle  1996   \n","15419              1 to 2            no change     1 vehicle  1996   \n","\n","       BasePolicy FraudFound  \n","0       Liability         No  \n","1       Collision         No  \n","2       Collision         No  \n","3       Liability         No  \n","4       Collision         No  \n","...           ...        ...  \n","15415   Collision        Yes  \n","15416   Liability         No  \n","15417   Collision        Yes  \n","15418  All Perils         No  \n","15419   Collision        Yes  \n","\n","[15420 rows x 33 columns]"],"text/html":["\n","  <div id=\"df-feb02499-9718-4636-9117-2c231f42d15b\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Month</th>\n","      <th>WeekOfMonth</th>\n","      <th>DayOfWeek</th>\n","      <th>Make</th>\n","      <th>AccidentArea</th>\n","      <th>DayOfWeekClaimed</th>\n","      <th>MonthClaimed</th>\n","      <th>WeekOfMonthClaimed</th>\n","      <th>Sex</th>\n","      <th>MaritalStatus</th>\n","      <th>...</th>\n","      <th>AgeOfPolicyHolder</th>\n","      <th>PoliceReportFiled</th>\n","      <th>WitnessPresent</th>\n","      <th>AgentType</th>\n","      <th>NumberOfSuppliments</th>\n","      <th>AddressChange-Claim</th>\n","      <th>NumberOfCars</th>\n","      <th>Year</th>\n","      <th>BasePolicy</th>\n","      <th>FraudFound</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Dec</td>\n","      <td>5</td>\n","      <td>Wednesday</td>\n","      <td>Honda</td>\n","      <td>Urban</td>\n","      <td>Tuesday</td>\n","      <td>Jan</td>\n","      <td>1</td>\n","      <td>Female</td>\n","      <td>Single</td>\n","      <td>...</td>\n","      <td>26 to 30</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>none</td>\n","      <td>1 year</td>\n","      <td>3 to 4</td>\n","      <td>1994</td>\n","      <td>Liability</td>\n","      <td>No</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>Jan</td>\n","      <td>3</td>\n","      <td>Wednesday</td>\n","      <td>Honda</td>\n","      <td>Urban</td>\n","      <td>Monday</td>\n","      <td>Jan</td>\n","      <td>4</td>\n","      <td>Male</td>\n","      <td>Single</td>\n","      <td>...</td>\n","      <td>31 to 35</td>\n","      <td>Yes</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>none</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1994</td>\n","      <td>Collision</td>\n","      <td>No</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>Oct</td>\n","      <td>5</td>\n","      <td>Friday</td>\n","      <td>Honda</td>\n","      <td>Urban</td>\n","      <td>Thursday</td>\n","      <td>Nov</td>\n","      <td>2</td>\n","      <td>Male</td>\n","      <td>Married</td>\n","      <td>...</td>\n","      <td>41 to 50</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>none</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1994</td>\n","      <td>Collision</td>\n","      <td>No</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>Jun</td>\n","      <td>2</td>\n","      <td>Saturday</td>\n","      <td>Toyota</td>\n","      <td>Rural</td>\n","      <td>Friday</td>\n","      <td>Jul</td>\n","      <td>1</td>\n","      <td>Male</td>\n","      <td>Married</td>\n","      <td>...</td>\n","      <td>51 to 65</td>\n","      <td>Yes</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>more than 5</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1994</td>\n","      <td>Liability</td>\n","      <td>No</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>Jan</td>\n","      <td>5</td>\n","      <td>Monday</td>\n","      <td>Honda</td>\n","      <td>Urban</td>\n","      <td>Tuesday</td>\n","      <td>Feb</td>\n","      <td>2</td>\n","      <td>Female</td>\n","      <td>Single</td>\n","      <td>...</td>\n","      <td>31 to 35</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>none</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1994</td>\n","      <td>Collision</td>\n","      <td>No</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>15415</th>\n","      <td>Nov</td>\n","      <td>4</td>\n","      <td>Friday</td>\n","      <td>Toyota</td>\n","      <td>Urban</td>\n","      <td>Tuesday</td>\n","      <td>Nov</td>\n","      <td>5</td>\n","      <td>Male</td>\n","      <td>Married</td>\n","      <td>...</td>\n","      <td>31 to 35</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>none</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1996</td>\n","      <td>Collision</td>\n","      <td>Yes</td>\n","    </tr>\n","    <tr>\n","      <th>15416</th>\n","      <td>Nov</td>\n","      <td>5</td>\n","      <td>Thursday</td>\n","      <td>Pontiac</td>\n","      <td>Urban</td>\n","      <td>Friday</td>\n","      <td>Dec</td>\n","      <td>1</td>\n","      <td>Male</td>\n","      <td>Married</td>\n","      <td>...</td>\n","      <td>31 to 35</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>more than 5</td>\n","      <td>no change</td>\n","      <td>3 to 4</td>\n","      <td>1996</td>\n","      <td>Liability</td>\n","      <td>No</td>\n","    </tr>\n","    <tr>\n","      <th>15417</th>\n","      <td>Nov</td>\n","      <td>5</td>\n","      <td>Thursday</td>\n","      <td>Toyota</td>\n","      <td>Rural</td>\n","      <td>Friday</td>\n","      <td>Dec</td>\n","      <td>1</td>\n","      <td>Male</td>\n","      <td>Single</td>\n","      <td>...</td>\n","      <td>26 to 30</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>1 to 2</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1996</td>\n","      <td>Collision</td>\n","      <td>Yes</td>\n","    </tr>\n","    <tr>\n","      <th>15418</th>\n","      <td>Dec</td>\n","      <td>1</td>\n","      <td>Monday</td>\n","      <td>Toyota</td>\n","      <td>Urban</td>\n","      <td>Thursday</td>\n","      <td>Dec</td>\n","      <td>2</td>\n","      <td>Female</td>\n","      <td>Married</td>\n","      <td>...</td>\n","      <td>31 to 35</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>more than 5</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1996</td>\n","      <td>All Perils</td>\n","      <td>No</td>\n","    </tr>\n","    <tr>\n","      <th>15419</th>\n","      <td>Dec</td>\n","      <td>2</td>\n","      <td>Wednesday</td>\n","      <td>Toyota</td>\n","      <td>Urban</td>\n","      <td>Thursday</td>\n","      <td>Dec</td>\n","      <td>3</td>\n","      <td>Male</td>\n","      <td>Single</td>\n","      <td>...</td>\n","      <td>26 to 30</td>\n","      <td>No</td>\n","      <td>No</td>\n","      <td>External</td>\n","      <td>1 to 2</td>\n","      <td>no change</td>\n","      <td>1 vehicle</td>\n","      <td>1996</td>\n","      <td>Collision</td>\n","      <td>Yes</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>15420 rows × 33 columns</p>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-feb02499-9718-4636-9117-2c231f42d15b')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-feb02499-9718-4636-9117-2c231f42d15b button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-feb02499-9718-4636-9117-2c231f42d15b');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":2}]},{"cell_type":"code","source":["data_df.isna().sum()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"3aRoSQH49kcS","executionInfo":{"status":"ok","timestamp":1680579465790,"user_tz":-180,"elapsed":372,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}},"outputId":"bdd0ea1c-c160-4e10-9b6b-903c9044c613"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["Month                   0\n","WeekOfMonth             0\n","DayOfWeek               0\n","Make                    0\n","AccidentArea            0\n","DayOfWeekClaimed        0\n","MonthClaimed            0\n","WeekOfMonthClaimed      0\n","Sex                     0\n","MaritalStatus           0\n","Age                     0\n","Fault                   0\n","PolicyType              0\n","VehicleCategory         0\n","VehiclePrice            0\n","PolicyNumber            0\n","RepNumber               0\n","Deductible              0\n","DriverRating            0\n","Days:Policy-Accident    0\n","Days:Policy-Claim       0\n","PastNumberOfClaims      0\n","AgeOfVehicle            0\n","AgeOfPolicyHolder       0\n","PoliceReportFiled       0\n","WitnessPresent          0\n","AgentType               0\n","NumberOfSuppliments     0\n","AddressChange-Claim     0\n","NumberOfCars            0\n","Year                    0\n","BasePolicy              0\n","FraudFound              0\n","dtype: int64"]},"metadata":{},"execution_count":3}]},{"cell_type":"code","source":["data_df.columns"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"i0E9rELS6vQr","executionInfo":{"status":"ok","timestamp":1680579466070,"user_tz":-180,"elapsed":285,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}},"outputId":"e29a2ea0-0b44-4ac5-ad5d-6dd77f5afea2"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["Index(['Month', 'WeekOfMonth', 'DayOfWeek', 'Make', 'AccidentArea',\n","       'DayOfWeekClaimed', 'MonthClaimed', 'WeekOfMonthClaimed', 'Sex',\n","       'MaritalStatus', 'Age', 'Fault', 'PolicyType', 'VehicleCategory',\n","       'VehiclePrice', 'PolicyNumber', 'RepNumber', 'Deductible',\n","       'DriverRating', 'Days:Policy-Accident', 'Days:Policy-Claim',\n","       'PastNumberOfClaims', 'AgeOfVehicle', 'AgeOfPolicyHolder',\n","       'PoliceReportFiled', 'WitnessPresent', 'AgentType',\n","       'NumberOfSuppliments', 'AddressChange-Claim', 'NumberOfCars', 'Year',\n","       'BasePolicy', 'FraudFound'],\n","      dtype='object')"]},"metadata":{},"execution_count":4}]},{"cell_type":"code","source":["for col in data_df.columns:\n","  print(col)\n","  print(data_df[col].value_counts())\n","  print()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"cjs4gV0-7MOl","executionInfo":{"status":"ok","timestamp":1680579466356,"user_tz":-180,"elapsed":291,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}},"outputId":"23437f06-b477-4aeb-c5ff-463bc6aa9577"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Month\n","Jan    1411\n","May    1367\n","Mar    1360\n","Jun    1321\n","Oct    1305\n","Dec    1285\n","Apr    1280\n","Feb    1266\n","Jul    1257\n","Sep    1240\n","Nov    1201\n","Aug    1127\n","Name: Month, dtype: int64\n","\n","WeekOfMonth\n","3    3640\n","2    3558\n","4    3398\n","1    3187\n","5    1637\n","Name: WeekOfMonth, dtype: int64\n","\n","DayOfWeek\n","Monday       2616\n","Friday       2445\n","Tuesday      2300\n","Thursday     2173\n","Wednesday    2159\n","Saturday     1982\n","Sunday       1745\n","Name: DayOfWeek, dtype: int64\n","\n","Make\n","Pontiac      3837\n","Toyota       3121\n","Honda        2801\n","Mazda        2354\n","Chevrolet    1681\n","Accura        472\n","Ford          450\n","VW            283\n","Dodge         109\n","Saab          108\n","Mercury        83\n","Saturn         58\n","Nisson         30\n","BMW            15\n","Jaguar          6\n","Porche          5\n","Mecedes         4\n","Ferrari         2\n","Lexus           1\n","Name: Make, dtype: int64\n","\n","AccidentArea\n","Urban    13822\n","Rural     1598\n","Name: AccidentArea, dtype: int64\n","\n","DayOfWeekClaimed\n","Monday       3757\n","Tuesday      3375\n","Wednesday    2951\n","Thursday     2660\n","Friday       2497\n","Saturday      127\n","Sunday         52\n","0               1\n","Name: DayOfWeekClaimed, dtype: int64\n","\n","MonthClaimed\n","Jan    1446\n","May    1411\n","Mar    1348\n","Oct    1339\n","Jun    1293\n","Feb    1287\n","Nov    1285\n","Apr    1271\n","Sep    1242\n","Jul    1225\n","Dec    1146\n","Aug    1126\n","0         1\n","Name: MonthClaimed, dtype: int64\n","\n","WeekOfMonthClaimed\n","2    3720\n","3    3583\n","1    3450\n","4    3433\n","5    1234\n","Name: WeekOfMonthClaimed, dtype: int64\n","\n","Sex\n","Male      13000\n","Female     2420\n","Name: Sex, dtype: int64\n","\n","MaritalStatus\n","Married     10625\n","Single       4684\n","Divorced       76\n","Widow          35\n","Name: MaritalStatus, dtype: int64\n","\n","Age\n","30    596\n","33    574\n","34    573\n","35    569\n","28    560\n","     ... \n","20     28\n","70     27\n","79     20\n","16      9\n","17      6\n","Name: Age, Length: 66, dtype: int64\n","\n","Fault\n","Policy Holder    11230\n","Third Party       4190\n","Name: Fault, dtype: int64\n","\n","PolicyType\n","Sedan - Collision       5584\n","Sedan - Liability       4987\n","Sedan - All Perils      4087\n","Sport - Collision        348\n","Utility - All Perils     340\n","Utility - Collision       30\n","Sport - All Perils        22\n","Utility - Liability       21\n","Sport - Liability          1\n","Name: PolicyType, dtype: int64\n","\n","VehicleCategory\n","Sedan      9671\n","Sport      5358\n","Utility     391\n","Name: VehicleCategory, dtype: int64\n","\n","VehiclePrice\n","20,000 to 29,000    8079\n","30,000 to 39,000    3533\n","more than 69,000    2164\n","less than 20,000    1096\n","40,000 to 59,000     461\n","60,000 to 69,000      87\n","Name: VehiclePrice, dtype: int64\n","\n","PolicyNumber\n","1        1\n","10361    1\n","10273    1\n","10274    1\n","10275    1\n","        ..\n","5145     1\n","5146     1\n","5147     1\n","5148     1\n","15420    1\n","Name: PolicyNumber, Length: 15420, dtype: int64\n","\n","RepNumber\n","7     1069\n","9      999\n","1      987\n","5      987\n","10     986\n","12     977\n","15     977\n","16     967\n","2      956\n","3      949\n","11     948\n","6      942\n","14     941\n","8      931\n","4      912\n","13     892\n","Name: RepNumber, dtype: int64\n","\n","Deductible\n","400    14838\n","700      311\n","500      263\n","300        8\n","Name: Deductible, dtype: int64\n","\n","DriverRating\n","1    3944\n","3    3884\n","2    3801\n","4    3791\n","Name: DriverRating, dtype: int64\n","\n","Days:Policy-Accident\n","more than 30    15247\n","none               55\n","8 to 15            55\n","15 to 30           49\n","1 to 7             14\n","Name: Days:Policy-Accident, dtype: int64\n","\n","Days:Policy-Claim\n","more than 30    15342\n","15 to 30           56\n","8 to 15            21\n","none                1\n","Name: Days:Policy-Claim, dtype: int64\n","\n","PastNumberOfClaims\n","2 to 4         5485\n","none           4352\n","1              3573\n","more than 4    2010\n","Name: PastNumberOfClaims, dtype: int64\n","\n","AgeOfVehicle\n","7 years        5807\n","more than 7    3981\n","6 years        3448\n","5 years        1357\n","new             373\n","4 years         229\n","3 years         152\n","2 years          73\n","Name: AgeOfVehicle, dtype: int64\n","\n","AgeOfPolicyHolder\n","31 to 35    5593\n","36 to 40    4043\n","41 to 50    2828\n","51 to 65    1392\n","26 to 30     613\n","over 65      508\n","16 to 17     320\n","21 to 25     108\n","18 to 20      15\n","Name: AgeOfPolicyHolder, dtype: int64\n","\n","PoliceReportFiled\n","No     14992\n","Yes      428\n","Name: PoliceReportFiled, dtype: int64\n","\n","WitnessPresent\n","No     15333\n","Yes       87\n","Name: WitnessPresent, dtype: int64\n","\n","AgentType\n","External    15179\n","Internal      241\n","Name: AgentType, dtype: int64\n","\n","NumberOfSuppliments\n","none           7047\n","more than 5    3867\n","1 to 2         2489\n","3 to 5         2017\n","Name: NumberOfSuppliments, dtype: int64\n","\n","AddressChange-Claim\n","no change         14324\n","4 to 8 years        631\n","2 to 3 years        291\n","1 year              170\n","under 6 months        4\n","Name: AddressChange-Claim, dtype: int64\n","\n","NumberOfCars\n","1 vehicle      14316\n","2 vehicles       709\n","3 to 4           372\n","5 to 8            21\n","more than 8        2\n","Name: NumberOfCars, dtype: int64\n","\n","Year\n","1994    6142\n","1995    5195\n","1996    4083\n","Name: Year, dtype: int64\n","\n","BasePolicy\n","Collision     5962\n","Liability     5009\n","All Perils    4449\n","Name: BasePolicy, dtype: int64\n","\n","FraudFound\n","No     14497\n","Yes      923\n","Name: FraudFound, dtype: int64\n","\n"]}]},{"cell_type":"markdown","source":["# Read and Preprocess the data"],"metadata":{"id":"xBv8NyF3_t8a"}},{"cell_type":"code","source":["# Convert categorical values to numeric\n","cols_to_keep_as_they_are = ['WeekOfMonth', 'WeekOfMonthClaimed', 'Age', 'PolicyNumber', 'RepNumber', 'DriverRating']\n","for col in data_df.drop(cols_to_keep_as_they_are, axis=1).columns:\n","  unique_vals = list(data_df[col].unique())\n","  replaced_vals = list(np.arange(len(unique_vals)))\n","  data_df[col].replace(unique_vals, replaced_vals, inplace=True)\n","\n","data_df.head()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"CgMlv-ps7r3T","executionInfo":{"status":"ok","timestamp":1680579468046,"user_tz":-180,"elapsed":1693,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}},"outputId":"f22536c6-2537-47bb-eb07-390a0bb8bd82"},"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/plain":["   Month  WeekOfMonth  DayOfWeek  Make  AccidentArea  DayOfWeekClaimed  \\\n","0      0            5          0     0             0                 0   \n","1      1            3          0     0             0                 1   \n","2      2            5          1     0             0                 2   \n","3      3            2          2     1             1                 3   \n","4      1            5          3     0             0                 0   \n","\n","   MonthClaimed  WeekOfMonthClaimed  Sex  MaritalStatus  ...  \\\n","0             0                   1    0              0  ...   \n","1             0                   4    1              0  ...   \n","2             1                   2    1              1  ...   \n","3             2                   1    1              1  ...   \n","4             3                   2    0              0  ...   \n","\n","   AgeOfPolicyHolder  PoliceReportFiled  WitnessPresent  AgentType  \\\n","0                  0                  0               0          0   \n","1                  1                  1               0          0   \n","2                  2                  0               0          0   \n","3                  3                  1               0          0   \n","4                  1                  0               0          0   \n","\n","   NumberOfSuppliments  AddressChange-Claim  NumberOfCars  Year  BasePolicy  \\\n","0                    0                    0             0     0           0   \n","1                    0                    1             1     0           1   \n","2                    0                    1             1     0           1   \n","3                    1                    1             1     0           0   \n","4                    0                    1             1     0           1   \n","\n","   FraudFound  \n","0           0  \n","1           0  \n","2           0  \n","3           0  \n","4           0  \n","\n","[5 rows x 33 columns]"],"text/html":["\n","  <div id=\"df-2276a253-a60d-41de-bae0-b790c1f2c0dd\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>Month</th>\n","      <th>WeekOfMonth</th>\n","      <th>DayOfWeek</th>\n","      <th>Make</th>\n","      <th>AccidentArea</th>\n","      <th>DayOfWeekClaimed</th>\n","      <th>MonthClaimed</th>\n","      <th>WeekOfMonthClaimed</th>\n","      <th>Sex</th>\n","      <th>MaritalStatus</th>\n","      <th>...</th>\n","      <th>AgeOfPolicyHolder</th>\n","      <th>PoliceReportFiled</th>\n","      <th>WitnessPresent</th>\n","      <th>AgentType</th>\n","      <th>NumberOfSuppliments</th>\n","      <th>AddressChange-Claim</th>\n","      <th>NumberOfCars</th>\n","      <th>Year</th>\n","      <th>BasePolicy</th>\n","      <th>FraudFound</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>5</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>...</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>4</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>...</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>2</td>\n","      <td>5</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>...</td>\n","      <td>2</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>...</td>\n","      <td>3</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>1</td>\n","      <td>5</td>\n","      <td>3</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>2</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>...</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>5 rows × 33 columns</p>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2276a253-a60d-41de-bae0-b790c1f2c0dd')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-2276a253-a60d-41de-bae0-b790c1f2c0dd button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-2276a253-a60d-41de-bae0-b790c1f2c0dd');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":6}]},{"cell_type":"code","source":["# abs(data_df.corr()['FraudFound']).sort_values(ascending=False)"],"metadata":{"id":"c4vzkIWyCJVI"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":["# PARAMETERS\n","TARGET = 'FraudFound'\n","TEST_SIZE = 0.25\n","RANDOM_STATE = 0\n","\n","# Split to train and test sets \n","X_train_df, X_test_df, y_train_df, y_test_df = train_test_split(data_df.drop(TARGET, axis=1),\n","                                                                data_df[TARGET], \n","                                                                test_size=TEST_SIZE,\n","                                                                shuffle=True,\n","                                                                random_state=RANDOM_STATE)\n","\n","# Scale data\n","scaler = MinMaxScaler()\n","X_train_sc = scaler.fit_transform(X_train_df)\n","X_train_df_sc = pd.DataFrame(X_train_sc, columns=X_train_df.columns, index=X_train_df.index)\n","X_test_sc = scaler.transform(X_test_df)\n","X_test_df_sc = pd.DataFrame(X_test_sc, columns=X_test_df.columns, index=X_test_df.index)\n","\n","print(X_train_df.shape, X_test_df.shape)\n","print(y_train_df.shape, y_test_df.shape)\n","\n","# Change 0's to -1's, because SVM requires that labels are +1 and -1\n","y_train_df[y_train_df == 0.] = -1\n","y_test_df[y_test_df == 0.] = -1"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"3LKn_BzoCWmE","executionInfo":{"status":"ok","timestamp":1680579468047,"user_tz":-180,"elapsed":7,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}},"outputId":"2bb3fff1-1461-4c13-9309-6b9ccdc6c5e9"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["(11565, 32) (3855, 32)\n","(11565,) (3855,)\n"]}]},{"cell_type":"markdown","source":["# 1. Synthetic Oversampling"],"metadata":{"id":"cn7MXYqBF2Az"}},{"cell_type":"code","source":["# create the classifiers\n","classifiers = {\n","    'random forest': RandomForestClassifier(n_estimators=100, random_state=0),\n","    'linear SVM': SVC(kernel='linear'),\n","    'naive bayes': GaussianNB(),\n","}\n","\n","y_test = y_test_df.to_numpy(dtype='int64')\n","\n","for name in classifiers.keys():\n","  print(f'======================== {name} ========================')\n","\n","  print(\"\\n***** WITHOUT over-sampling *****\")\n","  pipeline = make_pipeline(classifiers[name])\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))\n","\n","  print(\"\\n***** WITH over-sampling*****\")\n","  pipeline = make_pipeline(SMOTE(random_state=0, k_neighbors=5),\n","                           classifiers[name])\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"NfdJIW8IF4nC","executionInfo":{"status":"ok","timestamp":1680582283963,"user_tz":-180,"elapsed":2810195,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}},"outputId":"776ba63c-93ab-4283-bfab-b5ef27f332ba"},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["======================== random forest ========================\n","\n","***** WITHOUT over-sampling *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.94      1.00      0.02      0.97      0.15      0.02      3624\n","          1       0.71      0.02      1.00      0.04      0.15      0.02       231\n","\n","avg / total       0.93      0.94      0.08      0.91      0.15      0.02      3855\n","\n","\n","***** WITH over-sampling*****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.95      0.97      0.12      0.96      0.34      0.13      3624\n","          1       0.22      0.12      0.97      0.16      0.34      0.11       231\n","\n","avg / total       0.90      0.92      0.17      0.91      0.34      0.13      3855\n","\n","======================== linear SVM ========================\n","\n","***** WITHOUT over-sampling *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.94      0.99      0.02      0.97      0.15      0.02      3624\n","          1       0.14      0.02      0.99      0.04      0.15      0.02       231\n","\n","avg / total       0.89      0.93      0.08      0.91      0.15      0.02      3855\n","\n","\n","***** WITH over-sampling*****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.96      0.79      0.48      0.87      0.61      0.39      3624\n","          1       0.13      0.48      0.79      0.20      0.61      0.36       231\n","\n","avg / total       0.91      0.77      0.49      0.83      0.61      0.39      3855\n","\n","======================== naive bayes ========================\n","\n","***** WITHOUT over-sampling *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.95      0.94      0.21      0.95      0.45      0.21      3624\n","          1       0.20      0.21      0.94      0.20      0.45      0.19       231\n","\n","avg / total       0.90      0.90      0.26      0.90      0.45      0.21      3855\n","\n","\n","***** WITH over-sampling*****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.97      0.70      0.65      0.82      0.68      0.46      3624\n","          1       0.12      0.65      0.70      0.21      0.68      0.46       231\n","\n","avg / total       0.92      0.70      0.66      0.78      0.68      0.46      3855\n","\n"]}]},{"cell_type":"markdown","source":["Από τα παραπάνω αποτελέσματα παρατηρώ τα παρακάτω:\n","1. Για το **Random Forest**:\n","  * όταν εφαρμόζεται *synthetic oversampling* μειώνονται *ελάχιστα* οι μετρικές *precision* και *recall*, ωστόσο όλες οι υπόλοιπες μετρικές, εκτός της *f1*, που *παραμένει ίδια*, βελτιώνοντα σημαντικά.\n","\n","2. Για το **Linear SVM**, όταν εφαρμόζεται *synthetic oversampling*:\n","  * μειώνεται *ελάχιστα* η μετρική *precision* \n","  * μειώνονται *σημαντικά* οι μετρικές *recall* και *f1*\n","  * ωστόσο όλες οι υπόλοιπες μετρικές βελτιώνοντα σημαντικά.\n","\n","3. Για το **Naive Bayes**, όταν εφαρμόζεται *synthetic oversampling*::\n","  * βελτιώνεται *ελάχιστα* η μετρική *precision* \n","  * μειώνονται *σημαντικά* οι μετρικές *recall* και *f1*\n","  * ωστόσο όλες οι υπόλοιπες μετρικές βελτιώνοντα σημαντικά.\n","\n","Συνολικά, παρατηρούμε ότι:\n","* στην προκειμένη περίπτωση **με τη χρήση του *synthetic oversampling* επιτυγχάνεται ισορροπία στην επίδοση όλων των μετρικών και στους 3 παραπάνω αλγορίθμους**\n","* θα μπορούσαμε να πούμε ότι **η χρήση του *synthetic oversampling* ΔΕΝ συνίσταται όταν όταν μας ενδιαφέρει να βελτιστοποιήσουμε κάποια από τις μετρικές *precision*, *recall* ή *f1***.\n"],"metadata":{"id":"357aGg457fxl"}},{"cell_type":"markdown","source":["# 2. Undersampling - NearMiss"],"metadata":{"id":"ZlpBaAclITqj"}},{"cell_type":"code","source":["# create the classifiers\n","classifiers = {\n","    'random forest': RandomForestClassifier(n_estimators=100, random_state=0),\n","    'linear SVM': SVC(kernel='linear'),\n","    'naive bayes': GaussianNB(),\n","}\n","\n","y_test = y_test_df.to_numpy(dtype='int64')\n","\n","for name in classifiers.keys():\n","  print(f'======================== {name} ========================')\n","\n","  print(\"\\n***** WITHOUT under-sampling *****\")\n","  pipeline = make_pipeline(classifiers[name])\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))\n","\n","  print(\"\\n***** WITH NearMiss - version 1 *****\")\n","  pipeline = make_pipeline(NearMiss(version=1),\n","                           classifiers[name])\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))\n","\n","  print(\"\\n***** WITH NearMiss - version 2 *****\")\n","  pipeline = make_pipeline(NearMiss(version=2),\n","                           classifiers[name])\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))\n","\n","  print(\"\\n***** WITH NearMiss - version 3 *****\")\n","  pipeline = make_pipeline(NearMiss(version=3),\n","                           classifiers[name])\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"outputId":"5a7fa7e3-7cdb-4a32-e093-97fcb0880931","id":"d5CxYvGAITqk","executionInfo":{"status":"ok","timestamp":1680585121489,"user_tz":-180,"elapsed":2837545,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["======================== random forest ========================\n","\n","***** WITHOUT under-sampling *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.94      1.00      0.02      0.97      0.15      0.02      3624\n","          1       0.71      0.02      1.00      0.04      0.15      0.02       231\n","\n","avg / total       0.93      0.94      0.08      0.91      0.15      0.02      3855\n","\n","\n","***** WITH NearMiss - version 1 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.99      0.63      0.90      0.77      0.75      0.55      3624\n","          1       0.13      0.90      0.63      0.23      0.75      0.58       231\n","\n","avg / total       0.94      0.65      0.88      0.74      0.75      0.55      3855\n","\n","\n","***** WITH NearMiss - version 2 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.97      0.11      0.94      0.20      0.33      0.10      3624\n","          1       0.06      0.94      0.11      0.12      0.33      0.12       231\n","\n","avg / total       0.92      0.16      0.89      0.20      0.33      0.10      3855\n","\n","\n","***** WITH NearMiss - version 3 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.99      0.66      0.91      0.79      0.77      0.58      3624\n","          1       0.14      0.91      0.66      0.25      0.77      0.61       231\n","\n","avg / total       0.94      0.67      0.89      0.76      0.77      0.58      3855\n","\n","======================== linear SVM ========================\n","\n","***** WITHOUT under-sampling *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.94      0.99      0.02      0.97      0.15      0.02      3624\n","          1       0.14      0.02      0.99      0.04      0.15      0.02       231\n","\n","avg / total       0.89      0.93      0.08      0.91      0.15      0.02      3855\n","\n","\n","***** WITH NearMiss - version 1 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.98      0.61      0.84      0.75      0.71      0.50      3624\n","          1       0.12      0.84      0.61      0.21      0.71      0.52       231\n","\n","avg / total       0.93      0.62      0.82      0.72      0.71      0.50      3855\n","\n","\n","***** WITH NearMiss - version 2 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.98      0.63      0.84      0.77      0.73      0.52      3624\n","          1       0.13      0.84      0.63      0.22      0.73      0.54       231\n","\n","avg / total       0.93      0.64      0.82      0.74      0.73      0.52      3855\n","\n","\n","***** WITH NearMiss - version 3 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.98      0.66      0.81      0.79      0.73      0.52      3624\n","          1       0.13      0.81      0.66      0.23      0.73      0.54       231\n","\n","avg / total       0.93      0.67      0.80      0.76      0.73      0.53      3855\n","\n","======================== naive bayes ========================\n","\n","***** WITHOUT under-sampling *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.95      0.94      0.21      0.95      0.45      0.21      3624\n","          1       0.20      0.21      0.94      0.20      0.45      0.19       231\n","\n","avg / total       0.90      0.90      0.26      0.90      0.45      0.21      3855\n","\n","\n","***** WITH NearMiss - version 1 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.98      0.64      0.80      0.78      0.71      0.50      3624\n","          1       0.12      0.80      0.64      0.21      0.71      0.52       231\n","\n","avg / total       0.93      0.65      0.79      0.74      0.71      0.50      3855\n","\n","\n","***** WITH NearMiss - version 2 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.97      0.15      0.93      0.25      0.37      0.12      3624\n","          1       0.06      0.93      0.15      0.12      0.37      0.15       231\n","\n","avg / total       0.92      0.19      0.88      0.25      0.37      0.13      3855\n","\n","\n","***** WITH NearMiss - version 3 *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.98      0.64      0.82      0.78      0.73      0.52      3624\n","          1       0.13      0.82      0.64      0.22      0.73      0.54       231\n","\n","avg / total       0.93      0.65      0.81      0.74      0.73      0.52      3855\n","\n"]}]},{"cell_type":"markdown","source":["Από τα παραπάνω αποτελέσματα παρατηρώ τα παρακάτω:\n","1. Για το **Random Forest**:\n","  * όταν εφαρμόζεται η τεχνική *NearMiss 1* μειώνονται *σημαντικά* οι μετρικές *recall* και *f1*, ωστόσο όλες οι υπόλοιπες μετρικές βελτιώνοντα σημαντικά.\n","  * όταν εφαρμόζεται η τεχνική *NearMiss 2* μειώνονται *σημαντικά* οι μετρικές *recall* και *f1*, ωστόσο όλες οι υπόλοιπες μετρικές βελτιώνοντα σημαντικά, εκτός της *precision* που μειώνεται ελάχιστα.\n","  * όταν εφαρμόζεται η τεχνική *NearMiss 3* ισχύουν τα ίδια με την *NearMiss 1*.\n","\n","2. Για το **Linear SVM**, όταν εφαρμόζεται *synthetic oversampling*:\n","  * όταν εφαρμόζεται η τεχνική *NearMiss 1* μειώνονται *σημαντικά* οι μετρικές *recall* και *f1*, ωστόσο όλες οι υπόλοιπες μετρικές βελτιώνοντα σημαντικά.\n","  * όταν εφαρμόζονται οι τεχνικές *NearMiss 2* ή *NearMiss 3* ισχύουν τα ίδια με την *NearMiss 1*.\n","\n","3. Για το **Naive Bayes**, όταν εφαρμόζεται *synthetic oversampling*::\n","  * όταν εφαρμόζεται η τεχνική *NearMiss 1* μειώνονται *σημαντικά* οι μετρικές *recall* και *f1*, ωστόσο όλες οι υπόλοιπες μετρικές βελτιώνοντα σημαντικά.\n","  * όταν εφαρμόζεται η τεχνική *NearMiss 2* μειώνονται *ακόμη περισ΄΄οτερο* οι μετρικές *recall* και *f1*, μειώνονται η *geo* και η *iba*, ωστόσο όλες οι υπόλοιπες μετρικές βελτιώνοντα σημαντικά.\n","  * όταν εφαρμόζεται η τεχνική *NearMiss 3* ισχύουν τα ίδια με την *NearMiss 1*.\n","\n","Συνολικά, παρατηρούμε ότι:\n","* στην προκειμένη περίπτωση **με τη χρήση του *NearMiss* επιτυγχάνεται ισορροπία στην επίδοση όλων των μετρικών και στους 3 παραπάνω αλγορίθμους**.\n","* θα μπορούσαμε να πούμε ότι **η χρήση του *NearMiss* ΔΕΝ συνίσταται όταν όταν μας ενδιαφέρει να βελτιστοποιήσουμε κάποια από τις μετρικές *precision*, *recall* ή *f1***.\n","* Θα πρέπει να ελέγχεται ποιός από τους αλγορίθμους *NearMiss 1* ή *NearMiss 2* είναι καλύτερο στην κάθε περίπτωση.\n","* **Όπως είχαμε πει στο μάθημα, ο *NearMiss 3* δεν φαίνεται να αποτελεί ξεχωριστό αλγόριθμο, καθώς και στις τρεις περιπτώσεις πετυχαίνει τα ίδια αποτελέσματα σε σχέση με το *NearMiss 1***."],"metadata":{"id":"RravnSCPDP8X"}},{"cell_type":"markdown","source":["# 3. EasyEnsemble"],"metadata":{"id":"DyPB3f-TI9EF"}},{"cell_type":"code","source":["# create the classifiers\n","classifiers = {\n","    'random forest': RandomForestClassifier(n_estimators=100, random_state=0),\n","    'linear SVM': SVC(kernel='linear'),\n","    'naive bayes': GaussianNB(),\n","}\n","\n","y_test = y_test_df.to_numpy(dtype='int64')\n","\n","for name in classifiers.keys():\n","  print(f'======================== {name} ========================')\n","\n","  print(\"\\n***** WITHOUT EasyEnsemble *****\")\n","  pipeline = make_pipeline(classifiers[name])\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))\n","\n","  print(\"\\n***** WITH EasyEnsemble *****\")\n","  pipeline = make_pipeline(EasyEnsembleClassifier(random_state=42))\n","  pipeline.fit(X_train_df, y_train_df)\n","  y_preds = pipeline.predict(X_test_df)\n","  print(classification_report_imbalanced(y_test, y_preds))\n"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"outputId":"23a4185f-10d7-42f3-ba1c-8ee803834926","id":"8jCp9aREI9EF","executionInfo":{"status":"ok","timestamp":1680586581843,"user_tz":-180,"elapsed":1460366,"user":{"displayName":"Achil Pal","userId":"04608072963754583562"}}},"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["======================== random forest ========================\n","\n","***** WITHOUT EasyEnsemble *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.94      1.00      0.02      0.97      0.15      0.02      3624\n","          1       0.71      0.02      1.00      0.04      0.15      0.02       231\n","\n","avg / total       0.93      0.94      0.08      0.91      0.15      0.02      3855\n","\n","\n","***** WITH EasyEnsemble *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.99      0.62      0.86      0.76      0.73      0.52      3624\n","          1       0.13      0.86      0.62      0.22      0.73      0.54       231\n","\n","avg / total       0.93      0.63      0.84      0.73      0.73      0.52      3855\n","\n","======================== linear SVM ========================\n","\n","***** WITHOUT EasyEnsemble *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.94      0.99      0.02      0.97      0.15      0.02      3624\n","          1       0.14      0.02      0.99      0.04      0.15      0.02       231\n","\n","avg / total       0.89      0.93      0.08      0.91      0.15      0.02      3855\n","\n","\n","***** WITH EasyEnsemble *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.99      0.62      0.86      0.76      0.73      0.52      3624\n","          1       0.13      0.86      0.62      0.22      0.73      0.54       231\n","\n","avg / total       0.93      0.63      0.84      0.73      0.73      0.52      3855\n","\n","======================== naive bayes ========================\n","\n","***** WITHOUT EasyEnsemble *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.95      0.94      0.21      0.95      0.45      0.21      3624\n","          1       0.20      0.21      0.94      0.20      0.45      0.19       231\n","\n","avg / total       0.90      0.90      0.26      0.90      0.45      0.21      3855\n","\n","\n","***** WITH EasyEnsemble *****\n","                   pre       rec       spe        f1       geo       iba       sup\n","\n","         -1       0.99      0.62      0.86      0.76      0.73      0.52      3624\n","          1       0.13      0.86      0.62      0.22      0.73      0.54       231\n","\n","avg / total       0.93      0.63      0.84      0.73      0.73      0.52      3855\n","\n"]}]},{"cell_type":"markdown","source":["Από τα παραπάνω αποτελέσματα παρατηρώ τα παρακάτω:\n","1. Για το **Random Forest**:\n","  * όταν εφαρμόζεται o *Easy Ensemble* μειώνονται *σημαντικά* οι μετρικές *f1* και *recall*, ωστόσο όλες οι υπόλοιπες μετρικές, εκτός της *precision*, που *παραμένει ίδια*, βελτιώνοντα σημαντικά.\n","\n","2. Για το **Linear SVM**, όταν εφαρμόζεται *synthetic oversampling*:\n","  * όταν εφαρμόζεται o *Easy Ensemble* μειώνονται *σημαντικά* οι μετρικές *f1* και *recall*, ωστόσο όλες οι υπόλοιπες μετρικές, εκτός της *precision*, που *βελτιώνεται ελάχιστα*, βελτιώνοντα σημαντικά.\n","\n","3. Για το **Naive Bayes**, όταν εφαρμόζεται *synthetic oversampling*::\n","  * όταν εφαρμόζεται o *Easy Ensemble* μειώνονται *σημαντικά* οι μετρικές *f1* και *recall*, ωστόσο όλες οι υπόλοιπες μετρικές, εκτός της *precision*, που *βελτιώνεται ελάχιστα*, βελτιώνοντα σημαντικά.\n","\n","Συνολικά, παρατηρούμε ότι:\n","* στην προκειμένη περίπτωση **με τη χρήση του *Easy Ensemble* επιτυγχάνεται ισορροπία στην επίδοση όλων των μετρικών και στους 3 παραπάνω αλγορίθμους**\n","* θα μπορούσαμε να πούμε ότι **η χρήση του *synthetic oversampling* ΔΕΝ συνίσταται όταν όταν μας ενδιαφέρει να βελτιστοποιήσουμε κάποια από τις μετρικές *recall* ή *f1***.\n"],"metadata":{"id":"gt3wseRfI2BY"}},{"cell_type":"code","source":[],"metadata":{"id":"LC2xmidSJwmn"},"execution_count":null,"outputs":[]}]}