Skip to content

Created using Colaboratory #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aarison wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Created using Colaboratory #1

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 26 additions & 74 deletions labs/xgboostLab.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "8L5KyC6KP_yY",
"colab_type": "text"
"id": "8L5KyC6KP_yY"
},
"source": [
"# XGBoost Lab\n",
Expand Down Expand Up @@ -249,7 +248,6 @@
"cell_type": "code",
"metadata": {
"id": "d3c2iMbCP9aq",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 357
Expand Down Expand Up @@ -372,8 +370,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "b8hWOON-0qEU",
"colab_type": "text"
"id": "b8hWOON-0qEU"
},
"source": [
"And for our first prediction $f_0$ let's predict just the average value:\n"
Expand All @@ -383,7 +380,6 @@
"cell_type": "code",
"metadata": {
"id": "ddG9_pVPTwDW",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 357
Expand Down Expand Up @@ -512,8 +508,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "ijAYdpZd2fzx",
"colab_type": "text"
"id": "ijAYdpZd2fzx"
},
"source": [
"and the differences between our predictions and the actual values"
Expand All @@ -523,7 +518,6 @@
"cell_type": "code",
"metadata": {
"id": "CKG3OvcB1RWv",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 357
Expand Down Expand Up @@ -663,8 +657,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "JYYABnmp2niq",
"colab_type": "text"
"id": "JYYABnmp2niq"
},
"source": [
"That $y-f_0$ is the residual. What is left, or how far the first classifier was off. The residual is what the second classifier is trying to predict.\n",
Expand All @@ -678,7 +671,6 @@
"cell_type": "code",
"metadata": {
"id": "OVqyVkYF2XSO",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 357
Expand Down Expand Up @@ -841,8 +833,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "T72unS4a7HJP",
"colab_type": "text"
"id": "T72unS4a7HJP"
},
"source": [
"And the next classifier will try to predict $y-f_1$ based on x.\n",
Expand All @@ -867,7 +858,6 @@
"cell_type": "code",
"metadata": {
"id": "ma21zshP4umI",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 321
Expand Down Expand Up @@ -910,8 +900,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "U8XREkjWB5fS",
"colab_type": "text"
"id": "U8XREkjWB5fS"
},
"source": [
"This reminds me of a section of the *Hitchhiker's Guide to the Galaxy* by Douglas Adams, where Marvin, the robot, is asked to bring two hitchhikers to the bridge and he says:\n",
Expand All @@ -932,7 +921,6 @@
"cell_type": "code",
"metadata": {
"id": "83Bp2hvE6bcg",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 360
Expand Down Expand Up @@ -975,8 +963,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "XZmJUO_2CGez",
"colab_type": "text"
"id": "XZmJUO_2CGez"
},
"source": [
"It is a Tesla T4, which has 320 tensor cores.\n",
Expand All @@ -993,9 +980,7 @@
{
"cell_type": "code",
"metadata": {
"id": "UwUq3mqwB9EL",
"colab_type": "code",
"colab": {}
"id": "UwUq3mqwB9EL"
},
"source": [
"import pandas as pd\n",
Expand All @@ -1015,8 +1000,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "pmHyfTQeCUQs",
"colab_type": "text"
"id": "pmHyfTQeCUQs"
},
"source": [
"### Create an instance of the XGBoost classifier\n",
Expand All @@ -1027,7 +1011,6 @@
"cell_type": "code",
"metadata": {
"id": "acYGT2PzCRJV",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 154
Expand Down Expand Up @@ -1067,8 +1050,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "TX_yoP5kDKjl",
"colab_type": "text"
"id": "TX_yoP5kDKjl"
},
"source": [
"Let's take a look at those parameters.\n",
Expand All @@ -1085,7 +1067,6 @@
"cell_type": "code",
"metadata": {
"id": "MBxXaHTVCbt0",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 154
Expand Down Expand Up @@ -1121,8 +1102,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "xekQss6bFp65",
"colab_type": "text"
"id": "xekQss6bFp65"
},
"source": [
"### evaluate model\n",
Expand All @@ -1133,7 +1113,6 @@
"cell_type": "code",
"metadata": {
"id": "SO-caKnjFlG-",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
Expand Down Expand Up @@ -1165,8 +1144,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "hoX-aaylF8He",
"colab_type": "text"
"id": "hoX-aaylF8He"
},
"source": [
"We ran a state-of-the-art algorithm on a GPU. Yay us!\n",
Expand Down Expand Up @@ -1205,7 +1183,6 @@
"cell_type": "code",
"metadata": {
"id": "6O1qfbyAFvgW",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 554
Expand Down Expand Up @@ -1490,8 +1467,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "y-Iodm2UOrs7",
"colab_type": "text"
"id": "y-Iodm2UOrs7"
},
"source": [
"## divide features and labels\n",
Expand All @@ -1502,7 +1478,6 @@
"cell_type": "code",
"metadata": {
"id": "8I01XNJzPOI2",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 417
Expand Down Expand Up @@ -1774,8 +1749,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "hRTPkOHzNITR",
"colab_type": "text"
"id": "hRTPkOHzNITR"
},
"source": [
"Now let's one hot encode the features using sklearn's OneHotEncoder.\n"
Expand All @@ -1785,7 +1759,6 @@
"cell_type": "code",
"metadata": {
"id": "JCItEsf8K2TZ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 51
Expand Down Expand Up @@ -1819,8 +1792,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "QZyAn3WLZMqV",
"colab_type": "text"
"id": "QZyAn3WLZMqV"
},
"source": [
"Fantastic! \n",
Expand All @@ -1833,7 +1805,6 @@
"cell_type": "code",
"metadata": {
"id": "pc-AXr9FZGhW",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 51
Expand Down Expand Up @@ -1865,8 +1836,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "fRDTurCqa9nq",
"colab_type": "text"
"id": "fRDTurCqa9nq"
},
"source": [
"You may have noticed that we put a whopping 70% of the data in the test set. We did this because when we are just playing with things to gain an understanding we don't want to wait hours for a result.\n",
Expand All @@ -1880,9 +1850,7 @@
{
"cell_type": "code",
"metadata": {
"id": "1Py0OOb5aBy3",
"colab_type": "code",
"colab": {}
"id": "1Py0OOb5aBy3"
},
"source": [
"## TO DO call it model\n"
Expand All @@ -1893,8 +1861,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "eks7EiOebgkj",
"colab_type": "text"
"id": "eks7EiOebgkj"
},
"source": [
"Now let's say we want to find the best hyperparameter values for \n",
Expand All @@ -1909,9 +1876,7 @@
{
"cell_type": "code",
"metadata": {
"id": "JkqwXndrbfhP",
"colab_type": "code",
"colab": {}
"id": "JkqwXndrbfhP"
},
"source": [
"# TODO\n",
Expand All @@ -1923,8 +1888,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "b4GoFt4hc4zt",
"colab_type": "text"
"id": "b4GoFt4hc4zt"
},
"source": [
"\n",
Expand All @@ -1937,9 +1901,7 @@
{
"cell_type": "code",
"metadata": {
"id": "gZkjYTZHc1iP",
"colab_type": "code",
"colab": {}
"id": "gZkjYTZHc1iP"
},
"source": [
"# TO DO\n",
Expand All @@ -1957,8 +1919,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "3syNvBizdMYj",
"colab_type": "text"
"id": "3syNvBizdMYj"
},
"source": [
"Let's fit the model (this will take awhile)"
Expand All @@ -1968,7 +1929,6 @@
"cell_type": "code",
"metadata": {
"id": "BVxaKf8OdK9n",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 157
Expand Down Expand Up @@ -2011,8 +1971,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "lhJmfE6gM6IJ",
"colab_type": "text"
"id": "lhJmfE6gM6IJ"
},
"source": [
"Now let's see what the best parameters are, make predictions on our test data, and check accuracy..."
Expand All @@ -2022,7 +1981,6 @@
"cell_type": "code",
"metadata": {
"id": "CFPTUpq_dT0s",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
Expand Down Expand Up @@ -2051,9 +2009,7 @@
{
"cell_type": "code",
"metadata": {
"id": "PTboDXG8xsRV",
"colab_type": "code",
"colab": {}
"id": "PTboDXG8xsRV"
},
"source": [
"predictions = random_search.best_estimator_.predict(adult_test_features)"
Expand All @@ -2065,7 +2021,6 @@
"cell_type": "code",
"metadata": {
"id": "dA0tfa2n7t2w",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
Expand Down Expand Up @@ -2094,8 +2049,7 @@
{
"cell_type": "markdown",
"metadata": {
"id": "zmC50ToeLn7S",
"colab_type": "text"
"id": "zmC50ToeLn7S"
},
"source": [
"### This ends our first look at XGBoost\n",
Expand All @@ -2105,9 +2059,7 @@
{
"cell_type": "code",
"metadata": {
"id": "UTPwWCmh7y8k",
"colab_type": "code",
"colab": {}
"id": "UTPwWCmh7y8k"
},
"source": [
""
Expand Down