diff --git a/your-code/Orders.zip b/Orders.zip similarity index 100% rename from your-code/Orders.zip rename to Orders.zip diff --git a/your-code/Pokemon.csv b/Pokemon.csv similarity index 100% rename from your-code/Pokemon.csv rename to Pokemon.csv diff --git a/challenge-1.ipynb b/challenge-1.ipynb new file mode 100644 index 0000000..f2b82c1 --- /dev/null +++ b/challenge-1.ipynb @@ -0,0 +1,915 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Challenge 1\n", + "\n", + "In this challenge you will be working on **Pokemon**. You will answer a series of questions in order to practice dataframe calculation, aggregation, and transformation.\n", + "\n", + "![Pokemon](../images/pokemon.jpg)\n", + "\n", + "Follow the instructions below and enter your code.\n", + "\n", + "#### Import all required libraries." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "import numpy as np" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Import data set.\n", + "\n", + "Read the dataset `pokemon.csv` into a dataframe called `pokemon`.\n", + "\n", + "*Data set attributed to [Alberto Barradas](https://www.kaggle.com/abcsds/pokemon/)*" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "pokemon = pd.read_csv(\"Pokemon.csv\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Print first 10 rows of `pokemon`." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
#NameType 1Type 2TotalHPAttackDefenseSp. AtkSp. DefSpeedGenerationLegendary
01BulbasaurGrassPoison3184549496565451False
12IvysaurGrassPoison4056062638080601False
23VenusaurGrassPoison525808283100100801False
33VenusaurMega VenusaurGrassPoison62580100123122120801False
44CharmanderFireNaN3093952436050651False
55CharmeleonFireNaN4055864588065801False
66CharizardFireFlying534788478109851001False
76CharizardMega Charizard XFireDragon63478130111130851001False
86CharizardMega Charizard YFireFlying63478104781591151001False
97SquirtleWaterNaN3144448655064431False
\n", + "
" + ], + "text/plain": [ + " # Name Type 1 Type 2 Total HP Attack Defense \\\n", + "0 1 Bulbasaur Grass Poison 318 45 49 49 \n", + "1 2 Ivysaur Grass Poison 405 60 62 63 \n", + "2 3 Venusaur Grass Poison 525 80 82 83 \n", + "3 3 VenusaurMega Venusaur Grass Poison 625 80 100 123 \n", + "4 4 Charmander Fire NaN 309 39 52 43 \n", + "5 5 Charmeleon Fire NaN 405 58 64 58 \n", + "6 6 Charizard Fire Flying 534 78 84 78 \n", + "7 6 CharizardMega Charizard X Fire Dragon 634 78 130 111 \n", + "8 6 CharizardMega Charizard Y Fire Flying 634 78 104 78 \n", + "9 7 Squirtle Water NaN 314 44 48 65 \n", + "\n", + " Sp. Atk Sp. Def Speed Generation Legendary \n", + "0 65 65 45 1 False \n", + "1 80 80 60 1 False \n", + "2 100 100 80 1 False \n", + "3 122 120 80 1 False \n", + "4 60 50 65 1 False \n", + "5 80 65 80 1 False \n", + "6 109 85 100 1 False \n", + "7 130 85 100 1 False \n", + "8 159 115 100 1 False \n", + "9 50 64 43 1 False " + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "\n", + "pokemon.head(10)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(800, 13)" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pokemon.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "RangeIndex: 800 entries, 0 to 799\n", + "Data columns (total 13 columns):\n", + " # Column Non-Null Count Dtype \n", + "--- ------ -------------- ----- \n", + " 0 # 800 non-null int64 \n", + " 1 Name 800 non-null object\n", + " 2 Type 1 800 non-null object\n", + " 3 Type 2 414 non-null object\n", + " 4 Total 800 non-null int64 \n", + " 5 HP 800 non-null int64 \n", + " 6 Attack 800 non-null int64 \n", + " 7 Defense 800 non-null int64 \n", + " 8 Sp. Atk 800 non-null int64 \n", + " 9 Sp. Def 800 non-null int64 \n", + " 10 Speed 800 non-null int64 \n", + " 11 Generation 800 non-null int64 \n", + " 12 Legendary 800 non-null bool \n", + "dtypes: bool(1), int64(9), object(3)\n", + "memory usage: 75.9+ KB\n" + ] + } + ], + "source": [ + "\n", + "pokemon.info()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When you look at a data set, you often wonder what each column means. Some open-source data sets provide descriptions of the data set. In many cases, data descriptions are extremely useful for data analysts to perform work efficiently and successfully.\n", + "\n", + "For the `Pokemon.csv` data set, fortunately, the owner provided descriptions which you can see [here](https://www.kaggle.com/abcsds/pokemon/home). For your convenience, we are including the descriptions below. Read the descriptions and understand what each column means. This knowledge is helpful in your work with the data.\n", + "\n", + "| Column | Description |\n", + "| --- | --- |\n", + "| # | ID for each pokemon |\n", + "| Name | Name of each pokemon |\n", + "| Type 1 | Each pokemon has a type, this determines weakness/resistance to attacks |\n", + "| Type 2 | Some pokemon are dual type and have 2 |\n", + "| Total | A general guide to how strong a pokemon is |\n", + "| HP | Hit points, or health, defines how much damage a pokemon can withstand before fainting |\n", + "| Attack | The base modifier for normal attacks (eg. Scratch, Punch) |\n", + "| Defense | The base damage resistance against normal attacks |\n", + "| SP Atk | Special attack, the base modifier for special attacks (e.g. fire blast, bubble beam) |\n", + "| SP Def | The base damage resistance against special attacks |\n", + "| Speed | Determines which pokemon attacks first each round |\n", + "| Generation | Number of generation |\n", + "| Legendary | True if Legendary Pokemon False if not |" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Obtain the distinct values across `Type 1` and `Type 2`.\n", + "\n", + "Exctract all the values in `Type 1` and `Type 2`. Then create an array containing the distinct values across both fields." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Grass', 'Fire', 'Water', 'Bug', 'Normal', 'Poison', 'Electric',\n", + " 'Ground', 'Fairy', 'Fighting', 'Psychic', 'Rock', 'Ghost', 'Ice',\n", + " 'Dragon', 'Dark', 'Steel', 'Flying'], dtype=object)" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "pokemon[\"Type 1\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pokemon[\"Type 2\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "pokemon[\"Type 1\"].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pokemon[\"Type 2\"].value_counts()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Cleanup `Name` that contain \"Mega\".\n", + "\n", + "If you have checked out the pokemon names carefully enough, you should have found there are junk texts in the pokemon names which contain \"Mega\". We want to clean up the pokemon names. For instance, \"VenusaurMega Venusaur\" should be \"Mega Venusaur\", and \"CharizardMega Charizard X\" should be \"Mega Charizard X\"." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
#NameType 1Type 2TotalHPAttackDefenseSp. AtkSp. DefSpeedGenerationLegendary
01BulbasaurGrassPoison3184549496565451False
12IvysaurGrassPoison4056062638080601False
23VenusaurGrassPoison525808283100100801False
33Mega VenusaurGrassPoison62580100123122120801False
44CharmanderFireNaN3093952436050651False
55CharmeleonFireNaN4055864588065801False
66CharizardFireFlying534788478109851001False
76Mega Charizard XFireDragon63478130111130851001False
86Mega Charizard YFireFlying63478104781591151001False
97SquirtleWaterNaN3144448655064431False
\n", + "
" + ], + "text/plain": [ + " # Name Type 1 Type 2 Total HP Attack Defense Sp. Atk \\\n", + "0 1 Bulbasaur Grass Poison 318 45 49 49 65 \n", + "1 2 Ivysaur Grass Poison 405 60 62 63 80 \n", + "2 3 Venusaur Grass Poison 525 80 82 83 100 \n", + "3 3 Mega Venusaur Grass Poison 625 80 100 123 122 \n", + "4 4 Charmander Fire NaN 309 39 52 43 60 \n", + "5 5 Charmeleon Fire NaN 405 58 64 58 80 \n", + "6 6 Charizard Fire Flying 534 78 84 78 109 \n", + "7 6 Mega Charizard X Fire Dragon 634 78 130 111 130 \n", + "8 6 Mega Charizard Y Fire Flying 634 78 104 78 159 \n", + "9 7 Squirtle Water NaN 314 44 48 65 50 \n", + "\n", + " Sp. Def Speed Generation Legendary \n", + "0 65 45 1 False \n", + "1 80 60 1 False \n", + "2 100 80 1 False \n", + "3 120 80 1 False \n", + "4 50 65 1 False \n", + "5 65 80 1 False \n", + "6 85 100 1 False \n", + "7 85 100 1 False \n", + "8 115 100 1 False \n", + "9 64 43 1 False " + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "\n", + "# Creamos una función para dividir el nombre por \"Mega\" pero asegurarnos de que la palabra \"Mega\" sigue apareciendo en el nombre:\n", + "def nombre_Mega(nombre):\n", + " # Partir el nombre por \"Mega\"\n", + " partes = nombre.split(\"Mega\")\n", + " # Verificar si hay más de una parte y mantener \"Mega\" en la descripción\n", + " if len(partes) > 1:\n", + " # Reconstruir el nombre, asegurando que \"Mega\" esté presente\n", + " return \"Mega\" + ' ' + partes[1].strip() # Se agrega \"Mega\" y con el .strip() se limpia el espacio\n", + " return nombre # Retorna el nombre original si \"Mega\" no se encuentra\n", + "\n", + "# Especificamos la columna a la que queremos aplicar esta función\n", + "pokemon[\"Name\"] = pokemon[\"Name\"].apply(nombre_Mega)\n", + "pokemon.head(10)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Create a new column called `A/D Ratio` whose value equals to `Attack` devided by `Defense`.\n", + "\n", + "For instance, if a pokemon has the Attack score 49 and Defense score 49, the corresponding `A/D Ratio` is 49/49=1." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "pokemon[\"A/D Ratio\"]=pokemon[\"Attack\"]/pokemon[\"Defense\"]\n", + "pokemon.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Identify the pokemon with the highest `A/D Ratio`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "pokemon_highest_ratio = pokemon.sort_values(by='A/D Ratio', ascending=False).head(1)\n", + "pokemon_highest_ratio" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Deoxy Attack Forme es el Pokemon con el ratio A/D más alto" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Identify the pokemon with the lowest A/D Ratio." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pokemon_lowest_ratio = pokemon.sort_values(by='A/D Ratio', ascending=True).head(1)\n", + "pokemon_lowest_ratio" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Shuckle es el Pokemon con el ratio A/D más bajo" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Create a new column called `Combo Type` whose value combines `Type 1` and `Type 2`.\n", + "\n", + "Rules:\n", + "\n", + "* If both `Type 1` and `Type 2` have valid values, the `Combo Type` value should contain both values in the form of ` `. For example, if `Type 1` value is `Grass` and `Type 2` value is `Poison`, `Combo Type` will be `Grass-Poison`.\n", + "\n", + "* If `Type 1` has valid value but `Type 2` is not, `Combo Type` will be the same as `Type 1`. For example, if `Type 1` is `Fire` whereas `Type 2` is `NaN`, `Combo Type` will be `Fire`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def combinar_types(row):\n", + " # La función iterará sobre las filas, por lo que se indica que el parámetro es ese.\n", + " # Definimos los dos tipos de filas que recorrerá la función \n", + " type1= row[\"Type 1\"]\n", + " type2= row[\"Type 2\"]\n", + " # Investigamos y encontramos que hay un método que es pd.notna() que directamente encuentra si la fila contiene nulos o no\n", + "\n", + " # Establecemos las condiciones\n", + " # Si ambas filas (rows) son no nulas, se juntarán mediante un guión\n", + " if pd.notna(type1) and pd.notna(type2):\n", + " return f\"{type1}-{type2}\"\n", + " # Si únicamente la fila uno es no nula, se devolverá sólo el contenido de dicha fila\n", + " elif pd.notna(type1):\n", + " return type1\n", + " # Como sabemos que únicamente en el tipo 2 puede haber nulos, la ultima condición posible es que también en el tipo 1 haya nulos\n", + " else:\n", + " return np.nap\n", + "\n", + "# Aplicamos la función para crear la nueva columna:\n", + "pokemon[\"Combo Type\"] = pokemon.apply(combinar_types, axis=1)\n", + "pokemon.head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Identify the pokemon whose `A/D Ratio` are among the top 5." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "pokemon_top_5_ratio = pokemon.sort_values(by='A/D Ratio', ascending=False).head(5)\n", + "pokemon_top_5_ratio" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### For the 5 pokemon printed above, aggregate `Combo Type` and use a list to store the unique values.\n", + "\n", + "Your end product is a list containing the distinct `Combo Type` values of the 5 pokemon with the highest `A/D Ratio`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "pokemon_combo_type=pokemon_top_5_ratio[\"Combo Type\"].unique().tolist()\n", + "pokemon_combo_type" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### For each of the `Combo Type` values obtained from the previous question, calculate the mean scores of all numeric fields across all pokemon.\n", + "\n", + "Your output should look like below:\n", + "\n", + "![Aggregate](../images/aggregated-mean.png)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Seleccionamos solo los Pokémon en la lista pokemon_combo_type y los agrupamos por el tipo de pojemos, así que todos los Pokémon del mismo tipo se juntan.\n", + "# Seleccionamos las columnas sobre las que trabajar (todas aquellas con datos numéricos) y calculamos el promedio de cada columna para cada grupo de pokemos. \n", + "pokemon_combo_type_mean = pokemon[pokemon[\"Combo Type\"].isin(pokemon_combo_type)].groupby(\"Combo Type\")[[\"#\",\"Total\", \"HP\", \"Attack\", \"Defense\", \"Sp. Atk\", \"Sp. Def\", \"Speed\", \"Generation\", \"Legendary\", \"A/D Ratio\"]].mean()\n", + "pokemon_combo_type_mean" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/challenge-2.ipynb b/challenge-2.ipynb new file mode 100644 index 0000000..22f6b76 --- /dev/null +++ b/challenge-2.ipynb @@ -0,0 +1,704 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Challenge 2\n", + "\n", + "In this challenge we will continue working with the `Pokemon` dataset. We will attempt solving a slightly more complex problem in which we will practice the iterative data analysis process you leaned in [this video](https://www.youtube.com/watch?v=xOomNicqbkk).\n", + "\n", + "The problem statement is as follows:\n", + "\n", + "**You are at a Pokemon black market planning to buy a Pokemon for battle. All Pokemon are sold at the same price and you can only afford to buy one. You cannot choose which specific Pokemon to buy. However, you can specify the type of the Pokemon - one type that exists in either `Type 1` or `Type 2`. Which type should you choose in order to maximize your chance of receiving a good Pokemon?**\n", + "\n", + "To remind you about the 3 steps of iterative data analysis, they are:\n", + "\n", + "1. Setting Expectations\n", + "1. Collecting Information\n", + "1. Reacting to Data / Revising Expectations\n", + "\n", + "Following the iterative process, we'll guide you in completing the challenge." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "## Problem Solving Iteration 1\n", + "\n", + "In this iteration we'll analyze the problem and identify the breakthrough. The original question statement is kind of vague because we don't know what a *good pokemon* really means as represented in the data. We'll start by understanding the dataset and see if we can find some insights." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "# Import libraries\n", + "import numpy as np\n", + "import pandas as pd" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
#NameType 1Type 2TotalHPAttackDefenseSp. AtkSp. DefSpeedGenerationLegendary
01BulbasaurGrassPoison3184549496565451False
12IvysaurGrassPoison4056062638080601False
23VenusaurGrassPoison525808283100100801False
33VenusaurMega VenusaurGrassPoison62580100123122120801False
44CharmanderFireNaN3093952436050651False
..........................................
795719DiancieRockFairy60050100150100150506True
796719DiancieMega DiancieRockFairy700501601101601101106True
797720HoopaHoopa ConfinedPsychicGhost6008011060150130706True
798720HoopaHoopa UnboundPsychicDark6808016060170130806True
799721VolcanionFireWater6008011012013090706True
\n", + "

800 rows × 13 columns

\n", + "
" + ], + "text/plain": [ + " # Name Type 1 Type 2 Total HP Attack Defense \\\n", + "0 1 Bulbasaur Grass Poison 318 45 49 49 \n", + "1 2 Ivysaur Grass Poison 405 60 62 63 \n", + "2 3 Venusaur Grass Poison 525 80 82 83 \n", + "3 3 VenusaurMega Venusaur Grass Poison 625 80 100 123 \n", + "4 4 Charmander Fire NaN 309 39 52 43 \n", + ".. ... ... ... ... ... .. ... ... \n", + "795 719 Diancie Rock Fairy 600 50 100 150 \n", + "796 719 DiancieMega Diancie Rock Fairy 700 50 160 110 \n", + "797 720 HoopaHoopa Confined Psychic Ghost 600 80 110 60 \n", + "798 720 HoopaHoopa Unbound Psychic Dark 680 80 160 60 \n", + "799 721 Volcanion Fire Water 600 80 110 120 \n", + "\n", + " Sp. Atk Sp. Def Speed Generation Legendary \n", + "0 65 65 45 1 False \n", + "1 80 80 60 1 False \n", + "2 100 100 80 1 False \n", + "3 122 120 80 1 False \n", + "4 60 50 65 1 False \n", + ".. ... ... ... ... ... \n", + "795 100 150 50 6 True \n", + "796 160 110 110 6 True \n", + "797 150 130 70 6 True \n", + "798 170 130 80 6 True \n", + "799 130 90 70 6 True \n", + "\n", + "[800 rows x 13 columns]" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# Importing the dataset\n", + "pokemon=pd.read_csv(\"Pokemon.csv\")\n", + "pokemon" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "From the data it seems whether a pokemon is good depends on its abilities as represented in the fields of `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, `Speed`, and `Total`. We are not sure about `Generation` and `Legendary` because they are not necessarily the decisive factors of the pokemon abilities.\n", + "\n", + "But `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, `Speed`, and `Total` are a lot of fields! If we look at them all at once it's very complicated. This isn't Mission Impossible but it's ideal that we tackle this kind of problem after we learn Machine Learning (which you will do in Module 3). For now, is there a way to consolidate the fields we need to look into?\n", + "\n", + "Fortunately there seems to be a way. It appears the `Total` field is computed based on the other 6 fields. But we need to prove our theory. If we can approve there is a formula to compute `Total` based on the other 6 abilities, we only need to look into `Total`.\n", + "\n", + "We have the following expectation now:\n", + "\n", + "#### The `Total` field is computed based on `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, and `Speed`.\n", + "\n", + "We need to collect the following information:\n", + "\n", + "* **What is the formula to compute `Total`?**\n", + "* **Does the formula work for all pokemon?**\n", + "\n", + "In the cell below, make a hypothesis on how `Total` is computed and test your hypothesis." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
#NameType 1Type 2TotalHPAttackDefenseSp. AtkSp. DefSpeedGenerationLegendarySum
01BulbasaurGrassPoison3184549496565451False318
12IvysaurGrassPoison4056062638080601False405
23VenusaurGrassPoison525808283100100801False525
33VenusaurMega VenusaurGrassPoison62580100123122120801False625
44CharmanderFireNaN3093952436050651False309
\n", + "
" + ], + "text/plain": [ + " # Name Type 1 Type 2 Total HP Attack Defense \\\n", + "0 1 Bulbasaur Grass Poison 318 45 49 49 \n", + "1 2 Ivysaur Grass Poison 405 60 62 63 \n", + "2 3 Venusaur Grass Poison 525 80 82 83 \n", + "3 3 VenusaurMega Venusaur Grass Poison 625 80 100 123 \n", + "4 4 Charmander Fire NaN 309 39 52 43 \n", + "\n", + " Sp. Atk Sp. Def Speed Generation Legendary Sum \n", + "0 65 65 45 1 False 318 \n", + "1 80 80 60 1 False 405 \n", + "2 100 100 80 1 False 525 \n", + "3 122 120 80 1 False 625 \n", + "4 60 50 65 1 False 309 " + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# your code here\n", + "# your code here\n", + "# Total could be the sum of the other columns (HP, Attack, Defense, Sp. Atk, Sp. Def and Speed\n", + "# We try and do another column (which will be dropped once our hypothesis is met) :\n", + "pokemon[\"Sum\"]=pokemon[[\"HP\", \"Attack\", \"Defense\", \"Sp. Atk\", \"Sp. Def\", \"Speed\"]].sum(axis=1)\n", + "pokemon.head()\n", + "\n", + "# It seems that Column Total stands to the sum of the other columns." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# We make another check:\n", + "test1=pokemon[\"Total\"].sum()\n", + "test2=pokemon[\"Total\"].sum()\n", + "test1, test2\n", + "\n", + "# They are the same, so we can confirm that Total is the sum of the other columns" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pokemon.drop(columns=[\"Sum\"], inplace=True)\n", + "pokemon" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "pokemon.drop(columns=[\"HP\",\"Attack\",\"Defense\",\"Sp. Atk\",\"Sp. Def\",\"Speed\", \"Generation\", \"Legendary\"], inplace=True)\n", + "pokemon" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Problem Solving Iteration 2\n", + "\n", + "Now that we have consolidated the abilities fields, we can update the problem statement. The new problem statement is:\n", + "\n", + "### Which pokemon type is most likely to have the highest `Total` value?\n", + "\n", + "In the updated problem statement, we assume there is a certain relationship between the `Total` and the pokemon type. But we have two *type* fields (`Type 1` and `Type 2`) that have string values. In data analysis, string fields have to be transformed to numerical format in order to be analyzed. \n", + "\n", + "In addition, keep in mind that `Type 1` always has a value but `Type 2` is sometimes empty (having the `NaN` value). Also, the pokemon type we choose may be either in `Type 1` or `Type 2`.\n", + "\n", + "Now our expectation is:\n", + "\n", + "#### `Type 1` and `Type 2` string variables need to be converted to numerical variables in order to identify the relationship between `Total` and the pokemon type.\n", + "\n", + "The information we need to collect is:\n", + "\n", + "#### How to convert two string variables to numerical?\n", + "\n", + "Let's address the first question first. You can use a method called **One Hot Encoding** which is frequently used in machine learning to encode categorical string variables to numerical. The idea is to gather all the possible string values in a categorical field and create a numerical field for each unique string value. Each of those numerical fields uses `1` and `0` to indicate whether the data record has the corresponding categorical value. A detailed explanation of One Hot Encoding can be found in [this article](https://hackernoon.com/what-is-one-hot-encoding-why-and-when-do-you-have-to-use-it-e3c6186d008f). You will formally learn it in Module 3.\n", + "\n", + "For instance, if a pokemon has `Type 1` as `Poison` and `Type 2` as `Fire`, then its `Poison` and `Fire` fields are `1` whereas all other fields are `0`. If a pokemon has `Type 1` as `Water` and `Type 2` as `NaN`, then its `Water` field is `1` whereas all other fields are `0`.\n", + "\n", + "#### In the next cell, use One Hot Encoding to encode `Type 1` and `Type 2`. Use the pokemon type values as the names of the numerical fields you create.\n", + "\n", + "The new numerical variables you create should look like below:\n", + "\n", + "![One Hot Encoding](../images/one-hot-encoding.png)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Combinamos ambas columnas en una nueva llamada 'Combined_Type', manteniendo todos los valores\n", + "pokemon['Combined_Type'] = pokemon[['Type 1', 'Type 2']].fillna('').agg('-'.join, axis=1)\n", + "pokemon" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Aplicamos get_dummies en la nueva columna, separando por '-'\n", + "pokemon_types_dummies = pokemon['Combined_Type'].str.get_dummies(sep='-')\n", + "pokemon_types_dummies" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Finalmente, agregamos estas columnas de dummies al DataFrame original \n", + "pokemon = pd.concat([pokemon, pokemon_types_dummies], axis=1)\n", + "\n", + " # Eliminamos la columna auxiliar\n", + "pokemon = pokemon.drop(columns=['Combined_Type']) \n", + "pokemon" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Problem Solving Iteration 3\n", + "\n", + "Now we have encoded the pokemon types, we will identify the relationship between `Total` and the encoded fields. Our expectation is:\n", + "\n", + "#### There are relationships between `Total` and the encoded pokemon type variables and we need to identify the correlations.\n", + "\n", + "The information we need to collect is:\n", + "\n", + "#### How to identify the relationship between `Total` and the encoded pokemon type fields?\n", + "\n", + "There are multiple ways to answer this question. The easiest way is to use correlation. In the cell below, calculate the correlation of `Total` to each of the encoded fields. Rank the correlations and identify the #1 pokemon type that is most likely to have the highest `Total`." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Asegurar que las columnas que quieras comparar (en este caso, Total y los campos codificados de tipos de Pokémon) sean numéricas.\n", + "# Para ello, eliminamos las columnas no numéricas \n", + "pokemon.drop(columns=[\"Name\",\"Type 1\", \"Type 2\"], inplace=True)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Calculamos la correlación\n", + "correlations = pokemon.corr(method=\"pearson\")\n", + "correlations" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Para ver solo la correlación de Total\n", + "total_correlations = correlations[\"Total\"]\n", + "total_correlations\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Ordenamos las correlaciones\n", + "sorted_correlations = total_correlations.sort_values(ascending=False)\n", + "sorted_correlations" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Bonus Question\n", + "\n", + "Say now you can choose both `Type 1` and `Type 2` of the pokemon. In order to receive the best pokemon, which types will you choose?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# I would choose Dragon as Type 1 and Psychic as Type 2" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/challenge-3.ipynb b/challenge-3.ipynb new file mode 100644 index 0000000..9f3db63 --- /dev/null +++ b/challenge-3.ipynb @@ -0,0 +1,299 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Challenge 3\n", + "\n", + "In this challenge we will work on the `Orders` data set. In your work you will apply the thinking process and workflow we showed you in Challenge 2.\n", + "\n", + "You are serving as a Business Intelligence Analyst at the headquarter of an international fashion goods chain store. Your boss today asked you to do two things for her:\n", + "\n", + "**First, identify two groups of customers from the data set.** The first group is **VIP Customers** whose **aggregated expenses** at your global chain stores are **above the 95th percentile** (aka. 0.95 quantile). The second group is **Preferred Customers** whose **aggregated expenses** are **between the 75th and 95th percentile**.\n", + "\n", + "**Second, identify which country has the most of your VIP customers, and which country has the most of your VIP+Preferred Customers combined.**" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Q1: How to identify VIP & Preferred Customers?\n", + "\n", + "We start by importing all the required libraries:" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "# import required libraries\n", + "import numpy as np\n", + "import pandas as pd" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, extract and import `Orders` dataset into a dataframe variable called `orders`. Print the head of `orders` to overview the data:" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "orders=pd.read_csv(\"Pokemon.csv\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "\n", + "\"Identify VIP and Preferred Customers\" is the non-technical goal of your boss. You need to translate that goal into technical languages that data analysts use:\n", + "\n", + "## How to label customers whose aggregated `amount_spent` is in a given quantile range?\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We break down the main problem into several sub problems:\n", + "\n", + "#### Sub Problem 1: How to aggregate the `amount_spent` for unique customers?\n", + "\n", + "#### Sub Problem 2: How to select customers whose aggregated `amount_spent` is in a given quantile range?\n", + "\n", + "#### Sub Problem 3: How to label selected customers as \"VIP\" or \"Preferred\"?\n", + "\n", + "*Note: If you want to break down the main problem in a different way, please feel free to revise the sub problems above.*\n", + "\n", + "Now in the workspace below, tackle each of the sub problems using the iterative problem solving workflow. Insert cells as necessary to write your codes and explain your steps." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "orders.shape" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Tenemos casi 400.000 pedidos. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Obtenemos un nuevo dataframe agrupando los pedidos por clientes, haciendo un groupby por ID cliente\n", + "orders_by_customers = orders.groupby(\"CustomerID\")[\"amount_spent\"].sum().reset_index()\n", + "orders_by_customers" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Una vez hemos agrupado los pedidos en función del ID del cliente, vemos que tenemos 4400 clientes. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Sacamos un nuevo dataframe unicamente con los clientes cuyo amount_spent esté por encima del quantile 95\n", + "vip_clients= orders_by_customers[orders_by_customers[\"amount_spent\"]>= orders_by_customers[\"amount_spent\"].quantile(0.95)]\n", + "vip_clients" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Podemos decir que este dataframe con 2000 clientes corresponde a los clientes VIP\n", + "# Ya que son todos aquellos cuyos \"amount_spent\" están por encima del quantile 95" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# A continuación, sacamos los clientes con un percentil entre el 75 y el 95\n", + "# Calcular los percentiles\n", + "percentile_75 = orders_by_customers[\"amount_spent\"].quantile(0.75)\n", + "percentile_95 = orders_by_customers[\"amount_spent\"].quantile(0.95)\n", + "\n", + "# Filtrar los clientes que están entre el percentil 75 y el percentil 95\n", + "preferred_clients = orders_by_customers[(orders_by_customers[\"amount_spent\"] >= percentile_75) & \n", + " (orders_by_customers[\"amount_spent\"] < percentile_95)]\n", + "preferred_clients" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we'll leave it to you to solve Q2 & Q3, which you can leverage from your solution for Q1:\n", + "\n", + "## Q2: How to identify which country has the most VIP Customers?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "country_customers = orders.groupby([\"CustomerID\",\"Country\"])[\"amount_spent\"].sum().reset_index()\n", + "country_customers" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Sacamos un nuevo dataframe unicamente con los clientes cuyo amount_spent esté por encima del quantile 95\n", + "vip_clients_country= country_customers[country_customers[\"amount_spent\"]>= country_customers[\"amount_spent\"].quantile(0.95)]\n", + "vip_clients_country" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Contamos el numero de clientes VIP por pais mediante un groupby\n", + "vip_counts = vip_clients_country.groupby(\"Country\")[\"CustomerID\"].nunique().reset_index()\n", + "vip_counts" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Ordenamos los resultados para conocer qué país es el que tiene más clientes VIP\n", + "most_vip_country = vip_counts.sort_values(by=\"CustomerID\", ascending=False).head(1)\n", + "most_vip_country" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# United Kingdom es el país con más clientes VIP" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Q3: How to identify which country has the most VIP+Preferred Customers combined?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Para que sean VIP + Preferred se sacan todos aquellos clientes por encima del percentil 75\n", + "# Sacamos un nuevo dataframe unicamente con los clientes cuyo amount_spent esté por encima del quantile 75\n", + "vip_preferred_clients_country= country_customers[country_customers[\"amount_spent\"]>= country_customers[\"amount_spent\"].quantile(0.75)]\n", + "vip_preferred_clients_country" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Contamos el numero de clientes VIP y Preferred por pais mediante un groupby\n", + "vip_preferred_counts = vip_preferred_clients_country.groupby(\"Country\")[\"CustomerID\"].nunique().reset_index()\n", + "vip_preferred_counts" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# Ordenamos los resultados para conocer qué país es el que tiene más clientes VIP\n", + "most_vip_preferred_country = vip_preferred_counts.sort_values(by=\"CustomerID\", ascending=False).head(1)\n", + "most_vip_preferred_country" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# De nuevo, es United Kingdom el país con más clientes VIP y Preferred" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.4" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/your-code/challenge-1.ipynb b/your-code/challenge-1.ipynb deleted file mode 100644 index cd674cb..0000000 --- a/your-code/challenge-1.ipynb +++ /dev/null @@ -1,276 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Challenge 1\n", - "\n", - "In this challenge you will be working on **Pokemon**. You will answer a series of questions in order to practice dataframe calculation, aggregation, and transformation.\n", - "\n", - "![Pokemon](../images/pokemon.jpg)\n", - "\n", - "Follow the instructions below and enter your code.\n", - "\n", - "#### Import all required libraries." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# import libraries" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Import data set.\n", - "\n", - "Read the dataset `pokemon.csv` into a dataframe called `pokemon`.\n", - "\n", - "*Data set attributed to [Alberto Barradas](https://www.kaggle.com/abcsds/pokemon/)*" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# import dataset" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Print first 10 rows of `pokemon`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When you look at a data set, you often wonder what each column means. Some open-source data sets provide descriptions of the data set. In many cases, data descriptions are extremely useful for data analysts to perform work efficiently and successfully.\n", - "\n", - "For the `Pokemon.csv` data set, fortunately, the owner provided descriptions which you can see [here](https://www.kaggle.com/abcsds/pokemon/home). For your convenience, we are including the descriptions below. Read the descriptions and understand what each column means. This knowledge is helpful in your work with the data.\n", - "\n", - "| Column | Description |\n", - "| --- | --- |\n", - "| # | ID for each pokemon |\n", - "| Name | Name of each pokemon |\n", - "| Type 1 | Each pokemon has a type, this determines weakness/resistance to attacks |\n", - "| Type 2 | Some pokemon are dual type and have 2 |\n", - "| Total | A general guide to how strong a pokemon is |\n", - "| HP | Hit points, or health, defines how much damage a pokemon can withstand before fainting |\n", - "| Attack | The base modifier for normal attacks (eg. Scratch, Punch) |\n", - "| Defense | The base damage resistance against normal attacks |\n", - "| SP Atk | Special attack, the base modifier for special attacks (e.g. fire blast, bubble beam) |\n", - "| SP Def | The base damage resistance against special attacks |\n", - "| Speed | Determines which pokemon attacks first each round |\n", - "| Generation | Number of generation |\n", - "| Legendary | True if Legendary Pokemon False if not |" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Obtain the distinct values across `Type 1` and `Type 2`.\n", - "\n", - "Exctract all the values in `Type 1` and `Type 2`. Then create an array containing the distinct values across both fields." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Cleanup `Name` that contain \"Mega\".\n", - "\n", - "If you have checked out the pokemon names carefully enough, you should have found there are junk texts in the pokemon names which contain \"Mega\". We want to clean up the pokemon names. For instance, \"VenusaurMega Venusaur\" should be \"Mega Venusaur\", and \"CharizardMega Charizard X\" should be \"Mega Charizard X\"." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here\n", - "\n", - "\n", - "# test transformed data\n", - "pokemon.head(10)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Create a new column called `A/D Ratio` whose value equals to `Attack` devided by `Defense`.\n", - "\n", - "For instance, if a pokemon has the Attack score 49 and Defense score 49, the corresponding `A/D Ratio` is 49/49=1." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Identify the pokemon with the highest `A/D Ratio`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Identify the pokemon with the lowest A/D Ratio." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Create a new column called `Combo Type` whose value combines `Type 1` and `Type 2`.\n", - "\n", - "Rules:\n", - "\n", - "* If both `Type 1` and `Type 2` have valid values, the `Combo Type` value should contain both values in the form of ` `. For example, if `Type 1` value is `Grass` and `Type 2` value is `Poison`, `Combo Type` will be `Grass-Poison`.\n", - "\n", - "* If `Type 1` has valid value but `Type 2` is not, `Combo Type` will be the same as `Type 1`. For example, if `Type 1` is `Fire` whereas `Type 2` is `NaN`, `Combo Type` will be `Fire`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### Identify the pokemon whose `A/D Ratio` are among the top 5." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### For the 5 pokemon printed above, aggregate `Combo Type` and use a list to store the unique values.\n", - "\n", - "Your end product is a list containing the distinct `Combo Type` values of the 5 pokemon with the highest `A/D Ratio`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "#### For each of the `Combo Type` values obtained from the previous question, calculate the mean scores of all numeric fields across all pokemon.\n", - "\n", - "Your output should look like below:\n", - "\n", - "![Aggregate](../images/aggregated-mean.png)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/your-code/challenge-2.ipynb b/your-code/challenge-2.ipynb deleted file mode 100644 index d347731..0000000 --- a/your-code/challenge-2.ipynb +++ /dev/null @@ -1,195 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Challenge 2\n", - "\n", - "In this challenge we will continue working with the `Pokemon` dataset. We will attempt solving a slightly more complex problem in which we will practice the iterative data analysis process you leaned in [this video](https://www.youtube.com/watch?v=xOomNicqbkk).\n", - "\n", - "The problem statement is as follows:\n", - "\n", - "**You are at a Pokemon black market planning to buy a Pokemon for battle. All Pokemon are sold at the same price and you can only afford to buy one. You cannot choose which specific Pokemon to buy. However, you can specify the type of the Pokemon - one type that exists in either `Type 1` or `Type 2`. Which type should you choose in order to maximize your chance of receiving a good Pokemon?**\n", - "\n", - "To remind you about the 3 steps of iterative data analysis, they are:\n", - "\n", - "1. Setting Expectations\n", - "1. Collecting Information\n", - "1. Reacting to Data / Revising Expectations\n", - "\n", - "Following the iterative process, we'll guide you in completing the challenge." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "## Problem Solving Iteration 1\n", - "\n", - "In this iteration we'll analyze the problem and identify the breakthrough. The original question statement is kind of vague because we don't know what a *good pokemon* really means as represented in the data. We'll start by understanding the dataset and see if we can find some insights." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Import libraries\n", - "import numpy as np\n", - "import pandas as pd" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "scrolled": true - }, - "outputs": [], - "source": [ - "# Importing the dataset" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "From the data it seems whether a pokemon is good depends on its abilities as represented in the fields of `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, `Speed`, and `Total`. We are not sure about `Generation` and `Legendary` because they are not necessarily the decisive factors of the pokemon abilities.\n", - "\n", - "But `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, `Speed`, and `Total` are a lot of fields! If we look at them all at once it's very complicated. This isn't Mission Impossible but it's ideal that we tackle this kind of problem after we learn Machine Learning (which you will do in Module 3). For now, is there a way to consolidate the fields we need to look into?\n", - "\n", - "Fortunately there seems to be a way. It appears the `Total` field is computed based on the other 6 fields. But we need to prove our theory. If we can approve there is a formula to compute `Total` based on the other 6 abilities, we only need to look into `Total`.\n", - "\n", - "We have the following expectation now:\n", - "\n", - "#### The `Total` field is computed based on `HP`, `Attack`, `Defense`, `Sp. Atk`, `Sp. Def`, and `Speed`.\n", - "\n", - "We need to collect the following information:\n", - "\n", - "* **What is the formula to compute `Total`?**\n", - "* **Does the formula work for all pokemon?**\n", - "\n", - "In the cell below, make a hypothesis on how `Total` is computed and test your hypothesis." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Problem Solving Iteration 2\n", - "\n", - "Now that we have consolidated the abilities fields, we can update the problem statement. The new problem statement is:\n", - "\n", - "### Which pokemon type is most likely to have the highest `Total` value?\n", - "\n", - "In the updated problem statement, we assume there is a certain relationship between the `Total` and the pokemon type. But we have two *type* fields (`Type 1` and `Type 2`) that have string values. In data analysis, string fields have to be transformed to numerical format in order to be analyzed. \n", - "\n", - "In addition, keep in mind that `Type 1` always has a value but `Type 2` is sometimes empty (having the `NaN` value). Also, the pokemon type we choose may be either in `Type 1` or `Type 2`.\n", - "\n", - "Now our expectation is:\n", - "\n", - "#### `Type 1` and `Type 2` string variables need to be converted to numerical variables in order to identify the relationship between `Total` and the pokemon type.\n", - "\n", - "The information we need to collect is:\n", - "\n", - "#### How to convert two string variables to numerical?\n", - "\n", - "Let's address the first question first. You can use a method called **One Hot Encoding** which is frequently used in machine learning to encode categorical string variables to numerical. The idea is to gather all the possible string values in a categorical field and create a numerical field for each unique string value. Each of those numerical fields uses `1` and `0` to indicate whether the data record has the corresponding categorical value. A detailed explanation of One Hot Encoding can be found in [this article](https://hackernoon.com/what-is-one-hot-encoding-why-and-when-do-you-have-to-use-it-e3c6186d008f). You will formally learn it in Module 3.\n", - "\n", - "For instance, if a pokemon has `Type 1` as `Poison` and `Type 2` as `Fire`, then its `Poison` and `Fire` fields are `1` whereas all other fields are `0`. If a pokemon has `Type 1` as `Water` and `Type 2` as `NaN`, then its `Water` field is `1` whereas all other fields are `0`.\n", - "\n", - "#### In the next cell, use One Hot Encoding to encode `Type 1` and `Type 2`. Use the pokemon type values as the names of the numerical fields you create.\n", - "\n", - "The new numerical variables you create should look like below:\n", - "\n", - "![One Hot Encoding](../images/one-hot-encoding.png)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Problem Solving Iteration 3\n", - "\n", - "Now we have encoded the pokemon types, we will identify the relationship between `Total` and the encoded fields. Our expectation is:\n", - "\n", - "#### There are relationships between `Total` and the encoded pokemon type variables and we need to identify the correlations.\n", - "\n", - "The information we need to collect is:\n", - "\n", - "#### How to identify the relationship between `Total` and the encoded pokemon type fields?\n", - "\n", - "There are multiple ways to answer this question. The easiest way is to use correlation. In the cell below, calculate the correlation of `Total` to each of the encoded fields. Rank the correlations and identify the #1 pokemon type that is most likely to have the highest `Total`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Bonus Question\n", - "\n", - "Say now you can choose both `Type 1` and `Type 2` of the pokemon. In order to receive the best pokemon, which types will you choose?" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/your-code/challenge-3.ipynb b/your-code/challenge-3.ipynb deleted file mode 100644 index a42a586..0000000 --- a/your-code/challenge-3.ipynb +++ /dev/null @@ -1,147 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Challenge 3\n", - "\n", - "In this challenge we will work on the `Orders` data set. In your work you will apply the thinking process and workflow we showed you in Challenge 2.\n", - "\n", - "You are serving as a Business Intelligence Analyst at the headquarter of an international fashion goods chain store. Your boss today asked you to do two things for her:\n", - "\n", - "**First, identify two groups of customers from the data set.** The first group is **VIP Customers** whose **aggregated expenses** at your global chain stores are **above the 95th percentile** (aka. 0.95 quantile). The second group is **Preferred Customers** whose **aggregated expenses** are **between the 75th and 95th percentile**.\n", - "\n", - "**Second, identify which country has the most of your VIP customers, and which country has the most of your VIP+Preferred Customers combined.**" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Q1: How to identify VIP & Preferred Customers?\n", - "\n", - "We start by importing all the required libraries:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# import required libraries\n", - "import numpy as np\n", - "import pandas as pd" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Next, extract and import `Orders` dataset into a dataframe variable called `orders`. Print the head of `orders` to overview the data:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "---\n", - "\n", - "\"Identify VIP and Preferred Customers\" is the non-technical goal of your boss. You need to translate that goal into technical languages that data analysts use:\n", - "\n", - "## How to label customers whose aggregated `amount_spent` is in a given quantile range?\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We break down the main problem into several sub problems:\n", - "\n", - "#### Sub Problem 1: How to aggregate the `amount_spent` for unique customers?\n", - "\n", - "#### Sub Problem 2: How to select customers whose aggregated `amount_spent` is in a given quantile range?\n", - "\n", - "#### Sub Problem 3: How to label selected customers as \"VIP\" or \"Preferred\"?\n", - "\n", - "*Note: If you want to break down the main problem in a different way, please feel free to revise the sub problems above.*\n", - "\n", - "Now in the workspace below, tackle each of the sub problems using the iterative problem solving workflow. Insert cells as necessary to write your codes and explain your steps." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we'll leave it to you to solve Q2 & Q3, which you can leverage from your solution for Q1:\n", - "\n", - "## Q2: How to identify which country has the most VIP Customers?" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Q3: How to identify which country has the most VIP+Preferred Customers combined?" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# your code here" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.6.9" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -}