diff --git a/.ipynb_checkpoints/README-checkpoint.md b/.ipynb_checkpoints/README-checkpoint.md new file mode 100644 index 00000000..7bc0266a --- /dev/null +++ b/.ipynb_checkpoints/README-checkpoint.md @@ -0,0 +1,153 @@ +![logo_ironhack_blue 7](https://user-images.githubusercontent.com/23629340/40541063-a07a0a8a-601a-11e8-91b5-2f13e4e6b441.png) + +# LAB | Intro to Pandas +
+ +

Learning Goals

+ + + This lab allows you to practice and apply the concepts and techniques taught in class. + + Upon completion of this lab, you will be able to: + +- Create Pandas DataFrames from CSV files. +- Explore Pandas DataFrames to get the number of rows, columns, summary statistics and data types. +- Manipulate Series objects using various methods, including sort_values or value_counts. +- Access data in DataFrames through columns (as a dictionary or attribute) and by using loc or iloc to access rows. +- Filter and select data in DataFrames using simple comparison and logical operators + +
+
+ +
+ +
+ +

Prerequisites

+ + +Before this starting this lab, you should have learnt about: + +- Data types, operators and structures +- Flow control (if-else statements and loops) +- Functions + +
+
+ +
+ + +## Introduction + +Welcome to the Pandas Lab! In this lab, you will learn how to work with and create pandas series and dataframes. These are powerful tools for data manipulation and analysis in Python and allow you to create, manipulate, and analyze large, complex datasets with ease, making it a valuable library for any data scientist or analyst. + +You will also learn how to access their attributes and values, and manipulate them using a variety of built-in methods as well as learn how to filter and sort data and summarize and describe data. By the end of this lab, you will have a strong foundation in working with pandas series and dataframes, which you can apply to real-world data analysis problems. + +
+ +**Happy coding!** :heart: + +## About the dataset + +### Context +The dataset comprises customer data related to their vehicle insurance policies. It provides details about both the customers and the specific insurance coverage they've opted for. This data an be explored to segment similar kinds of customers. + +### Content +This dataset provides socio-economic information of customers alongside details of their insured vehicles. The data contains both categorical and numerical variables. The "Customer Lifetime Value" (CLV) has also been provided, which is derived from historical data. This is essential in understanding the customer purchase behavior. + +### Acknowledgements +UCI ML repository + +### Inspiration +The data is useful for clustering customers according to their behavior, thereby enabling targeted marketing of a new insurance policy. This helps pre-estimating a budget for marketing expenses. + +### Data Description + +- Customer: Customer ID. + +- ST: State where customers live. + +- Gender: Gender of the customer. + +- Education: Background education of customers. + +- Customer Lifetime Value: Customer Lifetime Value (CLV) is the total revenue the client will derive from their entire relationship with a customer. In other words, it is the predicted or calculated value of a customer over their entire duration as a policyholder with the insurance company. It is an estimation of the net profit that the insurance company expects to generate from a customer throughout their relationship with the company. CLV takes into account factors such as the duration of the customer's policy, premium payments, claim history, renewal likelihood, and potential additional services or products the customer may purchase. It helps insurers assess the long-term profitability and value associated with retaining a particular customer. + +- Income: Customer's earnings. + +- Monthly Premium Auto: The monthly premium amount a customer pays for their auto insurance. It represents the recurring cost that the insured person must pay to maintain their insurance policy and receive coverage for potential damages, accidents, or other covered events related to their vehicle. + +- Number of Open Complaints: Number of complaints the customer has opened. + +- Policy Type: Insurance policy categories include Corporate Auto, Personal Auto, and Special Auto. + +- Vehicle Class: Specifies the class of insured vehicles, such as Two-Door Car, Four-Door Car SUV, Luxury SUV, Sports Car, and Luxury Car. + +- Total Claim Amount: The sum of all claims made by the customer. It represents the total monetary value of all approved claims for incidents such as accidents, theft, vandalism, or other covered events. + +## Requirements + +- Fork this repo +- Clone it to your machine + +## Getting Started + +Complete the challenges in the `Jupyter Notebook` file. Follow the instructions and add your code and explanations as necessary. + +## Submission + +- Upon completion, run the following commands: + +```bash +git add . +git commit -m "Solved lab" +git push origin master +``` + +- Paste the link of your lab in Student Portal. + + +## FAQs +
+ I am stuck in the exercise and don't know how to solve the problem or where to start. +
+ + If you are stuck in your code and don't know how to solve the problem or where to start, you should take a step back and try to form a clear question about the specific issue you are facing. This will help you narrow down the problem and come up with potential solutions. + + + For example, is it a concept that you don't understand, or are you receiving an error message that you don't know how to fix? It is usually helpful to try to state the problem as clearly as possible, including any error messages you are receiving. This can help you communicate the issue to others and potentially get help from classmates or online resources. + + + Once you have a clear understanding of the problem, you will be able to start working toward the solution. + + [Back to top](#faqs) + +
+ + +
+ I am unable to push changes to the repository. What should I do? +
+ +There are a couple of possible reasons why you may be unable to *push* changes to a Git repository: + +1. **You have not committed your changes:** Before you can push your changes to the repository, you need to commit them using the `git commit` command. Make sure you have committed your changes and try pushing again. To do this, run the following terminal commands from the project folder: + ```bash + git add . + git commit -m "Your commit message" + git push + ``` +2. **You do not have permission to push to the repository:** If you have cloned the repository directly from the main Ironhack repository without making a *Fork* first, you do not have write access to the repository. +To check which remote repository you have cloned, run the following terminal command from the project folder: + ```bash + git remote -v + ``` +If the link shown is the same as the main Ironhack repository, you will need to fork the repository to your GitHub account first and then clone your fork to your local machine to be able to push the changes. + +**Note**: You should make a copy of your local code to avoid losing it in the process. + + [Back to top](#faqs) + +
+ diff --git a/.ipynb_checkpoints/lab-dw-pandas-checkpoint.ipynb b/.ipynb_checkpoints/lab-dw-pandas-checkpoint.ipynb new file mode 100644 index 00000000..fbd46831 --- /dev/null +++ b/.ipynb_checkpoints/lab-dw-pandas-checkpoint.ipynb @@ -0,0 +1,259 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "25d7736c-ba17-4aff-b6bb-66eba20fbf4e", + "metadata": {}, + "source": [ + "# Lab | Pandas" + ] + }, + { + "cell_type": "markdown", + "id": "d1973e9e-8be6-4039-b70e-d73ee0d94c99", + "metadata": {}, + "source": [ + "In this lab, we will be working with the customer data from an insurance company, which can be found in the CSV file located at the following link: https://raw.githubusercontent.com/data-bootcamp-v4/data/main/file1.csv\n", + "\n", + "The data includes information such as customer ID, state, gender, education, income, and other variables that can be used to perform various analyses.\n", + "\n", + "Throughout the lab, we will be using the pandas library in Python to manipulate and analyze the data. Pandas is a powerful library that provides various data manipulation and analysis tools, including the ability to load and manipulate data from a variety of sources, including CSV files." + ] + }, + { + "cell_type": "markdown", + "id": "8045146f-f4f7-44d9-8cd9-130d6400c73a", + "metadata": {}, + "source": [ + "### Data Description\n", + "\n", + "- Customer - Customer ID\n", + "\n", + "- ST - State where customers live\n", + "\n", + "- Gender - Gender of the customer\n", + "\n", + "- Education - Background education of customers \n", + "\n", + "- Customer Lifetime Value - Customer lifetime value(CLV) is the total revenue the client will derive from their entire relationship with a customer. In other words, is the predicted or calculated value of a customer over their entire duration as a policyholder with the insurance company. It is an estimation of the net profit that the insurance company expects to generate from a customer throughout their relationship with the company. Customer Lifetime Value takes into account factors such as the duration of the customer's policy, premium payments, claim history, renewal likelihood, and potential additional services or products the customer may purchase. It helps insurers assess the long-term profitability and value associated with retaining a particular customer.\n", + "\n", + "- Income - Customers income\n", + "\n", + "- Monthly Premium Auto - Amount of money the customer pays on a monthly basis as a premium for their auto insurance coverage. It represents the recurring cost that the insured person must pay to maintain their insurance policy and receive coverage for potential damages, accidents, or other covered events related to their vehicle.\n", + "\n", + "- Number of Open Complaints - Number of complaints the customer opened\n", + "\n", + "- Policy Type - There are three type of policies in car insurance (Corporate Auto, Personal Auto, and Special Auto)\n", + "\n", + "- Vehicle Class - Type of vehicle classes that customers have Two-Door Car, Four-Door Car SUV, Luxury SUV, Sports Car, and Luxury Car\n", + "\n", + "- Total Claim Amount - the sum of all claims made by the customer. It represents the total monetary value of all approved claims for incidents such as accidents, theft, vandalism, or other covered events.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3a72419b-20fc-4905-817a-8c83abc59de6", + "metadata": {}, + "source": [ + "External Resources: https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9" + ] + }, + { + "cell_type": "markdown", + "id": "8f8ece17-e919-4e23-96c0-c7c59778436a", + "metadata": {}, + "source": [ + "## Challenge 1: Understanding the data\n", + "\n", + "In this challenge, you will use pandas to explore a given dataset. Your task is to gain a deep understanding of the data by analyzing its characteristics, dimensions, and statistical properties." + ] + }, + { + "cell_type": "markdown", + "id": "91437bd5-59a6-49c0-8150-ef0e6e6eb253", + "metadata": {}, + "source": [ + "- Identify the dimensions of the dataset by determining the number of rows and columns it contains.\n", + "- Determine the data types of each column and evaluate whether they are appropriate for the nature of the variable. You should also provide suggestions for fixing any incorrect data types.\n", + "- Identify the number of unique values for each column and determine which columns appear to be categorical. You should also describe the unique values of each categorical column and the range of values for numerical columns, and give your insights.\n", + "- Compute summary statistics such as mean, median, mode, standard deviation, and quartiles to understand the central tendency and distribution of the data for numerical columns. You should also provide your conclusions based on these summary statistics.\n", + "- Compute summary statistics for categorical columns and providing your conclusions based on these statistics." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dd4e8cd8-a6f6-486c-a5c4-1745b0c035f4", + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here" + ] + }, + { + "cell_type": "markdown", + "id": "4a703890-63db-4944-b7ab-95a4f8185120", + "metadata": {}, + "source": [ + "## Challenge 2: analyzing the data" + ] + }, + { + "cell_type": "markdown", + "id": "0776a403-c56a-452f-ac33-5fd4fdb06fc7", + "metadata": {}, + "source": [ + "### Exercise 1" + ] + }, + { + "cell_type": "markdown", + "id": "eedbc484-da4d-4f9c-9343-e1d44311a87e", + "metadata": {}, + "source": [ + "The marketing team wants to know the top 5 less common customer locations. Create a pandas Series object that contains the customer locations and their frequencies, and then retrieve the top 5 less common locations in ascending order." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2dca5073-4520-4f42-9390-4b92733284ed", + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here" + ] + }, + { + "cell_type": "markdown", + "id": "0ce80f43-4afa-43c7-a78a-c917444da4e0", + "metadata": {}, + "source": [ + "### Exercise 2\n", + "\n", + "The sales team wants to know the total number of policies sold for each type of policy. Create a pandas Series object that contains the policy types and their total number of policies sold, and then retrieve the policy type with the highest number of policies sold." + ] + }, + { + "cell_type": "markdown", + "id": "a9f13997-1555-4f98-aca6-970fda1d2c3f", + "metadata": {}, + "source": [ + "*Hint:*\n", + "- *Using value_counts() method simplifies this analysis.*\n", + "- *Futhermore, there is a method that returns the index of the maximum value in a column or row.*\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bcfad6c1-9af2-4b0b-9aa9-0dc5c17473c0", + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here" + ] + }, + { + "cell_type": "markdown", + "id": "0b863fd3-bf91-4d5d-86eb-be29ed9f5b70", + "metadata": {}, + "source": [ + "### Exercise 3\n", + "\n", + "The sales team wants to know if customers with Personal Auto have a lower income than those with Corporate Auto. How does the average income compare between the two policy types?" + ] + }, + { + "cell_type": "markdown", + "id": "b1386d75-2810-4aa1-93e0-9485aa12d552", + "metadata": {}, + "source": [ + "- Use *loc* to create two dataframes: one containing only Personal Auto policies and one containing only Corporate Auto policies.\n", + "- Calculate the average income for each policy.\n", + "- Print the results." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0c0563cf-6f8b-463d-a321-651a972f82e5", + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here" + ] + }, + { + "cell_type": "markdown", + "id": "80b16c27-f4a5-4727-a229-1f88671cf4e2", + "metadata": {}, + "source": [ + "### Bonus: Exercise 4\n" + ] + }, + { + "cell_type": "markdown", + "id": "ac584986-299b-475f-ac2e-928c16c3f512", + "metadata": {}, + "source": [ + "Your goal is to identify customers with a high policy claim amount.\n", + "\n", + "Instructions:\n", + "\n", + "- Review again the statistics for total claim amount to gain an understanding of the data.\n", + "- To identify potential areas for improving customer retention and profitability, we want to focus on customers with a high policy claim amount. Consider customers with a high policy claim amount to be those in the top 25% of the total claim amount. Create a pandas DataFrame object that contains information about customers with a policy claim amount greater than the 75th percentile.\n", + "- Use DataFrame methods to calculate summary statistics about the high policy claim amount data. " + ] + }, + { + "cell_type": "markdown", + "id": "4e3af5f1-6023-4b05-9c01-d05392daa650", + "metadata": {}, + "source": [ + "*Note: When analyzing data, we often want to focus on certain groups of values to gain insights. Percentiles are a useful tool to help us define these groups. A percentile is a measure that tells us what percentage of values in a dataset are below a certain value. For example, the 75th percentile represents the value below which 75% of the data falls. Similarly, the 25th percentile represents the value below which 25% of the data falls. When we talk about the top 25%, we are referring to the values that fall above the 75th percentile, which represent the top quarter of the data. On the other hand, when we talk about the bottom 25%, we are referring to the values that fall below the 25th percentile, which represent the bottom quarter of the data. By focusing on these groups, we can identify patterns and trends that may be useful for making decisions and taking action.*\n", + "\n", + "*Hint: look for a method that gives you the percentile or quantile 0.75 and 0.25 for a Pandas Series.*" + ] + }, + { + "cell_type": "markdown", + "id": "2d234634-50bd-41e0-88f7-d5ba684455d1", + "metadata": {}, + "source": [ + "*Hint 2: check `Boolean selection according to the values of a single column` in https://towardsdatascience.com/filtering-data-frames-in-pandas-b570b1f834b9*" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b731bca6-a760-4860-a27b-a33efa712ce0", + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/lab-dw-pandas.ipynb b/lab-dw-pandas.ipynb index fbd46831..d34447fc 100644 --- a/lab-dw-pandas.ipynb +++ b/lab-dw-pandas.ipynb @@ -8,6 +8,14 @@ "# Lab | Pandas" ] }, + { + "cell_type": "code", + "execution_count": null, + "id": "ae9f1267-4bd0-4c61-8067-dd7af142a312", + "metadata": {}, + "outputs": [], + "source": [] + }, { "cell_type": "markdown", "id": "d1973e9e-8be6-4039-b70e-d73ee0d94c99", @@ -82,12 +90,495 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "id": "dd4e8cd8-a6f6-486c-a5c4-1745b0c035f4", "metadata": {}, "outputs": [], "source": [ - "# Your code here" + "import pandas as pd\n", + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "5c19d900-764d-4540-83c0-c197491a24d5", + "metadata": {}, + "outputs": [], + "source": [ + "df = pd.read_csv(\"https://raw.githubusercontent.com/data-bootcamp-v4/data/main/file1.csv\")" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "adb0be52-0b9e-48b2-96e7-9ae719f31f02", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(4008, 11)" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.shape\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "000de4ca-7a65-4517-8678-92274a7a321b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Customer object\n", + "ST object\n", + "GENDER object\n", + "Education object\n", + "Customer Lifetime Value object\n", + "Income float64\n", + "Monthly Premium Auto float64\n", + "Number of Open Complaints object\n", + "Policy Type object\n", + "Vehicle Class object\n", + "Total Claim Amount float64\n", + "dtype: object" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.dtypes" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "c61f804c-189d-4cf1-b1dc-824831aedf9b", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Customer 1071\n", + "ST 8\n", + "GENDER 5\n", + "Education 6\n", + "Customer Lifetime Value 1027\n", + "Income 774\n", + "Monthly Premium Auto 132\n", + "Number of Open Complaints 6\n", + "Policy Type 3\n", + "Vehicle Class 6\n", + "Total Claim Amount 761\n", + "dtype: int64" + ] + }, + "execution_count": 17, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.nunique()" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "98c7e22f-9349-425a-9538-1e5038c522e3", + "metadata": {}, + "outputs": [], + "source": [ + "## Categorical columns = ST / GENDER / Education / Number of Open Complaints / Policty Type / Vehicle class" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "b722ad71-3bbd-4ccc-af4c-0d0d2a522c85", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Washington', 'Arizona', 'Nevada', 'California', 'Oregon', 'Cali',\n", + " 'AZ', 'WA', nan], dtype=object)" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df[\"ST\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "96668f32-c07f-4051-8e6a-3d8cefb03862", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([nan, 'F', 'M', 'Femal', 'Male', 'female'], dtype=object)" + ] + }, + "execution_count": 29, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df[\"GENDER\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "id": "052f9dff-2002-4510-9ede-79179ed0adf9", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Master', 'Bachelor', 'High School or Below', 'College',\n", + " 'Bachelors', 'Doctor', nan], dtype=object)" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df[\"Education\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "id": "5e1db50c-6274-4a88-947c-ce9f078c38fd", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['1/0/00', '1/2/00', '1/1/00', '1/3/00', '1/5/00', '1/4/00', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 33, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df[\"Number of Open Complaints\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "id": "672e7c49-273e-435a-a3d0-5aeabd8dbfb5", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Personal Auto', 'Corporate Auto', 'Special Auto', nan],\n", + " dtype=object)" + ] + }, + "execution_count": 37, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df[\"Policy Type\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "6ee2b86d-054e-446c-9839-19a2a0540c07", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array(['Four-Door Car', 'Two-Door Car', 'SUV', 'Luxury SUV', 'Sports Car',\n", + " 'Luxury Car', nan], dtype=object)" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df[\"Vehicle Class\"].unique()" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "aab098f1-7bda-4964-856a-5da3cf347e52", + "metadata": {}, + "outputs": [], + "source": [ + "## Categorical columns = GENDER / Education / Number of Open Complaints / Policty Type / Vehicle class" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "id": "03f3f376-041b-40d5-9eed-2e9270e885bf", + "metadata": { + "jp-MarkdownHeadingCollapsed": true + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
IncomeMonthly Premium AutoTotal Claim Amount
count1071.0000001071.0000001071.000000
mean39295.701214193.234360404.986909
std30469.4270601601.190369293.027260
min0.00000061.0000000.382107
25%14072.00000068.000000202.157702
50%36234.00000083.000000354.729129
75%64631.000000109.500000532.800000
max99960.00000035354.0000002893.239678
\n", + "
" + ], + "text/plain": [ + " Income Monthly Premium Auto Total Claim Amount\n", + "count 1071.000000 1071.000000 1071.000000\n", + "mean 39295.701214 193.234360 404.986909\n", + "std 30469.427060 1601.190369 293.027260\n", + "min 0.000000 61.000000 0.382107\n", + "25% 14072.000000 68.000000 202.157702\n", + "50% 36234.000000 83.000000 354.729129\n", + "75% 64631.000000 109.500000 532.800000\n", + "max 99960.000000 35354.000000 2893.239678" + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "df.describe()" + ] + }, + { + "cell_type": "code", + "execution_count": 64, + "id": "a0c07d07-9abe-43dd-b1ff-e1c98fc02350", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
unique valuemost frequent valuefrequence of this valuemissing valuevalue count
GENDER5F4573054{'F': 457, 'M': 413, 'Male': 39, 'female': 28,...
Education6Bachelor3242937{'Bachelor': 324, 'College': 313, 'High School...
Policy Type3Personal Auto7802937{'Personal Auto': 780, 'Corporate Auto': 234, ...
Vehicle Class6Four-Door Car5762937{'Four-Door Car': 576, 'Two-Door Car': 205, 'S...
Number of Open Complaints61/0/008302937{'1/0/00': 830, '1/1/00': 138, '1/2/00': 50, '...
\n", + "
" + ], + "text/plain": [ + " unique value most frequent value \\\n", + "GENDER 5 F \n", + "Education 6 Bachelor \n", + "Policy Type 3 Personal Auto \n", + "Vehicle Class 6 Four-Door Car \n", + "Number of Open Complaints 6 1/0/00 \n", + "\n", + " frequence of this value missing value \\\n", + "GENDER 457 3054 \n", + "Education 324 2937 \n", + "Policy Type 780 2937 \n", + "Vehicle Class 576 2937 \n", + "Number of Open Complaints 830 2937 \n", + "\n", + " value count \n", + "GENDER {'F': 457, 'M': 413, 'Male': 39, 'female': 28,... \n", + "Education {'Bachelor': 324, 'College': 313, 'High School... \n", + "Policy Type {'Personal Auto': 780, 'Corporate Auto': 234, ... \n", + "Vehicle Class {'Four-Door Car': 576, 'Two-Door Car': 205, 'S... \n", + "Number of Open Complaints {'1/0/00': 830, '1/1/00': 138, '1/2/00': 50, '... " + ] + }, + "execution_count": 64, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "\n", + "categorical_cols = [\"GENDER\", \"Education\", \"Policy Type\", \"Vehicle Class\", \"Number of Open Complaints\"]\n", + "\n", + "\n", + "resume = {}\n", + "for col in categorical_cols:\n", + " resume[col] = {\n", + " \"unique value\": df[col].nunique(),\n", + " \"most frequent value\": df[col].mode().iloc[0] if not df[col].mode().empty else None,\n", + " \"frequence of this value\": df[col].value_counts().iloc[0] if not df[col].value_counts().empty else None,\n", + " \"missing value\": df[col].isnull().sum(),\n", + " \"value count\": df[col].value_counts().to_dict(),\n", + " }\n", + "\n", + "# Convertir le résumé en DataFrame pour une meilleure lisibilité\n", + "resume_df = pd.DataFrame(resume).T\n", + "\n", + "resume_df.head()" ] }, { @@ -116,12 +607,30 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 82, "id": "2dca5073-4520-4f42-9390-4b92733284ed", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ST\n", + "AZ 25\n", + "WA 30\n", + "Washington 81\n", + "Nevada 98\n", + "Cali 120\n", + "Name: count, dtype: int64\n" + ] + } + ], "source": [ - "# Your code here" + "location_frequencies = df['ST'].value_counts()\n", + "\n", + "top_5_least_common = location_frequencies.nsmallest(5).sort_values(ascending=True)\n", + "\n", + "print(top_5_least_common)" ] }, { @@ -135,23 +644,174 @@ ] }, { - "cell_type": "markdown", - "id": "a9f13997-1555-4f98-aca6-970fda1d2c3f", + "cell_type": "code", + "execution_count": 84, + "id": "874f54da-3a13-4343-ac76-d95e84c0e993", "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
CustomerSTGENDEREducationCustomer Lifetime ValueIncomeMonthly Premium AutoNumber of Open ComplaintsPolicy TypeVehicle ClassTotal Claim Amount
0RB50392WashingtonNaNMasterNaN0.01000.01/0/00Personal AutoFour-Door Car2.704934
1QZ44356ArizonaFBachelor697953.59%0.094.01/0/00Personal AutoFour-Door Car1131.464935
2AI49188NevadaFBachelor1288743.17%48767.0108.01/0/00Personal AutoTwo-Door Car566.472247
3WW63253CaliforniaMBachelor764586.18%0.0106.01/0/00Corporate AutoSUV529.881344
4GA49547WashingtonMHigh School or Below536307.65%36357.068.01/0/00Personal AutoFour-Door Car17.269323
\n", + "
" + ], + "text/plain": [ + " Customer ST GENDER Education Customer Lifetime Value \\\n", + "0 RB50392 Washington NaN Master NaN \n", + "1 QZ44356 Arizona F Bachelor 697953.59% \n", + "2 AI49188 Nevada F Bachelor 1288743.17% \n", + "3 WW63253 California M Bachelor 764586.18% \n", + "4 GA49547 Washington M High School or Below 536307.65% \n", + "\n", + " Income Monthly Premium Auto Number of Open Complaints Policy Type \\\n", + "0 0.0 1000.0 1/0/00 Personal Auto \n", + "1 0.0 94.0 1/0/00 Personal Auto \n", + "2 48767.0 108.0 1/0/00 Personal Auto \n", + "3 0.0 106.0 1/0/00 Corporate Auto \n", + "4 36357.0 68.0 1/0/00 Personal Auto \n", + "\n", + " Vehicle Class Total Claim Amount \n", + "0 Four-Door Car 2.704934 \n", + "1 Four-Door Car 1131.464935 \n", + "2 Two-Door Car 566.472247 \n", + "3 SUV 529.881344 \n", + "4 Four-Door Car 17.269323 " + ] + }, + "execution_count": 84, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "*Hint:*\n", - "- *Using value_counts() method simplifies this analysis.*\n", - "- *Futhermore, there is a method that returns the index of the maximum value in a column or row.*\n" + "df.head()" ] }, { "cell_type": "code", - "execution_count": null, - "id": "bcfad6c1-9af2-4b0b-9aa9-0dc5c17473c0", + "execution_count": 90, + "id": "98bf8513-1dfc-438b-94c9-f5f27978878f", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Policy Type\n", + "Personal Auto 780\n", + "Name: count, dtype: int64\n" + ] + } + ], "source": [ - "# Your code here" + "policy_frequency = df['Policy Type'].value_counts()\n", + "\n", + "highest_number_of_policy_sold = policy_frequency.nlargest(1).sort_values(ascending=False)\n", + "\n", + "print(highest_number_of_policy_sold)" ] }, { @@ -176,12 +836,33 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 96, "id": "0c0563cf-6f8b-463d-a321-651a972f82e5", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Average Income for Personal Auto: $38,180.70\n", + "Average Income for Corporate Auto: $41,390.31\n" + ] + } + ], "source": [ - "# Your code here" + "df = df.dropna(subset=['Income'])\n", + "\n", + "\n", + "personal_auto_df = df.loc[df['Policy Type'] == 'Personal Auto']\n", + "corporate_auto_df = df.loc[df['Policy Type'] == 'Corporate Auto']\n", + "\n", + "\n", + "average_income_personal = personal_auto_df['Income'].mean()\n", + "average_income_corporate = corporate_auto_df['Income'].mean()\n", + "\n", + "\n", + "print(f\"Average Income for Personal Auto: ${average_income_personal:,.2f}\")\n", + "print(f\"Average Income for Corporate Auto: ${average_income_corporate:,.2f}\")" ] }, { @@ -251,7 +932,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.13" + "version": "3.12.7" } }, "nbformat": 4,