Text_Analytics_Project_PreProcessing-Final.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Text Analytics Project**\n",
    "**Problem statement**: Identifying memes with hate or sarcastic content by mining the text and Image information using Machine learning models.\n",
    "**Goal of this Project**: Building a decent classifier which can understand the sarcasm and hate by using supervised Machine learning \n",
    "**Tools used**: Python\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "**Notebook Roadmap**:\n",
    "### Used Google VISION API to extract the text out of the memes\n",
    "1. Loading the Memes dataset(Manually labelled sarcasm and category lables apart from hateful/non-hateful labels)\n",
    "2. Pre-processing & Cleaning\n",
    "  1. Expanding the contractions\n",
    "  2. Word segmentation\n",
    "  3. Spelling correction using language model\n",
    "  4. Basic cleaning(Removing ascii characters, removing numbers, custom contractions)\n",
    "  5. Sentiment Analysis(Exploratory purposes - didn't use in the model building)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#importing all the libraries\n",
    "import os\n",
    "import re\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import cv2\n",
    "import pytesseract\n",
    "import matplotlib.pyplot as plt\n",
    "from pytesseract import Output\n",
    "import nltk\n",
    "from nltk import FreqDist\n",
    "from textblob import TextBlob\n",
    "import seaborn as sns\n",
    "from symspellpy.symspellpy import SymSpell\n",
    "from keras.utils import to_categorical \n",
    "import pkg_resources\n",
    "#pip install lmproof\n",
    "from sentistrength import PySentiStr\n",
    "import spellchecker\n",
    "from textblob import TextBlob\n",
    "import lmproof\n",
    "from spellchecker import SpellChecker\n",
    "spell = SpellChecker()\n",
    "import spacy\n",
    "import en_core_web_sm\n",
    "# nlp = en_core_web_sm.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#changing the directory to the data directory\n",
    "path = 'C:/Users/nagar/OneDrive/Desktop/Facebook meme data'\n",
    "os.chdir(path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "#loading only required columns\n",
    "names = [ 'Image_id', 'Hate_label', 'FB_text', 'google_raw_text',\n",
    "       'label_list', 'object_list', 'Semi_cleaned_text', 'Sarcasm_label',\n",
    "       'Category']\n",
    "df = pd.read_csv('Final_data.csv',names= names,header=0)\n",
    "df.dropna(inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "#changing the type of labels\n",
    "df['Category'] = df['Category'].astype('int32')\n",
    "df['Sarcasm_label'] = df['Sarcasm_label'].astype('int32')\n",
    "df = df.reset_index()\n",
    "df_original = df.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pre-processing and cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Why do we have to do a lot of pre-processing & cleaning using language models?**\n",
    "1. Google API extracted text was so messy and it needs a lot of cleaning\n",
    "2. Lot of memes has words like(af,asf,I'm)--> we used custom contractions & language model(pycontractions- glove twitter model) to fix the contractions as there are so many internet slang words, it is better to use glove twitter to resolve contractions\n",
    "3. Word segmentation has to be done as there were few memes with confusing text with mix up of words adjacent to each other(we have used sym spell dictionary method to tackle this)\n",
    "4. A lot of memes generally have spelling mistakes, to restore the context, we have used language model to fix it\n",
    "5. we also did basic cleaning to handle the text for sentiment analysis and for modelling too\n",
    "\n",
    "**Why language models and why not other things like dictionary based approach or something simple?**\n",
    "Language models tend to restore or correct the context more because they must have pre-trained on lot of examples and it would outperform dictionary based methods when we want to restore the symantics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Contractions --> converting I'm to I am and so on"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pycontractions import Contractions\n",
    "cont = Contractions(api_key=\"glove-twitter-100\")\n",
    "cont.load_models()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def contractions_model(df):\n",
    "    text_list = df.to_list()\n",
    "    text_list = list(cont.expand_texts(text_list))\n",
    "    return text_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Word segmentation(\"idon't thinkso thisisa rightplace\") --> \"i don't think so this is a right place\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# word segmentation\n",
    "# Set max_dictionary_edit_distance to avoid spelling correction\n",
    "sym_spell = SymSpell(max_dictionary_edit_distance=0, prefix_length=7)\n",
    "dictionary_path = pkg_resources.resource_filename(\"symspellpy\", \"frequency_dictionary_en_82_765.txt\")\n",
    "# term_index is the column of the term and count_index is the\n",
    "# column of the term frequency\n",
    "sym_spell.load_dictionary(dictionary_path, term_index=0, count_index=1)\n",
    "def word_segmentation(text):\n",
    "    # a sentence without any spaces\n",
    "    result = sym_spell.word_segmentation(text)\n",
    "    return result.corrected_string"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Correcting the spelling using lmProof Language model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\nagar\\anaconda3\\lib\\site-packages\\transformers\\modeling_auto.py:821: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "import lmproof\n",
    "import spacy\n",
    "import en_core_web_sm\n",
    "proof_reader = lmproof.load(\"en\")\n",
    "def correct_spelling(text):\n",
    "    return proof_reader.proofread(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Basic Cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "def basic_cleaning(text):\n",
    "    text = re.sub(r'[A-Z]{2,}',' ',text)\n",
    "    text = text.lower()\n",
    "    text = re.sub(r'[^a-zA-Z]',' ',text)\n",
    "    text = re.sub(r' +',' ',text)\n",
    "    words = nltk.tokenize.word_tokenize(text)\n",
    "#     lemma = nltk.stem.WordNetLemmatizer()\n",
    "#     lemmatized = [lemma.lemmatize(w) for w in words]\n",
    "    return ' '.join(words)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Main Cleaning function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "import contractions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Removing Non-ASCII characters\n",
    "def cleaning_text(text):\n",
    "    text = re.sub(r'[^\\x00-\\x7F]+','', text)\n",
    "    text = re.sub(r'\\s+',' ',text)\n",
    "    text = contractions.fix(text)\n",
    "    text = word_segmentation(text)\n",
    "    return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-129-5b9093e28905>:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['Semi_cleaned_text'][i] = cleaning_text(df['Semi_cleaned_text'][i])\n"
     ]
    }
   ],
   "source": [
    "for i in range(0,len(df)):\n",
    "    df['Semi_cleaned_text'][i] = cleaning_text(df['Semi_cleaned_text'][i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Semi_cleaned_text'] = contractions_model(df['Semi_cleaned_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-131-adc5cf8d0c67>:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['Semi_cleaned_text'][i] = basic_cleaning(df['Semi_cleaned_text'][i])\n"
     ]
    }
   ],
   "source": [
    "for i in range(0,len(df)):\n",
    "    df['Semi_cleaned_text'][i] = basic_cleaning(df['Semi_cleaned_text'][i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-132-5745a80a482a>:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['Semi_cleaned_text'][i] = correct_spelling(df['Semi_cleaned_text'][i])\n"
     ]
    }
   ],
   "source": [
    "for i in range(0,len(df)):\n",
    "    df['Semi_cleaned_text'][i] = correct_spelling(df['Semi_cleaned_text'][i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Custom contractions for replacing misspelled words and confusing words for sentiment analysis and better word2vec embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('New_df.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking for misspelled words to make custom contractions list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = ' '\n",
    "for i in range(len(df['Semi_cleaned_text'])):\n",
    "    text = text + ' ' + df['Semi_cleaned_text'][i]\n",
    "words = list(set(text.split()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [],
   "source": [
    "import enchant\n",
    "Eng_dict = enchant.Dict(\"en_US\")\n",
    "Corpus_words= words\n",
    "Checked_Word_List=[]\n",
    "UnChecked_Word_List=[]\n",
    "for word in Corpus_words:\n",
    "    if Eng_dict.check(word)==True:\n",
    "        Checked_Word_List.append(word)\n",
    "    else:\n",
    "        UnChecked_Word_List.append(word)\n",
    "\n",
    "from spellchecker import SpellChecker\n",
    "spell = SpellChecker()\n",
    "# find those words that may be misspelled\n",
    "misspelled = spell.unknown(words)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "custom_dict = {\n",
    "    'fuk':'fuck',\n",
    "    'af':'as fuck',\n",
    "    'asf':'as fuck',\n",
    "    'n': 'and',\n",
    "    'tom':'from',\n",
    "    'wite': 'white',\n",
    "    'kenned':'ken',\n",
    "    'pedo': 'pedofile',\n",
    "    'vel':' ',\n",
    "    'aaf':' ',\n",
    "    'jou': 'so you',\n",
    "    'adhd': 'attention-deficit hyperactivity disorder',\n",
    "    'conf': 'confused',\n",
    "    'idler':'hitler',\n",
    "    'nti': 'attention',\n",
    "    'tric':'tricare',\n",
    "    'znd':'second',\n",
    "    'congressed':'congress',\n",
    "    'clhwitz': 'auschwitz',\n",
    "    'colo':'colorized',\n",
    "    'pend':'happened',\n",
    "    'lem':'them',\n",
    "    'tel':'tell',\n",
    "    'cams':'canada',\n",
    "    'aya in':'again',\n",
    "    'whited':'white',\n",
    "    'cuter':'cute',\n",
    "    'irk' : 'irritate',\n",
    "    'tran':'tranny',\n",
    "    'rehab': 'rehabilitation',\n",
    "    'wi hens': 'when',\n",
    "    'imma':'i am going to',\n",
    "    'eww':'disgust',\n",
    "    'tromp':'trump',\n",
    "    'oof':'extreme pain',\n",
    "    'cuck':'cock',\n",
    "    'meathead':'stupid',\n",
    "    'hornier':'horny',\n",
    "    'quran':'quran',\n",
    "    'bruh':'bro',\n",
    "    'poops':'poop',\n",
    "    'burqa':'burqa',\n",
    "    'dumpster':'dumpster',\n",
    "    'chemo':'chemo therapy',\n",
    "    'rofl':'roll on the floor laughing',\n",
    "    'isl':'islam',\n",
    "    'nyt':'new york times',\n",
    "    'trons':'trans',\n",
    "    'fwd':'forward',\n",
    "    'whies':'whites',\n",
    "    'iynch':'lynch',\n",
    "    'dumber':'dumb',\n",
    "    'olla':'alla',\n",
    "    'tts':'tits',\n",
    "    'uive':'give',\n",
    "    'muh':'much',\n",
    "    'yolo':'you only live once',\n",
    "    'ghns':'guns',\n",
    "    'paie':'paid',\n",
    "    'qum':'cum',\n",
    "    'cuter':'cute',\n",
    "    'jaly':'july',\n",
    "    'kool':'cool',\n",
    "    'wite':'white',\n",
    "    'arb':'arab',\n",
    "    'uld':'old',\n",
    "    'vagine':'vagina',\n",
    "    'fah':'far',\n",
    "    'gayest':'gay',\n",
    "    'uty':'duty',\n",
    "    'wyo':'who',\n",
    "    'ery':'very',\n",
    "    'yot':'you',\n",
    "    'nte':'note',\n",
    "    'raci':'race',\n",
    "    'ilh':'i will',\n",
    "    'mlk':'milk',\n",
    "    'bruhh':'bro',\n",
    "    'tney':'they',\n",
    "    'mussing':'missing',\n",
    "    'frt':'fart'\n",
    "    \n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [],
   "source": [
    "#fixing custom contractions\n",
    "def expand_contractions(text):\n",
    "    temp = text.split() \n",
    "    res = [] \n",
    "    for wrd in temp: \n",
    "        res.append(custom_dict.get(wrd, wrd)) \n",
    "    res = ' '.join(res) \n",
    "    return str(res)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-172-2c3d01488ecc>:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['Semi_cleaned_text'][i] = expand_contractions(df['Semi_cleaned_text'][i])\n"
     ]
    }
   ],
   "source": [
    "for i in range(0,len(df)):\n",
    "    df['Semi_cleaned_text'][i] = expand_contractions(df['Semi_cleaned_text'][i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('final_dataframe.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sentiment Analysis - Exploratory purposes -- haven't used in the modelling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [],
   "source": [
    "#sentiment analysis\n",
    "senti = PySentiStr()\n",
    "senti.setSentiStrengthLanguageFolderPath(\"C:/Users/nagar/OneDrive/Desktop/Facebook meme data/Senti/SentiStrength_Data\")\n",
    "senti.setSentiStrengthPath('C:/Users/nagar/OneDrive/Desktop/Facebook meme data/Senti/SentiStrength.jar')\n",
    "def sentiment_analysis(df):\n",
    "    sentiment = []\n",
    "    for i in range(len(df)):\n",
    "        sentiment.append(senti.getSentiment(df[i],score= 'scale'))\n",
    "    return sentiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [],
   "source": [
    "sentiment_list = sentiment_analysis(df['Semi_cleaned_text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Sentiment_Scores'] = sentiment_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-198-77e3cb159140>:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  df['Sentiment_Scores'][i] = df['Sentiment_Scores'][i][0]\n"
     ]
    }
   ],
   "source": [
    "for i in range(len(df)):\n",
    "    df['Sentiment_Scores'][i] = df['Sentiment_Scores'][i][0]  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'count of memes')"
      ]
     },
     "execution_count": 199,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de7xVZb3v8c9XvF/wclgacmmhogVkFCu13O7c6U7UUvN0wbNTTItyq+XuJqivcnc2O9pmltvUgzfEY95SE1NTNNFMyBaGIKKFgrqEAG9B6aHA3/ljPFNHy7nWmGsxb7C+79drvOYYzxjjGb85F8zfHOMZ43kUEZiZmXVns0YHYGZmzc/JwszMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4VZjUn6s6Q9Gh2H2YZwsrCqk7RU0l8lDehUPk9SSGptTGR/F8tZkpakL/IOSTdUqd5Zkj6fL4uI7SPimWrU38NYlko6tN7HtU2Tk4XVyhLguNKCpPcA2zQunLdIGg8cDxwaEdsDbcB9jY1q0yNp80bHYNXjZGG1cg1wQm55PDA9v4GkrSR9X9JzklZIulTSNmndwekX/zclrZS0XNIxko6Q9HtJL0s6K1fXZpImSnpa0kuSbpS0SxexfQC4OyKeBoiIP0bE1FxdO0q6Ih3zBUn/IalfWneipIdS3K+ks5PD07rJwEHARemM5aJUHpL2SvPTJF0s6a60za8lvUPSD1N9T0p6Xy6W3SXdLGlVOtaXc+vOTe9zuqQ1khZKakvrrgGGAren43yz84cgaYCkn0t6NX2ev5K0WVo3RNIt6bgv5d7LZpLOkfRs+rtMl7RjWtea3uvJkp4DfpnKT5K0KL2/uyW9M5VL0gWpnj9Jmi9pVBd/M2u0iPDkqaoTsBQ4FHgKeDfQD3geeCcQQGva7ofADGAXYAfgduC7ad3BwDrgW8AWwBeAVcBP0rYjgf8H7JG2PwOYAwwGtgL+D3BdF/F9FngZ+AbZWUW/Tut/lvbfDtgVeAT4Ylp3IvC3FE8/4BRgGaC0fhbw+U71BbBXmp8GvAiMAbYm+0JdQpZY+wH/Adyftt0MmJs+gy2BPYBngMPS+nPTZ3BE2ve7wJzOf4du/k7fBS5Nn+8WZIlOqa7HgAvSZ7A18A9pn5OAxSmW7YFbgGvSutb0Xqen/bYBjknbvxvYHDgHeDhtf1h6fzul474bGNjof7+euvj30ugAPG16E28li3PSF9JYYGb6soj0pSLgL8Ceuf0+CCxJ8wcDr5e+yMkSRAD757afCxyT5hcBh+TWDUxf6pt3EeO/APemGF4CJqby3YC1wDa5bY/LfYGfCCzOrds2xfWOtDyL4mRxWW7d6cCi3PJ7gFfT/P7Ac53qmgRclebPBe7NrRsBvN7579DN3+k7wG2l2Dr9HVaV++zILtf9a255n9LnnEsWe+TW3wWcnFveDHiN7IfDR4DfAwcAmzX6362n7idfU7RaugZ4EBhGp0tQQAvZF+1cSaWy0q/akpciYn2afz29rsitf53s1y1kXz63Snojt3492Zf/C50Di4hrgWslbUH26/daSb8DXiH7lb08F9dmZGdGJX/M1fNa2m57Ktf5PXT3nnaX9GpufT/gV+ViIfsS3lrS5hGxroI4ziNLOPek9zA1IqYAQ4Bnu6hjd+DZ3PKzZIlit1xZ/rN6J/AjSefnygQMiohfpstbPwaGSroV+HpErK4gdqszt1lYzUTEs2SXWI4gu1yR9yLZF+PIiNgpTTtG1uDcG88Dh+fq2ikito6ItyWKTjH+LSJuAuYDo1I9a4EBuXr6R8TICuOoZjfOz5OdaeXf0w4RcUQ1YomINRHxtYjYA/g48FVJh6TjDu2igXoZWQIoGUp2uTCf8PLHfZ7sEl7+PWwTEQ+nGC6MiDFklxX3Jrs0aE3IycJq7WTgIxHxl3xhRLwBXAZcIGlXAEmDJB3Wy+NcCkzONZ62SDq63IapkfpISTukBtvDyb6sfhMRy4F7gPMl9U/r95T04QrjWEF2Pb8aHgFWSzpT0jaS+kkaJekD1YhF0sck7aXstGI12ZnY+nTc5cAUSdtJ2lrSgWm364B/kzRM0vbAfwI3dHMmcykwSdLIdMwdJX0qzX9A0v7p7O4vZO0v67uoxxrMycJqKiKejoj2LlafSdb4OUfSarI2hH16eagfkTWW3yNpDVlj9/5dbLsaOAt4DngV+C/glIh4KK0/gaxB+Qmyy1I/JWsDqTSOT6Y7fy7szRspSZfgPg6MJjtDexG4HNixwiq+C5yT7nb6epn1w8k+8z8Ds4GLI2JW7rh7kX1GHcBn0j5X8tblxSVkX/Cnd/MebgW+B1yf/saPA4en1f3JfjC8QnY56yXg+xW+N6uz0h0cZmZmXfKZhZmZFXKyMDOzQk4WZmZWyMnCzMwKbbIP5Q0YMCBaW1sbHYaZ2UZl7ty5L0ZES+fyTTZZtLa20t7e1R2bZmZWjqRny5X7MpSZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaFapYs0oDv96eB2hdK+koq30XSTEl/SK875/aZJGmxpKfy4xpIGiNpQVp3oXJDmJmZWe3V8sxiHfC1iHg32Ri7p0oaAUwE7ouI4WTj+U4ESOvGkQ1CMxa4WFJpiM1LgAlk/e8PT+vNzKxOavYEdxpxbHmaXyNpETAIOBo4OG12NdkA92em8usjYi2wRNJiYD9JS4H+ETEbQNJ0sjGT76pV7Ga90TrxjqrWt3TKkVWtz2xD1KXNQlIr8D7gN8BuKZGUEsquabNB/P1A7x2pbFCa71xe7jgTJLVLal+1alU134KZWZ9W82SRxum9GTgjIlZ3t2mZsuim/O2FEVMjoi0i2lpa3tYPlpmZ9VJNk0UaiP1m4NqIuCUVr5A0MK0fCKxM5R3AkNzug4FlqXxwmXIzM6uTWt4NJeAKYFFE/CC3agYwPs2PB27LlY+TtJWkYWQN2Y+kS1VrJB2Q6jwht4+ZmdVBLbsoPxA4HlggaV4qOwuYAtwo6WTgOeBTABGxUNKNwBNkd1KdGhHr036nANOAbcgatt24bWZWR7W8G+ohyrc3ABzSxT6TgcllytuBUdWLzszMesJPcJuZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzArVcljVKyWtlPR4ruwGSfPStLQ0gp6kVkmv59ZdmttnjKQFkhZLujANrWpmZnVUy2FVpwEXAdNLBRHxmdK8pPOBP+W2fzoiRpep5xJgAjAHuBMYi4dVNTOrq5qdWUTEg8DL5dals4NPA9d1V4ekgUD/iJgdEUGWeI6pdqxmZta9RrVZHASsiIg/5MqGSfqdpAckHZTKBgEduW06UpmZmdVRLS9Ddec4/v6sYjkwNCJekjQG+JmkkUC59onoqlJJE8guWTF06NAqhmtm1rfV/cxC0ubAscANpbKIWBsRL6X5ucDTwN5kZxKDc7sPBpZ1VXdETI2Itohoa2lpqUX4ZmZ9UiMuQx0KPBkRb15ektQiqV+a3wMYDjwTEcuBNZIOSO0cJwC3NSBmM7M+rZa3zl4HzAb2kdQh6eS0ahxvb9j+R2C+pMeAnwJfiohS4/gpwOXAYrIzDt8JZWZWZzVrs4iI47ooP7FM2c3AzV1s3w6MqmpwZmbWI36C28zMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCtVyWNUrJa2U9Hiu7FxJL0ial6YjcusmSVos6SlJh+XKx0hakNZdmMbiNjOzOqrlmcU0YGyZ8gsiYnSa7gSQNIJsbO6RaZ+LJfVL218CTACGp6lcnWZmVkM1SxYR8SDwcoWbHw1cHxFrI2IJsBjYT9JAoH9EzI6IAKYDx9QmYjMz60oj2ixOkzQ/XabaOZUNAp7PbdORygal+c7lZUmaIKldUvuqVauqHbeZWZ9V72RxCbAnMBpYDpyfysu1Q0Q35WVFxNSIaIuItpaWlg2N1czMkromi4hYERHrI+IN4DJgv7SqAxiS23QwsCyVDy5TbmZmdVTXZJHaIEo+AZTulJoBjJO0laRhZA3Zj0TEcmCNpAPSXVAnALfVM2YzM4PNa1WxpOuAg4EBkjqAbwMHSxpNdilpKfBFgIhYKOlG4AlgHXBqRKxPVZ1CdmfVNsBdaTIzszqqWbKIiOPKFF/RzfaTgcllytuBUVUMzczMeshPcJuZWaEeJQtJO0vat1bBmJlZcypMFpJmSeovaRfgMeAqST+ofWhmZtYsKjmz2DEiVgPHAldFxBjg0NqGZWZmzaSSZLF5uuX108DPaxyPmZk1oUqSxXeAu4GnI+K3kvYA/lDbsMzMrJkU3jobETcBN+WWnwH+Zy2DMjOz5lJJA/feku4rjUshaV9J59Q+NDMzaxaVXIa6DJgE/A0gIuaTjT1hZmZ9RCXJYtuIeKRT2bpaBGNmZs2pkmTxoqQ9SV2DS/okWffiZmbWR1TSN9SpwFTgXZJeAJYAn61pVGZm1lQquRvqGeBQSdsBm0XEmtqHZWZmzaQwWUjaiWwciVayB/QAiIgv1zQyMzNrGpVchroTmAMsAN6obThmZtaMKkkWW0fEV2seiZmZNa1K7oa6RtIXJA2UtEtpqnlkZmbWNCpJFn8FzgNmA3PT1F60k6QrJa0sPfmdys6T9KSk+ZJuTe0hSGqV9LqkeWm6NLfPGEkLJC2WdKFKjSZmZlY3lSSLrwJ7RURrRAxL0x4V7DcNGNupbCYwKiL2BX5P9mR4ydMRMTpNX8qVXwJMAIanqXOdZmZWY5Uki4XAaz2tOCIeBF7uVHZPRJSe/p4DDO6ujtQ1ev+ImB0RAUwHjulpLGZmtmEqaeBeD8yTdD+wtlRYhVtnTwJuyC0Pk/Q7YDVwTkT8ChgEdOS26UhlZUmaQHYWwtChQzcwPDMzK6kkWfwsTVUj6Wyy/qWuTUXLgaER8ZKkMcDPJI0EyrVPRFf1RsRUsqfNaWtr63I7MzPrmUqe4L5a0jZkX+ZPbegBJY0HPgYcki4tERFrSWctETFX0tPA3mRnEvlLVYOBZRsag5mZ9Uwl41l8HJgH/CItj5Y0ozcHkzQWOBM4KiJey5W3SOqX5vcga8h+JiKWA2skHZDugjoBuK03xzYzs96rpIH7XGA/4FWAiJgHDCvaSdJ1ZLfb7iOpQ9LJwEXADsDMTrfI/iMwX9JjwE+BL0VEqXH8FOByYDHwNHBXhe/NzMyqpJI2i3UR8adOjzcUtgdExHFliq/oYtubgZu7WNcOjKogTjMzq5FKksXjkv4X0E/ScODLwMO1DcvMzJpJJZehTgdGkjVAX0d2a+sZtQzKzMyaSyV3Q70GnJ0mMzPrgyoZz6INOIs0nkWpPHXZYWZmfUAlbRbXAt/A41mYmfVZlSSLVRHRq+cqzMxs01BJsvi2pMuB+/j7vqFuqVlUZmbWVCpJFp8D3gVswVuXoQJwsjAz6yMqSRbvjYj31DwSMzNrWpU8ZzFH0oiaR2JmZk2rkjOLfwDGS1pC1mYhIHzrrJlZ31FJsvAwpmZmfVwlT3A/W49AzMyseVXSZmFmZn1cl8lC0lb1DMTMzJpXd2cWswEkXVOnWMzMrEl112axZRov+0OSju280k9wm5n1Hd2dWXwJOADYCfh4p+ljRRVLulLSSkmP58p2kTRT0h/S6865dZMkLZb0lKTDcuVjJC1I6y5UpyH7zMys9ro8s4iIh4CHJLVHRNnhUAtMIxtze3qubCJwX0RMkTQxLZ+ZHvobRzbI0u7AvZL2joj1wCXABGAOcCfZrbweh9vMrI4quRvqGklflvTTNJ0uaYuinSLiQeDlTsVHA1en+auBY3Ll10fE2ohYAiwG9pM0EOgfEbMjIsgSzzGYmVldVZIsLgbGpNeLgfeT/drvjd0iYjlAet01lQ8Cns9t15HKBqX5zuVlSZogqV1S+6pVq3oZopmZdVbJE9wfiIj35pZ/KemxKsdRrh0iuikvKyKmAlMB2trautzOzMx6ppIzi/WS9iwtSNoDWN/L461Il5ZIrytTeQcwJLfdYGBZKh9cptzMzOqokmTxDeB+SbMkPQD8EvhaL483Axif5scDt+XKx0naStIwYDjwSLpUtUbSAekuqBNy+5iZWZ1U0jfUfZKGA/uQXRZ6MiLWFuyGpOuAg4EBkjqAbwNTgBslnQw8B3wqHWOhpBuBJ4B1wKnpTiiAU8jurNqG7C4o3wllZlZnlbRZkJLD/J5UHBHHdbHqkC62nwxMLlPeDozqybHNzKy63JGgmZkVcrIwM7NChclC0n2VlJmZ2aaryzYLSVsD25I1UO/MW8889CfrksPMzPqI7hq4vwicQZYY5vJWslgN/LjGcZmZWRPpriPBHwE/knR6RPx3HWMye5vWiXdUvc6lU46sep1mm6pKnrP4b0kfAlrz20fE9C53MjOzTUphskgj5e0JzOOtbj5KPcCamVkfUMlDeW3AiNRFuJmZ9UGVPGfxOPCOWgdiZmbNq5IziwHAE5IeAd7sEyoijqpZVGZm1lQqSRbn1joIMzNrbpXcDfVAPQIxM7PmVcndUGt4a3S6LYEtgL9ERP9aBmZmZs2jkjOLHfLLko4B9qtZRGZm1nR63OtsRPwM+EgNYjEzsyZVyWWoY3OLm5E9d9HrZy4k7QPckCvaA/gWsBPwBWBVKj8rIu5M+0wCTiZ7KPDLEXF3b49vZmY9V8ndUB/Pza8DlgJH9/aAEfEUMBpAUj/gBeBW4HPABRHx/fz2kkYA44CRZJ0a3itp79ywq2ZmVmOVtFl8robHPwR4OiKeldTVNkcD16ehXZdIWkzWZjK7hnGZmVlOJYMfDZZ0q6SVklZIulnS4CodfxxwXW75NEnzJV2ZxtAAGAQ8n9umI5WZmVmdVNLAfRUwg+wS0CDg9lS2QSRtCRwF3JSKLiHrsHA0sBw4v7Rpmd3LtplImiCpXVL7qlWrym1iZma9UEmyaImIqyJiXZqmAS1VOPbhwKMRsQIgIlZExPqIeAO4jLduz+0AhuT2GwwsK1dhREyNiLaIaGtpqUaIZmYGlSWLFyV9VlK/NH0WeKkKxz6O3CUoSQNz6z5B1oEhZGc14yRtJWkYMBx4pArHNzOzClVyN9RJwEXABWSXfx5OZb0maVvgn8mGbi35L0mj0zGWltZFxEJJNwJPkN2NdarvhDIzq69K7oZ6jqxtoWoi4jXgf3QqO76b7ScDk6sZg5mZVa6Su6GulrRTbnlnSVfWNiwzM2smlbRZ7BsRr5YWIuIV4H21C8nMzJpNJclis9wzD0jahcraOszMbBNRyZf++cDDkn5K1vj8adx+YGbWp1TSwD1dUjtZT7MCjo2IJ2oemZmZNY2KLiel5OAEYWbWR/V4PAszM+t7nCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaF3CGgAdA68Y6q1rd0ypFVrc/MGstnFmZmVqghyULSUkkLJM1LnRQiaRdJMyX9Ib3mu0WfJGmxpKckHdaImM3M+rJGnln8U0SMjoi2tDwRuC8ihgP3pWUkjQDGASOBscDFkvo1ImAzs76qmdosjgYOTvNXA7OAM1P59RGxFlgiaTGwHzC7ATGabdTcNmW91agziwDukTRX0oRUtltELAdIr7um8kHA87l9O1LZ20iaIKldUvuqVatqFLqZWd/TqDOLAyNimaRdgZmSnuxmW5Upi3IbRsRUYCpAW1tb2W3MzKznGnJmERHL0utK4Fayy0orJA0ESK8r0+YdwJDc7oOBZfWL1szM6p4sJG0naYfSPPBR4HFgBjA+bTYeuC3NzwDGSdpK0jBgOPBIfaM2M+vbGnEZajfgVkml4/8kIn4h6bfAjZJOBp4DPgUQEQsl3Ug2rOs64NSIWN+AuM3M+qy6J4uIeAZ4b5nyl4BDuthnMjC5xqGZmVkX/AS3mZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVasQY3EMk3S9pkaSFkr6Sys+V9IKkeWk6IrfPJEmLJT0l6bB6x2xm1tc1YgzudcDXIuJRSTsAcyXNTOsuiIjv5zeWNAIYB4wEdgfulbS3x+E2M6ufup9ZRMTyiHg0za8BFgGDutnlaOD6iFgbEUuAxcB+tY/UzMxKGnFm8SZJrcD7gN8ABwKnSToBaCc7+3iFLJHMye3WQRfJRdIEYALA0KFDaxa3mdVG68Q7qlrf0ilHVrW+vqxhDdyStgduBs6IiNXAJcCewGhgOXB+adMyu0e5OiNiakS0RURbS0tLDaI2M+ubGpIsJG1BliiujYhbACJiRUSsj4g3gMt461JTBzAkt/tgYFk94zUz6+sacTeUgCuARRHxg1z5wNxmnwAeT/MzgHGStpI0DBgOPFKveM3MrDFtFgcCxwMLJM1LZWcBx0kaTXaJaSnwRYCIWCjpRuAJsjupTt3Y7oSq9nVY8LVYM6uvuieLiHiI8u0Qd3azz2Rgcs2CMjOzbvkJbjMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK7TRJAtJYyU9JWmxpImNjsfMrC9pxBjcPSapH/Bj4J+BDuC3kmZExBO1OF4txsw2s01Dtb8flk45sqr11crGcmaxH7A4Ip6JiL8C1wNHNzgmM7M+QxHR6BgKSfokMDYiPp+Wjwf2j4jTOm03AZiQFvcBnurlIQcAL/Zy31pyXD3juHrGcfXMphrXOyOipXPhRnEZClCZsrdluYiYCkzd4INJ7RHRtqH1VJvj6hnH1TOOq2f6Wlwby2WoDmBIbnkwsKxBsZiZ9TkbS7L4LTBc0jBJWwLjgBkNjsnMrM/YKC5DRcQ6SacBdwP9gCsjYmEND7nBl7JqxHH1jOPqGcfVM30qro2igdvMzBprY7kMZWZmDeRkYWZmhZwsCkj6uqSQNKDRsQBI+t+S5kuaJ+keSbs3OiYASedJejLFdquknRodE4CkT0laKOkNSQ2/zbEZu62RdKWklZIeb3QseZKGSLpf0qL0N/xKo2MCkLS1pEckPZbi+vdGx5QnqZ+k30n6eTXrdbLohqQhZF2MPNfoWHLOi4h9I2I08HPgW40OKJkJjIqIfYHfA5MaHE/J48CxwIONDiTXbc3hwAjgOEkjGhsVANOAsY0Ooox1wNci4t3AAcCpTfJ5rQU+EhHvBUYDYyUd0OCY8r4CLKp2pU4W3bsA+CZlHgBslIhYnVvcjiaJLSLuiYh1aXEO2bMwDRcRiyKit0/yV1tTdlsTEQ8CLzc6js4iYnlEPJrm15B9AQ5qbFQQmT+nxS3S1BT/DyUNBo4ELq923U4WXZB0FPBCRDzW6Fg6kzRZ0vPAv9A8ZxZ5JwF3NTqIJjQIeD633EETfPltDCS1Au8DftPYSDLpUs88YCUwMyKaIi7gh2Q/cN+odsUbxXMWtSLpXuAdZVadDZwFfLS+EWW6iysibouIs4GzJU0CTgO+3QxxpW3OJrt8cG09Yqo0riZRUbc19vckbQ/cDJzR6cy6YSJiPTA6tc3dKmlURDS0zUfSx4CVETFX0sHVrr9PJ4uIOLRcuaT3AMOAxyRBdknlUUn7RcQfGxVXGT8B7qBOyaIoLknjgY8Bh0QdH+DpwefVaO62pockbUGWKK6NiFsaHU9nEfGqpFlkbT6NvkHgQOAoSUcAWwP9Jf3fiPhsNSr3ZagyImJBROwaEa0R0Ur2n/z99UgURSQNzy0eBTzZqFjyJI0FzgSOiojXGh1Pk3K3NT2g7JfaFcCiiPhBo+MpkdRSuttP0jbAoTTB/8OImBQRg9N31jjgl9VKFOBksTGaIulxSfPJLpM1xe2EwEXADsDMdFvvpY0OCEDSJyR1AB8E7pB0d6NiSTcAlLqtWQTcWONuayoi6TpgNrCPpA5JJzc6puRA4HjgI+nf1Lz0q7nRBgL3p/+DvyVrs6jqbarNyN19mJlZIZ9ZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysrBNjqSzU2+gpd559+9lPaPzt2pKOqrWPcVKOljSh2p5DLPe6NNPcNumR9IHyZ4if39ErE1dy2/Zy+pGA23AnQARMYPaP0R3MPBn4OEaHwdJm+c6fzTrls8sbFMzEHgxItYCRMSLEbEMQNIYSQ9ImivpbkkDU/ksSd9LYxT8XtJB6Qnr7wCfSWcnn5F0oqSL0j7TJF2Sxlt4RtKH07gQiyRNKwUj6aOSZkt6VNJNqZ8jJC2V9O+pfIGkd6XO8r4E/Fs65kH5N5aOUXo47XeSdkjl30x1PCZpSiobLWmO3hpfZOfce/1PSQ8AX+nqMzF7m4jw5GmTmYDtgXlkY2pcDHw4lW9B9mu9JS1/Brgyzc8Czk/zRwD3pvkTgYtydb+5TDYGxPVknQMeDawG3kP2A2wu2VnJALJxNLZL+5wJfCvNLwVOT/P/Clye5s8Fvt7Fe7sdODD3PjcnGxvjYWDbVL5Lep2fe+/fAX6Ye68XF30mnjx1nnwZyjYpEfFnSWOAg4B/Am5I7QztwCiy7kgA+gHLc7uWOqmbC7RWeLjbIyIkLQBWRMQCAEkLUx2DyQY5+nU65pZk3WqUO+axFRzv18APJF0L3BIRHZIOBa6K1B9XRLwsaUdgp4h4IO13NXBTrp4b0us+dP+ZmL3JycI2OZF1Hz0LmJW+yMeTfSEvjIgPdrHb2vS6nsr/X5T2eSM3X1rePNU1MyKOq8YxI2KKpDvIzn7mpEQhet7N+V/Sq+j+MzF7k9ssbJMiaZ9OPfOOBp4FngJaUgM4kraQNLKgujVknSP21hzgQPNqeTIAAADMSURBVEl7pWNuK2nv3h5T0p6R9Yj8PbIzpXcB9wAnSdo2bbNLRPwJeCXX5nE88ECZKnvzmVgf5WRhm5rtgaslPZF6BR0BnBvZMKafBL4n6TGydo2iW1TvB0aUGrh7GkhErCJr57guxTKH7Au+O7cDnyjXwA2ckXocfgx4HbgrIn5BdodWu7KR276eth0PnJeOO5qs3aJzfL35TKyPcq+zZmZWyGcWZmZWyMnCzMwKOVmYmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFfr/mTe7BJXfcdwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(df['Sentiment_Scores'], bins = 15)\n",
    "plt.title(f'Meme Sentiment scores')\n",
    "plt.xlabel('Sentiment score')\n",
    "plt.ylabel('count of memes')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([  5.,  97., 215.,   0., 351., 697.,   0., 157.,  57.,   4.]),\n",
       " array([-4.0, -3.3, -2.6, -1.9000000000000004, -1.2000000000000002, -0.5,\n",
       "        0.1999999999999993, 0.8999999999999995, 1.5999999999999996, 2.3,\n",
       "        3.0], dtype=object),\n",
       " <a list of 10 Patch objects>)"
      ]
     },
     "execution_count": 215,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAARQElEQVR4nO3df6jdd33H8efLVKvzB7Zr0mVJWDIIztRpdZesUhibURutNN2gENkkbIXsjzgUHC5RmMgIVAS3wdaNoM7A1BLUktAyNcssMnBNb2vVpmnWzGpzl9hcFVEnRBLf++N+u50m595zkntOz70fng8I3+/3cz7f833dS/vKN99zzvekqpAkteUFkw4gSRo9y12SGmS5S1KDLHdJapDlLkkNumrSAQCuu+66Wr9+/aRjSNKy8vDDD3+/qlb2e2xJlPv69euZnp6edAxJWlaSfHe+x7wsI0kNstwlqUGWuyQ1yHKXpAZZ7pLUoIHlnuRVSR7t+fPjJO9Ncm2Sw0me7JbX9OyzJ8nJJCeS3DLeH0GSdLGB5V5VJ6rqxqq6Efgt4GfAvcBu4EhVbQSOdNsk2QRsB24AtgJ3J1kxpvySpD4u97LMFuC/quq7wDZgfze+H7i9W98G3FNV56rqKeAksHkUYSVJw7ncct8OfLZbv76qzgB0y1Xd+BrgVM8+M93YcyTZmWQ6yfTs7OxlxpAkLWToT6gmeRFwG7Bn0NQ+Y5d8I0hV7QP2AUxNTfmNIVqy1u++fyLH/c5dt07kuGrD5Zy5vw14pKqe6bafSbIaoFue7cZngHU9+60FTi82qCRpeJdT7u/k/y/JABwCdnTrO4CDPePbk1ydZAOwETi62KCSpOENdVkmyS8BbwH+tGf4LuBAkjuBp4E7AKrqWJIDwOPAeWBXVV0YaWpJ0oKGKveq+hnwyxeN/YC5d8/0m78X2LvodJKkK+InVCWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1KChyj3JK5N8LskTSY4neWOSa5McTvJkt7ymZ/6eJCeTnEhyy/jiS5L6GfbM/W+BL1bVbwCvA44Du4EjVbURONJtk2QTsB24AdgK3J1kxaiDS5LmN7Dck7wC+B3gEwBV9fOq+hGwDdjfTdsP3N6tbwPuqapzVfUUcBLYPOrgkqT5DXPm/uvALPBPSb6e5ONJXgpcX1VnALrlqm7+GuBUz/4z3Zgk6XkyTLlfBbwB+Ieqej3wP3SXYOaRPmN1yaRkZ5LpJNOzs7NDhZUkDWeYcp8BZqrqwW77c8yV/TNJVgN0y7M989f17L8WOH3xk1bVvqqaqqqplStXXml+SVIfA8u9qr4HnEryqm5oC/A4cAjY0Y3tAA5264eA7UmuTrIB2AgcHWlqSdKCrhpy3p8Bn07yIuDbwB8z9xfDgSR3Ak8DdwBU1bEkB5j7C+A8sKuqLow8uSRpXkOVe1U9Ckz1eWjLPPP3AnsXkUuStAh+QlWSGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQUOVe5LvJPlWkkeTTHdj1yY5nOTJbnlNz/w9SU4mOZHklnGFlyT1dzln7r9XVTdW1VS3vRs4UlUbgSPdNkk2AduBG4CtwN1JVowwsyRpgMVcltkG7O/W9wO394zfU1Xnquop4CSweRHHkSRdpmHLvYAvJ3k4yc5u7PqqOgPQLVd142uAUz37znRjz5FkZ5LpJNOzs7NXll6S1NdVQ867uapOJ1kFHE7yxAJz02esLhmo2gfsA5iamrrkcUnSlRvqzL2qTnfLs8C9zF1meSbJaoBuebabPgOs69l9LXB6VIElSYMNLPckL03y8mfXgbcCjwGHgB3dtB3AwW79ELA9ydVJNgAbgaOjDi5Jmt8wl2WuB+5N8uz8z1TVF5M8BBxIcifwNHAHQFUdS3IAeBw4D+yqqgtjSS9J6mtguVfVt4HX9Rn/AbBlnn32AnsXnU6SdEX8hKokNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSg4Yu9yQrknw9yX3d9rVJDid5slte0zN3T5KTSU4kuWUcwSVJ87ucM/f3AMd7tncDR6pqI3Ck2ybJJmA7cAOwFbg7yYrRxJUkDWOock+yFrgV+HjP8DZgf7e+H7i9Z/yeqjpXVU8BJ4HNo4krSRrGsGfufwO8H/hFz9j1VXUGoFuu6sbXAKd65s10Y8+RZGeS6STTs7Ozlx1ckjS/geWe5B3A2ap6eMjnTJ+xumSgal9VTVXV1MqVK4d8aknSMK4aYs7NwG1J3g68GHhFkn8GnkmyuqrOJFkNnO3mzwDrevZfC5weZWhJ0sIGnrlX1Z6qWltV65l7ofTfquqPgEPAjm7aDuBgt34I2J7k6iQbgI3A0ZEnlyTNa5gz9/ncBRxIcifwNHAHQFUdS3IAeBw4D+yqqguLTipJGtpllXtVPQA80K3/ANgyz7y9wN5FZpMkXSE/oSpJDbLcJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ1azO0HpOfN+t33TzqCtKx45i5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQZa7JDVoYLkneXGSo0m+keRYkg9349cmOZzkyW55Tc8+e5KcTHIiyS3j/AEkSZca5sz9HPCmqnodcCOwNclNwG7gSFVtBI502yTZBGwHbgC2AncnWTGO8JKk/gaWe835abf5wu5PAduA/d34fuD2bn0bcE9Vnauqp4CTwOaRppYkLWioa+5JViR5FDgLHK6qB4Hrq+oMQLdc1U1fA5zq2X2mG7v4OXcmmU4yPTs7u5ifQZJ0kaHKvaouVNWNwFpgc5LXLDA9/Z6iz3Puq6qpqppauXLlcGklSUO5rHfLVNWPgAeYu5b+TJLVAN3ybDdtBljXs9ta4PSik0qShjbMu2VWJnllt/4S4M3AE8AhYEc3bQdwsFs/BGxPcnWSDcBG4Oiog0uS5jfM1+ytBvZ373h5AXCgqu5L8jXgQJI7gaeBOwCq6liSA8DjwHlgV1VdGE98SVI/A8u9qr4JvL7P+A+ALfPssxfYu+h0kqQr4idUJalBlrskNchyl6QGWe6S1CDLXZIaZLlLUoMsd0lqkOUuSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAYN8wXZWmLW775/Ysf+zl23TuzYkoY38Mw9ybokX0lyPMmxJO/pxq9NcjjJk93ymp599iQ5meREklvG+QNIki41zGWZ88D7qurVwE3AriSbgN3AkaraCBzptuke2w7cAGwF7k6yYhzhJUn9DSz3qjpTVY906z8BjgNrgG3A/m7afuD2bn0bcE9Vnauqp4CTwOZRB5ckze+yXlBNsh54PfAgcH1VnYG5vwCAVd20NcCpnt1murGLn2tnkukk07Ozs5efXJI0r6HLPcnLgM8D762qHy80tc9YXTJQta+qpqpqauXKlcPGkCQNYahyT/JC5or901X1hW74mSSru8dXA2e78RlgXc/ua4HTo4krSRrGMO+WCfAJ4HhVfaznoUPAjm59B3CwZ3x7kquTbAA2AkdHF1mSNMgw73O/GXgX8K0kj3ZjHwDuAg4kuRN4GrgDoKqOJTkAPM7cO212VdWFkSeXJM1rYLlX1b/T/zo6wJZ59tkL7F1ELknSInj7AUlqkOUuSQ2y3CWpQZa7JDXIcpekBlnuktQgy12SGuSXdUi6xKS+EMYvgxkdz9wlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGWe6S1CDLXZIaNLDck3wyydkkj/WMXZvkcJInu+U1PY/tSXIyyYkkt4wruCRpfsOcuX8K2HrR2G7gSFVtBI502yTZBGwHbuj2uTvJipGllSQNZWC5V9VXgR9eNLwN2N+t7wdu7xm/p6rOVdVTwElg84iySpKGdKXX3K+vqjMA3XJVN74GONUzb6Ybu0SSnUmmk0zPzs5eYQxJUj+j/iam9BmrfhOrah+wD2BqaqrvnKVuUt9WI0mDXOmZ+zNJVgN0y7Pd+AywrmfeWuD0lceTJF2JKy33Q8CObn0HcLBnfHuSq5NsADYCRxcXUZJ0uQZelknyWeB3geuSzAAfAu4CDiS5E3gauAOgqo4lOQA8DpwHdlXVhTFllyTNY2C5V9U753loyzzz9wJ7FxNKkrQ4fkJVkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNchyl6QGjfreMpJ0xSZ1v6bv3HXrRI47Tp65S1KDLHdJapDlLkkNstwlqUGWuyQ1yHKXpAZZ7pLUIMtdkhpkuUtSgyx3SWqQ5S5JDbLcJalBlrskNWhs5Z5ka5ITSU4m2T2u40iSLjWWW/4mWQH8PfAWYAZ4KMmhqnp8HMeb1G1CJWmpGtf93DcDJ6vq2wBJ7gG2AWMpd0lajEmeII7rXvLjKvc1wKme7Rngt3snJNkJ7Ow2f5rkxCKOdx3w/UXs/3xaTlnhorz5yASTDLacfrcDsy6x33VTv9ulJB9ZVN5fm++BcZV7+ozVczaq9gH7RnKwZLqqpkbxXOO2nLLC8spr1vFZTnmXU1YYX95xvaA6A6zr2V4LnB7TsSRJFxlXuT8EbEyyIcmLgO3AoTEdS5J0kbFclqmq80neDXwJWAF8sqqOjeNYnZFc3nmeLKessLzymnV8llPe5ZQVxpQ3VTV4liRpWfETqpLUIMtdkhrUVLkn+fMkleS6SWdZSJK/SvLNJI8m+XKSX510pvkk+WiSJ7q89yZ55aQzLSTJHUmOJflFkiX5drjldGuOJJ9McjbJY5POMkiSdUm+kuR499/AeyadaT5JXpzkaJJvdFk/POpjNFPuSdYxd7uDpyedZQgfrarXVtWNwH3AX0460AIOA6+pqtcC/wnsmXCeQR4D/gD46qSD9NNza463AZuAdybZNNlUC/oUsHXSIYZ0HnhfVb0auAnYtYR/t+eAN1XV64Abga1JbhrlAZopd+Cvgfdz0YellqKq+nHP5ktZwpmr6stVdb7b/A/mPrOwZFXV8apazKedx+3/bs1RVT8Hnr01x5JUVV8FfjjpHMOoqjNV9Ui3/hPgOHOfll9yas5Pu80Xdn9G2gNNlHuS24D/rqpvTDrLsJLsTXIK+EOW9pl7rz8B/mXSIZa5frfmWJIFtJwlWQ+8Hnhwsknml2RFkkeBs8Dhqhpp1nHdfmDkkvwr8Ct9Hvog8AHgrc9vooUtlLeqDlbVB4EPJtkDvBv40PMasMegrN2cDzL3z95PP5/Z+hkm7xI28NYcWpwkLwM+D7z3on8lLylVdQG4sXsd694kr6mqkb22sWzKvare3G88yW8CG4BvJIG5ywaPJNlcVd97HiM+x3x5+/gMcD8TLPdBWZPsAN4BbKkl8MGIy/jdLkXemmOMkryQuWL/dFV9YdJ5hlFVP0ryAHOvbYys3Jf9ZZmq+lZVraqq9VW1nrn/ed4wyWIfJMnGns3bgCcmlWWQJFuBvwBuq6qfTTpPA7w1x5hk7uzuE8DxqvrYpPMsJMnKZ995luQlwJsZcQ8s+3Jfpu5K8liSbzJ3OWnJvmUL+Dvg5cDh7q2b/zjpQAtJ8vtJZoA3Avcn+dKkM/XqXpx+9tYcx4EDY741x6Ik+SzwNeBVSWaS3DnpTAu4GXgX8Kbuv9VHk7x90qHmsRr4StcBDzF3zf2+UR7A2w9IUoM8c5ekBlnuktQgy12SGmS5S1KDLHdJapDlLkkNstwlqUH/C2qFo8TrDqWSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#grouping the dataset by party to get the average estimates of vocabulary for different parties\n",
    "hate_df = df[df['Hate_label'] == 1]\n",
    "plt.hist(hate_df['Sentiment_Scores'])\n",
    "# plt.title('Avg vocabulary in different parties')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([6.000e+00, 1.430e+02, 4.070e+02, 6.400e+02, 0.000e+00, 1.362e+03,\n",
       "        3.540e+02, 1.670e+02, 7.000e+00, 1.000e+00]),\n",
       " array([-4.0, -3.2, -2.4, -1.5999999999999996, -0.7999999999999998, 0.0,\n",
       "        0.8000000000000007, 1.6000000000000005, 2.4000000000000004, 3.2,\n",
       "        4.0], dtype=object),\n",
       " <a list of 10 Patch objects>)"
      ]
     },
     "execution_count": 216,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAASEklEQVR4nO3df6hf913H8efLZHa/LGvpbY25wRslzKV17sclVgcy7LTRjqYKhQznghaCI+r8xZZYsIgEIpNNh3YStrqM1ZYwNxpWui1mG0XoVm+7dm2axUZbm7tmzdWh6xSi6d7+8T2V726/+fH9fm/u97af5wMu33Pe53POed8093VPz6+kqpAkteH7Jt2AJGn5GPqS1BBDX5IaYuhLUkMMfUlqyOpJN3Aul112Wc3MzEy6DUl6UXnggQf+raqmFtdXfOjPzMwwNzc36TYk6UUlyb8Oqp/z9E6S25KcTPLogGV/kKSSXNZX25XkWJKjSa7tq785ySPdsg8lyajfjCRpNOdzTv9jwObFxSTrgJ8DnuqrbQS2Ald269yaZFW3+MPAdmBD9/WCbUqSLqxzhn5V3Qt8a8CiDwLvBfof6d0C3FlVp6rqCeAYsCnJGuDiqrqveo8Afxy4YezuJUlDGenunSTXA9+oqocXLVoLHO+bn+9qa7vpxfUzbX97krkkcwsLC6O0KEkaYOjQT/JK4GbgjwYtHlCrs9QHqqq9VTVbVbNTUy+4+CxJGtEod+/8KLAeeLi7FjsNPJhkE70j+HV9Y6eBp7v69IC6JGkZDX2kX1WPVNXlVTVTVTP0Av1NVfVN4ACwNclFSdbTu2B7f1WdAJ5NcnV31867gLuW7tuQJJ2P87ll8w7gPuC1SeaT3HSmsVV1GNgPPAZ8FthRVc91i98NfITexd1/Bu4Zs3dJ0pCy0t+nPzs7Wz6cJUnDSfJAVc0urq/4J3KllWpm590T2/eTe66b2L714uYL1ySpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1JBzhn6S25KcTPJoX+39Sb6e5GtJPp3kNX3LdiU5luRokmv76m9O8ki37ENJsvTfjiTpbM7nSP9jwOZFtYPAVVX1euCfgF0ASTYCW4Eru3VuTbKqW+fDwHZgQ/e1eJuSpAvsnKFfVfcC31pU+3xVne5mvwxMd9NbgDur6lRVPQEcAzYlWQNcXFX3VVUBHwduWKpvQpJ0fpbinP6vA/d002uB433L5rva2m56cX2gJNuTzCWZW1hYWIIWJUkwZugnuRk4Ddz+fGnAsDpLfaCq2ltVs1U1OzU1NU6LkqQ+q0ddMck24O3ANd0pG+gdwa/rGzYNPN3VpwfUJUnLaKQj/SSbgfcB11fVf/ctOgBsTXJRkvX0LtjeX1UngGeTXN3dtfMu4K4xe5ckDemcR/pJ7gDeClyWZB64hd7dOhcBB7s7L79cVb9RVYeT7Aceo3faZ0dVPddt6t307gR6Bb1rAPcgSVpW5wz9qnrHgPJHzzJ+N7B7QH0OuGqo7iRJS8onciWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkPOGfpJbktyMsmjfbVLkxxM8nj3eUnfsl1JjiU5muTavvqbkzzSLftQkiz9tyNJOpvzOdL/GLB5UW0ncKiqNgCHunmSbAS2Ald269yaZFW3zoeB7cCG7mvxNiVJF9g5Q7+q7gW+tai8BdjXTe8Dbuir31lVp6rqCeAYsCnJGuDiqrqvqgr4eN86kqRlMuo5/Suq6gRA93l5V18LHO8bN9/V1nbTi+sDJdmeZC7J3MLCwogtSpIWW+oLuYPO09dZ6gNV1d6qmq2q2ampqSVrTpJaN2roP9OdsqH7PNnV54F1feOmgae7+vSAuiRpGY0a+geAbd30NuCuvvrWJBclWU/vgu393SmgZ5Nc3d21866+dSRJy2T1uQYkuQN4K3BZknngFmAPsD/JTcBTwI0AVXU4yX7gMeA0sKOqnus29W56dwK9Arin+5IkLaNzhn5VveMMi645w/jdwO4B9TngqqG6kyQtKZ/IlaSGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWrIWKGf5HeTHE7yaJI7krw8yaVJDiZ5vPu8pG/8riTHkhxNcu347UuShjFy6CdZC/w2MFtVVwGrgK3ATuBQVW0ADnXzJNnYLb8S2AzcmmTVeO1LkoYx7umd1cArkqwGXgk8DWwB9nXL9wE3dNNbgDur6lRVPQEcAzaNuX9J0hBGDv2q+gbwZ8BTwAngP6vq88AVVXWiG3MCuLxbZS1wvG8T813tBZJsTzKXZG5hYWHUFiVJi4xzeucSekfv64EfAl6V5J1nW2VArQYNrKq9VTVbVbNTU1OjtihJWmSc0ztvA56oqoWq+l/gU8BPA88kWQPQfZ7sxs8D6/rWn6Z3OkiStEzGCf2ngKuTvDJJgGuAI8ABYFs3ZhtwVzd9ANia5KIk64ENwP1j7F+SNKTVo65YVV9J8kngQeA08FVgL/BqYH+Sm+j9YrixG384yX7gsW78jqp6bsz+JUlDGDn0AarqFuCWReVT9I76B43fDeweZ5+SpNH5RK4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIaM9c8lSs+b2Xn3xPb95J7rJrZv6cXGI31JaoihL0kNGSv0k7wmySeTfD3JkSQ/leTSJAeTPN59XtI3fleSY0mOJrl2/PYlScMY90j/L4DPVtWPAT8BHAF2AoeqagNwqJsnyUZgK3AlsBm4NcmqMfcvSRrCyKGf5GLgZ4CPAlTV/1TVfwBbgH3dsH3ADd30FuDOqjpVVU8Ax4BNo+5fkjS8cY70fwRYAP4myVeTfCTJq4ArquoEQPd5eTd+LXC8b/35riZJWibjhP5q4E3Ah6vqjcB/0Z3KOYMMqNXAgcn2JHNJ5hYWFsZoUZLUb5zQnwfmq+or3fwn6f0SeCbJGoDu82Tf+HV9608DTw/acFXtrarZqpqdmpoao0VJUr+RQ7+qvgkcT/LarnQN8BhwANjW1bYBd3XTB4CtSS5Ksh7YANw/6v4lScMb94nc3wJuT/L9wL8Av0bvF8n+JDcBTwE3AlTV4ST76f1iOA3sqKrnxty/JGkIY4V+VT0EzA5YdM0Zxu8Gdo+zT0nS6HwiV5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNWTs0E+yKslXk3ymm780ycEkj3efl/SN3ZXkWJKjSa4dd9+SpOEsxZH+e4AjffM7gUNVtQE41M2TZCOwFbgS2AzcmmTVEuxfknSeVo+zcpJp4DpgN/B7XXkL8NZueh/wJeB9Xf3OqjoFPJHkGLAJuG+cHvS9ZnbePekWJK1g4x7p/znwXuC7fbUrquoEQPd5eVdfCxzvGzff1V4gyfYkc0nmFhYWxmxRkvS8kUM/yduBk1X1wPmuMqBWgwZW1d6qmq2q2ampqVFblCQtMs7pnbcA1yf5ReDlwMVJPgE8k2RNVZ1IsgY42Y2fB9b1rT8NPD3G/iVJQxr5SL+qdlXVdFXN0LtA+4WqeidwANjWDdsG3NVNHwC2JrkoyXpgA3D/yJ1LkoY21oXcM9gD7E9yE/AUcCNAVR1Osh94DDgN7Kiq5y7A/iVJZ7AkoV9VX6J3lw5V9e/ANWcYt5venT6SpAnwiVxJasiFOL0j6QKb1PMYT+65biL71dLxSF+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqyMihn2Rdki8mOZLkcJL3dPVLkxxM8nj3eUnfOruSHEtyNMm1S/ENSJLO3zhH+qeB36+q1wFXAzuSbAR2AoeqagNwqJunW7YVuBLYDNyaZNU4zUuShjNy6FfViap6sJt+FjgCrAW2APu6YfuAG7rpLcCdVXWqqp4AjgGbRt2/JGl4S3JOP8kM8EbgK8AVVXUCer8YgMu7YWuB432rzXe1QdvbnmQuydzCwsJStChJYglCP8mrgb8Dfqeqvn22oQNqNWhgVe2tqtmqmp2amhq3RUlSZ6zQT/IyeoF/e1V9qis/k2RNt3wNcLKrzwPr+lafBp4eZ/+SpOGMc/dOgI8CR6rqA32LDgDbuultwF199a1JLkqyHtgA3D/q/iVJw1s9xrpvAX4VeCTJQ13tD4E9wP4kNwFPATcCVNXhJPuBx+jd+bOjqp4bY/+SpCGNHPpV9Q8MPk8PcM0Z1tkN7B51n5Ima2bn3RPZ75N7rpvIfl+KfCJXkhpi6EtSQwx9SWrIOBdydQaTOu8pSefikb4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYse+gn2ZzkaJJjSXYu9/4lqWXLGvpJVgF/BfwCsBF4R5KNy9mDJLVsuf9h9E3Asar6F4AkdwJbgMcuxM78B8qll4ZJ/iw/uee6ie37Qlju0F8LHO+bnwd+cvGgJNuB7d3sd5IcHXF/lwH/NuK6F5J9DeesfeVPl7GT7/Wi/POaoBdlXy/iv18/PKi43KGfAbV6QaFqL7B37J0lc1U1O+52lpp9Dce+hmNfw2mtr+W+kDsPrOubnwaeXuYeJKlZyx36/whsSLI+yfcDW4EDy9yDJDVrWU/vVNXpJL8JfA5YBdxWVYcv4C7HPkV0gdjXcOxrOPY1nKb6StULTqlLkl6ifCJXkhpi6EtSQ5oJ/SR/kKSSXDbpXgCS/EmSryV5KMnnk/zQpHsCSPL+JF/vevt0ktdMuieAJDcmOZzku0kmfnvdSnydSJLbkpxM8uike+mXZF2SLyY50v03fM+kewJI8vIk9yd5uOvrjyfdU78kq5J8NclnlnK7TYR+knXAzwFPTbqXPu+vqtdX1RuAzwB/NOmGOgeBq6rq9cA/Absm3M/zHgV+Gbh30o2s4NeJfAzYPOkmBjgN/H5VvQ64GtixQv68TgE/W1U/AbwB2Jzk6gn31O89wJGl3mgToQ98EHgvAx4Em5Sq+nbf7KtYIb1V1eer6nQ3+2V6z1JMXFUdqapRn8xeav//OpGq+h/g+deJTFRV3Qt8a9J9LFZVJ6rqwW76WXpBtnayXUH1fKebfVn3tSJ+DpNMA9cBH1nqbb/kQz/J9cA3qurhSfeyWJLdSY4Dv8LKOdLv9+vAPZNuYgUa9DqRiYfYi0GSGeCNwFcm20lPdwrlIeAkcLCqVkRfwJ/TO1D97lJveLlfw3BBJPl74AcHLLoZ+EPg55e3o56z9VVVd1XVzcDNSXYBvwncshL66sbcTO9/y29fjp7Ot68V4rxeJ6LvleTVwN8Bv7Po/3QnpqqeA97QXbv6dJKrqmqi10SSvB04WVUPJHnrUm//JRH6VfW2QfUkPw6sBx5OAr1TFQ8m2VRV35xUXwP8LXA3yxT65+oryTbg7cA1tYwPcgzx5zVpvk5kSEleRi/wb6+qT026n8Wq6j+SfIneNZFJXwh/C3B9kl8EXg5cnOQTVfXOpdj4S/r0TlU9UlWXV9VMVc3Q+2F903IE/rkk2dA3ez3w9Un10i/JZuB9wPVV9d+T7meF8nUiQ0jviOujwJGq+sCk+3lekqnn705L8grgbayAn8Oq2lVV011mbQW+sFSBDy/x0F/h9iR5NMnX6J1+WhG3sQF/CfwAcLC7nfSvJ90QQJJfSjIP/BRwd5LPTaqX7kL3868TOQLsv8CvEzkvSe4A7gNem2Q+yU2T7qnzFuBXgZ/t/k491B3FTtoa4Ivdz+A/0junv6S3R65EvoZBkhrikb4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ35P2zyAJ41Mcg3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#grouping the dataset by party to get the average estimates of vocabulary for different parties\n",
    "sarcasm_df = df[df['Sarcasm_label'] == 1]\n",
    "plt.hist(sarcasm_df['Sentiment_Scores'])\n",
    "# plt.title('Avg vocabulary in different parties')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the sentiment analysis for overall dataset, hate only and sarcasm only, we can see that negative trend for overall and the sarcasm only is close enough, this could be because there is a lot of sarcasm in our memes(mostly abusive). Sentistrength has given us better insights than other tools like textblob or NLTK vader because sentistrength has been trained on short texts like tweets & internet slang"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 326,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('New_df.csv')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}