Using the Twitter API for Tweet Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Twitter data\n",
    "\n",
    "## Copyright and Licensing\n",
    "\n",
    "You are free to use or adapt this notebook for any purpose you'd like. However, please respect the [Simplified BSD License](https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition/blob/master/LICENSE.txt) that governs its use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Twitter API Access\n",
    "\n",
    "Twitter implements OAuth 1.0A as its standard authentication mechanism, and in order to use it to make requests to Twitter's API, you'll need to go to https://dev.twitter.com/apps and create a sample application.\n",
    "\n",
    "Choose any name for your application, write a description and use `http://google.com` for the website.\n",
    "\n",
    "Under **Key and Access Tokens**, there are four primary identifiers you'll need to note for an OAuth 1.0A workflow: \n",
    "* consumer key, \n",
    "* consumer secret, \n",
    "* access token, and \n",
    "* access token secret (Click on Create Access Token to create those).\n",
    "\n",
    "Note that you will need an ordinary Twitter account in order to login, create an app, and get these credentials."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first time you execute the notebook, add all credentials so that you can save them in the `pkl` file, then you can remove the secret keys from the notebook because they will just be loaded from the `pkl` file.\n",
    "\n",
    "The `pkl` file contains sensitive information that can be used to take control of your twitter acccount, **do not share it**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if not os.path.exists('secret_twitter_credentials.pkl'):\n",
    "    Twitter={}\n",
    "    Twitter['Consumer Key'] = ''\n",
    "    Twitter['Consumer Secret'] = ''\n",
    "    Twitter['Access Token'] = ''\n",
    "    Twitter['Access Token Secret'] = ''\n",
    "    with open('secret_twitter_credentials.pkl','wb') as f:\n",
    "        pickle.dump(Twitter, f)\n",
    "else:\n",
    "    Twitter=pickle.load(open('secret_twitter_credentials.pkl','rb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install the `twitter` package to interface with the Twitter API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: twitter in /Users/altintas/anaconda3/lib/python3.6/site-packages\r\n"
     ]
    }
   ],
   "source": [
    "!pip install twitter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1. Authorizing an application to access Twitter account data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<twitter.api.Twitter object at 0x10f581b00>\n"
     ]
    }
   ],
   "source": [
    "import twitter\n",
    "\n",
    "auth = twitter.oauth.OAuth(Twitter['Access Token'],\n",
    "                           Twitter['Access Token Secret'],\n",
    "                           Twitter['Consumer Key'],\n",
    "                           Twitter['Consumer Secret'])\n",
    "\n",
    "twitter_api = twitter.Twitter(auth=auth)\n",
    "\n",
    "# Nothing to see by displaying twitter_api except that it's now a\n",
    "# defined variable\n",
    "\n",
    "print(twitter_api)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2. Retrieving trends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Twitter identifies locations using the Yahoo! Where On Earth ID.\n",
    "\n",
    "The Yahoo! Where On Earth ID for the entire world is 1.\n",
    "See https://dev.twitter.com/docs/api/1.1/get/trends/place and\n",
    "http://developer.yahoo.com/geo/geoplanet/\n",
    "\n",
    "look at the BOSS placefinder here: https://developer.yahoo.com/boss/placefinder/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "WORLD_WOE_ID = 1\n",
    "US_WOE_ID = 23424977"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look for the WOEID for [san-diego](http://woeid.rosselliot.co.nz/lookup/san%20diego%20%20ca)\n",
    "\n",
    "You can change it to another location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "LOCAL_WOE_ID=2487889\n",
    "\n",
    "# Prefix ID with the underscore for query string parameterization.\n",
    "# Without the underscore, the twitter package appends the ID value\n",
    "# to the URL itself as a special case keyword argument.\n",
    "\n",
    "world_trends = twitter_api.trends.place(_id=WORLD_WOE_ID)\n",
    "us_trends = twitter_api.trends.place(_id=US_WOE_ID)\n",
    "local_trends = twitter_api.trends.place(_id=LOCAL_WOE_ID)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'as_of': '2017-05-19T10:24:54Z',\n",
       "  'created_at': '2017-05-19T10:22:19Z',\n",
       "  'locations': [{'name': 'Worldwide', 'woeid': 1}],\n",
       "  'trends': [{'name': '#CryingInTheClubMusicVideo',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23CryingInTheClubMusicVideo',\n",
       "    'tweet_volume': 15923,\n",
       "    'url': 'http://twitter.com/search?q=%23CryingInTheClubMusicVideo'},\n",
       "   {'name': 'Julian Assange',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Julian+Assange%22',\n",
       "    'tweet_volume': 33930,\n",
       "    'url': 'http://twitter.com/search?q=%22Julian+Assange%22'},\n",
       "   {'name': '#19Mayıs1919',\n",
       "    'promoted_content': None,\n",
       "    'query': '%2319May%C4%B1s1919',\n",
       "    'tweet_volume': 176154,\n",
       "    'url': 'http://twitter.com/search?q=%2319May%C4%B1s1919'},\n",
       "   {'name': '성소수자',\n",
       "    'promoted_content': None,\n",
       "    'query': '%EC%84%B1%EC%86%8C%EC%88%98%EC%9E%90',\n",
       "    'tweet_volume': 64679,\n",
       "    'url': 'http://twitter.com/search?q=%EC%84%B1%EC%86%8C%EC%88%98%EC%9E%90'},\n",
       "   {'name': '#StripThatDown',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23StripThatDown',\n",
       "    'tweet_volume': 256116,\n",
       "    'url': 'http://twitter.com/search?q=%23StripThatDown'},\n",
       "   {'name': '#SözcüSusarsaTürkiyeSusar',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23S%C3%B6zc%C3%BCSusarsaT%C3%BCrkiyeSusar',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23S%C3%B6zc%C3%BCSusarsaT%C3%BCrkiyeSusar'},\n",
       "   {'name': '#死亡から予測変換で生還めざす',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%E6%AD%BB%E4%BA%A1%E3%81%8B%E3%82%89%E4%BA%88%E6%B8%AC%E5%A4%89%E6%8F%9B%E3%81%A7%E7%94%9F%E9%82%84%E3%82%81%E3%81%96%E3%81%99',\n",
       "    'tweet_volume': 20005,\n",
       "    'url': 'http://twitter.com/search?q=%23%E6%AD%BB%E4%BA%A1%E3%81%8B%E3%82%89%E4%BA%88%E6%B8%AC%E5%A4%89%E6%8F%9B%E3%81%A7%E7%94%9F%E9%82%84%E3%82%81%E3%81%96%E3%81%99'},\n",
       "   {'name': 'Hayırlı Cumalar',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Hay%C4%B1rl%C4%B1+Cumalar%22',\n",
       "    'tweet_volume': 41180,\n",
       "    'url': 'http://twitter.com/search?q=%22Hay%C4%B1rl%C4%B1+Cumalar%22'},\n",
       "   {'name': '宮崎駿監督',\n",
       "    'promoted_content': None,\n",
       "    'query': '%E5%AE%AE%E5%B4%8E%E9%A7%BF%E7%9B%A3%E7%9D%A3',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%E5%AE%AE%E5%B4%8E%E9%A7%BF%E7%9B%A3%E7%9D%A3'},\n",
       "   {'name': '포비아',\n",
       "    'promoted_content': None,\n",
       "    'query': '%ED%8F%AC%EB%B9%84%EC%95%84',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%ED%8F%AC%EB%B9%84%EC%95%84'},\n",
       "   {'name': 'FGO舞台',\n",
       "    'promoted_content': None,\n",
       "    'query': 'FGO%E8%88%9E%E5%8F%B0',\n",
       "    'tweet_volume': 13615,\n",
       "    'url': 'http://twitter.com/search?q=FGO%E8%88%9E%E5%8F%B0'},\n",
       "   {'name': 'Spor Bayramı',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Spor+Bayram%C4%B1%22',\n",
       "    'tweet_volume': 66872,\n",
       "    'url': 'http://twitter.com/search?q=%22Spor+Bayram%C4%B1%22'},\n",
       "   {'name': '꿈돌이',\n",
       "    'promoted_content': None,\n",
       "    'query': '%EA%BF%88%EB%8F%8C%EC%9D%B4',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%EA%BF%88%EB%8F%8C%EC%9D%B4'},\n",
       "   {'name': 'Hanouna',\n",
       "    'promoted_content': None,\n",
       "    'query': 'Hanouna',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=Hanouna'},\n",
       "   {'name': 'Stanley Greene',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Stanley+Greene%22',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%22Stanley+Greene%22'},\n",
       "   {'name': 'Lopetegui',\n",
       "    'promoted_content': None,\n",
       "    'query': 'Lopetegui',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=Lopetegui'},\n",
       "   {'name': '#الجمعه',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%A7%D9%84%D8%AC%D9%85%D8%B9%D9%87',\n",
       "    'tweet_volume': 82996,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%A7%D9%84%D8%AC%D9%85%D8%B9%D9%87'},\n",
       "   {'name': '#SWISHSWISH',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23SWISHSWISH',\n",
       "    'tweet_volume': 142037,\n",
       "    'url': 'http://twitter.com/search?q=%23SWISHSWISH'},\n",
       "   {'name': '#شتسوي_وقت_العصبيه',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%B4%D8%AA%D8%B3%D9%88%D9%8A_%D9%88%D9%82%D8%AA_%D8%A7%D9%84%D8%B9%D8%B5%D8%A8%D9%8A%D9%87',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%B4%D8%AA%D8%B3%D9%88%D9%8A_%D9%88%D9%82%D8%AA_%D8%A7%D9%84%D8%B9%D8%B5%D8%A8%D9%8A%D9%87'},\n",
       "   {'name': '#JaiLavakusaFirstLook',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23JaiLavakusaFirstLook',\n",
       "    'tweet_volume': 67937,\n",
       "    'url': 'http://twitter.com/search?q=%23JaiLavakusaFirstLook'},\n",
       "   {'name': '#FelizFinde',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23FelizFinde',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23FelizFinde'},\n",
       "   {'name': '#صلوا_علي_النبي',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%B5%D9%84%D9%88%D8%A7_%D8%B9%D9%84%D9%8A_%D8%A7%D9%84%D9%86%D8%A8%D9%8A',\n",
       "    'tweet_volume': 24233,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%B5%D9%84%D9%88%D8%A7_%D8%B9%D9%84%D9%8A_%D8%A7%D9%84%D9%86%D8%A8%D9%8A'},\n",
       "   {'name': '#나는어떤맛',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%EB%82%98%EB%8A%94%EC%96%B4%EB%96%A4%EB%A7%9B',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%EB%82%98%EB%8A%94%EC%96%B4%EB%96%A4%EB%A7%9B'},\n",
       "   {'name': '#言うだけタダだから欲しいアンソロ呟く',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%E8%A8%80%E3%81%86%E3%81%A0%E3%81%91%E3%82%BF%E3%83%80%E3%81%A0%E3%81%8B%E3%82%89%E6%AC%B2%E3%81%97%E3%81%84%E3%82%A2%E3%83%B3%E3%82%BD%E3%83%AD%E5%91%9F%E3%81%8F',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%E8%A8%80%E3%81%86%E3%81%A0%E3%81%91%E3%82%BF%E3%83%80%E3%81%A0%E3%81%8B%E3%82%89%E6%AC%B2%E3%81%97%E3%81%84%E3%82%A2%E3%83%B3%E3%82%BD%E3%83%AD%E5%91%9F%E3%81%8F'},\n",
       "   {'name': '#AFLCatsDogs',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23AFLCatsDogs',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23AFLCatsDogs'},\n",
       "   {'name': '#フォロワーが体験した事が無さそうな体験',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%E3%83%95%E3%82%A9%E3%83%AD%E3%83%AF%E3%83%BC%E3%81%8C%E4%BD%93%E9%A8%93%E3%81%97%E3%81%9F%E4%BA%8B%E3%81%8C%E7%84%A1%E3%81%95%E3%81%9D%E3%81%86%E3%81%AA%E4%BD%93%E9%A8%93',\n",
       "    'tweet_volume': 59674,\n",
       "    'url': 'http://twitter.com/search?q=%23%E3%83%95%E3%82%A9%E3%83%AD%E3%83%AF%E3%83%BC%E3%81%8C%E4%BD%93%E9%A8%93%E3%81%97%E3%81%9F%E4%BA%8B%E3%81%8C%E7%84%A1%E3%81%95%E3%81%9D%E3%81%86%E3%81%AA%E4%BD%93%E9%A8%93'},\n",
       "   {'name': '#Okja',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23Okja',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23Okja'},\n",
       "   {'name': '#BuenViernes',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23BuenViernes',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23BuenViernes'},\n",
       "   {'name': '#VendrediLecture',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23VendrediLecture',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23VendrediLecture'},\n",
       "   {'name': '#27MTomaMadrid',\n",
       "    'promoted_content': None,\n",
       "    'query': '%2327MTomaMadrid',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%2327MTomaMadrid'},\n",
       "   {'name': '#ReuniondeAgencias',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23ReuniondeAgencias',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23ReuniondeAgencias'},\n",
       "   {'name': '#جمعه_مباركه',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%AC%D9%85%D8%B9%D9%87_%D9%85%D8%A8%D8%A7%D8%B1%D9%83%D9%87',\n",
       "    'tweet_volume': 10275,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%AC%D9%85%D8%B9%D9%87_%D9%85%D8%A8%D8%A7%D8%B1%D9%83%D9%87'},\n",
       "   {'name': '#NRLWarriorsDragons',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23NRLWarriorsDragons',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23NRLWarriorsDragons'},\n",
       "   {'name': '#내가_동물_이었다면',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%EB%82%B4%EA%B0%80_%EB%8F%99%EB%AC%BC_%EC%9D%B4%EC%97%88%EB%8B%A4%EB%A9%B4',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%EB%82%B4%EA%B0%80_%EB%8F%99%EB%AC%BC_%EC%9D%B4%EC%97%88%EB%8B%A4%EB%A9%B4'},\n",
       "   {'name': '#FrenchGP',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23FrenchGP',\n",
       "    'tweet_volume': 10910,\n",
       "    'url': 'http://twitter.com/search?q=%23FrenchGP'},\n",
       "   {'name': '#اكثر_شي_يروقني',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%A7%D9%83%D8%AB%D8%B1_%D8%B4%D9%8A_%D9%8A%D8%B1%D9%88%D9%82%D9%86%D9%8A',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%A7%D9%83%D8%AB%D8%B1_%D8%B4%D9%8A_%D9%8A%D8%B1%D9%88%D9%82%D9%86%D9%8A'},\n",
       "   {'name': '#CarpoolHarryoke',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23CarpoolHarryoke',\n",
       "    'tweet_volume': 63895,\n",
       "    'url': 'http://twitter.com/search?q=%23CarpoolHarryoke'},\n",
       "   {'name': '#NRLBroncosTigers',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23NRLBroncosTigers',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23NRLBroncosTigers'},\n",
       "   {'name': '#19maggio',\n",
       "    'promoted_content': None,\n",
       "    'query': '%2319maggio',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%2319maggio'},\n",
       "   {'name': '#CHIvCRU',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23CHIvCRU',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23CHIvCRU'},\n",
       "   {'name': '#culturegeek',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23culturegeek',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23culturegeek'},\n",
       "   {'name': '#DTBYWalangIwanan',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23DTBYWalangIwanan',\n",
       "    'tweet_volume': 67258,\n",
       "    'url': 'http://twitter.com/search?q=%23DTBYWalangIwanan'},\n",
       "   {'name': '#ONADublin',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23ONADublin',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23ONADublin'},\n",
       "   {'name': '#الملكي_بطلا_لكاس_الملك',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%A7%D9%84%D9%85%D9%84%D9%83%D9%8A_%D8%A8%D8%B7%D9%84%D8%A7_%D9%84%D9%83%D8%A7%D8%B3_%D8%A7%D9%84%D9%85%D9%84%D9%83',\n",
       "    'tweet_volume': 19786,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%A7%D9%84%D9%85%D9%84%D9%83%D9%8A_%D8%A8%D8%B7%D9%84%D8%A7_%D9%84%D9%83%D8%A7%D8%B3_%D8%A7%D9%84%D9%85%D9%84%D9%83'},\n",
       "   {'name': '#EngagePrague',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23EngagePrague',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23EngagePrague'},\n",
       "   {'name': '#FeteDesVoisins',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23FeteDesVoisins',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23FeteDesVoisins'},\n",
       "   {'name': '#NoEnCibeles',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23NoEnCibeles',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23NoEnCibeles'},\n",
       "   {'name': '#KathrynAvonIkawNa',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23KathrynAvonIkawNa',\n",
       "    'tweet_volume': 32110,\n",
       "    'url': 'http://twitter.com/search?q=%23KathrynAvonIkawNa'},\n",
       "   {'name': '#friyay',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23friyay',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23friyay'},\n",
       "   {'name': '#MociónParaEcharlos',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23Moci%C3%B3nParaEcharlos',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23Moci%C3%B3nParaEcharlos'}]}]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "world_trends[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'twitter.api.TwitterListResponse'>\n",
      "['trends', 'as_of', 'created_at', 'locations']\n",
      "[{'name': '#StripThatDown', 'url': 'http://twitter.com/search?q=%23StripThatDown', 'promoted_content': None, 'query': '%23StripThatDown', 'tweet_volume': 254535}, {'name': '#Scandal', 'url': 'http://twitter.com/search?q=%23Scandal', 'promoted_content': None, 'query': '%23Scandal', 'tweet_volume': 122522}, {'name': '#10NewsAt11', 'url': 'http://twitter.com/search?q=%2310NewsAt11', 'promoted_content': None, 'query': '%2310NewsAt11', 'tweet_volume': None}, {'name': 'Chris Cornell', 'url': 'http://twitter.com/search?q=%22Chris+Cornell%22', 'promoted_content': None, 'query': '%22Chris+Cornell%22', 'tweet_volume': 806757}, {'name': 'Roger Ailes', 'url': 'http://twitter.com/search?q=%22Roger+Ailes%22', 'promoted_content': None, 'query': '%22Roger+Ailes%22', 'tweet_volume': 334331}, {'name': 'Times Square', 'url': 'http://twitter.com/search?q=%22Times+Square%22', 'promoted_content': None, 'query': '%22Times+Square%22', 'tweet_volume': 353784}, {'name': '#BikeToWorkDay', 'url': 'http://twitter.com/search?q=%23BikeToWorkDay', 'promoted_content': None, 'query': '%23BikeToWorkDay', 'tweet_volume': None}, {'name': 'Julian Assange', 'url': 'http://twitter.com/search?q=%22Julian+Assange%22', 'promoted_content': None, 'query': '%22Julian+Assange%22', 'tweet_volume': 32574}, {'name': '#CryingInTheClubMusicVideo', 'url': 'http://twitter.com/search?q=%23CryingInTheClubMusicVideo', 'promoted_content': None, 'query': '%23CryingInTheClubMusicVideo', 'tweet_volume': 11351}, {'name': 'OKJA', 'url': 'http://twitter.com/search?q=OKJA', 'promoted_content': None, 'query': 'OKJA', 'tweet_volume': 23218}, {'name': \"We're LIVE ON-AIR\", 'url': 'http://twitter.com/search?q=%22We%27re+LIVE+ON-AIR%22', 'promoted_content': None, 'query': '%22We%27re+LIVE+ON-AIR%22', 'tweet_volume': None}, {'name': 'Alien Covenant', 'url': 'http://twitter.com/search?q=%22Alien+Covenant%22', 'promoted_content': None, 'query': '%22Alien+Covenant%22', 'tweet_volume': 16008}, {'name': 'Residente', 'url': 'http://twitter.com/search?q=Residente', 'promoted_content': None, 'query': 'Residente', 'tweet_volume': 40870}, {'name': 'Farmington Rd', 'url': 'http://twitter.com/search?q=%22Farmington+Rd%22', 'promoted_content': None, 'query': '%22Farmington+Rd%22', 'tweet_volume': None}, {'name': 'Car Boys', 'url': 'http://twitter.com/search?q=%22Car+Boys%22', 'promoted_content': None, 'query': '%22Car+Boys%22', 'tweet_volume': None}, {'name': 'Prince William Pkwy', 'url': 'http://twitter.com/search?q=%22Prince+William+Pkwy%22', 'promoted_content': None, 'query': '%22Prince+William+Pkwy%22', 'tweet_volume': None}, {'name': 'Scott Deem', 'url': 'http://twitter.com/search?q=%22Scott+Deem%22', 'promoted_content': None, 'query': '%22Scott+Deem%22', 'tweet_volume': None}, {'name': '$110.5 Million', 'url': 'http://twitter.com/search?q=%22%24110.5+Million%22', 'promoted_content': None, 'query': '%22%24110.5+Million%22', 'tweet_volume': 11276}, {'name': 'Lee Circle', 'url': 'http://twitter.com/search?q=%22Lee+Circle%22', 'promoted_content': None, 'query': '%22Lee+Circle%22', 'tweet_volume': None}, {'name': 'Ruby Rose', 'url': 'http://twitter.com/search?q=%22Ruby+Rose%22', 'promoted_content': None, 'query': '%22Ruby+Rose%22', 'tweet_volume': None}, {'name': 'YOU KNOW I USED TO BE IN 1D', 'url': 'http://twitter.com/search?q=%22YOU+KNOW+I+USED+TO+BE+IN+1D%22', 'promoted_content': None, 'query': '%22YOU+KNOW+I+USED+TO+BE+IN+1D%22', 'tweet_volume': None}, {'name': 'Ivanka and Jared', 'url': 'http://twitter.com/search?q=%22Ivanka+and+Jared%22', 'promoted_content': None, 'query': '%22Ivanka+and+Jared%22', 'tweet_volume': None}, {'name': 'Destiny 2', 'url': 'http://twitter.com/search?q=%22Destiny+2%22', 'promoted_content': None, 'query': '%22Destiny+2%22', 'tweet_volume': 131786}, {'name': 'Joe Lieberman', 'url': 'http://twitter.com/search?q=%22Joe+Lieberman%22', 'promoted_content': None, 'query': '%22Joe+Lieberman%22', 'tweet_volume': 119702}, {'name': 'All-NBA', 'url': 'http://twitter.com/search?q=All-NBA', 'promoted_content': None, 'query': 'All-NBA', 'tweet_volume': 114861}, {'name': 'T-Wayne', 'url': 'http://twitter.com/search?q=T-Wayne', 'promoted_content': None, 'query': 'T-Wayne', 'tweet_volume': 35141}, {'name': '#SWISHSWISH', 'url': 'http://twitter.com/search?q=%23SWISHSWISH', 'promoted_content': None, 'query': '%23SWISHSWISH', 'tweet_volume': 141391}, {'name': '#NSFW90sShows', 'url': 'http://twitter.com/search?q=%23NSFW90sShows', 'promoted_content': None, 'query': '%23NSFW90sShows', 'tweet_volume': 22443}, {'name': '#FridayFeeling', 'url': 'http://twitter.com/search?q=%23FridayFeeling', 'promoted_content': None, 'query': '%23FridayFeeling', 'tweet_volume': 32621}, {'name': '#ONADublin', 'url': 'http://twitter.com/search?q=%23ONADublin', 'promoted_content': None, 'query': '%23ONADublin', 'tweet_volume': None}, {'name': '#CarpoolHarryoke', 'url': 'http://twitter.com/search?q=%23CarpoolHarryoke', 'promoted_content': None, 'query': '%23CarpoolHarryoke', 'tweet_volume': 63543}, {'name': '#BlackEyedSusanDay', 'url': 'http://twitter.com/search?q=%23BlackEyedSusanDay', 'promoted_content': None, 'query': '%23BlackEyedSusanDay', 'tweet_volume': None}, {'name': '#EngagePrague', 'url': 'http://twitter.com/search?q=%23EngagePrague', 'promoted_content': None, 'query': '%23EngagePrague', 'tweet_volume': None}, {'name': '#pgday', 'url': 'http://twitter.com/search?q=%23pgday', 'promoted_content': None, 'query': '%23pgday', 'tweet_volume': None}, {'name': '#DEAD', 'url': 'http://twitter.com/search?q=%23DEAD', 'promoted_content': None, 'query': '%23DEAD', 'tweet_volume': 22687}, {'name': '#ChelseaChampions', 'url': 'http://twitter.com/search?q=%23ChelseaChampions', 'promoted_content': None, 'query': '%23ChelseaChampions', 'tweet_volume': 11122}, {'name': '#MalcolmX', 'url': 'http://twitter.com/search?q=%23MalcolmX', 'promoted_content': None, 'query': '%23MalcolmX', 'tweet_volume': None}, {'name': '#IWFLeadChange', 'url': 'http://twitter.com/search?q=%23IWFLeadChange', 'promoted_content': None, 'query': '%23IWFLeadChange', 'tweet_volume': None}, {'name': '#SupernaturaI', 'url': 'http://twitter.com/search?q=%23SupernaturaI', 'promoted_content': None, 'query': '%23SupernaturaI', 'tweet_volume': None}, {'name': '#BreakALittle', 'url': 'http://twitter.com/search?q=%23BreakALittle', 'promoted_content': None, 'query': '%23BreakALittle', 'tweet_volume': None}, {'name': '#springio17', 'url': 'http://twitter.com/search?q=%23springio17', 'promoted_content': None, 'query': '%23springio17', 'tweet_volume': None}, {'name': '#newmusicfriday', 'url': 'http://twitter.com/search?q=%23newmusicfriday', 'promoted_content': None, 'query': '%23newmusicfriday', 'tweet_volume': None}, {'name': '#NevaLeft', 'url': 'http://twitter.com/search?q=%23NevaLeft', 'promoted_content': None, 'query': '%23NevaLeft', 'tweet_volume': None}, {'name': '#ReactEurope', 'url': 'http://twitter.com/search?q=%23ReactEurope', 'promoted_content': None, 'query': '%23ReactEurope', 'tweet_volume': None}, {'name': '#JerseyBoys', 'url': 'http://twitter.com/search?q=%23JerseyBoys', 'promoted_content': None, 'query': '%23JerseyBoys', 'tweet_volume': None}, {'name': '#TheGodBox', 'url': 'http://twitter.com/search?q=%23TheGodBox', 'promoted_content': None, 'query': '%23TheGodBox', 'tweet_volume': None}, {'name': '#TheKingAndI', 'url': 'http://twitter.com/search?q=%23TheKingAndI', 'promoted_content': None, 'query': '%23TheKingAndI', 'tweet_volume': None}, {'name': '#GreysAnatomy', 'url': 'http://twitter.com/search?q=%23GreysAnatomy', 'promoted_content': None, 'query': '%23GreysAnatomy', 'tweet_volume': 100943}, {'name': '#io17', 'url': 'http://twitter.com/search?q=%23io17', 'promoted_content': None, 'query': '%23io17', 'tweet_volume': 26987}, {'name': '#okwx', 'url': 'http://twitter.com/search?q=%23okwx', 'promoted_content': None, 'query': '%23okwx', 'tweet_volume': 15390}]\n"
     ]
    }
   ],
   "source": [
    "trends=local_trends\n",
    "print(type(trends))\n",
    "print(list(trends[0].keys()))\n",
    "print(trends[0]['trends'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3. Displaying API responses as pretty-printed JSON"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      " {\n",
      "  \"trends\": [\n",
      "   {\n",
      "    \"name\": \"Julian Assange\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Julian+Assange%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Julian+Assange%22\",\n",
      "    \"tweet_volume\": 32574\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#CryingInTheClubMusicVideo\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23CryingInTheClubMusicVideo\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23CryingInTheClubMusicVideo\",\n",
      "    \"tweet_volume\": 11351\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#SWISHSWISH\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23SWISHSWISH\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23SWISHSWISH\",\n",
      "    \"tweet_volume\": 141391\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#StripThatDown\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23StripThatDown\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23StripThatDown\",\n",
      "    \"tweet_volume\": 254535\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#NSFW90sShows\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23NSFW90sShows\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23NSFW90sShows\",\n",
      "    \"tweet_volume\": 22443\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#FridayFeeling\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23FridayFeeling\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23FridayFeeling\",\n",
      "    \"tweet_volume\": 32621\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"OKJA\",\n",
      "    \"url\": \"http://twitter.com/search?q=OKJA\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"OKJA\",\n",
      "    \"tweet_volume\": 23218\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"We're LIVE ON-AIR\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22We%27re+LIVE+ON-AIR%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22We%27re+LIVE+ON-AIR%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Alien Covenant\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Alien+Covenant%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Alien+Covenant%22\",\n",
      "    \"tweet_volume\": 16008\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Residente\",\n",
      "    \"url\": \"http://twitter.com/search?q=Residente\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"Residente\",\n",
      "    \"tweet_volume\": 40870\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Farmington Rd\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Farmington+Rd%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Farmington+Rd%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Car Boys\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Car+Boys%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Car+Boys%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Prince William Pkwy\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Prince+William+Pkwy%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Prince+William+Pkwy%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Scott Deem\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Scott+Deem%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Scott+Deem%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"$110.5 Million\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22%24110.5+Million%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22%24110.5+Million%22\",\n",
      "    \"tweet_volume\": 11276\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Lee Circle\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Lee+Circle%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Lee+Circle%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Ruby Rose\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Ruby+Rose%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Ruby+Rose%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Jean-Michel Basquiat\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Jean-Michel+Basquiat%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Jean-Michel+Basquiat%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"YOU KNOW I USED TO BE IN 1D\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22YOU+KNOW+I+USED+TO+BE+IN+1D%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22YOU+KNOW+I+USED+TO+BE+IN+1D%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Ivanka and Jared\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Ivanka+and+Jared%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Ivanka+and+Jared%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Chris Cornell\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Chris+Cornell%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Chris+Cornell%22\",\n",
      "    \"tweet_volume\": 806757\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Times Square\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Times+Square%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Times+Square%22\",\n",
      "    \"tweet_volume\": 353784\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Destiny 2\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Destiny+2%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Destiny+2%22\",\n",
      "    \"tweet_volume\": 131786\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Soundgarden\",\n",
      "    \"url\": \"http://twitter.com/search?q=Soundgarden\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"Soundgarden\",\n",
      "    \"tweet_volume\": 324566\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Joe Lieberman\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Joe+Lieberman%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Joe+Lieberman%22\",\n",
      "    \"tweet_volume\": 119702\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"All-NBA\",\n",
      "    \"url\": \"http://twitter.com/search?q=All-NBA\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"All-NBA\",\n",
      "    \"tweet_volume\": 114861\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"T-Wayne\",\n",
      "    \"url\": \"http://twitter.com/search?q=T-Wayne\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"T-Wayne\",\n",
      "    \"tweet_volume\": 35141\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"Robert Mueller\",\n",
      "    \"url\": \"http://twitter.com/search?q=%22Robert+Mueller%22\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%22Robert+Mueller%22\",\n",
      "    \"tweet_volume\": 83930\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#ONADublin\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23ONADublin\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23ONADublin\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#CarpoolHarryoke\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23CarpoolHarryoke\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23CarpoolHarryoke\",\n",
      "    \"tweet_volume\": 63543\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#BlackEyedSusanDay\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23BlackEyedSusanDay\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23BlackEyedSusanDay\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#EngagePrague\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23EngagePrague\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23EngagePrague\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#pgday\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23pgday\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23pgday\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#DEAD\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23DEAD\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23DEAD\",\n",
      "    \"tweet_volume\": 22687\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#ChelseaChampions\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23ChelseaChampions\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23ChelseaChampions\",\n",
      "    \"tweet_volume\": 11122\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#nationalbiketoworkday\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23nationalbiketoworkday\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23nationalbiketoworkday\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#MalcolmX\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23MalcolmX\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23MalcolmX\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#IWFLeadChange\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23IWFLeadChange\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23IWFLeadChange\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#SupernaturaI\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23SupernaturaI\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23SupernaturaI\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#BreakALittle\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23BreakALittle\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23BreakALittle\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#springio17\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23springio17\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23springio17\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#newmusicfriday\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23newmusicfriday\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23newmusicfriday\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#NevaLeft\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23NevaLeft\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23NevaLeft\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#ReactEurope\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23ReactEurope\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23ReactEurope\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#JerseyBoys\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23JerseyBoys\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23JerseyBoys\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#TheGodBox\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23TheGodBox\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23TheGodBox\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#TheKingAndI\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23TheKingAndI\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23TheKingAndI\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#Scandal\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23Scandal\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23Scandal\",\n",
      "    \"tweet_volume\": 122522\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#GreysAnatomy\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23GreysAnatomy\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23GreysAnatomy\",\n",
      "    \"tweet_volume\": 100943\n",
      "   },\n",
      "   {\n",
      "    \"name\": \"#io17\",\n",
      "    \"url\": \"http://twitter.com/search?q=%23io17\",\n",
      "    \"promoted_content\": null,\n",
      "    \"query\": \"%23io17\",\n",
      "    \"tweet_volume\": 26987\n",
      "   }\n",
      "  ],\n",
      "  \"as_of\": \"2017-05-19T10:24:54Z\",\n",
      "  \"created_at\": \"2017-05-19T10:17:26Z\",\n",
      "  \"locations\": [\n",
      "   {\n",
      "    \"name\": \"United States\",\n",
      "    \"woeid\": 23424977\n",
      "   }\n",
      "  ]\n",
      " }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "print((json.dumps(us_trends[:2], indent=1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 4. Computing the intersection of two sets of trends"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('----------', 'world')\n",
      "#19Mayıs1919,#جمعه_مباركه,#CarpoolHarryoke,#NRLBroncosTigers,#19maggio,#ReuniondeAgencias,#VendrediLecture,#الجمعه,#AFLCatsDogs,Hanouna,#صلوا_علي_النبي,Hayırlı Cumalar,#나는어떤맛,#フォロワーが体験した事が無さそうな体験,#27MTomaMadrid,#friyay,#StripThatDown,FGO舞台,#言うだけタダだから欲しいアンソロ呟く,#SWISHSWISH,꿈돌이,#내가_동물_이었다면,#culturegeek,성소수자,#FeteDesVoisins,#اكثر_شي_يروقني,#JaiLavakusaFirstLook,Julian Assange,#ONADublin,#KathrynAvonIkawNa,#BuenViernes,Lopetegui,#DTBYWalangIwanan,宮崎駿監督,#NRLWarriorsDragons,#Okja,#MociónParaEcharlos,#الملكي_بطلا_لكاس_الملك,Spor Bayramı,#死亡から予測変換で生還めざす,#شتسوي_وقت_العصبيه,#FelizFinde,#EngagePrague,#CHIvCRU,포비아,Stanley Greene,#FrenchGP,#CryingInTheClubMusicVideo,#SözcüSusarsaTürkiyeSusar,#NoEnCibeles\n",
      "('----------', 'us')\n",
      "#TheKingAndI,Destiny 2,#CarpoolHarryoke,All-NBA,#NevaLeft,Jean-Michel Basquiat,Prince William Pkwy,$110.5 Million,OKJA,#IWFLeadChange,YOU KNOW I USED TO BE IN 1D,Times Square,Residente,#pgday,#BreakALittle,Scott Deem,#JerseyBoys,#StripThatDown,Soundgarden,#SWISHSWISH,Chris Cornell,Alien Covenant,#DEAD,#nationalbiketoworkday,#newmusicfriday,#TheGodBox,We're LIVE ON-AIR,Joe Lieberman,#MalcolmX,Julian Assange,#ONADublin,#ReactEurope,#BlackEyedSusanDay,Ivanka and Jared,#io17,Farmington Rd,Lee Circle,Robert Mueller,T-Wayne,#GreysAnatomy,#EngagePrague,#ChelseaChampions,Car Boys,#FridayFeeling,#springio17,#CryingInTheClubMusicVideo,#Scandal,#SupernaturaI,#NSFW90sShows,Ruby Rose\n",
      "('----------', 'san diego')\n",
      "#TheKingAndI,Destiny 2,#CarpoolHarryoke,All-NBA,#NevaLeft,#10NewsAt11,Roger Ailes,Prince William Pkwy,$110.5 Million,OKJA,#IWFLeadChange,YOU KNOW I USED TO BE IN 1D,Times Square,Residente,#pgday,#BreakALittle,#okwx,Scott Deem,#JerseyBoys,#StripThatDown,#SWISHSWISH,Chris Cornell,Alien Covenant,#DEAD,#newmusicfriday,#TheGodBox,We're LIVE ON-AIR,Joe Lieberman,#MalcolmX,Julian Assange,#ONADublin,#ReactEurope,#BlackEyedSusanDay,Ivanka and Jared,#io17,Farmington Rd,Lee Circle,T-Wayne,#GreysAnatomy,#EngagePrague,#ChelseaChampions,Car Boys,#FridayFeeling,#BikeToWorkDay,#CryingInTheClubMusicVideo,#springio17,#Scandal,#SupernaturaI,#NSFW90sShows,Ruby Rose\n",
      "('==========', 'intersection of world and us')\n",
      "{'#CarpoolHarryoke', '#EngagePrague', '#SWISHSWISH', '#StripThatDown', 'Julian Assange', '#ONADublin', '#CryingInTheClubMusicVideo'}\n",
      "('==========', 'intersection of us and san-diego')\n",
      "{'#TheKingAndI', 'Destiny 2', '#CarpoolHarryoke', 'All-NBA', '#NevaLeft', 'Prince William Pkwy', '$110.5 Million', 'OKJA', '#IWFLeadChange', 'YOU KNOW I USED TO BE IN 1D', 'Times Square', 'Residente', '#pgday', '#BreakALittle', 'Scott Deem', '#JerseyBoys', '#StripThatDown', '#SWISHSWISH', 'Chris Cornell', 'Alien Covenant', '#DEAD', '#newmusicfriday', '#TheGodBox', \"We're LIVE ON-AIR\", 'Joe Lieberman', '#MalcolmX', 'Julian Assange', '#ReactEurope', '#BlackEyedSusanDay', 'Ivanka and Jared', '#io17', '#CryingInTheClubMusicVideo', 'Farmington Rd', 'Lee Circle', 'T-Wayne', '#GreysAnatomy', '#EngagePrague', '#ChelseaChampions', 'Car Boys', '#FridayFeeling', '#springio17', '#ONADublin', '#Scandal', '#SupernaturaI', '#NSFW90sShows', 'Ruby Rose'}\n"
     ]
    }
   ],
   "source": [
    "trends_set = {}\n",
    "trends_set['world'] = set([trend['name'] \n",
    "                        for trend in world_trends[0]['trends']])\n",
    "\n",
    "trends_set['us'] = set([trend['name'] \n",
    "                     for trend in us_trends[0]['trends']]) \n",
    "\n",
    "trends_set['san diego'] = set([trend['name'] \n",
    "                     for trend in local_trends[0]['trends']]) \n",
    "\n",
    "for loc in ['world','us','san diego']:\n",
    "    print(('-'*10,loc))\n",
    "    print((','.join(trends_set[loc])))\n",
    "\n",
    "print(( '='*10,'intersection of world and us'))\n",
    "print((trends_set['world'].intersection(trends_set['us'])))\n",
    "\n",
    "print(('='*10,'intersection of us and san-diego'))\n",
    "print((trends_set['san diego'].intersection(trends_set['us'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5. Collecting search results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the variable `q` to a trending topic, \n",
    "or anything else for that matter. The example query below\n",
    "was a trending topic when this content was being developed\n",
    "and is used throughout the remainder of this chapter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "q = '#MTVAwards' \n",
    "\n",
    "number = 100\n",
    "\n",
    "# See https://dev.twitter.com/docs/api/1.1/get/search/tweets\n",
    "\n",
    "search_results = twitter_api.search.tweets(q=q, count=number)\n",
    "\n",
    "statuses = search_results['statuses']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(statuses)\n",
    "#print(statuses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Twitter often returns duplicate results, we can filter them out checking for duplicate texts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "all_text = []\n",
    "filtered_statuses = []\n",
    "for s in statuses:\n",
    "    if not s[\"text\"] in all_text:\n",
    "        filtered_statuses.append(s)\n",
    "        all_text.append(s[\"text\"])\n",
    "statuses = filtered_statuses     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "43"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(statuses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['#MTVAwards Benji e Fede ❣️',\n",
       " \"RT @tylergposey: I'm taking over @mtv's snapchat account from the official #mtvawards + #ToyotaCHR after party. Head over to snapchat now t…\",\n",
       " 'RT @MTVAwards: #ThrowbackThursday to @AllTimeLow slaying the #MTVAwards Festival stage, even in crazy weather 🤘 https://t.co/UBWhsylJ4T',\n",
       " 'RT @MTV: Don’t turn off the lights during this brand new clip of \"IT\" from the #MTVAwards airing right now! https://t.co/2rTf7HcATO',\n",
       " 'RT @johnanjos: Millie win the Best Actor in a Show with Stranger Things  #MTVAwards https://t.co/FlT0KpNQPV',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'Ready to invest? $XYL is definitely worth checking out today! https://t.co/GHMEbo8mch #TigresBicampeón #MTVAwards',\n",
       " \"RT @tylergposey: I'm taking over @mtv's snapchat account from the official #mtvawards + #ToyotaCHR after party. Head over to snapchat now t…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @MTV: So inspired by @TherealTaraji's #MTVAwards acceptance speech for Best Fight Against the System 💥 https://t.co/n9xL147Qvb\",\n",
       " 'RT @THR: .@VinDiesel accepts #MTVAwards for #F8, thanks \"a generation who was willing to accept this multicultural franchise\" https://t.co/…',\n",
       " 'RT @5H_locks: Camila Cabello Lockscreen\\n❊ rt se salvou \\n❊ fav se gostar\\n❊ print se usar  // bEEh\\n\\n#MTVAwards https://t.co/cfDxLtLzCq',\n",
       " \"RT @billyraycyrus: Can't wait to watch you perform tonight @noahcyrus! #StayTogether #NC17 #MTVAwards @MTV https://t.co/XJdwuvwkhV\",\n",
       " \"RT @extratv: .@noahcyrus looking fab at the #MTVawards! Can't wait for her performance! https://t.co/NdlPHpenuZ https://t.co/srjOtVTg0b\",\n",
       " 'RT @MTV: ❤️ @HaileeSteinfeld and @ADAMDEVINE make the perfect Belle + Beast. #MTVAwards https://t.co/FoDRhy0CNn',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @KatherineLanBRA: Nova/antiga foto de Katherine Langford e Dylan Minnette com fã após o #MTVAwards https://t.co/z2P2tGRGGa',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @hSwrote: Copyrighted by @hSwrote 💕💕💕💕#MTVAwards #AllRise #Yankees #ThePartyTour https://t.co/tNVIuQqIBC',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @LaNacionGeek: Listado de ganadores #MTVAwards https://t.co/0Z4V5QJW8E https://t.co/Q8dljQbKwV',\n",
       " 'Sì ricomincia #MTVawards Benji &amp; Fede',\n",
       " 'RT @lilyachty: Keeping it fresh for the #MTVAwards! Tune in Sun. May 7 at 8/7c on @MTV &amp; catch the nominees &amp; performers here!\\xa0https://t.co…',\n",
       " 'RT @EiProfeta: Emma Watson emociona con un precioso discurso en los #MTVAwards. https://t.co/op063RbUFq',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " '#MTVAWARDS @LittleMix https://t.co/SZj7kihdh4',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @TwitterTV: The #MTVAwards are getting an extra dose of 📺\\n\\nCatch the action 🎬 streaming LIVE, right now. https://t.co/JBzDhQTeXQ',\n",
       " 'RT @MTV: BE STILL, MY HEART ❤️ #MTVAwards https://t.co/oYKEY3v8jN',\n",
       " 'RT @moonlight_mov: 🎊MTV Movie &amp; TV Awardsにてベストキス賞受賞!!🎊\\n\\nMTVアワードにて『ムーンライト』がベストキス賞を受賞しました！アシュトン＆ジャレール、おめでとうございます🙌\\n\\n#MTVAwards \\n#Moonlight \\n#ム…',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @enews: This is how we'd like to enter every room from now on. #MTVAwards https://t.co/JGIc1RhtO5\",\n",
       " 'RT @lilyachty: Keeping it fresh for the #MTVAwards! Tune in Sun. May 7 at 8/7c on @MTV &amp; catch the nominees &amp; performers here!\\xa0https://t.co…',\n",
       " \"RT @MTV: You know it's gonna be a fun time when @jackgilinsky and @JackJ arrive. #MTVAwards https://t.co/pVfpJV3hKn\",\n",
       " 'RT @1027KIISFM: Jack &amp; Jack at the #mtvawards 😆😍 @jackgilinsky @jackj @jackandjack https://t.co/5UsBKOpEvt',\n",
       " \"RT @enews: This time, an actor won't be receiving the Generation Award at the #MTVAwards, the Fast &amp; Furious franchise will: https://t.co/O…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @enews: This time, an actor won't be receiving the Generation Award at the #MTVAwards, the Fast &amp; Furious franchise will: https://t.co/O…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @MTV: You know it's gonna be a fun time when @jackgilinsky and @JackJ arrive. #MTVAwards https://t.co/pVfpJV3hKn\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @Sotili: No sólo #DafneKeen se llevó la película de #Logan también se llevo el discurso de agradecimiento en los  #MTVAwards https://t.c…',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @gxmezkindness: Me prohíbo olvidar este momento #MTVAwards https://t.co/28p1tYlMHH',\n",
       " 'RT @MTVAwards: #ThrowbackThursday to @AllTimeLow slaying the #MTVAwards Festival stage, even in crazy weather 🤘 https://t.co/UBWhsylJ4T',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @GalGadot: Hey everyone – I’ll be at the @MTVAwards tomorrow with a ‘wonder’-ful surprise…#WonderWoman 😉 #MTVAwards @WonderWomanFilm htt…',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @OfficialCyrens: 🚨win Noah's signed pass from the #MTVAwards🚨 share the #StayTogetherVideo to keep it climbing on the YT viral chart! ht…\",\n",
       " 'RT @moonlight_mov: 🎊MTV Movie &amp; TV Awardsにてベストキス賞受賞!!🎊\\n\\nMTVアワードにて『ムーンライト』がベストキス賞を受賞しました！アシュトン＆ジャレール、おめでとうございます🙌\\n\\n#MTVAwards \\n#Moonlight \\n#ム…',\n",
       " 'RT @chrissyteigen: Yes, I do my own stunts. New episode of @spikelsb tonight before the #mtvawards at 7:30pm EST! @zendaya vs @TomHolland19…',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"@lilkesh's suit dat he wore in his music video #fakelove exactly like wat @wizkidayo wore to #mtvawards.\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @GirlsGoneWildTH: 7 พฤษภาคม : ภาพล่าสุด Cara Delevingne ที่งาน #MTVAwards\\xa0 จัดขึ้นในลอสแองเจลิส หล่อเข้มมาก ใจสั่นไปหมดแล้วพี่ร่า 😍😝 htt…',\n",
       " 'RT @GirlsGoneWildTH: Emma Watson ร่วมงาน #MTVAwards โดยเอ็มม่าได้รับรางวัล “นักแสดงยอดเยี่ยม” จากภาพยนตร์เรื่อง “Beauty And The Beast” ❤️ h…',\n",
       " 'RT @MTVAwards: #ThrowbackThursday to @AllTimeLow slaying the #MTVAwards Festival stage, even in crazy weather 🤘 https://t.co/UBWhsylJ4T',\n",
       " 'RT @MTVAwards: #ThrowbackThursday to @AllTimeLow slaying the #MTVAwards Festival stage, even in crazy weather 🤘 https://t.co/UBWhsylJ4T',\n",
       " '#DafBama2017_MarcoMengoni  @mengonimarco Buongiorno #Esercito! votiamo? anche 100 voti per gli #MTVAwards !',\n",
       " \"RT @syco: .@Camila_Cabello performed 'Hey Ma' with @JBALVIN and @pitbull at the #MTVAwards 🙌💃 | 📸: @MTV https://t.co/kMBe4FyuNU\",\n",
       " \"RT @syco: .@Camila_Cabello performed 'Hey Ma' with @JBALVIN and @pitbull at the #MTVAwards 🙌💃 | 📸: @MTV https://t.co/kMBe4FyuNU\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @EiProfeta: Emma Watson emociona con un precioso discurso en los #MTVAwards. https://t.co/op063RbUFq',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @tgaud82: Has it really come to this that this show is so bad  now they have to combine both tv and movies together ?  #MTVAwards',\n",
       " \"RT @MTV: You know it's gonna be a fun time when @jackgilinsky and @JackJ arrive. #MTVAwards https://t.co/pVfpJV3hKn\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @PeytonList: Post rain #MTVAwards https://t.co/UnoizJRx8D',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @THR: \\'Moonlight\\' stars Ashton Sanders &amp; Jharrel Jerome win Best Kiss: \"This is for the others. The misfits.\" #MTVAwards https://t.co/Qt…',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @MTVAwards: #ThrowbackThursday to @AllTimeLow slaying the #MTVAwards Festival stage, even in crazy weather 🤘 https://t.co/UBWhsylJ4T',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @Delia_X23: Seriously, look how cute Hugh Jackman and Dafne Keen are together 💜💜💜💜 #MTVAwards https://t.co/0wuOnq2x3W',\n",
       " 'RT @HermanasCoders5: Odio llegar tarde a mi casa por la escuela 😠😭\\n#MTVEXPOPCD9 #MTVCHALLENGCD9 #MTVAwards\\nAnii🍭',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " 'RT @stelovesrdj: This cast is so precious  #MTVAwards https://t.co/UjpalPuaFh',\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\",\n",
       " \"RT @AllTimeLow: Throwback to the @MTV Movie &amp; TV Awards Festival. Still can't believe we played in a hailstorm in LA. #MTVawards \\n\\nhttps://…\"]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[s['text'] for s in search_results['statuses']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      " \"created_at\": \"Fri May 19 10:24:21 +0000 2017\",\n",
      " \"id\": 865513450260250624,\n",
      " \"id_str\": \"865513450260250624\",\n",
      " \"text\": \"#MTVAwards Benji e Fede \\u2763\\ufe0f\",\n",
      " \"truncated\": false,\n",
      " \"entities\": {\n",
      "  \"hashtags\": [\n",
      "   {\n",
      "    \"text\": \"MTVAwards\",\n",
      "    \"indices\": [\n",
      "     0,\n",
      "     10\n",
      "    ]\n",
      "   }\n",
      "  ],\n",
      "  \"symbols\": [],\n",
      "  \"user_mentions\": [],\n",
      "  \"urls\": []\n",
      " },\n",
      " \"metadata\": {\n",
      "  \"iso_language_code\": \"und\",\n",
      "  \"result_type\": \"recent\"\n",
      " },\n",
      " \"source\": \"<a href=\\\"http://twitter.com/download/android\\\" rel=\\\"nofollow\\\">Twitter for Android</a>\",\n",
      " \"in_reply_to_status_id\": null,\n",
      " \"in_reply_to_status_id_str\": null,\n",
      " \"in_reply_to_user_id\": null,\n",
      " \"in_reply_to_user_id_str\": null,\n",
      " \"in_reply_to_screen_name\": null,\n",
      " \"user\": {\n",
      "  \"id\": 3391607301,\n",
      "  \"id_str\": \"3391607301\",\n",
      "  \"name\": \"Alisea\\ud83d\\udc7c\",\n",
      "  \"screen_name\": \"AliseaBatazzi\",\n",
      "  \"location\": \"\",\n",
      "  \"description\": \"Non ti amo per noia, o per solitudine, o per capriccio. \\nTi amo perch\\u00e9 il desiderio di te \\u00e9 pi\\u00f9 forte di qualsiasi felicit\\u00e0. \\ud83d\\udc99\",\n",
      "  \"url\": null,\n",
      "  \"entities\": {\n",
      "   \"description\": {\n",
      "    \"urls\": []\n",
      "   }\n",
      "  },\n",
      "  \"protected\": false,\n",
      "  \"followers_count\": 66,\n",
      "  \"friends_count\": 80,\n",
      "  \"listed_count\": 0,\n",
      "  \"created_at\": \"Tue Jul 28 08:04:35 +0000 2015\",\n",
      "  \"favourites_count\": 1497,\n",
      "  \"utc_offset\": null,\n",
      "  \"time_zone\": null,\n",
      "  \"geo_enabled\": false,\n",
      "  \"verified\": false,\n",
      "  \"statuses_count\": 1218,\n",
      "  \"lang\": \"it\",\n",
      "  \"contributors_enabled\": false,\n",
      "  \"is_translator\": false,\n",
      "  \"is_translation_enabled\": false,\n",
      "  \"profile_background_color\": \"000000\",\n",
      "  \"profile_background_image_url\": \"http://abs.twimg.com/images/themes/theme1/bg.png\",\n",
      "  \"profile_background_image_url_https\": \"https://abs.twimg.com/images/themes/theme1/bg.png\",\n",
      "  \"profile_background_tile\": false,\n",
      "  \"profile_image_url\": \"http://pbs.twimg.com/profile_images/817400932103258113/8UpBT8v-_normal.jpg\",\n",
      "  \"profile_image_url_https\": \"https://pbs.twimg.com/profile_images/817400932103258113/8UpBT8v-_normal.jpg\",\n",
      "  \"profile_banner_url\": \"https://pbs.twimg.com/profile_banners/3391607301/1479360107\",\n",
      "  \"profile_link_color\": \"9266CC\",\n",
      "  \"profile_sidebar_border_color\": \"000000\",\n",
      "  \"profile_sidebar_fill_color\": \"000000\",\n",
      "  \"profile_text_color\": \"000000\",\n",
      "  \"profile_use_background_image\": false,\n",
      "  \"has_extended_profile\": true,\n",
      "  \"default_profile\": false,\n",
      "  \"default_profile_image\": false,\n",
      "  \"following\": false,\n",
      "  \"follow_request_sent\": false,\n",
      "  \"notifications\": false,\n",
      "  \"translator_type\": \"none\"\n",
      " },\n",
      " \"geo\": null,\n",
      " \"coordinates\": null,\n",
      " \"place\": null,\n",
      " \"contributors\": null,\n",
      " \"is_quote_status\": false,\n",
      " \"retweet_count\": 0,\n",
      " \"favorite_count\": 0,\n",
      " \"favorited\": false,\n",
      " \"retweeted\": false,\n",
      " \"lang\": \"und\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# Show one sample search result by slicing the list...\n",
    "print(json.dumps(statuses[0], indent=1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "# The result of the list comprehension is a list with only one element that\n",
    "# can be accessed by its index and set to the variable t\n",
    "t = statuses[0]\n",
    "#[ status for status in statuses \n",
    "#          if status['id'] == 316948241264549888 ][0]\n",
    "\n",
    "# Explore the variable t to get familiarized with the data structure...\n",
    "\n",
    "print(t['retweet_count'])\n",
    "print(t['retweeted'])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 6. Extracting text, screen names, and hashtags from tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      " \"#MTVAwards Benji e Fede \\u2763\\ufe0f\",\n",
      " \"RT @tylergposey: I'm taking over @mtv's snapchat account from the official #mtvawards + #ToyotaCHR after party. Head over to snapchat now t\\u2026\",\n",
      " \"RT @MTVAwards: #ThrowbackThursday to @AllTimeLow slaying the #MTVAwards Festival stage, even in crazy weather \\ud83e\\udd18 https://t.co/UBWhsylJ4T\",\n",
      " \"RT @MTV: Don\\u2019t turn off the lights during this brand new clip of \\\"IT\\\" from the #MTVAwards airing right now! https://t.co/2rTf7HcATO\",\n",
      " \"RT @johnanjos: Millie win the Best Actor in a Show with Stranger Things  #MTVAwards https://t.co/FlT0KpNQPV\"\n",
      "]\n",
      "[\n",
      " \"tylergposey\",\n",
      " \"MTV\",\n",
      " \"MTVAwards\",\n",
      " \"AllTimeLow\",\n",
      " \"MTV\"\n",
      "]\n",
      "[\n",
      " \"MTVAwards\",\n",
      " \"mtvawards\",\n",
      " \"ToyotaCHR\",\n",
      " \"ThrowbackThursday\",\n",
      " \"MTVAwards\"\n",
      "]\n",
      "[\n",
      " \"#MTVAwards\",\n",
      " \"Benji\",\n",
      " \"e\",\n",
      " \"Fede\",\n",
      " \"\\u2763\\ufe0f\"\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "status_texts = [ status['text'] \n",
    "                 for status in statuses ]\n",
    "\n",
    "screen_names = [ user_mention['screen_name'] \n",
    "                 for status in statuses\n",
    "                     for user_mention in status['entities']['user_mentions'] ]\n",
    "\n",
    "hashtags = [ hashtag['text'] \n",
    "             for status in statuses\n",
    "                 for hashtag in status['entities']['hashtags'] ]\n",
    "\n",
    "# Compute a collection of all words from all tweets\n",
    "words = [ w \n",
    "          for t in status_texts \n",
    "              for w in t.split() ]\n",
    "\n",
    "# Explore the first 5 items for each...\n",
    "\n",
    "print(json.dumps(status_texts[0:5], indent=1))\n",
    "print(json.dumps(screen_names[0:5], indent=1)) \n",
    "print(json.dumps(hashtags[0:5], indent=1))\n",
    "print(json.dumps(words[0:5], indent=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 7. Creating a basic frequency distribution from the words in tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('RT', 37), ('#MTVAwards', 30), ('the', 23), ('to', 11), ('&amp;', 8), ('at', 7), ('in', 6), ('@MTV:', 5), ('a', 5), ('is', 5), ('for', 5), ('se', 5), ('and', 5), ('from', 4), ('this', 4), ('@MTV', 4), ('de', 4), ('#mtvawards', 3), ('now', 3), ('of', 3)]\n",
      "\n",
      "[('MTV', 10), ('MTVAwards', 2), ('AllTimeLow', 2), ('THR', 2), ('noahcyrus', 2), ('hSwrote', 2), ('enews', 2), ('jackgilinsky', 2), ('JackJ', 2), ('GirlsGoneWildTH', 2), ('tylergposey', 1), ('johnanjos', 1), ('TherealTaraji', 1), ('vindiesel', 1), ('5H_locks', 1), ('billyraycyrus', 1), ('extratv', 1), ('HaileeSteinfeld', 1), ('ADAMDEVINE', 1), ('KatherineLanBRA', 1)]\n",
      "\n",
      "[('MTVAwards', 35), ('mtvawards', 4), ('MTVawards', 3), ('ToyotaCHR', 1), ('ThrowbackThursday', 1), ('TigresBicampeón', 1), ('F8', 1), ('StayTogether', 1), ('NC17', 1), ('AllRise', 1), ('Yankees', 1), ('ThePartyTour', 1), ('MTVAWARDS', 1), ('Moonlight', 1), ('DafneKeen', 1), ('Logan', 1), ('WonderWoman', 1), ('StayTogetherVideo', 1), ('fakelove', 1), ('DafBama2017_MarcoMengoni', 1)]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from collections import Counter\n",
    "\n",
    "for item in [words, screen_names, hashtags]:\n",
    "    c = Counter(item)\n",
    "    print(c.most_common()[:20]) # top 10\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 8. Create a prettyprint function to display tuples in a nice tabular format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def prettyprint_counts(label, list_of_tuples):\n",
    "    print(\"\\n{:^20} | {:^6}\".format(label, \"Count\"))\n",
    "    print(\"*\"*40)\n",
    "    for k,v in list_of_tuples:\n",
    "        print(\"{:20} | {:>6}\".format(k,v))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "        Word         | Count \n",
      "****************************************\n",
      "RT                   |     37\n",
      "#MTVAwards           |     30\n",
      "the                  |     23\n",
      "to                   |     11\n",
      "&amp;                |      8\n",
      "at                   |      7\n",
      "in                   |      6\n",
      "@MTV:                |      5\n",
      "a                    |      5\n",
      "is                   |      5\n",
      "\n",
      "    Screen Name      | Count \n",
      "****************************************\n",
      "MTV                  |     10\n",
      "MTVAwards            |      2\n",
      "AllTimeLow           |      2\n",
      "THR                  |      2\n",
      "noahcyrus            |      2\n",
      "hSwrote              |      2\n",
      "enews                |      2\n",
      "jackgilinsky         |      2\n",
      "JackJ                |      2\n",
      "GirlsGoneWildTH      |      2\n",
      "\n",
      "      Hashtag        | Count \n",
      "****************************************\n",
      "MTVAwards            |     35\n",
      "mtvawards            |      4\n",
      "MTVawards            |      3\n",
      "ToyotaCHR            |      1\n",
      "ThrowbackThursday    |      1\n",
      "TigresBicampeón      |      1\n",
      "F8                   |      1\n",
      "StayTogether         |      1\n",
      "NC17                 |      1\n",
      "AllRise              |      1\n"
     ]
    }
   ],
   "source": [
    "for label, data in (('Word', words), \n",
    "                    ('Screen Name', screen_names), \n",
    "                    ('Hashtag', hashtags)):\n",
    "    \n",
    "    c = Counter(data)\n",
    "    prettyprint_counts(label, c.most_common()[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 9. Finding the most popular retweets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can build another `prettyprint` function to print entire tweets with their retweet count.\n",
    "\n",
    "We also want to split the text of the tweet in up to 3 lines, if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "row_template = \"{:^7} | {:^15} | {:50}\"\n",
    "def prettyprint_tweets(list_of_tuples):\n",
    "    print()\n",
    "    print(row_template.format(\"Count\", \"Screen Name\", \"Text\"))\n",
    "    print(\"*\"*30)\n",
    "    for count, screen_name, text in list_of_tuples:\n",
    "        print(row_template.format(count, screen_name, text[:50]))\n",
    "        if len(text) > 50:\n",
    "            print(row_template.format(\"\", \"\", text[50:100]))\n",
    "            if len(text) > 100:\n",
    "                print(row_template.format(\"\", \"\", text[100:]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Count  |   Screen Name   | Text                                              \n",
      "******************************\n",
      " 14466  |       MTV       | RT @MTV: Don’t turn off the lights during this bra\n",
      "        |                 | nd new clip of \"IT\" from the #MTVAwards airing rig\n",
      "        |                 | ht now! https://t.co/2rTf7HcATO                   \n",
      " 12523  |       MTV       | RT @MTV: BE STILL, MY HEART ❤️ #MTVAwards https://\n",
      "        |                 | t.co/oYKEY3v8jN                                   \n",
      " 6964   |      enews      | RT @enews: This is how we'd like to enter every ro\n",
      "        |                 | om from now on. #MTVAwards https://t.co/JGIc1RhtO5\n",
      " 5066   |       THR       | RT @THR: 'Moonlight' stars Ashton Sanders &amp; Jh\n",
      "        |                 | arrel Jerome win Best Kiss: \"This is for the other\n",
      "        |                 | s. The misfits.\" #MTVAwards https://t.co/Qt…      \n",
      " 4934   |  chrissyteigen  | RT @chrissyteigen: Yes, I do my own stunts. New ep\n",
      "        |                 | isode of @spikelsb tonight before the #mtvawards a\n",
      "        |                 | t 7:30pm EST! @zendaya vs @TomHolland19…          \n"
     ]
    }
   ],
   "source": [
    "retweets = [\n",
    "            # Store out a tuple of these three values ...\n",
    "            (status['retweet_count'], \n",
    "             status['retweeted_status']['user']['screen_name'],\n",
    "             status['text'].replace(\"\\n\",\"\\\\\")) \n",
    "            \n",
    "            # ... for each status ...\n",
    "            for status in statuses \n",
    "            \n",
    "            # ... so long as the status meets this condition.\n",
    "                if 'retweeted_status' in status\n",
    "           ]\n",
    "\n",
    "# Slice off the first 5 from the sorted results and display each item in the tuple\n",
    "\n",
    "prettyprint_tweets(sorted(retweets, reverse=True)[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}