07_twitter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Twitter data\n",
    "\n",
    "## Copyright and Licensing\n",
    "\n",
    "You are free to use or adapt this notebook for any purpose you'd like. However, please respect the [Simplified BSD License](https://github.com/ptwobrussell/Mining-the-Social-Web-2nd-Edition/blob/master/LICENSE.txt) that governs its use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Twitter API Access\n",
    "\n",
    "Twitter implements OAuth 1.0A as its standard authentication mechanism, and in order to use it to make requests to Twitter's API, you'll need to go to https://dev.twitter.com/apps and create a sample application.\n",
    "\n",
    "Choose any name for your application, write a description and use `http://google.com` for the website.\n",
    "\n",
    "Under **Key and Access Tokens**, there are four primary identifiers you'll need to note for an OAuth 1.0A workflow: consumer key, consumer secret, access token, and access token secret (Click on Create Access Token to create those). Note that you will need an ordinary Twitter account in order to login, create an app, and get these credentials."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first time you execute the notebook, add all credentials so that you can save them in the `pkl` file, then you can remove the secret keys from the notebook because they will just be loaded from the `pkl` file.\n",
    "\n",
    "The `pkl` file contains sensitive information that can be used to take control of your twitter acccount, **do not share it**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "if not os.path.exists('secret_twitter_credentials.pkl'):\n",
    "    Twitter={}\n",
    "    Twitter['Consumer Key'] = ''\n",
    "    Twitter['Consumer Secret'] = ''\n",
    "    Twitter['Access Token'] = ''\n",
    "    Twitter['Access Token Secret'] = ''\n",
    "    with open('secret_twitter_credentials.pkl','wb') as f:\n",
    "        pickle.dump(Twitter, f)\n",
    "else:\n",
    "    Twitter=pickle.load(open('secret_twitter_credentials.pkl','rb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install the `twitter` package to interface with the Twitter API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: twitter in /home/zonca/anaconda/lib/python3.5/site-packages\r\n"
     ]
    }
   ],
   "source": [
    "!pip install twitter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1. Authorizing an application to access Twitter account data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<twitter.api.Twitter object at 0x7f983401edd8>\n"
     ]
    }
   ],
   "source": [
    "import twitter\n",
    "\n",
    "auth = twitter.oauth.OAuth(Twitter['Access Token'],\n",
    "                           Twitter['Access Token Secret'],\n",
    "                           Twitter['Consumer Key'],\n",
    "                           Twitter['Consumer Secret'])\n",
    "\n",
    "twitter_api = twitter.Twitter(auth=auth)\n",
    "\n",
    "# Nothing to see by displaying twitter_api except that it's now a\n",
    "# defined variable\n",
    "\n",
    "print(twitter_api)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2. Retrieving trends"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Twitter identifies locations using the Yahoo! Where On Earth ID.\n",
    "\n",
    "The Yahoo! Where On Earth ID for the entire world is 1.\n",
    "See https://dev.twitter.com/docs/api/1.1/get/trends/place and\n",
    "http://developer.yahoo.com/geo/geoplanet/\n",
    "\n",
    "look at the BOSS placefinder here: https://developer.yahoo.com/boss/placefinder/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "WORLD_WOE_ID = 1\n",
    "US_WOE_ID = 23424977"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look for the WOEID for [san-diego](http://woeid.rosselliot.co.nz/lookup/san%20diego%20%20ca)\n",
    "\n",
    "You can change it to another location."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "LOCAL_WOE_ID=2487889\n",
    "\n",
    "# Prefix ID with the underscore for query string parameterization.\n",
    "# Without the underscore, the twitter package appends the ID value\n",
    "# to the URL itself as a special case keyword argument.\n",
    "\n",
    "world_trends = twitter_api.trends.place(_id=WORLD_WOE_ID)\n",
    "us_trends = twitter_api.trends.place(_id=US_WOE_ID)\n",
    "local_trends = twitter_api.trends.place(_id=LOCAL_WOE_ID)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'as_of': '2017-05-08T05:57:50Z',\n",
       "  'created_at': '2017-05-08T05:53:27Z',\n",
       "  'locations': [{'name': 'Worldwide', 'woeid': 1}],\n",
       "  'trends': [{'name': '#MTVAwards',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23MTVAwards',\n",
       "    'tweet_volume': 1078929,\n",
       "    'url': 'http://twitter.com/search?q=%23MTVAwards'},\n",
       "   {'name': '#MasterChefMx',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23MasterChefMx',\n",
       "    'tweet_volume': 39744,\n",
       "    'url': 'http://twitter.com/search?q=%23MasterChefMx'},\n",
       "   {'name': '#CorazonesRotosMC',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23CorazonesRotosMC',\n",
       "    'tweet_volume': 13996,\n",
       "    'url': 'http://twitter.com/search?q=%23CorazonesRotosMC'},\n",
       "   {'name': 'Joe West',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Joe+West%22',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%22Joe+West%22'},\n",
       "   {'name': '#حاجات_تفتح_نفسي',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%AD%D8%A7%D8%AC%D8%A7%D8%AA_%D8%AA%D9%81%D8%AA%D8%AD_%D9%86%D9%81%D8%B3%D9%8A',\n",
       "    'tweet_volume': 28268,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%AD%D8%A7%D8%AC%D8%A7%D8%AA_%D8%AA%D9%81%D8%AA%D8%AD_%D9%86%D9%81%D8%B3%D9%8A'},\n",
       "   {'name': '#RHOA',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23RHOA',\n",
       "    'tweet_volume': 115361,\n",
       "    'url': 'http://twitter.com/search?q=%23RHOA'},\n",
       "   {'name': 'Guerra Mundial Z',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Guerra+Mundial+Z%22',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%22Guerra+Mundial+Z%22'},\n",
       "   {'name': 'Stranger Things',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Stranger+Things%22',\n",
       "    'tweet_volume': 66201,\n",
       "    'url': 'http://twitter.com/search?q=%22Stranger+Things%22'},\n",
       "   {'name': 'GW明け',\n",
       "    'promoted_content': None,\n",
       "    'query': 'GW%E6%98%8E%E3%81%91',\n",
       "    'tweet_volume': 67452,\n",
       "    'url': 'http://twitter.com/search?q=GW%E6%98%8E%E3%81%91'},\n",
       "   {'name': 'Erica Rivas',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Erica+Rivas%22',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%22Erica+Rivas%22'},\n",
       "   {'name': 'Jake Arrieta',\n",
       "    'promoted_content': None,\n",
       "    'query': '%22Jake+Arrieta%22',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%22Jake+Arrieta%22'},\n",
       "   {'name': '#سقوط_طالب_من_الدرج',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%B3%D9%82%D9%88%D8%B7_%D8%B7%D8%A7%D9%84%D8%A8_%D9%85%D9%86_%D8%A7%D9%84%D8%AF%D8%B1%D8%AC',\n",
       "    'tweet_volume': 13940,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%B3%D9%82%D9%88%D8%B7_%D8%B7%D8%A7%D9%84%D8%A8_%D9%85%D9%86_%D8%A7%D9%84%D8%AF%D8%B1%D8%AC'},\n",
       "   {'name': '#FelizLunes',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23FelizLunes',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23FelizLunes'},\n",
       "   {'name': '#무엇을_어디에_넣어줄지_말해보자',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%EB%AC%B4%EC%97%87%EC%9D%84_%EC%96%B4%EB%94%94%EC%97%90_%EB%84%A3%EC%96%B4%EC%A4%84%EC%A7%80_%EB%A7%90%ED%95%B4%EB%B3%B4%EC%9E%90',\n",
       "    'tweet_volume': 14082,\n",
       "    'url': 'http://twitter.com/search?q=%23%EB%AC%B4%EC%97%87%EC%9D%84_%EC%96%B4%EB%94%94%EC%97%90_%EB%84%A3%EC%96%B4%EC%A4%84%EC%A7%80_%EB%A7%90%ED%95%B4%EB%B3%B4%EC%9E%90'},\n",
       "   {'name': '#PurposeTourManila',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23PurposeTourManila',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23PurposeTourManila'},\n",
       "   {'name': '#MejorHablar',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23MejorHablar',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23MejorHablar'},\n",
       "   {'name': '#VenezuelaUrgente',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23VenezuelaUrgente',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23VenezuelaUrgente'},\n",
       "   {'name': '#JesusCristoEstáVivo',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23JesusCristoEst%C3%A1Vivo',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23JesusCristoEst%C3%A1Vivo'},\n",
       "   {'name': '#MLBxESPN',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23MLBxESPN',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23MLBxESPN'},\n",
       "   {'name': '#محمد_بن_سلمان_يزلزل_ايران',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D9%85%D8%AD%D9%85%D8%AF_%D8%A8%D9%86_%D8%B3%D9%84%D9%85%D8%A7%D9%86_%D9%8A%D8%B2%D9%84%D8%B2%D9%84_%D8%A7%D9%8A%D8%B1%D8%A7%D9%86',\n",
       "    'tweet_volume': 26888,\n",
       "    'url': 'http://twitter.com/search?q=%23%D9%85%D8%AD%D9%85%D8%AF_%D8%A8%D9%86_%D8%B3%D9%84%D9%85%D8%A7%D9%86_%D9%8A%D8%B2%D9%84%D8%B2%D9%84_%D8%A7%D9%8A%D8%B1%D8%A7%D9%86'},\n",
       "   {'name': '#EnLiguillaVoy',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23EnLiguillaVoy',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23EnLiguillaVoy'},\n",
       "   {'name': '#SegundaComClubeSDV',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23SegundaComClubeSDV',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23SegundaComClubeSDV'},\n",
       "   {'name': '#نغرد_عشان',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D9%86%D8%BA%D8%B1%D8%AF_%D8%B9%D8%B4%D8%A7%D9%86',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%D9%86%D8%BA%D8%B1%D8%AF_%D8%B9%D8%B4%D8%A7%D9%86'},\n",
       "   {'name': '#FodderScam',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23FodderScam',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23FodderScam'},\n",
       "   {'name': '#AltaTensión',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23AltaTensi%C3%B3n',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23AltaTensi%C3%B3n'},\n",
       "   {'name': '#AmericanRaceTNT',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23AmericanRaceTNT',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23AmericanRaceTNT'},\n",
       "   {'name': '#OnceUponAMusical',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23OnceUponAMusical',\n",
       "    'tweet_volume': 62479,\n",
       "    'url': 'http://twitter.com/search?q=%23OnceUponAMusical'},\n",
       "   {'name': '#ProfileInCourage',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23ProfileInCourage',\n",
       "    'tweet_volume': 38825,\n",
       "    'url': 'http://twitter.com/search?q=%23ProfileInCourage'},\n",
       "   {'name': '#ShowtimeMAYForever',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23ShowtimeMAYForever',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23ShowtimeMAYForever'},\n",
       "   {'name': '#pazartesi',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23pazartesi',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23pazartesi'},\n",
       "   {'name': '#unite2017tokyo',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23unite2017tokyo',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23unite2017tokyo'},\n",
       "   {'name': '#DANCEDANCE',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23DANCEDANCE',\n",
       "    'tweet_volume': 35335,\n",
       "    'url': 'http://twitter.com/search?q=%23DANCEDANCE'},\n",
       "   {'name': '#kokkai',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23kokkai',\n",
       "    'tweet_volume': 14538,\n",
       "    'url': 'http://twitter.com/search?q=%23kokkai'},\n",
       "   {'name': '#iBribedKejriwal',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23iBribedKejriwal',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23iBribedKejriwal'},\n",
       "   {'name': '#في_حبك_قلت',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D9%81%D9%8A_%D8%AD%D8%A8%D9%83_%D9%82%D9%84%D8%AA',\n",
       "    'tweet_volume': 63619,\n",
       "    'url': 'http://twitter.com/search?q=%23%D9%81%D9%8A_%D8%AD%D8%A8%D9%83_%D9%82%D9%84%D8%AA'},\n",
       "   {'name': '#صباحات_الاهلي',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%B5%D8%A8%D8%A7%D8%AD%D8%A7%D8%AA_%D8%A7%D9%84%D8%A7%D9%87%D9%84%D9%8A',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%B5%D8%A8%D8%A7%D8%AD%D8%A7%D8%AA_%D8%A7%D9%84%D8%A7%D9%87%D9%84%D9%8A'},\n",
       "   {'name': '#NoMeGustaQue',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23NoMeGustaQue',\n",
       "    'tweet_volume': 11318,\n",
       "    'url': 'http://twitter.com/search?q=%23NoMeGustaQue'},\n",
       "   {'name': '#MTVActAppVillalobos',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23MTVActAppVillalobos',\n",
       "    'tweet_volume': 166316,\n",
       "    'url': 'http://twitter.com/search?q=%23MTVActAppVillalobos'},\n",
       "   {'name': '#DeboDecir',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23DeboDecir',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23DeboDecir'},\n",
       "   {'name': '#MagicianDatingProblems',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23MagicianDatingProblems',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23MagicianDatingProblems'},\n",
       "   {'name': '#LaVozKids',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23LaVozKids',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23LaVozKids'},\n",
       "   {'name': '#BillionsSeasonFinale',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23BillionsSeasonFinale',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23BillionsSeasonFinale'},\n",
       "   {'name': '#IntoTheBadlands',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23IntoTheBadlands',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23IntoTheBadlands'},\n",
       "   {'name': '#EntMEWomen',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23EntMEWomen',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23EntMEWomen'},\n",
       "   {'name': '#SinMiedoTuca',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23SinMiedoTuca',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23SinMiedoTuca'},\n",
       "   {'name': '#اجد_في_عيناك',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%A7%D8%AC%D8%AF_%D9%81%D9%8A_%D8%B9%D9%8A%D9%86%D8%A7%D9%83',\n",
       "    'tweet_volume': 18045,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%A7%D8%AC%D8%AF_%D9%81%D9%8A_%D8%B9%D9%8A%D9%86%D8%A7%D9%83'},\n",
       "   {'name': '#تجمع_التربيه_الخاصه_بجده2',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23%D8%AA%D8%AC%D9%85%D8%B9_%D8%A7%D9%84%D8%AA%D8%B1%D8%A8%D9%8A%D9%87_%D8%A7%D9%84%D8%AE%D8%A7%D8%B5%D9%87_%D8%A8%D8%AC%D8%AF%D9%872',\n",
       "    'tweet_volume': None,\n",
       "    'url': 'http://twitter.com/search?q=%23%D8%AA%D8%AC%D9%85%D8%B9_%D8%A7%D9%84%D8%AA%D8%B1%D8%A8%D9%8A%D9%87_%D8%A7%D9%84%D8%AE%D8%A7%D8%B5%D9%87_%D8%A8%D8%AC%D8%AF%D9%872'},\n",
       "   {'name': '#Independiente',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23Independiente',\n",
       "    'tweet_volume': 12124,\n",
       "    'url': 'http://twitter.com/search?q=%23Independiente'},\n",
       "   {'name': '#SallyYatesIsAPatriot',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23SallyYatesIsAPatriot',\n",
       "    'tweet_volume': 105190,\n",
       "    'url': 'http://twitter.com/search?q=%23SallyYatesIsAPatriot'},\n",
       "   {'name': '#DafBama2017_SuperJunior',\n",
       "    'promoted_content': None,\n",
       "    'query': '%23DafBama2017_SuperJunior',\n",
       "    'tweet_volume': 20857,\n",
       "    'url': 'http://twitter.com/search?q=%23DafBama2017_SuperJunior'}]}]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "world_trends[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'twitter.api.TwitterListResponse'>\n",
      "['as_of', 'created_at', 'trends', 'locations']\n",
      "[{'promoted_content': None, 'url': 'http://twitter.com/search?q=%23MTVAwards', 'name': '#MTVAwards', 'query': '%23MTVAwards', 'tweet_volume': 1078929}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23RHOA', 'name': '#RHOA', 'query': '%23RHOA', 'tweet_volume': 115361}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23LigaMXeng', 'name': '#LigaMXeng', 'query': '%23LigaMXeng', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23frenchelection', 'name': '#frenchelection', 'query': '%23frenchelection', 'tweet_volume': 152976}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23NW88JR', 'name': '#NW88JR', 'query': '%23NW88JR', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Chasen+Shreve%22', 'name': 'Chasen Shreve', 'query': '%22Chasen+Shreve%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Jake+Arrieta%22', 'name': 'Jake Arrieta', 'query': '%22Jake+Arrieta%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Wade+Davis%22', 'name': 'Wade Davis', 'query': '%22Wade+Davis%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Cheesecake+Factory%22', 'name': 'Cheesecake Factory', 'query': '%22Cheesecake+Factory%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=Bryant', 'name': 'Bryant', 'query': 'Bryant', 'tweet_volume': 13697}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Brian+Duensing%22', 'name': 'Brian Duensing', 'query': '%22Brian+Duensing%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Patrick+Beverley%22', 'name': 'Patrick Beverley', 'query': '%22Patrick+Beverley%22', 'tweet_volume': 10006}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Aaron+Boone%22', 'name': 'Aaron Boone', 'query': '%22Aaron+Boone%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Bobby+Brown%22', 'name': 'Bobby Brown', 'query': '%22Bobby+Brown%22', 'tweet_volume': 21613}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Stranger+Things%22', 'name': 'Stranger Things', 'query': '%22Stranger+Things%22', 'tweet_volume': 66201}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Jump+Out+The+Window%22', 'name': 'Jump Out The Window', 'query': '%22Jump+Out+The+Window%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Adam+Warren%22', 'name': 'Adam Warren', 'query': '%22Adam+Warren%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22New+MLB%22', 'name': 'New MLB', 'query': '%22New+MLB%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Eric+Gordon%22', 'name': 'Eric Gordon', 'query': '%22Eric+Gordon%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Kings+of+Leon%22', 'name': 'Kings of Leon', 'query': '%22Kings+of+Leon%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Princess+Diana%22', 'name': 'Princess Diana', 'query': '%22Princess+Diana%22', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%22Big+Sean%22', 'name': 'Big Sean', 'query': '%22Big+Sean%22', 'tweet_volume': 36977}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23ProfileInCourage', 'name': '#ProfileInCourage', 'query': '%23ProfileInCourage', 'tweet_volume': 38825}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23digitalhealth', 'name': '#digitalhealth', 'query': '%23digitalhealth', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23Yankees', 'name': '#Yankees', 'query': '%23Yankees', 'tweet_volume': 11203}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23AmericanRaceTNT', 'name': '#AmericanRaceTNT', 'query': '%23AmericanRaceTNT', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23Billions', 'name': '#Billions', 'query': '%23Billions', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23MagicianDatingProblems', 'name': '#MagicianDatingProblems', 'query': '%23MagicianDatingProblems', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23TheLeftovers', 'name': '#TheLeftovers', 'query': '%23TheLeftovers', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23IntoTheBadlands', 'name': '#IntoTheBadlands', 'query': '%23IntoTheBadlands', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23OnceUponAMusical', 'name': '#OnceUponAMusical', 'query': '%23OnceUponAMusical', 'tweet_volume': 62479}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23GoCubsGo', 'name': '#GoCubsGo', 'query': '%23GoCubsGo', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23UnitedShades', 'name': '#UnitedShades', 'query': '%23UnitedShades', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23Veep', 'name': '#Veep', 'query': '%23Veep', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23IWishINeverHadTo', 'name': '#IWishINeverHadTo', 'query': '%23IWishINeverHadTo', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23SallyYatesIsAPatriot', 'name': '#SallyYatesIsAPatriot', 'query': '%23SallyYatesIsAPatriot', 'tweet_volume': 105190}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23LastWeekTonight', 'name': '#LastWeekTonight', 'query': '%23LastWeekTonight', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23WhyIHatedSchool', 'name': '#WhyIHatedSchool', 'query': '%23WhyIHatedSchool', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23KUWTK', 'name': '#KUWTK', 'query': '%23KUWTK', 'tweet_volume': 11534}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%234WordLetDowns', 'name': '#4WordLetDowns', 'query': '%234WordLetDowns', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23PlaylistatUniversal', 'name': '#PlaylistatUniversal', 'query': '%23PlaylistatUniversal', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23ShadesOfBlue', 'name': '#ShadesOfBlue', 'query': '%23ShadesOfBlue', 'tweet_volume': 11041}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23TheArrangement', 'name': '#TheArrangement', 'query': '%23TheArrangement', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23LastManOnEarth', 'name': '#LastManOnEarth', 'query': '%23LastManOnEarth', 'tweet_volume': None}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23Next100Days', 'name': '#Next100Days', 'query': '%23Next100Days', 'tweet_volume': 64398}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23MasterChefMx', 'name': '#MasterChefMx', 'query': '%23MasterChefMx', 'tweet_volume': 39744}, {'promoted_content': None, 'url': 'http://twitter.com/search?q=%23SiliconValleyHBO', 'name': '#SiliconValleyHBO', 'query': '%23SiliconValleyHBO', 'tweet_volume': None}]\n"
     ]
    }
   ],
   "source": [
    "trends=local_trends\n",
    "print(type(trends))\n",
    "print(list(trends[0].keys()))\n",
    "print(trends[0]['trends'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3. Displaying API responses as pretty-printed JSON"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      " {\n",
      "  \"as_of\": \"2017-05-08T05:57:50Z\",\n",
      "  \"created_at\": \"2017-05-08T05:53:27Z\",\n",
      "  \"trends\": [\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Joe+West%22\",\n",
      "    \"name\": \"Joe West\",\n",
      "    \"query\": \"%22Joe+West%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23RHOA\",\n",
      "    \"name\": \"#RHOA\",\n",
      "    \"query\": \"%23RHOA\",\n",
      "    \"tweet_volume\": 115361\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23MTVAwards\",\n",
      "    \"name\": \"#MTVAwards\",\n",
      "    \"query\": \"%23MTVAwards\",\n",
      "    \"tweet_volume\": 1078929\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23AmericanRaceTNT\",\n",
      "    \"name\": \"#AmericanRaceTNT\",\n",
      "    \"query\": \"%23AmericanRaceTNT\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23ProfileInCourage\",\n",
      "    \"name\": \"#ProfileInCourage\",\n",
      "    \"query\": \"%23ProfileInCourage\",\n",
      "    \"tweet_volume\": 38825\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23Billions\",\n",
      "    \"name\": \"#Billions\",\n",
      "    \"query\": \"%23Billions\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Chasen+Shreve%22\",\n",
      "    \"name\": \"Chasen Shreve\",\n",
      "    \"query\": \"%22Chasen+Shreve%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Jake+Arrieta%22\",\n",
      "    \"name\": \"Jake Arrieta\",\n",
      "    \"query\": \"%22Jake+Arrieta%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Wade+Davis%22\",\n",
      "    \"name\": \"Wade Davis\",\n",
      "    \"query\": \"%22Wade+Davis%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Chris+Carter%22\",\n",
      "    \"name\": \"Chris Carter\",\n",
      "    \"query\": \"%22Chris+Carter%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Cheesecake+Factory%22\",\n",
      "    \"name\": \"Cheesecake Factory\",\n",
      "    \"query\": \"%22Cheesecake+Factory%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=Bryant\",\n",
      "    \"name\": \"Bryant\",\n",
      "    \"query\": \"Bryant\",\n",
      "    \"tweet_volume\": 13697\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Brian+Duensing%22\",\n",
      "    \"name\": \"Brian Duensing\",\n",
      "    \"query\": \"%22Brian+Duensing%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Patrick+Beverley%22\",\n",
      "    \"name\": \"Patrick Beverley\",\n",
      "    \"query\": \"%22Patrick+Beverley%22\",\n",
      "    \"tweet_volume\": 10006\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Aaron+Boone%22\",\n",
      "    \"name\": \"Aaron Boone\",\n",
      "    \"query\": \"%22Aaron+Boone%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Bobby+Brown%22\",\n",
      "    \"name\": \"Bobby Brown\",\n",
      "    \"query\": \"%22Bobby+Brown%22\",\n",
      "    \"tweet_volume\": 21613\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Stranger+Things%22\",\n",
      "    \"name\": \"Stranger Things\",\n",
      "    \"query\": \"%22Stranger+Things%22\",\n",
      "    \"tweet_volume\": 66201\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Jump+Out+The+Window%22\",\n",
      "    \"name\": \"Jump Out The Window\",\n",
      "    \"query\": \"%22Jump+Out+The+Window%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Adam+Warren%22\",\n",
      "    \"name\": \"Adam Warren\",\n",
      "    \"query\": \"%22Adam+Warren%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22New+MLB%22\",\n",
      "    \"name\": \"New MLB\",\n",
      "    \"query\": \"%22New+MLB%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Eric+Gordon%22\",\n",
      "    \"name\": \"Eric Gordon\",\n",
      "    \"query\": \"%22Eric+Gordon%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Kings+of+Leon%22\",\n",
      "    \"name\": \"Kings of Leon\",\n",
      "    \"query\": \"%22Kings+of+Leon%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Princess+Diana%22\",\n",
      "    \"name\": \"Princess Diana\",\n",
      "    \"query\": \"%22Princess+Diana%22\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%22Big+Sean%22\",\n",
      "    \"name\": \"Big Sean\",\n",
      "    \"query\": \"%22Big+Sean%22\",\n",
      "    \"tweet_volume\": 36977\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23MagicianDatingProblems\",\n",
      "    \"name\": \"#MagicianDatingProblems\",\n",
      "    \"query\": \"%23MagicianDatingProblems\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23TheLeftovers\",\n",
      "    \"name\": \"#TheLeftovers\",\n",
      "    \"query\": \"%23TheLeftovers\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23IntoTheBadlands\",\n",
      "    \"name\": \"#IntoTheBadlands\",\n",
      "    \"query\": \"%23IntoTheBadlands\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23OnceUponAMusical\",\n",
      "    \"name\": \"#OnceUponAMusical\",\n",
      "    \"query\": \"%23OnceUponAMusical\",\n",
      "    \"tweet_volume\": 62479\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23UnitedShades\",\n",
      "    \"name\": \"#UnitedShades\",\n",
      "    \"query\": \"%23UnitedShades\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23Veep\",\n",
      "    \"name\": \"#Veep\",\n",
      "    \"query\": \"%23Veep\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23IWishINeverHadTo\",\n",
      "    \"name\": \"#IWishINeverHadTo\",\n",
      "    \"query\": \"%23IWishINeverHadTo\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23SallyYatesIsAPatriot\",\n",
      "    \"name\": \"#SallyYatesIsAPatriot\",\n",
      "    \"query\": \"%23SallyYatesIsAPatriot\",\n",
      "    \"tweet_volume\": 105190\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23LastWeekTonight\",\n",
      "    \"name\": \"#LastWeekTonight\",\n",
      "    \"query\": \"%23LastWeekTonight\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23WhyIHatedSchool\",\n",
      "    \"name\": \"#WhyIHatedSchool\",\n",
      "    \"query\": \"%23WhyIHatedSchool\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23KUWTK\",\n",
      "    \"name\": \"#KUWTK\",\n",
      "    \"query\": \"%23KUWTK\",\n",
      "    \"tweet_volume\": 11534\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23ShadesOfBlue\",\n",
      "    \"name\": \"#ShadesOfBlue\",\n",
      "    \"query\": \"%23ShadesOfBlue\",\n",
      "    \"tweet_volume\": 11041\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%234WordLetDowns\",\n",
      "    \"name\": \"#4WordLetDowns\",\n",
      "    \"query\": \"%234WordLetDowns\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23PlaylistatUniversal\",\n",
      "    \"name\": \"#PlaylistatUniversal\",\n",
      "    \"query\": \"%23PlaylistatUniversal\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23TheArrangement\",\n",
      "    \"name\": \"#TheArrangement\",\n",
      "    \"query\": \"%23TheArrangement\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23LastManOnEarth\",\n",
      "    \"name\": \"#LastManOnEarth\",\n",
      "    \"query\": \"%23LastManOnEarth\",\n",
      "    \"tweet_volume\": null\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23Next100Days\",\n",
      "    \"name\": \"#Next100Days\",\n",
      "    \"query\": \"%23Next100Days\",\n",
      "    \"tweet_volume\": 64398\n",
      "   },\n",
      "   {\n",
      "    \"promoted_content\": null,\n",
      "    \"url\": \"http://twitter.com/search?q=%23MasterChefMx\",\n",
      "    \"name\": \"#MasterChefMx\",\n",
      "    \"query\": \"%23MasterChefMx\",\n",
      "    \"tweet_volume\": 39744\n",
      "   }\n",
      "  ],\n",
      "  \"locations\": [\n",
      "   {\n",
      "    \"name\": \"United States\",\n",
      "    \"woeid\": 23424977\n",
      "   }\n",
      "  ]\n",
      " }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "print((json.dumps(us_trends[:2], indent=1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 4. Computing the intersection of two sets of trends"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('----------', 'world')\n",
      "#نغرد_عشان,#IntoTheBadlands,#RHOA,Joe West,#ProfileInCourage,#AltaTensión,#LaVozKids,#FodderScam,#سقوط_طالب_من_الدرج,#محمد_بن_سلمان_يزلزل_ايران,#SallyYatesIsAPatriot,#DeboDecir,#EntMEWomen,#pazartesi,#SinMiedoTuca,#SegundaComClubeSDV,#MagicianDatingProblems,#Independiente,#PurposeTourManila,#DafBama2017_SuperJunior,#حاجات_تفتح_نفسي,#NoMeGustaQue,#MLBxESPN,#DANCEDANCE,Erica Rivas,#OnceUponAMusical,#في_حبك_قلت,#iBribedKejriwal,#ShowtimeMAYForever,#تجمع_التربيه_الخاصه_بجده2,#무엇을_어디에_넣어줄지_말해보자,GW明け,#CorazonesRotosMC,#VenezuelaUrgente,#MasterChefMx,Guerra Mundial Z,#EnLiguillaVoy,Stranger Things,#AmericanRaceTNT,#JesusCristoEstáVivo,#MTVAwards,#FelizLunes,#kokkai,#MTVActAppVillalobos,Jake Arrieta,#BillionsSeasonFinale,#MejorHablar,#unite2017tokyo,#صباحات_الاهلي,#اجد_في_عيناك\n",
      "('----------', 'us')\n",
      "Kings of Leon,Wade Davis,#LastWeekTonight,#IntoTheBadlands,#RHOA,Joe West,Aaron Boone,#ProfileInCourage,#IWishINeverHadTo,#SallyYatesIsAPatriot,Eric Gordon,#LastManOnEarth,Chris Carter,#MagicianDatingProblems,#PlaylistatUniversal,Princess Diana,#UnitedShades,#TheLeftovers,#OnceUponAMusical,Bryant,#ShadesOfBlue,Big Sean,#Next100Days,Cheesecake Factory,#WhyIHatedSchool,#MasterChefMx,Jump Out The Window,Chasen Shreve,Stranger Things,#AmericanRaceTNT,#4WordLetDowns,#TheArrangement,#MTVAwards,Bobby Brown,#Billions,#Veep,Jake Arrieta,New MLB,Adam Warren,#KUWTK,Brian Duensing,Patrick Beverley\n",
      "('----------', 'san diego')\n",
      "Kings of Leon,Wade Davis,#LastWeekTonight,#IntoTheBadlands,#RHOA,#digitalhealth,Aaron Boone,#ProfileInCourage,#SiliconValleyHBO,#IWishINeverHadTo,#SallyYatesIsAPatriot,Eric Gordon,#LastManOnEarth,#MagicianDatingProblems,#PlaylistatUniversal,Princess Diana,#UnitedShades,#TheLeftovers,#OnceUponAMusical,Bryant,#GoCubsGo,#ShadesOfBlue,Big Sean,#Next100Days,Cheesecake Factory,#WhyIHatedSchool,#NW88JR,#Yankees,#MasterChefMx,Jump Out The Window,#LigaMXeng,Chasen Shreve,Stranger Things,#AmericanRaceTNT,#4WordLetDowns,#TheArrangement,#MTVAwards,Bobby Brown,#frenchelection,#Billions,#Veep,Jake Arrieta,New MLB,Adam Warren,#KUWTK,Brian Duensing,Patrick Beverley\n",
      "('==========', 'intersection of world and us')\n",
      "{'#MTVAwards', '#IntoTheBadlands', '#MagicianDatingProblems', '#RHOA', 'Joe West', '#MasterChefMx', '#ProfileInCourage', 'Jake Arrieta', '#OnceUponAMusical', '#SallyYatesIsAPatriot', 'Stranger Things', '#AmericanRaceTNT'}\n",
      "('==========', 'intersection of us and san-diego')\n",
      "{'Kings of Leon', 'Wade Davis', '#LastWeekTonight', '#IntoTheBadlands', '#RHOA', 'Aaron Boone', '#ProfileInCourage', '#IWishINeverHadTo', '#SallyYatesIsAPatriot', 'Eric Gordon', '#LastManOnEarth', '#MagicianDatingProblems', '#PlaylistatUniversal', 'Princess Diana', '#UnitedShades', '#TheLeftovers', '#OnceUponAMusical', 'Bryant', '#ShadesOfBlue', 'Big Sean', '#Next100Days', 'Cheesecake Factory', '#WhyIHatedSchool', '#MasterChefMx', 'Jump Out The Window', 'Chasen Shreve', 'Stranger Things', '#AmericanRaceTNT', '#4WordLetDowns', '#TheArrangement', '#MTVAwards', 'Bobby Brown', '#Billions', '#Veep', 'Jake Arrieta', 'New MLB', 'Adam Warren', '#KUWTK', 'Brian Duensing', 'Patrick Beverley'}\n"
     ]
    }
   ],
   "source": [
    "trends_set = {}\n",
    "trends_set['world'] = set([trend['name'] \n",
    "                        for trend in world_trends[0]['trends']])\n",
    "\n",
    "trends_set['us'] = set([trend['name'] \n",
    "                     for trend in us_trends[0]['trends']]) \n",
    "\n",
    "trends_set['san diego'] = set([trend['name'] \n",
    "                     for trend in local_trends[0]['trends']]) \n",
    "\n",
    "for loc in ['world','us','san diego']:\n",
    "    print(('-'*10,loc))\n",
    "    print((','.join(trends_set[loc])))\n",
    "\n",
    "print(( '='*10,'intersection of world and us'))\n",
    "print((trends_set['world'].intersection(trends_set['us'])))\n",
    "\n",
    "print(('='*10,'intersection of us and san-diego'))\n",
    "print((trends_set['san diego'].intersection(trends_set['us'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5. Collecting search results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the variable `q` to a trending topic, \n",
    "or anything else for that matter. The example query below\n",
    "was a trending topic when this content was being developed\n",
    "and is used throughout the remainder of this chapter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "q = '#MTVAwards' \n",
    "\n",
    "count = 100\n",
    "\n",
    "# See https://dev.twitter.com/docs/api/1.1/get/search/tweets\n",
    "\n",
    "search_results = twitter_api.search.tweets(q=q, count=count)\n",
    "\n",
    "statuses = search_results['statuses']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(statuses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Twitter often returns duplicate results, we can filter them out checking for duplicate texts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "all_text = []\n",
    "filtered_statuses = []\n",
    "for s in statuses:\n",
    "    if not s[\"text\"] in all_text:\n",
    "        filtered_statuses.append(s)\n",
    "        all_text.append(s[\"text\"])\n",
    "statuses = filtered_statuses     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "91"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(statuses)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['RT @Kaitiii: Found these crazy kids backstage at the #MTVAwards festival 🌸 @JackAllTimeLow @zackalltimelow @AlexAllTimeLow @riandawson http…',\n",
       " 'RT @blkgirlsrelate: shooketh #mtvawards https://t.co/H2CEtJpJ3G',\n",
       " 'RT @THR: Emma Watson: \"This is the first acting award in history that doesn\\'t separate nominees by their sex.\" #MTVAwards https://t.co/Adpo…',\n",
       " 'RT @__girlsgonewild: 7 พฤษภาคม : ภาพล่าสุด Cara Delevingne ที่งาน #MTVAwards\\xa0 จัดขึ้นในลอสแองเจลิส หล่อเข้มมาก ใจสั่นไปหมดแล้วพี่ร่า 😍😝 htt…',\n",
       " 'RT @13ReasonsWhyBRA: 📸 | Christian Navarro via Instagram com o Dylan Minnette.\\n#MTVAwards https://t.co/4zh4dPtnUr',\n",
       " 'RT @Camila_Cabello: GUYYS!! So excited to perform \"Hey Ma\" w @pitbull and @JBALVIN , see you tonight at 8/7c on @MTV! #MTVAwards https://t.…',\n",
       " 'RT @ohmxrvel: Ganó Stranger Things. La vida me sonríe  #MTVAwards',\n",
       " \"RT @MTVteenwolf: Look who I ran into! @tylergposey at the #mtvawards + #ToyotaCHR after party 😍 let's go make some mischief! https://t.co/n…\",\n",
       " 'RT @MTV: ICYMI -- @milliebbrown won Best Actor in a TV Show at the #MTVAwards: https://t.co/X4E1meCn9v https://t.co/ynSwF9VMlt',\n",
       " 'RT @federicodevito: Eu só queria estar nesse grupinho! #MTVAwards https://t.co/vs7sCcorNR',\n",
       " 'RT @MTV: This is too precious for words. #MTVAwards https://t.co/Xu7BpoZL84',\n",
       " 'RT @tomhsources: MY FAVS. #MTVAwards https://t.co/UYaluahLDS',\n",
       " 'RT @finnwolfhardbr: FOTO | Finn, @FelloWyatt e @jackgrazer4ya no Red Carpet do #MTVAwards https://t.co/4BKirZAFOg',\n",
       " 'RT @13ReasonsFans: Brandon Flynn was looking all types of fine tonight #MTVAwards https://t.co/8eZV5ZAVt4',\n",
       " 'RT @extratv: \"Hey guys, I\\'m sitting next to #SpiderMan!\" @zendaya @TomHolland1996 @SpiderManMovie \\n#MTVAwards https://t.co/GWFNgfB979',\n",
       " \"RT @LogoTV: .@AsiaKateDillon presenting the first acting award in history that doesn't base nominees on gender. #MTVAwards 👏 👏 https://t.co…\",\n",
       " 'The latest The Steve Lapa Daily! https://t.co/A1LfhXAAiQ Thanks to @1067theFan @cnnasiapr @chadjennings22 #mtvawards #moonlight',\n",
       " 'RT @ZacEfronFF: Millie Bobby Brown Wins Best Actor in a Show #MTVAwards \\n🍿 #ZacEfron @AADaddario @milliebbrown https://t.co/Cb3qveALQf',\n",
       " 'RT @13ReasonsWhyBRA: 📸 | Christian Navarro via Instagram com o Dylan Minnette e Katherine Langford.\\n#MTVAwards https://t.co/Bsfv8W9XGL',\n",
       " 'RT @THR: Emma Watson: \"This is the first acting award in history that doesn\\'t separate nominees by their sex.\" #MTVAwards https://t.co/Adpo…',\n",
       " 'RT @lacosacine: Una serie de ellos dos.\\n#MTVAwards https://t.co/ZLrF3M9cyN',\n",
       " 'RT @isabelamoner: carpet #MTVAwards #transformers https://t.co/g64Lv6RXyW',\n",
       " 'RT @MTV: Welcome to the #MTVAwards, @tylergposey 😈 https://t.co/eAPMvXU6KN',\n",
       " 'RT @latinregui: quiero empapelar toda mi casa con estas fotos\\n\\n#MTVAwards https://t.co/LCuvtvvnWU',\n",
       " 'RT @SMGomezDatos: El cast completo de #13ReasonsWhy en los MTV Movie Awards. 😱😍💖 #SelenaBBMAs #MTVAwards https://t.co/Z7wr67FKmt',\n",
       " 'Over 100 HQ Photos added of @milliebbrown at the #MTVAwards - https://t.co/bnfYvWpEAe #MillieBobbyBrown… https://t.co/tnoQxbQRuv',\n",
       " 'RT @MTV: Sliding into your DMs like... 😏 #MTVAwards @tylergposey https://t.co/wdEyJk0enW',\n",
       " 'RT @EdinahJane: DEUS ABENÇOE CAMILA CABELLO E SEU SANGUE LATINO \\nMIL VEZES AMÉM CAMILA CABELLO\\n#MTVAwards https://t.co/Y2oCK7TGfk',\n",
       " 'RT @TeenVogue: Because @Camila_Cabello is QUEEN. #MTVAwards https://t.co/8k1iFQis0f',\n",
       " 'RT @LostAtHogwarts: Congratulations to Emma Watson for winning the Best Actor in a Movie Award at #MTVAwards 😍😍https://t.co/r5Dn66yIor',\n",
       " 'RT @MTV: 💜 @shelleyhennig, @tylergposey, and @hollandroden have arrived at the #MTVAwards festival: https://t.co/GKnQbCFW23 https://t.co/Bv…',\n",
       " 'RT @elevenoah: LINDOS LINDOS LINDOS  #MTVAwards https://t.co/AzFRdnL84K',\n",
       " 'RT @krupnan: มีงาน #MTVAwards \\nงานศัพท์ พรมแดง ก็ต้องมา😆\\nStylish/Trendy ดูดีมีสไตล์ อินเทรนด์\\nGlam/Glamorous หรูเริ่ดเชิดอลัง!\\nSmoking hot…',\n",
       " 'RT @MTVLA: .@RuPaul ¡FELICIDADES! 💅🏼 #MTVAwards https://t.co/JdXY10gTVq',\n",
       " 'RT @sollgmz: um elenco é um elenco #mtvawards https://t.co/p52LyoYzJH',\n",
       " 'RT @MTV: “You’re the Spiderman!” Here’s your first look at a brand new @SpiderManMovie clip, exclusively from the #MTVAwards airing right n…',\n",
       " 'RT @gossipseriesfr: Tyler Posey au #MTVAwards https://t.co/RAM9CATi8M',\n",
       " 'RT @__girlsgonewild: Emma Watson ร่วมงาน #MTVAwards โดยเอ็มม่าได้รับรางวัล “นักแสดงยอดเยี่ยม” จากภาพยนตร์เรื่อง “Beauty And The Beast” ❤️ h…',\n",
       " 'RT @tthegingerninja: #MTVAwards HOLLAND RODEN E EMMA WATSON NELLA STESSA FOTO STO URLANDO https://t.co/yaK5YI0QiE',\n",
       " 'RT @MTV: “You’re the Spiderman!” Here’s your first look at a brand new @SpiderManMovie clip, exclusively from the #MTVAwards airing right n…',\n",
       " 'RT @fardore: Another day, Another slay @Zendaya 👏🏾 #MTVAwards https://t.co/VjLVTj8Br0',\n",
       " 'RT @CCabelloSP: Actuación completa de Hey Ma en los #MTVAwards https://t.co/Jat367VEft',\n",
       " 'RT @MTV: Shoutout to @EmmaWatson, @joshgad, and the Beauty and the Beast crew for winning Movie of the Year at the #MTVAwards https://t.co/…',\n",
       " 'RT @NYCSubwayQueen: I will never forget this #KristenStewart #MTVMovieAwards2017 #MTVAwards https://t.co/nRxrZsNKs4',\n",
       " 'RT @ShayItaly: I am specchless,Shay looks AMAZING! 😍 #MTVMovieAwards2017 #MTVAwards https://t.co/7fKs57wHUo',\n",
       " 'RT @HollandRItalia: 🎥 Holland nella sua Instagram Story #MTVAwards @hollandroden https://t.co/wuNSGOrH0S',\n",
       " 'RT @moviesmusic_th: ภาพของสาวสวย #ZaraLarsson จากงาน #MTVAwards\\xa0 ค่ะ แต่งตัวเก๋มากกก เมคอัพก็เป๊ะ แถมถ่ายรูปกับ Camila Cabello ด้วย https:/…',\n",
       " 'RT @MTVNews: ✨ @Camila_Cabello gives a sneak peek at her #MTVAwards rehearsal with @Pitbull and @JBALVIN 🔥 see their full performance tonig…',\n",
       " 'RT @MTV: Don’t turn off the lights during this brand new clip of \"IT\" from the #MTVAwards airing right now! https://t.co/2rTf7HcATO',\n",
       " 'RT @PopCrave: Congratulations to the cast and crew of \"Stranger Things\" for winning \"Show of the Year\" at the #MTVAwards! https://t.co/qFwH…',\n",
       " 'RT @LostAtHogwarts: Congratulations to Emma Watson for winning the Best Actor in a Movie Award at #MTVAwards 😍😍https://t.co/r5Dn66yIor',\n",
       " 'RT @gossipseriesfr: Miles Heizer au #MTVAwards. https://t.co/aV9c0QY7mN',\n",
       " 'RT @MTV: BE STILL, MY HEART ❤️ #MTVAwards https://t.co/oYKEY3v8jN',\n",
       " 'RT @gxmezkindness: Me he encariñado tanto con ellos #MTVAwards https://t.co/ja0o5ba5T6',\n",
       " 'RT @PopCrave: Congrats to the cast of \"The Fast and the Furious\" franchise as they won the MTV Generation Award at the #MTVAwards! https://…',\n",
       " '#13ReasonsWhy to watch @netflix RT @MTV: Look at my children at the #MTVAwards\\xa0!!! https://t.co/G6XFxYkTPo https://t.co/qjtbbRAcsi',\n",
       " 'RT @13ReasonsWhyBRA: 📸 | Tommy Dorfman via Instagram com a Katherine Langford. #MTVAwards https://t.co/MEF1qdIPGW',\n",
       " 'RT @heizermilesbr: 🎥| FOFO! Miles dando um oi para o Brasil no tapete vermelho do #MTVAwards ! \\n\\nQuem mais sentiu um orgulho de ser brasile…',\n",
       " 'RT @MTV: This is too precious for words. #MTVAwards https://t.co/Xu7BpoZL84',\n",
       " 'RT @lovesxbooks: \"El primer premio que no distingue a los nominados por su género\" \\nGracias por existir Emma Watson 👏🏽 #MTVAwards https://t…',\n",
       " 'RT @Variety: .@VinDiesel accepts the MTV Generation Award for #TheFastAndTheFurious franchise this Sunday at the #MTVAwards https://t.co/GJ…',\n",
       " 'RT @DylanOBrienPOL: NIE O DYLANIE | Tyler na #MTVAwards zachował się kochanie,bo pomimo deszczu, wyszedł do fanów i zrobił wiele świetnych…',\n",
       " \"RT @camilizws: para o caso de você ter perdido a performance de Hey Ma, i'm shook \\n\\nAMÉM CAMILA CABELLO\\n#MTVAwards\\nhttps://t.co/01qDVIEHcC\",\n",
       " 'RT @MTVteenwolf: 🐺🐾 #MTVAwards https://t.co/Ya7iK6JAv9',\n",
       " 'RT @RapUp: 😍😍\\xa0@BigSean and @JheneAiko at the #MTVAwards https://t.co/tAWtkv5MnV',\n",
       " 'RT @SA_Heroes: Congratulations to our funny-guy #TrevorNoah @TrevorNoah for winning #BestHost at the #MTVAwards',\n",
       " 'RT @AmyHargreavesNY: Awwww all my kiddies up there at\\n#MTVAwards @MTV  Warms the heart!! https://t.co/Hy5mlD3Swi',\n",
       " \"RT @enews: We know who's really running the show here. 💁 #MTVAwards https://t.co/pMghCqpQCC\",\n",
       " 'RT @TVGuide: #Logan’s Hugh Jackman &amp; Dafne Keen win best duo https://t.co/ebEqHJxRj9 #MTVAwards https://t.co/O3yxgLsTLx',\n",
       " 'RT @MTV: And your 2017 MTV Movie &amp; TV Awards winners are... https://t.co/43AhrhwFDt #MTVAwards https://t.co/UtIkSIwgr1',\n",
       " 'RT @CaraUpdateTh: ภาพล่าสุด Cara Delevingne และ Dane DeHaan เข้าร่วมงาน #MTVAwards ในลอสแองเจลิส 😍 ความหล่อของคู่นี้ https://t.co/ILaBf2EsBf',\n",
       " 'RT @mtvcanada: HEY MA 🔥 #MTVAwards @Camila_Cabello https://t.co/ecOsfe1xaL',\n",
       " 'RT @klauspoetic: HOLA NECESITO UNA AMBULANCIA GRACIAS #MTVAwards https://t.co/FijPHnoYuo',\n",
       " 'RT @MTV: Don’t turn off the lights during this brand new clip of \"IT\" from the #MTVAwards airing right now! https://t.co/2rTf7HcATO',\n",
       " 'Trevor Noah is winning in life bafethu what an inspiration . #MtvAwards',\n",
       " 'RT @CCabelloNews: Camila on the #MTVAWARDS red carpet #6 https://t.co/Aalpqz0eBg',\n",
       " 'RT @MTV: Let @dylanobrien guide you through a first look at Maze Runner: The Death Cure, exclusively for the #MTVAwards tonight at 8/7c! 💥…',\n",
       " 'RT @MTV: in love w/ these two!!!!!!!!! #mtvawards https://t.co/dZAmDlBfz3',\n",
       " 'RT @MTV: Seeing the @Stranger_Things cast with the @13ReasonsWhy cast at the #MTVAwards is too much for my heart to handle 💗 https://t.co/p…',\n",
       " 'RT @rhi_robertson: They look so amazing #MTVAwards https://t.co/g1KXBJvHsT',\n",
       " 'The 14 Best and Worst Dressed Celebs at the #MTVAwards https://t.co/WKJmBCEnIQ https://t.co/WOMj1HgjDy',\n",
       " 'RT @paisaregui: EL MEJOR CAST #MTVAwards https://t.co/13b7sP0vku',\n",
       " 'RT @MTV: “You’re the Spiderman!” Here’s your first look at a brand new @SpiderManMovie clip, exclusively from the #MTVAwards airing right n…',\n",
       " 'RT @idolsxpurpose: MIREN LO QUE ES EL DETRAS DE ESCENA DE THE DEATH CURE.\\nESTOY MUY EMOCIONADA Y NO PUEDO ESPERAR MÁS  #MTVAwards https://t…',\n",
       " 'RT @MTV: BE STILL, MY HEART ❤️ #MTVAwards https://t.co/oYKEY3v8jN',\n",
       " 'RT @BET: S/O to Daniel Kaluuya for winning the Next Generation Award at the #MTVAwards! \\n\\nvia @GIPHY https://t.co/lt89e4sLIo',\n",
       " 'RT @sollgmz: minha meta de amizade é ter amigos que fiquem tão felizes por mim como eles #mtvawards https://t.co/dkruTzkTXw',\n",
       " \"RT @LogoTV: .@AsiaKateDillon presenting the first acting award in history that doesn't base nominees on gender. #MTVAwards 👏 👏 https://t.co…\",\n",
       " \"RT @MTV: 🔥 @Zendaya's #MTVAwards look was EVERYTHING: https://t.co/0J7iurVBrO https://t.co/V4UFKgo1Pz\",\n",
       " 'RT @MTV: Look at my children at the #MTVAwards!!! https://t.co/GKnQbCFW23 https://t.co/vLzHNJM4Si',\n",
       " 'RT @seituttozayn: COME SI FA A NON AMARLI, È UN CAST MAGNIFICO #MTVAwards https://t.co/2GJv0d5ZO4',\n",
       " 'RT @MTV: Thank you for your beautiful Best Actor in a Movie acceptance speech at the #MTVAwards, @EmmaWatson 💖 https://t.co/iGN3nQQylL',\n",
       " \"RT @LogoTV: .@AsiaKateDillon presenting the first acting award in history that doesn't base nominees on gender. #MTVAwards 👏 👏 https://t.co…\",\n",
       " 'RT @camilizws: - vamos sair?\\n- ah não, eu preciso estudar\\n- te pago um lanche\\n-\\n\\n#MTVAwards https://t.co/MENtdfh6qS',\n",
       " 'RT @sollgmz: o elenco de 13 reasons why todo reunido foi a gente que pediu sim, que família linda #mtvawards https://t.co/AyJo0pGTde',\n",
       " 'RT @bestofsthings: STRANGER THINGS WINS SHOW OF THE YEAR #MTVAwards https://t.co/7bKKdoGGRv',\n",
       " \"RT @GeeksOfColor: Check out the new clip from 'Spider-Man Homecoming'. #MTVAwards https://t.co/SWOpvtDj1M\",\n",
       " 'RT @frogwheelers: nunca vou esquecer desse momento \\n#MTVAwards https://t.co/Y4k5unwgQ3',\n",
       " 'RT @PeytonList: Post rain #MTVAwards https://t.co/UnoizJRx8D',\n",
       " 'RT @MarkDice: #MTVAwards first presenter was a \"gender non-binary person\" who doesn\\'t identify as either male, nor female.  Also known as #…']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[s['text'] for s in search_results['statuses']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      " \"in_reply_to_screen_name\": null,\n",
      " \"lang\": \"en\",\n",
      " \"truncated\": false,\n",
      " \"metadata\": {\n",
      "  \"result_type\": \"recent\",\n",
      "  \"iso_language_code\": \"en\"\n",
      " },\n",
      " \"created_at\": \"Mon May 08 05:57:34 +0000 2017\",\n",
      " \"entities\": {\n",
      "  \"symbols\": [],\n",
      "  \"user_mentions\": [\n",
      "   {\n",
      "    \"id\": 20468965,\n",
      "    \"screen_name\": \"Kaitiii\",\n",
      "    \"indices\": [\n",
      "     3,\n",
      "     11\n",
      "    ],\n",
      "    \"name\": \"Kaitlyn Vella\",\n",
      "    \"id_str\": \"20468965\"\n",
      "   },\n",
      "   {\n",
      "    \"id\": 18378446,\n",
      "    \"screen_name\": \"JackAllTimeLow\",\n",
      "    \"indices\": [\n",
      "     75,\n",
      "     90\n",
      "    ],\n",
      "    \"name\": \"Jack Barakat\",\n",
      "    \"id_str\": \"18378446\"\n",
      "   },\n",
      "   {\n",
      "    \"id\": 29300489,\n",
      "    \"screen_name\": \"zackalltimelow\",\n",
      "    \"indices\": [\n",
      "     91,\n",
      "     106\n",
      "    ],\n",
      "    \"name\": \"Zachary Merrick\",\n",
      "    \"id_str\": \"29300489\"\n",
      "   },\n",
      "   {\n",
      "    \"id\": 18734884,\n",
      "    \"screen_name\": \"AlexAllTimeLow\",\n",
      "    \"indices\": [\n",
      "     107,\n",
      "     122\n",
      "    ],\n",
      "    \"name\": \"Alexander William\",\n",
      "    \"id_str\": \"18734884\"\n",
      "   },\n",
      "   {\n",
      "    \"id\": 22596730,\n",
      "    \"screen_name\": \"riandawson\",\n",
      "    \"indices\": [\n",
      "     123,\n",
      "     134\n",
      "    ],\n",
      "    \"name\": \"Rian Dawson\",\n",
      "    \"id_str\": \"22596730\"\n",
      "   }\n",
      "  ],\n",
      "  \"hashtags\": [\n",
      "   {\n",
      "    \"text\": \"MTVAwards\",\n",
      "    \"indices\": [\n",
      "     53,\n",
      "     63\n",
      "    ]\n",
      "   }\n",
      "  ],\n",
      "  \"urls\": []\n",
      " },\n",
      " \"favorite_count\": 0,\n",
      " \"favorited\": false,\n",
      " \"in_reply_to_status_id_str\": null,\n",
      " \"in_reply_to_user_id\": null,\n",
      " \"geo\": null,\n",
      " \"source\": \"<a href=\\\"http://twitter.com\\\" rel=\\\"nofollow\\\">Twitter Web Client</a>\",\n",
      " \"is_quote_status\": false,\n",
      " \"retweet_count\": 245,\n",
      " \"retweeted_status\": {\n",
      "  \"in_reply_to_screen_name\": null,\n",
      "  \"lang\": \"en\",\n",
      "  \"truncated\": true,\n",
      "  \"metadata\": {\n",
      "   \"result_type\": \"recent\",\n",
      "   \"iso_language_code\": \"en\"\n",
      "  },\n",
      "  \"created_at\": \"Sun May 07 21:36:47 +0000 2017\",\n",
      "  \"entities\": {\n",
      "   \"symbols\": [],\n",
      "   \"user_mentions\": [\n",
      "    {\n",
      "     \"id\": 18378446,\n",
      "     \"screen_name\": \"JackAllTimeLow\",\n",
      "     \"indices\": [\n",
      "      62,\n",
      "      77\n",
      "     ],\n",
      "     \"name\": \"Jack Barakat\",\n",
      "     \"id_str\": \"18378446\"\n",
      "    },\n",
      "    {\n",
      "     \"id\": 29300489,\n",
      "     \"screen_name\": \"zackalltimelow\",\n",
      "     \"indices\": [\n",
      "      78,\n",
      "      93\n",
      "     ],\n",
      "     \"name\": \"Zachary Merrick\",\n",
      "     \"id_str\": \"29300489\"\n",
      "    },\n",
      "    {\n",
      "     \"id\": 18734884,\n",
      "     \"screen_name\": \"AlexAllTimeLow\",\n",
      "     \"indices\": [\n",
      "      94,\n",
      "      109\n",
      "     ],\n",
      "     \"name\": \"Alexander William\",\n",
      "     \"id_str\": \"18734884\"\n",
      "    }\n",
      "   ],\n",
      "   \"hashtags\": [\n",
      "    {\n",
      "     \"text\": \"MTVAwards\",\n",
      "     \"indices\": [\n",
      "      40,\n",
      "      50\n",
      "     ]\n",
      "    }\n",
      "   ],\n",
      "   \"urls\": [\n",
      "    {\n",
      "     \"url\": \"https://t.co/5Pw7j1DbiZ\",\n",
      "     \"indices\": [\n",
      "      111,\n",
      "      134\n",
      "     ],\n",
      "     \"expanded_url\": \"https://twitter.com/i/web/status/861334017878052864\",\n",
      "     \"display_url\": \"twitter.com/i/web/status/8\\u2026\"\n",
      "    }\n",
      "   ]\n",
      "  },\n",
      "  \"possibly_sensitive\": false,\n",
      "  \"favorite_count\": 861,\n",
      "  \"favorited\": false,\n",
      "  \"in_reply_to_status_id_str\": null,\n",
      "  \"in_reply_to_user_id\": null,\n",
      "  \"geo\": null,\n",
      "  \"source\": \"<a href=\\\"http://twitter.com/download/iphone\\\" rel=\\\"nofollow\\\">Twitter for iPhone</a>\",\n",
      "  \"is_quote_status\": false,\n",
      "  \"retweet_count\": 245,\n",
      "  \"retweeted\": false,\n",
      "  \"in_reply_to_user_id_str\": null,\n",
      "  \"contributors\": null,\n",
      "  \"id_str\": \"861334017878052864\",\n",
      "  \"user\": {\n",
      "   \"lang\": \"en\",\n",
      "   \"is_translation_enabled\": false,\n",
      "   \"profile_sidebar_border_color\": \"000000\",\n",
      "   \"is_translator\": false,\n",
      "   \"created_at\": \"Mon Feb 09 21:26:58 +0000 2009\",\n",
      "   \"entities\": {\n",
      "    \"url\": {\n",
      "     \"urls\": [\n",
      "      {\n",
      "       \"url\": \"https://t.co/1AGQTOPksO\",\n",
      "       \"indices\": [\n",
      "        0,\n",
      "        23\n",
      "       ],\n",
      "       \"expanded_url\": \"http://Instagram.com/kaitlynvella\",\n",
      "       \"display_url\": \"Instagram.com/kaitlynvella\"\n",
      "      }\n",
      "     ]\n",
      "    },\n",
      "    \"description\": {\n",
      "     \"urls\": []\n",
      "    }\n",
      "   },\n",
      "   \"description\": \"social media manager @mtv, the end.\",\n",
      "   \"protected\": false,\n",
      "   \"profile_background_image_url_https\": \"https://pbs.twimg.com/profile_background_images/378800000171294397/bDvCchVV.jpeg\",\n",
      "   \"profile_image_url_https\": \"https://pbs.twimg.com/profile_images/849427592218324992/DZQ7llkP_normal.jpg\",\n",
      "   \"default_profile\": false,\n",
      "   \"time_zone\": \"Eastern Time (US & Canada)\",\n",
      "   \"id\": 20468965,\n",
      "   \"notifications\": false,\n",
      "   \"profile_background_image_url\": \"http://pbs.twimg.com/profile_background_images/378800000171294397/bDvCchVV.jpeg\",\n",
      "   \"geo_enabled\": true,\n",
      "   \"contributors_enabled\": false,\n",
      "   \"profile_sidebar_fill_color\": \"252429\",\n",
      "   \"profile_text_color\": \"666666\",\n",
      "   \"default_profile_image\": false,\n",
      "   \"profile_background_color\": \"000000\",\n",
      "   \"profile_image_url\": \"http://pbs.twimg.com/profile_images/849427592218324992/DZQ7llkP_normal.jpg\",\n",
      "   \"profile_use_background_image\": false,\n",
      "   \"has_extended_profile\": true,\n",
      "   \"profile_link_color\": \"430084\",\n",
      "   \"name\": \"Kaitlyn Vella\",\n",
      "   \"profile_background_tile\": false,\n",
      "   \"id_str\": \"20468965\",\n",
      "   \"utc_offset\": -14400,\n",
      "   \"friends_count\": 994,\n",
      "   \"favourites_count\": 18087,\n",
      "   \"verified\": true,\n",
      "   \"following\": false,\n",
      "   \"follow_request_sent\": false,\n",
      "   \"profile_banner_url\": \"https://pbs.twimg.com/profile_banners/20468965/1421809462\",\n",
      "   \"listed_count\": 193,\n",
      "   \"url\": \"https://t.co/1AGQTOPksO\",\n",
      "   \"screen_name\": \"Kaitiii\",\n",
      "   \"statuses_count\": 26137,\n",
      "   \"translator_type\": \"regular\",\n",
      "   \"location\": \"new york\",\n",
      "   \"followers_count\": 22980\n",
      "  },\n",
      "  \"id\": 861334017878052864,\n",
      "  \"place\": null,\n",
      "  \"text\": \"Found these crazy kids backstage at the #MTVAwards festival \\ud83c\\udf38 @JackAllTimeLow @zackalltimelow @AlexAllTimeLow\\u2026 https://t.co/5Pw7j1DbiZ\",\n",
      "  \"coordinates\": null,\n",
      "  \"in_reply_to_status_id\": null\n",
      " },\n",
      " \"in_reply_to_user_id_str\": null,\n",
      " \"contributors\": null,\n",
      " \"retweeted\": false,\n",
      " \"id_str\": \"861460043052466176\",\n",
      " \"user\": {\n",
      "  \"lang\": \"en\",\n",
      "  \"is_translation_enabled\": false,\n",
      "  \"profile_sidebar_border_color\": \"FFFFFF\",\n",
      "  \"is_translator\": false,\n",
      "  \"created_at\": \"Fri Nov 01 22:01:19 +0000 2013\",\n",
      "  \"entities\": {\n",
      "   \"url\": {\n",
      "    \"urls\": [\n",
      "     {\n",
      "      \"url\": \"https://t.co/ZE3SBnrMHG\",\n",
      "      \"indices\": [\n",
      "       0,\n",
      "       23\n",
      "      ],\n",
      "      \"expanded_url\": \"https://www.wattpad.com/user/ILove5SOS247\",\n",
      "      \"display_url\": \"wattpad.com/user/ILove5SOS\\u2026\"\n",
      "     }\n",
      "    ]\n",
      "   },\n",
      "   \"description\": {\n",
      "    \"urls\": []\n",
      "   }\n",
      "  },\n",
      "  \"description\": \"LOML is @lovemarissa143/ R5 1-27-16/ SLFL 9-2-16/ HV 3-7-17/ Dalton /HV 3/6/ The Vamps band acc + 3/4 /R5 band acc/ mom of 5SOFF\",\n",
      "  \"protected\": false,\n",
      "  \"profile_background_image_url_https\": \"https://pbs.twimg.com/profile_background_images/550548662611226626/7juUPicI.jpeg\",\n",
      "  \"profile_image_url_https\": \"https://pbs.twimg.com/profile_images/856946825412001792/__tJe8Yp_normal.jpg\",\n",
      "  \"default_profile\": false,\n",
      "  \"time_zone\": null,\n",
      "  \"id\": 2169144331,\n",
      "  \"notifications\": false,\n",
      "  \"profile_background_image_url\": \"http://pbs.twimg.com/profile_background_images/550548662611226626/7juUPicI.jpeg\",\n",
      "  \"geo_enabled\": true,\n",
      "  \"contributors_enabled\": false,\n",
      "  \"profile_sidebar_fill_color\": \"E3E2DE\",\n",
      "  \"profile_text_color\": \"634047\",\n",
      "  \"default_profile_image\": false,\n",
      "  \"profile_background_color\": \"EDECE9\",\n",
      "  \"profile_image_url\": \"http://pbs.twimg.com/profile_images/856946825412001792/__tJe8Yp_normal.jpg\",\n",
      "  \"profile_use_background_image\": true,\n",
      "  \"has_extended_profile\": true,\n",
      "  \"profile_link_color\": \"19CF86\",\n",
      "  \"name\": \"Sarah\",\n",
      "  \"profile_background_tile\": true,\n",
      "  \"id_str\": \"2169144331\",\n",
      "  \"utc_offset\": null,\n",
      "  \"friends_count\": 2608,\n",
      "  \"favourites_count\": 7329,\n",
      "  \"verified\": false,\n",
      "  \"following\": false,\n",
      "  \"follow_request_sent\": false,\n",
      "  \"profile_banner_url\": \"https://pbs.twimg.com/profile_banners/2169144331/1492411155\",\n",
      "  \"listed_count\": 49,\n",
      "  \"url\": \"https://t.co/ZE3SBnrMHG\",\n",
      "  \"screen_name\": \"ILove5SOS247\",\n",
      "  \"statuses_count\": 233548,\n",
      "  \"translator_type\": \"none\",\n",
      "  \"location\": \"cuddling with Michael\",\n",
      "  \"followers_count\": 3276\n",
      " },\n",
      " \"id\": 861460043052466176,\n",
      " \"place\": null,\n",
      " \"text\": \"RT @Kaitiii: Found these crazy kids backstage at the #MTVAwards festival \\ud83c\\udf38 @JackAllTimeLow @zackalltimelow @AlexAllTimeLow @riandawson http\\u2026\",\n",
      " \"coordinates\": null,\n",
      " \"in_reply_to_status_id\": null\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "# Show one sample search result by slicing the list...\n",
    "print(json.dumps(statuses[0], indent=1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "245\n",
      "False\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "# The result of the list comprehension is a list with only one element that\n",
    "# can be accessed by its index and set to the variable t\n",
    "t = statuses[0]\n",
    "#[ status for status in statuses \n",
    "#          if status['id'] == 316948241264549888 ][0]\n",
    "\n",
    "# Explore the variable t to get familiarized with the data structure...\n",
    "\n",
    "print(t['retweet_count'])\n",
    "print(t['retweeted'])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 6. Extracting text, screen names, and hashtags from tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      " \"RT @Kaitiii: Found these crazy kids backstage at the #MTVAwards festival \\ud83c\\udf38 @JackAllTimeLow @zackalltimelow @AlexAllTimeLow @riandawson http\\u2026\",\n",
      " \"RT @blkgirlsrelate: shooketh #mtvawards https://t.co/H2CEtJpJ3G\",\n",
      " \"RT @THR: Emma Watson: \\\"This is the first acting award in history that doesn't separate nominees by their sex.\\\" #MTVAwards https://t.co/Adpo\\u2026\",\n",
      " \"RT @__girlsgonewild: 7 \\u0e1e\\u0e24\\u0e29\\u0e20\\u0e32\\u0e04\\u0e21 : \\u0e20\\u0e32\\u0e1e\\u0e25\\u0e48\\u0e32\\u0e2a\\u0e38\\u0e14 Cara Delevingne \\u0e17\\u0e35\\u0e48\\u0e07\\u0e32\\u0e19 #MTVAwards\\u00a0 \\u0e08\\u0e31\\u0e14\\u0e02\\u0e36\\u0e49\\u0e19\\u0e43\\u0e19\\u0e25\\u0e2d\\u0e2a\\u0e41\\u0e2d\\u0e07\\u0e40\\u0e08\\u0e25\\u0e34\\u0e2a \\u0e2b\\u0e25\\u0e48\\u0e2d\\u0e40\\u0e02\\u0e49\\u0e21\\u0e21\\u0e32\\u0e01 \\u0e43\\u0e08\\u0e2a\\u0e31\\u0e48\\u0e19\\u0e44\\u0e1b\\u0e2b\\u0e21\\u0e14\\u0e41\\u0e25\\u0e49\\u0e27\\u0e1e\\u0e35\\u0e48\\u0e23\\u0e48\\u0e32 \\ud83d\\ude0d\\ud83d\\ude1d htt\\u2026\",\n",
      " \"RT @13ReasonsWhyBRA: \\ud83d\\udcf8 | Christian Navarro via Instagram com o Dylan Minnette.\\n#MTVAwards https://t.co/4zh4dPtnUr\"\n",
      "]\n",
      "[\n",
      " \"Kaitiii\",\n",
      " \"JackAllTimeLow\",\n",
      " \"zackalltimelow\",\n",
      " \"AlexAllTimeLow\",\n",
      " \"riandawson\"\n",
      "]\n",
      "[\n",
      " \"MTVAwards\",\n",
      " \"mtvawards\",\n",
      " \"MTVAwards\",\n",
      " \"MTVAwards\",\n",
      " \"MTVAwards\"\n",
      "]\n",
      "[\n",
      " \"RT\",\n",
      " \"@Kaitiii:\",\n",
      " \"Found\",\n",
      " \"these\",\n",
      " \"crazy\"\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "status_texts = [ status['text'] \n",
    "                 for status in statuses ]\n",
    "\n",
    "screen_names = [ user_mention['screen_name'] \n",
    "                 for status in statuses\n",
    "                     for user_mention in status['entities']['user_mentions'] ]\n",
    "\n",
    "hashtags = [ hashtag['text'] \n",
    "             for status in statuses\n",
    "                 for hashtag in status['entities']['hashtags'] ]\n",
    "\n",
    "# Compute a collection of all words from all tweets\n",
    "words = [ w \n",
    "          for t in status_texts \n",
    "              for w in t.split() ]\n",
    "\n",
    "# Explore the first 5 items for each...\n",
    "\n",
    "print(json.dumps(status_texts[0:5], indent=1))\n",
    "print(json.dumps(screen_names[0:5], indent=1)) \n",
    "print(json.dumps(hashtags[0:5], indent=1))\n",
    "print(json.dumps(words[0:5], indent=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 7. Creating a basic frequency distribution from the words in tweets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('RT', 87), ('#MTVAwards', 74), ('the', 42), ('at', 26), ('@MTV:', 17), ('a', 12), ('to', 12), ('for', 10), ('and', 9), ('in', 8)]\n",
      "\n",
      "[('MTV', 19), ('Camila_Cabello', 4), ('tylergposey', 4), ('13ReasonsWhyBRA', 3), ('Zendaya', 3), ('milliebbrown', 3), ('sollgmz', 3), ('pitbull', 2), ('__girlsgonewild', 2), ('MTVteenwolf', 2)]\n",
      "\n",
      "[('MTVAwards', 82), ('mtvawards', 7), ('13ReasonsWhy', 2), ('MTVMovieAwards2017', 2), ('KristenStewart', 1), ('ZacEfron', 1), ('Logan', 1), ('moonlight', 1), ('TheFastAndTheFurious', 1), ('transformers', 1)]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from collections import Counter\n",
    "\n",
    "for item in [words, screen_names, hashtags]:\n",
    "    c = Counter(item)\n",
    "    print(c.most_common()[:10]) # top 10\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 8. Create a prettyprint function to display tuples in a nice tabular format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def prettyprint_counts(label, list_of_tuples):\n",
    "    print(\"\\n{:^20} | {:^6}\".format(label, \"Count\"))\n",
    "    print(\"*\"*30)\n",
    "    for k,v in list_of_tuples:\n",
    "        print(\"{:20} | {:>6}\".format(k,v))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "        Word         | Count \n",
      "******************************\n",
      "RT                   |     87\n",
      "#MTVAwards           |     74\n",
      "the                  |     42\n",
      "at                   |     26\n",
      "@MTV:                |     17\n",
      "a                    |     12\n",
      "to                   |     12\n",
      "for                  |     10\n",
      "and                  |      9\n",
      "in                   |      8\n",
      "\n",
      "    Screen Name      | Count \n",
      "******************************\n",
      "MTV                  |     19\n",
      "Camila_Cabello       |      4\n",
      "tylergposey          |      4\n",
      "13ReasonsWhyBRA      |      3\n",
      "Zendaya              |      3\n",
      "milliebbrown         |      3\n",
      "sollgmz              |      3\n",
      "pitbull              |      2\n",
      "__girlsgonewild      |      2\n",
      "MTVteenwolf          |      2\n",
      "\n",
      "      Hashtag        | Count \n",
      "******************************\n",
      "MTVAwards            |     82\n",
      "mtvawards            |      7\n",
      "13ReasonsWhy         |      2\n",
      "MTVMovieAwards2017   |      2\n",
      "KristenStewart       |      1\n",
      "ZacEfron             |      1\n",
      "Logan                |      1\n",
      "moonlight            |      1\n",
      "TheFastAndTheFurious |      1\n",
      "transformers         |      1\n"
     ]
    }
   ],
   "source": [
    "for label, data in (('Word', words), \n",
    "                    ('Screen Name', screen_names), \n",
    "                    ('Hashtag', hashtags)):\n",
    "    \n",
    "    c = Counter(data)\n",
    "    prettyprint_counts(label, c.most_common()[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 9. Finding the most popular retweets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can build another `prettyprint` function to print entire tweets with their retweet count.\n",
    "\n",
    "We also want to split the text of the tweet in up to 3 lines, if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "row_template = \"{:^7} | {:^15} | {:50}\"\n",
    "def prettyprint_tweets(list_of_tuples):\n",
    "    print()\n",
    "    print(row_template.format(\"Count\", \"Screen Name\", \"Text\"))\n",
    "    print(\"*\"*30)\n",
    "    for count, screen_name, text in list_of_tuples:\n",
    "        print(row_template.format(count, screen_name, text[:50]))\n",
    "        if len(text) > 50:\n",
    "            print(row_template.format(\"\", \"\", text[50:100]))\n",
    "            if len(text) > 100:\n",
    "                print(row_template.format(\"\", \"\", text[100:]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      " Count  |   Screen Name   | Text                                                                                                                                        \n",
      "******************************\n",
      "   9967 |  Camila_Cabello | RT @Camila_Cabello: GUYYS!! So excited to perform \n",
      "        |                 | \"Hey Ma\" w @pitbull and @JBALVIN , see you tonight\n",
      "        |                 |  at 8/7c on @MTV! #MTVAwards https://t.…          \n",
      "   6871 |             MTV | RT @MTV: “You’re the Spiderman!” Here’s your first\n",
      "        |                 |  look at a brand new @SpiderManMovie clip, exclusi\n",
      "        |                 | vely from the #MTVAwards airing right n…          \n",
      "   5420 |             MTV | RT @MTV: This is too precious for words. #MTVAward\n",
      "        |                 | s https://t.co/Xu7BpoZL84                         \n",
      "   4923 |             MTV | RT @MTV: BE STILL, MY HEART ❤️ #MTVAwards https://\n",
      "        |                 | t.co/oYKEY3v8jN                                   \n",
      "   4835 |             MTV | RT @MTV: Let @dylanobrien guide you through a firs\n",
      "        |                 | t look at Maze Runner: The Death Cure, exclusively\n",
      "        |                 |  for the #MTVAwards tonight at 8/7c! 💥…           \n"
     ]
    }
   ],
   "source": [
    "retweets = [\n",
    "            # Store out a tuple of these three values ...\n",
    "            (status['retweet_count'], \n",
    "             status['retweeted_status']['user']['screen_name'],\n",
    "             status['text'].replace(\"\\n\",\"\\\\\")) \n",
    "            \n",
    "            # ... for each status ...\n",
    "            for status in statuses \n",
    "            \n",
    "            # ... so long as the status meets this condition.\n",
    "                if 'retweeted_status' in status\n",
    "           ]\n",
    "\n",
    "# Slice off the first 5 from the sorted results and display each item in the tuple\n",
    "\n",
    "prettyprint_tweets(sorted(retweets, reverse=True)[:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}