Skip to content
This repository has been archived by the owner on May 14, 2019. It is now read-only.

outdated - small utility to extract members from twitter lists

License

Notifications You must be signed in to change notification settings

Mango-information-systems/twiLiMe

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

twiLiMe - extracting members from twitter lists

Small node.js utility to extract all twitter usernames contained from twitter lists.

features

  • Extracts all twitter usernames from many lists into a single txt file.
  • No twitter authentication required - only public lists are browsable
  • Resumes extraction when interrupted by twitter hourly rate limiting.

Installation

  1. Get the source code from github:

       git clone git://github.com/mef/twiLiMe.git 
       cd twiLiMe
     
  2. Install dependency: request node.js module

       npm install -d
     
  3. You're ready to go

How to use

  1. write the names of the lists you want to process inside lists.json file. All you need is the name of list owner and name of the list. Example file:

    {
      "lists":[
        {
          "owner": "Support",
          "listName": "international-support"
        },
        {
          "owner": "crystal",
          "listName": "twizzles"
        }
      ]
    }
  2. execute the application :

       node twiLiMe.js
     
  3. read output from the command line. Execution is completed successfully in case the following output is displayed: members from all lists have been extracted successfully :) ... results were saved!

The file results.txt stores the list of usernames.

guidelines

During the processing, the lists.json file will be updated to store the progress of the extraction. keep a copy of the original file in case you want to re-use it for another purpose

Twitter rate limit

In case you want to extract large amount of data, you will reach the twitter API's hourly rate limit: https://dev.twitter.com/docs/rate-limiting.

In such case re-run the previous step every hour to resume the extraction.

You can use CRON or another scheduling tool to do this.

## Dependencies

This app uses Mikeal's request library. Obviously, you should also have node.js working on your machine.

## Status Under development. Basic tests have been performed and were successful under nodejs. 0.4.12.

JSON response from twitter sometimes fails to parse correctly.

To do

support user authentication to increase rate limit and get able to extract from private lists to which the authenticated user has access

License

This software is licensed under MIT License, please read LICENSE file. Do not hesitate to ask in case you want to use it under terms of another open source license.

About

outdated - small utility to extract members from twitter lists

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published