forked from ropensci-archive/rtweet
-
Notifications
You must be signed in to change notification settings - Fork 0
/
README.Rmd
371 lines (279 loc) · 13.8 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, echo = FALSE}
knitr::opts_chunk$set(
eval = FALSE,
collapse = TRUE,
comment = "#>",
fig.path = "README-"
)
```
# rtweet <img src="man/figures/logo.png" width="160px" align="right" />
[](https://travis-ci.org/ropensci/rtweet)
[](https://cran.r-project.org/package=rtweet)
[](https://codecov.io/gh/ropensci/rtweet?branch=master)
[](https://zenodo.org/badge/latestdoi/64161359)
[](https://github.com/ropensci/software-review/issues/302)
[](https://www.repostatus.org/#active)
[](https://www.tidyverse.org/lifecycle/#maturing)
[](https://doi.org/10.21105/joss.01829)
R client for accessing Twitter's REST and stream APIs. Check out the [rtweet package documentation website](https://rtweet.info).
### Package Functionality
There are several R packages for interacting with Twitter's APIs. See how {rtweet} compares to these others in the chart below.
| Task | [rtweet](https://github.com/ropensci/rtweet) | [twitteR](https://github.com/geoffjentry/twitteR) | [streamR](https://github.com/pablobarbera/streamR) | [RTwitterAPI](https://github.com/joyofdata/RTwitterAPI) |
|------------------------------|--------|---------|---------|-------------|
| Available on CRAN | ✅ | ✅ | ✅ | ❌ |
| Updated since 2016 | ✅ | ❌ | ✅ | ❌ |
| Non-'developer' access | ✅ | ❌ | ❌ | ❌ |
| Extended tweets (280 chars) | ✅ | ❌ | ✅ | ❌ |
| Parses JSON data | ✅ | ✅ | ✅ | ❌ |
| Converts to data frames | ✅ | ✅ | ✅ | ❌ |
| Automated pagination | ✅ | ❌ | ❌ | ❌ |
| Search tweets | ✅ | ✅ | ❌ | ❓ |
| Search users | ✅ | ❌ | ❌ | ❓ |
| Stream sample | ✅ | ❌ | ✅ | ❌ |
| Stream keywords | ✅ | ❌ | ✅ | ❌ |
| Stream users | ✅ | ❌ | ✅ | ❌ |
| Get friends | ✅ | ✅ | ❌ | ✅ |
| Get timelines | ✅ | ✅ | ❌ | ❓ |
| Get mentions | ✅ | ✅ | ❌ | ❓ |
| Get favorites | ✅ | ✅ | ❌ | ❓ |
| Get trends | ✅ | ✅ | ❌ | ❓ |
| Get list members | ✅ | ❌ | ❌ | ❓ |
| Get list memberships | ✅ | ❌ | ❌ | ❓ |
| Get list statuses | ✅ | ❌ | ❌ | ❓ |
| Get list subscribers | ✅ | ❌ | ❌ | ❓ |
| Get list subscriptions | ✅ | ❌ | ❌ | ❓ |
| Get list users | ✅ | ❌ | ❌ | ❓ |
| Lookup collections | ✅ | ❌ | ❌ | ❓ |
| Lookup friendships | ✅ | ✅ | ❌ | ❓ |
| Lookup statuses | ✅ | ✅ | ❌ | ❓ |
| Lookup users | ✅ | ✅ | ❌ | ❓ |
| Get retweeters | ✅ | ✅ | ❌ | ❓ |
| Get retweets | ✅ | ✅ | ❌ | ❓ |
| Post tweets | ✅ | ✅ | ❌ | ❌ |
| Post favorite | ✅ | ❌ | ❌ | ❌ |
| Post follow | ✅ | ❌ | ❌ | ❌ |
| Post messsage | ✅ | ✅ | ❌ | ❌ |
| Post mute | ✅ | ❌ | ❌ | ❌ |
| Premium 30 day | ✅ | ❌ | ❌ | ❌ |
| Premium full archive | ✅ | ❌ | ❌ | ❌ |
| Run package tests | ✅ | ❌ | ❌ | ❌ |
## Responsible use
**{{rtweet}}** should be used in strict accordance with Twitter's [developer terms](https://developer.twitter.com/en/developer-terms/more-on-restricted-use-cases).
## Installation
To get the current released version from CRAN:
```{r}
## install rtweet from CRAN
install.packages("rtweet")
## load rtweet package
library(rtweet)
```
To get the current development version from Github:
```{r}
## install remotes package if it's not already
if (!requireNamespace("remotes", quietly = TRUE)) {
install.packages("remotes")
}
## install dev version of rtweet from github
remotes::install_github("ropensci/rtweet")
## load rtweet package
library(rtweet)
```
## Usage
All you need is a **Twitter account** (user name and password) and you can be up in running in minutes!
Simply send a request to Twitter's API (with a function like `search_tweets()`, `get_timeline()`, `get_followers()`, `get_favorites()`, etc.) during an interactive session of R, authorize the embedded **`rstats2twitter`** app (approve the browser popup), and your token will be created and saved/stored (for future sessions) for you!
### API authorization
All users must be authorized to interact with Twitter's APIs. To become authorized, simply use a function like `search_tweets()`, `get_timeline()`, `get_followers()`, or `get_favorites()` in an interactive session an authorize via web browser popup on behalf of your Twitter account!
**It is no longer necessary to obtain a developer account and create your own Twitter application to use Twitter's API.** You may still choose to do this (gives you more stability and permissions; see the table at the bottom of this section), but **{rtweet}** should work out of the box assuming (a) you are working in an interactive/live session of R and (b) you have installed the [**{httpuv}**](https://github.com/rstudio/httpuv) package.
+ If you still want to apply for a developer account and create your own application, see the `auth` vignette (or the API authorization section below) for additional instructions: [https://rtweet.info/articles/auth.html](https://rtweet.info/articles/auth.html).
| Task | rstats2twitter | user-app |
|------------------------------|----------------|----------|
| Work interactively | ✅ | ✅ |
| Search/lookup tweets/users | ✅ | ✅ |
| Get friends/followers | ✅ | ✅ |
| Get timelines/favorites | ✅ | ✅ |
| Get lists/collections | ✅ | ✅ |
| Post tweets | ❌ | ✅ |
| Run package tests | ❌ | ✅ |
| Use Bearer token | ❌ | ✅ |
| Read/Write Direct Messages | ❌ | ✅ |
### Vignettes
[Obtaining and using Twitter API tokens](https://rtweet.info/articles/auth.html)
```{r}
## quick overview of rtweet functions
vignette("auth", package = "rtweet")
```
[Quick overview of rtweet package](https://rtweet.info/articles/intro.html)
```{r}
## quick overview of rtweet functions
vignette("intro", package = "rtweet")
```
[Live streaming tweets data](https://rtweet.info/articles/stream.html)
```{r}
## working with the stream
vignette("stream", package = "rtweet")
```
[Troubleshooting common rtweet problems](https://rtweet.info/articles/FAQ.html)
```{r}
## working with the stream
vignette("FAQ", package = "rtweet")
```
### Package features
#### Search tweets
Search for up to 18,000 (non-retweeted) tweets containing the rstats hashtag.
```{r}
## search for 18000 tweets using the rstats hashtag
rt <- search_tweets(
"#rstats", n = 18000, include_rts = FALSE
)
```
Quickly visualize frequency of tweets over time using `ts_plot()`.
```{r}
## plot time series of tweets
rt %>%
ts_plot("3 hours") +
ggplot2::theme_minimal() +
ggplot2::theme(plot.title = ggplot2::element_text(face = "bold")) +
ggplot2::labs(
x = NULL, y = NULL,
title = "Frequency of #rstats Twitter statuses from past 9 days",
subtitle = "Twitter status (tweet) counts aggregated using three-hour intervals",
caption = "\nSource: Data collected from Twitter's REST API via rtweet"
)
```
Twitter rate limits cap the number of search results returned to
18,000 every 15 minutes. To request more than that, simply set
`retryonratelimit = TRUE` and rtweet will wait for rate limit
resets for you.
```{r}
## search for 250,000 tweets containing the word data
rt <- search_tweets(
"data", n = 250000, retryonratelimit = TRUE
)
```
Search by geo-location---for example, find 10,000 tweets in the English
language sent from the United States. *Note: `lookup_coords()` requires users have [a Google API key](https://developers.google.com/maps/documentation/javascript/tutorial)*
```{r}
## search for 10,000 tweets sent from the US
rt <- search_tweets(
"lang:en", geocode = lookup_coords("usa"), n = 10000
)
## create lat/lng variables using all available tweet and profile geo-location data
rt <- lat_lng(rt)
## plot state boundaries
par(mar = c(0, 0, 0, 0))
maps::map("state", lwd = .25)
## plot lat and lng points onto state map
with(rt, points(lng, lat, pch = 20, cex = .75, col = rgb(0, .3, .7, .75)))
```
#### Stream tweets
Randomly sample (approximately 1%) from the live stream of all tweets.
```{r}
## random sample for 30 seconds (default)
rt <- stream_tweets("")
```
Stream all geo enabled tweets from London for 60 seconds.
```{r}
## stream tweets from london for 60 seconds
rt <- stream_tweets(lookup_coords("london, uk"), timeout = 60)
```
Stream all tweets mentioning realDonaldTrump or Trump for a week.
```{r}
## stream london tweets for a week (60 secs x 60 mins * 24 hours * 7 days)
stream_tweets(
"realdonaldtrump,trump",
timeout = 60 * 60 * 24 * 7,
file_name = "tweetsabouttrump.json",
parse = FALSE
)
## read in the data as a tidy tbl data frame
djt <- parse_stream("tweetsabouttrump.json")
```
#### Get friends
Retrieve a list of all the accounts a **user follows**.
```{r}
## get user IDs of accounts followed by CNN
cnn_fds <- get_friends("cnn")
## lookup data on those accounts
cnn_fds_data <- lookup_users(cnn_fds$user_id)
```
##### Get followers
Retrieve a list of the **accounts following** a user.
```{r}
## get user IDs of accounts following CNN
cnn_flw <- get_followers("cnn", n = 75000)
## lookup data on those accounts
cnn_flw_data <- lookup_users(cnn_flw$user_id)
```
Or if you really want ALL of their followers:
```{r}
## how many total follows does cnn have?
cnn <- lookup_users("cnn")
## get them all (this would take a little over 5 days)
cnn_flw <- get_followers(
"cnn", n = cnn$followers_count, retryonratelimit = TRUE
)
```
#### Get timelines
Get the most recent 3,200 tweets from cnn, BBCWorld, and foxnews.
```{r}
## get user IDs of accounts followed by CNN
tmls <- get_timelines(c("cnn", "BBCWorld", "foxnews"), n = 3200)
## plot the frequency of tweets for each user over time
tmls %>%
dplyr::filter(created_at > "2017-10-29") %>%
dplyr::group_by(screen_name) %>%
ts_plot("days", trim = 1L) +
ggplot2::geom_point() +
ggplot2::theme_minimal() +
ggplot2::theme(
legend.title = ggplot2::element_blank(),
legend.position = "bottom",
plot.title = ggplot2::element_text(face = "bold")) +
ggplot2::labs(
x = NULL, y = NULL,
title = "Frequency of Twitter statuses posted by news organization",
subtitle = "Twitter status (tweet) counts aggregated by day from October/November 2017",
caption = "\nSource: Data collected from Twitter's REST API via rtweet"
)
```
#### Get favorites
Get the 3,000 most recently favorited statuses by JK Rowling.
```{r}
jkr <- get_favorites("jk_rowling", n = 3000)
```
#### Search users
Search for 1,000 users with the rstats hashtag in their profile bios.
```{r}
## search for users with #rstats in their profiles
usrs <- search_users("#rstats", n = 1000)
```
#### Get trends
Discover what's currently trending in San Francisco.
```{r}
sf <- get_trends("san francisco")
```
#### Post actions
- Posting (tweeting from R console) or reading direct messages require
additional permissions
- If you'd like to post Twitter statuses, follow or unfollow accounts,
and/or read your direct messages, you'll need to create your own
Twitter app
- To create your own Twitter app, follow the instructions in the
authorization vignette on [obtaining and using access tokens](https://rtweet.info/articles/auth.html)
## Contact
Communicating with Twitter's APIs relies on an internet connection,
which can sometimes be inconsistent. With that said, if you encounter
an obvious bug for which there is not already an active
[issue](https://github.com/ropensci/rtweet/issues), please [create a
new issue](https://github.com/ropensci/rtweet/issues/new) with all
code used (preferably a reproducible example) on Github.
[](https://ropensci.org)