Skip to content

Commit

Permalink
V3 structural updates
Browse files Browse the repository at this point in the history
  • Loading branch information
debruine committed Feb 15, 2024
1 parent c50ba51 commit 7ca4fd4
Show file tree
Hide file tree
Showing 78 changed files with 8,067 additions and 6,700 deletions.
138 changes: 5 additions & 133 deletions 05-summary.qmd → 04-summary.qmd
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Data Summaries {#sec-summary}

THIS CHAPTER IS CURRENTLY UNDERGOING REVISION

## Intended Learning Outcomes {#sec-ilo-summary .unnumbered}

* Be able to summarise data by groups
Expand All @@ -8,51 +10,19 @@

## Walkthrough video {#sec-walkthrough-summary .unnumbered}

There is a walkthrough video of this chapter available via [Echo360.](https://echo360.org.uk/media/6783dc21-2603-4579-bc19-5ae971638e85/public) Please note that there may have been minor edits to the book since the video was recorded. Where there are differences, the book should always take precedence.
There is a walkthrough video of this chapter available via [Echo360](). Please note that there may have been minor edits to the book since the video was recorded. Where there are differences, the book should always take precedence.

## Set-up {#sec-setup-summary}

First, create a new project for the work we'll do in this chapter named `r path("05-summary")`. Second, download the data for this chapter (<a href="data/ncod_tweets.rds" download>ncod_tweets.rds</a>) and save it in your project data folder. Finally, open and save and new R Markdown document named `summary.Rmd`, delete the welcome text and load the required packages for this chapter.
First, create a new project for the work we'll do in this chapter named `r path("04-summary")`. Second, download the data for this chapter (<a href="data/ncod_tweets.rds" download>ncod_tweets.rds</a>) and save it in your project data folder. Finally, open and save and new R Markdown document named `summary.Rmd`, delete the welcome text and load the required packages for this chapter.

```{r setup-summary, message=FALSE, verbatim="r setup, include=FALSE"}
library(tidyverse) # data wrangling functions
library(rtweet) # for searching tweets
library(kableExtra) # for nice tables
```

Download the [Data transformation cheat sheet](https://raw.githubusercontent.com/rstudio/cheatsheets/main/data-transformation.pdf).

## Social media data

In this chapter we're going to analyse social media data, specifically data from Twitter. There are two broad types of data you can obtain from Twitter; data scraped from Twitter using purpose-built packages such as <pkg>rtweet</pkg>, and data provided via [Twitter Analytics](https://analytics.twitter.com/) for any accounts for which you have access.

**This chapter was written in late 2021 and a lot of things have changed at Twitter since then. Importantly, starting 9th February 2023, Twitter have announced that access to their API will no longer be free, which means that the search functions below will not work unless you have a paid account. We have provided the data you need for this chapter so it is not a problem for your learning and in version 3 of ADS, we will remove any reliance on Twitter data.**

For this chapter, we'll use data scraped from Twitter using <pkg>rtweet</pkg>. In order to use these functions, you need to have a Twitter account. Don't worry if you don't have one; we'll provide the data in the examples below for you.

<pkg>rtweet</pkg> has a lot of flexibility, for example, you can search for tweets that contain a certain hashtag or word, tweets by a specific user, or tweets that meet certain conditions like location or whether the user is verified.

For the dataset for this chapter, we used the `search_tweets()` function to find the last 30K tweets with the hashtag [#NationalComingOutDay](https://en.wikipedia.org/wiki/National_Coming_Out_Day). This is mainly interesting around October 11th (the date of National Coming Out Day), so we've provided the relevant data for you that we scraped at that time.

If you have a Twitter account, you can complete this chapter using your own data and any hashtag that interests you. When you run the `search_tweets()` function, you will be asked to sign in to your Twitter account.

```{r, eval = FALSE}
tweets <- search_tweets(q = "#NationalComingOutDay",
n = 30000,
include_rts = FALSE)
```

### R objects

If you're working with live social media data, every time you run a query it's highly likely you will get a different set of data as new tweets are added. Additionally, the Twitter API places limits on how much data you can download and searches are limited to data from the last 6-9 days. Consequently, it can be useful to save the results of your initial search. `saveRDS` is a useful function that allows you to save any object in your environment to disk.

```{r eval = FALSE}
saveRDS(tweets, file = "data/ncod_tweets.rds")
```

After you run `search_tweets()` and save the results, set that code chunk to `eval = FALSE` or comment out that code so your script doesn't run the search and overwrite your saved data every time you knit it.

To load an `.rds` file, you can use the `readRDS()` function. If you don't have access to a Twitter account, or to ensure that you get the same output as the rest of this chapter, you can download <a href="data/ncod_tweets.rds" download>ncod_tweets.rds</a> and load it using this function.

```{r}
tweets <- readRDS("data/ncod_tweets.rds")
Expand Down Expand Up @@ -180,28 +150,6 @@ tweet_summary <- tweets %>% # start with the object tweets and then

Notice that `summarise()` no longer needs the first argument to be the data table, it is pulled in from the pipe. The power of the pipe may not be obvious now, but it will soon prove its worth.

### Inline coding

To insert those values into the text of a report you can use inline coding. First. we'll create another set of objects that contain the first and last date of the tweets in our dataset. `format()` formats the dates to day/month/year.

```{r}
date_from <- tweet_summary$min_date %>%
format("%d %B, %Y")
date_to <- tweet_summary$max_date %>%
format("%d %B, %Y")
```

Then you can insert values from these objects and the tables you created with `summarise()` using inline R (note the dollar sign notation to get the value of the `n` column from the table `tweet_summary`).

```{verbatim, lang="md"}
There were `r tweet_summary$n` tweets between `r date_from` and `r date_to`.
```

Knit your Markdown to see how the variables inside the inline code get replaced by their values.

> There were `r tweet_summary$n` tweets between `r date_from` and `r date_to`.
Ok, let's get back on track.

## Counting

Expand Down Expand Up @@ -236,32 +184,6 @@ tweets %>% count(is_quote, is_retweet)
`r mcq1`
:::

### Inline coding 2

Let's do another example of inline coding that writes up a summary of the most prolific tweeters to demonstrate a few additional functions. First, we need to create some additional objects to use with inline R:

* `nrow()` simply counts the number of rows in a dataset so if you have one user/participant/customer per row, this is an easy way to do a head count.
* `slice()` chooses a particular row of data, in this case the first row. Because we sorted our data, this will therefore be the user with the most tweets.
* `pull()` pulls out a single variable.
* The combination of `slice()` and `pull()` allows you to choose a single observation from a single variable.

```{r}
unique_users <- nrow(tweets_per_user)
most_prolific <- slice(tweets_per_user, 1) %>%
pull(screen_name)
most_prolific_n <- slice(tweets_per_user, 1) %>%
pull(n)
```

Then add the inline code to your report...

```{verbatim, lang="md"}
There were `r unique_users` unique accounts tweeting about #NationalComingOutDay. `r most_prolific` was the most prolific tweeter, with `r most_prolific_n` tweets.
```

...and knit your Markdown to see the output:

There were `r unique_users` unique accounts tweeting about #NationalComingOutDay. `r most_prolific` was the most prolific tweeter, with `r most_prolific_n` tweets.

## Grouping {#sec-grouping}

Expand Down Expand Up @@ -376,54 +298,6 @@ mcq1 <- c(answer = "`tweets %>% group_by(source) %>% filter(n() >= 10)`",
`r mcq1`
:::

### Inline coding 3

There's a huge amount of data reported for each tweet, including things like the URLs of the tweets and any media attached to them. This means we can produce output like the below reproducibly and using inline coding.

```{r echo = FALSE}
orig <- filter(most_fav, !is_quote)
quote <- filter(most_fav, is_quote)
```

The most favourited `r orig$favorite_count` original tweet was by [`r orig$screen_name`](`r orig$status_url`):

--------------------------------------------------

> `r orig$text`
![](`r orig$ext_media_url`)

------------------------------------------------

To produce this, first we split `most_fav`, so that we have one object that contains the data from the original tweet and one object that contains the data from the quote tweet.

```{r, echo = TRUE}
orig <- filter(most_fav,is_quote == FALSE)
quote <- filter(most_fav,is_quote == TRUE)
```

The inline code is then as follows:

```{verbatim, lang="md"}
The most favourited `r orig$favorite_count` original tweet was by [`r orig$screen_name`](`r orig$status_url`):
--------------------------------------------------
> `r orig$text`
![](`r orig$ext_media_url`)
--------------------------------------------------
```

This is quite complicated so let's break it down.

* The first bit of inline coding is fairly standard and is what you have used before.
* The second bit of inline coding inserts a URL. The content of the `[]` is the text that will be displayed. The content of `()` is the underlying URL. In both cases, the content is being pulled from the dataset. In this case, the text is `screen_name` and `status_url` links to the tweet.
* The line of dashes creates the solid line in the knitted output.
* The `>` symbol changes the format to a block quote.
* The image is then included using the format `![](url)`, which is an alternative method of including images in Markdown.


## Exercises

Expand All @@ -445,7 +319,5 @@ glossary_table(as_kable = FALSE) |>
## Further resources {#sec-resources-summary}

* [Data transformation cheat sheet](https://raw.githubusercontent.com/rstudio/cheatsheets/main/data-transformation.pdf)
* [Chapter 5: Data Transformation ](http://r4ds.had.co.nz/transform.html) in *R for Data Science*
* [Intro to rtweet](https://docs.ropensci.org/rtweet/articles/rtweet.html)
* [Tidy Text](https://www.tidytextmining.com/index.html)
* [Chapter 5: Data Transformation ](https://r4ds.hadley.nz/data-transform.html) in *R for Data Science*
* [kableExtra vignettes](https://cran.r-project.org/web/packages/kableExtra/vignettes/awesome_table_in_html.html)
File renamed without changes.
14 changes: 14 additions & 0 deletions 06-ai.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# AI {#sec-ai}

CONTENT COMING SOON!

## Intended Learning Outcomes {#sec-ilo-ai .unnumbered}

* A
* B

## Walkthrough video {#sec-walkthrough-ai .unnumbered}

There is a walkthrough video of this chapter available via [Echo360](). Please note that there may have been minor edits to the book since the video was recorded. Where there are differences, the book should always take precedence.

## Set-up {#sec-setup-ai}
2 changes: 1 addition & 1 deletion 12-license.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ Some material from this book was adapted from @reprores2 and @nordmann_2021.

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.6365077.svg)](https://doi.org/10.5281/zenodo.6365077)

Nordmann, E. & DeBruine, L. (2023) Applied Data Skills. v2.0. Retrieved from https://psyteachr.github.io/ads-v2/ doi: [10.5281/zenodo.6365077](https://doi.org/10.5281/zenodo.6365077)
Nordmann, E. & DeBruine, L. (2023) Applied Data Skills. v3.0. Retrieved from https://psyteachr.github.io/ads-v3/ doi: [10.5281/zenodo.6365077](https://doi.org/10.5281/zenodo.6365077)

6 changes: 3 additions & 3 deletions CITATION
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ authors:
given-names: "Lisa"
orcid: "https://orcid.org/0000-0002-7523-5539"
title: "Applied Data Skills"
version: 2.0
date-released: 2023-01-01
url: "https://github.com/psyteachr/ads-v2/"
version: 3.0
date-released: 2024-03-01
url: "https://github.com/psyteachr/ads-v3/"
7 changes: 3 additions & 4 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: ads-v2
Package: ads-v3
Title: Applied Data Skills
Version: 2.0
Version: 3.0
Authors@R:
c(
person(
Expand Down Expand Up @@ -52,8 +52,7 @@ Imports:
ggbump,
waffle,
DT,
showtext,
spotifyr
showtext
Remotes:
psyteachr/glossary,
ropensci/rnaturalearthhires
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Applied Data Skills (v2)
# Applied Data Skills (v3)

<!-- badges: start -->
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.6365077.svg)](https://doi.org/10.5281/zenodo.6365077)
<!-- badges: end -->

The 2023 version of the course book for the Applied Data Skills microcredential course in the School of Psychology and Neuroscience at the University of Glasgow.
The 2024 version of the course book for the Applied Data Skills microcredential course in the School of Psychology and Neuroscience at the University of Glasgow.

This repository contains the source files for the interactive textbook:

Nordmann, E. & DeBruine, L. (2023). Applied Data Skills. Version 1.0. Retrieved from https://psyteachr.github.io/ads-v2/. doi: [10.5281/zenodo.6365077](https://doi.org/10.5281/zenodo.6365077)
Nordmann, E. & DeBruine, L. (2024). Applied Data Skills. Version 3.0. Retrieved from https://psyteachr.github.io/ads-v2/. doi: [10.5281/zenodo.6365077](https://doi.org/10.5281/zenodo.6365077)

See https://psyteachr.github.io/ads-v1/ for version 1.
See <https://psyteachr.github.io/ads-v1/> for version 1 (2022) and <https://psyteachr.github.io/ads-v2/> for version 2 (2023).



Expand Down
16 changes: 16 additions & 0 deletions _freeze/04-summary/execute-results/html.json

Large diffs are not rendered by default.

16 changes: 16 additions & 0 deletions _freeze/app-import/execute-results/html.json

Large diffs are not rendered by default.

16 changes: 7 additions & 9 deletions _quarto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ book:
# hypothesis:
# theme: clean
# openSidebar: false
repo-url: https://github.com/psyteachr/ads-v2/
repo-url: https://github.com/psyteachr/ads-v3/
repo-branch: master
repo-actions: [edit, issue, source]
# downloads: [pdf, epub]
Expand All @@ -33,10 +33,10 @@ book:
# background: light
margin-header: "<a href='https://doi.org/10.5281/zenodo.6365077'><img src='https://zenodo.org/badge/DOI/10.5281/zenodo.6365077.svg' alt='DOI'></a>"
page-footer:
left: "CC-BY 2022, psyTeachR"
left: "CC-BY 2024, psyTeachR"
right:
- icon: github
href: https://github.com/psyteachr/ads-v2
href: https://github.com/psyteachr/ads-v3
- icon: twitter
href: https://twitter.com/psyteachr
- icon: https://zenodo.org/badge/DOI/10.5281/zenodo.6365077.svg
Expand All @@ -46,9 +46,9 @@ book:
- 01-intro.qmd
- 02-reports.qmd
- 03-viz.qmd
- 04-data.qmd
- 05-summary.qmd
- 06-formative.qmd
- 04-summary.qmd
- 05-formative.qmd
- 06-ai.qmd
- 07-joins.qmd
- 08-tidy.qmd
- 09-wrangle.qmd
Expand All @@ -62,12 +62,10 @@ book:
- app-conventions.qmd
- app-teams.qmd
- app-debugging.qmd
- app-import.qmd
- app-datatypes.qmd
- app-dates.qmd
- app-styling.qmd
# - app-hashtags.qmd
# - app-twitter.qmd
- app-spotify.qmd
- app-webpage.qmd


Expand Down
25 changes: 0 additions & 25 deletions _render.R

This file was deleted.

Loading

0 comments on commit 7ca4fd4

Please sign in to comment.