diff --git a/.Rproj.user/EEE9B81B/pcs/source-pane.pper b/.Rproj.user/EEE9B81B/pcs/source-pane.pper
index ddca97d..902cc6f 100644
--- a/.Rproj.user/EEE9B81B/pcs/source-pane.pper
+++ b/.Rproj.user/EEE9B81B/pcs/source-pane.pper
@@ -1,3 +1,3 @@
{
- "activeTab": 2
+ "activeTab": 0
}
\ No newline at end of file
diff --git a/.quarto/_freeze/02-wrangling/execute-results/html.json b/.quarto/_freeze/02-wrangling/execute-results/html.json
index 3abe2c8..e5f22d3 100644
--- a/.quarto/_freeze/02-wrangling/execute-results/html.json
+++ b/.quarto/_freeze/02-wrangling/execute-results/html.json
@@ -1,10 +1,8 @@
{
- "hash": "1740a607e9de7ca322e49f331bb6e0b3",
+ "hash": "85f2fa0ec2f8baeeb06d4b656ca711c6",
"result": {
- "markdown": "# Data wrangling I {#sec-wrangling}\n\n## Intended Learning Outcomes {.unnumbered}\n\nIn the next two chapters, we will build on the data wrangling skills from level 1. We will revisit all the functions you have already encountered (and might have forgotten over the summer break) and introduce 2 or 3 new functions. These two chapters will provide an opportunity to revise and apply the functions to a novel dataset.\n\nBy the end of this chapter, you should be able to:\n\n- apply familiar data wrangling functions to novel datasets\n- read and interpret error messages\n- realise there are several ways of getting to the results\n- export data objects as csv files\n\nThe main purpose of this chapter and @sec-wrangling2 is to wrangle your data into shape for data visualisation (@sec-dataviz and @sec-dataviz2). For the two chapters, we will:\n\n1. calculate demographics\n2. tidy 3 different questionnaires with varying degrees of complexity\n3. solve an error mode problem\n4. join all data objects together\n\n## [Individual Walkthrough]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\nBefore we start, we need to set up some things.\n\n\n## Activity 1: Setup\n\n* We will be working on the **dataset by Pownall et al. (2023)** again, which means we can still use the project we created last week. The data files will already be there, so no need to download them again.\n* To **open the project** in RStudio, go to the folder in which you stored the project and the data last time, and double click on the project icon.\n* **Create a new `.Rmd` file** for chapter 2 and save it to your project folder. Name it something meaningful (e.g., “chapter_02.Rmd”, “02_data_wrangling.Rmd”). See @sec-rmd if you need some guidance.\n* In your newly created `.Rmd` file, delete everything below line 12 (after the set-up code chunk).\n\n\n\n## Activity 2: Load in the libraries and read in the data\n\nWe will use `tidyverse` today, and we want to create a data object `data_prp` that stores the data from the file `prp_data_reduced.csv`.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(???)\ndata_prp <- read_csv(\"???\")\n```\n:::\n\n\n\n\n:::\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(tidyverse)\ndata_prp <- read_csv(\"prp_data_reduced.csv\")\n```\n:::\n\n\n:::\n\nIf you need a quick reminder what the dataset was about, have a look at the abstract in @sec-download_data_ch1. We also addressed the changes we made to the dataset there.\n\nAnd remember to have a quick `glimpse()` at your data.\n\n\n\n## Activity 3: Calculating demographics\n\nLet’s start with some simple data-wrangling steps to compute demographics for our original dataset, `data_prp`. First, we want to determine how many participants took part in the study by Pownall et al. (2023) and compute the mean age and the standard deviation of age for the sample.\n\n\n\n### ... for the full sample using `summarise()`\n\nThe `summarise()` function is part of the **\"Wickham Six\"** alongside `group_by()`, `select()`, `filter()`, `mutate()`, and `arrange()`. You used them plenty of times last year.\n\nWithin `summarise()`, we can use the `n()` function, which calculates the number of rows in the dataset. Since each row corresponds to a unique participant, this gives us the total number of participants.\n\nTo calculate the mean age and the standard deviation of age, we need to use the functions `mean()` and `sd()` on the column `Age` respectively.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: There were 2 warnings in `summarise()`.\nThe first warning was:\nℹ In argument: `mean_age = mean(Age)`.\nCaused by warning in `mean.default()`:\n! argument is not numeric or logical: returning NA\nℹ Run `dplyr::last_dplyr_warnings()` to see the 1 remaining warning.\n```\n:::\n\n```{.r .cell-code}\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nR did not give us an error message per se, but the output is not quite as expected either. There are `NA` values in the `mean_age` and `sd_age` columns. Looking at the warning message and at `Age`, can you explain what happened?\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nThe warning message says: `argument is not numeric or logical: returning NA` If we look at the `Age` column more closely, we can see that it's a character data type.\n\n:::\n\n\n\n#### Fixing `Age` {.unnumbered}\n\nMight be wise to look at the unique answers in column `Age` to determine what is wrong. We can do that with the function `distinct()`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nage_distinct <- data_prp %>% \n distinct(Age)\n\nage_distinct\n```\n:::\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Show the unique values of `Age`.\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n:::\n\n::: columns\n\n::: column\n\nOne cell has the string \"years\" added to their number 25, which has converted the entire column into a character column.\n\nWe can easily fix this by extracting only the numbers from the column and converting it into a numeric data type. The `parse_number()` function, which is part of the `tidyverse` package, handles both steps in one go (so there’s no need to load additional packages).\n\nWe will combine this with the `mutate()` function to create a new column called `Age` (containing those numeric values), effectively replacing the old `Age` column (which had the character values).\n\n:::\n\n::: column\n\n{target=\"_blank\"})](images/parse_number.png){width=\"95%\"}\n\n:::\n\n:::\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndata_prp <- data_prp %>% \n mutate(Age = parse_number(Age))\n\ntypeof(data_prp$Age) # fixed\n```\n\n::: {.cell-output .cell-output-stdout}\n```\n[1] \"double\"\n```\n:::\n:::\n\n\n\n\n#### Computing summary stats {.unnumbered}\n\nExcellent. Now that the numbers are in a numeric format, let's try calculating the demographics for the total sample again.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nEven though there's no error or warning, the table still shows `NA` values for `mean_age` and `sd_age`. So, what could possibly be wrong now?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nDid you notice that the `Age` column in `age_distinct` contains some missing values (`NA`)? To be honest, it's easier to spot this issue in the actual R output than in the printed HTML page.\n\n:::\n\n\n\n#### Computing summary stats - third attempt {.unnumbered}\n\nTo ensure R ignores missing values during calculations, we need to add the extra argument `na.rm = TRUE` to the `mean()` and `sd()` functions.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age, na.rm = TRUE), # mean age\n sd_age = sd(Age, na.rm = TRUE)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nFinally, we’ve got it! 🥳 Third time's the charm!\n\n\n\n### ... per gender using `summarise()` and `group_by()`\n\nNow we want to compute the summary statistics for each gender. The code inside the `summarise()` function remains unchanged; we just need to use the `group_by()` function beforehand to tell R that we want to compute the summary statistics for each group separately. It’s also a good practice to use `ungroup()` afterwards, so you are not taking groupings forward unintentionally.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% # split data up into groups (here Gender)\n summarise(n = n(), # participant number \n mean_age = mean(Age, na.rm = TRUE), # mean age \n sd_age = sd(Age, na.rm = TRUE)) %>% # standard deviation of age\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Adding percentages\n\nSometimes, it may be useful to calculate percentages, such as for the gender split. You can do this by adding a line within the `summarise()` function to perform the calculation. All we need to do is take the number of female, male, and non-binary participants (stored in the `n` column of `demo_by_gender`), divide it by the total number of participants (stored in the `n` column of `demo_total`), and multiply by 100. Let's add `percentage` to the `summarise()` function of `demo_by_gender`. Make sure that the code for `percentages` is placed after the value for `n` has been computed.\n\nAccessing the value of `n` for the different gender categories is straightforward because we can refer back to it directly. However, since the total number of participants is stored in a different data object, we need to use a base R function to access it – specifically the `$` operator. To do this, you simply type the name of the data object (in this case, `demo_total`), followed by the `$` symbol (with no spaces), and then the name of the column you want to retrieve (in this case, `n`). The general pattern is `data$column`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n # n from the line above divided by n from demo_total *100\n percentage = n/demo_total$n *100, \n mean_age = mean(Age, na.rm = TRUE), \n sd_age = sd(Age, na.rm = TRUE)) %>% \n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n::: {.callout-tip collapse=\"true\"}\n\n## Tip for decimal places - use `round()`\n\nNot super important, because you could round the values by yourself when writing up your reports, but if you wanted to tidy up the decimal places in the output, you can do that using the `round()` function. You would need to \"wrap\" it around your computations and specify how many decimal places you want to display (for example `mean(Age)` would turn into `round(mean(Age), 1)`). It may look odd for `percentage`, just make sure the number that specifies the decimal places is placed **within** the round function. The default value is 0 (meaning no decimal spaces).\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n percentage = round(n/demo_total$n *100, 2), # percentage with 2 decimal places\n mean_age = round(mean(Age, na.rm = TRUE), 1), # mean Age with 1 decimal place\n sd_age = round(sd(Age, na.rm = TRUE), 3)) %>% # sd Age with 3 decimal places\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 4: Questionable Research Practices (QRPs) {#sec-ch2_act4}\n\n#### The main goal is to compute the mean QRP score per participant for time point 1. {.unnumbered}\n\nLooking at the QRP data at time point 1, you determine that\n\n* individual item columns are , and\n* according to the codebook, there are reverse-coded items in this questionnaire.\n\nSo, we just have to **compute an average score for items 1 to 11** as items 12 to 15 are distractor items. Seems quite straightforward.\n\nThe downside is that individual items are each in a separate column, i.e., in **wide format**, and everything would be easier if the items were arranged in **long format**.\n\nLet's tackle this problem in steps. Best would be to create a separate data object for that. If we wanted to compute this within `data_prp`, it would turn into a nightmare.\n\n* **Step 1**: Select the relevant columns `Code`, and `QRPs_1_Time1` to `QRPs_1_Time1` and store them in an object called `qrp_t1`\n* **Step 2**: Pivot the data from wide format to long format using `pivot_longer()` so we can calculate the average score more easily (in step 3)\n* **Step 3**: Calculate the average QRP score (`QRPs_Acceptance_Time1_mean`) per participant using `group_by()` and `summarise()`\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_t1 <- data_prp %>% \n #Step 1\n select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%\n # Step 2\n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(Code) %>% # grouping py participant id\n summarise(QRPs_Acceptance_Time1_mean = mean(Scores)) %>% # calculating the average Score\n ungroup() # just make it a habit\n```\n:::\n\n\n::: {.callout-caution icon=\"false\" collapse=\"true\"}\n\n## Explain the individual functions\n\n::: panel-tabset\n\n## `select ()`\n\nThe select function allows to include or exclude certain variables (columns). Here we want to focus on the participant ID column (i.e., `Code`) and the QRP items at time point 1. We can either list them all individually, i.e., Code, QRPs_1_Time1, QRPs_2_Time1, QRPs_3_Time1, and so forth (you get the gist), but that would take forever to type.\n\nA shortcut is to use the colon operator `:`. It allows us to select all columns that fall within the range of `first_column_name` to `last_column_name`. We can apply this here since the QRP items (1 to 11) are sequentially listed in `data_prp`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step1 <- data_prp %>% \n select(Code, QRPs_1_Time1:QRPs_11_Time1)\n\n# show first 5 rows of qrp_step1\nhead(qrp_step1, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nHow many rows/observations and columns/variables do we have in `qrp_step1`?\n\n* rows/observations: \n* columns/variables: \n\n## `pivot_longer()`\n\nAs you can see, the table we got from Step 1 is in wide format. To get it into wide format, we need to define:\n\n* the columns that need to be reshuffled from wide into long format (`col` argument). Here we selected \"everything except the `Code` column\", as indicated by `-Code` \\[minus `Code`\\]. However, `QRPs_1_Time1:QRPs_11_Time1` would also work and give you the exact same result.\n* the `names_to` argument. R is creating a new column in which all the column names from the columns you selected in `col` will be stored in. Here we are naming this column \"Items\" but you could pick something equally sensible if you like.\n* the `values_to` argument. R creates this second column to store all responses the participants gave to the individual questions, i.e., all the numbers in this case. We named it \"Scores\" here, but you could have called it something different, like \"Responses\"\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step2 <- qrp_step1 %>% \n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\")\n\n# show first 15 rows of qrp_step2\nhead(qrp_step2, n = 15)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nNow, have a look at `qrp_step2`. In total, we now have rows/observations, per participant, and columns/variables.\n\n## `group_by()` and `summarise()`\n\nThis follows exactly the same sequence we used when calculating descriptive statistics by gender. The only difference is that we are now grouping the data by the participant's `Code` instead of `Gender`.\n\n`summarise()` works exactly the same way: `summarise(new_column_name = function_to_calculate_something(column_name_of_numeric_values))`\n\nThe `function_to_calculate_something` can be `mean()`, `sd()` or `sum()` for mean scores, standard deviations, or summed-up scores respectively. You could also use `min()` or `max()` if you wanted to determine the lowest or the highest score for each participant.\n\n:::\n\n:::\n\n::: callout-tip\n\nYou could **rename the columns whilst selecting** them. The pattern would be `select(new_name = old_name)`. For example, if we wanted to select variable `Code` and rename it as `Participant_ID`, we could do that.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nrenaming_col <- data_prp %>% \n select(Participant_ID = Code)\n\nhead(renaming_col, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 5: Knitting\n\nOnce you've completed your R Markdown file, the final step is to \"knit\" it, which converts the `.Rmd` file into a HTML file. Knitting combines your code, text, and output (like tables and plots) into a single cohesive document. This is a really good way to check your code is working.\n\nTo knit the file, **click the Knit button** at the top of your RStudio window. The document will be generated and, depending on your setting, automatically opened in the viewer in the `Output pane` or an external browser window.\n\nIf any errors occur during knitting, RStudio will show you an error message with details to help you troubleshoot.\n\nIf you want to **intentionally keep any errors** we tackled today to keep a reference on how you solved them, you could add `error=TRUE` or `eval=FALSE` to the code chunk that isn't running.\n\n\n\n## Activity 6: Export a data object as a csv\n\nTo avoid having to repeat the same steps in the next chapter, it's a good idea to save the data objects you've created today as csv files. You can do this by using the `write_csv()` function from the `readr` package. The csv files will appear in your project folder.\n\nThe basic syntax is:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_object, \"filename.csv\")\n```\n:::\n\n\nNow, let's export the objects `data_prp` and `qrp_t1`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_prp, \"data_prp_for_ch3.csv\")\n```\n:::\n\n\nHere we named the file `data_prp_for_ch3.csv`, so we wouldn't override the original data csv file `prp_data_reduced.csv`. However, feel free to choose a name that makes sense to you.\n\n::: {.callout-note icon=\"false\"}\n\n## Your Turn\n\nExport `qrp_t1`.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(qrp_t1, \"qrp_t1.csv\")\n```\n:::\n\n\n:::\n\n:::\n\nCheck that your csv files have appeared in your project folder, and you're all set!\n\n**That’s it for Chapter 2: Individual Walkthrough.**\n\n## [Pair-coding]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n\n::: {.cell layout-align=\"center\"}\n\n:::\n\n\n\nWe will continue working with the data from Binfet et al. (2021), focusing on the randomised controlled trial of therapy dog interventions. Today, our goal is to **calculate an average `Flourishing` score for each participant** at time point 1 (pre-intervention) using the raw data file `dog_data_raw`. Currently, the data looks like this:\n\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Task 1: Open the R project you created last week {.unnumbered}\n\nIf you haven’t created an R project for the lab yet, please do so now. If you already have one set up, go ahead and open it.\n\n\n### Task 2: Open your `.Rmd` file from last week {.unnumbered}\n\nSince we haven’t used it much yet, feel free to continue using the `.Rmd` file you created last week in Task 2.\n\n\n### Task 3: Load in the library and read in the data {.unnumbered}\n\nThe data should be in your project folder. If you didn’t download it last week, or if you’d like a fresh copy, you can download the data again here: [data_pair_ch1](data/data_pair_ch1.zip \"download\").\n\nWe will be using the `tidyverse` package today, and the data file we need to read in is `dog_data_raw.csv`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(???)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"???\")\n```\n:::\n\n\n:::\n\n\n### Task 4: Calculating the mean for `Flourishing_pre` {.unnumbered}\n\n* **Step 1**: Select all relevant columns, including participant ID and all 8 items from the `Flourishing` questionnaire completed before the intervention. Store this data in an object called `data_flourishing`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nLook at the codebook. Try to determine:\n\n* The variable name of the column where the participant ID is stored.\n* The items related to the Flourishing scale at the pre-intervention stage.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nFrom the codebook, we know that:\n\n* The participant ID column is called `RID`.\n* The Flourishing items at the pre-intervention stage start with `F1_`.\n\n:::\n\n:::\n\n\n* **Step 2**: Pivot the data from wide format to long format so we can calculate the average score more easily (in step 3).\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nWhich pivot function should you use? We have `pivot_wider()` and `pivot_longer()` to choose from.\n\nWe also need 3 arguments in that function:\n\n* The columns you want to select (e.g., all the Flourishing items),\n* The name of the column where the current column headings will be stored (e.g., \"Questionnaire\"),\n* The name of the column that should store all the values (e.g., \"Responses\").\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n pivot_longer(cols = ???, names_to = \"???\", values_to = \"???\")\n```\n:::\n\n\n:::\n\n:::\n\n* **Step 3**: Calculate the average Flourishing score per participant and name this column `Flourishing_pre` to match the table above.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nBefore summarising the mean, you may need to group the data.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n group_by(???) %>% \n summarise(Flourishing_pre = ???(???)) %>% \n ungroup()\n```\n:::\n\n:::\n\n:::\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(tidyverse)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"dog_data_raw.csv\")\n\n# Task 4: Tidying \ndata_flourishing <- dog_data_raw %>% \n # Step 1\n select(RID, F1_1:F1_8) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Questionnaire\", values_to = \"Responses\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_pre = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\n:::\n\n\n\n## [Test your knowledge and challenge yourself]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n### Knowledge check {.unnumbered}\n\n#### Question 1 {.unnumbered}\n\nWhich function of the Wickham Six would you use to include or exclude certain variables (columns)? \n\n\n#### Question 2 {.unnumbered}\n\nWhich function of the Wickham Six would you use to create new columns or modify existing columns in a dataframe? \n\n\n#### Question 3 {.unnumbered}\n\n\nWhich function of the Wickham Six would you use to organise data into groups based on one or more columns? \n\n\n\n#### Question 4 {.unnumbered}\n\nWhich function of the Wickham Six would you use to sort the rows of a dataframe based on the values in one or more columns? \n\n\n\n#### Question 5 {.unnumbered}\n\nWhich function of the Wickham Six would NOT modify the original dataframe? \n\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain these answers\n\n| Function | Description |\n|:-------------|:------------------------------------------------------|\n| `select()` | Include or exclude certain variables/columns |\n| `filter()` | Include or exclude certain observations/rows |\n| `mutate()` | Creates new columns or modifies existing ones |\n| `arrange()` | Changes the order of the rows |\n| `group_by()` | Split data into groups based on one or more variables |\n| `summarise()`| Creates a new dataframe returning one row for each combination of grouping variables |\n\n\nTechnically, the first five functions operate on the existing data object, making adjustments like sorting the data (e.g., with `arrange()`), reducing the number of rows (e.g., with `filter()`), reducing the number of columns (e.g., with `select()`), or adding new columns (e.g., with `mutate()`). In contrast, `summarise()` fundamentally alters the structure of the original dataframe by generating a completely new dataframe that contains only summary statistics, rather than retaining the original rows and columns.\n\n:::\n\n\n\n### Error mode {.unnumbered}\n\nSome of the code chunks contain mistakes and result in errors, while others do not produce the expected results. Your task is to identify any issues, explain why they occurred, and, if possible, fix them.\n\nWe will use a few built-in datasets, such as `billboard` and `starwars`, to help you replicate the errors in your own R environment. You can view the data either by typing the dataset name directly into your console or by storing the data as a separate object in your `Global Environment`.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nbillboard\n\nstarwars_data = starwars\n```\n:::\n\n\n\n\n#### Question 6 {.unnumbered}\n\nCurrently, the weekly song rankings for Billboard Top 100 in 2000 are in wide format, with each week in a separate column. The following code is supposed to transpose the wide-format `billboard` data into long format:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(names_to = \"weeks\", values_to = \"rank\")\n```\n\n::: {.cell-output .cell-output-error}\n```\nError in `pivot_longer()`:\n! `cols` must select at least one column.\n```\n:::\n:::\n\n\nWhat does this error message mean and how do you fix it?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe error message indicates that the `cols` argument is missing in the function. This means the function doesn’t know which columns to transpose from wide format to long format.\n\nFIX: Add `cols = wk1:wk76` to the function to select columns from wk1 to wk76. Alternatively, `cols = starts_with(\"wk\")` would also work since all columns start with the letter combination \"wk\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(cols = wk1:wk76, names_to = \"weeks\", values_to = \"rank\")\n# OR\nlong_data <- billboard %>% \n pivot_longer(cols = starts_with(\"wk\"), names_to = \"weeks\", values_to = \"rank\")\n```\n:::\n\n\n:::\n\n\n\n#### Question 7 {.unnumbered}\n\nThe following code is intended to calculate the mean height of all the characters in the built-in `starwars` dataset, grouped by their gender. \n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = height)\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: Returning more (or less) than 1 row per `summarise()` group was deprecated in\ndplyr 1.1.0.\nℹ Please use `reframe()` instead.\nℹ When switching from `summarise()` to `reframe()`, remember that `reframe()`\n always returns an ungrouped data frame and adjust accordingly.\n```\n:::\n:::\n\n\nThe code runs, but it's giving us some weird warning and the output is also not as expected. What steps should we take to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe aggregation function `mean()` is missing from within `summarise()`. Without it, the function does not perform any aggregation and returns *all* rows with only the columns for gender and height.\n\nFIX: Wrap the `mean()` function around the variable you want to aggregate, here `height`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height))\n```\n:::\n\n\n:::\n\n\n\n#### Question 8 {.unnumbered}\n\nFollowing up on Question 7, we now have `summary_data` that looks approximately correct - it has the expected rows and column numbers, however, the cell values are \"weird\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\nCan you explain what is happening here? And how can we modify the code to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nLook at the original `starwars` data. You will notice that some of the characters with feminine and masculine gender entries have missing height values. However, all four characters without a specified gender have provided their height.\n\nFIX: We need to add `na.rm = TRUE` to the `mean()` function to ensure that R ignores missing values before aggregating the data.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height, na.rm = TRUE))\n\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n### Challenge yourself {.unnumbered}\n\nIf you want to **challenge yourself** and further apply the skills from Chapter 2, you can wrangle the data from `dog_data_raw` for additional questionnaires from either the pre- and/or post-intervention stages:\n\n* Calculate the mean score for `flourishing_post` for each participant.\n* Calculate the mean score for the `PANAS` (Positive and/or Negative Affect) per participant\n* Calculate the mean score for happiness (`SHS`) per participant\n\nThe 3 steps are equivalent for those questionnaires - select, pivot, group_by and summarise; you just have to \"replace\" the questionnaire items involved.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution for **Challenge yourself**\n\nFlourishing post-intervention\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n## flourishing_post\nflourishing_post <- dog_data_raw %>% \n # Step 1\n select(RID, starts_with(\"F2\")) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Names\", values_to = \"Response\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_post = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\nThe PANAS could be solved more concisely with the skills we learn in @sec-wrangling2, but for now, you would have solved it this way:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# PANAS - positive affect pre\nPANAS_PA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_3, PN1_5, PN1_7, PN1_8, PN1_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - positive affect post\nPANAS_PA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_3, PN2_5, PN2_7, PN2_8, PN2_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_post = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect pre\nPANAS_NA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_1, PN1_2, PN1_4, PN1_6, PN1_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect post\nPANAS_NA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_1, PN2_2, PN2_4, PN2_6, PN2_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_post = mean(Scores)) %>% \n ungroup()\n```\n:::\n\n\nHappiness scale\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# happiness_pre\nhappiness_pre <- dog_data_raw %>% \n # Step 1\n select(RID, HA1_1, HA1_2, HA1_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_pre = mean(Score)) %>% \n ungroup()\n\n#happiness_post\nhappiness_post <- dog_data_raw %>% \n # Step 1\n select(RID, HA2_1, HA2_2, HA2_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_post = mean(Score)) %>% \n ungroup()\n```\n:::\n\n\n:::\n",
- "supporting": [
- "02-wrangling_files"
- ],
+ "markdown": "# Data wrangling I {#sec-wrangling}\n\n## Intended Learning Outcomes {.unnumbered}\n\nIn the next two chapters, we will build on the data wrangling skills from level 1. We will revisit all the functions you have already encountered (and might have forgotten over the summer break) and introduce 2 or 3 new functions. These two chapters will provide an opportunity to revise and apply the functions to a novel dataset.\n\nBy the end of this chapter, you should be able to:\n\n- apply familiar data wrangling functions to novel datasets\n- read and interpret error messages\n- realise there are several ways of getting to the results\n- export data objects as csv files\n\nThe main purpose of this chapter and @sec-wrangling2 is to wrangle your data into shape for data visualisation (@sec-dataviz and @sec-dataviz2). For the two chapters, we will:\n\n1. calculate demographics\n2. tidy 3 different questionnaires with varying degrees of complexity\n3. solve an error mode problem\n4. join all data objects together\n\n## [Individual Walkthrough]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\nBefore we start, we need to set up some things.\n\n\n## Activity 1: Setup\n\n* We will be working on the **dataset by Pownall et al. (2023)** again, which means we can still use the project we created last week. The data files will already be there, so no need to download them again.\n* To **open the project** in RStudio, go to the folder in which you stored the project and the data last time, and double click on the project icon.\n* **Create a new `.Rmd` file** for chapter 2 and save it to your project folder. Name it something meaningful (e.g., “chapter_02.Rmd”, “02_data_wrangling.Rmd”). See @sec-rmd if you need some guidance.\n* In your newly created `.Rmd` file, delete everything below line 12 (after the set-up code chunk).\n\n\n\n## Activity 2: Load in the libraries and read in the data\n\nWe will use `tidyverse` today, and we want to create a data object `data_prp` that stores the data from the file `prp_data_reduced.csv`.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(???)\ndata_prp <- read_csv(\"???\")\n```\n:::\n\n\n\n\n:::\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(tidyverse)\ndata_prp <- read_csv(\"prp_data_reduced.csv\")\n```\n:::\n\n\n:::\n\nIf you need a quick reminder what the dataset was about, have a look at the abstract in @sec-download_data_ch1. We also addressed the changes we made to the dataset there.\n\nAnd remember to have a quick `glimpse()` at your data.\n\n\n\n## Activity 3: Calculating demographics\n\nLet’s start with some simple data-wrangling steps to compute demographics for our original dataset, `data_prp`. First, we want to determine how many participants took part in the study by Pownall et al. (2023) and compute the mean age and the standard deviation of age for the sample.\n\n\n\n### ... for the full sample using `summarise()`\n\nThe `summarise()` function is part of the **\"Wickham Six\"** alongside `group_by()`, `select()`, `filter()`, `mutate()`, and `arrange()`. You used them plenty of times last year.\n\nWithin `summarise()`, we can use the `n()` function, which calculates the number of rows in the dataset. Since each row corresponds to a unique participant, this gives us the total number of participants.\n\nTo calculate the mean age and the standard deviation of age, we need to use the functions `mean()` and `sd()` on the column `Age` respectively.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: There were 2 warnings in `summarise()`.\nThe first warning was:\nℹ In argument: `mean_age = mean(Age)`.\nCaused by warning in `mean.default()`:\n! argument is not numeric or logical: returning NA\nℹ Run `dplyr::last_dplyr_warnings()` to see the 1 remaining warning.\n```\n:::\n\n```{.r .cell-code}\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nR did not give us an error message per se, but the output is not quite as expected either. There are `NA` values in the `mean_age` and `sd_age` columns. Looking at the warning message and at `Age`, can you explain what happened?\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nThe warning message says: `argument is not numeric or logical: returning NA` If we look at the `Age` column more closely, we can see that it's a character data type.\n\n:::\n\n\n\n#### Fixing `Age` {.unnumbered}\n\nMight be wise to look at the unique answers in column `Age` to determine what is wrong. We can do that with the function `distinct()`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nage_distinct <- data_prp %>% \n distinct(Age)\n\nage_distinct\n```\n:::\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Show the unique values of `Age`.\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n:::\n\n::: columns\n\n::: column\n\nOne cell has the string \"years\" added to their number 25, which has converted the entire column into a character column.\n\nWe can easily fix this by extracting only the numbers from the column and converting it into a numeric data type. The `parse_number()` function, which is part of the `tidyverse` package, handles both steps in one go (so there’s no need to load additional packages).\n\nWe will combine this with the `mutate()` function to create a new column called `Age` (containing those numeric values), effectively replacing the old `Age` column (which had the character values).\n\n:::\n\n::: column\n\n{target=\"_blank\"})](images/parse_number.png){width=\"95%\"}\n\n:::\n\n:::\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndata_prp <- data_prp %>% \n mutate(Age = parse_number(Age))\n\ntypeof(data_prp$Age) # fixed\n```\n\n::: {.cell-output .cell-output-stdout}\n```\n[1] \"double\"\n```\n:::\n:::\n\n\n\n\n#### Computing summary stats {.unnumbered}\n\nExcellent. Now that the numbers are in a numeric format, let's try calculating the demographics for the total sample again.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nEven though there's no error or warning, the table still shows `NA` values for `mean_age` and `sd_age`. So, what could possibly be wrong now?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nDid you notice that the `Age` column in `age_distinct` contains some missing values (`NA`)? To be honest, it's easier to spot this issue in the actual R output than in the printed HTML page.\n\n:::\n\n\n\n#### Computing summary stats - third attempt {.unnumbered}\n\nTo ensure R ignores missing values during calculations, we need to add the extra argument `na.rm = TRUE` to the `mean()` and `sd()` functions.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age, na.rm = TRUE), # mean age\n sd_age = sd(Age, na.rm = TRUE)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nFinally, we’ve got it! 🥳 Third time's the charm!\n\n\n\n### ... per gender using `summarise()` and `group_by()`\n\nNow we want to compute the summary statistics for each gender. The code inside the `summarise()` function remains unchanged; we just need to use the `group_by()` function beforehand to tell R that we want to compute the summary statistics for each group separately. It’s also a good practice to use `ungroup()` afterwards, so you are not taking groupings forward unintentionally.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% # split data up into groups (here Gender)\n summarise(n = n(), # participant number \n mean_age = mean(Age, na.rm = TRUE), # mean age \n sd_age = sd(Age, na.rm = TRUE)) %>% # standard deviation of age\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Adding percentages\n\nSometimes, it may be useful to calculate percentages, such as for the gender split. You can do this by adding a line within the `summarise()` function to perform the calculation. All we need to do is take the number of female, male, and non-binary participants (stored in the `n` column of `demo_by_gender`), divide it by the total number of participants (stored in the `n` column of `demo_total`), and multiply by 100. Let's add `percentage` to the `summarise()` function of `demo_by_gender`. Make sure that the code for `percentages` is placed after the value for `n` has been computed.\n\nAccessing the value of `n` for the different gender categories is straightforward because we can refer back to it directly. However, since the total number of participants is stored in a different data object, we need to use a base R function to access it – specifically the `$` operator. To do this, you simply type the name of the data object (in this case, `demo_total`), followed by the `$` symbol (with no spaces), and then the name of the column you want to retrieve (in this case, `n`). The general pattern is `data$column`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n # n from the line above divided by n from demo_total *100\n percentage = n/demo_total$n *100, \n mean_age = mean(Age, na.rm = TRUE), \n sd_age = sd(Age, na.rm = TRUE)) %>% \n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n::: {.callout-tip collapse=\"true\"}\n\n## Tip for decimal places - use `round()`\n\nNot super important, because you could round the values by yourself when writing up your reports, but if you wanted to tidy up the decimal places in the output, you can do that using the `round()` function. You would need to \"wrap\" it around your computations and specify how many decimal places you want to display (for example `mean(Age)` would turn into `round(mean(Age), 1)`). It may look odd for `percentage`, just make sure the number that specifies the decimal places is placed **within** the round function. The default value is 0 (meaning no decimal spaces).\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n percentage = round(n/demo_total$n *100, 2), # percentage with 2 decimal places\n mean_age = round(mean(Age, na.rm = TRUE), 1), # mean Age with 1 decimal place\n sd_age = round(sd(Age, na.rm = TRUE), 3)) %>% # sd Age with 3 decimal places\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 4: Questionable Research Practices (QRPs) {#sec-ch2_act4}\n\n#### The main goal is to compute the mean QRP score per participant for time point 1. {.unnumbered}\n\nAt the moment, the data is in wide format. The table below shows data from the first 3 participants:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nhead(data_prp, n = 3)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\nLooking at the QRP data at time point 1, you determine that\n\n* individual item columns are , and\n* according to the codebook, there are reverse-coded items in this questionnaire.\n\nAccording to the codebook and the data table above, we just have to **compute the average score for QRP items to **, since items to are distractor items. Seems quite straightforward.\n\nHowever, as you can see in the table above, each item is in a separate column, meaning the data is in **wide format**. It would be much easier to calculate the mean scores if the items were arranged in **long format**.\n\n\nLet’s tackle this problem step by step. It’s best to create a separate data object for this. If we tried to compute it within `data_prp`, it could quickly become messy.\n\n\n* **Step 1**: Select the relevant columns `Code`, and `QRPs_1_Time1` to `QRPs_1_Time1` and store them in an object called `qrp_t1`\n* **Step 2**: Pivot the data from wide format to long format using `pivot_longer()` so we can calculate the average score more easily (in step 3)\n* **Step 3**: Calculate the average QRP score (`QRPs_Acceptance_Time1_mean`) per participant using `group_by()` and `summarise()`\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_t1 <- data_prp %>% \n #Step 1\n select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%\n # Step 2\n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(Code) %>% # grouping by participant id\n summarise(QRPs_Acceptance_Time1_mean = mean(Scores)) %>% # calculating the average Score\n ungroup() # just make it a habit\n```\n:::\n\n\n::: {.callout-caution icon=\"false\" collapse=\"true\"}\n\n## Explain the individual functions\n\n::: panel-tabset\n\n## `select ()`\n\nThe select function allows to include or exclude certain variables (columns). Here we want to focus on the participant ID column (i.e., `Code`) and the QRP items at time point 1. We can either list them all individually, i.e., Code, QRPs_1_Time1, QRPs_2_Time1, QRPs_3_Time1, and so forth (you get the gist), but that would take forever to type.\n\nA shortcut is to use the colon operator `:`. It allows us to select all columns that fall within the range of `first_column_name` to `last_column_name`. We can apply this here since the QRP items (1 to 11) are sequentially listed in `data_prp`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step1 <- data_prp %>% \n select(Code, QRPs_1_Time1:QRPs_11_Time1)\n\n# show first 5 rows of qrp_step1\nhead(qrp_step1, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nHow many rows/observations and columns/variables do we have in `qrp_step1`?\n\n* rows/observations: \n* columns/variables: \n\n## `pivot_longer()`\n\nAs you can see, the table we got from Step 1 is in wide format. To get it into wide format, we need to define:\n\n* the columns that need to be reshuffled from wide into long format (`col` argument). Here we selected \"everything except the `Code` column\", as indicated by `-Code` \\[minus `Code`\\]. However, `QRPs_1_Time1:QRPs_11_Time1` would also work and give you the exact same result.\n* the `names_to` argument. R is creating a new column in which all the column names from the columns you selected in `col` will be stored in. Here we are naming this column \"Items\" but you could pick something equally sensible if you like.\n* the `values_to` argument. R creates this second column to store all responses the participants gave to the individual questions, i.e., all the numbers in this case. We named it \"Scores\" here, but you could have called it something different, like \"Responses\"\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step2 <- qrp_step1 %>% \n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\")\n\n# show first 15 rows of qrp_step2\nhead(qrp_step2, n = 15)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nNow, have a look at `qrp_step2`. In total, we now have rows/observations, per participant, and columns/variables.\n\n## `group_by()` and `summarise()`\n\nThis follows exactly the same sequence we used when calculating descriptive statistics by gender. The only difference is that we are now grouping the data by the participant's `Code` instead of `Gender`.\n\n`summarise()` works exactly the same way: `summarise(new_column_name = function_to_calculate_something(column_name_of_numeric_values))`\n\nThe `function_to_calculate_something` can be `mean()`, `sd()` or `sum()` for mean scores, standard deviations, or summed-up scores respectively. You could also use `min()` or `max()` if you wanted to determine the lowest or the highest score for each participant.\n\n:::\n\n:::\n\n::: callout-tip\n\nYou could **rename the columns whilst selecting** them. The pattern would be `select(new_name = old_name)`. For example, if we wanted to select variable `Code` and rename it as `Participant_ID`, we could do that.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nrenaming_col <- data_prp %>% \n select(Participant_ID = Code)\n\nhead(renaming_col, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 5: Knitting\n\nOnce you've completed your R Markdown file, the final step is to \"knit\" it, which converts the `.Rmd` file into a HTML file. Knitting combines your code, text, and output (like tables and plots) into a single cohesive document. This is a really good way to check your code is working.\n\nTo knit the file, **click the Knit button** at the top of your RStudio window. The document will be generated and, depending on your setting, automatically opened in the viewer in the `Output pane` or an external browser window.\n\nIf any errors occur during knitting, RStudio will show you an error message with details to help you troubleshoot.\n\nIf you want to **intentionally keep any errors** we tackled today to keep a reference on how you solved them, you could add `error=TRUE` or `eval=FALSE` to the code chunk that isn't running.\n\n\n\n## Activity 6: Export a data object as a csv\n\nTo avoid having to repeat the same steps in the next chapter, it's a good idea to save the data objects you've created today as csv files. You can do this by using the `write_csv()` function from the `readr` package. The csv files will appear in your project folder.\n\nThe basic syntax is:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_object, \"filename.csv\")\n```\n:::\n\n\nNow, let's export the objects `data_prp` and `qrp_t1`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_prp, \"data_prp_for_ch3.csv\")\n```\n:::\n\n\nHere we named the file `data_prp_for_ch3.csv`, so we wouldn't override the original data csv file `prp_data_reduced.csv`. However, feel free to choose a name that makes sense to you.\n\n::: {.callout-note icon=\"false\"}\n\n## Your Turn\n\nExport `qrp_t1`.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(qrp_t1, \"qrp_t1.csv\")\n```\n:::\n\n\n:::\n\n:::\n\nCheck that your csv files have appeared in your project folder, and you're all set!\n\n**That’s it for Chapter 2: Individual Walkthrough.**\n\n## [Pair-coding]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n\n::: {.cell layout-align=\"center\"}\n\n:::\n\n\n\nWe will continue working with the data from Binfet et al. (2021), focusing on the randomised controlled trial of therapy dog interventions. Today, our goal is to **calculate an average `Flourishing` score for each participant** at time point 1 (pre-intervention) using the raw data file `dog_data_raw`. Currently, the data looks like this:\n\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Task 1: Open the R project you created last week {.unnumbered}\n\nIf you haven’t created an R project for the lab yet, please do so now. If you already have one set up, go ahead and open it.\n\n\n### Task 2: Open your `.Rmd` file from last week {.unnumbered}\n\nSince we haven’t used it much yet, feel free to continue using the `.Rmd` file you created last week in Task 2.\n\n\n### Task 3: Load in the library and read in the data {.unnumbered}\n\nThe data should be in your project folder. If you didn’t download it last week, or if you’d like a fresh copy, you can download the data again here: [data_pair_ch1](data/data_pair_ch1.zip \"download\").\n\nWe will be using the `tidyverse` package today, and the data file we need to read in is `dog_data_raw.csv`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(???)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"???\")\n```\n:::\n\n\n:::\n\n\n### Task 4: Calculating the mean for `Flourishing_pre` {.unnumbered}\n\n\n* **Step 1**: Select all relevant columns from `dog_data_raw`, including participant ID and all items from the `Flourishing` questionnaire completed before the intervention. Store this data in an object called `data_flourishing`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nLook at the codebook. Try to determine:\n\n* The variable name of the column where the participant ID is stored.\n* The items related to the Flourishing scale at the pre-intervention stage.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nFrom the codebook, we know that:\n\n* The participant ID column is called `RID`.\n* The Flourishing items at the pre-intervention stage start with `F1_`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndata_flourishing <- ??? %>% \n select(???, F1_???:F1_???)\n```\n:::\n\n\n:::\n\n:::\n\n\n* **Step 2**: Pivot the data from wide format to long format so we can calculate the average score more easily (in step 3).\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nWhich pivot function should you use? We have `pivot_wider()` and `pivot_longer()` to choose from.\n\nWe also need 3 arguments in that function:\n\n* The columns you want to select (e.g., all the Flourishing items),\n* The name of the column where the current column headings will be stored (e.g., \"Questionnaire\"),\n* The name of the column that should store all the values (e.g., \"Responses\").\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nWe need `pivot_longer()`. You already encountered `pivot_longer()` in first year (or in the individual walkthrough if you have already completed this Chapter). The 3 arguments was also a give-away; `pivot_wider()` only requires 2 arguments.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n pivot_longer(cols = ???, names_to = \"???\", values_to = \"???\")\n```\n:::\n\n\n:::\n\n:::\n\n* **Step 3**: Calculate the average Flourishing score per participant and name this column `Flourishing_pre` to match the table above.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nBefore summarising the mean, you may need to group the data.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nTo compute an average score **per participant**, we would need to group by participant ID first.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n group_by(???) %>% \n summarise(Flourishing_pre = mean(???)) %>% \n ungroup()\n```\n:::\n\n:::\n\n:::\n\n\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(tidyverse)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"dog_data_raw.csv\")\n\n# Task 4: Tidying \ndata_flourishing <- dog_data_raw %>% \n # Step 1\n select(RID, F1_1:F1_8) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Questionnaire\", values_to = \"Responses\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_pre = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\n:::\n\n\n\n## [Test your knowledge and challenge yourself]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n### Knowledge check {.unnumbered}\n\n#### Question 1 {.unnumbered}\n\nWhich function of the Wickham Six would you use to include or exclude certain variables (columns)? \n\n\n#### Question 2 {.unnumbered}\n\nWhich function of the Wickham Six would you use to create new columns or modify existing columns in a dataframe? \n\n\n#### Question 3 {.unnumbered}\n\n\nWhich function of the Wickham Six would you use to organise data into groups based on one or more columns? \n\n\n\n#### Question 4 {.unnumbered}\n\nWhich function of the Wickham Six would you use to sort the rows of a dataframe based on the values in one or more columns? \n\n\n\n#### Question 5 {.unnumbered}\n\nWhich function of the Wickham Six would NOT modify the original dataframe? \n\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain these answers\n\n| Function | Description |\n|:-------------|:------------------------------------------------------|\n| `select()` | Include or exclude certain variables/columns |\n| `filter()` | Include or exclude certain observations/rows |\n| `mutate()` | Creates new columns or modifies existing ones |\n| `arrange()` | Changes the order of the rows |\n| `group_by()` | Split data into groups based on one or more variables |\n| `summarise()`| Creates a new dataframe returning one row for each combination of grouping variables |\n\n\nTechnically, the first five functions operate on the existing data object, making adjustments like sorting the data (e.g., with `arrange()`), reducing the number of rows (e.g., with `filter()`), reducing the number of columns (e.g., with `select()`), or adding new columns (e.g., with `mutate()`). In contrast, `summarise()` fundamentally alters the structure of the original dataframe by generating a completely new dataframe that contains only summary statistics, rather than retaining the original rows and columns.\n\n:::\n\n\n\n### Error mode {.unnumbered}\n\nSome of the code chunks contain mistakes and result in errors, while others do not produce the expected results. Your task is to identify any issues, explain why they occurred, and, if possible, fix them.\n\nWe will use a few built-in datasets, such as `billboard` and `starwars`, to help you replicate the errors in your own R environment. You can view the data either by typing the dataset name directly into your console or by storing the data as a separate object in your `Global Environment`.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nbillboard\n\nstarwars_data = starwars\n```\n:::\n\n\n\n\n#### Question 6 {.unnumbered}\n\nCurrently, the weekly song rankings for Billboard Top 100 in 2000 are in wide format, with each week in a separate column. The following code is supposed to transpose the wide-format `billboard` data into long format:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(names_to = \"weeks\", values_to = \"rank\")\n```\n\n::: {.cell-output .cell-output-error}\n```\nError in `pivot_longer()`:\n! `cols` must select at least one column.\n```\n:::\n:::\n\n\nWhat does this error message mean and how do you fix it?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe error message indicates that the `cols` argument is missing in the function. This means the function doesn’t know which columns to transpose from wide format to long format.\n\nFIX: Add `cols = wk1:wk76` to the function to select columns from wk1 to wk76. Alternatively, `cols = starts_with(\"wk\")` would also work since all columns start with the letter combination \"wk\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(cols = wk1:wk76, names_to = \"weeks\", values_to = \"rank\")\n# OR\nlong_data <- billboard %>% \n pivot_longer(cols = starts_with(\"wk\"), names_to = \"weeks\", values_to = \"rank\")\n```\n:::\n\n\n:::\n\n\n\n#### Question 7 {.unnumbered}\n\nThe following code is intended to calculate the mean height of all the characters in the built-in `starwars` dataset, grouped by their gender. \n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = height)\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: Returning more (or less) than 1 row per `summarise()` group was deprecated in\ndplyr 1.1.0.\nℹ Please use `reframe()` instead.\nℹ When switching from `summarise()` to `reframe()`, remember that `reframe()`\n always returns an ungrouped data frame and adjust accordingly.\n```\n:::\n:::\n\n\nThe code runs, but it's giving us some weird warning and the output is also not as expected. What steps should we take to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe aggregation function `mean()` is missing from within `summarise()`. Without it, the function does not perform any aggregation and returns *all* rows with only the columns for gender and height.\n\nFIX: Wrap the `mean()` function around the variable you want to aggregate, here `height`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height))\n```\n:::\n\n\n:::\n\n\n\n#### Question 8 {.unnumbered}\n\nFollowing up on Question 7, we now have `summary_data` that looks approximately correct - it has the expected rows and column numbers, however, the cell values are \"weird\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\nCan you explain what is happening here? And how can we modify the code to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nLook at the original `starwars` data. You will notice that some of the characters with feminine and masculine gender entries have missing height values. However, all four characters without a specified gender have provided their height.\n\nFIX: We need to add `na.rm = TRUE` to the `mean()` function to ensure that R ignores missing values before aggregating the data.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height, na.rm = TRUE))\n\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n### Challenge yourself {.unnumbered}\n\nIf you want to **challenge yourself** and further apply the skills from Chapter 2, you can wrangle the data from `dog_data_raw` for additional questionnaires from either the pre- and/or post-intervention stages:\n\n* Calculate the mean score for `flourishing_post` for each participant.\n* Calculate the mean score for the `PANAS` (Positive and/or Negative Affect) per participant\n* Calculate the mean score for happiness (`SHS`) per participant\n\nThe 3 steps are equivalent for those questionnaires - select, pivot, group_by and summarise; you just have to \"replace\" the questionnaire items involved.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution for **Challenge yourself**\n\nFlourishing post-intervention\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n## flourishing_post\nflourishing_post <- dog_data_raw %>% \n # Step 1\n select(RID, starts_with(\"F2\")) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Names\", values_to = \"Response\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_post = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\nThe PANAS could be solved more concisely with the skills we learn in @sec-wrangling2, but for now, you would have solved it this way:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# PANAS - positive affect pre\nPANAS_PA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_3, PN1_5, PN1_7, PN1_8, PN1_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - positive affect post\nPANAS_PA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_3, PN2_5, PN2_7, PN2_8, PN2_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_post = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect pre\nPANAS_NA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_1, PN1_2, PN1_4, PN1_6, PN1_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect post\nPANAS_NA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_1, PN2_2, PN2_4, PN2_6, PN2_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_post = mean(Scores)) %>% \n ungroup()\n```\n:::\n\n\nHappiness scale\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# happiness_pre\nhappiness_pre <- dog_data_raw %>% \n # Step 1\n select(RID, HA1_1, HA1_2, HA1_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_pre = mean(Score)) %>% \n ungroup()\n\n#happiness_post\nhappiness_post <- dog_data_raw %>% \n # Step 1\n select(RID, HA2_1, HA2_2, HA2_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_post = mean(Score)) %>% \n ungroup()\n```\n:::\n\n\n:::\n",
+ "supporting": [],
"filters": [
"rmarkdown/pagebreak.lua"
],
diff --git a/.quarto/cites/index.json b/.quarto/cites/index.json
index b2004e5..02e2ca5 100644
--- a/.quarto/cites/index.json
+++ b/.quarto/cites/index.json
@@ -1 +1 @@
-{"11-multiple-regression.qmd":[],"05-dataviz2.qmd":[],"09-simple-regression.qmd":[],"12-factorial-anova.qmd":[],"06-paired.qmd":[],"08-paired.qmd":[],"instructions.qmd":["usethis"],"appendix-a-installing-r.qmd":[],"appendix-c-exporting-server.qmd":[],"07-independent.qmd":[],"09-correlation.qmd":[],"05-chi-square-one-sample.qmd":[],"01-basics.qmd":[],"04-dataviz.qmd":[],"10-multiple-regression.qmd":[],"04-dataviz2.qmd":[],"index.qmd":[],"02-wrangling.qmd":[],"webexercises.qmd":[],"13-factorial-anova.qmd":[],"05-independent.qmd":[],"appendix-x-How-to-cite-R.qmd":[],"appendix-b-updating-packages.qmd":[],"04-prob-binom-one-sample.qmd":[],"11-one-way-anova.qmd":[],"07-paired.qmd":[],"references.qmd":[],"10-regression.qmd":[],"appendix-y-license.qmd":[],"03-wrangling2.qmd":[],"06-independent.qmd":[],"12-one-way-anova.qmd":[],"08-correlation.qmd":[],"04-chi-square-one-sample.qmd":[],"06-chi-square-one-sample.qmd":[],"07-apes.qmd":[],"03-dataviz.qmd":[],"appendix-d-symbols.qmd":[]}
+{"09-correlation.qmd":[],"09-simple-regression.qmd":[],"05-dataviz2.qmd":[],"07-apes.qmd":[],"11-one-way-anova.qmd":[],"11-multiple-regression.qmd":[],"06-paired.qmd":[],"08-paired.qmd":[],"appendix-c-exporting-server.qmd":[],"06-independent.qmd":[],"04-dataviz.qmd":[],"10-regression.qmd":[],"references.qmd":[],"04-dataviz2.qmd":[],"index.qmd":[],"13-factorial-anova.qmd":[],"01-basics.qmd":[],"08-correlation.qmd":[],"02-wrangling.qmd":[],"12-one-way-anova.qmd":[],"appendix-d-symbols.qmd":[],"10-multiple-regression.qmd":[],"03-wrangling2.qmd":[],"05-independent.qmd":[],"instructions.qmd":["usethis"],"03-dataviz.qmd":[],"appendix-a-installing-r.qmd":[],"appendix-x-How-to-cite-R.qmd":[],"07-independent.qmd":[],"06-chi-square-one-sample.qmd":[],"04-chi-square-one-sample.qmd":[],"appendix-b-updating-packages.qmd":[],"05-chi-square-one-sample.qmd":[],"12-factorial-anova.qmd":[],"webexercises.qmd":[],"07-paired.qmd":[],"appendix-y-license.qmd":[],"04-prob-binom-one-sample.qmd":[]}
diff --git a/.quarto/xref/7a0d69cd b/.quarto/xref/7a0d69cd
index a5059bd..f0417c8 100644
--- a/.quarto/xref/7a0d69cd
+++ b/.quarto/xref/7a0d69cd
@@ -1 +1 @@
-{"entries":[{"order":{"section":[2,0,0,0,0,0,0],"number":1},"key":"sec-wrangling"},{"caption":"2.4 Activity 4: Questionable Research Practices (QRPs)","order":{"section":[2,4,0,0,0,0,0],"number":1},"key":"sec-ch2_act4"}],"options":{"chapter-id":"sec-wrangling","chapters":true},"headings":["intended-learning-outcomes","individual-walkthrough","activity-1-setup","activity-2-load-in-the-libraries-and-read-in-the-data","activity-3-calculating-demographics","for-the-full-sample-using-summarise","fixing-age","computing-summary-stats","computing-summary-stats---third-attempt","per-gender-using-summarise-and-group_by","adding-percentages","sec-ch2_act4","the-main-goal-is-to-compute-the-mean-qrp-score-per-participant-for-time-point-1.","activity-5-knitting","activity-6-export-a-data-object-as-a-csv","pair-coding","task-1-open-the-r-project-you-created-last-week","task-2-open-your-.rmd-file-from-last-week","task-3-load-in-the-library-and-read-in-the-data","task-4-calculating-the-mean-for-flourishing_pre","test-your-knowledge-and-challenge-yourself","knowledge-check","question-1","question-2","question-3","question-4","question-5","error-mode","question-6","question-7","question-8","challenge-yourself","sec-wrangling"]}
\ No newline at end of file
+{"entries":[{"key":"sec-wrangling","order":{"section":[2,0,0,0,0,0,0],"number":1}},{"caption":"2.4 Activity 4: Questionable Research Practices (QRPs)","key":"sec-ch2_act4","order":{"section":[2,4,0,0,0,0,0],"number":1}}],"headings":["intended-learning-outcomes","individual-walkthrough","activity-1-setup","activity-2-load-in-the-libraries-and-read-in-the-data","activity-3-calculating-demographics","for-the-full-sample-using-summarise","fixing-age","computing-summary-stats","computing-summary-stats---third-attempt","per-gender-using-summarise-and-group_by","adding-percentages","sec-ch2_act4","the-main-goal-is-to-compute-the-mean-qrp-score-per-participant-for-time-point-1.","activity-5-knitting","activity-6-export-a-data-object-as-a-csv","pair-coding","task-1-open-the-r-project-you-created-last-week","task-2-open-your-.rmd-file-from-last-week","task-3-load-in-the-library-and-read-in-the-data","task-4-calculating-the-mean-for-flourishing_pre","test-your-knowledge-and-challenge-yourself","knowledge-check","question-1","question-2","question-3","question-4","question-5","error-mode","question-6","question-7","question-8","challenge-yourself","sec-wrangling"],"options":{"chapter-id":"sec-wrangling","chapters":true}}
\ No newline at end of file
diff --git a/02-wrangling.qmd b/02-wrangling.qmd
index 924163e..a3346f5 100644
--- a/02-wrangling.qmd
+++ b/02-wrangling.qmd
@@ -261,16 +261,25 @@ demo_by_gender
#### The main goal is to compute the mean QRP score per participant for time point 1. {.unnumbered}
+At the moment, the data is in wide format. The table below shows data from the first 3 participants:
+
+```{r}
+head(data_prp, n = 3)
+```
+
+
Looking at the QRP data at time point 1, you determine that
* individual item columns are `r mcq(c(answer = "numeric", x = "character"))`, and
* according to the codebook, there are `r mcq(c(answer = "no", x = "some"))` reverse-coded items in this questionnaire.
-So, we just have to **compute an average score for items 1 to 11** as items 12 to 15 are distractor items. Seems quite straightforward.
+According to the codebook and the data table above, we just have to **compute the average score for QRP items `r fitb("1")` to `r fitb("11")`**, since items `r fitb("12")` to `r fitb("15")` are distractor items. Seems quite straightforward.
-The downside is that individual items are each in a separate column, i.e., in **wide format**, and everything would be easier if the items were arranged in **long format**.
+However, as you can see in the table above, each item is in a separate column, meaning the data is in **wide format**. It would be much easier to calculate the mean scores if the items were arranged in **long format**.
+
+
+Let’s tackle this problem step by step. It’s best to create a separate data object for this. If we tried to compute it within `data_prp`, it could quickly become messy.
-Let's tackle this problem in steps. Best would be to create a separate data object for that. If we wanted to compute this within `data_prp`, it would turn into a nightmare.
* **Step 1**: Select the relevant columns `Code`, and `QRPs_1_Time1` to `QRPs_1_Time1` and store them in an object called `qrp_t1`
* **Step 2**: Pivot the data from wide format to long format using `pivot_longer()` so we can calculate the average score more easily (in step 3)
@@ -283,7 +292,7 @@ qrp_t1 <- data_prp %>%
# Step 2
pivot_longer(cols = -Code, names_to = "Items", values_to = "Scores") %>%
# Step 3
- group_by(Code) %>% # grouping py participant id
+ group_by(Code) %>% # grouping by participant id
summarise(QRPs_Acceptance_Time1_mean = mean(Scores)) %>% # calculating the average Score
ungroup() # just make it a habit
```
@@ -479,7 +488,8 @@ dog_data_raw <- read_csv("???")
### Task 4: Calculating the mean for `Flourishing_pre` {.unnumbered}
-* **Step 1**: Select all relevant columns, including participant ID and all 8 items from the `Flourishing` questionnaire completed before the intervention. Store this data in an object called `data_flourishing`.
+
+* **Step 1**: Select all relevant columns from `dog_data_raw`, including participant ID and all items from the `Flourishing` questionnaire completed before the intervention. Store this data in an object called `data_flourishing`.
::: {.callout-note collapse="true" icon="false"}
@@ -500,6 +510,11 @@ From the codebook, we know that:
* The participant ID column is called `RID`.
* The Flourishing items at the pre-intervention stage start with `F1_`.
+```{r eval=FALSE}
+data_flourishing <- ??? %>%
+ select(???, F1_???:F1_???)
+```
+
:::
:::
@@ -524,6 +539,8 @@ We also need 3 arguments in that function:
## More concrete hint
+We need `pivot_longer()`. You already encountered `pivot_longer()` in first year (or in the individual walkthrough if you have already completed this Chapter). The 3 arguments was also a give-away; `pivot_wider()` only requires 2 arguments.
+
```{r eval=FALSE}
pivot_longer(cols = ???, names_to = "???", values_to = "???")
```
@@ -545,9 +562,11 @@ Before summarising the mean, you may need to group the data.
## More concrete hint
+To compute an average score **per participant**, we would need to group by participant ID first.
+
```{r eval=FALSE}
group_by(???) %>%
- summarise(Flourishing_pre = ???(???)) %>%
+ summarise(Flourishing_pre = mean(???)) %>%
ungroup()
```
:::
@@ -555,6 +574,8 @@ Before summarising the mean, you may need to group the data.
:::
+
+
::: {.callout-caution collapse="true" icon="false"}
## Solution
diff --git a/_freeze/02-wrangling/execute-results/html.json b/_freeze/02-wrangling/execute-results/html.json
index 3abe2c8..e5f22d3 100644
--- a/_freeze/02-wrangling/execute-results/html.json
+++ b/_freeze/02-wrangling/execute-results/html.json
@@ -1,10 +1,8 @@
{
- "hash": "1740a607e9de7ca322e49f331bb6e0b3",
+ "hash": "85f2fa0ec2f8baeeb06d4b656ca711c6",
"result": {
- "markdown": "# Data wrangling I {#sec-wrangling}\n\n## Intended Learning Outcomes {.unnumbered}\n\nIn the next two chapters, we will build on the data wrangling skills from level 1. We will revisit all the functions you have already encountered (and might have forgotten over the summer break) and introduce 2 or 3 new functions. These two chapters will provide an opportunity to revise and apply the functions to a novel dataset.\n\nBy the end of this chapter, you should be able to:\n\n- apply familiar data wrangling functions to novel datasets\n- read and interpret error messages\n- realise there are several ways of getting to the results\n- export data objects as csv files\n\nThe main purpose of this chapter and @sec-wrangling2 is to wrangle your data into shape for data visualisation (@sec-dataviz and @sec-dataviz2). For the two chapters, we will:\n\n1. calculate demographics\n2. tidy 3 different questionnaires with varying degrees of complexity\n3. solve an error mode problem\n4. join all data objects together\n\n## [Individual Walkthrough]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\nBefore we start, we need to set up some things.\n\n\n## Activity 1: Setup\n\n* We will be working on the **dataset by Pownall et al. (2023)** again, which means we can still use the project we created last week. The data files will already be there, so no need to download them again.\n* To **open the project** in RStudio, go to the folder in which you stored the project and the data last time, and double click on the project icon.\n* **Create a new `.Rmd` file** for chapter 2 and save it to your project folder. Name it something meaningful (e.g., “chapter_02.Rmd”, “02_data_wrangling.Rmd”). See @sec-rmd if you need some guidance.\n* In your newly created `.Rmd` file, delete everything below line 12 (after the set-up code chunk).\n\n\n\n## Activity 2: Load in the libraries and read in the data\n\nWe will use `tidyverse` today, and we want to create a data object `data_prp` that stores the data from the file `prp_data_reduced.csv`.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(???)\ndata_prp <- read_csv(\"???\")\n```\n:::\n\n\n\n\n:::\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(tidyverse)\ndata_prp <- read_csv(\"prp_data_reduced.csv\")\n```\n:::\n\n\n:::\n\nIf you need a quick reminder what the dataset was about, have a look at the abstract in @sec-download_data_ch1. We also addressed the changes we made to the dataset there.\n\nAnd remember to have a quick `glimpse()` at your data.\n\n\n\n## Activity 3: Calculating demographics\n\nLet’s start with some simple data-wrangling steps to compute demographics for our original dataset, `data_prp`. First, we want to determine how many participants took part in the study by Pownall et al. (2023) and compute the mean age and the standard deviation of age for the sample.\n\n\n\n### ... for the full sample using `summarise()`\n\nThe `summarise()` function is part of the **\"Wickham Six\"** alongside `group_by()`, `select()`, `filter()`, `mutate()`, and `arrange()`. You used them plenty of times last year.\n\nWithin `summarise()`, we can use the `n()` function, which calculates the number of rows in the dataset. Since each row corresponds to a unique participant, this gives us the total number of participants.\n\nTo calculate the mean age and the standard deviation of age, we need to use the functions `mean()` and `sd()` on the column `Age` respectively.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: There were 2 warnings in `summarise()`.\nThe first warning was:\nℹ In argument: `mean_age = mean(Age)`.\nCaused by warning in `mean.default()`:\n! argument is not numeric or logical: returning NA\nℹ Run `dplyr::last_dplyr_warnings()` to see the 1 remaining warning.\n```\n:::\n\n```{.r .cell-code}\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nR did not give us an error message per se, but the output is not quite as expected either. There are `NA` values in the `mean_age` and `sd_age` columns. Looking at the warning message and at `Age`, can you explain what happened?\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nThe warning message says: `argument is not numeric or logical: returning NA` If we look at the `Age` column more closely, we can see that it's a character data type.\n\n:::\n\n\n\n#### Fixing `Age` {.unnumbered}\n\nMight be wise to look at the unique answers in column `Age` to determine what is wrong. We can do that with the function `distinct()`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nage_distinct <- data_prp %>% \n distinct(Age)\n\nage_distinct\n```\n:::\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Show the unique values of `Age`.\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n:::\n\n::: columns\n\n::: column\n\nOne cell has the string \"years\" added to their number 25, which has converted the entire column into a character column.\n\nWe can easily fix this by extracting only the numbers from the column and converting it into a numeric data type. The `parse_number()` function, which is part of the `tidyverse` package, handles both steps in one go (so there’s no need to load additional packages).\n\nWe will combine this with the `mutate()` function to create a new column called `Age` (containing those numeric values), effectively replacing the old `Age` column (which had the character values).\n\n:::\n\n::: column\n\n{target=\"_blank\"})](images/parse_number.png){width=\"95%\"}\n\n:::\n\n:::\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndata_prp <- data_prp %>% \n mutate(Age = parse_number(Age))\n\ntypeof(data_prp$Age) # fixed\n```\n\n::: {.cell-output .cell-output-stdout}\n```\n[1] \"double\"\n```\n:::\n:::\n\n\n\n\n#### Computing summary stats {.unnumbered}\n\nExcellent. Now that the numbers are in a numeric format, let's try calculating the demographics for the total sample again.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nEven though there's no error or warning, the table still shows `NA` values for `mean_age` and `sd_age`. So, what could possibly be wrong now?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nDid you notice that the `Age` column in `age_distinct` contains some missing values (`NA`)? To be honest, it's easier to spot this issue in the actual R output than in the printed HTML page.\n\n:::\n\n\n\n#### Computing summary stats - third attempt {.unnumbered}\n\nTo ensure R ignores missing values during calculations, we need to add the extra argument `na.rm = TRUE` to the `mean()` and `sd()` functions.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age, na.rm = TRUE), # mean age\n sd_age = sd(Age, na.rm = TRUE)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nFinally, we’ve got it! 🥳 Third time's the charm!\n\n\n\n### ... per gender using `summarise()` and `group_by()`\n\nNow we want to compute the summary statistics for each gender. The code inside the `summarise()` function remains unchanged; we just need to use the `group_by()` function beforehand to tell R that we want to compute the summary statistics for each group separately. It’s also a good practice to use `ungroup()` afterwards, so you are not taking groupings forward unintentionally.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% # split data up into groups (here Gender)\n summarise(n = n(), # participant number \n mean_age = mean(Age, na.rm = TRUE), # mean age \n sd_age = sd(Age, na.rm = TRUE)) %>% # standard deviation of age\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Adding percentages\n\nSometimes, it may be useful to calculate percentages, such as for the gender split. You can do this by adding a line within the `summarise()` function to perform the calculation. All we need to do is take the number of female, male, and non-binary participants (stored in the `n` column of `demo_by_gender`), divide it by the total number of participants (stored in the `n` column of `demo_total`), and multiply by 100. Let's add `percentage` to the `summarise()` function of `demo_by_gender`. Make sure that the code for `percentages` is placed after the value for `n` has been computed.\n\nAccessing the value of `n` for the different gender categories is straightforward because we can refer back to it directly. However, since the total number of participants is stored in a different data object, we need to use a base R function to access it – specifically the `$` operator. To do this, you simply type the name of the data object (in this case, `demo_total`), followed by the `$` symbol (with no spaces), and then the name of the column you want to retrieve (in this case, `n`). The general pattern is `data$column`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n # n from the line above divided by n from demo_total *100\n percentage = n/demo_total$n *100, \n mean_age = mean(Age, na.rm = TRUE), \n sd_age = sd(Age, na.rm = TRUE)) %>% \n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n::: {.callout-tip collapse=\"true\"}\n\n## Tip for decimal places - use `round()`\n\nNot super important, because you could round the values by yourself when writing up your reports, but if you wanted to tidy up the decimal places in the output, you can do that using the `round()` function. You would need to \"wrap\" it around your computations and specify how many decimal places you want to display (for example `mean(Age)` would turn into `round(mean(Age), 1)`). It may look odd for `percentage`, just make sure the number that specifies the decimal places is placed **within** the round function. The default value is 0 (meaning no decimal spaces).\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n percentage = round(n/demo_total$n *100, 2), # percentage with 2 decimal places\n mean_age = round(mean(Age, na.rm = TRUE), 1), # mean Age with 1 decimal place\n sd_age = round(sd(Age, na.rm = TRUE), 3)) %>% # sd Age with 3 decimal places\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 4: Questionable Research Practices (QRPs) {#sec-ch2_act4}\n\n#### The main goal is to compute the mean QRP score per participant for time point 1. {.unnumbered}\n\nLooking at the QRP data at time point 1, you determine that\n\n* individual item columns are , and\n* according to the codebook, there are reverse-coded items in this questionnaire.\n\nSo, we just have to **compute an average score for items 1 to 11** as items 12 to 15 are distractor items. Seems quite straightforward.\n\nThe downside is that individual items are each in a separate column, i.e., in **wide format**, and everything would be easier if the items were arranged in **long format**.\n\nLet's tackle this problem in steps. Best would be to create a separate data object for that. If we wanted to compute this within `data_prp`, it would turn into a nightmare.\n\n* **Step 1**: Select the relevant columns `Code`, and `QRPs_1_Time1` to `QRPs_1_Time1` and store them in an object called `qrp_t1`\n* **Step 2**: Pivot the data from wide format to long format using `pivot_longer()` so we can calculate the average score more easily (in step 3)\n* **Step 3**: Calculate the average QRP score (`QRPs_Acceptance_Time1_mean`) per participant using `group_by()` and `summarise()`\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_t1 <- data_prp %>% \n #Step 1\n select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%\n # Step 2\n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(Code) %>% # grouping py participant id\n summarise(QRPs_Acceptance_Time1_mean = mean(Scores)) %>% # calculating the average Score\n ungroup() # just make it a habit\n```\n:::\n\n\n::: {.callout-caution icon=\"false\" collapse=\"true\"}\n\n## Explain the individual functions\n\n::: panel-tabset\n\n## `select ()`\n\nThe select function allows to include or exclude certain variables (columns). Here we want to focus on the participant ID column (i.e., `Code`) and the QRP items at time point 1. We can either list them all individually, i.e., Code, QRPs_1_Time1, QRPs_2_Time1, QRPs_3_Time1, and so forth (you get the gist), but that would take forever to type.\n\nA shortcut is to use the colon operator `:`. It allows us to select all columns that fall within the range of `first_column_name` to `last_column_name`. We can apply this here since the QRP items (1 to 11) are sequentially listed in `data_prp`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step1 <- data_prp %>% \n select(Code, QRPs_1_Time1:QRPs_11_Time1)\n\n# show first 5 rows of qrp_step1\nhead(qrp_step1, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nHow many rows/observations and columns/variables do we have in `qrp_step1`?\n\n* rows/observations: \n* columns/variables: \n\n## `pivot_longer()`\n\nAs you can see, the table we got from Step 1 is in wide format. To get it into wide format, we need to define:\n\n* the columns that need to be reshuffled from wide into long format (`col` argument). Here we selected \"everything except the `Code` column\", as indicated by `-Code` \\[minus `Code`\\]. However, `QRPs_1_Time1:QRPs_11_Time1` would also work and give you the exact same result.\n* the `names_to` argument. R is creating a new column in which all the column names from the columns you selected in `col` will be stored in. Here we are naming this column \"Items\" but you could pick something equally sensible if you like.\n* the `values_to` argument. R creates this second column to store all responses the participants gave to the individual questions, i.e., all the numbers in this case. We named it \"Scores\" here, but you could have called it something different, like \"Responses\"\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step2 <- qrp_step1 %>% \n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\")\n\n# show first 15 rows of qrp_step2\nhead(qrp_step2, n = 15)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nNow, have a look at `qrp_step2`. In total, we now have rows/observations, per participant, and columns/variables.\n\n## `group_by()` and `summarise()`\n\nThis follows exactly the same sequence we used when calculating descriptive statistics by gender. The only difference is that we are now grouping the data by the participant's `Code` instead of `Gender`.\n\n`summarise()` works exactly the same way: `summarise(new_column_name = function_to_calculate_something(column_name_of_numeric_values))`\n\nThe `function_to_calculate_something` can be `mean()`, `sd()` or `sum()` for mean scores, standard deviations, or summed-up scores respectively. You could also use `min()` or `max()` if you wanted to determine the lowest or the highest score for each participant.\n\n:::\n\n:::\n\n::: callout-tip\n\nYou could **rename the columns whilst selecting** them. The pattern would be `select(new_name = old_name)`. For example, if we wanted to select variable `Code` and rename it as `Participant_ID`, we could do that.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nrenaming_col <- data_prp %>% \n select(Participant_ID = Code)\n\nhead(renaming_col, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 5: Knitting\n\nOnce you've completed your R Markdown file, the final step is to \"knit\" it, which converts the `.Rmd` file into a HTML file. Knitting combines your code, text, and output (like tables and plots) into a single cohesive document. This is a really good way to check your code is working.\n\nTo knit the file, **click the Knit button** at the top of your RStudio window. The document will be generated and, depending on your setting, automatically opened in the viewer in the `Output pane` or an external browser window.\n\nIf any errors occur during knitting, RStudio will show you an error message with details to help you troubleshoot.\n\nIf you want to **intentionally keep any errors** we tackled today to keep a reference on how you solved them, you could add `error=TRUE` or `eval=FALSE` to the code chunk that isn't running.\n\n\n\n## Activity 6: Export a data object as a csv\n\nTo avoid having to repeat the same steps in the next chapter, it's a good idea to save the data objects you've created today as csv files. You can do this by using the `write_csv()` function from the `readr` package. The csv files will appear in your project folder.\n\nThe basic syntax is:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_object, \"filename.csv\")\n```\n:::\n\n\nNow, let's export the objects `data_prp` and `qrp_t1`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_prp, \"data_prp_for_ch3.csv\")\n```\n:::\n\n\nHere we named the file `data_prp_for_ch3.csv`, so we wouldn't override the original data csv file `prp_data_reduced.csv`. However, feel free to choose a name that makes sense to you.\n\n::: {.callout-note icon=\"false\"}\n\n## Your Turn\n\nExport `qrp_t1`.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(qrp_t1, \"qrp_t1.csv\")\n```\n:::\n\n\n:::\n\n:::\n\nCheck that your csv files have appeared in your project folder, and you're all set!\n\n**That’s it for Chapter 2: Individual Walkthrough.**\n\n## [Pair-coding]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n\n::: {.cell layout-align=\"center\"}\n\n:::\n\n\n\nWe will continue working with the data from Binfet et al. (2021), focusing on the randomised controlled trial of therapy dog interventions. Today, our goal is to **calculate an average `Flourishing` score for each participant** at time point 1 (pre-intervention) using the raw data file `dog_data_raw`. Currently, the data looks like this:\n\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Task 1: Open the R project you created last week {.unnumbered}\n\nIf you haven’t created an R project for the lab yet, please do so now. If you already have one set up, go ahead and open it.\n\n\n### Task 2: Open your `.Rmd` file from last week {.unnumbered}\n\nSince we haven’t used it much yet, feel free to continue using the `.Rmd` file you created last week in Task 2.\n\n\n### Task 3: Load in the library and read in the data {.unnumbered}\n\nThe data should be in your project folder. If you didn’t download it last week, or if you’d like a fresh copy, you can download the data again here: [data_pair_ch1](data/data_pair_ch1.zip \"download\").\n\nWe will be using the `tidyverse` package today, and the data file we need to read in is `dog_data_raw.csv`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(???)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"???\")\n```\n:::\n\n\n:::\n\n\n### Task 4: Calculating the mean for `Flourishing_pre` {.unnumbered}\n\n* **Step 1**: Select all relevant columns, including participant ID and all 8 items from the `Flourishing` questionnaire completed before the intervention. Store this data in an object called `data_flourishing`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nLook at the codebook. Try to determine:\n\n* The variable name of the column where the participant ID is stored.\n* The items related to the Flourishing scale at the pre-intervention stage.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nFrom the codebook, we know that:\n\n* The participant ID column is called `RID`.\n* The Flourishing items at the pre-intervention stage start with `F1_`.\n\n:::\n\n:::\n\n\n* **Step 2**: Pivot the data from wide format to long format so we can calculate the average score more easily (in step 3).\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nWhich pivot function should you use? We have `pivot_wider()` and `pivot_longer()` to choose from.\n\nWe also need 3 arguments in that function:\n\n* The columns you want to select (e.g., all the Flourishing items),\n* The name of the column where the current column headings will be stored (e.g., \"Questionnaire\"),\n* The name of the column that should store all the values (e.g., \"Responses\").\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n pivot_longer(cols = ???, names_to = \"???\", values_to = \"???\")\n```\n:::\n\n\n:::\n\n:::\n\n* **Step 3**: Calculate the average Flourishing score per participant and name this column `Flourishing_pre` to match the table above.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nBefore summarising the mean, you may need to group the data.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n group_by(???) %>% \n summarise(Flourishing_pre = ???(???)) %>% \n ungroup()\n```\n:::\n\n:::\n\n:::\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(tidyverse)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"dog_data_raw.csv\")\n\n# Task 4: Tidying \ndata_flourishing <- dog_data_raw %>% \n # Step 1\n select(RID, F1_1:F1_8) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Questionnaire\", values_to = \"Responses\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_pre = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\n:::\n\n\n\n## [Test your knowledge and challenge yourself]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n### Knowledge check {.unnumbered}\n\n#### Question 1 {.unnumbered}\n\nWhich function of the Wickham Six would you use to include or exclude certain variables (columns)? \n\n\n#### Question 2 {.unnumbered}\n\nWhich function of the Wickham Six would you use to create new columns or modify existing columns in a dataframe? \n\n\n#### Question 3 {.unnumbered}\n\n\nWhich function of the Wickham Six would you use to organise data into groups based on one or more columns? \n\n\n\n#### Question 4 {.unnumbered}\n\nWhich function of the Wickham Six would you use to sort the rows of a dataframe based on the values in one or more columns? \n\n\n\n#### Question 5 {.unnumbered}\n\nWhich function of the Wickham Six would NOT modify the original dataframe? \n\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain these answers\n\n| Function | Description |\n|:-------------|:------------------------------------------------------|\n| `select()` | Include or exclude certain variables/columns |\n| `filter()` | Include or exclude certain observations/rows |\n| `mutate()` | Creates new columns or modifies existing ones |\n| `arrange()` | Changes the order of the rows |\n| `group_by()` | Split data into groups based on one or more variables |\n| `summarise()`| Creates a new dataframe returning one row for each combination of grouping variables |\n\n\nTechnically, the first five functions operate on the existing data object, making adjustments like sorting the data (e.g., with `arrange()`), reducing the number of rows (e.g., with `filter()`), reducing the number of columns (e.g., with `select()`), or adding new columns (e.g., with `mutate()`). In contrast, `summarise()` fundamentally alters the structure of the original dataframe by generating a completely new dataframe that contains only summary statistics, rather than retaining the original rows and columns.\n\n:::\n\n\n\n### Error mode {.unnumbered}\n\nSome of the code chunks contain mistakes and result in errors, while others do not produce the expected results. Your task is to identify any issues, explain why they occurred, and, if possible, fix them.\n\nWe will use a few built-in datasets, such as `billboard` and `starwars`, to help you replicate the errors in your own R environment. You can view the data either by typing the dataset name directly into your console or by storing the data as a separate object in your `Global Environment`.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nbillboard\n\nstarwars_data = starwars\n```\n:::\n\n\n\n\n#### Question 6 {.unnumbered}\n\nCurrently, the weekly song rankings for Billboard Top 100 in 2000 are in wide format, with each week in a separate column. The following code is supposed to transpose the wide-format `billboard` data into long format:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(names_to = \"weeks\", values_to = \"rank\")\n```\n\n::: {.cell-output .cell-output-error}\n```\nError in `pivot_longer()`:\n! `cols` must select at least one column.\n```\n:::\n:::\n\n\nWhat does this error message mean and how do you fix it?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe error message indicates that the `cols` argument is missing in the function. This means the function doesn’t know which columns to transpose from wide format to long format.\n\nFIX: Add `cols = wk1:wk76` to the function to select columns from wk1 to wk76. Alternatively, `cols = starts_with(\"wk\")` would also work since all columns start with the letter combination \"wk\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(cols = wk1:wk76, names_to = \"weeks\", values_to = \"rank\")\n# OR\nlong_data <- billboard %>% \n pivot_longer(cols = starts_with(\"wk\"), names_to = \"weeks\", values_to = \"rank\")\n```\n:::\n\n\n:::\n\n\n\n#### Question 7 {.unnumbered}\n\nThe following code is intended to calculate the mean height of all the characters in the built-in `starwars` dataset, grouped by their gender. \n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = height)\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: Returning more (or less) than 1 row per `summarise()` group was deprecated in\ndplyr 1.1.0.\nℹ Please use `reframe()` instead.\nℹ When switching from `summarise()` to `reframe()`, remember that `reframe()`\n always returns an ungrouped data frame and adjust accordingly.\n```\n:::\n:::\n\n\nThe code runs, but it's giving us some weird warning and the output is also not as expected. What steps should we take to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe aggregation function `mean()` is missing from within `summarise()`. Without it, the function does not perform any aggregation and returns *all* rows with only the columns for gender and height.\n\nFIX: Wrap the `mean()` function around the variable you want to aggregate, here `height`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height))\n```\n:::\n\n\n:::\n\n\n\n#### Question 8 {.unnumbered}\n\nFollowing up on Question 7, we now have `summary_data` that looks approximately correct - it has the expected rows and column numbers, however, the cell values are \"weird\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\nCan you explain what is happening here? And how can we modify the code to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nLook at the original `starwars` data. You will notice that some of the characters with feminine and masculine gender entries have missing height values. However, all four characters without a specified gender have provided their height.\n\nFIX: We need to add `na.rm = TRUE` to the `mean()` function to ensure that R ignores missing values before aggregating the data.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height, na.rm = TRUE))\n\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n### Challenge yourself {.unnumbered}\n\nIf you want to **challenge yourself** and further apply the skills from Chapter 2, you can wrangle the data from `dog_data_raw` for additional questionnaires from either the pre- and/or post-intervention stages:\n\n* Calculate the mean score for `flourishing_post` for each participant.\n* Calculate the mean score for the `PANAS` (Positive and/or Negative Affect) per participant\n* Calculate the mean score for happiness (`SHS`) per participant\n\nThe 3 steps are equivalent for those questionnaires - select, pivot, group_by and summarise; you just have to \"replace\" the questionnaire items involved.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution for **Challenge yourself**\n\nFlourishing post-intervention\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n## flourishing_post\nflourishing_post <- dog_data_raw %>% \n # Step 1\n select(RID, starts_with(\"F2\")) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Names\", values_to = \"Response\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_post = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\nThe PANAS could be solved more concisely with the skills we learn in @sec-wrangling2, but for now, you would have solved it this way:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# PANAS - positive affect pre\nPANAS_PA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_3, PN1_5, PN1_7, PN1_8, PN1_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - positive affect post\nPANAS_PA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_3, PN2_5, PN2_7, PN2_8, PN2_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_post = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect pre\nPANAS_NA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_1, PN1_2, PN1_4, PN1_6, PN1_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect post\nPANAS_NA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_1, PN2_2, PN2_4, PN2_6, PN2_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_post = mean(Scores)) %>% \n ungroup()\n```\n:::\n\n\nHappiness scale\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# happiness_pre\nhappiness_pre <- dog_data_raw %>% \n # Step 1\n select(RID, HA1_1, HA1_2, HA1_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_pre = mean(Score)) %>% \n ungroup()\n\n#happiness_post\nhappiness_post <- dog_data_raw %>% \n # Step 1\n select(RID, HA2_1, HA2_2, HA2_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_post = mean(Score)) %>% \n ungroup()\n```\n:::\n\n\n:::\n",
- "supporting": [
- "02-wrangling_files"
- ],
+ "markdown": "# Data wrangling I {#sec-wrangling}\n\n## Intended Learning Outcomes {.unnumbered}\n\nIn the next two chapters, we will build on the data wrangling skills from level 1. We will revisit all the functions you have already encountered (and might have forgotten over the summer break) and introduce 2 or 3 new functions. These two chapters will provide an opportunity to revise and apply the functions to a novel dataset.\n\nBy the end of this chapter, you should be able to:\n\n- apply familiar data wrangling functions to novel datasets\n- read and interpret error messages\n- realise there are several ways of getting to the results\n- export data objects as csv files\n\nThe main purpose of this chapter and @sec-wrangling2 is to wrangle your data into shape for data visualisation (@sec-dataviz and @sec-dataviz2). For the two chapters, we will:\n\n1. calculate demographics\n2. tidy 3 different questionnaires with varying degrees of complexity\n3. solve an error mode problem\n4. join all data objects together\n\n## [Individual Walkthrough]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\nBefore we start, we need to set up some things.\n\n\n## Activity 1: Setup\n\n* We will be working on the **dataset by Pownall et al. (2023)** again, which means we can still use the project we created last week. The data files will already be there, so no need to download them again.\n* To **open the project** in RStudio, go to the folder in which you stored the project and the data last time, and double click on the project icon.\n* **Create a new `.Rmd` file** for chapter 2 and save it to your project folder. Name it something meaningful (e.g., “chapter_02.Rmd”, “02_data_wrangling.Rmd”). See @sec-rmd if you need some guidance.\n* In your newly created `.Rmd` file, delete everything below line 12 (after the set-up code chunk).\n\n\n\n## Activity 2: Load in the libraries and read in the data\n\nWe will use `tidyverse` today, and we want to create a data object `data_prp` that stores the data from the file `prp_data_reduced.csv`.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(???)\ndata_prp <- read_csv(\"???\")\n```\n:::\n\n\n\n\n:::\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlibrary(tidyverse)\ndata_prp <- read_csv(\"prp_data_reduced.csv\")\n```\n:::\n\n\n:::\n\nIf you need a quick reminder what the dataset was about, have a look at the abstract in @sec-download_data_ch1. We also addressed the changes we made to the dataset there.\n\nAnd remember to have a quick `glimpse()` at your data.\n\n\n\n## Activity 3: Calculating demographics\n\nLet’s start with some simple data-wrangling steps to compute demographics for our original dataset, `data_prp`. First, we want to determine how many participants took part in the study by Pownall et al. (2023) and compute the mean age and the standard deviation of age for the sample.\n\n\n\n### ... for the full sample using `summarise()`\n\nThe `summarise()` function is part of the **\"Wickham Six\"** alongside `group_by()`, `select()`, `filter()`, `mutate()`, and `arrange()`. You used them plenty of times last year.\n\nWithin `summarise()`, we can use the `n()` function, which calculates the number of rows in the dataset. Since each row corresponds to a unique participant, this gives us the total number of participants.\n\nTo calculate the mean age and the standard deviation of age, we need to use the functions `mean()` and `sd()` on the column `Age` respectively.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: There were 2 warnings in `summarise()`.\nThe first warning was:\nℹ In argument: `mean_age = mean(Age)`.\nCaused by warning in `mean.default()`:\n! argument is not numeric or logical: returning NA\nℹ Run `dplyr::last_dplyr_warnings()` to see the 1 remaining warning.\n```\n:::\n\n```{.r .cell-code}\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nR did not give us an error message per se, but the output is not quite as expected either. There are `NA` values in the `mean_age` and `sd_age` columns. Looking at the warning message and at `Age`, can you explain what happened?\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nThe warning message says: `argument is not numeric or logical: returning NA` If we look at the `Age` column more closely, we can see that it's a character data type.\n\n:::\n\n\n\n#### Fixing `Age` {.unnumbered}\n\nMight be wise to look at the unique answers in column `Age` to determine what is wrong. We can do that with the function `distinct()`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nage_distinct <- data_prp %>% \n distinct(Age)\n\nage_distinct\n```\n:::\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Show the unique values of `Age`.\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n:::\n\n::: columns\n\n::: column\n\nOne cell has the string \"years\" added to their number 25, which has converted the entire column into a character column.\n\nWe can easily fix this by extracting only the numbers from the column and converting it into a numeric data type. The `parse_number()` function, which is part of the `tidyverse` package, handles both steps in one go (so there’s no need to load additional packages).\n\nWe will combine this with the `mutate()` function to create a new column called `Age` (containing those numeric values), effectively replacing the old `Age` column (which had the character values).\n\n:::\n\n::: column\n\n{target=\"_blank\"})](images/parse_number.png){width=\"95%\"}\n\n:::\n\n:::\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndata_prp <- data_prp %>% \n mutate(Age = parse_number(Age))\n\ntypeof(data_prp$Age) # fixed\n```\n\n::: {.cell-output .cell-output-stdout}\n```\n[1] \"double\"\n```\n:::\n:::\n\n\n\n\n#### Computing summary stats {.unnumbered}\n\nExcellent. Now that the numbers are in a numeric format, let's try calculating the demographics for the total sample again.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age), # mean age\n sd_age = sd(Age)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nEven though there's no error or warning, the table still shows `NA` values for `mean_age` and `sd_age`. So, what could possibly be wrong now?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Answer\n\nDid you notice that the `Age` column in `age_distinct` contains some missing values (`NA`)? To be honest, it's easier to spot this issue in the actual R output than in the printed HTML page.\n\n:::\n\n\n\n#### Computing summary stats - third attempt {.unnumbered}\n\nTo ensure R ignores missing values during calculations, we need to add the extra argument `na.rm = TRUE` to the `mean()` and `sd()` functions.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_total <- data_prp %>% \n summarise(n = n(), # participant number\n mean_age = mean(Age, na.rm = TRUE), # mean age\n sd_age = sd(Age, na.rm = TRUE)) # standard deviation of age\n\ndemo_total\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nFinally, we’ve got it! 🥳 Third time's the charm!\n\n\n\n### ... per gender using `summarise()` and `group_by()`\n\nNow we want to compute the summary statistics for each gender. The code inside the `summarise()` function remains unchanged; we just need to use the `group_by()` function beforehand to tell R that we want to compute the summary statistics for each group separately. It’s also a good practice to use `ungroup()` afterwards, so you are not taking groupings forward unintentionally.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% # split data up into groups (here Gender)\n summarise(n = n(), # participant number \n mean_age = mean(Age, na.rm = TRUE), # mean age \n sd_age = sd(Age, na.rm = TRUE)) %>% # standard deviation of age\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Adding percentages\n\nSometimes, it may be useful to calculate percentages, such as for the gender split. You can do this by adding a line within the `summarise()` function to perform the calculation. All we need to do is take the number of female, male, and non-binary participants (stored in the `n` column of `demo_by_gender`), divide it by the total number of participants (stored in the `n` column of `demo_total`), and multiply by 100. Let's add `percentage` to the `summarise()` function of `demo_by_gender`. Make sure that the code for `percentages` is placed after the value for `n` has been computed.\n\nAccessing the value of `n` for the different gender categories is straightforward because we can refer back to it directly. However, since the total number of participants is stored in a different data object, we need to use a base R function to access it – specifically the `$` operator. To do this, you simply type the name of the data object (in this case, `demo_total`), followed by the `$` symbol (with no spaces), and then the name of the column you want to retrieve (in this case, `n`). The general pattern is `data$column`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n # n from the line above divided by n from demo_total *100\n percentage = n/demo_total$n *100, \n mean_age = mean(Age, na.rm = TRUE), \n sd_age = sd(Age, na.rm = TRUE)) %>% \n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n::: {.callout-tip collapse=\"true\"}\n\n## Tip for decimal places - use `round()`\n\nNot super important, because you could round the values by yourself when writing up your reports, but if you wanted to tidy up the decimal places in the output, you can do that using the `round()` function. You would need to \"wrap\" it around your computations and specify how many decimal places you want to display (for example `mean(Age)` would turn into `round(mean(Age), 1)`). It may look odd for `percentage`, just make sure the number that specifies the decimal places is placed **within** the round function. The default value is 0 (meaning no decimal spaces).\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndemo_by_gender <- data_prp %>% \n group_by(Gender) %>% \n summarise(n = n(), \n percentage = round(n/demo_total$n *100, 2), # percentage with 2 decimal places\n mean_age = round(mean(Age, na.rm = TRUE), 1), # mean Age with 1 decimal place\n sd_age = round(sd(Age, na.rm = TRUE), 3)) %>% # sd Age with 3 decimal places\n ungroup()\n\ndemo_by_gender\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 4: Questionable Research Practices (QRPs) {#sec-ch2_act4}\n\n#### The main goal is to compute the mean QRP score per participant for time point 1. {.unnumbered}\n\nAt the moment, the data is in wide format. The table below shows data from the first 3 participants:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nhead(data_prp, n = 3)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\nLooking at the QRP data at time point 1, you determine that\n\n* individual item columns are , and\n* according to the codebook, there are reverse-coded items in this questionnaire.\n\nAccording to the codebook and the data table above, we just have to **compute the average score for QRP items to **, since items to are distractor items. Seems quite straightforward.\n\nHowever, as you can see in the table above, each item is in a separate column, meaning the data is in **wide format**. It would be much easier to calculate the mean scores if the items were arranged in **long format**.\n\n\nLet’s tackle this problem step by step. It’s best to create a separate data object for this. If we tried to compute it within `data_prp`, it could quickly become messy.\n\n\n* **Step 1**: Select the relevant columns `Code`, and `QRPs_1_Time1` to `QRPs_1_Time1` and store them in an object called `qrp_t1`\n* **Step 2**: Pivot the data from wide format to long format using `pivot_longer()` so we can calculate the average score more easily (in step 3)\n* **Step 3**: Calculate the average QRP score (`QRPs_Acceptance_Time1_mean`) per participant using `group_by()` and `summarise()`\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_t1 <- data_prp %>% \n #Step 1\n select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%\n # Step 2\n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(Code) %>% # grouping by participant id\n summarise(QRPs_Acceptance_Time1_mean = mean(Scores)) %>% # calculating the average Score\n ungroup() # just make it a habit\n```\n:::\n\n\n::: {.callout-caution icon=\"false\" collapse=\"true\"}\n\n## Explain the individual functions\n\n::: panel-tabset\n\n## `select ()`\n\nThe select function allows to include or exclude certain variables (columns). Here we want to focus on the participant ID column (i.e., `Code`) and the QRP items at time point 1. We can either list them all individually, i.e., Code, QRPs_1_Time1, QRPs_2_Time1, QRPs_3_Time1, and so forth (you get the gist), but that would take forever to type.\n\nA shortcut is to use the colon operator `:`. It allows us to select all columns that fall within the range of `first_column_name` to `last_column_name`. We can apply this here since the QRP items (1 to 11) are sequentially listed in `data_prp`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step1 <- data_prp %>% \n select(Code, QRPs_1_Time1:QRPs_11_Time1)\n\n# show first 5 rows of qrp_step1\nhead(qrp_step1, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nHow many rows/observations and columns/variables do we have in `qrp_step1`?\n\n* rows/observations: \n* columns/variables: \n\n## `pivot_longer()`\n\nAs you can see, the table we got from Step 1 is in wide format. To get it into wide format, we need to define:\n\n* the columns that need to be reshuffled from wide into long format (`col` argument). Here we selected \"everything except the `Code` column\", as indicated by `-Code` \\[minus `Code`\\]. However, `QRPs_1_Time1:QRPs_11_Time1` would also work and give you the exact same result.\n* the `names_to` argument. R is creating a new column in which all the column names from the columns you selected in `col` will be stored in. Here we are naming this column \"Items\" but you could pick something equally sensible if you like.\n* the `values_to` argument. R creates this second column to store all responses the participants gave to the individual questions, i.e., all the numbers in this case. We named it \"Scores\" here, but you could have called it something different, like \"Responses\"\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nqrp_step2 <- qrp_step1 %>% \n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\")\n\n# show first 15 rows of qrp_step2\nhead(qrp_step2, n = 15)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\nNow, have a look at `qrp_step2`. In total, we now have rows/observations, per participant, and columns/variables.\n\n## `group_by()` and `summarise()`\n\nThis follows exactly the same sequence we used when calculating descriptive statistics by gender. The only difference is that we are now grouping the data by the participant's `Code` instead of `Gender`.\n\n`summarise()` works exactly the same way: `summarise(new_column_name = function_to_calculate_something(column_name_of_numeric_values))`\n\nThe `function_to_calculate_something` can be `mean()`, `sd()` or `sum()` for mean scores, standard deviations, or summed-up scores respectively. You could also use `min()` or `max()` if you wanted to determine the lowest or the highest score for each participant.\n\n:::\n\n:::\n\n::: callout-tip\n\nYou could **rename the columns whilst selecting** them. The pattern would be `select(new_name = old_name)`. For example, if we wanted to select variable `Code` and rename it as `Participant_ID`, we could do that.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nrenaming_col <- data_prp %>% \n select(Participant_ID = Code)\n\nhead(renaming_col, n = 5)\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n## Activity 5: Knitting\n\nOnce you've completed your R Markdown file, the final step is to \"knit\" it, which converts the `.Rmd` file into a HTML file. Knitting combines your code, text, and output (like tables and plots) into a single cohesive document. This is a really good way to check your code is working.\n\nTo knit the file, **click the Knit button** at the top of your RStudio window. The document will be generated and, depending on your setting, automatically opened in the viewer in the `Output pane` or an external browser window.\n\nIf any errors occur during knitting, RStudio will show you an error message with details to help you troubleshoot.\n\nIf you want to **intentionally keep any errors** we tackled today to keep a reference on how you solved them, you could add `error=TRUE` or `eval=FALSE` to the code chunk that isn't running.\n\n\n\n## Activity 6: Export a data object as a csv\n\nTo avoid having to repeat the same steps in the next chapter, it's a good idea to save the data objects you've created today as csv files. You can do this by using the `write_csv()` function from the `readr` package. The csv files will appear in your project folder.\n\nThe basic syntax is:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_object, \"filename.csv\")\n```\n:::\n\n\nNow, let's export the objects `data_prp` and `qrp_t1`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(data_prp, \"data_prp_for_ch3.csv\")\n```\n:::\n\n\nHere we named the file `data_prp_for_ch3.csv`, so we wouldn't override the original data csv file `prp_data_reduced.csv`. However, feel free to choose a name that makes sense to you.\n\n::: {.callout-note icon=\"false\"}\n\n## Your Turn\n\nExport `qrp_t1`.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nwrite_csv(qrp_t1, \"qrp_t1.csv\")\n```\n:::\n\n\n:::\n\n:::\n\nCheck that your csv files have appeared in your project folder, and you're all set!\n\n**That’s it for Chapter 2: Individual Walkthrough.**\n\n## [Pair-coding]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n\n::: {.cell layout-align=\"center\"}\n\n:::\n\n\n\nWe will continue working with the data from Binfet et al. (2021), focusing on the randomised controlled trial of therapy dog interventions. Today, our goal is to **calculate an average `Flourishing` score for each participant** at time point 1 (pre-intervention) using the raw data file `dog_data_raw`. Currently, the data looks like this:\n\n\n\n::: {.cell layout-align=\"center\"}\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\n\n### Task 1: Open the R project you created last week {.unnumbered}\n\nIf you haven’t created an R project for the lab yet, please do so now. If you already have one set up, go ahead and open it.\n\n\n### Task 2: Open your `.Rmd` file from last week {.unnumbered}\n\nSince we haven’t used it much yet, feel free to continue using the `.Rmd` file you created last week in Task 2.\n\n\n### Task 3: Load in the library and read in the data {.unnumbered}\n\nThe data should be in your project folder. If you didn’t download it last week, or if you’d like a fresh copy, you can download the data again here: [data_pair_ch1](data/data_pair_ch1.zip \"download\").\n\nWe will be using the `tidyverse` package today, and the data file we need to read in is `dog_data_raw.csv`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(???)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"???\")\n```\n:::\n\n\n:::\n\n\n### Task 4: Calculating the mean for `Flourishing_pre` {.unnumbered}\n\n\n* **Step 1**: Select all relevant columns from `dog_data_raw`, including participant ID and all items from the `Flourishing` questionnaire completed before the intervention. Store this data in an object called `data_flourishing`.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nLook at the codebook. Try to determine:\n\n* The variable name of the column where the participant ID is stored.\n* The items related to the Flourishing scale at the pre-intervention stage.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nFrom the codebook, we know that:\n\n* The participant ID column is called `RID`.\n* The Flourishing items at the pre-intervention stage start with `F1_`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\ndata_flourishing <- ??? %>% \n select(???, F1_???:F1_???)\n```\n:::\n\n\n:::\n\n:::\n\n\n* **Step 2**: Pivot the data from wide format to long format so we can calculate the average score more easily (in step 3).\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nWhich pivot function should you use? We have `pivot_wider()` and `pivot_longer()` to choose from.\n\nWe also need 3 arguments in that function:\n\n* The columns you want to select (e.g., all the Flourishing items),\n* The name of the column where the current column headings will be stored (e.g., \"Questionnaire\"),\n* The name of the column that should store all the values (e.g., \"Responses\").\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nWe need `pivot_longer()`. You already encountered `pivot_longer()` in first year (or in the individual walkthrough if you have already completed this Chapter). The 3 arguments was also a give-away; `pivot_wider()` only requires 2 arguments.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n pivot_longer(cols = ???, names_to = \"???\", values_to = \"???\")\n```\n:::\n\n\n:::\n\n:::\n\n* **Step 3**: Calculate the average Flourishing score per participant and name this column `Flourishing_pre` to match the table above.\n\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## Hint\n\nBefore summarising the mean, you may need to group the data.\n\n::: {.callout-note collapse=\"true\" icon=\"false\"}\n\n## More concrete hint\n\nTo compute an average score **per participant**, we would need to group by participant ID first.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n group_by(???) %>% \n summarise(Flourishing_pre = mean(???)) %>% \n ungroup()\n```\n:::\n\n:::\n\n:::\n\n\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# loading tidyverse into the library\nlibrary(tidyverse)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"dog_data_raw.csv\")\n\n# Task 4: Tidying \ndata_flourishing <- dog_data_raw %>% \n # Step 1\n select(RID, F1_1:F1_8) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Questionnaire\", values_to = \"Responses\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_pre = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\n:::\n\n\n\n## [Test your knowledge and challenge yourself]{style=\"color: #F39C12; text-transform: uppercase;\"} {.unnumbered}\n\n### Knowledge check {.unnumbered}\n\n#### Question 1 {.unnumbered}\n\nWhich function of the Wickham Six would you use to include or exclude certain variables (columns)? \n\n\n#### Question 2 {.unnumbered}\n\nWhich function of the Wickham Six would you use to create new columns or modify existing columns in a dataframe? \n\n\n#### Question 3 {.unnumbered}\n\n\nWhich function of the Wickham Six would you use to organise data into groups based on one or more columns? \n\n\n\n#### Question 4 {.unnumbered}\n\nWhich function of the Wickham Six would you use to sort the rows of a dataframe based on the values in one or more columns? \n\n\n\n#### Question 5 {.unnumbered}\n\nWhich function of the Wickham Six would NOT modify the original dataframe? \n\n\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain these answers\n\n| Function | Description |\n|:-------------|:------------------------------------------------------|\n| `select()` | Include or exclude certain variables/columns |\n| `filter()` | Include or exclude certain observations/rows |\n| `mutate()` | Creates new columns or modifies existing ones |\n| `arrange()` | Changes the order of the rows |\n| `group_by()` | Split data into groups based on one or more variables |\n| `summarise()`| Creates a new dataframe returning one row for each combination of grouping variables |\n\n\nTechnically, the first five functions operate on the existing data object, making adjustments like sorting the data (e.g., with `arrange()`), reducing the number of rows (e.g., with `filter()`), reducing the number of columns (e.g., with `select()`), or adding new columns (e.g., with `mutate()`). In contrast, `summarise()` fundamentally alters the structure of the original dataframe by generating a completely new dataframe that contains only summary statistics, rather than retaining the original rows and columns.\n\n:::\n\n\n\n### Error mode {.unnumbered}\n\nSome of the code chunks contain mistakes and result in errors, while others do not produce the expected results. Your task is to identify any issues, explain why they occurred, and, if possible, fix them.\n\nWe will use a few built-in datasets, such as `billboard` and `starwars`, to help you replicate the errors in your own R environment. You can view the data either by typing the dataset name directly into your console or by storing the data as a separate object in your `Global Environment`.\n\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nbillboard\n\nstarwars_data = starwars\n```\n:::\n\n\n\n\n#### Question 6 {.unnumbered}\n\nCurrently, the weekly song rankings for Billboard Top 100 in 2000 are in wide format, with each week in a separate column. The following code is supposed to transpose the wide-format `billboard` data into long format:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(names_to = \"weeks\", values_to = \"rank\")\n```\n\n::: {.cell-output .cell-output-error}\n```\nError in `pivot_longer()`:\n! `cols` must select at least one column.\n```\n:::\n:::\n\n\nWhat does this error message mean and how do you fix it?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe error message indicates that the `cols` argument is missing in the function. This means the function doesn’t know which columns to transpose from wide format to long format.\n\nFIX: Add `cols = wk1:wk76` to the function to select columns from wk1 to wk76. Alternatively, `cols = starts_with(\"wk\")` would also work since all columns start with the letter combination \"wk\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nlong_data <- billboard %>% \n pivot_longer(cols = wk1:wk76, names_to = \"weeks\", values_to = \"rank\")\n# OR\nlong_data <- billboard %>% \n pivot_longer(cols = starts_with(\"wk\"), names_to = \"weeks\", values_to = \"rank\")\n```\n:::\n\n\n:::\n\n\n\n#### Question 7 {.unnumbered}\n\nThe following code is intended to calculate the mean height of all the characters in the built-in `starwars` dataset, grouped by their gender. \n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = height)\n```\n\n::: {.cell-output .cell-output-stderr}\n```\nWarning: Returning more (or less) than 1 row per `summarise()` group was deprecated in\ndplyr 1.1.0.\nℹ Please use `reframe()` instead.\nℹ When switching from `summarise()` to `reframe()`, remember that `reframe()`\n always returns an ungrouped data frame and adjust accordingly.\n```\n:::\n:::\n\n\nThe code runs, but it's giving us some weird warning and the output is also not as expected. What steps should we take to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nThe aggregation function `mean()` is missing from within `summarise()`. Without it, the function does not perform any aggregation and returns *all* rows with only the columns for gender and height.\n\nFIX: Wrap the `mean()` function around the variable you want to aggregate, here `height`.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height))\n```\n:::\n\n\n:::\n\n\n\n#### Question 8 {.unnumbered}\n\nFollowing up on Question 7, we now have `summary_data` that looks approximately correct - it has the expected rows and column numbers, however, the cell values are \"weird\".\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n\nCan you explain what is happening here? And how can we modify the code to fix this?\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Explain the solution\n\nLook at the original `starwars` data. You will notice that some of the characters with feminine and masculine gender entries have missing height values. However, all four characters without a specified gender have provided their height.\n\nFIX: We need to add `na.rm = TRUE` to the `mean()` function to ensure that R ignores missing values before aggregating the data.\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\nsummary_data <- starwars %>%\n group_by(gender) %>%\n summarise(mean_height = mean(height, na.rm = TRUE))\n\nsummary_data\n```\n\n::: {.cell-output-display}\n
\n:::\n:::\n\n\n:::\n\n\n\n### Challenge yourself {.unnumbered}\n\nIf you want to **challenge yourself** and further apply the skills from Chapter 2, you can wrangle the data from `dog_data_raw` for additional questionnaires from either the pre- and/or post-intervention stages:\n\n* Calculate the mean score for `flourishing_post` for each participant.\n* Calculate the mean score for the `PANAS` (Positive and/or Negative Affect) per participant\n* Calculate the mean score for happiness (`SHS`) per participant\n\nThe 3 steps are equivalent for those questionnaires - select, pivot, group_by and summarise; you just have to \"replace\" the questionnaire items involved.\n\n::: {.callout-caution collapse=\"true\" icon=\"false\"}\n\n## Solution for **Challenge yourself**\n\nFlourishing post-intervention\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n## flourishing_post\nflourishing_post <- dog_data_raw %>% \n # Step 1\n select(RID, starts_with(\"F2\")) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Names\", values_to = \"Response\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_post = mean(Response)) %>% \n ungroup()\n```\n:::\n\n\nThe PANAS could be solved more concisely with the skills we learn in @sec-wrangling2, but for now, you would have solved it this way:\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# PANAS - positive affect pre\nPANAS_PA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_3, PN1_5, PN1_7, PN1_8, PN1_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - positive affect post\nPANAS_PA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_3, PN2_5, PN2_7, PN2_8, PN2_10) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_PA_post = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect pre\nPANAS_NA_pre <- dog_data_raw %>% \n # Step 1\n select(RID, PN1_1, PN1_2, PN1_4, PN1_6, PN1_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_pre = mean(Scores)) %>% \n ungroup()\n\n# PANAS - negative affect post\nPANAS_NA_post <- dog_data_raw %>% \n # Step 1\n select(RID, PN2_1, PN2_2, PN2_4, PN2_6, PN2_9) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(PANAS_NA_post = mean(Scores)) %>% \n ungroup()\n```\n:::\n\n\nHappiness scale\n\n\n::: {.cell layout-align=\"center\"}\n\n```{.r .cell-code}\n# happiness_pre\nhappiness_pre <- dog_data_raw %>% \n # Step 1\n select(RID, HA1_1, HA1_2, HA1_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_pre = mean(Score)) %>% \n ungroup()\n\n#happiness_post\nhappiness_post <- dog_data_raw %>% \n # Step 1\n select(RID, HA2_1, HA2_2, HA2_3) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Item\", values_to = \"Score\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(SHS_post = mean(Score)) %>% \n ungroup()\n```\n:::\n\n\n:::\n",
+ "supporting": [],
"filters": [
"rmarkdown/pagebreak.lua"
],
diff --git a/docs/02-wrangling.html b/docs/02-wrangling.html
index 3984ccb..765d7df 100644
--- a/docs/02-wrangling.html
+++ b/docs/02-wrangling.html
@@ -740,6 +740,485 @@
2.4 Activity 4: Questionable Research Practices (QRPs)
The main goal is to compute the mean QRP score per participant for time point 1.
+
At the moment, the data is in wide format. The table below shows data from the first 3 participants:
+
+
head(data_prp, n =3)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Code
+
Gender
+
Age
+
Ethnicity
+
Secondyeargrade
+
Opptional_mod
+
Opptional_mod_1_TEXT
+
Research_exp
+
Research_exp_1_TEXT
+
Plan_prereg
+
SATS28_1_Affect_Time1
+
SATS28_2_Affect_Time1
+
SATS28_3_Affect_Time1
+
SATS28_4_Affect_Time1
+
SATS28_5_Affect_Time1
+
SATS28_6_Affect_Time1
+
SATS28_7_CognitiveCompetence_Time1
+
SATS28_8_CognitiveCompetence_Time1
+
SATS28_9_CognitiveCompetence_Time1
+
SATS28_10_CognitiveCompetence_Time1
+
SATS28_11_CognitiveCompetence_Time1
+
SATS28_12_CognitiveCompetence_Time1
+
SATS28_13_Value_Time1
+
SATS28_14_Value_Time1
+
SATS28_15_Value_Time1
+
SATS28_16_Value_Time1
+
SATS28_17_Value_Time1
+
SATS28_18_Value_Time1
+
SATS28_19_Value_Time1
+
SATS28_20_Value_Time1
+
SATS28_21_Value_Time1
+
SATS28_22_Difficulty_Time1
+
SATS28_23_Difficulty_Time1
+
SATS28_24_Difficulty_Time1
+
SATS28_25_Difficulty_Time1
+
SATS28_26_Difficulty_Time1
+
SATS28_27_Difficulty_Time1
+
SATS28_28_Difficulty_Time1
+
QRPs_1_Time1
+
QRPs_2_Time1
+
QRPs_3_Time1
+
QRPs_4_Time1
+
QRPs_5_Time1
+
QRPs_6_Time1
+
QRPs_7_Time1
+
QRPs_8_Time1
+
QRPs_9_Time1
+
QRPs_10_Time1
+
QRPs_11_Time1
+
QRPs_12NotQRP_Time1
+
QRPs_13NotQRP_Time1
+
QRPs_14NotQRP_Time1
+
QRPs_15NotQRP_Time1
+
Understanding_OS_1_Time1
+
Understanding_OS_2_Time1
+
Understanding_OS_3_Time1
+
Understanding_OS_4_Time1
+
Understanding_OS_5_Time1
+
Understanding_OS_6_Time1
+
Understanding_OS_7_Time1
+
Understanding_OS_8_Time1
+
Understanding_OS_9_Time1
+
Understanding_OS_10_Time1
+
Understanding_OS_11_Time1
+
Understanding_OS_12_Time1
+
Pre_reg_group
+
Other_OS_behav_2
+
Other_OS_behav_4
+
Other_OS_behav_5
+
Closely_follow
+
SATS28_Affect_Time2_mean
+
SATS28_CognitiveCompetence_Time2_mean
+
SATS28_Value_Time2_mean
+
SATS28_Difficulty_Time2_mean
+
QRPs_Acceptance_Time2_mean
+
Time2_Understanding_OS
+
Supervisor_1
+
Supervisor_2
+
Supervisor_3
+
Supervisor_4
+
Supervisor_5
+
Supervisor_6
+
Supervisor_7
+
Supervisor_8
+
Supervisor_9
+
Supervisor_10
+
Supervisor_11
+
Supervisor_12
+
Supervisor_13
+
Supervisor_14
+
Supervisor_15_R
+
+
+
+
Tr10
+
2
+
22
+
White European
+
2
+
1
+
Research methods in first year
+
2
+
NA
+
1
+
4
+
5
+
3
+
4
+
5
+
5
+
4
+
2
+
2
+
6
+
4
+
3
+
1
+
7
+
7
+
2
+
1
+
3
+
3
+
2
+
2
+
3
+
5
+
2
+
6
+
4
+
4
+
1
+
7
+
7
+
5
+
7
+
3
+
4
+
5
+
7
+
6
+
7
+
7
+
2
+
1
+
1
+
2
+
2
+
2
+
2
+
6
+
Entirely confident
+
Entirely confident
+
6
+
6
+
Entirely confident
+
Entirely confident
+
Entirely confident
+
Entirely confident
+
1
+
1
+
1
+
NA
+
2
+
3.500000
+
4.166667
+
3.000000
+
2.857143
+
5.636364
+
5.583333
+
5
+
5
+
6
+
6
+
5
+
5
+
1
+
5
+
6
+
5
+
NA
+
4
+
4
+
5
+
1
+
+
+
Bi07
+
2
+
20
+
White British
+
3
+
2
+
NA
+
2
+
NA
+
3
+
5
+
6
+
2
+
5
+
5
+
6
+
2
+
2
+
2
+
7
+
3
+
5
+
1
+
7
+
7
+
1
+
1
+
6
+
3
+
1
+
1
+
2
+
6
+
2
+
7
+
2
+
5
+
7
+
7
+
7
+
2
+
7
+
3
+
7
+
7
+
7
+
7
+
6
+
7
+
2
+
1
+
4
+
4
+
2
+
Not at all confident
+
Not at all confident
+
Not at all confident
+
6
+
Entirely confident
+
Not at all confident
+
3
+
6
+
6
+
2
+
2
+
1
+
NA
+
NA
+
NA
+
2
+
3.166667
+
4.666667
+
6.222222
+
2.857143
+
5.454546
+
3.333333
+
7
+
6
+
7
+
7
+
7
+
7
+
1
+
5
+
7
+
7
+
7
+
5
+
2
+
7
+
1
+
+
+
SK03
+
2
+
22
+
White British
+
1
+
2
+
NA
+
2
+
NA
+
1
+
5
+
3
+
5
+
2
+
5
+
2
+
2
+
2
+
2
+
6
+
5
+
3
+
2
+
6
+
6
+
3
+
3
+
5
+
3
+
4
+
3
+
5
+
5
+
2
+
5
+
2
+
5
+
5
+
7
+
7
+
6
+
6
+
7
+
6
+
7
+
7
+
7
+
5
+
7
+
1
+
1
+
3
+
2
+
6
+
2
+
3
+
6
+
6
+
5
+
2
+
5
+
5
+
5
+
4
+
5
+
1
+
NA
+
NA
+
NA
+
2
+
4.833333
+
6.166667
+
6.000000
+
4.000000
+
6.272727
+
5.416667
+
7
+
7
+
7
+
7
+
7
+
7
+
1
+
7
+
7
+
7
+
7
+
7
+
5
+
7
+
1
+
+
+
+
+
+
+
+
Looking at the QRP data at time point 1, you determine that
individual item columns are reverse-coded items in this questionnaire.
-
So, we just have to compute an average score for items 1 to 11 as items 12 to 15 are distractor items. Seems quite straightforward.
-
The downside is that individual items are each in a separate column, i.e., in wide format, and everything would be easier if the items were arranged in long format.
-
Let’s tackle this problem in steps. Best would be to create a separate data object for that. If we wanted to compute this within data_prp, it would turn into a nightmare.
+
According to the codebook and the data table above, we just have to compute the average score for QRP items to , since items to are distractor items. Seems quite straightforward.
+
However, as you can see in the table above, each item is in a separate column, meaning the data is in wide format. It would be much easier to calculate the mean scores if the items were arranged in long format.
+
Let’s tackle this problem step by step. It’s best to create a separate data object for this. If we tried to compute it within data_prp, it could quickly become messy.
Step 1: Select the relevant columns Code, and QRPs_1_Time1 to QRPs_1_Time1 and store them in an object called qrp_t1
@@ -763,15 +1242,15 @@
-
qrp_t1 <- data_prp %>%
-#Step 1
-select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%
-# Step 2
-pivot_longer(cols =-Code, names_to ="Items", values_to ="Scores") %>%
-# Step 3
-group_by(Code) %>%# grouping py participant id
-summarise(QRPs_Acceptance_Time1_mean =mean(Scores)) %>%# calculating the average Score
-ungroup() # just make it a habit
+
qrp_t1 <- data_prp %>%
+#Step 1
+select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%
+# Step 2
+pivot_longer(cols =-Code, names_to ="Items", values_to ="Scores") %>%
+# Step 3
+group_by(Code) %>%# grouping by participant id
+summarise(QRPs_Acceptance_Time1_mean =mean(Scores)) %>%# calculating the average Score
+ungroup() # just make it a habit
@@ -796,11 +1275,11 @@
The select function allows to include or exclude certain variables (columns). Here we want to focus on the participant ID column (i.e., Code) and the QRP items at time point 1. We can either list them all individually, i.e., Code, QRPs_1_Time1, QRPs_2_Time1, QRPs_3_Time1, and so forth (you get the gist), but that would take forever to type.
A shortcut is to use the colon operator :. It allows us to select all columns that fall within the range of first_column_name to last_column_name. We can apply this here since the QRP items (1 to 11) are sequentially listed in data_prp.
-
qrp_step1 <- data_prp %>%
-select(Code, QRPs_1_Time1:QRPs_11_Time1)
-
-# show first 5 rows of qrp_step1
-head(qrp_step1, n =5)
+
qrp_step1 <- data_prp %>%
+select(Code, QRPs_1_Time1:QRPs_11_Time1)
+
+# show first 5 rows of qrp_step1
+head(qrp_step1, n =5)
@@ -924,11 +1403,11 @@
the values_to argument. R creates this second column to store all responses the participants gave to the individual questions, i.e., all the numbers in this case. We named it “Scores” here, but you could have called it something different, like “Responses”
-
qrp_step2 <- qrp_step1 %>%
-pivot_longer(cols =-Code, names_to ="Items", values_to ="Scores")
-
-# show first 15 rows of qrp_step2
-head(qrp_step2, n =15)
+
qrp_step2 <- qrp_step1 %>%
+pivot_longer(cols =-Code, names_to ="Items", values_to ="Scores")
+
+# show first 15 rows of qrp_step2
+head(qrp_step2, n =15)
@@ -1042,10 +1521,10 @@
You could rename the columns whilst selecting them. The pattern would be select(new_name = old_name). For example, if we wanted to select variable Code and rename it as Participant_ID, we could do that.
To avoid having to repeat the same steps in the next chapter, it’s a good idea to save the data objects you’ve created today as csv files. You can do this by using the write_csv() function from the readr package. The csv files will appear in your project folder.
The basic syntax is:
-
write_csv(data_object, "filename.csv")
+
write_csv(data_object, "filename.csv")
Now, let’s export the objects data_prp and qrp_t1.
-
write_csv(data_prp, "data_prp_for_ch3.csv")
+
write_csv(data_prp, "data_prp_for_ch3.csv")
Here we named the file data_prp_for_ch3.csv, so we wouldn’t override the original data csv file prp_data_reduced.csv. However, feel free to choose a name that makes sense to you.
@@ -1117,7 +1596,7 @@
-
write_csv(qrp_t1, "qrp_t1.csv")
+
write_csv(qrp_t1, "qrp_t1.csv")
@@ -1259,11 +1738,11 @@
-
# loading tidyverse into the library
-library(???)
-
-# reading in `dog_data_raw.csv`
-dog_data_raw <-read_csv("???")
+
# loading tidyverse into the library
+library(???)
+
+# reading in `dog_data_raw.csv`
+dog_data_raw <-read_csv("???")
@@ -1272,7 +1751,7 @@
-Step 1: Select all relevant columns, including participant ID and all 8 items from the Flourishing questionnaire completed before the intervention. Store this data in an object called data_flourishing.
+Step 1: Select all relevant columns from dog_data_raw, including participant ID and all items from the Flourishing questionnaire completed before the intervention. Store this data in an object called data_flourishing.
@@ -1308,6 +1787,10 @@
The participant ID column is called RID.
The Flourishing items at the pre-intervention stage start with F1_.
We need pivot_longer(). You already encountered pivot_longer() in first year (or in the individual walkthrough if you have already completed this Chapter). The 3 arguments was also a give-away; pivot_wider() only requires 2 arguments.
Some of the code chunks contain mistakes and result in errors, while others do not produce the expected results. Your task is to identify any issues, explain why they occurred, and, if possible, fix them.
We will use a few built-in datasets, such as billboard and starwars, to help you replicate the errors in your own R environment. You can view the data either by typing the dataset name directly into your console or by storing the data as a separate object in your Global Environment.
-
billboard
-
-starwars_data = starwars
+
billboard
+
+starwars_data = starwars
Question 6
Currently, the weekly song rankings for Billboard Top 100 in 2000 are in wide format, with each week in a separate column. The following code is supposed to transpose the wide-format billboard data into long format:
-
long_data <- billboard %>%
-pivot_longer(names_to ="weeks", values_to ="rank")
+
long_data <- billboard %>%
+pivot_longer(names_to ="weeks", values_to ="rank")
Error in `pivot_longer()`:
! `cols` must select at least one column.
@@ -1559,11 +2044,11 @@
The error message indicates that the cols argument is missing in the function. This means the function doesn’t know which columns to transpose from wide format to long format.
FIX: Add cols = wk1:wk76 to the function to select columns from wk1 to wk76. Alternatively, cols = starts_with("wk") would also work since all columns start with the letter combination “wk”.
-
long_data <- billboard %>%
-pivot_longer(cols = wk1:wk76, names_to ="weeks", values_to ="rank")
-# OR
-long_data <- billboard %>%
-pivot_longer(cols =starts_with("wk"), names_to ="weeks", values_to ="rank")
+
long_data <- billboard %>%
+pivot_longer(cols = wk1:wk76, names_to ="weeks", values_to ="rank")
+# OR
+long_data <- billboard %>%
+pivot_longer(cols =starts_with("wk"), names_to ="weeks", values_to ="rank")
@@ -1571,9 +2056,9 @@
Question 7
The following code is intended to calculate the mean height of all the characters in the built-in starwars dataset, grouped by their gender.
Warning: Returning more (or less) than 1 row per `summarise()` group was deprecated in
dplyr 1.1.0.
@@ -1598,9 +2083,9 @@
The aggregation function mean() is missing from within summarise(). Without it, the function does not perform any aggregation and returns all rows with only the columns for gender and height.
FIX: Wrap the mean() function around the variable you want to aggregate, here height.
Following up on Question 7, we now have summary_data that looks approximately correct - it has the expected rows and column numbers, however, the cell values are “weird”.
-
summary_data
+
summary_data
@@ -1650,11 +2135,11 @@
Look at the original starwars data. You will notice that some of the characters with feminine and masculine gender entries have missing height values. However, all four characters without a specified gender have provided their height.
FIX: We need to add na.rm = TRUE to the mean() function to ensure that R ignores missing values before aggregating the data.
diff --git a/docs/search.json b/docs/search.json
index e1b0746..1048a7c 100644
--- a/docs/search.json
+++ b/docs/search.json
@@ -116,7 +116,7 @@
"href": "02-wrangling.html#sec-ch2_act4",
"title": "2 Data wrangling I",
"section": "\n2.4 Activity 4: Questionable Research Practices (QRPs)",
- "text": "2.4 Activity 4: Questionable Research Practices (QRPs)\nThe main goal is to compute the mean QRP score per participant for time point 1.\nLooking at the QRP data at time point 1, you determine that\n\nindividual item columns are \nnumeric\ncharacter, and\naccording to the codebook, there are \nno\nsome reverse-coded items in this questionnaire.\n\nSo, we just have to compute an average score for items 1 to 11 as items 12 to 15 are distractor items. Seems quite straightforward.\nThe downside is that individual items are each in a separate column, i.e., in wide format, and everything would be easier if the items were arranged in long format.\nLet’s tackle this problem in steps. Best would be to create a separate data object for that. If we wanted to compute this within data_prp, it would turn into a nightmare.\n\n\nStep 1: Select the relevant columns Code, and QRPs_1_Time1 to QRPs_1_Time1 and store them in an object called qrp_t1\n\n\nStep 2: Pivot the data from wide format to long format using pivot_longer() so we can calculate the average score more easily (in step 3)\n\nStep 3: Calculate the average QRP score (QRPs_Acceptance_Time1_mean) per participant using group_by() and summarise()\n\n\n\nqrp_t1 <- data_prp %>% \n #Step 1\n select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%\n # Step 2\n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(Code) %>% # grouping py participant id\n summarise(QRPs_Acceptance_Time1_mean = mean(Scores)) %>% # calculating the average Score\n ungroup() # just make it a habit\n\n\n\n\n\n\n\nExplain the individual functions\n\n\n\n\n\n\n\nselect ()\npivot_longer()\ngroup_by() and summarise()\n\n\n\nThe select function allows to include or exclude certain variables (columns). Here we want to focus on the participant ID column (i.e., Code) and the QRP items at time point 1. We can either list them all individually, i.e., Code, QRPs_1_Time1, QRPs_2_Time1, QRPs_3_Time1, and so forth (you get the gist), but that would take forever to type.\nA shortcut is to use the colon operator :. It allows us to select all columns that fall within the range of first_column_name to last_column_name. We can apply this here since the QRP items (1 to 11) are sequentially listed in data_prp.\n\nqrp_step1 <- data_prp %>% \n select(Code, QRPs_1_Time1:QRPs_11_Time1)\n\n# show first 5 rows of qrp_step1\nhead(qrp_step1, n = 5)\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCode\nQRPs_1_Time1\nQRPs_2_Time1\nQRPs_3_Time1\nQRPs_4_Time1\nQRPs_5_Time1\nQRPs_6_Time1\nQRPs_7_Time1\nQRPs_8_Time1\nQRPs_9_Time1\nQRPs_10_Time1\nQRPs_11_Time1\n\n\n\nTr10\n7\n7\n5\n7\n3\n4\n5\n7\n6\n7\n7\n\n\nBi07\n7\n7\n2\n7\n3\n7\n7\n7\n7\n6\n7\n\n\nSK03\n7\n7\n6\n6\n7\n6\n7\n7\n7\n5\n7\n\n\nSM95\n7\n7\n2\n6\n7\n5\n7\n7\n4\n2\n4\n\n\nSt01\n7\n7\n6\n7\n2\n7\n7\n7\n7\n5\n7\n\n\n\n\n\n\nHow many rows/observations and columns/variables do we have in qrp_step1?\n\nrows/observations: \n\ncolumns/variables: \n\n\n\n\nAs you can see, the table we got from Step 1 is in wide format. To get it into wide format, we need to define:\n\nthe columns that need to be reshuffled from wide into long format (col argument). Here we selected “everything except the Code column”, as indicated by -Code [minus Code]. However, QRPs_1_Time1:QRPs_11_Time1 would also work and give you the exact same result.\nthe names_to argument. R is creating a new column in which all the column names from the columns you selected in col will be stored in. Here we are naming this column “Items” but you could pick something equally sensible if you like.\nthe values_to argument. R creates this second column to store all responses the participants gave to the individual questions, i.e., all the numbers in this case. We named it “Scores” here, but you could have called it something different, like “Responses”\n\n\nqrp_step2 <- qrp_step1 %>% \n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\")\n\n# show first 15 rows of qrp_step2\nhead(qrp_step2, n = 15)\n\n\n\n\nCode\nItems\nScores\n\n\n\nTr10\nQRPs_1_Time1\n7\n\n\nTr10\nQRPs_2_Time1\n7\n\n\nTr10\nQRPs_3_Time1\n5\n\n\nTr10\nQRPs_4_Time1\n7\n\n\nTr10\nQRPs_5_Time1\n3\n\n\nTr10\nQRPs_6_Time1\n4\n\n\nTr10\nQRPs_7_Time1\n5\n\n\nTr10\nQRPs_8_Time1\n7\n\n\nTr10\nQRPs_9_Time1\n6\n\n\nTr10\nQRPs_10_Time1\n7\n\n\nTr10\nQRPs_11_Time1\n7\n\n\nBi07\nQRPs_1_Time1\n7\n\n\nBi07\nQRPs_2_Time1\n7\n\n\nBi07\nQRPs_3_Time1\n2\n\n\nBi07\nQRPs_4_Time1\n7\n\n\n\n\n\n\nNow, have a look at qrp_step2. In total, we now have rows/observations, per participant, and columns/variables.\n\n\nThis follows exactly the same sequence we used when calculating descriptive statistics by gender. The only difference is that we are now grouping the data by the participant’s Code instead of Gender.\nsummarise() works exactly the same way: summarise(new_column_name = function_to_calculate_something(column_name_of_numeric_values))\nThe function_to_calculate_something can be mean(), sd() or sum() for mean scores, standard deviations, or summed-up scores respectively. You could also use min() or max() if you wanted to determine the lowest or the highest score for each participant.\n\n\n\n\n\n\n\n\n\n\n\n\nTip\n\n\n\nYou could rename the columns whilst selecting them. The pattern would be select(new_name = old_name). For example, if we wanted to select variable Code and rename it as Participant_ID, we could do that.\n\nrenaming_col <- data_prp %>% \n select(Participant_ID = Code)\n\nhead(renaming_col, n = 5)\n\n\n\n\nParticipant_ID\n\n\n\nTr10\n\n\nBi07\n\n\nSK03\n\n\nSM95\n\n\nSt01"
+ "text": "2.4 Activity 4: Questionable Research Practices (QRPs)\nThe main goal is to compute the mean QRP score per participant for time point 1.\nAt the moment, the data is in wide format. The table below shows data from the first 3 participants:\n\nhead(data_prp, n = 3)\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCode\nGender\nAge\nEthnicity\nSecondyeargrade\nOpptional_mod\nOpptional_mod_1_TEXT\nResearch_exp\nResearch_exp_1_TEXT\nPlan_prereg\nSATS28_1_Affect_Time1\nSATS28_2_Affect_Time1\nSATS28_3_Affect_Time1\nSATS28_4_Affect_Time1\nSATS28_5_Affect_Time1\nSATS28_6_Affect_Time1\nSATS28_7_CognitiveCompetence_Time1\nSATS28_8_CognitiveCompetence_Time1\nSATS28_9_CognitiveCompetence_Time1\nSATS28_10_CognitiveCompetence_Time1\nSATS28_11_CognitiveCompetence_Time1\nSATS28_12_CognitiveCompetence_Time1\nSATS28_13_Value_Time1\nSATS28_14_Value_Time1\nSATS28_15_Value_Time1\nSATS28_16_Value_Time1\nSATS28_17_Value_Time1\nSATS28_18_Value_Time1\nSATS28_19_Value_Time1\nSATS28_20_Value_Time1\nSATS28_21_Value_Time1\nSATS28_22_Difficulty_Time1\nSATS28_23_Difficulty_Time1\nSATS28_24_Difficulty_Time1\nSATS28_25_Difficulty_Time1\nSATS28_26_Difficulty_Time1\nSATS28_27_Difficulty_Time1\nSATS28_28_Difficulty_Time1\nQRPs_1_Time1\nQRPs_2_Time1\nQRPs_3_Time1\nQRPs_4_Time1\nQRPs_5_Time1\nQRPs_6_Time1\nQRPs_7_Time1\nQRPs_8_Time1\nQRPs_9_Time1\nQRPs_10_Time1\nQRPs_11_Time1\nQRPs_12NotQRP_Time1\nQRPs_13NotQRP_Time1\nQRPs_14NotQRP_Time1\nQRPs_15NotQRP_Time1\nUnderstanding_OS_1_Time1\nUnderstanding_OS_2_Time1\nUnderstanding_OS_3_Time1\nUnderstanding_OS_4_Time1\nUnderstanding_OS_5_Time1\nUnderstanding_OS_6_Time1\nUnderstanding_OS_7_Time1\nUnderstanding_OS_8_Time1\nUnderstanding_OS_9_Time1\nUnderstanding_OS_10_Time1\nUnderstanding_OS_11_Time1\nUnderstanding_OS_12_Time1\nPre_reg_group\nOther_OS_behav_2\nOther_OS_behav_4\nOther_OS_behav_5\nClosely_follow\nSATS28_Affect_Time2_mean\nSATS28_CognitiveCompetence_Time2_mean\nSATS28_Value_Time2_mean\nSATS28_Difficulty_Time2_mean\nQRPs_Acceptance_Time2_mean\nTime2_Understanding_OS\nSupervisor_1\nSupervisor_2\nSupervisor_3\nSupervisor_4\nSupervisor_5\nSupervisor_6\nSupervisor_7\nSupervisor_8\nSupervisor_9\nSupervisor_10\nSupervisor_11\nSupervisor_12\nSupervisor_13\nSupervisor_14\nSupervisor_15_R\n\n\n\nTr10\n2\n22\nWhite European\n2\n1\nResearch methods in first year\n2\nNA\n1\n4\n5\n3\n4\n5\n5\n4\n2\n2\n6\n4\n3\n1\n7\n7\n2\n1\n3\n3\n2\n2\n3\n5\n2\n6\n4\n4\n1\n7\n7\n5\n7\n3\n4\n5\n7\n6\n7\n7\n2\n1\n1\n2\n2\n2\n2\n6\nEntirely confident\nEntirely confident\n6\n6\nEntirely confident\nEntirely confident\nEntirely confident\nEntirely confident\n1\n1\n1\nNA\n2\n3.500000\n4.166667\n3.000000\n2.857143\n5.636364\n5.583333\n5\n5\n6\n6\n5\n5\n1\n5\n6\n5\nNA\n4\n4\n5\n1\n\n\nBi07\n2\n20\nWhite British\n3\n2\nNA\n2\nNA\n3\n5\n6\n2\n5\n5\n6\n2\n2\n2\n7\n3\n5\n1\n7\n7\n1\n1\n6\n3\n1\n1\n2\n6\n2\n7\n2\n5\n7\n7\n7\n2\n7\n3\n7\n7\n7\n7\n6\n7\n2\n1\n4\n4\n2\nNot at all confident\nNot at all confident\nNot at all confident\n6\nEntirely confident\nNot at all confident\n3\n6\n6\n2\n2\n1\nNA\nNA\nNA\n2\n3.166667\n4.666667\n6.222222\n2.857143\n5.454546\n3.333333\n7\n6\n7\n7\n7\n7\n1\n5\n7\n7\n7\n5\n2\n7\n1\n\n\nSK03\n2\n22\nWhite British\n1\n2\nNA\n2\nNA\n1\n5\n3\n5\n2\n5\n2\n2\n2\n2\n6\n5\n3\n2\n6\n6\n3\n3\n5\n3\n4\n3\n5\n5\n2\n5\n2\n5\n5\n7\n7\n6\n6\n7\n6\n7\n7\n7\n5\n7\n1\n1\n3\n2\n6\n2\n3\n6\n6\n5\n2\n5\n5\n5\n4\n5\n1\nNA\nNA\nNA\n2\n4.833333\n6.166667\n6.000000\n4.000000\n6.272727\n5.416667\n7\n7\n7\n7\n7\n7\n1\n7\n7\n7\n7\n7\n5\n7\n1\n\n\n\n\n\n\n\n\nLooking at the QRP data at time point 1, you determine that\n\nindividual item columns are \nnumeric\ncharacter, and\naccording to the codebook, there are \nno\nsome reverse-coded items in this questionnaire.\n\nAccording to the codebook and the data table above, we just have to compute the average score for QRP items to , since items to are distractor items. Seems quite straightforward.\nHowever, as you can see in the table above, each item is in a separate column, meaning the data is in wide format. It would be much easier to calculate the mean scores if the items were arranged in long format.\nLet’s tackle this problem step by step. It’s best to create a separate data object for this. If we tried to compute it within data_prp, it could quickly become messy.\n\n\nStep 1: Select the relevant columns Code, and QRPs_1_Time1 to QRPs_1_Time1 and store them in an object called qrp_t1\n\n\nStep 2: Pivot the data from wide format to long format using pivot_longer() so we can calculate the average score more easily (in step 3)\n\nStep 3: Calculate the average QRP score (QRPs_Acceptance_Time1_mean) per participant using group_by() and summarise()\n\n\n\nqrp_t1 <- data_prp %>% \n #Step 1\n select(Code, QRPs_1_Time1:QRPs_11_Time1) %>%\n # Step 2\n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\") %>% \n # Step 3\n group_by(Code) %>% # grouping by participant id\n summarise(QRPs_Acceptance_Time1_mean = mean(Scores)) %>% # calculating the average Score\n ungroup() # just make it a habit\n\n\n\n\n\n\n\nExplain the individual functions\n\n\n\n\n\n\n\nselect ()\npivot_longer()\ngroup_by() and summarise()\n\n\n\nThe select function allows to include or exclude certain variables (columns). Here we want to focus on the participant ID column (i.e., Code) and the QRP items at time point 1. We can either list them all individually, i.e., Code, QRPs_1_Time1, QRPs_2_Time1, QRPs_3_Time1, and so forth (you get the gist), but that would take forever to type.\nA shortcut is to use the colon operator :. It allows us to select all columns that fall within the range of first_column_name to last_column_name. We can apply this here since the QRP items (1 to 11) are sequentially listed in data_prp.\n\nqrp_step1 <- data_prp %>% \n select(Code, QRPs_1_Time1:QRPs_11_Time1)\n\n# show first 5 rows of qrp_step1\nhead(qrp_step1, n = 5)\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCode\nQRPs_1_Time1\nQRPs_2_Time1\nQRPs_3_Time1\nQRPs_4_Time1\nQRPs_5_Time1\nQRPs_6_Time1\nQRPs_7_Time1\nQRPs_8_Time1\nQRPs_9_Time1\nQRPs_10_Time1\nQRPs_11_Time1\n\n\n\nTr10\n7\n7\n5\n7\n3\n4\n5\n7\n6\n7\n7\n\n\nBi07\n7\n7\n2\n7\n3\n7\n7\n7\n7\n6\n7\n\n\nSK03\n7\n7\n6\n6\n7\n6\n7\n7\n7\n5\n7\n\n\nSM95\n7\n7\n2\n6\n7\n5\n7\n7\n4\n2\n4\n\n\nSt01\n7\n7\n6\n7\n2\n7\n7\n7\n7\n5\n7\n\n\n\n\n\n\nHow many rows/observations and columns/variables do we have in qrp_step1?\n\nrows/observations: \n\ncolumns/variables: \n\n\n\n\nAs you can see, the table we got from Step 1 is in wide format. To get it into wide format, we need to define:\n\nthe columns that need to be reshuffled from wide into long format (col argument). Here we selected “everything except the Code column”, as indicated by -Code [minus Code]. However, QRPs_1_Time1:QRPs_11_Time1 would also work and give you the exact same result.\nthe names_to argument. R is creating a new column in which all the column names from the columns you selected in col will be stored in. Here we are naming this column “Items” but you could pick something equally sensible if you like.\nthe values_to argument. R creates this second column to store all responses the participants gave to the individual questions, i.e., all the numbers in this case. We named it “Scores” here, but you could have called it something different, like “Responses”\n\n\nqrp_step2 <- qrp_step1 %>% \n pivot_longer(cols = -Code, names_to = \"Items\", values_to = \"Scores\")\n\n# show first 15 rows of qrp_step2\nhead(qrp_step2, n = 15)\n\n\n\n\nCode\nItems\nScores\n\n\n\nTr10\nQRPs_1_Time1\n7\n\n\nTr10\nQRPs_2_Time1\n7\n\n\nTr10\nQRPs_3_Time1\n5\n\n\nTr10\nQRPs_4_Time1\n7\n\n\nTr10\nQRPs_5_Time1\n3\n\n\nTr10\nQRPs_6_Time1\n4\n\n\nTr10\nQRPs_7_Time1\n5\n\n\nTr10\nQRPs_8_Time1\n7\n\n\nTr10\nQRPs_9_Time1\n6\n\n\nTr10\nQRPs_10_Time1\n7\n\n\nTr10\nQRPs_11_Time1\n7\n\n\nBi07\nQRPs_1_Time1\n7\n\n\nBi07\nQRPs_2_Time1\n7\n\n\nBi07\nQRPs_3_Time1\n2\n\n\nBi07\nQRPs_4_Time1\n7\n\n\n\n\n\n\nNow, have a look at qrp_step2. In total, we now have rows/observations, per participant, and columns/variables.\n\n\nThis follows exactly the same sequence we used when calculating descriptive statistics by gender. The only difference is that we are now grouping the data by the participant’s Code instead of Gender.\nsummarise() works exactly the same way: summarise(new_column_name = function_to_calculate_something(column_name_of_numeric_values))\nThe function_to_calculate_something can be mean(), sd() or sum() for mean scores, standard deviations, or summed-up scores respectively. You could also use min() or max() if you wanted to determine the lowest or the highest score for each participant.\n\n\n\n\n\n\n\n\n\n\n\n\nTip\n\n\n\nYou could rename the columns whilst selecting them. The pattern would be select(new_name = old_name). For example, if we wanted to select variable Code and rename it as Participant_ID, we could do that.\n\nrenaming_col <- data_prp %>% \n select(Participant_ID = Code)\n\nhead(renaming_col, n = 5)\n\n\n\n\nParticipant_ID\n\n\n\nTr10\n\n\nBi07\n\n\nSK03\n\n\nSM95\n\n\nSt01"
},
{
"objectID": "02-wrangling.html#activity-5-knitting",
@@ -137,7 +137,7 @@
"href": "02-wrangling.html#pair-coding",
"title": "2 Data wrangling I",
"section": "Pair-coding",
- "text": "Pair-coding\nWe will continue working with the data from Binfet et al. (2021), focusing on the randomised controlled trial of therapy dog interventions. Today, our goal is to calculate an average Flourishing score for each participant at time point 1 (pre-intervention) using the raw data file dog_data_raw. Currently, the data looks like this:\n\n\n\n\n\nRID\nF1_1\nF1_2\nF1_3\nF1_4\nF1_5\nF1_6\nF1_7\nF1_8\n\n\n\n1\n6\n7\n5\n5\n7\n7\n6\n6\n\n\n2\n5\n7\n6\n5\n5\n5\n5\n4\n\n\n3\n5\n5\n5\n6\n6\n6\n5\n5\n\n\n4\n7\n6\n7\n7\n7\n6\n7\n4\n\n\n5\n5\n5\n4\n6\n7\n7\n7\n6\n\n\n\n\n\n\nHowever, we want the data to look like this:\n\n\n\n\n\nRID\nFlourishing_pre\n\n\n\n1\n6.125\n\n\n2\n5.250\n\n\n3\n5.375\n\n\n4\n6.375\n\n\n5\n5.875\n\n\n\n\n\n\nTask 1: Open the R project you created last week\nIf you haven’t created an R project for the lab yet, please do so now. If you already have one set up, go ahead and open it.\nTask 2: Open your .Rmd file from last week\nSince we haven’t used it much yet, feel free to continue using the .Rmd file you created last week in Task 2.\nTask 3: Load in the library and read in the data\nThe data should be in your project folder. If you didn’t download it last week, or if you’d like a fresh copy, you can download the data again here: data_pair_ch1.\nWe will be using the tidyverse package today, and the data file we need to read in is dog_data_raw.csv.\n\n\n\n\n\n\nHint\n\n\n\n\n\n\n# loading tidyverse into the library\nlibrary(???)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"???\")\n\n\n\n\nTask 4: Calculating the mean for Flourishing_pre\n\n\n\nStep 1: Select all relevant columns, including participant ID and all 8 items from the Flourishing questionnaire completed before the intervention. Store this data in an object called data_flourishing.\n\n\n\n\n\n\n\nHint\n\n\n\n\n\nLook at the codebook. Try to determine:\n\nThe variable name of the column where the participant ID is stored.\nThe items related to the Flourishing scale at the pre-intervention stage.\n\n\n\n\n\n\n\nMore concrete hint\n\n\n\n\n\nFrom the codebook, we know that:\n\nThe participant ID column is called RID.\nThe Flourishing items at the pre-intervention stage start with F1_.\n\n\n\n\n\n\n\n\n\nStep 2: Pivot the data from wide format to long format so we can calculate the average score more easily (in step 3).\n\n\n\n\n\n\n\nHint\n\n\n\n\n\nWhich pivot function should you use? We have pivot_wider() and pivot_longer() to choose from.\nWe also need 3 arguments in that function:\n\nThe columns you want to select (e.g., all the Flourishing items),\nThe name of the column where the current column headings will be stored (e.g., “Questionnaire”),\nThe name of the column that should store all the values (e.g., “Responses”).\n\n\n\n\n\n\n\nMore concrete hint\n\n\n\n\n\n\n pivot_longer(cols = ???, names_to = \"???\", values_to = \"???\")\n\n\n\n\n\n\n\n\n\nStep 3: Calculate the average Flourishing score per participant and name this column Flourishing_pre to match the table above.\n\n\n\n\n\n\n\nHint\n\n\n\n\n\nBefore summarising the mean, you may need to group the data.\n\n\n\n\n\n\nMore concrete hint\n\n\n\n\n\n\n group_by(???) %>% \n summarise(Flourishing_pre = ???(???)) %>% \n ungroup()\n\n\n\n\n\n\n\n\n\n\n\n\n\nSolution\n\n\n\n\n\n\n# loading tidyverse into the library\nlibrary(tidyverse)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"dog_data_raw.csv\")\n\n# Task 4: Tidying \ndata_flourishing <- dog_data_raw %>% \n # Step 1\n select(RID, F1_1:F1_8) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Questionnaire\", values_to = \"Responses\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_pre = mean(Response)) %>% \n ungroup()"
+ "text": "Pair-coding\nWe will continue working with the data from Binfet et al. (2021), focusing on the randomised controlled trial of therapy dog interventions. Today, our goal is to calculate an average Flourishing score for each participant at time point 1 (pre-intervention) using the raw data file dog_data_raw. Currently, the data looks like this:\n\n\n\n\n\nRID\nF1_1\nF1_2\nF1_3\nF1_4\nF1_5\nF1_6\nF1_7\nF1_8\n\n\n\n1\n6\n7\n5\n5\n7\n7\n6\n6\n\n\n2\n5\n7\n6\n5\n5\n5\n5\n4\n\n\n3\n5\n5\n5\n6\n6\n6\n5\n5\n\n\n4\n7\n6\n7\n7\n7\n6\n7\n4\n\n\n5\n5\n5\n4\n6\n7\n7\n7\n6\n\n\n\n\n\n\nHowever, we want the data to look like this:\n\n\n\n\n\nRID\nFlourishing_pre\n\n\n\n1\n6.125\n\n\n2\n5.250\n\n\n3\n5.375\n\n\n4\n6.375\n\n\n5\n5.875\n\n\n\n\n\n\nTask 1: Open the R project you created last week\nIf you haven’t created an R project for the lab yet, please do so now. If you already have one set up, go ahead and open it.\nTask 2: Open your .Rmd file from last week\nSince we haven’t used it much yet, feel free to continue using the .Rmd file you created last week in Task 2.\nTask 3: Load in the library and read in the data\nThe data should be in your project folder. If you didn’t download it last week, or if you’d like a fresh copy, you can download the data again here: data_pair_ch1.\nWe will be using the tidyverse package today, and the data file we need to read in is dog_data_raw.csv.\n\n\n\n\n\n\nHint\n\n\n\n\n\n\n# loading tidyverse into the library\nlibrary(???)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"???\")\n\n\n\n\nTask 4: Calculating the mean for Flourishing_pre\n\n\n\nStep 1: Select all relevant columns from dog_data_raw, including participant ID and all items from the Flourishing questionnaire completed before the intervention. Store this data in an object called data_flourishing.\n\n\n\n\n\n\n\nHint\n\n\n\n\n\nLook at the codebook. Try to determine:\n\nThe variable name of the column where the participant ID is stored.\nThe items related to the Flourishing scale at the pre-intervention stage.\n\n\n\n\n\n\n\nMore concrete hint\n\n\n\n\n\nFrom the codebook, we know that:\n\nThe participant ID column is called RID.\nThe Flourishing items at the pre-intervention stage start with F1_.\n\n\ndata_flourishing <- ??? %>% \n select(???, F1_???:F1_???)\n\n\n\n\n\n\n\n\n\nStep 2: Pivot the data from wide format to long format so we can calculate the average score more easily (in step 3).\n\n\n\n\n\n\n\nHint\n\n\n\n\n\nWhich pivot function should you use? We have pivot_wider() and pivot_longer() to choose from.\nWe also need 3 arguments in that function:\n\nThe columns you want to select (e.g., all the Flourishing items),\nThe name of the column where the current column headings will be stored (e.g., “Questionnaire”),\nThe name of the column that should store all the values (e.g., “Responses”).\n\n\n\n\n\n\n\nMore concrete hint\n\n\n\n\n\nWe need pivot_longer(). You already encountered pivot_longer() in first year (or in the individual walkthrough if you have already completed this Chapter). The 3 arguments was also a give-away; pivot_wider() only requires 2 arguments.\n\n pivot_longer(cols = ???, names_to = \"???\", values_to = \"???\")\n\n\n\n\n\n\n\n\n\nStep 3: Calculate the average Flourishing score per participant and name this column Flourishing_pre to match the table above.\n\n\n\n\n\n\n\nHint\n\n\n\n\n\nBefore summarising the mean, you may need to group the data.\n\n\n\n\n\n\nMore concrete hint\n\n\n\n\n\nTo compute an average score per participant, we would need to group by participant ID first.\n\n group_by(???) %>% \n summarise(Flourishing_pre = mean(???)) %>% \n ungroup()\n\n\n\n\n\n\n\n\n\n\n\n\n\nSolution\n\n\n\n\n\n\n# loading tidyverse into the library\nlibrary(tidyverse)\n\n# reading in `dog_data_raw.csv`\ndog_data_raw <- read_csv(\"dog_data_raw.csv\")\n\n# Task 4: Tidying \ndata_flourishing <- dog_data_raw %>% \n # Step 1\n select(RID, F1_1:F1_8) %>% \n # Step 2\n pivot_longer(cols = -RID, names_to = \"Questionnaire\", values_to = \"Responses\") %>% \n # Step 3\n group_by(RID) %>% \n summarise(Flourishing_pre = mean(Response)) %>% \n ungroup()"
},
{
"objectID": "02-wrangling.html#test-your-knowledge-and-challenge-yourself",