Error running by_split()

[tommygunawan]

Hello. My team is analyzing data collected from task called trail making test. We created a fn_score function that is able to compute a score for each participant (sum of time taken to complete each trial). When we tried running the by_split() function, we ran into the following error:

split_scores <- by_split(
  df_tmt,
  df_tmt$subject,
  fn_score,
  method = "random",
  replace = TRUE,
  split_p = 1,
  replications = 1000,
)

Error in checkForRemoteErrors(val) : 8 nodes produced errors; first error: could not find function "%>%"

This error persist even if we use first_second or odd_even for the splitting. We have successfully computed split-half reliabilities for many of our tasks and this was the first time we're encountering this error. Any ideas on what may be causing it?

[tpronk]

Hey, thanks for your question! By default, splithalfr imports the function %>% from the dplyr package, which in turn imports it from magrittr. If your use this function inside your own code, you'll first need to import it yourself by importing dplyr or magrittr.

However, if you use it inside of the fn_score function passed to by_split, things might be a bit more complicated because the splithalfr uses multi-core processing by default. If this is the case..

1. Could you run a little test by setting ncores = 1?
2. Could you share your fn_score function?

Example:

split_scores <- by_split(
  df_tmt,
  df_tmt$subject,
  fn_score,
  method = "random",
  replace = TRUE,
  split_p = 1,
  replications = 1000,
  ncores = 1
)

[tommygunawan]

Hi Thomas, thanks for your response! You were right, the problem was with multi-core processing. Setting ncores = 1 fixes this issue.

For reference in case anyone encounters a similar issue: we used the trail making test from Inquisit. We used total time to complete the task as our primary score. The fn_score function below computes time taken per trial for the two practice and two test blocks, and add them all up.

fn_score <- function (df) {
  new_tmt <- df %>% mutate(c1 = values.sampleATime - lag(values.sampleATime))
  new_tmt1 <- new_tmt %>% mutate(c2 = values.trail1Time - lag(values.trail1Time))
  new_tmt2 <- new_tmt1 %>% mutate(c3 = values.sampleBTime - lag(values.sampleBTime))
  new_tmt3 <- new_tmt2 %>% mutate(c4 = values.trail2Time - lag(values.trail2Time))
                                  
  new_tmt3$c5 <- new_tmt3$c1 + new_tmt3$c2 + new_tmt3$c3 + new_tmt3$c4
  tmt_score <- sum(new_tmt3$c5,na.rm = TRUE)

  return (tmt_score)
}

Thank you again for your help!

[tpronk]

Thanks for checking my hunch and offering a workaround for fellow researchers (and developers). I'll dive a bit deeper into the issue below, also as notes for a future update to splithalfr. The story is rather technical, so I'd say: if ncores = 1 is fast enough for your purposes, just stick with that :).

Serial mode

If you specify ncores = 1, then splithalfr runs in serial mode, where it uses R's apply function to calculate the split scores. As a consequence, inside of your fn_score function, you got access to any variables that belong to the global environment.

To make this a bit more concrete, using the RAPI vignette as a starting point, the code below works fine.

# Define a new function, `my_sum` in the global environment
my_sum = sum
# Use `my_sum` inside of `fn_score`
fn_score <- function (ds) {
  return (my_sum(ds$score))
}
# Running in serial mode, so `my_sum` is accessible from inside `fn_score`
split_scores <- by_split(
  ds_rapi,
  ds_rapi$twnr,
  fn_score,
  replications = 4,
  ncores = 1
)

Parallel mode

If you don't specify ncores, the splithalfr will distribute the computations across each of your CPU cores (i.e., the computations are parallelized). To achieve this, it creates a cluster using R's parallel package, where each core used in this cluster is called a "node". Now, these nodes do have access to R's base functions (like mean and sum), but they don't have access to the other variables and functions in the global environment.

So... the code below does NOT work:

# Define a new function, `my_sum` in the global environment
my_sum = sum
# Use `my_sum` inside of `fn_score`
fn_score <- function (ds) {
  return (my_sum(ds$score))
}
# Running in parallel mode, so `my_sum` is NOT accessible from inside `fn_score`
split_scores <- by_split(
  ds_rapi,
  ds_rapi$twnr,
  fn_score,
  replications = 4
)

Workaround

• One workaround is simply setting ncores = 1.
• Another workaround I used in the IAT vignette is defining a custom function (fn_block) inside of fn_score.
• Since you use dplyr/magrittr functions such as %>% and mutate, you could instead import dplyr inside of fn_score, like below. However, this might be a bit slow; I'm not sure.

fn_score <- function (df) {
  library(dplyr)
  new_tmt <- df %>% mutate(c1 = values.sampleATime - lag(values.sampleATime))
  ...
}

Solution

A really elegant solution would have splithalfr offer a way to have fn_score access dplyr, without needing to import it each time fn_score is called. What I could do is add an argument to by_split where you could list all the packages/functions you'd like available inside of fn_score. In technical terms: add those packages/functions to clusterExport over here. I'll ask around a bit how other devs do this, to see if this is the best approach.