Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large difference in run-times of test files on being measured with testthatQuantity wrapper #6

Open
analyticalmonk opened this issue Jun 22, 2015 · 2 comments

Comments

@analyticalmonk
Copy link
Owner

@tdhock Hey Toby, this was what I was talking about.

The results

Run-time for entire file with original version

          test_name     seconds status branch           date_time
1  basic case works 0.001079901   pass master 2015-01-08 14:09:43
2 NULLs are dropped 0.001078013   pass master 2015-01-08 14:09:43
3       test-join.r 0.002519254   pass master 2015-01-08 14:09:43
4  basic case works 0.001080284   pass master 2015-01-08 14:03:37
5 NULLs are dropped 0.000931637   pass master 2015-01-08 14:03:37
6       test-join.r 0.002546496   pass master 2015-01-08 14:03:37

and
Run-time for entire file with substituted code (testthatQuantity)

          test_name     seconds status branch           date_time
1  basic case works 0.001073032   pass master 2015-01-08 14:09:43
2 NULLs are dropped 0.000929690   pass master 2015-01-08 14:09:43
3       test-join.r 0.132559892   pass master 2015-01-08 14:09:43
4  basic case works 0.001076601   pass master 2015-01-08 14:03:37
5 NULLs are dropped 0.000931337   pass master 2015-01-08 14:03:37
6       test-join.r 0.132972392   pass master 2015-01-08 14:03:37

The background

   seconds_file <- if(require(microbenchmark)){
      times <- microbenchmark(test = {
        source(temp_file1, local = T)
      }, times = 3)
      times$time/1e9
    } else {
      replicate(3, {
        time_vec <- system.time( {
          source(temp_file1, local = T)
        } )
        time_vec[["elapsed"]]
      })
    }

and

seconds_file2 <- {
      times <- microbenchmark(times = 3, source(file = temp_file2, local = T))
      times$time/1e9
    }

where the tempfiles 1 and 2 correspond to the non-substituted and substituted version of the test-code.

  t_lines <- readLines(test_path)
  q_lines <- sub("test_that(", "testthatQuantity(", t_lines, fixed = TRUE)
  temp_file1 <- tempfile()
  temp_file2 <- tempfile()
  writeLines(t_lines, temp_file1)
  writeLines(q_lines, temp_file2)
@tdhock
Copy link

tdhock commented Jul 14, 2015

any update?

@tdhock
Copy link

tdhock commented Jul 24, 2015

if you do this same test with the current version of Rperform what is the result? if it is OK now can you please explain what was the trick you used to get it to work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants