Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sos challenge #2

Open
wants to merge 42 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
f4b9291
Automatic results upload after execution for CdmExecutionSettings
azimov Feb 28, 2023
e45354f
doc strings
azimov Feb 28, 2023
70e1196
doc strings
azimov Feb 28, 2023
1e9a045
Fixed bad type check
azimov Feb 28, 2023
fed5682
missing comma
azimov Feb 28, 2023
d9fa6d7
Revert "missing comma"
azimov Feb 28, 2023
ed42a8c
missing comma
azimov Feb 28, 2023
321d371
missing comma correction
azimov Feb 28, 2023
dec5dd0
whitespace
azimov Feb 28, 2023
800a27f
logic
azimov Feb 28, 2023
a2d6003
expermient with tar_env
azimov Mar 1, 2023
ec789b2
expermient with tar_env removed
azimov Mar 1, 2023
589423e
experiment with tar_env
azimov Mar 1, 2023
bd713ae
experiment with tar_env
azimov Mar 1, 2023
a7e8728
targets tar_script for readability
azimov Mar 1, 2023
2ee618e
Result upload where module functions for specs don't exist handled
azimov Mar 2, 2023
67223ec
Result schema creation and improve readability of code executed in re…
azimov Mar 4, 2023
12b7902
Module execution environment for code readability
azimov Mar 4, 2023
630b09a
Renamed module exec file
azimov Mar 4, 2023
f80849e
Example file
azimov Mar 4, 2023
79c1ed7
Merge branch 'develop' into results-upload
azimov Mar 4, 2023
825e0a6
Removed code
azimov Mar 4, 2023
4a97546
Merge branch 'develop-azimov' into results-upload
azimov Mar 7, 2023
825a81b
missing doc files
azimov Mar 9, 2023
ec56393
error
azimov Mar 10, 2023
bad1192
docstring changes
azimov Mar 14, 2023
aa1a5bf
Merge branch 'OHDSI:main' into results-upload
azimov Mar 14, 2023
b7531d1
Added info logging for result upload
azimov Apr 10, 2023
549b0bb
Merge branch 'develop' into results-upload
azimov Apr 24, 2023
92cf9ad
Allow unit tests to run on mac by using a file backend
azimov Apr 24, 2023
c3b61a4
Merge branch 'develop' into results-upload
azimov Apr 27, 2023
46c597e
Updated run script
azimov Apr 27, 2023
0846de6
Updated run script
azimov Apr 27, 2023
0cf869d
Update DatabaseMetaData.R
DrTTrousers May 3, 2023
967a912
Merge pull request #1 from DrTTrousers/develop(ru)
DrTTrousers May 3, 2023
693f2f4
Implement better storeConnectionDetails - fixes #74
anthonysena May 8, 2023
1f42d1f
Add check for tempEmulationSchema - fixes #73
anthonysena May 8, 2023
9e62f14
Ensure uniqueness for all cdm tables listed
anthonysena May 8, 2023
5bfafe2
Use renv::install for mandatory dependencies
anthonysena May 9, 2023
7789202
Bump CG Module Ref in test analysisSpecification.json
anthonysena May 9, 2023
a147a57
Add option to pass tempEmulationSchema in execution settings and add …
anthonysena May 10, 2023
d0af4d8
Merge branch 'main' into sos-challenge
DrTTrousers May 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion R/DatabaseMetaData.R
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ createDatabaseMetaData <- function(executionSettings, keyringName = NULL) {
connection = connection,
databaseSchema = executionSettings$cdmDatabaseSchema
)
cdmTableList <- tolower(cdmTableList)
cdmTableList <- unique(tolower(cdmTableList))

if (!length(cdmTableList[which(x = cdmTableList %in% requiredTables)]) == length(requiredTables)) {
missingCdmTables <- requiredTables[!(requiredTables %in% cdmTableList)]
Expand Down
11 changes: 11 additions & 0 deletions R/Execution.R
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,17 @@ execute <- function(analysisSpecifications,
checkmate::assertChoice(x = keyringName, choices = keyringList$keyring, null.ok = TRUE, add = errorMessages)
checkmate::reportAssertions(collection = errorMessages)

# Assert that the temp emulation schema is set if required for the dbms
# specified by the executionSettings
connectionDetails <- retrieveConnectionDetails(
connectionDetailsReference = executionSettings$connectionDetailsReference,
keyringName = keyringName
)
DatabaseConnector::assertTempEmulationSchemaSet(
dbms = connectionDetails$dbms,
tempEmulationSchema = getOption("sqlRenderTempEmulationSchema")
)

modules <- ensureAllModulesInstantiated(analysisSpecifications)

if (is.null(executionScriptFolder)) {
Expand Down
18 changes: 9 additions & 9 deletions R/ModuleInstantiation.R
Original file line number Diff line number Diff line change
Expand Up @@ -191,15 +191,15 @@ instantiateModule <- function(module, version, remoteRepo, remoteUsername, modul
stop(message)
}

script <- "
renv::restore(prompt = FALSE)
if (!require('ParallelLogger', quietly = TRUE)) {
install.packages('ParallelLogger')
}
if (!require('keyring', quietly = TRUE)) {
install.packages('keyring')
}
"
script <- paste(
c(
"renv::install(c('ParallelLogger', 'keyring'), prompt = FALSE)",
sprintf("ParallelLogger::addDefaultFileLogger(file.path('%s', 'moduleInitLog.txt'))", moduleFolder),
sprintf("ParallelLogger::addDefaultErrorReportLogger(fileName = file.path('%s', 'moduleInitErrorReport.R'))", moduleFolder),
"renv::restore(prompt = FALSE)"
),
collapse = "\n"
)
tempScriptFile <- tempfile(fileext = ".R")
fileConn <- file(tempScriptFile)
writeLines(script, fileConn)
Expand Down
2 changes: 2 additions & 0 deletions R/RunModule.R
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ runModule <- function(analysisSpecifications, keyringSettings, moduleIndex, exec

options(andromedaTempFolder = file.path(jobContext$moduleExecutionSettings$workFolder, 'andromedaTemp'))

options(tempEmulationSchema = jobContext$moduleExecutionSettings$tempEmulationSchema)

if (Sys.getenv('FORCE_RENV_USE', '') == 'TRUE') {
renv::use(lockfile = 'renv.lock')
}
Expand Down
4 changes: 3 additions & 1 deletion R/Settings.R
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ createCdmExecutionSettings <- function(connectionDetailsReference,
workDatabaseSchema,
cdmDatabaseSchema,
cohortTableNames = CohortGenerator::getCohortTableNames(cohortTable = "cohort"),
tempEmulationSchema = getOption("sqlRenderTempEmulationSchema"),
workFolder,
resultsFolder,
minCellCount = 5,
Expand All @@ -125,6 +126,7 @@ createCdmExecutionSettings <- function(connectionDetailsReference,
workDatabaseSchema = workDatabaseSchema,
cdmDatabaseSchema = cdmDatabaseSchema,
cohortTableNames = cohortTableNames,
tempEmulationSchema = tempEmulationSchema,
workFolder = workFolder,
resultsFolder = resultsFolder,
minCellCount = minCellCount,
Expand Down Expand Up @@ -214,7 +216,7 @@ storeConnectionDetails <- function(connectionDetails, connectionDetailsReference
if (is.function(connectionDetails[[i]])) {
detail <- connectionDetails[[i]]()
if (is.null(detail)) {
connectionDetails[[i]] <- ""
connectionDetails[[i]] <- list(NULL) # Fixes Issue #74
} else {
connectionDetails[[i]] <- connectionDetails[[i]]()
}
Expand Down
2 changes: 1 addition & 1 deletion inst/testdata/analysisSpecification.json
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@
"moduleSpecifications": [
{
"module": "CohortGeneratorModule",
"version": "0.1.0",
"version": "0.1.1-1",
"remoteRepo": "github.com",
"remoteUsername": "ohdsi",
"settings": {
Expand Down
1 change: 1 addition & 0 deletions man/Strategus-package.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/createCdmExecutionSettings.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 42 additions & 0 deletions man/withModuleRenv.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.