`roxygenize` cannot find Bioconductor pkg dependency when setting environment #507

averissimo · 2023-08-16T09:21:45Z

Before filing a bug

I have installed the latest dev version of {precommit} with remotes::install_github('lorenzwalthert/precommit')
I have installed the latest hook revisions (update with precommit::autoupdate())
I have installed the latest release of the upstream Python framework pre-comit as described under the update instructions.

Describe the bug

Can't install the environment if roxygenize plugin has bioc::MultiAssayExperiment and bioc::SummarizedExperiment additional dependencies

To Reproduce
Steps to reproduce the behavior:

git clone https://github.com/insightsengineering/teal.modules.general
change something and run git commit -m "something"
See error when environment is being installed

Expected behavior

Install all the additional dependencies

Additional context

My operating system is Ubuntu 23.04:
Project doesn't use {renv}

Error

⊞ Click here to see the full log `/home//.cache/pre-commit/pre-commit.log`

version information

pre-commit version: 3.3.3
git --version: git version 2.39.2
sys.version:
    3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:17) [GCC 12.2.0]
sys.executable: /home/averissimo/.local/share/r-miniconda/envs/r-precommit/bin/python3.11
os.name: posix
sys.platform: linux

error information

An unexpected error has occurred: CalledProcessError: command: ('/usr/bin/Rscript', '--no-save', '--no-restore', '--no-site-file', '--no-environ', '/tmp/tmpfwe_2gfa/script.R', 'checkmate', 'digest', 'dplyr', 'ggplot2', 'glue', 'grDevices', 'lifecycle', 'logger', 'magrittr', 'methods', 'bioc::MultiAssayExperiment', 'R6', 'rlang', 'rtables', 'shinyjs', 'shinyWidgets', 'stats', 'bioc::SummarizedExperiment', 'utils', 'yaml')
return code: 1
stdout:
    Installing BiocManager [1.30.22] ...
    	OK [linked cache in 0.14 milliseconds]
    * Installed 1 package in 11 milliseconds.
stderr:
    Error: package 'GenomeInfoDbData' is not available
    In addition: Warning message:
    no MRAN records available from repository URL '/src/contrib' 
    Traceback (most recent calls last):
    30: renv::install(commandArgs(trailingOnly = TRUE))
    29: retrieve(names(remotes))
    28: handler(package, renv_retrieve_impl(package))
    27: renv_retrieve_impl(package)
    26: renv_retrieve_successful(record, path)
    25: renv_retrieve_successful_recurse(deps)
    24: renv_retrieve_successful_recurse_impl(remote)
    23: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    22: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    21: renv_retrieve_successful_recurse_impl_one(remote)
    20: renv_retrieve_impl(remote)
    19: renv_retrieve_successful(record, path)
    18: renv_retrieve_successful_recurse(deps)
    17: renv_retrieve_successful_recurse_impl(remote)
    16: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    15: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    14: renv_retrieve_successful_recurse_impl_one(remote)
    13: renv_retrieve_impl(remote)
    12: renv_retrieve_successful(record, path)
    11: renv_retrieve_successful_recurse(deps)
    10: renv_retrieve_successful_recurse_impl(remote)
     9: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
     8: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
     7: renv_retrieve_successful_recurse_impl_one(remote)
     6: renv_retrieve_impl(remote)
     5: records[[package]] %||% renv_retrieve_missing_record(package)
     4: renv_retrieve_missing_record(package)
     3: renv_available_packages_latest(package)
     2: stopf("package '%s' is not available", package)
     1: stop(sprintf(fmt, ...), call. = call.)
    Execution halted

Traceback (most recent call last):
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/error_handler.py", line 73, in error_handler
    yield
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/main.py", line 386, in main
    return hook_impl(
           ^^^^^^^^^^
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/commands/hook_impl.py", line 271, in hook_impl
    return retv | run(config, store, ns)
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/commands/run.py", line 442, in run
    install_hook_envs(to_install, store)
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/repository.py", line 248, in install_hook_envs
    _hook_install(hook)
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/repository.py", line 95, in _hook_install
    lang.install_environment(
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/languages/r.py", line 154, in install_environment
    cmd_output_b(
  File "/home/averissimo/.local/share/r-miniconda/envs/r-precommit/lib/python3.11/site-packages/pre_commit/util.py", line 110, in cmd_output_b
    raise CalledProcessError(returncode, cmd, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('/usr/bin/Rscript', '--no-save', '--no-restore', '--no-site-file', '--no-environ', '/tmp/tmpfwe_2gfa/script.R', 'checkmate', 'digest', 'dplyr', 'ggplot2', 'glue', 'grDevices', 'lifecycle', 'logger', 'magrittr', 'methods', 'bioc::MultiAssayExperiment', 'R6', 'rlang', 'rtables', 'shinyjs', 'shinyWidgets', 'stats', 'bioc::SummarizedExperiment', 'utils', 'yaml')
return code: 1
stdout:
    Installing BiocManager [1.30.22] ...
    	OK [linked cache in 0.14 milliseconds]
    * Installed 1 package in 11 milliseconds.
stderr:
    Error: package 'GenomeInfoDbData' is not available
    In addition: Warning message:
    no MRAN records available from repository URL '/src/contrib' 
    Traceback (most recent calls last):
    30: renv::install(commandArgs(trailingOnly = TRUE))
    29: retrieve(names(remotes))
    28: handler(package, renv_retrieve_impl(package))
    27: renv_retrieve_impl(package)
    26: renv_retrieve_successful(record, path)
    25: renv_retrieve_successful_recurse(deps)
    24: renv_retrieve_successful_recurse_impl(remote)
    23: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    22: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    21: renv_retrieve_successful_recurse_impl_one(remote)
    20: renv_retrieve_impl(remote)
    19: renv_retrieve_successful(record, path)
    18: renv_retrieve_successful_recurse(deps)
    17: renv_retrieve_successful_recurse_impl(remote)
    16: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    15: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    14: renv_retrieve_successful_recurse_impl_one(remote)
    13: renv_retrieve_impl(remote)
    12: renv_retrieve_successful(record, path)
    11: renv_retrieve_successful_recurse(deps)
    10: renv_retrieve_successful_recurse_impl(remote)
     9: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
     8: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
     7: renv_retrieve_successful_recurse_impl_one(remote)
     6: renv_retrieve_impl(remote)
     5: records[[package]] %||% renv_retrieve_missing_record(package)
     4: renv_retrieve_missing_record(package)
     3: renv_available_packages_latest(package)
     2: stopf("package '%s' is not available", package)
     1: stop(sprintf(fmt, ...), call. = call.)
    Execution halted

stderr:
    Error: package 'GenomeInfoDbData' is not available
    In addition: Warning message:
    no MRAN records available from repository URL '/src/contrib' 
    Traceback (most recent calls last):

The text was updated successfully, but these errors were encountered:

lorenzwalthert · 2023-08-16T09:26:43Z

I think it's related to MRAN being retired. I suggest you to pre-commit clean and then install the latest {renv} and try again.

lorenzwalthert · 2023-08-16T09:45:13Z

Just to make sure, you have bioc::GenomeInfoDbData in your additional_dependencies, not just GenomeInfoDbData?

averissimo · 2023-08-16T09:46:23Z

I believe GenomeInfoDbData is a dependency of one of the other bioc packages.

edit: However, it still fails when adding - bioc::GenomeInfoDbData as a dependency

averissimo · 2023-08-16T09:49:06Z

The problem still persists with [email protected] installed

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"

r> packageVersion("renv")
[1] ‘1.0.2’
r> packageVersion("precommit")
[1] ‘0.3.2.9020’

averissimo · 2023-08-16T10:06:48Z

A colleague is also facing the same problem in MacOSX

⊞ Click here to see the full log `/home/user/.cache/pre-commit/pre-commit.log`

version information

pre-commit version: 3.3.3
git --version: git version 2.39.2 (Apple Git-143)
sys.version:
    3.11.4 (main, Jul 25 2023, 17:36:13) [Clang 14.0.3 (clang-1403.0.22.14.1)]
sys.executable: /opt/homebrew/Cellar/pre-commit/3.3.3/libexec/bin/python3.11
os.name: posix
sys.platform: darwin

error information

An unexpected error has occurred: CalledProcessError: command: ('/usr/local/bin/Rscript', '--no-save', '--no-restore', '--no-site-file', '--no-environ', '/var/folders/gy/xtc2dqrx62s36gj1p8l_9zxh0000gn/T/tmp9qotwaga/script.R', 'checkmate', 'digest', 'dplyr', 'ggplot2', 'glue', 'grDevices', 'lifecycle', 'logger', 'magrittr', 'methods', 'bioc::MultiAssayExperiment', 'R6', 'rlang', 'rtables', 'shinyjs', 'shinyWidgets', 'stats', 'bioc::SummarizedExperiment', 'utils', 'yaml', 'bioc::GenomeInfoDbData')
return code: 1
stdout:
    Retrieving 'https://packagemanager.rstudio.com/all/latest/bin/macosx/big-sur-x86_64/contrib/4.3/ggplot2_3.4.3.tgz' ...
    	OK [file is up to date]
    Installing BiocManager [1.30.22] ...
    	OK [linked cache in 0.37 milliseconds]
    * Installed 1 package in 20 milliseconds.
stderr:
    Error: package 'GenomeInfoDbData' is not available
    In addition: Warning message:
    no MRAN records available from repository URL '/bin/macosx/big-sur-x86_64/contrib/4.3' 
    Traceback (most recent calls last):
    28: renv::install(commandArgs(trailingOnly = TRUE))
    27: retrieve(names(remotes))
    26: handler(package, renv_retrieve_impl(package))
    25: renv_retrieve_impl(package)
    24: renv_retrieve_successful(record, path)
    23: renv_retrieve_successful_recurse(deps)
    22: renv_retrieve_successful_recurse_impl(remote)
    21: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    20: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    19: renv_retrieve_successful_recurse_impl_one(remote)
    18: renv_retrieve_impl(remote)
    17: renv_retrieve_successful(record, path)
    16: renv_retrieve_successful_recurse(deps)
    15: renv_retrieve_successful_recurse_impl(remote)
    14: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    13: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    12: renv_retrieve_successful_recurse_impl_one(remote)
    11: renv_retrieve_impl(remote)
    10: renv_retrieve_successful(record, path)
     9: renv_retrieve_successful_recurse(deps)
     8: renv_retrieve_successful_recurse_impl(remote)
     7: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
     6: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
     5: renv_retrieve_successful_recurse_impl_one(remote)
     4: renv_retrieve_impl(remote)
     3: renv_available_packages_latest(package)
     2: stopf("package '%s' is not available", package)
     1: stop(sprintf(fmt, ...), call. = call.)
    Execution halted

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/error_handler.py", line 73, in error_handler
    yield
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/main.py", line 386, in main
    return hook_impl(
           ^^^^^^^^^^
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/commands/hook_impl.py", line 271, in hook_impl
    return retv | run(config, store, ns)
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/commands/run.py", line 442, in run
    install_hook_envs(to_install, store)
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/repository.py", line 248, in install_hook_envs
    _hook_install(hook)
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/repository.py", line 95, in _hook_install
    lang.install_environment(
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/languages/r.py", line 154, in install_environment
    cmd_output_b(
  File "/opt/homebrew/Cellar/pre-commit/3.3.3/libexec/lib/python3.11/site-packages/pre_commit/util.py", line 110, in cmd_output_b
    raise CalledProcessError(returncode, cmd, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('/usr/local/bin/Rscript', '--no-save', '--no-restore', '--no-site-file', '--no-environ', '/var/folders/gy/xtc2dqrx62s36gj1p8l_9zxh0000gn/T/tmp9qotwaga/script.R', 'checkmate', 'digest', 'dplyr', 'ggplot2', 'glue', 'grDevices', 'lifecycle', 'logger', 'magrittr', 'methods', 'bioc::MultiAssayExperiment', 'R6', 'rlang', 'rtables', 'shinyjs', 'shinyWidgets', 'stats', 'bioc::SummarizedExperiment', 'utils', 'yaml', 'bioc::GenomeInfoDbData')
return code: 1
stdout:
    Retrieving 'https://packagemanager.rstudio.com/all/latest/bin/macosx/big-sur-x86_64/contrib/4.3/ggplot2_3.4.3.tgz' ...
    	OK [file is up to date]
    Installing BiocManager [1.30.22] ...
    	OK [linked cache in 0.37 milliseconds]
    * Installed 1 package in 20 milliseconds.
stderr:
    Error: package 'GenomeInfoDbData' is not available
    In addition: Warning message:
    no MRAN records available from repository URL '/bin/macosx/big-sur-x86_64/contrib/4.3' 
    Traceback (most recent calls last):
    28: renv::install(commandArgs(trailingOnly = TRUE))
    27: retrieve(names(remotes))
    26: handler(package, renv_retrieve_impl(package))
    25: renv_retrieve_impl(package)
    24: renv_retrieve_successful(record, path)
    23: renv_retrieve_successful_recurse(deps)
    22: renv_retrieve_successful_recurse_impl(remote)
    21: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    20: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    19: renv_retrieve_successful_recurse_impl_one(remote)
    18: renv_retrieve_impl(remote)
    17: renv_retrieve_successful(record, path)
    16: renv_retrieve_successful_recurse(deps)
    15: renv_retrieve_successful_recurse_impl(remote)
    14: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
    13: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
    12: renv_retrieve_successful_recurse_impl_one(remote)
    11: renv_retrieve_impl(remote)
    10: renv_retrieve_successful(record, path)
     9: renv_retrieve_successful_recurse(deps)
     8: renv_retrieve_successful_recurse_impl(remote)
     7: dynamic(key = list(remote = remote), value = renv_retrieve_successful_recurse_impl_one(remote))
     6: `_renv_dynamic_objects`[[id]] %??% {
            dlog("dynamic", "memoizing dynamic value for '%s'", id)
            value
        }
     5: renv_retrieve_successful_recurse_impl_one(remote)
     4: renv_retrieve_impl(remote)
     3: renv_available_packages_latest(package)
     2: stopf("package '%s' is not available", package)
     1: stop(sprintf(fmt, ...), call. = call.)
    Execution halted

lorenzwalthert · 2023-08-16T10:51:50Z

I believe GenomeInfoDbData is a dependency of one of the other bioc packages.

Ok. {renv} should sort these ones out, but adding should not hurt. I can reproduce the issue with https://github.com/lorenzwalthert/precommit.507. Seems like the problem is that we have options(pkgType = "binary") set in the Python pre-commit source code. That means that packages for which no binaries are available seem not installable... This means a new release of pre-commit Python would be required to allow your use case.

I note that implementing #494 would also solve your problem, which is the better solution anyways. I suggest you deactivate the roxygenise hook for now. If you are interested in contributint to #494 (should be mostly cp for the existing roxygenise hook), let me know.

lorenzwalthert · 2023-08-16T10:53:39Z

I note that the solution outlined in the wiki also wont' work for me, i.e. installing the package with {renv} in your global environment and depend on the cache.

averissimo · 2023-08-16T11:40:06Z

Thanks @lorenzwalthert , I've installed a modified pre-commit locally with changes on R options and it works as a current workaround (works on my particular use cose both with install.packages.compile.from.source defined and without).

I'll keep an eye on #494 and I'll write there if I find the time to pick up that issue.

diff --git a/pre_commit/languages/r.py b/pre_commit/languages/r.py
index 6feb065..8a1205b 100644
--- a/pre_commit/languages/r.py
+++ b/pre_commit/languages/r.py
@@ -167,8 +167,7 @@ def _inline_r_setup(code: str) -> str:
     with_option = [
         textwrap.dedent("""\
         options(
-            install.packages.compile.from.source = "never",
-            pkgType = "binary"
+          install.packages.compile.from.source = "never"
         )
         """),
         code,

averissimo closed this as completed Aug 16, 2023

averissimo mentioned this issue Aug 17, 2023

only hard deps in pre-commit - roxygenize - additional deps insightsengineering/teal#891

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`roxygenize` cannot find Bioconductor pkg dependency when setting environment #507

`roxygenize` cannot find Bioconductor pkg dependency when setting environment #507

averissimo commented Aug 16, 2023

version information

error information

lorenzwalthert commented Aug 16, 2023

lorenzwalthert commented Aug 16, 2023 •

edited

Loading

averissimo commented Aug 16, 2023 •

edited

Loading

averissimo commented Aug 16, 2023 •

edited

Loading

averissimo commented Aug 16, 2023 •

edited

Loading

version information

error information

lorenzwalthert commented Aug 16, 2023 •

edited

Loading

lorenzwalthert commented Aug 16, 2023

averissimo commented Aug 16, 2023

roxygenize cannot find Bioconductor pkg dependency when setting environment #507

roxygenize cannot find Bioconductor pkg dependency when setting environment #507

Comments

averissimo commented Aug 16, 2023

version information

error information

lorenzwalthert commented Aug 16, 2023

lorenzwalthert commented Aug 16, 2023 • edited Loading

averissimo commented Aug 16, 2023 • edited Loading

averissimo commented Aug 16, 2023 • edited Loading

averissimo commented Aug 16, 2023 • edited Loading

version information

error information

lorenzwalthert commented Aug 16, 2023 • edited Loading

lorenzwalthert commented Aug 16, 2023

averissimo commented Aug 16, 2023

`roxygenize` cannot find Bioconductor pkg dependency when setting environment #507

`roxygenize` cannot find Bioconductor pkg dependency when setting environment #507

lorenzwalthert commented Aug 16, 2023 •

edited

Loading

averissimo commented Aug 16, 2023 •

edited

Loading

averissimo commented Aug 16, 2023 •

edited

Loading

averissimo commented Aug 16, 2023 •

edited

Loading

lorenzwalthert commented Aug 16, 2023 •

edited

Loading