Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add devel image #165

Merged
merged 41 commits into from
Nov 8, 2023
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
1bf5f11
add checks for devel
pharmaverse-bot Oct 24, 2023
6097514
fix branch name
pharmaverse-bot Oct 24, 2023
1236e00
update push job
pharmaverse-bot Oct 24, 2023
80516da
re-test
pharmaverse-bot Oct 24, 2023
e79372c
fix checks
pharmaverse-bot Oct 25, 2023
eef1305
fix typo
pharmaverse-bot Oct 25, 2023
506ff15
fix
pharmaverse-bot Oct 25, 2023
2ce17db
fix
pharmaverse-bot Oct 25, 2023
a5b6036
fix
pharmaverse-bot Oct 25, 2023
40a30b0
install rcmdcheck for devel im
pharmaverse-bot Oct 25, 2023
1dc1389
templates: retrieve temp folder
pharmaverse-bot Oct 27, 2023
aad06e3
add checks on existing branches - templates
pharmaverse-bot Oct 27, 2023
06ce9fc
debug traces
pharmaverse-bot Oct 27, 2023
466e5fa
fix cp cmd
pharmaverse-bot Oct 27, 2023
91acb1d
fix
pharmaverse-bot Oct 27, 2023
d0e2ad9
debug traces
pharmaverse-bot Oct 27, 2023
1f82e55
add other debug traces
pharmaverse-bot Oct 27, 2023
9b0a4b6
update
pharmaverse-bot Oct 27, 2023
d3011fe
add other debug traces
pharmaverse-bot Oct 27, 2023
6445514
test other folder
pharmaverse-bot Oct 27, 2023
b2c7ab6
fix condition branch
pharmaverse-bot Oct 27, 2023
0c23a4a
Merge branch 'main' into add_devel_image
pharmaverse-bot Nov 7, 2023
d34d1ea
fix checks
pharmaverse-bot Nov 7, 2023
40aa7d3
test checks on maintenance versions
pharmaverse-bot Nov 7, 2023
31f569e
fix
pharmaverse-bot Nov 7, 2023
c43f3f4
trigger checks
pharmaverse-bot Nov 7, 2023
9f978eb
try ignore note
pharmaverse-bot Nov 7, 2023
f93f2b1
retry
pharmaverse-bot Nov 7, 2023
5148dad
try another regexp
pharmaverse-bot Nov 7, 2023
a12985d
retry
pharmaverse-bot Nov 7, 2023
82e47cd
rework rcmdcheck
pharmaverse-bot Nov 7, 2023
8e817e1
fix
pharmaverse-bot Nov 7, 2023
381d596
fix
pharmaverse-bot Nov 7, 2023
19cfc99
fix
pharmaverse-bot Nov 7, 2023
b272317
fix
pharmaverse-bot Nov 7, 2023
b6a0f89
fix
pharmaverse-bot Nov 7, 2023
889d1f9
cleanup
pharmaverse-bot Nov 7, 2023
c021700
change r-im-creator tag
pharmaverse-bot Nov 7, 2023
e7c2913
Update check-templates.yml
dgrassellyb Nov 8, 2023
ea523c7
add cron job
pharmaverse-bot Nov 8, 2023
829a8f1
Merge branch 'add_devel_image' of github.com:pharmaverse/admiralci in…
pharmaverse-bot Nov 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@
^admiralci.*\.tgz$
^staged_dependencies.yaml$
^README.Rmd$
^.devcontainer
^.devcontainer
13 changes: 9 additions & 4 deletions .github/workflows/check-templates.yml
Original file line number Diff line number Diff line change
Expand Up @@ -365,13 +365,18 @@ jobs:
id: branch
run: |
repo_name=$(basename "${{ github.repository }}")
echo "source-branch=${{ env.source-branch }}_${repo_name}@devel" >> $GITHUB_OUTPUT
source_br="${{ env.source-branch }}_${repo_name}@devel"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cicdguy : since this PR includes several changes I just add some comments ! here I added this change for pharmaverseadam datasets creation (since we push updates to a new branch, I just make a check before to see if the branch exists - if not we do not trigger the "commit and push changes" step (then we avoid error cases that we often have since pharmaverseadam maintainers might sometimes forget to merge some pending PRs..)
I also thought creating unique branch but would be super messy I guess

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call.

BTW you do not need the @devel suffix anymore for the branch names since everything now uses main as the default branch

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes indeed you're right :) just from some old runs I guess I just forgot to delete it 🙈

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(done)

source_grep=$(git branch -a | grep $source_br)
if [ -z $source_grep ]; then branch_exists=false; else branch_exists=true; fi
echo "source-branch=$source_br" >> $GITHUB_OUTPUT
echo "branch-exists=$branch_exists" >> $GITHUB_OUTPUT
warn_msg="WARNING:: the branch $source_br already exists in pharmaverseadam, please merge it and delete it, and re-execute this job"
if [ '${{ steps.changes.outputs.diff }}' == 'true' ] && [ '$branch_exists' == 'true' ]; then echo "$warn_msg"; fi
shell: bash {0}


# if diff detected, push changes
- name: Commit and push changes in {{ env.repo }}
if: ${{ inputs.push-templates-data && steps.changes.outputs.diff == 'true' }}
if: ${{ inputs.push-templates-data && steps.changes.outputs.diff == 'true' && steps.branch.outputs.branch-exists == 'false' }}
uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_message: templates data updates from ${{ github.repository }}
Expand All @@ -382,7 +387,7 @@ jobs:
create_branch: true

- name: Create Pull Request
if: ${{ inputs.push-templates-data && steps.changes.outputs.diff == 'true' }}
if: ${{ inputs.push-templates-data && steps.changes.outputs.diff == 'true' && steps.branch.outputs.branch-exists == 'false' }}
uses: actions/github-script@v6
with:
github-token: ${{ secrets.PHARMAVERSE_BOT }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/propagate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ on:
- renv/**
- .Rprofile
- .devcontainer/*
workflow_dispatch: # give possibility to run it manually

jobs:
integrity:
Expand Down
40 changes: 38 additions & 2 deletions .github/workflows/push-docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,10 @@ jobs:
run: |
package_name=$(grep "Package:" DESCRIPTION | awk '{print $NF}')
r_version=$(basename $(dirname ${{ matrix.value }})) # TODO: check here if legacy renv (if yes, delete _legacy suffix and add _legacy suffix to docker im?)
echo "r_version=$r_version" >> $GITHUB_OUTPUT
echo "image_name=$package_name-$r_version" >> $GITHUB_OUTPUT
renv=$(echo "${{ matrix.value }}" | tr -d '"')
echo "renv_lock_path=$renv" >> $GITHUB_OUTPUT
echo "r_version=$r_version" >> $GITHUB_OUTPUT
echo "image_name=$package_name-$r_version" >> $GITHUB_OUTPUT

- name: Call deploy docker image action for every renv
uses: insightsengineering/r-image-creator@v1
Expand All @@ -94,5 +94,41 @@ jobs:
repo-user: ${{ github.actor }} # pharmaverse-bot
repo-token: "${{ secrets.GITHUB_TOKEN }}" # ${{ secrets.PHARMAVERSE_BOT }}

deploy-image-devel:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add step to deploy devel image

runs-on: ubuntu-latest

# Token permissions
permissions:
contents: read
packages: write

steps:
- name: Checkout repo
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Set image specs
id: image_specs
run: |
package_name=$(grep "Package:" DESCRIPTION | awk '{print $NF}')
echo "image_name=$package_name-devel" >> $GITHUB_OUTPUT

- name: Call deploy docker image action devel
uses: insightsengineering/r-image-creator@v1
with:
image-name: "${{ steps.image_specs.outputs.image_name }}"
tag-latest: true
base-image: "rocker/rstudio:devel"
sysdeps: qpdf,libxt-dev,curl,npm,libicu-dev,libcurl4-openssl-dev,libssl-dev,make,zlib1g-dev,libfontconfig1-dev,libfreetype6-dev,libfribidi-dev,libharfbuzz-dev,libjpeg-dev,libpng-dev,libtiff-dev,pandoc,libxml2-dev,libgit2-dev,libgit2-dev,jq
description-file: "./DESCRIPTION"
repository-owner: ${{ github.repository_owner }}
repo-user: ${{ github.actor }} # pharmaverse-bot
repo-token: "${{ secrets.GITHUB_TOKEN }}" # ${{ secrets.PHARMAVERSE_BOT }}


# refacto todo: image-name, sysdeps on get-renv-list job


# note: in case of 403 error when pushing to ghcr : link current repo to the given package registry - https://github.com/docker/build-push-action/issues/687
# (got to https://github.com/<user name>?tab=packages to go to packages settings) and there https://github.com/users/<user name>/packages/container/admiralci-4.0/settings
27 changes: 25 additions & 2 deletions .github/workflows/r-cmd-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ jobs:
- {r: '4.1', repos: 'https://packagemanager.posit.co/cran/2021-05-03/'}
- {r: '4.2', repos: 'https://packagemanager.posit.co/cran/2022-01-03/'}
- {r: '4.3', repos: 'https://packagemanager.posit.co/cran/__linux__/focal/latest'}
- {r: 'devel', repos: 'https://packagemanager.posit.co/cran/__linux__/focal/latest'} # not needed for devel ?

env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
Expand Down Expand Up @@ -81,11 +82,33 @@ jobs:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
#####################c#####################

- name: Check Version
id: check_version
run: |
maintenance_version="F"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here this is a step to check if version has this pattern x.y.x.m with m >= 9000 - if this is the case then on next step we are doing this : Sys.setenv("R_CHECK_CRAN_INCOMING_SKIP_LARGE_VERSION" = TRUE) to avoid having notes on rcmdchecks

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend just setting this as a top-level environment var instead of a session-specific var.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah sure? (because I see this more like temp setup to just enable checks in case of dev version) - but np to change this !

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, never mind. Your implementation should work. Keep it as-is.

description_dat <- readLines("DESCRIPTION")
for (i in seq_along(description_dat)) {
if (grepl("^Version:", description_dat[i])) {
current_version <- sub("^Version: ", "", description_dat[i])
version_parts <- strsplit(current_version, "\\.")[[1]]
# check if maintenance version - if yes, temp update the DESCRIPTION for rcmdchecks
if (length(version_parts) == 4) {
print("Maintenance version detected (format X.Y.Z.M with M >= 9000)")
maintenance_version="T"
}
}
}
cat(sprintf("maintenance_version=%s", maintenance_version), file = Sys.getenv("GITHUB_OUTPUT"), append = TRUE)
shell: Rscript {0}

- name: Check
env:
_R_CHECK_CRAN_INCOMING_REMOTE_: false
_R_CHECK_FORCE_SUGGESTS_: false
run: |
if ("${{steps.check_version.outputs.maintenance_version}}" == "T"){
Sys.setenv("_R_CHECK_CRAN_INCOMING_SKIP_LARGE_VERSION_" = TRUE)
}
if (!requireNamespace("rcmdcheck", quietly = TRUE)) install.packages("rcmdcheck")
options(crayon.enabled = TRUE)
check_error_on <- "${{ inputs.error-on }}"
Expand All @@ -103,5 +126,5 @@ jobs:
if: failure()
uses: actions/upload-artifact@main
with:
name: ${{ runner.os }}-r${{ matrix.config.r }}-results
path: check
name: r${{ matrix.config.r }}-results
path: check
Loading