Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the app deployments #41

Merged
merged 124 commits into from
Oct 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
5f82eb3
remove formatters
m7pr Jul 18, 2023
9a41c12
feat: update the deploy workflow to test deployment for `exploratory`…
vedhav Jul 25, 2023
5475cc4
fix: using the release version of R instead of relying on lockfile
vedhav Jul 25, 2023
ef89aaf
fix: provide reproducible package versions and remove renv from explo…
vedhav Jul 25, 2023
5f703f4
feat: deploy all the apps and remove the renv from all the sub-direct…
vedhav Jul 26, 2023
5a0372e
feat: adding front end tests for two apps
vedhav Jul 27, 2023
c249587
feat: adding deployments for main and release versions
vedhav Jul 27, 2023
13f637f
fix: increase the timeout so the elements are loaded in time
vedhav Jul 28, 2023
d293cda
feat: adding front end tests for all the other apps
vedhav Jul 28, 2023
d6cf957
fix: adding a system dependency for python virtual env
vedhav Jul 28, 2023
fac5b07
fix: increase the startup timeout
vedhav Jul 28, 2023
23bc8f2
chore: validate the correct number of tabs in the safety app
vedhav Jul 28, 2023
0c262e6
merge main
m7pr Jul 28, 2023
c4a43b9
teal.widgets -> teal.data
m7pr Jul 28, 2023
d94a288
formatters_var to col_
m7pr Jul 31, 2023
d41dcba
Merge remote-tracking branch 'origin/main' into deploy-apps@main
Jul 31, 2023
24ec23d
Merge branch 'main' into 447_remove_formatters@main
m7pr Jul 31, 2023
af17610
revert: remove front-end tests for python and early-dev apps
Jul 31, 2023
dcf98fb
Merge remote-tracking branch 'origin/447_remove_formatters@main' into…
Jul 31, 2023
69d9351
fix: explicitly deploy the app.R file
vedhav Aug 1, 2023
0a52bb1
feat: update source script to install dependencies from r-universe
vedhav Aug 1, 2023
23a55df
docs: update the readme files with new install instructions
vedhav Aug 1, 2023
f93412f
fix: provide the cdisc data
vedhav Aug 1, 2023
3fe4510
Merge branch 'main' into deploy-apps@main
vedhav Aug 1, 2023
acb8c40
fix: also install and update the package dependencies
vedhav Aug 4, 2023
0ff4830
Merge branch 'main' into deploy-apps@main
vedhav Aug 4, 2023
047259b
Merge branch 'main' into deploy-apps@main
vedhav Aug 4, 2023
561138d
feat: using the same repos for package dependencies
vedhav Aug 7, 2023
1c728bd
chore: remove unwanted function arguments
vedhav Aug 7, 2023
75b3984
chore: explicitly mention the dependency directory
vedhav Aug 7, 2023
0bee950
feat: restore renv for basic-teal app during deployment
vedhav Aug 9, 2023
1e55370
fix: update teal.widgets
vedhav Aug 9, 2023
f4547ce
fix: update teal.slice
vedhav Aug 9, 2023
62687bb
fix: set repo options right before deployment
vedhav Aug 9, 2023
5820a75
fix: deploy app with working directory
vedhav Aug 9, 2023
0045fbc
chore: move install rsconnect step within deployment step
vedhav Aug 9, 2023
56a5adf
[skip deploy] Update renv.lock file for basic-teal app
vedhav Aug 9, 2023
451335a
feat: init renv to all the other apps
vedhav Aug 9, 2023
8c42e90
commit renv.lock file only for main apps
vedhav Aug 9, 2023
db6920c
chore: update packages for RNA-seq app
vedhav Aug 9, 2023
9782db5
[skip deploy] Update renv.lock file for basic-teal app
vedhav Aug 9, 2023
23074f7
feat: install package dependencies without renv for release version o…
vedhav Aug 9, 2023
02045f6
feat: pushing the changes using force
vedhav Aug 9, 2023
ec1b475
Revert "feat: pushing the changes using force"
vedhav Aug 9, 2023
6da4cbc
[skip deploy] Update renv.lock file for RNA-seq app
vedhav Aug 9, 2023
7a7b824
chore: rename the deployment env channels
vedhav Aug 16, 2023
ba42867
fix: install R packages without renv for both the channels
vedhav Aug 16, 2023
4cf642f
[skip deploy] Update renv.lock file for basic-teal app
vedhav Aug 16, 2023
cd43aba
feat: updating the renv packages before snapshot
Aug 16, 2023
cdf52f5
Revert "[skip deploy] Update renv.lock file for basic-teal app"
Aug 16, 2023
42d9688
Revert "feat: updating the renv packages before snapshot"
Aug 16, 2023
949eddd
feat: restoring the basic-teal app using NEST packages from GitHub
vedhav Aug 16, 2023
b09a388
Revert "feat: restoring the basic-teal app using NEST packages from G…
Aug 16, 2023
37ae7c5
feat: using renv for non-teal packages and installing teal packages d…
Aug 16, 2023
37a0686
[skip deploy] Update renv.lock file for RNA-seq app
vedhav Aug 16, 2023
2ab953b
feat: using renv for non-teal packages and installing teal packages d…
Aug 16, 2023
92b9a3b
Merge branch 'deploy-apps@main' of github.com:insightsengineering/tea…
Aug 16, 2023
fd50d64
revert: restore the renv cache
vedhav Aug 16, 2023
d5db00f
fix: restore and update from renv
vedhav Aug 17, 2023
7f317a9
fix: setting local repo options
vedhav Aug 17, 2023
1aff1a7
[skip deploy] Update renv.lock file for basic-teal app
vedhav Aug 17, 2023
b9899c1
feat: explicitly update only packages from lock file
vedhav Aug 17, 2023
66dc55d
feat: add a step to update repo channel pre restore
vedhav Aug 17, 2023
86fe4f6
[skip deploy] Update renv.lock file for basic-teal app
vedhav Aug 17, 2023
ec7b86d
feat: include RNA-seq app in deployment
vedhav Aug 17, 2023
0a42325
[skip deploy] Update renv.lock file for RNA-seq app
vedhav Aug 17, 2023
7ba85d0
feat: update renv lock files for all the apps and deploy them
Aug 17, 2023
2429a64
docs: update the readme and website with new stable link
vedhav Aug 17, 2023
a05d5b9
[skip deploy] Update renv.lock file for python app
vedhav Aug 17, 2023
dadca1a
Update README.md
vedhav Aug 17, 2023
111e558
feat: excluding just channel packages from renv + update renv.lock files
Aug 17, 2023
1468fd9
feat: update the sourceme script to work with the renv changes
vedhav Aug 17, 2023
93f97f4
fix: fix typo
vedhav Aug 17, 2023
8d3da0a
[skip deploy] Update renv.lock file for basic-teal app
vedhav Aug 17, 2023
f3b5e24
feat: using GitHub as source and update CI to modify lockfile before …
vedhav Aug 18, 2023
0740922
fix: set the GitHub PAT
vedhav Aug 18, 2023
b9e4ccd
feat: test package installation
vedhav Aug 18, 2023
53598af
feat: auto run CI on all push
vedhav Aug 18, 2023
195d7fd
feat: restore the deploy step
vedhav Aug 18, 2023
ba6305c
chore: checkout the repo without dedicated path
vedhav Aug 18, 2023
0b9228c
feat: update non-github packages
vedhav Aug 23, 2023
f4b9bd0
fix: commit the renv.lock files using just git commands
vedhav Aug 23, 2023
59425a1
[skip deploy] Update renv.lock file for basic-teal app
actions-user Aug 23, 2023
24e7436
feat: deploy all the apps
vedhav Aug 23, 2023
c259e45
feat: update renv.lock files for all the apps with GitHub source
vedhav Aug 23, 2023
6ad8529
chore: update the nestcolor with GitHub source
vedhav Aug 23, 2023
f0229ef
fix: install osprey and teal.osprey from GitHub
Aug 23, 2023
f0109b8
[skip deploy] Update renv.lock file for RNA-seq app
actions-user Aug 23, 2023
ba491ca
[skip deploy] Update renv.lock file for exploratory app
actions-user Aug 23, 2023
2364847
[skip deploy] Update renv.lock file for efficacy app
actions-user Aug 23, 2023
6305b97
[skip deploy] Update renv.lock file for longitudinal app
actions-user Aug 23, 2023
d88fabd
[skip deploy] Update renv.lock file for patient-profile app
actions-user Aug 23, 2023
de5c271
[skip deploy] Update renv.lock file for safety app
actions-user Aug 23, 2023
c8a1047
[skip deploy] Update renv.lock file for early-dev app
actions-user Aug 23, 2023
fe52473
fix: skip auto-commit when renv.lock is not modified
vedhav Aug 28, 2023
ebef894
feat: deploy both stable and dev channels
vedhav Aug 28, 2023
e0c4eb5
docs: trim the comment for restore_and_run
vedhav Aug 28, 2023
0802fb6
fix: fetch remotes before checkout
vedhav Aug 28, 2023
c422452
feat: restore the user's current working directory after app run
vedhav Aug 28, 2023
8b7dc85
chore: remove unused package_repo from source script and readme
vedhav Aug 28, 2023
d409204
docs: update docs about renv and GitHub PAT
Aug 28, 2023
942cc55
fix: always install system dependencies for python and cypress
vedhav Aug 28, 2023
fde85f5
Update .github/workflows/deploy.yaml
vedhav Aug 28, 2023
0783434
Update .github/workflows/deploy.yaml
vedhav Aug 28, 2023
fa1d984
chore: use the proper name term 'owner' instead of 'username'
vedhav Aug 28, 2023
cd274ec
chore: update the npm packages to their latest versions
vedhav Aug 28, 2023
4111f09
chore: migrate the cypress tests to cypress 12
vedhav Aug 28, 2023
b87a8a8
Merge branch 'main' into deploy-apps@main
vedhav Aug 28, 2023
80a01ee
Merge branch 'main' into deploy-apps@main
vedhav Aug 30, 2023
4773e17
chore: deploy apps
vedhav Aug 30, 2023
66c450f
fix: push all directory content so other assets can be accessed
vedhav Aug 30, 2023
3264004
chore: bundle only the app.R and static files for the deployment
vedhav Aug 30, 2023
1f94193
Merge branch 'main' into deploy-apps@main
vedhav Aug 31, 2023
655fd7c
fix: update renv env for the python app
vedhav Sep 1, 2023
d053a82
fix: stash and apply changes before checkout
vedhav Sep 1, 2023
abf44c0
Merge branch 'main' into deploy-apps@main
vedhav Sep 5, 2023
7d27aa0
Merge branch 'main' into deploy-apps@main
donyunardi Sep 15, 2023
3c24cff
Update deploy.yaml
walkowif Sep 22, 2023
c7570c7
Update deploy.yaml
walkowif Sep 22, 2023
980eaa2
Update deploy.yaml
walkowif Sep 22, 2023
c4d9c8d
Fix false-positive deployment status (#73)
vedhav Sep 26, 2023
9ffee72
Merge branch 'main' into deploy-apps@main
vedhav Oct 3, 2023
138ec6b
feat: update the renv.lock file during restore and runing the app loc…
vedhav Oct 3, 2023
878437e
[skip deploy] update branch references
vedhav Oct 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 134 additions & 38 deletions .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,18 @@ name: Deploy Apps 🚀

on:
push:
branches:
- main
tags:
- "*"
workflow_dispatch:
#schedule:
# - cron: '12 3 * * *'
schedule:
- cron: "12 3 * * *"

concurrency:
group: publish-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

env:
SHINYAPPSIO_ACCOUNT: genentech
APP_PREFIX: NEST
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

jobs:
deploy:
Expand All @@ -29,45 +27,143 @@ jobs:
image: ghcr.io/insightsengineering/rstudio_4.3.1_bioc_3.17:latest
if: >
!contains(github.event.commits[0].message, '[skip deploy]')
strategy:
fail-fast: false
matrix:
directory:
[
"RNA-seq",
"basic-teal",
"efficacy",
"exploratory",
"longitudinal",
"early-dev",
"patient-profile",
"python",
"safety",
]
channel: ["stable", "dev"]
steps:
- name: Checkout repo 🛎
uses: actions/checkout@v3

- name: Set channel related constants
run: |
if [ "${{ matrix.channel }}" = "stable" ]; then
echo "BRANCH_NAME=main" >> $GITHUB_ENV # TODO: Change the branch name to main
else
echo "BRANCH_NAME=dev" >> $GITHUB_ENV # TODO: Change the branch name to deval
fi

- name: Check if cypress test exists
id: find-cypress
run: |
if [ -d ${{ matrix.directory }}/tests/cypress ]; then
echo "has-cypress-tests=true" >> $GITHUB_OUTPUT
else
echo "Skipping frontend tests because cypress directory does not exist."
echo "has-cypress-tests=false" >> $GITHUB_OUTPUT
fi

- name: Setup system dependencies for cypress and python app
run: >
apt-get update && apt-get install --yes
libgtk2.0-0 libgbm-dev libnotify-dev libgconf-2-4 xvfb python3.10-venv

- name: Setup Node
uses: actions/setup-node@v3
if: steps.find-cypress.outputs.has-cypress-tests == 'true'
with:
path: ${{ github.event.repository.name }}
node-version: 16

- name: Overwrite the global Rprofile ✍️
- name: Restore renv from cache
uses: actions/cache@v3
env:
CACHE_KEY: renv-${{ runner.arch }}-${{ runner.os }}-${{ matrix.directory }}-${{ matrix.channel }}
with:
path: ${{ matrix.directory }}/renv/library
key: ${{ env.CACHE_KEY }}-${{ hashFiles(format('{0}/renv.lock', matrix.directory)) }}
restore-keys: ${{ env.CACHE_KEY }}-

- name: Update renv.lock file with updated GitHub packages
shell: Rscript {0}
run: |
cat > /usr/local/lib/R/etc/Rprofile.site <<RPROFILE
options(
repos = c(
Pharmaverse = "https://pharmaverse.r-universe.dev",
CRAN = "https://cloud.r-project.org",
BioC = BiocManager::repositories()
),
download.file.method = 'libcurl',
HTTPUserAgent = sprintf(
"R/%s R (%s)",
getRversion(),
paste(getRversion(), R.version["platform"], R.version["arch"], R.version["os"])
)
)
RPROFILE
setwd("${{ matrix.directory }}")
lockfile <- renv::lockfile_read()
pkg_name_structure <- ifelse("${{ matrix.channel }}" == "stable", "%s/%s@*release", "%s/%s")
for (package in lockfile$Packages) {
if (package$Source == "GitHub") {
renv::record(sprintf(pkg_name_structure, package$RemoteUsername, package$Package))
}
}

- name: Deploy apps 🖨 ${{ github.event.repository.name }} 🎨
- name: Install R packages using renv and update the renv snapshot
shell: Rscript {0}
working-directory: ${{ matrix.directory }}
run: |
Sys.setenv(GITHUB_PAT="${{ secrets.GITHUB_TOKEN }}")
devtools::install_local(
".",
upgrade = "never",
dependencies = TRUE
options(renv.config.cache.symlinks = FALSE)
lockfile_pkgs <- renv::lockfile_read()$Package
github_pkgs <- names(lockfile_pkgs)[sapply(lockfile_pkgs, function(x) x$Source == "GitHub")]
renv::restore(clean = TRUE)
renv::update(exclude = github_pkgs)
renv::snapshot()

- name: Print the new renv.lock file for ${{ matrix.directory }}
working-directory: ${{ matrix.directory }}
run: cat renv.lock

- name: Front end test to check if the app works fine
if: steps.find-cypress.outputs.has-cypress-tests == 'true'
continue-on-error: true
uses: cypress-io/github-action@v6
with:
build: npm install cypress --save-dev
working-directory: ${{ matrix.directory }}/tests
start: npm run run-app
wait-on: "http://localhost:3333"
wait-on-timeout: 500

- name: Install deploy R package dependencies
shell: Rscript {0}
working-directory: ${{ matrix.directory }}
run: |
install.packages(c("BiocManager", "rsconnect"))

- name: Deploy 🖨 ${{ matrix.directory }} 🎨
shell: Rscript {0}
working-directory: ${{ matrix.directory }}
run: |
rsconnect::setAccountInfo(
name = "${{ env.SHINYAPPSIO_ACCOUNT }}",
token = "${{ secrets.SHINYAPPSIO_TOKEN }}",
secret = "${{ secrets.SHINYAPPSIO_SECRET }}",
server = "shinyapps.io"
)
teal.gallery::deploy_all_apps(
api_key = "${{ secrets.SHINYAPPSIO_TOKEN }}",
api_secret = "${{ secrets.SHINYAPPSIO_SECRET }}",
title_prefix = "NEST",
title_suffix = "${{ github.ref_name }}",
server = "https://${{ env.SHINYAPPSIO_ACCOUNT }}.shinyapps.io",
create_manifest = TRUE
rsconnect::deployApp(
appFiles = c("app.R"),
appName = rsconnect::generateAppName("${{ env.APP_PREFIX }}_${{ matrix.directory }}_${{ matrix.channel }}"),
appTitle = "${{ env.APP_PREFIX }}_${{ matrix.directory }}_${{ matrix.channel }}",
account = "${{ env.SHINYAPPSIO_ACCOUNT }}",
upload = TRUE,
logLevel = "normal",
lint = FALSE,
forceUpdate = TRUE
)
working-directory: ${{ github.event.repository.name }}
shell: Rscript {0}

- name: Commit updated renv.lock file
run: |
git config --global user.email "[email protected]"
git config --global user.name "GitHub Action"
git config --global --add safe.directory /__w/teal.gallery/teal.gallery
git fetch
git stash
git checkout ${{ env.BRANCH_NAME }}
git stash apply
git pull
if [ -n "$(git diff --staged)" ]; then
git add ${{ matrix.directory }}/renv.lock
git commit -m "[skip deploy] Update renv.lock file for ${{ matrix.directory }} app"
git push origin ${{ env.BRANCH_NAME }}
else
echo "renv.lock was not modified. Nothing to commit.";
fi
43 changes: 28 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,27 +15,34 @@ A gallery of sample apps based on the [`teal`](https://github.com/insightsengine

### Development version

- [basic-teal](https://genentech.shinyapps.io/NEST_basic-teal_main/)
- [exploratory](https://genentech.shinyapps.io/NEST_exploratory_main/)
- [safety](https://genentech.shinyapps.io/NEST_safety_main/)
- [efficacy](https://genentech.shinyapps.io/NEST_efficacy_main/)
- [patient-profile](https://genentech.shinyapps.io/NEST_patient-profile_main/)
- [early-dev](https://genentech.shinyapps.io/NEST_early-dev_main/)
- [longitudinal](https://genentech.shinyapps.io/NEST_longitudinal_main/)
- [RNA-seq](https://genentech.shinyapps.io/NEST_RNA-seq_main/)
- [python](https://genentech.shinyapps.io/NEST_python_main/)
- [basic-teal](https://genentech.shinyapps.io/NEST_basic-teal_stable/)
- [exploratory](https://genentech.shinyapps.io/NEST_exploratory_stable/)
- [safety](https://genentech.shinyapps.io/NEST_safety_stable/)
- [efficacy](https://genentech.shinyapps.io/NEST_efficacy_stable/)
- [patient-profile](https://genentech.shinyapps.io/NEST_patient-profile_stable/)
- [early-dev](https://genentech.shinyapps.io/NEST_early-dev_stable/)
- [longitudinal](https://genentech.shinyapps.io/NEST_longitudinal_stable/)
- [RNA-seq](https://genentech.shinyapps.io/NEST_RNA-seq_stable/)
- [python](https://genentech.shinyapps.io/NEST_python_stable/)

## Running the apps

You can run any of these apps by just executing these two lines of code in your R console.

By sourcing the [sourceme.R](https://github.com/insightsengineering/teal.gallery/blob/main/utils/sourceme.R) file you make sure that you have access to the `restore_and_run()`

Note: Make sure to install [renv](https://rstudio.github.io/renv/index.html) before you do this. Also, it is recommended that you create and use a Github PAT. Once you have the PAT, run the following:

```R
Sys.setenv(GITHUB_PAT = "your_access_token_here")
```

Running the `restore_and_run("APP_NAME")` will run the `APP_NAME` by restoring the packages using [renv](https://rstudio.github.io/renv/)

```R
source("https://raw.github.com/insightsengineering/teal.gallery/main/_internal/utils/sourceme.R")
# Assuming you want to run the "basic-teal" app. Refer to the list above to know which apps are possible

# Run the basic-teal app
restore_and_run("basic-teal")
```

Expand All @@ -47,8 +54,14 @@ All `teal` sample apps are wrapped into this package for the sake of portability

Adding a sample app involves the following steps:

1. Copy each sample app into a file named `app.R` into a sensible folder name inside it's own directory. The directory is also the `APP_NAME`.
2. Make sure that `{renv}` is used for the teal app.
3. Create a GIF recording ([KAP](https://getkap.co/) is a good tool for this). Make sure that the dimensions of the GIF is 970x555 px and the size is about 1 MB. (It can be done by recording using KAP in 1470x840 px and rendering 5fps and downsizing 33%). Place the GIF inside the `_internal/quarto/assets/img` directory. Also, make sure that the name of the GIF is `APP_NAME.gif`. Also, make sure to place a static image with the name dimention called `APP_NAME.png` that will be displayed in the demo page when the card is not hovered.
4. Update the `_internal/quarto/demo-apps.yml` with a new `app/title`. This should be the `APP_NAME`.
5. Run the `_internal/utils/generate_app_readme.R` to generate the readme for the app inside it's own directory.
#### Hard requirements

1. Copy each sample app into a file named `app.R` into a sensible folder name inside it's own directory. The directory has to be named with the `APP_NAME`.
2. Update the `_internal/quarto/demo-apps.yml` with a new `app/title`. This should be the `APP_NAME`.
3. Run the `_internal/utils/generate_app_readme.R` to generate the readme for the app inside it's own directory.

#### Optional/Soft requirements

1. You can snapshot the teal app dependencies using `{renv}` but make sure to snapshot using GitHub references to the teal packages.
2. Create a GIF recording ([KAP](https://getkap.co/) is a good tool for this). Make sure that the dimensions of the GIF is 970x555 px and the size is about 1 MB. (It can be done by recording using KAP in 1470x840 px and rendering 5fps and downsizing 33%). Place the GIF inside the `_internal/quarto/assets/img` directory. Also, make sure that the name of the GIF is `APP_NAME.gif`. Also, make sure to place a static image with the name dimention called `APP_NAME.png` that will be displayed in the demo page when the card is not hovered.
3. Add front-end tests with the help of cypress. Copy the contents of the `js` directory within some other app's directory inside your app directory to get the node dependencies. Place the cypress tests inside the `tests/cypress` inside your app's directory. Please refer to an existing app's tests so that the `.github/deploy.yaml` will automaticall run the cypress tests.
6 changes: 4 additions & 2 deletions RNA-seq/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@
### Run the app yourself

source("https://raw.github.com/insightsengineering/teal.gallery/main/_internal/utils/sourceme.R")
restore_and_run("RNA-seq")

# Run the app
restore_and_run("RNA-seq", package_repo = "https://insightsengineering.r-universe.dev")

### View the deployed app

Deployed app: <https://genentech.shinyapps.io/NEST_RNA-seq_main>
Deployed app: <https://genentech.shinyapps.io/NEST_RNA-seq_stable>

### Preview the app

Expand Down
Loading