Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerfile Cleaning #60

Merged
merged 29 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
16d7406
Dockerfile Cleaning
AlexAxthelm May 21, 2024
94be280
Correctly place envvar
AlexAxthelm May 21, 2024
3a98866
Update Static Labels
AlexAxthelm May 22, 2024
17a2bb1
Use Fully qualified base image
AlexAxthelm May 22, 2024
ac7c247
Remove empty Docker LABELs
AlexAxthelm May 22, 2024
0819291
Merge branch 'main' into refactor/55-restructure-dockerfile
AlexAxthelm May 22, 2024
30ce652
Use improved docker labels
AlexAxthelm May 22, 2024
8674fa9
Trigger CI
AlexAxthelm May 22, 2024
b7b9db3
Trigger CI
AlexAxthelm May 22, 2024
3af9ce2
Trigger CI
AlexAxthelm May 22, 2024
35db556
Trigger CI
AlexAxthelm May 22, 2024
bf6485e
Trigger CI
AlexAxthelm May 22, 2024
c57eb5a
Trigger CI
AlexAxthelm May 22, 2024
d118b8a
Use single dockerfile for multiple CRAN-ish source
AlexAxthelm May 22, 2024
58b874a
Further cleaning to setting Rprofile
AlexAxthelm May 22, 2024
cfcf5a7
Fix syntax error, allow loading `Rprofile.site`
AlexAxthelm May 22, 2024
1ef39d8
Suppress unneeded message from `pak`
AlexAxthelm May 22, 2024
1471499
Don't use `root` as user
AlexAxthelm May 22, 2024
1c887bf
Fix typo in Rprofile.site
AlexAxthelm May 22, 2024
7667576
Fix for non-root user
AlexAxthelm May 22, 2024
4a236d0
Rearrange
AlexAxthelm May 22, 2024
94ca006
Use absolute path in config
AlexAxthelm May 23, 2024
f947d21
Change portfolio paths for new working dir
AlexAxthelm May 23, 2024
40fb7aa
Update output path
AlexAxthelm May 23, 2024
d0b34fd
Put outputs in user dir
AlexAxthelm May 23, 2024
04cacaf
use main branch of docker action
AlexAxthelm May 23, 2024
fe1e187
Update bind mount in testing script
AlexAxthelm May 23, 2024
388b728
Change permissions for output dir
AlexAxthelm May 23, 2024
5514c02
Fix typo
AlexAxthelm May 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,11 +82,12 @@ jobs:
WORKSPACE: ${{ github.workspace }}
PARAMETERS: ${{ steps.prepare.outputs.parameters }}
run: |
chmod -R 777 "${WORKSPACE}/${OUTPUT_DIR}"
AlexAxthelm marked this conversation as resolved.
Show resolved Hide resolved
docker run \
--network none \
--env LOG_LEVEL=DEBUG \
--mount type=bind,readonly,source=${WORKSPACE}/${PACTA_DATA_DIR},target=/pacta-data \
--mount type=bind,source=${WORKSPACE}/${OUTPUT_DIR},target=/output_dir \
--mount type=bind,source=${WORKSPACE}/${OUTPUT_DIR},target=/home/workflow-pacta/output_dir \
--mount type=bind,source=${WORKSPACE}/${INPUT_DIR},target=/input_dir \
$FULL_IMAGE_NAME \
$PARAMETERS
Expand Down
86 changes: 39 additions & 47 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,30 +1,20 @@
# using rocker r-vers as a base with R 4.3.1
# https://hub.docker.com/r/rocker/r-ver
# https://rocker-project.org/images/versioned/r-ver.html

# set proper base image
ARG R_VERS="4.3.1"
FROM rocker/r-ver:$R_VERS AS base
FROM docker.io/rocker/r-ver:4.3.1 AS base

# set Docker image labels
LABEL org.opencontainers.image.source=https://github.com/RMI-PACTA/workflow.pacta
LABEL org.opencontainers.image.description="Docker image to run PACTA"
LABEL org.opencontainers.image.description="Docker image to run PACTA analysis"
LABEL org.opencontainers.image.licenses=MIT
LABEL org.opencontainers.image.title=""
LABEL org.opencontainers.image.revision=""
LABEL org.opencontainers.image.version=""
LABEL org.opencontainers.image.vendor=""
LABEL org.opencontainers.image.base.name=""
LABEL org.opencontainers.image.ref.name=""
LABEL org.opencontainers.image.authors=""
LABEL org.opencontainers.image.title="workflow.pacta"
LABEL org.opencontainers.image.vendor="RMI"
LABEL org.opencontainers.image.base.name="docker.io/rocker/r-ver:4.3.1"
LABEL org.opencontainers.image.authors="Alex Axthelm, CJ Yetman, Jackson Hoffart"

AlexAxthelm marked this conversation as resolved.
Show resolved Hide resolved
# set apt-get to noninteractive mode
ARG DEBIAN_FRONTEND="noninteractive"
ARG DEBCONF_NOWARNINGS="yes"

AlexAxthelm marked this conversation as resolved.
Show resolved Hide resolved
# install system dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
&& DEBIAN_FRONTEND="noninteractive" \
apt-get install -y --no-install-recommends \
git=1:2.34.* \
libcurl4-openssl-dev=7.81.* \
libgit2-dev=1.1.* \
Expand All @@ -35,52 +25,54 @@ RUN apt-get update \
&& chmod -R a+rwX /root \
&& rm -rf /var/lib/apt/lists/*

# set frozen CRAN repo
ARG CRAN_REPO="https://packagemanager.posit.co/cran/2023-10-30"
RUN echo "options(repos = c(CRAN = '$CRAN_REPO'), pkg.sysreqs = FALSE)" >> "${R_HOME}/etc/Rprofile.site"
# set frozen CRAN repo and RProfile.site
# This block makes use of the builtin ARG $TARGETPLATFORM (See:
# https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
# ) to pick the correct CRAN-like repo, which will let us target binaries fo
# supported platforms
ARG TARGETPLATFORM
RUN PACKAGE_PIN_DATE="2023-10-30" && \
echo "TARGETPLATFORM: $TARGETPLATFORM" && \
if [ "$TARGETPLATFORM" = "linux/amd64" ] && grep -q -e "Jammy Jellyfish" "/etc/os-release" ; then \
CRAN_LIKE_URL="https://packagemanager.posit.co/cran/__linux__/jammy/$PACKAGE_PIN_DATE"; \
else \
CRAN_LIKE_URL="https://packagemanager.posit.co/cran/$PACKAGE_PIN_DATE"; \
fi && \
echo "CRAN_LIKE_URL: $CRAN_LIKE_URL" && \
printf "options(\n \
repos = c(CRAN = '%s'),\n \
pak.no_extra_messages = TRUE,\n \
pkg.sysreqs = FALSE,\n \
pkg.sysreqs_db_update = FALSE,\n \
pkg.sysreqs_update = FALSE\n \
)\n" \
"$CRAN_LIKE_URL" \
> "${R_HOME}/etc/Rprofile.site"

# copy in DESCRIPTION from this repo
COPY DESCRIPTION /DESCRIPTION
COPY DESCRIPTION /workflow.pacta/DESCRIPTION

# install pak, find dependencises from DESCRIPTION, and install them.
RUN Rscript -e "\
install.packages('pak'); \
deps <- pak::local_deps(root = '.'); \
deps <- pak::local_deps(root = '/workflow.pacta'); \
pkg_deps <- deps[!deps[['direct']], 'ref']; \
print(pkg_deps); \
pak::pak(pkg_deps); \
"

FROM base AS install-pacta

# PACTA R package tags
ARG allocate_tag="/tree/main"
ARG audit_tag="/tree/main"
ARG import_tag="/tree/main"
ARG utils_tag="/tree/main"

ARG allocate_url="https://github.com/rmi-pacta/pacta.portfolio.allocate"
ARG audit_url="https://github.com/rmi-pacta/pacta.portfolio.audit"
ARG import_url="https://github.com/rmi-pacta/pacta.portfolio.import"
ARG utils_url="https://github.com/rmi-pacta/pacta.portfolio.utils"

# install R package dependencies
RUN Rscript -e "\
gh_pkgs <- \
c( \
paste0('$allocate_url', '$allocate_tag'), \
paste0('$audit_url', '$audit_tag'), \
paste0('$import_url', '$import_tag'), \
paste0('$utils_url', '$utils_tag') \
); \
print(gh_pkgs); \
pak::pak(c(gh_pkgs)); \
"

COPY . /workflow.pacta/

RUN Rscript -e "pak::local_install(root = '/workflow.pacta')"

# set default run behavior
ENTRYPOINT ["Rscript", "--vanilla", "/workflow.pacta/inst/extdata/scripts/run_pacta.R"]
CMD ["/workflow.pacta/input_dir/default_config.json"]

# Create and use non-root user
RUN useradd -m -s /bin/bash workflow-pacta
USER workflow-pacta
WORKDIR /home/workflow-pacta

2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ services:
read_only: true
- type: bind
source: ${output_dir:-./output_dir}
target: /output_dir
target: /home/workflow-pacta/output_dir
read_only: false
- type: bind
source: ${input_dir:-./input_dir}
Expand Down
6 changes: 3 additions & 3 deletions input_dir/default_config.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"data_dir": "../pacta-data/2023Q4",
"portfolio_path": "input_dir/default_portfolio.csv",
"output_dir": "output_dir",
"data_dir": "/pacta-data/2023Q4",
"portfolio_path": "/workflow.pacta/input_dir/default_portfolio.csv",
"output_dir": "/home/workflow-pacta/output_dir",
AlexAxthelm marked this conversation as resolved.
Show resolved Hide resolved
"start_year": 2023,
"equity_market_list": ["GlobalMarket", "DevelopedMarket", "EmergingMarket"],
"scenario_sources_list": ["GECO2023", "ISF2023", "WEO2023"],
Expand Down
6 changes: 3 additions & 3 deletions tests/config/default_2022Q4.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
"holdingsDate": "2022Q4",
"pactaDataURL": "https://pactadatadev.file.core.windows.net/workflow-data-preparation-outputs/2022Q4_20240426T113151Z",
"parameters": {
"data_dir": "../pacta-data/2022Q4",
"portfolio_path": "input_dir/default_portfolio.csv",
"output_dir": "output_dir",
"data_dir": "/pacta-data/2022Q4",
"portfolio_path": "/workflow.pacta/input_dir/default_portfolio.csv",
"output_dir": "/home/workflow-pacta/output_dir",
"start_year": 2022,
"equity_market_list": ["GlobalMarket", "DevelopedMarket", "EmergingMarket"],
"scenario_sources_list": ["GECO2022", "ISF2021", "WEO2022"],
Expand Down
Loading