Skip to content

Commit

Permalink
CI updates (ThummeTo#146)
Browse files Browse the repository at this point in the history 
* set julia 1.10 for exmples

* Added checks to Example action (ThummeTo#144)

* Update Example.yml

Added check for success of jupyter examples, fail action if example-building fails, prevents autocommit to examples branch

* relaxed compats

Update Project.toml

---------

Co-authored-by: ThummeTo <[email protected]>

* relaxed Diffeq version

for testing

* updated dependent actions to latest

* Revert "updated dependent actions to latest"

This reverts commit 871eade.

* updated dependency action versions

* fixes for artifact upload

see breaking changes of actions/upload-artifact@v4

* changed from artifacts to branch for gathering

due to changes in artifact behavior

* added retry on merge conflict

* fixes for example action

* Update Example.yml

* Update Example.yml

* Update Project.toml

* force pkg to update in CI

* debug checks

* makeshift versioning fix

* fix for makeshift fix

* Update Example.yml

dont use "latest" in workflows

* Update Eval.yml

dont use "latest" in workflows

* Update TestLTS.yml

dont use "latest" in workflows

* Update TestLatest.yml

dont use "latest" in workflows

---------

Co-authored-by: ThummeTo <[email protected]>
  • Loading branch information
@0815Creeper @ThummeTo
0815Creeper and ThummeTo authored Sep 5, 2024
1 parent f1db228 commit 322f028
Show file tree
Hide file tree
Showing 6 changed files with 173 additions and 188 deletions.
112 changes: 56 additions & 56 deletions .github/workflows/Documentation.yml
View file
Original file line number Diff line number Diff line change
@@ -1,56 +1,56 @@
name: Documentation

on:
# manual trigger
workflow_dispatch:
# trigger from an other action
repository_dispatch:
types: [trigger-docu]

pull_request:
push:
branches:
- main
paths:
- 'docs/**'
- 'README.md'
- '.github/workflows/Documentation.yml'
tags:
- '*'

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: "Check out repository"
uses: actions/checkout@v3
with:
ref: ''

- name: "Set up Julia"
uses: julia-actions/setup-julia@v1
with:
version: '1' # 1.6
arch: x64

- name: "Copy examples, readme"
env:
WORK_TREE: './docs/src/examples'
SRC_BRANCH: 'origin/examples'
SRC_FOLDER_PATH: 'examples'
# copy the examples from the examples branch (.md, .svg) and copy the readme in the docs folder
run: |
git fetch
git --work-tree=$WORK_TREE/.. checkout $SRC_BRANCH -- $SRC_FOLDER_PATH/jupyter-src/*.md $SRC_FOLDER_PATH/jupyter-src/*.svg $SRC_FOLDER_PATH/jupyter-src/*.gif $SRC_FOLDER_PATH/pluto-src/*.html
mv $WORK_TREE/jupyter-src/* $WORK_TREE
rm -r $WORK_TREE/jupyter-src
cp ./README.md ./docs/src/index.md
- name: "Install dependencies"
run: julia --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate()'

- name: "Build and deploy"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # For authentication with GitHub Actions token
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }} # For authentication with SSH deploy key
run: julia --project=docs/ docs/make.jl
name: Documentation

on:
# manual trigger
workflow_dispatch:
# trigger from an other action
repository_dispatch:
types: [trigger-docu]

pull_request:
push:
branches:
- main
paths:
- 'docs/**'
- 'README.md'
- '.github/workflows/Documentation.yml'
tags:
- '*'

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: "Check out repository"
uses: actions/checkout@v4
with:
ref: ''

- name: "Set up Julia"
uses: julia-actions/setup-julia@latest
with:
version: '1' # 1.6
arch: x64

- name: "Copy examples, readme"
env:
WORK_TREE: './docs/src/examples'
SRC_BRANCH: 'origin/examples'
SRC_FOLDER_PATH: 'examples'
# copy the examples from the examples branch (.md, .svg) and copy the readme in the docs folder
run: |
git fetch
git --work-tree=$WORK_TREE/.. checkout $SRC_BRANCH -- $SRC_FOLDER_PATH/jupyter-src/*.md $SRC_FOLDER_PATH/jupyter-src/*.svg $SRC_FOLDER_PATH/jupyter-src/*.gif $SRC_FOLDER_PATH/pluto-src/*.html
mv $WORK_TREE/jupyter-src/* $WORK_TREE
rm -r $WORK_TREE/jupyter-src
cp ./README.md ./docs/src/index.md
- name: "Install dependencies"
run: julia --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate(); Pkg.update()'

- name: "Build and deploy"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # For authentication with GitHub Actions token
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }} # For authentication with SSH deploy key
run: julia --project=docs/ docs/make.jl
4 changes: 2 additions & 2 deletions .github/workflows/Eval.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Check out repository
uses: actions/checkout@v2
uses: actions/checkout@v4

# Set up Julia
- name: "Set up Julia"
Expand All @@ -32,7 +32,7 @@ jobs:

# Set up cache
- name: "Set up cache"
uses: actions/cache@v2
uses: actions/cache@v4
env:
cache-name: cache-artifacts
with:
Expand Down
167 changes: 76 additions & 91 deletions .github/workflows/Example.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,23 +12,23 @@ on:
- 'examples/pluto-src/**'
- '.github/workflows/Example.yml'
- 'Project.toml'

jobs:
jypiter:
jupyter:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [windows-latest] # , ubuntu-latest]
file-name: [simple_hybrid_CS, simple_hybrid_ME, juliacon_2023]
julia-version: ['1.8']
julia-version: ['1.10']
julia-arch: [x64]
experimental: [false]

steps:
- name: "Check out repository"
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: "Set up Julia"
uses: julia-actions/setup-julia@v1
with:
Expand All @@ -37,15 +37,15 @@ jobs:

- name: "Install dependencies"
run: julia --project=examples/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate()'

- name: "Install packages"
run: pip install jupyter nbconvert
- name: "Execute and synchronize all files"

- name: "Execute notebook"
env:
FILE: examples/jupyter-src/${{ matrix.file-name }}.ipynb
run: |
jupyter nbconvert --ExecutePreprocessor.kernel_name="julia-1.8" --to notebook --inplace --execute ${{ env.FILE }}
jupyter nbconvert --ExecutePreprocessor.kernel_name="julia-${{ matrix.julia-version }}" --to notebook --inplace --execute ${{ env.FILE }}
jupyter nbconvert --to script ${{ env.FILE }}
jupyter nbconvert --to markdown ${{ env.FILE }}
Expand All @@ -57,105 +57,90 @@ jobs:
awk '{if($0~/<img src="data:image\/gif;base64,[[:alpha:],[:digit:],\/,+,=]*" \/>/) {sub(/<img src="data:image\/gif;base64,[[:alpha:],[:digit:],\/,+,=]*" \/>/,"![gif](${{ matrix.file-name }}_files\/gif_"++i".gif)")}}1' examples/jupyter-src/${{ matrix.file-name }}.md > examples/jupyter-src/tmp_${{ matrix.file-name }}.md
mv -Force examples/jupyter-src/tmp_${{ matrix.file-name }}.md examples/jupyter-src/${{ matrix.file-name }}.md
echo "gifs should be fixed"
- name: Archive examples artifacts (success)
if: success() && matrix.os == 'windows-latest'
uses: actions/upload-artifact@v3
- name: "run generated jl script to determine success of example building"
run: julia --project=examples/ examples/jupyter-src/${{ matrix.file-name }}.jl

- name: "auto-commit (retry on merge)"
if: success() && github.event_name != 'pull_request' && github.branch == 'main'
uses: nick-fields/retry@v3
env:
CI_COMMIT_MESSAGE: jupyter-example-${{ matrix.os }}-${{ matrix.file-name }}-${{ matrix.julia-version }}-${{ matrix.julia-arch }}-${{ matrix.experimental }}[${{ github.ref }}]
CI_COMMIT_AUTHOR: github-actions[bot]
EXAMPLES_PATH: examples
# Fetch all and clear the stash list. Include all files from the examples folder to the stash and switch the branch.
# Reset the branch and remove all current files in the examples folder.
# Checkout the last stash to restore the new notebooks and apply the stash index to restore all other new files in the folder.
with:
name: jupyter-examples
path: examples/jupyter-src/${{ matrix.file-name }}*

timeout_minutes: 999
max_attempts: 10
warning_on_retry: false
shell: bash
command: |
git fetch --all
git stash clear
git stash --include-untracked -- ${{ env.EXAMPLES_PATH }}
git switch examples
git reset --hard origin/examples
git checkout stash -f -- ${{ env.EXAMPLES_PATH }}
git stash apply --index
git stash drop
git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
git config --global user.email "${{ env.CI_COMMIT_AUTHOR }}@users.noreply.github.com"
git config --global core.autocrlf false
git add ${{ env.EXAMPLES_PATH }}
git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
git push origin examples || (git reset --soft HEAD~1 && (exit 1))
pluto:
runs-on: ubuntu-latest
steps:
- name: "Check out repository"
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: "Set up Julia"
uses: julia-actions/setup-julia@v1
with:
version: '1.10'

- run: julia -e 'using Pkg; Pkg.add("PlutoSliderServer"); Pkg.add("FMIFlux")'
- run: julia -e 'using PlutoSliderServer; PlutoSliderServer.export_directory("examples/pluto-src")'

- name: Archive examples artifacts (success)
if: success()
uses: actions/upload-artifact@v3
with:
name: pluto-examples
path: examples/pluto-src/*

filecheck:
needs: [jypiter, pluto]
runs-on: ubuntu-latest
steps:
- name: Download jupyter examples
uses: actions/download-artifact@v3
with:
name: jupyter-examples
path: examples/jupyter-src/

- name: Download pluto examples
uses: actions/download-artifact@v3
with:
name: pluto-examples
path: examples/pluto-src/

- name: Check if the example files generated are valid (if jupyter-examples failed, svgs are missing; jupyter command does not fail even if examples fail)
uses: andstor/file-existence-action@v3
with:
files: "examples/jupyter-src/*/*.svg"
fail: true

auto-commit:
needs: [jypiter, pluto, filecheck]
if: github.event_name != 'pull_request'
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v3

- name: Download jupyter examples
uses: actions/download-artifact@v3
with:
name: jupyter-examples
path: examples/jupyter-src/

- name: Download pluto examples
uses: actions/download-artifact@v3
with:
name: pluto-examples
path: examples/pluto-src/

- name: auto-commit

- name: "auto-commit (retry on merge)"
if: success() && github.event_name != 'pull_request' && github.branch == 'main'
uses: nick-fields/retry@v3
env:
CI_COMMIT_MESSAGE: Jupyter modified .ipynb & exported md files; Pluto static html exported files
CI_COMMIT_MESSAGE: pluto-examples[${{ github.ref }}]
CI_COMMIT_AUTHOR: github-actions[bot]
EXAMPLES_PATH: examples
# Fetch all and clear the stash list. Include all files from the examples/jupyter-src folder to the stash and switch the branch.
# Reset the branch and remove all current files in the examples/jupyter-src folder.
# Fetch all and clear the stash list. Include all files from the examples folder to the stash and switch the branch.
# Reset the branch and remove all current files in the examples folder.
# Checkout the last stash to restore the new notebooks and apply the stash index to restore all other new files in the folder.
run: |
git fetch --all
git stash clear
git stash --include-untracked -- ${{ env.EXAMPLES_PATH }}
git switch examples
git reset --hard origin/examples
rm -rf ${{ env.EXAMPLES_PATH }}
git checkout stash -f -- ${{ env.EXAMPLES_PATH }}
git stash apply --index
git stash drop
git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
git config --global user.email "${{ env.CI_COMMIT_AUTHOR }}@users.noreply.github.com"
git config --global core.autocrlf false
git add ${{ env.EXAMPLES_PATH }}
git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
git push origin examples
with:
timeout_minutes: 999
max_attempts: 10
warning_on_retry: false
shell: bash
command: |
git fetch --all
git stash clear
git stash --include-untracked -- ${{ env.EXAMPLES_PATH }}
git switch examples
git reset --hard origin/examples
git checkout stash -f -- ${{ env.EXAMPLES_PATH }}
git stash apply --index
git stash drop
git config --global user.name "${{ env.CI_COMMIT_AUTHOR }}"
git config --global user.email "${{ env.CI_COMMIT_AUTHOR }}@users.noreply.github.com"
git config --global core.autocrlf false
git pull
git add ${{ env.EXAMPLES_PATH }}
git commit -m "${{ env.CI_COMMIT_MESSAGE }}"
git push origin examples || (git reset --soft HEAD~1 && (exit 1))
call-docu:
needs: auto-commit
if: github.event_name != 'pull_request'
needs: [jupyter, pluto]
if: github.event_name != 'pull_request' && github.branch == 'main'
runs-on: ubuntu-latest
steps:
# Trigger an repoisitory dispath event
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/TestLTS.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
steps:
# Checks-out your repository
- name: Check out repository
uses: actions/checkout@v3
uses: actions/checkout@v4

# Set up Julia
- name: "Set up Julia"
Expand All @@ -37,7 +37,7 @@ jobs:

# Set up cache
- name: "Set up cache"
uses: actions/cache@v3
uses: actions/cache@v4
env:
cache-name: cache-artifacts
with:
Expand All @@ -54,4 +54,4 @@ jobs:

# Run the tests
- name: "Run tests"
uses: julia-actions/julia-runtest@v1
uses: julia-actions/julia-runtest@v1
Loading

0 comments on commit 322f028

Please sign in to comment.