Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk: testing for failures #48356

Closed
wants to merge 24 commits into from
1 change: 1 addition & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,7 @@ workflows:
ignore:
- master
- bulk
- /bulk.*/
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kind of hacky, but I did it because master is hard-coded in this yml file as the root branch and so building this would try to build everything from bulk.

Azure has the option to exclude PRs with the target of bulk so I did that on the bulk branch itself before branching this..

matrix:
parameters:
os:
Expand Down
87 changes: 8 additions & 79 deletions .github/workflows/Bulk.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,87 +2,18 @@ name: Bulk branch
on:
push:
branches:
- bulk
- bulk-osx-failures
jobs:
build-linux:
name: Bulk Linux Builds
if: "contains(github.event.head_commit.message, '[ci run]')"
runs-on: ubuntu-latest
strategy:
fail-fast: false
max-parallel: 6
matrix:
runner: [0, 1, 2, 3, 4, 5]
steps:
- uses: actions/checkout@v4
with:
# checkout as BiocondaBot in order to have the permission to push fail logs
token: ${{secrets.BIOCONDA_BOT_REPO_TOKEN}}

- name: set git user
run: |
git config user.name BiocondaBot
git config user.email [email protected]

- name: set path
run: echo "/opt/mambaforge/bin" >> $GITHUB_PATH

- name: Fetch conda install script
run: |
wget https://raw.githubusercontent.com/bioconda/bioconda-common/bulk/{common,install-and-set-up-conda,configure-conda}.sh

- name: Restore cache
id: cache
uses: actions/cache@v4
with:
path: /opt/mambaforge
key: ${{ runner.os }}--bulk--${{ hashFiles('install-and-set-up-conda.sh', 'common.sh', 'configure-conda.sh') }}

- name: Set up bioconda-utils
if: steps.cache.outputs.cache-hit != 'true'
run: bash install-and-set-up-conda.sh

# This script can be used to reconfigure conda to use the right channel setup.
# This has to be done after the cache is restored, because
# the channel setup is not cached as it resides in the home directory.
# We could use a system-wide (and therefore cached) channel setup,
# but mamba does not support that at the time of implementation
# (it ignores settings made with --system).
- name: Configure conda
run: bash configure-conda.sh

- name: Build and upload
env:
ANACONDA_TOKEN: ${{ secrets.ANACONDA_TOKEN }}
INVOLUCRO_AUTH: ${{ secrets.INVOLUCRO_AUTH }}
QUAY_OAUTH_TOKEN: ${{ secrets.QUAY_OAUTH_TOKEN }}
# Mimic circleci
OSTYPE: "linux-gnu"
CI: "true"
run: |
set -e
eval "$(conda shell.bash hook)"
conda activate bioconda
echo '============'
conda info --all
conda config --show-sources
python -c 'import bioconda_utils.utils as u ; import pathlib as p ; print(*(f"{f}:\n{p.Path(f).read_text()}" for f in u.load_conda_build_config().exclusive_config_files), sep="\n")'
echo '============'
bioconda-utils build recipes config.yml \
--worker-offset ${{ matrix.runner }} --n-workers 6 \
--docker --mulled-test --anaconda-upload --mulled-upload-target biocontainers \
--lint --record-build-failures --skiplist-leafs
conda clean -y --all

build-osx:
name: Bulk OSX Builds
if: "contains(github.event.head_commit.message, '[ci run]')"
runs-on: macos-13
strategy:
fail-fast: false
max-parallel: 4
max-parallel: 1
matrix:
runner: [0, 1, 2, 3]
runner: [0] #, 1, 2, 3]
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -99,7 +30,7 @@ jobs:

- name: Fetch conda install script
run: |
wget https://raw.githubusercontent.com/bioconda/bioconda-common/bulk/{common,install-and-set-up-conda,configure-conda}.sh
wget https://raw.githubusercontent.com/bioconda/bioconda-common/bulk-osx-failures/{common,install-and-set-up-conda,configure-conda}.sh

- name: Restore cache
id: cache
Expand Down Expand Up @@ -133,17 +64,15 @@ jobs:
set -e
eval "$(conda shell.bash hook)"
conda activate bioconda

# The SDK isn't actually cached, so reinstall it
source common.sh
run_conda_forge_build_setup

echo $MACOSX_DEPLOYMENT_TARGET
echo $MACOSX_SDK_VERSION
echo '============'
conda info --all
conda config --show-sources
conda activate bioconda
python -c 'import bioconda_utils.utils as u ; import pathlib as p ; print(*(f"{f}:\n{p.Path(f).read_text()}" for f in u.load_conda_build_config().exclusive_config_files), sep="\n")'
echo '============'
bioconda-utils build recipes config.yml \
--worker-offset ${{ matrix.runner }} --n-workers 4 \
--lint --anaconda-upload --record-build-failures --skiplist-leafs
bioconda-utils build recipes config.yml --packages bowtie
conda clean -y --all
104 changes: 0 additions & 104 deletions recipes/bowtie/build_failure.osx-64.yaml

This file was deleted.

1 change: 0 additions & 1 deletion recipes/bowtie/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ source:
- rename_version_to_version_txt.patch

build:
skip: True # [py<30]
number: 8
run_exports:
- {{ pin_subpackage("bowtie", max_pin="x.x") }}
Expand Down