Skip to content

Commit

Permalink
Merge pull request #159 from nf-core/dev
Browse files Browse the repository at this point in the history
Dev -> Master for 1.10.0 release
  • Loading branch information
ejseqera authored May 16, 2023
2 parents 084e5ef + 1b9d187 commit 2706782
Show file tree
Hide file tree
Showing 48 changed files with 588 additions and 343 deletions.
2 changes: 1 addition & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ trim_trailing_whitespace = true
indent_size = 4
indent_style = space

[*.{md,yml,yaml,html,css,scss,js,cff}]
[*.{md,yml,yaml,html,css,scss,js}]
indent_size = 2

# These files are edited and tested upstream in nf-core/modules
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,6 @@ body:
* Nextflow version _(eg. 22.10.1)_
* Hardware _(eg. HPC, Desktop, Cloud)_
* Executor _(eg. slurm, local, awsbatch)_
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
* OS _(eg. CentOS Linux, macOS, Linux Mint)_
* Version of nf-core/fetchngs _(eg. 1.1, 1.5, 1.8.2)_
3 changes: 2 additions & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/fetc

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/fetchngs/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/fetchngs _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/fetchngs/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/fetchngs _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
Expand Down
31 changes: 0 additions & 31 deletions .github/workflows/awsfulltest.yml

This file was deleted.

29 changes: 0 additions & 29 deletions .github/workflows/awstest.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
- name: Check PRs
if: github.repository == 'nf-core/fetchngs'
run: |
{ [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/fetchngs ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
{ [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/fetchngs ]] && [[ $GITHUB_HEAD_REF == "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
# If the above check failed, post a comment on the PR explaining the failure
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
Expand Down
24 changes: 24 additions & 0 deletions .github/workflows/clean-up.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: "Close user-tagged issues and PRs"
on:
schedule:
- cron: "0 0 * * 0" # Once a week

jobs:
clean-up:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v7
with:
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
close-issue-message: "This issue was closed because it has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor and then staled for 20 days with no activity."
days-before-stale: 30
days-before-close: 20
days-before-pr-close: -1
any-of-labels: "awaiting-changes,awaiting-feedback"
exempt-issue-labels: "WIP"
exempt-pr-labels: "WIP"
repo-token: "${{ secrets.GITHUB_TOKEN }}"
81 changes: 81 additions & 0 deletions .github/workflows/cloud_tests_full.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
name: full-sized tests on cloud providers
run-name: Submitting workflow to all cloud providers using full sized data
on:
release:
types: [published]
workflow_dispatch:
inputs:
platform:
description: "Platform to run test"
required: true
default: "all"
type: choice
options:
- all
- aws
- azure
- gcp
jobs:
run-full-tests-on-aws:
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'aws' || !github.event.inputs }}
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_CE_AWS_CPU }}
workdir: "${{ secrets.TOWER_BUCKET_AWS }}/work/fetchngs/work-${{ github.sha }}"
run_name: "aws_fetchngs_full"
profiles: test_full,public_aws_ecr
parameters: |
{
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
"outdir": "${{ secrets.TOWER_BUCKET_AWS }}/fetchngs/results-${{ github.sha }}"
}
- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
run-full-tests-on-gcp:
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'gcp' || !github.event.inputs }}
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_CE_GCP_CPU }}
workdir: "${{ secrets.TOWER_BUCKET_GCP }}/work/fetchngs/work-${{ github.sha }}"
run_name: "gcp_fetchngs_full"
profiles: test_full
parameters: |
{
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
"outdir": "${{ secrets.TOWER_BUCKET_GCP }}/fetchngs/results-${{ github.sha }}"
}
- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
run-full-tests-on-azure:
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'azure' || !github.event.inputs }}
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_CE_AZURE_CPU }}
workdir: "${{ secrets.TOWER_BUCKET_AZURE }}/work/fetchngs/work-${{ github.sha }}"
run_name: "azure_fetchngs_full"
profiles: test_full
parameters: |
{
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
"outdir": "${{ secrets.TOWER_BUCKET_AZURE }}/fetchngs/results-${{ github.sha }}"
}
- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
76 changes: 76 additions & 0 deletions .github/workflows/cloud_tests_small.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
name: small-sized tests on cloud providers
run-name: Submitting workflow to all cloud providers using small sized data
on:
workflow_dispatch:
inputs:
platform:
description: "Platform to run test"
required: true
default: "all"
type: choice
options:
- all
- aws
- azure
- gcp
jobs:
run-small-tests-on-aws:
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'aws' }}
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_CE_AWS_CPU }}
workdir: "${{ secrets.TOWER_BUCKET_AWS }}/work/fetchngs/work-${{ github.sha }}"
run_name: "aws_fetchngs_small"
profiles: test,public_aws_ecr
parameters: |
{
"outdir": "${{ secrets.TOWER_BUCKET_AWS }}/fetchngs/results-test-${{ github.sha }}"
}
- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
run-small-tests-on-gcp:
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'gcp' }}
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_CE_GCP_CPU }}
workdir: "${{ secrets.TOWER_BUCKET_GCP }}/work/fetchngs/work-${{ github.sha }}"
run_name: "gcp_fetchngs_small"
profiles: test
parameters: |
{
"outdir": "${{ secrets.TOWER_BUCKET_GCP }}/fetchngs/results-test-${{ github.sha }}"
}
- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
run-small-tests-on-azure:
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'azure' }}
runs-on: ubuntu-latest
steps:
- uses: seqeralabs/action-tower-launch@v1
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_CE_AZURE_CPU }}
workdir: "${{ secrets.TOWER_BUCKET_AZURE }}/work/fetchngs/work-${{ github.sha }}"
run_name: "azure_fetchngs_small"
profiles: test
parameters: |
{
"outdir": "${{ secrets.TOWER_BUCKET_AZURE }}/fetchngs/results-test-${{ github.sha }}"
}
- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
2 changes: 1 addition & 1 deletion .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:

- uses: actions/setup-python@v4
with:
python-version: "3.7"
python-version: "3.8"
architecture: "x64"

- name: Install dependencies
Expand Down
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
repos:
- repo: https://github.com/pre-commit/mirrors-prettier
rev: "v2.7.1"
hooks:
- id: prettier
40 changes: 40 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,46 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [[1.10.0](https://github.com/nf-core/fetchngs/releases/tag/1.10.0)] - 2023-05-16

### Credits

Special thanks to the following for their contributions to the release:

- [Adam Talbot](https://github.com/adamrtalbot)
- [Esha Joshi](https://github.com/ejseqera)
- [Maxime Garcia](https://github.com/maxulysse)
- [Moritz E. Beber](https://github.com/Midnighter)
- [Rob Syme](https://github.com/robsyme)
- [sirclockalot](https://github.com/sirclockalot)

Thank you to everyone else that has contributed by reporting bugs, enhancements or in any other way, shape or form.

### Enhancements & fixes

- [#85](https://github.com/nf-core/fetchngs/issues/85) - Not able to fetch metadata for ERR ids associated with ArrayExpress
- [#104](https://github.com/nf-core/fetchngs/issues/104) - Add support back in for [GEO IDs](https://www.ncbi.nlm.nih.gov/geo) (removed in v1.7)
- [#129](https://github.com/nf-core/fetchngs/issues/129) - Pipeline is working with SRA run ids but failing with corresponding Biosample ids
- [#138](https://github.com/nf-core/fetchngs/issues/138) - Add support for downloading protected dbGAP data using a JWT file
- [#144](https://github.com/nf-core/fetchngs/issues/144) - Add support to download 10X Genomics data
- [PR #140](https://github.com/nf-core/fetchngs/pull/140) - Bumped modules version to allow for sratools download of sralite format files
- [PR #147](https://github.com/nf-core/fetchngs/pull/147) - Updated pipeline template to [nf-core/tools 2.8](https://github.com/nf-core/tools/releases/tag/2.8)
- [PR #148](https://github.com/nf-core/fetchngs/pull/148) - Fix default metadata fields for ENA API v2.0
- [PR #150](https://github.com/nf-core/fetchngs/pull/150) - Add infrastructure and CI for multi-cloud full-sized tests run via Nextflow Tower
- [PR #157](https://github.com/nf-core/fetchngs/pull/157) - Add `public_aws_ecr.config` to source mulled containers when using `public.ecr.aws` Docker Biocontainer registry

### Software dependencies

| Dependency | Old version | New version |
| --------------- | ----------- | ----------- |
| `synapseclient` | 2.6.0 | 2.7.1 |

> **NB:** Dependency has been **updated** if both old and new version information is present.
>
> **NB:** Dependency has been **added** if just the new version information is present.
>
> **NB:** Dependency has been **removed** if new version information isn't present.
## [[1.9](https://github.com/nf-core/fetchngs/releases/tag/1.9)] - 2022-12-21

### Enhancements & fixes
Expand Down
Loading

0 comments on commit 2706782

Please sign in to comment.