Skip to content

Commit 4af8299

Browse files
SusiJoSusiJo
authored
Merge pull request #159 from nf-core/nf-core-template-merge-3.3.2
Nf core template merge 3.3.2
2 parents c03848e + bb7fdf9 commit 4af8299

File tree

72 files changed

+1279
-832
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

72 files changed

+1279
-832
lines changed

‎.editorconfig‎

Lines changed: 0 additions & 33 deletions
This file was deleted.

‎.github/CONTRIBUTING.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# nf-core/metapep: Contributing Guidelines
1+
# `nf-core/metapep`: Contributing Guidelines
22

33
Hi there!
44
Many thanks for taking an interest in improving nf-core/metapep.
@@ -55,36 +55,36 @@ These tests are run both with the latest available version of `Nextflow` and als
5555

5656
:warning: Only in the unlikely and regretful event of a release happening with a bug.
5757

58-
- On your own fork, make a new branch `patch` based on `upstream/master`.
58+
- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`.
5959
- Fix the bug, and bump version (X.Y.Z+1).
60-
- A PR should be made on `master` from patch to directly this particular bug.
60+
- Open a pull-request from `patch` to `main`/`master` with the changes.
6161

6262
## Getting help
6363

6464
For further information/help, please consult the [nf-core/metapep documentation](https://nf-co.re/metapep/usage) and don't hesitate to get in touch on the nf-core Slack [#metapep](https://nfcore.slack.com/channels/metapep) channel ([join our Slack here](https://nf-co.re/join/slack)).
6565

6666
## Pipeline contribution conventions
6767

68-
To make the nf-core/metapep code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
68+
To make the `nf-core/metapep` code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
6969

7070
### Adding a new step
7171

7272
If you wish to contribute a new step, please use the following coding standards:
7373

74-
1. Define the corresponding input channel into your new process from the expected previous process channel
74+
1. Define the corresponding input channel into your new process from the expected previous process channel.
7575
2. Write the process block (see below).
7676
3. Define the output channel if needed (see below).
7777
4. Add any new parameters to `nextflow.config` with a default (see below).
7878
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core pipelines schema build` tool).
7979
6. Add sanity checks and validation for all relevant parameters.
8080
7. Perform local tests to validate that the new code works as expected.
81-
8. If applicable, add a new test command in `.github/workflow/ci.yml`.
81+
8. If applicable, add a new test in the `tests` directory.
8282
9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module.
8383
10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.
8484

8585
### Default values
8686

87-
Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.
87+
Parameters should be initialised / defined with default values within the `params` scope in `nextflow.config`.
8888

8989
Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json`.
9090

‎.github/ISSUE_TEMPLATE/bug_report.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ body:
99
1010
- [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
1111
- [nf-core/metapep pipeline documentation](https://nf-co.re/metapep/usage)
12-
1312
- type: textarea
1413
id: description
1514
attributes:

‎.github/actions/get-shards/action.yml‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: "Get number of shards"
2+
description: "Get the number of nf-test shards for the current CI job"
3+
inputs:
4+
max_shards:
5+
description: "Maximum number of shards allowed"
6+
required: true
7+
paths:
8+
description: "Component paths to test"
9+
required: false
10+
tags:
11+
description: "Tags to pass as argument for nf-test --tag parameter"
12+
required: false
13+
outputs:
14+
shard:
15+
description: "Array of shard numbers"
16+
value: ${{ steps.shards.outputs.shard }}
17+
total_shards:
18+
description: "Total number of shards"
19+
value: ${{ steps.shards.outputs.total_shards }}
20+
runs:
21+
using: "composite"
22+
steps:
23+
- name: Install nf-test
24+
uses: nf-core/setup-nf-test@v1
25+
with:
26+
version: ${{ env.NFT_VER }}
27+
- name: Get number of shards
28+
id: shards
29+
shell: bash
30+
run: |
31+
# Run nf-test with dynamic parameter
32+
nftest_output=$(nf-test test \
33+
--profile +docker \
34+
$(if [ -n "${{ inputs.tags }}" ]; then echo "--tag ${{ inputs.tags }}"; fi) \
35+
--dry-run \
36+
--ci \
37+
--changed-since HEAD^) || {
38+
echo "nf-test command failed with exit code $?"
39+
echo "Full output: $nftest_output"
40+
exit 1
41+
}
42+
echo "nf-test dry-run output: $nftest_output"
43+
44+
# Default values for shard and total_shards
45+
shard="[]"
46+
total_shards=0
47+
48+
# Check if there are related tests
49+
if echo "$nftest_output" | grep -q 'No tests to execute'; then
50+
echo "No related tests found."
51+
else
52+
# Extract the number of related tests
53+
number_of_shards=$(echo "$nftest_output" | sed -n 's|.*Executed \([0-9]*\) tests.*|\1|p')
54+
if [[ -n "$number_of_shards" && "$number_of_shards" -gt 0 ]]; then
55+
shards_to_run=$(( $number_of_shards < ${{ inputs.max_shards }} ? $number_of_shards : ${{ inputs.max_shards }} ))
56+
shard=$(seq 1 "$shards_to_run" | jq -R . | jq -c -s .)
57+
total_shards="$shards_to_run"
58+
else
59+
echo "Unexpected output format. Falling back to default values."
60+
fi
61+
fi
62+
63+
# Write to GitHub Actions outputs
64+
echo "shard=$shard" >> $GITHUB_OUTPUT
65+
echo "total_shards=$total_shards" >> $GITHUB_OUTPUT
66+
67+
# Debugging output
68+
echo "Final shard array: $shard"
69+
echo "Total number of shards: $total_shards"

‎.github/actions/nf-test/action.yml‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
name: "nf-test Action"
2+
description: "Runs nf-test with common setup steps"
3+
inputs:
4+
profile:
5+
description: "Profile to use"
6+
required: true
7+
shard:
8+
description: "Shard number for this CI job"
9+
required: true
10+
total_shards:
11+
description: "Total number of test shards(NOT the total number of matrix jobs)"
12+
required: true
13+
paths:
14+
description: "Test paths"
15+
required: true
16+
tags:
17+
description: "Tags to pass as argument for nf-test --tag parameter"
18+
required: false
19+
runs:
20+
using: "composite"
21+
steps:
22+
- name: Setup Nextflow
23+
uses: nf-core/setup-nextflow@v2
24+
with:
25+
version: "${{ env.NXF_VERSION }}"
26+
27+
- name: Set up Python
28+
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
29+
with:
30+
python-version: "3.13"
31+
32+
- name: Install icdiff
33+
run: pip install icdiff
34+
shell: bash
35+
36+
- name: Run icdiff
37+
run: icdiff --help
38+
shell: bash
39+
40+
- name: Install nf-test
41+
uses: nf-core/setup-nf-test@v1
42+
with:
43+
version: "${{ env.NFT_VER }}"
44+
install-pdiff: true
45+
46+
- name: Setup apptainer
47+
if: contains(inputs.profile, 'singularity')
48+
uses: eWaterCycle/setup-apptainer@main
49+
50+
- name: Set up Singularity
51+
if: contains(inputs.profile, 'singularity')
52+
shell: bash
53+
run: |
54+
mkdir -p $NXF_SINGULARITY_CACHEDIR
55+
mkdir -p $NXF_SINGULARITY_LIBRARYDIR
56+
57+
- name: Conda setup
58+
if: contains(inputs.profile, 'conda')
59+
uses: conda-incubator/setup-miniconda@505e6394dae86d6a5c7fbb6e3fb8938e3e863830 # v3
60+
with:
61+
auto-update-conda: true
62+
conda-solver: libmamba
63+
conda-remove-defaults: true
64+
65+
# Set up Nextflow secrets for NCBI access
66+
- name: Set Nextflow secrets
67+
if: env.NCBI_EMAIL != '' && env.NCBI_KEY != ''
68+
shell: bash
69+
run: |
70+
nextflow secrets set NCBI_EMAIL "$NCBI_EMAIL"
71+
nextflow secrets set NCBI_KEY "$NCBI_KEY"
72+
73+
- name: Run nf-test
74+
shell: bash
75+
env:
76+
NFT_WORKDIR: ${{ env.NFT_WORKDIR }}
77+
NFT_DIFF: ${{ env.NFT_DIFF }}
78+
NFT_DIFF_ARGS: ${{ env.NFT_DIFF_ARGS }}
79+
run: |
80+
nf-test test \
81+
--profile=+${{ inputs.profile }} \
82+
$(if [ -n "${{ inputs.tags }}" ]; then echo "--tag ${{ inputs.tags }}"; fi) \
83+
--ci \
84+
--changed-since HEAD^ \
85+
--verbose \
86+
--tap=test.tap \
87+
--shard ${{ inputs.shard }}/${{ inputs.total_shards }}
88+
89+
# Save the absolute path of the test.tap file to the output
90+
echo "tap_file_path=$(realpath test.tap)" >> $GITHUB_OUTPUT
91+
92+
- name: Generate test summary
93+
if: always()
94+
shell: bash
95+
run: |
96+
# Add header if it doesn't exist (using a token file to track this)
97+
if [ ! -f ".summary_header" ]; then
98+
echo "# 🚀 nf-test results" >> $GITHUB_STEP_SUMMARY
99+
echo "" >> $GITHUB_STEP_SUMMARY
100+
echo "| Status | Test Name | Profile | Shard |" >> $GITHUB_STEP_SUMMARY
101+
echo "|:------:|-----------|---------|-------|" >> $GITHUB_STEP_SUMMARY
102+
touch .summary_header
103+
fi
104+
105+
if [ -f test.tap ]; then
106+
while IFS= read -r line; do
107+
if [[ $line =~ ^ok ]]; then
108+
test_name="${line#ok }"
109+
# Remove the test number from the beginning
110+
test_name="${test_name#* }"
111+
echo "| ✅ | ${test_name} | ${{ inputs.profile }} | ${{ inputs.shard }}/${{ inputs.total_shards }} |" >> $GITHUB_STEP_SUMMARY
112+
elif [[ $line =~ ^not\ ok ]]; then
113+
test_name="${line#not ok }"
114+
# Remove the test number from the beginning
115+
test_name="${test_name#* }"
116+
echo "| ❌ | ${test_name} | ${{ inputs.profile }} | ${{ inputs.shard }}/${{ inputs.total_shards }} |" >> $GITHUB_STEP_SUMMARY
117+
fi
118+
done < test.tap
119+
else
120+
echo "| ⚠️ | No test results found | ${{ inputs.profile }} | ${{ inputs.shard }}/${{ inputs.total_shards }} |" >> $GITHUB_STEP_SUMMARY
121+
fi
122+
123+
- name: Clean up
124+
if: always()
125+
shell: bash
126+
run: |
127+
sudo rm -rf /home/ubuntu/tests/

‎.github/workflows/awsfulltest.yml‎

Lines changed: 13 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,27 @@
11
name: nf-core AWS full size tests
2-
# This workflow is triggered on PRs opened against the master branch.
2+
# This workflow is triggered on PRs opened against the main/master branch.
33
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
44
# It runs the -profile 'test_full' on AWS batch
55

66
on:
7-
pull_request:
8-
branches:
9-
- master
107
workflow_dispatch:
118
pull_request_review:
129
types: [submitted]
10+
release:
11+
types: [published]
1312

1413
jobs:
1514
run-platform:
1615
name: Run AWS full tests
17-
# run only if the PR is approved by at least 2 reviewers and against the master branch or manually triggered
18-
if: github.repository == 'nf-core/metapep' && github.event.review.state == 'approved' && github.event.pull_request.base.ref == 'master' || github.event_name == 'workflow_dispatch'
16+
# run only if the PR is approved by at least 2 reviewers and against the master/main branch or manually triggered
17+
if: github.repository == 'nf-core/metapep' && github.event.review.state == 'approved' && (github.event.pull_request.base.ref == 'master' || github.event.pull_request.base.ref == 'main') || github.event_name == 'workflow_dispatch' || github.event_name == 'release'
1918
runs-on: ubuntu-latest
2019
steps:
21-
- uses: octokit/[email protected]
22-
id: check_approvals
23-
with:
24-
route: GET /repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews
25-
env:
26-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
27-
- id: test_variables
28-
if: github.event_name != 'workflow_dispatch'
20+
- name: Set revision variable
21+
id: revision
2922
run: |
30-
JSON_RESPONSE='${{ steps.check_approvals.outputs.data }}'
31-
CURRENT_APPROVALS_COUNT=$(echo $JSON_RESPONSE | jq -c '[.[] | select(.state | contains("APPROVED")) ] | length')
32-
test $CURRENT_APPROVALS_COUNT -ge 2 || exit 1 # At least 2 approvals are required
23+
echo "revision=${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'release') && github.sha || 'dev' }}" >> "$GITHUB_OUTPUT"
24+
3325
- name: Launch workflow via Seqera Platform
3426
uses: seqeralabs/action-tower-launch@v2
3527
# TODO nf-core: You can customise AWS full pipeline tests as required
@@ -39,16 +31,16 @@ jobs:
3931
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
4032
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
4133
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
42-
revision: ${{ github.sha }}
43-
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/metapep/work-${{ github.sha }}
34+
revision: ${{ steps.revision.outputs.revision }}
35+
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/metapep/work-${{ steps.revision.outputs.revision }}
4436
parameters: |
4537
{
4638
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
47-
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/metapep/results-${{ github.sha }}"
39+
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/metapep/results-${{ steps.revision.outputs.revision }}"
4840
}
4941
profiles: test_full
5042

51-
- uses: actions/upload-artifact@v4
43+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
5244
with:
5345
name: Seqera Platform debug log file
5446
path: |

‎.github/workflows/awstest.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ jobs:
2525
}
2626
profiles: test
2727

28-
- uses: actions/upload-artifact@v4
28+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
2929
with:
3030
name: Seqera Platform debug log file
3131
path: |

0 commit comments

Comments
 (0)