Skip to content

Commit b971ba1

Browse files
authored
Merge pull request #1593 from nf-core/dev
Dev -> Master for 3.20.0
2 parents 0bb032c + eb070e1 commit b971ba1

File tree

98 files changed

+9139
-773
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

98 files changed

+9139
-773
lines changed

.github/actions/nf-test/action.yml

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,12 +54,20 @@ runs:
5454
conda-solver: libmamba
5555
conda-remove-defaults: true
5656

57-
# TODO Skip failing conda tests and document their failures
58-
# https://github.com/nf-core/modules/issues/7017
57+
# Set up secrets
58+
- name: Set up Nextflow secrets
59+
if: env.SENTIEON_ENCRYPTION_KEY != '' && env.SENTIEON_LICENSE_MESSAGE != ''
60+
shell: bash
61+
run: |
62+
python -m pip install cryptography
63+
nextflow secrets set SENTIEON_AUTH_DATA $(python3 .github/actions/nf-test/license_message.py encrypt --key "$SENTIEON_ENCRYPTION_KEY" --message "$SENTIEON_LICENSE_MESSAGE")
64+
5965
- name: Run nf-test
6066
shell: bash
6167
env:
6268
NFT_WORKDIR: ${{ env.NFT_WORKDIR }}
69+
SENTIEON_LICSRVR_IP: ${{ env.SENTIEON_LICSRVR_IP }}
70+
SENTIEON_AUTH_MECH: "GitHub Actions - token"
6371
run: |
6472
nf-test test \
6573
--profile=+${{ inputs.profile }} \
Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
#!/usr/bin/env python3
2+
3+
#########################################
4+
# Author: [DonFreed](https://github.com/DonFreed)
5+
# File: license_message.py
6+
# Source: https://github.com/DonFreed/docker-actions-test/blob/main/.github/scripts/license_message.py
7+
# Source+commit: https://github.com/DonFreed/docker-actions-test/blob/aa1051a9f53b3a1e801953748d062cad74dca9a9/.github/scripts/license_message.py
8+
# Download Date: 2023-07-04, commit: aa1051a
9+
# This source code is licensed under the BSD 2-Clause license
10+
#########################################
11+
12+
"""
13+
Functions for generating and sending license messages
14+
"""
15+
16+
# Modified from - https://stackoverflow.com/a/59835994
17+
18+
import argparse
19+
import base64
20+
import calendar
21+
import re
22+
import secrets
23+
import sys
24+
25+
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
26+
from datetime import datetime as dt
27+
28+
MESSAGE_TIMEOUT = 60 * 60 * 24 # Messages are valid for 1 day
29+
NONCE_BYTES = 12
30+
31+
32+
class DecryptionTimeout(Exception):
33+
# Decrypting a message that is too old
34+
pass
35+
36+
37+
def generate_key():
38+
key = secrets.token_bytes(32)
39+
return key
40+
41+
42+
def handle_generate_key(args):
43+
key = generate_key()
44+
key_b64 = base64.b64encode(key)
45+
print(key_b64.decode("utf-8"), file=args.outfile)
46+
47+
48+
def encrypt_message(key, message):
49+
nonce = secrets.token_bytes(NONCE_BYTES)
50+
timestamp = calendar.timegm(dt.now().utctimetuple())
51+
data = timestamp.to_bytes(10, byteorder="big") + b"__" + message
52+
ciphertext = nonce + AESGCM(key).encrypt(nonce, data, b"")
53+
return ciphertext
54+
55+
56+
def handle_encrypt_message(args):
57+
key = base64.b64decode(args.key.encode("utf-8"))
58+
message = args.message.encode("utf-8")
59+
ciphertext = encrypt_message(key, message)
60+
ciphertext_b64 = base64.b64encode(ciphertext)
61+
print(ciphertext_b64.decode("utf-8"), file=args.outfile)
62+
63+
64+
def decrypt_message(key, ciphertext, timeout=MESSAGE_TIMEOUT):
65+
nonce, ciphertext = ciphertext[:NONCE_BYTES], ciphertext[NONCE_BYTES:]
66+
message = AESGCM(key).decrypt(nonce, ciphertext, b"")
67+
68+
msg_timestamp, message = re.split(b"__", message, maxsplit=1)
69+
msg_timestamp = int.from_bytes(msg_timestamp, byteorder="big")
70+
timestamp = calendar.timegm(dt.now().utctimetuple())
71+
if (timestamp - msg_timestamp) > timeout:
72+
raise DecryptionTimeout("The message has an expired timeout")
73+
return message.decode("utf-8")
74+
75+
76+
def handle_decrypt_message(args):
77+
key = base64.b64decode(args.key.encode("utf-8"))
78+
ciphertext = base64.b64decode(args.message.encode("utf-8"))
79+
message = decrypt_message(key, ciphertext, timeout=args.timeout)
80+
print(str(message), file=args.outfile)
81+
82+
83+
def parse_args(argv=None):
84+
parser = argparse.ArgumentParser(description=__doc__)
85+
parser.add_argument("--outfile", default=sys.stdout, type=argparse.FileType("w"), help="The output file")
86+
87+
subparsers = parser.add_subparsers(help="Available sub-commands")
88+
89+
gen_parser = subparsers.add_parser("generate_key", help="Generate a random key string")
90+
gen_parser.set_defaults(func=handle_generate_key)
91+
92+
encrypt_parser = subparsers.add_parser("encrypt", help="Encrypt a message")
93+
encrypt_parser.add_argument("--key", required=True, help="The encryption key")
94+
encrypt_parser.add_argument("--message", required=True, help="Message to encrypt")
95+
encrypt_parser.set_defaults(func=handle_encrypt_message)
96+
97+
decrypt_parser = subparsers.add_parser("decrypt", help="Decyrpt a message")
98+
decrypt_parser.add_argument("--key", required=True, help="The encryption key")
99+
decrypt_parser.add_argument("--message", required=True, help="Message to decrypt")
100+
decrypt_parser.add_argument(
101+
"--timeout",
102+
default=MESSAGE_TIMEOUT,
103+
type=int,
104+
help="A message timeout. Decryption will fail for older messages",
105+
)
106+
decrypt_parser.set_defaults(func=handle_decrypt_message)
107+
108+
return parser.parse_args(argv)
109+
110+
111+
if __name__ == "__main__":
112+
args = parse_args()
113+
args.func(args)

.github/workflows/awsfulltest.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ jobs:
4040
{
4141
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
4242
"aligner": "${{ matrix.aligner }}",
43-
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/rnaseq/results-${{ steps.revision.outputs.revision }}"
43+
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/rnaseq/results-${{ steps.revision.outputs.revision }}/aligner_${{ matrix.aligner }}/"
4444
}
4545
profiles: test_full
4646

.github/workflows/linting.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ jobs:
1313
steps:
1414
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
1515

16-
- name: Set up Python 3.12
16+
- name: Set up Python 3.13
1717
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
1818
with:
1919
python-version: "3.13"

.github/workflows/linting_comment.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
runs-on: ubuntu-latest
1212
steps:
1313
- name: Download lint results
14-
uses: dawidd6/action-download-artifact@4c1e823582f43b179e2cbb49c3eade4e41f992e2 # v10
14+
uses: dawidd6/action-download-artifact@ac66b43f0e6a346234dd65d4d0c8fbb31cb316e5 # v11
1515
with:
1616
workflow: linting.yml
1717
workflow_conclusion: completed

.github/workflows/nf-test.yml

Lines changed: 27 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,5 @@
11
name: Run nf-test
22
on:
3-
push:
4-
paths-ignore:
5-
- "docs/**"
6-
- "**/meta.yml"
7-
- "**/*.md"
8-
- "**/*.png"
9-
- "**/*.svg"
103
pull_request:
114
paths-ignore:
125
- "docs/**"
@@ -37,7 +30,7 @@ jobs:
3730
nf-test-changes:
3831
name: nf-test-changes
3932
runs-on: # use self-hosted runners
40-
- runs-on=$-nf-test-changes
33+
- runs-on=${{ github.run_id }}-nf-test-changes
4134
- runner=4cpu-linux-x64
4235
outputs:
4336
shard: ${{ steps.set-shards.outputs.shard }}
@@ -71,7 +64,7 @@ jobs:
7164
needs: [nf-test-changes]
7265
if: ${{ needs.nf-test-changes.outputs.total_shards != '0' }}
7366
runs-on: # use self-hosted runners
74-
- runs-on=$-nf-test
67+
- runs-on=${{ github.run_id }}-nf-test
7568
- runner=4cpu-linux-x64
7669
strategy:
7770
fail-fast: false
@@ -87,7 +80,7 @@ jobs:
8780
- isMain: false
8881
profile: "singularity"
8982
NXF_VER:
90-
- "24.04.2"
83+
- "24.10.5"
9184
- "latest-everything"
9285
env:
9386
NXF_ANSI_LOG: false
@@ -99,23 +92,43 @@ jobs:
9992
fetch-depth: 0
10093

10194
- name: Run nf-test
95+
id: run_nf_test
10296
uses: ./.github/actions/nf-test
97+
continue-on-error: ${{ matrix.NXF_VER == 'latest-everything' }}
10398
env:
104-
NFT_DIFF: ${{ env.NFT_DIFF }}
105-
NFT_DIFF_ARGS: ${{ env.NFT_DIFF_ARGS }}
10699
NFT_WORKDIR: ${{ env.NFT_WORKDIR }}
100+
SENTIEON_AUTH_MECH: "GitHub Actions - token"
101+
SENTIEON_ENCRYPTION_KEY: ${{ secrets.SENTIEON_ENCRYPTION_KEY }}
102+
SENTIEON_LICENSE_MESSAGE: ${{ secrets.SENTIEON_LICENSE_MESSAGE }}
103+
SENTIEON_LICSRVR_IP: ${{ secrets.SENTIEON_LICSRVR_IP }}
107104
with:
108105
profile: ${{ matrix.profile }}
109106
shard: ${{ matrix.shard }}
110107
total_shards: ${{ env.TOTAL_SHARDS }}
108+
109+
- name: Report test status
110+
if: ${{ always() }}
111+
run: |
112+
if [[ "${{ steps.run_nf_test.outcome }}" == "failure" ]]; then
113+
echo "::error::Test with ${{ matrix.NXF_VER }} failed"
114+
# Add to workflow summary
115+
echo "## ❌ Test failed: ${{ matrix.profile }} | ${{ matrix.NXF_VER }} | Shard ${{ matrix.shard }}/${{ env.TOTAL_SHARDS }}" >> $GITHUB_STEP_SUMMARY
116+
if [[ "${{ matrix.NXF_VER }}" == "latest-everything" ]]; then
117+
echo "::warning::Test with latest-everything failed but will not cause workflow failure. Please check if the error is expected or if it needs fixing."
118+
fi
119+
if [[ "${{ matrix.NXF_VER }}" != "latest-everything" ]]; then
120+
exit 1
121+
fi
122+
fi
123+
111124
confirm-pass:
112125
needs: [nf-test]
113126
if: always()
114127
runs-on: # use self-hosted runners
115-
- runs-on=$-confirm-pass
128+
- runs-on=${{ github.run_id }}-confirm-pass
116129
- runner=2cpu-linux-x64
117130
steps:
118-
- name: One or more tests failed
131+
- name: One or more tests failed (excluding latest-everything)
119132
if: ${{ contains(needs.*.result, 'failure') }}
120133
run: exit 1
121134

@@ -134,11 +147,3 @@ jobs:
134147
echo "DEBUG: toJSON(needs) = ${{ toJSON(needs) }}"
135148
echo "DEBUG: toJSON(needs.*.result) = ${{ toJSON(needs.*.result) }}"
136149
echo "::endgroup::"
137-
138-
- name: Clean Workspace # Purge the workspace in case it's running on a self-hosted runner
139-
if: always()
140-
run: |
141-
ls -la ./
142-
rm -rf ./* || true
143-
rm -rf ./.??* || true
144-
ls -la ./

.github/workflows/release-announcements.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ jobs:
3030
bsky-post:
3131
runs-on: ubuntu-latest
3232
steps:
33-
- uses: zentered/bluesky-post-action@4aa83560bb3eac05dbad1e5f221ee339118abdd2 # v0.2.0
33+
- uses: zentered/bluesky-post-action@6461056ea355ea43b977e149f7bf76aaa572e5e8 # v0.3.0
3434
with:
3535
post: |
3636
Pipeline release! ${{ github.repository }} v${{ github.event.release.tag_name }} - ${{ github.event.release.name }}!

.nf-core.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,15 @@ lint:
1010
nextflow_config:
1111
- config_defaults:
1212
- params.ribo_database_manifest
13-
nf_core_version: 3.3.1
13+
nf_core_version: 3.3.2
1414
repository_type: pipeline
1515
template:
16-
author: "Harshil Patel, Phil Ewels, Rickard Hammar\xE9n"
17-
description: RNA sequencing analysis pipeline for gene/isoform quantification and
18-
extensive quality control.
16+
author: "Harshil Patel, Phil Ewels, Rickard Hammarén"
17+
description: RNA sequencing analysis pipeline for gene/isoform quantification
18+
and extensive quality control.
1919
force: false
2020
is_nfcore: true
2121
name: rnaseq
2222
org: nf-core
2323
outdir: .
24-
version: 3.19.0
24+
version: 3.20.0

.pre-commit-config.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ repos:
44
hooks:
55
- id: prettier
66
additional_dependencies:
7-
- prettier@3.5.0
7+
- prettier@3.6.2
88
- repo: https://github.com/pre-commit/pre-commit-hooks
99
rev: v5.0.0
1010
hooks:

CHANGELOG.md

Lines changed: 33 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,37 @@
33
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
44
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
55

6-
# 3.19.0 - 2025-06-06
6+
## 3.20.0
7+
8+
### Credits
9+
10+
Special thanks to the following for their contributions to the release:
11+
12+
- [Friederike Hanssen](https://github.com/friederikehanssen)
13+
- [Ido Tamir](https://github.com/idot)
14+
- [Jonathan Manning](https://github.com/pinin4fjords)
15+
- [Maxime Garcia](https://github.com/maxulysse)
16+
- [Usman Rashid](https://github.com/GallVp)
17+
18+
### Enhancements & fixes
19+
20+
- [PR #1568](https://github.com/nf-core/rnaseq/pull/1568) - Bump version after release 3.19.0
21+
- [PR #1571](https://github.com/nf-core/rnaseq/pull/1571) - For umitools use only umi_dedup.sorted.log in multiqc_report
22+
- [PR #1573](https://github.com/nf-core/rnaseq/pull/1573) - Fix salmon.merged.SummarizedExperiment.rds name collision
23+
- [PR #1585](https://github.com/nf-core/rnaseq/pull/1585) - Update awsfulltest.yml to restore aligner-wise outputs
24+
- [PR #1580](https://github.com/nf-core/rnaseq/pull/1580) - Template update for nf-core/tools v3.3.2
25+
- [PR #1590](https://github.com/nf-core/rnaseq/pull/1590) - Addition of Sentieon STAR
26+
- [PR #1594](https://github.com/nf-core/rnaseq/pull/1594) - Exclude star rsem pca from snaps
27+
- [PR #1595](https://github.com/nf-core/rnaseq/pull/1595) - Exclude unstable star_rsem clusterings from snaps
28+
29+
### Software dependencies
30+
31+
| Dependency | Old version | New version |
32+
| ---------- | ----------- | ----------- |
33+
| `MultiQC` | 1.29 | 1.30 |
34+
| `Sentieon` | | 202503.01 |
35+
36+
## [[3.19.0](https://github.com/nf-core/rnaseq/releases/tag/3.19.0)] - 2025-06-10
737

838
### Credits
939

@@ -15,6 +45,7 @@ Special thanks to the following for their contributions to the release:
1545
- [Ben Sherman](https://github.com/bentsherman)
1646
- [Dave Carlson](https://github.com/davidecarlson)
1747
- [Gabriel Lichtenstein](https://github.com/glichtenstein)
48+
- [Jonathan Manning](https://github.com/pinin4fjords)
1849
- [Lorenzo Fontana](https://github.com/fntlnz)
1950
- [Matthias Hörtenhuber](https://github.com/mashehu)
2051
- [Milos Micik](https://github.com/milos7250)
@@ -41,7 +72,7 @@ Special thanks to the following for their contributions to the release:
4172
- [PR #1565](https://github.com/nf-core/rnaseq/pull/1565) - Improve reproducibility with Conda
4273
- [PR #1567](https://github.com/nf-core/rnaseq/pull/1567) - Prerelease 3.19.0 fixes
4374

44-
# 3.18.0 - 2024-12-19
75+
## [[3.18.0](https://github.com/nf-core/rnaseq/releases/tag/3.18.0)] - 2024-12-19
4576

4677
### Credits
4778

0 commit comments

Comments
 (0)