-
Notifications
You must be signed in to change notification settings - Fork 735
Description
Bug report
If using nf-schema, Nextflow will determine an Azure path to be a directory if it's missing a trailing slash:
ERROR ~ Validation of pipeline parameters failed!
-- Check '.nextflow.log' file for details
The following invalid input values have been detected:
* --outdir (az://pipeline/output): 'az://pipeline/output' is not a directory, but a file
-- Check script '/home/username/.nextflow/assets/nf-core/rnaseq/subworkflows/nf-core/utils_nfschema_plugin/main.nf' at line: 39 or see '.nextflow.log' file for more details
Expected behavior and actual behavior
nf-schema checks Nextflow.file.directory
for whether a file is a directory, but for Azure this is inaccurate because it's just checking for the trailing slash:
nextflow/plugins/nf-azure/src/main/nextflow/cloud/azure/nio/AzFileAttributes.groovy
Line 61 in a5756da
directory = client.blobName.endsWith('/') |
For most situations, this is fine but when using nf-schema this becomes an issue because it halts the pipeline.
Instead, we should be able to have a more robust way of checking if the file is a directory on Azure, it's probably possible using the SDKs.
I will add a bypass for nf-schema.
Steps to reproduce the problem
main.nf
:
// Print the parameters for inspection
log.info """
Parameters:
s3_file: ${params.s3_file ?: 'not set'}
gs_file: ${params.gs_file ?: 'not set'}
az_file: ${params.az_file ?: 'not set'}
"""
workflow {
println "Cloud path validation example completed successfully!"
}
nextflow_schema.json
:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/nf-core/nf-schema/master/examples/cloudpath/nextflow_schema.json",
"title": "Cloud path validation example",
"description": "Example demonstrating cloud path validation",
"type": "object",
"properties": {
"s3_file": {
"type": "string",
"format": "file-path",
"description": "An S3 file path"
},
"gs_file": {
"type": "string",
"format": "file-path",
"description": "A Google Cloud Storage file path"
},
"az_file": {
"type": "string",
"format": "file-path",
"description": "An Azure Blob Storage file path"
}
}
}
Program output
(Copy and paste the output produced by the failing execution. Please highlight it as a code block. Whenever possible upload the .nextflow.log
file.)
Environment
- Nextflow version: 25.04.7 build 5955
- nf-schema version: 2.5.1
- Java version: openjdk version "17.0.3" 2022-04-19
- Operating system: macOS
- Bash version: zsh
Additional context
Slack thread: https://nextflow.slack.com/archives/C02TPRRLCF4/p1758724547528979?thread_ts=1727918072.021049&cid=C02TPRRLCF4
nf-schema issue: nextflow-io/nf-schema#16