Support Nextflow typed syntax for process inputs/outputs#4
Merged
Conversation
Add parsing support for Nextflow's typed I/O declarations
(nextflow.preview.types = true) which use a different syntax than
traditional DSL2:
- Typed tuple inputs: (meta, bam): Tuple<Map, Path>
- Typed simple inputs: x: Integer, bam: Path
- Named assignment outputs: txt = tuple(meta, file("*.txt"))
- Topic sections: topic:/>> 'name' as section delimiters
Also adds comprehensive test suite for nf_parser module (53 tests)
covering both traditional and typed syntax patterns.
Fixes issue where pipelines using typed syntax would generate
documentation with no inputs or outputs listed.
The Nextflow LSP returns bare emit names (e.g. 'txt', 'bam') as outputs for typed processes, rather than qualified declarations. Add handling for these bare identifier outputs in _parse_single_output. Also add tests for ? type in typed tuples (LSP representation of Map type) and integration tests matching real LSP hover output from nf-fgsv pipeline. Verified against fulcrumgenomics/nf-fgsv: all 4 processes now correctly show both inputs and outputs.
When the Nextflow LSP returns only bare emit names for typed process
outputs (e.g. 'txt', 'bam'), read the actual .nf source file to get
the full output declarations (e.g. txt = tuple(meta, file("*.txt"))).
- Add enrich_outputs_from_source() in nf_parser.py to parse the
source output block and replace bare outputs with rich declarations
- Add _parse_tuple_components() to handle mixed qualified/bare tuple
elements (e.g. tuple(meta, file("*.txt")) -> val(meta), file(...))
- Call enrichment from extractor when bare outputs are detected
- Add 6 tests for the enrichment function
Verified against nf-fgsv: outputs now show val(meta), file("*") with
type=tuple and emit names, matching the quality of typed inputs.
README headings like '## Inputs' generated id='inputs', colliding
with the page-level section id='inputs' used for Parameters navigation.
When JavaScript called getElementById('inputs'), it found the README
heading first (earlier in DOM), leaving the Parameters section hidden.
Fix by prefixing all README heading IDs with 'readme-' (e.g.
'readme-inputs'), keeping section IDs intact for navigation.
The pattern field from nextflow_schema.json was already included in JSON, YAML, and Markdown output but was missing from the HTML template. Add it after the enum block in the parameter card.
- Parse @param and @return tags from LSP hover and apply descriptions to matching ProcessInput/ProcessOutput objects - Support named @return tags (@return txt Description) so each output gets its own description instead of sharing a single _return key - Fall back to reading Groovydoc directly from .nf source files when the LSP returns no param docs (common with typed processes) - Support both @param/@return and Inputs:/Outputs: bullet-list formats - Handle Groovydoc not immediately adjacent to process declaration (e.g. nextflow.preview.types = true between comment and process) - Render all LSP-sourced descriptions through |markdown filter in HTML (consistent with meta.yml descriptions) - Move pattern field from a separate block to an inline yellow badge in the parameter card, with dark mode support Adds 8 tests for Groovydoc parsing and param description matching.
Fulcrum Genomics structural variant calling pipeline, demonstrating typed Nextflow syntax (nextflow.preview.types) with Groovydoc support.
Read .nf source files once per process instead of twice (once for Groovydoc parsing, once for output enrichment). Extract duplicated section-boundary regexes and _return_ key prefix strings as shared constants to keep parsers in sync. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…parsing # Conflicts: # docs/examples.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for Nextflow's typed syntax (
nextflow.preview.types = true) which was causing processes to generate documentation with no inputs, no outputs, and no Groovydoc descriptions.Reported via community feedback on fulcrumgenomics/nf-fgsv.
Changes
Typed process I/O parsing (
nf_parser.py)(meta, bam): Tuple<Map, Path>andTuple<?, Path>(LSP?for Map)x: Integer,bam: Pathtxt,bam).nfsource files to recover full declarations (e.g.txt = tuple(meta, file("*.txt")))tuple(meta, file("*.txt")))topic:as a section delimiter so topic blocks don't bleed into output parsingeach,val, etc.) being misparsed as typed inputsGroovydoc support (
extractor.py,lsp_client.py)@param/@returntag descriptions to process inputs and outputs@returntags (@return txt Description) so each output gets its own description.nfsource files when the LSP omits it (common with typed processes)@param/@returntags andInputs:/Outputs:bullet-list Groovydoc formatsnextflow.preview.types = truebetween comment and process)HTML template fixes (
html.html,html.py)## Inputsgeneratedid="inputs", shadowing the<section id="inputs">used for navigation. Prefix all README heading IDs withreadme-patternrestriction as an inline yellow badge in parameter cards (with dark mode support)|markdownfilter (process inputs, outputs, workflow I/O, function params) for consistent formattingTests
tests/test_nf_parser.pycovering traditional and typed syntaxtests/test_extractor.pyfor Groovydoc parsing and param description matchingExamples