Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions library-skills/parabricks/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: >-
license: CC-BY-4.0 AND Apache-2.0
metadata:
author: "Angel Pizarro <apizarro@nvidia.com>"
version: "1.3.0"
version: "1.4.0"
tags:
- parabricks
- genomics
Expand Down Expand Up @@ -85,9 +85,9 @@ clinical interpretation, or promise exact runtime without benchmark data.

## Instructions

1. Identify the requested release. The option references cover v4.7.1; examples
1. Identify the requested release. The option references cover v4.8.0; examples
from another release are labeled. A software version and image tag can
differ: NVIDIA's 4.7.1 image is tagged `4.7.1-1`. Preserve the user's verified
differ: NVIDIA's 4.8.0 image is tagged `4.8.0-1`. Preserve the user's verified
image selection; use [command-conventions.md](references/command-conventions.md)
for the shared launch and validation contract.
2. Classify the request:
Expand Down Expand Up @@ -209,3 +209,5 @@ the `pbrun` portion; the shared container envelope is documented once in
<https://docs.nvidia.com/clara/parabricks/get-started>
- Overview:
<https://docs.nvidia.com/clara/parabricks/about-parabricks>
- 4.8.0 release notes (changes from 4.7.1):
<https://docs.nvidia.com/clara/parabricks/v4.8.0/about-parabricks/release-notes/4-8-0-1-release-notes>
157 changes: 131 additions & 26 deletions library-skills/parabricks/evals/evals.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion library-skills/parabricks/evals/files/rna-batch.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"parabricks_version": "4.7.1",
"parabricks_version": "4.8.0",
"reference_assets": "The FASTA and STAR library use the same reference sequences and annotation; the library was built by STAR 2.7.2a. Input data paths are deployment paths, not local files to probe.",
"jobs": [
{
Expand Down
2 changes: 1 addition & 1 deletion library-skills/parabricks/evals/files/rna-contracts.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"parabricks_version": "4.7.1",
"parabricks_version": "4.8.0",
"reference_assets": "The FASTA and STAR library use the same reference sequences and annotation; the library was built by STAR 2.7.2a. Input data paths are deployment paths, not local files to probe.",
"jobs": [
{
Expand Down
14 changes: 8 additions & 6 deletions library-skills/parabricks/references/command-conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ their Tools column for the selected command.

Tool examples give the command inside the container. Add one Docker envelope
only when requested; preserve a requested bare `pbrun` command or argument array.
Use an exact NVIDIA release tag (4.7.1 uses `4.7.1-1`), never `latest` or an
Use an exact NVIDIA release tag (4.8.0 uses `4.8.0-1`), never `latest` or an
implicit tag. For reproducible execution, resolve that release to its trusted
NGC manifest digest and record the digest with the run. An image's local
`RepoDigests` after an authorized pull can identify the downloaded manifest;
Expand Down Expand Up @@ -48,11 +48,11 @@ consent rules. Preparing a workload command does not authorize executing it.

## Translating options

The mapping tables primarily describe Parabricks 4.7.1. Check each selected
release's NVIDIA tool manual before finalizing syntax, defaults, optional flags,
input modes, or output naming. An upstream option without a documented mapping
has no assumed Parabricks equivalent; do not pass it through or borrow an option
from another `pbrun` tool. Flag availability does not establish biological parity.
Check each selected release's NVIDIA tool manual before finalizing syntax,
defaults, optional flags, input modes, or output naming. An upstream option
without a documented mapping has no assumed Parabricks equivalent; do not pass
it through or borrow an option from another `pbrun` tool. Flag availability
does not establish biological parity.
In option tables, **—** in the baseline column denotes a Parabricks-only control.
If an upstream option is absent from both the tool-specific table and its
scoped rows in `shared-options.md`, treat it as unmapped until the selected
Expand Down Expand Up @@ -106,6 +106,8 @@ or outputs. For example, `rna_fq2bam` needs both `--output-dir` for generated
STAR outputs and `--out-bam` for the final BAM. A Docker output mount does not
supply either argument. Retain explicit placeholders when a required path is
unknown. Check this from the command text without launching a container.
Appending `--dry-run` has `pbrun` validate the arguments and input paths
without a GPU; it is still a container launch that needs authorization.

Before execution, check that every input and required index resolves through
the mounts and that reference, known-sites, model, and graph resources are
Expand Down
2 changes: 1 addition & 1 deletion library-skills/parabricks/references/pbrun-bam2fq.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

## Preserving reads during conversion

For Parabricks 4.7.0, paired reads use `--out-suffixF` and `--out-suffixF2`.
Paired reads use `--out-suffixF` and `--out-suffixF2`.
Orphan first mates, orphan second mates, and unpaired reads are **ignored**
unless `--out-suffixO`, `--out-suffixO2`, and `--out-suffixS`, respectively, are
supplied. A single unpaired output does not capture both orphan categories.
Expand Down
3 changes: 2 additions & 1 deletion library-skills/parabricks/references/pbrun-deepsomatic.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ selected version before finalizing.

## Gotchas

Parabricks DeepSomatic `--gvcf` option actually produces both .g.vcf and .vcf files with the same name. Do not generate separate commands for gvcf and vcf outputs.
Parabricks `deepsomatic` has no `--gvcf` option; write the VCF with
`--out-variants`.

## DeepSomatic Option Mapping

Expand Down
7 changes: 4 additions & 3 deletions library-skills/parabricks/references/pbrun-deepvariant.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,12 @@ version.
Parabricks DeepVariant `--gvcf` option actually produces both .g.vcf and .vcf files with the same name. Do not generate separate commands for gvcf and vcf outputs.

The small model is enabled by default. Do not add `--enable-small-model`; that
flag is not in the current tool reference. For WES, pass `--disable-small-model`
together with `--use-wes-model` when matching Google DeepVariant.
flag is not in the current tool reference. For WES, `--use-wes-model` also
disables the small model, matching Google DeepVariant; in 4.7.1, also pass
`--disable-small-model`.

With both `--proposed-variants` and `--gvcf`, Parabricks warns and ignores
`--run-partition`; the [4.7.1 manual](https://docs.nvidia.com/clara/parabricks/tool-reference/tools/deepvariant)
`--run-partition`; the [manual](https://docs.nvidia.com/clara/parabricks/tool-reference/tools/deepvariant)
also warns of a slowdown for that combination. Preserve the requested candidate
input and gVCF output, and omit the ignored partition flag. Removing it does not
prove a speedup, and increasing GPU streams cannot restore partitioning. Measure
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ Use this when translating BWA-MEM plus Google DeepVariant germline workflows to
`pbrun deepvariant_germline`. Preprocessing and shared DeepVariant mappings for
`deepvariant_germline` are in [shared-options.md](shared-options.md). The small
model is enabled by default; use `--disable-small-model` to turn it off. For WES,
pass `--disable-small-model` with `--use-wes-model` to match Google DeepVariant.
`--use-wes-model` also disables it, matching Google DeepVariant; in 4.7.1, also
pass `--disable-small-model`.

| Baseline option | `pbrun deepvariant_germline` equivalent | Notes |
| --- | --- | --- |
Expand Down
1 change: 1 addition & 0 deletions library-skills/parabricks/references/pbrun-fq2bam.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ markdup/BQSR, intervals, GPU write controls, and similar) are in
| Single-end FASTQ input | `--in-se-fq` | Single-end FASTQ input. |
| FASTQ manifest or file list | `--in-fq-list`, `--in-se-fq-list` | Parabricks manifest; no direct BWA-MEM flag. |
| `bwa mem -t` | `--bwa-cpu-thread-pool`, `--num-cpu-threads-per-stage`, `--bwa-primary-cpus` | Partial; CPU controls split across stages. |
| `bwa mem -SP5M` for Hi-C | `--bwa-options="-SP5M"` | Add other supported BWA-MEM options to the same string. |
| `bwa mem` alignment-only output | `--align-only` | Stops after BWA-MEM; no sort/markdup. |
| GATK/Picard `SortSam --SORT_ORDER coordinate` | Default `fq2bam` behavior | Coordinate sorting is part of the normal workflow. |
| GATK/Picard `MarkDuplicates -I` | Implicit sorted intermediate | Sorted alignments feed duplicate marking unless disabled. |
Expand Down
8 changes: 4 additions & 4 deletions library-skills/parabricks/references/pbrun-germline.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
4. For preprocessing-only questions, route to
`pbrun-fq2bam.md` and related FASTQ/BAM references; for runtime readiness, use
`runtime-environment.md`.
For prepared BAM/CRAM input, use `pbrun-haplotypecaller.md`; the 4.7.1
germline pipeline does not document `--in-bam`.
For prepared BAM/CRAM input, use `pbrun-haplotypecaller.md`; the germline
pipeline has no `--in-bam` option.

## Command Shape

Expand All @@ -42,7 +42,7 @@ markdup/BQSR, intervals, and shared execution controls) for `germline` are in
| GATK `HaplotypeCaller --output`, `-O` | `--out-variants` | VCF/gVCF output. |
| GATK `HaplotypeCaller --emit-ref-confidence GVCF` | `--gvcf` | gVCF mode. |
| GATK `HaplotypeCaller --sample-ploidy` | `--ploidy` | Haploid/diploid where documented. |
| GATK `HaplotypeCaller` annotation/output-mode/pruning controls | `--haplotypecaller-options` | Pass supported HaplotypeCaller options as one string. |
| GATK `HaplotypeCaller` annotation/output-mode/pruning controls | `--haplotypecaller-options` | Pass supported HaplotypeCaller options as one string. `AS_*` annotations require `--gvcf`. |
| `--java-options`, GATK engine/common flags | No direct equivalent | Not exposed as GATK engine controls. |

## germline Options Without BWA-MEM/GATK Equivalents
Expand All @@ -54,7 +54,7 @@ Shared Parabricks pipeline and GPU controls for `germline` are in
| --- | --- |
| `--run-partition` | Genome partitions for HaplotypeCaller (one process per partition). Not fq2bam GPU-process partitioning. |
| `--bwa-gpu-num-per-partition` | BWA GPUs per worker when `--run-partition` is set (default: 2; must divide `--num-gpus`). |
| `--num-streams-per-gpu` | HaplotypeCaller streams (default: 1). Not a partition flag. |
| `--num-streams-per-gpu` | HaplotypeCaller streams (default: `auto`). Not a partition flag. |

## Validation

Expand Down
20 changes: 18 additions & 2 deletions library-skills/parabricks/references/pbrun-giraffe.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,29 @@ pbrun giraffe \
Always verify exact graph resource flags and output options against the selected
version before finalizing. Do not guess graph/index file combinations.

For **4.7.0**, the GBZ graph, distance index, minimizer index, and zipcodes file
The GBZ graph, distance index, minimizer index, and zipcodes file
are required inputs. Supply `--zipcodes-name` explicitly; do not infer a
compatible zipcodes file from a minimizer filename. Use `--ref-paths` when the
output BAM needs a supplied path order or sequence dictionary.

Giraffe uses vg v1.75.1: generate index files with vg v1.75.1, regenerating any
built with an older vg, and supply a GBZ version 1 graph. vg 1.72.0 and later
write version 2 by default, which fails to load; convert it with
`vg gbwt -Z graph.gbz -g graph-v1.gbz --gbz-v1` (vg 1.74.0 or later). Indexes
built from the same graph stay valid after that conversion. In 4.7.1, giraffe
uses vg v1.70.0; build its indexes with that version.

To produce a **coordinate-sorted BAM without duplicate marking**, use
`--no-markdups`. `--align-only` stops before coordinate sorting and therefore
does not meet that output requirement. Preserve the supplied sample and all
read-group fields using the Giraffe-specific flags in the mapping below.

An `--out-bam` path ending in `.cram` writes CRAM. It requires a
`--ref` FASTA that matches the graph and the paths in `--ref-paths`, cannot be
combined with `--align-only`, and `--run-ref-verification` is recommended the
first time a reference is used. Dropping `--align-only` for CRAM also turns on
duplicate marking; add `--no-markdups` if duplicates should stay unmarked.

## Performance Guidance

Prefer the documented automatic stream selection for general commands: leave
Expand Down Expand Up @@ -73,8 +86,9 @@ equivalent.
| `vg giraffe` fragment length controls | `--max-fragment-length`, `--fragment-mean`, `--fragment-stdev` | Same fragment-distribution role. |
| `vg giraffe --max-multimaps`, `-M` | `--max-multimaps` | Max alignments per read (default: 1). |
| `vg giraffe --copy-comment` | `--copy-comment` | Appends FASTQ comment to BAM output via auxiliary tag. |
| `vg giraffe --supplementary` | `--supplementary` | Identify and report supplementary alignments (off by default). |
| Alignment-only output | `--align-only` | Stops after `vg giraffe` alignment output; does not coordinate-sort. |
| GATK/Picard `SortSam --OUTPUT`, `-O` | `--out-bam` | Final BAM output path. |
| GATK/Picard `SortSam --OUTPUT`, `-O` | `--out-bam` | Final BAM output path, or CRAM for a `.cram` path. |
| GATK/Picard duplicate-marking behavior | `--markdups-*`, `--optical-duplicate-pixel-distance`, `--no-markdups` | Parabricks exposes the duplicate-marking controls documented for this pipeline. |
| GATK/Picard `--version` | `--version` | Same version-reporting role. |
| Upstream `vg giraffe` options not listed here | No direct equivalent | Not exposed by current Parabricks docs for `pbrun giraffe`. |
Expand All @@ -88,6 +102,8 @@ equivalent.
| `--nstreams`, `--num-cpu-threads-per-gpu`, `--batch-size`, `--write-threads`, `--work-queue-capacity` | Parabricks GPU/CPU pipeline scheduling controls. Prefer `--nstreams auto` for default guidance; use integer stream counts only for benchmarked/manual tuning. |
| `--minimizers-gpu` | Parabricks GPU offload for minimizers/seeds in supported single-end runs. |
| `--memory-limit`, `--low-memory` | Parabricks host/GPU memory controls for the pipeline. |
| `--in-se-cram`, `--in-cram-ref`, `--cram-read-threads` | Single-end CRAM input; aligned reads are re-aligned and need `--in-cram-ref` to decode. Paired-end CRAM is not supported. |
| `--ultima-tags`, `--tp-output-order` | Carry Ultima per-base `tp`/`t0` tags from `--in-se-cram` input to the output. |

## Validation

Expand Down
9 changes: 6 additions & 3 deletions library-skills/parabricks/references/pbrun-haplotypecaller.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,11 @@ GPU-accelerated HaplotypeCaller counterpart, but the CLI is not one-to-one:
some GATK flags are direct Parabricks flags, while other supported original
HaplotypeCaller options must be passed through `--haplotypecaller-options`.
Use the spelling documented for that inner parser, which differs from GATK's
top-level CLI for some options. For example, 4.7.1 accepts one quoted value
`'-min-pruning 4 -pcr-indel-model NONE'`; those are single-hyphen inner flags.
top-level CLI for some options. For example, the inner parser takes one quoted
value `'-min-pruning 4 -pcr-indel-model NONE'`; those are single-hyphen inner
flags. Allele-specific (`AS_*`) annotations, whether from `-G` or from `-A`
inside `--haplotypecaller-options`, require `--gvcf`; Parabricks rejects them
during argument checking.
`--in-recal-file` applies the supplied recalibration table during calling;
creating that table earlier did not itself rewrite the input BAM. A separate
ApplyBQSR BAM is unnecessary when using this caller input.
Expand Down Expand Up @@ -66,7 +69,7 @@ ApplyBQSR BAM is unnecessary when using this caller input.
| `--adaptive-pruning` | Parabricks-exposed graph pruning control. |
| `--force-call-filtered-alleles` | Parabricks force-calling behavior tied to its documented allele input. |
| `--filter-reads-too-long`, `--no-alt-contigs` | Parabricks read/contig filtering conveniences. |
| `--htvc-low-memory`, `--num-htvc-threads`, `--num-streams-per-gpu`, `--run-partition`, `--gpu-num-per-partition` | Parabricks GPU/partition performance controls. `--num-streams-per-gpu` default 1. |
| `--htvc-low-memory`, `--num-htvc-threads`, `--num-streams-per-gpu`, `--run-partition`, `--gpu-num-per-partition` | Parabricks GPU/partition performance controls. `--num-streams-per-gpu` defaults to `auto`. |

## Validation

Expand Down
32 changes: 18 additions & 14 deletions library-skills/parabricks/references/pbrun-indexgvcf.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

## First Steps

1. Confirm the input is a GVCF that needs indexing.
1. Confirm the input is a GVCF that needs indexing. `haplotypecaller` and
`deepvariant` already write `<output>.tbi` for `.gz` output, so check for
that index before planning an `indexgvcf` run.
2. Collect required inputs:
- Input GVCF path, and whether its directory is writable inside the container.
- Where the index must end up. There is no documented output-path option; the
`.tbi` is written next to the input (see Output Naming below).
index is written next to the input (see Output Naming below).
3. Ask whether the output will be consumed by `genotypegvcf` or another tool.

## Command Shape
Expand All @@ -21,9 +23,10 @@ Verify exact flags against the selected version.

## Output Naming and Read-Only Inputs

- The index name is the input filename with `.tbi` appended
(`sample.g.vcf.gz` → `sample.g.vcf.gz.tbi`). Do not use or invent an `--out`
flag; none is documented for `indexgvcf`.
- The index name is the input filename with `.tbi` appended for a `.gz` input
(`sample.g.vcf.gz` → `sample.g.vcf.gz.tbi`), or `.idx` for an uncompressed
`.g.vcf`. Do not use or invent an `--out` flag; none is documented for
`indexgvcf`.
- If the original GVCF is on a read-only mount (for example `/data`), indexing it
in place fails. Index a **writable copy** instead and leave the original
untouched.
Expand All @@ -36,19 +39,19 @@ Verify exact flags against the selected version.
cp -p /data/<sample.g.vcf.gz> /results/<sample.g.vcf.gz>
```

Ask the user before replacing an existing GVCF or `.tbi`, and make clear the
commands are a plan until the user runs them.
For an uncompressed `.g.vcf`, check for `<sample.g.vcf>.idx` instead of the
`.tbi`. Ask the user before replacing an existing GVCF or index, and make
clear the commands are a plan until the user runs them.

## IndexFeatureFile Option Mapping

Use this mapping when translating a GATK `IndexFeatureFile` command for GVCF
indexing to `pbrun indexgvcf`. Parabricks documents `indexgvcf` as
creating a `.tbi` index by appending `.tbi` to the input GVCF filename.
indexing to `pbrun indexgvcf`.

| GATK option | `pbrun indexgvcf` equivalent | Notes |
| --- | --- | --- |
| `--input`, `-I` | `--input` | Required gVCF/gVCF.GZ input. |
| Implicit output index path | Implicit `.tbi` next to input | Parabricks docs state the output name is determined by appending `.tbi` to the input GVCF filename. |
| Implicit output index path | Implicit `.tbi` (`.gz` input) or `.idx` (`.g.vcf`) next to input | Not configurable; see Output Naming and Read-Only Inputs. |
| GATK output path override, cloud flags, `--arguments_file`, and Java options | No direct equivalent | Not exposed by current Parabricks docs for `indexgvcf`. |

## Validation
Expand All @@ -58,9 +61,10 @@ creating a `.tbi` index by appending `.tbi` to the input GVCF filename.

## Guardrails

- Do not use this for ordinary VCF indexing unless the selected version
documents that use.
- Index only GVCFs and ordinary `.vcf.gz` files; other inputs are not
documented.
- Do not run genotyping here; route to `pbrun-genotypegvcf.md`.
- Do not invent output index suffixes; verify version behavior.
- Do not overwrite an existing GVCF or `.tbi`, and do not index a read-only
original in place; use a writable copy after checking the destination.
- Do not overwrite an existing GVCF or index (`.tbi`, or `.idx` for an
uncompressed `.g.vcf`), and do not index a read-only original in place; use
a writable copy after checking the destination.
Loading
Loading