Dataflow & Produces¶

DV Flow Manager uses dataflow dependencies to connect tasks together. Each task can:

Consume input data from its dependencies
Produce output data for dependent tasks

The produces feature allows tasks to declare what output datasets they create, enabling workflow validation and task discovery.

Overview¶

When a task lists another task in its needs, it receives the output data from that dependency as input. To ensure compatibility and help with task discovery, tasks should declare:

produces - What output datasets this task creates
consumes - What input datasets this task accepts

Example:

package:
  name: my_flow
  tasks:
    # Producer declares what it outputs
    - name: VerilogCompiler
      produces:
        - type: std.FileSet
          filetype: verilog
      run: |
        # ... compile verilog files

    # Consumer declares what it needs
    - name: Simulator
      needs: [VerilogCompiler]
      consumes:
        - type: std.FileSet
          filetype: verilog
      run: |
        # ... run simulation

The validator checks that VerilogCompiler’s produces matches Simulator’s consumes, warning if there’s a mismatch.

Declaring Produces¶

Basic Declaration¶

Declare output datasets using a list of pattern dictionaries:

- name: MyTask
  produces:
    - type: std.FileSet
      filetype: verilog
  run: echo "produce verilog files"

Each pattern describes one type of output. Tasks can produce multiple types:

- name: Compiler
  produces:
    - type: std.FileSet
      filetype: verilog
    - type: std.FileSet
      filetype: verilogInclude
    - type: custom.BuildLog
      format: json

Parameter References¶

Produces patterns can reference task parameters using ${{ }} syntax:

- name: GenericProducer
  with:
    output_type:
      type: str
      value: verilog
  produces:
    - type: std.FileSet
      filetype: "${{ params.output_type }}"
  run: echo "produce ${{ output_type }} files"

The parameter reference is evaluated when the task graph is built, creating a concrete produces pattern for that task instance.

Complex Patterns¶

Produces patterns can have multiple attributes to precisely describe outputs:

- name: VendorTool
  produces:
    - type: std.FileSet
      filetype: verilog
      vendor: synopsys
      version: "2023.09"
      optimization: speed
  run: echo "vendor-specific output"

Optional Declaration¶

Declaring produces is optional. Tasks without produces are assumed to have unknown or dynamic outputs, which is acceptable for validation:

# No produces declared - that's OK
- name: DynamicTask
  run: echo "outputs vary at runtime"

# Explicitly declare no outputs
- name: SideEffectTask
  produces: []
  run: echo "only side effects, no data output"

Inheritance¶

When a task uses another task as a base, it inherits and extends the base task’s produces patterns:

- name: BaseCompiler
  produces:
    - type: std.FileSet
      filetype: verilog
  run: echo "base compilation"

- name: AdvancedCompiler
  uses: BaseCompiler
  produces:
    - type: std.FileSet
      filetype: coverage
      format: ucdb
  run: echo "advanced compilation with coverage"

AdvancedCompiler produces both:

std.FileSet with filetype: verilog (inherited from base)
std.FileSet with filetype: coverage (added by derived task)

This matches the principle that derived tasks add capabilities to base tasks.

Validation¶

The dfm validate command checks produces/consumes compatibility between connected tasks.

Compatibility Rules¶

Two tasks are compatible when:

Consumer has consumes: all - Accepts any produces
Producer has no produces - Unknown outputs assumed compatible
OR Logic - If ANY consumer pattern matches ANY producer pattern, the dataflow is valid

Pattern Matching¶

A consumer pattern matches a producer pattern when:

All attributes in the consumer pattern exist in the producer pattern
All attribute values match exactly
Producer can have additional attributes (subset matching)

Examples:

✅ Compatible - Exact Match:

# Producer
produces:
  - type: std.FileSet
    filetype: verilog

# Consumer
consumes:
  - type: std.FileSet
    filetype: verilog

✅ Compatible - Subset Match:

# Producer has extra attributes
produces:
  - type: std.FileSet
    filetype: verilog
    vendor: synopsys

# Consumer only requires some attributes
consumes:
  - type: std.FileSet
    filetype: verilog

⚠️ Warning - Mismatch:

# Producer
produces:
  - type: std.FileSet
    filetype: verilog

# Consumer needs different type
consumes:
  - type: std.FileSet
    filetype: vhdl  # Mismatch!

OR Logic Example¶

When multiple patterns exist, ANY match makes it valid:

# Producer outputs both verilog and vhdl
- name: MultiProducer
  produces:
    - type: std.FileSet
      filetype: verilog
    - type: std.FileSet
      filetype: vhdl

# Consumer accepts either
- name: FlexibleConsumer
  needs: [MultiProducer]
  consumes:
    - type: std.FileSet
      filetype: verilog  # This matches, so it's valid!
    - type: std.FileSet
      filetype: systemverilog

Warnings vs Errors¶

Dataflow mismatches generate warnings, not errors. This allows:

Flexible workflows during development
Dynamic outputs not known at definition time
Gradual adoption of produces declarations

To see validation warnings:

$ dfm validate
Package: my_flow
  Tasks: 5
  Types: 2

Warnings (1):
  WARNING: Task 'Consumer' consumes [{'type': 'std.FileSet', 'filetype': 'vhdl'}]
           but 'Producer' produces [{'type': 'std.FileSet', 'filetype': 'verilog'}].
           No consume pattern matches any produce pattern.

✓ Validation passed
  (1 warning(s))

Task Discovery¶

The show command can filter tasks by their produces patterns.

Finding Tasks by Output¶

Use --produces to find tasks that produce specific outputs:

# Find tasks that produce verilog files
$ dfm show tasks --produces "type=std.FileSet,filetype=verilog"

Task                              Description
────────────────────────────────────────────────────
my_flow.VerilogCompiler          Compiles Verilog RTL
my_flow.PreProcessor             Preprocesses sources

Viewing Task Details¶

The show task command displays produces information:

$ dfm show task VerilogCompiler

Task:        my_flow.VerilogCompiler
Package:     my_flow
Base:        -
Scope:       -

Parameters:
  optimization  str     O2    Optimization level

Produces:
  - type=std.FileSet, filetype=verilog, optimization=O2

Direct Needs:
  - my_flow.SourceFiles

Filter Syntax¶

The produces filter uses comma-separated key=value pairs:

# Single attribute
--produces "type=std.FileSet"

# Multiple attributes (all must match)
--produces "type=std.FileSet,filetype=verilog"

# Complex filter
--produces "type=std.FileSet,filetype=verilog,vendor=synopsys"

JSON Output¶

For programmatic use, add --json:

$ dfm show tasks --produces "type=std.FileSet" --json

{
  "command": "show tasks",
  "filters": {
    "produces": "type=std.FileSet"
  },
  "results": [
    {
      "name": "my_flow.VerilogCompiler",
      "package": "my_flow",
      "desc": "Compiles Verilog RTL",
      "produces": [
        {"type": "std.FileSet", "filetype": "verilog"}
      ]
    }
  ]
}

Best Practices¶

When to Declare Produces¶

✅ Do declare produces for:

Tasks with predictable, structured outputs
Tasks that produce FileSet or custom data items
Reusable library tasks
Tasks where output type matters to consumers

❌ Don’t declare produces for:

Tasks with purely side effects (no data outputs)
Tasks with highly dynamic outputs not known until runtime
Quick one-off tasks in a specific workflow

Being Specific¶

More specific produces patterns enable better validation:

# Less helpful - too generic
produces:
  - type: std.FileSet

# Better - includes key characteristics
produces:
  - type: std.FileSet
    filetype: verilog
    stage: compiled

# Best - fully describes the output
produces:
  - type: std.FileSet
    filetype: verilog
    stage: compiled
    vendor: synopsys
    optimization: speed

Documenting Outputs¶

Use task description and documentation to explain produces:

- name: AdvancedCompiler
  desc: Compiles Verilog with vendor-specific optimizations
  doc: |
    This task compiles Verilog RTL using vendor-specific tools,
    producing optimized netlists and optional coverage databases.

    Outputs:
    - Compiled Verilog netlist (optimized for speed)
    - UCDB coverage database (if coverage enabled)
  produces:
    - type: std.FileSet
      filetype: verilog
      optimization: speed
    - type: std.FileSet
      filetype: coverage
      format: ucdb

Common Patterns¶

File Transformation Pipeline¶

tasks:
  - name: PreProcess
    produces:
      - type: std.FileSet
        filetype: verilog
        stage: preprocessed
    run: echo "preprocess"

  - name: Compile
    needs: [PreProcess]
    consumes:
      - type: std.FileSet
        filetype: verilog
    produces:
      - type: std.FileSet
        filetype: verilog
        stage: compiled
    run: echo "compile"

  - name: Optimize
    needs: [Compile]
    consumes:
      - type: std.FileSet
        filetype: verilog
        stage: compiled
    produces:
      - type: std.FileSet
        filetype: verilog
        stage: optimized
    run: echo "optimize"

Multi-Output Tasks¶

- name: BuildAndAnalyze
  produces:
    - type: std.FileSet
      filetype: executable
    - type: std.FileSet
      filetype: coverage
      format: lcov
    - type: std.FileSet
      filetype: log
  run: |
    # Build creates executable, coverage data, and logs

Parameterized Outputs¶

- name: GenericBuilder
  with:
    target:
      type: str
      value: debug
    format:
      type: str
      value: elf
  produces:
    - type: std.FileSet
      filetype: executable
      target: "${{ params.target }}"
      format: "${{ params.format }}"
  run: echo "build for ${{ target }}"

Troubleshooting¶

Validation Warnings¶

Problem: Getting dataflow mismatch warnings

Solutions:

Check produces pattern matches consumes:

$ dfm show task ProducerTask
$ dfm show task ConsumerTask

Make produces more specific or consumes less specific
Verify parameter values are correct
Check task inheritance - derived tasks extend produces

Problem: Warning says “No consume pattern matches any produce pattern”

Solution: This means NONE of the consumer’s patterns match ANY of the producer’s patterns. Check for typos in attribute names or values.

Missing Produces¶

Problem: Tasks not appearing in --produces filter

Solutions:

Verify produces is declared in task definition
Check produces patterns match filter exactly
Ensure task is in the loaded package

# Debug: show all tasks to verify it exists
$ dfm show tasks

# Debug: show specific task to see its produces
$ dfm show task MyTask

Parameter Evaluation¶

Problem: Parameter references not being evaluated

Solutions:

Use correct syntax: ${{ params.name }} not ${{ name }}
Verify parameter is defined in task’s with section
Check parameter has a value (default or override)

- name: MyTask
  with:
    output_type:
      type: str
      value: verilog  # Must have a value!
  produces:
    - type: std.FileSet
      filetype: "${{ params.output_type }}"

Dataflow & Produces¶

Overview¶

Declaring Produces¶

Basic Declaration¶

Parameter References¶

Complex Patterns¶

Optional Declaration¶

Inheritance¶

Validation¶

Compatibility Rules¶

Pattern Matching¶

OR Logic Example¶

Warnings vs Errors¶

Task Discovery¶

Finding Tasks by Output¶

Viewing Task Details¶

Filter Syntax¶

JSON Output¶

Best Practices¶

When to Declare Produces¶

Being Specific¶

Documenting Outputs¶

Common Patterns¶

File Transformation Pipeline¶

Multi-Output Tasks¶

Parameterized Outputs¶

Troubleshooting¶

Validation Warnings¶

Missing Produces¶

Parameter Evaluation¶

See Also¶

DV Flow Manager

Navigation

Related Topics