Script ↔ Dataflow I/O

Shell tasks (shell: bash and friends) exchange data with the dataflow graph through a GitHub-Actions-style contract: environment variables carry inputs into the script, and append-only files carry outputs back to the graph. The files are located by DFM_* environment variables that the runner sets, so a script just appends to $DFM_OUTPUT (etc.) and the runner parses the file after the process exits.

This is fully backward compatible: ${{ }} parameter substitution still works, and a script that ignores all of the channels below behaves exactly as before.

Inputs the runner provides

Env var	Kind	Contents
DFM_RUNDIR	path	Task run directory (alias: TASK_RUNDIR)
DFM_SRCDIR	path	Task source directory (alias: TASK_SRCDIR)
DFM_TASK_NAME	string	Fully-qualified task name
DFM_PARAM_<NAME>	string	Each declared param. Scalars verbatim; list/map as compact JSON; booleans as true/false.
DFM_PARAMS	path	JSON file: { "<name>": <value>, ... } — full structured params
DFM_INPUTS	path	JSON file: [ <data item>, ... ] — every consumed input item
DFM_MEMENTO	path	JSON file of the prior memento (absent on the first run)

DFM_INPUTS is a path to a file rather than inline JSON so that large filesets cannot blow the environment-size limit. Scalar params are also inlined as DFM_PARAM_* for the common, ergonomic case.

Outputs the script provides

All output files are pre-created empty, so a script can blindly >> to them.

Env var	Format	Becomes
DFM_OUTPUT	JSONL — one data-item object per line	output items (filesets, etc.)
DFM_ENV	KEY=VALUE lines (+ heredoc form)	a std.Env item (vals) → downstream env
DFM_PATH	one directory per line	a std.Env item (prepend_path) → downstream PATH
DFM_MARKERS	JSONL of {severity, msg, loc?}	markers
DFM_MEMENTO_OUT	JSON object (write)	memento for the next run’s up-to-date check
exit code	integer	status (authoritative — markers never flip it)

File formats

DFM_OUTPUT — typed data items (JSONL). Each line is one item; type is required. A std.FileSet with basedir: "." is rewritten to the rundir:

echo '{"type":"std.FileSet","filetype":"verilogSource","basedir":".","files":["gen.sv"]}' >> "$DFM_OUTPUT"

DFM_ENV — GHA-compatible key/value, including the multiline heredoc form:

echo "SIM_SEED=42" >> "$DFM_ENV"
{ echo "BANNER<<EOF"; cat banner.txt; echo "EOF"; } >> "$DFM_ENV"

The runner folds the accumulated keys (and DFM_PATH directories) into a single std.Env item, so they flow downstream through the same env-merging path that std.Env inputs already use.

The dfm-out helper

Hand-writing JSON in bash is the rough edge of this design. The bundled dfm-out console script writes the files for you (it reads the DFM_* paths from its own environment, and is placed on PATH for shell tasks):

dfm-out fileset --filetype verilogSource gen.sv pkg.sv   # → $DFM_OUTPUT
dfm-out env  SIM_SEED=42                                  # → $DFM_ENV
dfm-out path /opt/tools/bin                               # → $DFM_PATH
dfm-out error "synthesis failed" --file top.sv --line 10  # → $DFM_MARKERS
dfm-out item --type my_pkg.Report key=val n:=3           # n:= ⇒ JSON-typed value

python -m dv_flow.mgr.out is an equivalent fallback. Scripts may always fall back to raw echo >> "$DFM_OUTPUT".

Worked example

- name: GenRtl
  shell: bash
  consumes: [{type: std.FileSet}]
  produces: [{type: std.FileSet}]
  with:
    top:   {type: str, value: "soc_top"}
    seeds: {type: list, value: [1, 2, 3]}
  run: |
    echo "Generating for $DFM_PARAM_TOP"          # scalar param, inline
    echo "seeds JSON: $DFM_PARAM_SEEDS"           # list param as JSON

    # Read upstream filesets structurally
    jq -r '.[] | select(.type=="std.FileSet") | .files[]' "$DFM_INPUTS" > srclist.txt

    python gen.py --top "$DFM_PARAM_TOP" --srcs srclist.txt --out gen.sv

    # Hand results back to the graph
    dfm-out fileset --filetype verilogSource gen.sv
    dfm-out env GEN_OK=1
    [ -s gen.sv ] || dfm-out error "generator produced empty output"

Downstream tasks that consumes: std.FileSet automatically see gen.sv; downstream shell tasks see GEN_OK in their environment.

Coming from GitHub Actions

GitHub Actions	DV Flow
$GITHUB_OUTPUT	$DFM_OUTPUT (typed JSONL, not key=value)
$GITHUB_ENV	$DFM_ENV
$GITHUB_PATH	$DFM_PATH
INPUT_<NAME>	DFM_PARAM_<NAME>
$GITHUB_STEP_SUMMARY	$DFM_MARKERS (structured markers)

Note

TASK_SRCDIR / TASK_RUNDIR remain as aliases for one release and are then removed in favor of DFM_SRCDIR / DFM_RUNDIR.