CLI Reference

Installation

pip install vrdev

Authentication

export VR_API_KEY=vr_live_...

Commands

vr verify

Run a single verifier.

vr verify --verifier <id> --ground-truth '<json>'

| Flag | Description | |------|-------------| | --verifier | Verifier ID (e.g., code.python.tests_pass) | | --ground-truth | JSON string with input data | | --config | Optional JSON config overrides | | --output | Output format: json, table, minimal |

vr compose

Run a pipeline of verifiers.

vr compose \
  --verifiers tau2.retail.order_cancelled,rubric.email.tone_professional \
  --ground-truth '{"order_id": "ORD-42"}' \
  --policy fail_closed

| Flag | Description | |------|-------------| | --verifiers | Comma-separated list of verifier IDs | | --policy | fail_closed (default), fail_open, escalation, or ensemble | | --ground-truth | JSON string with input data |

vr batch

Run verifier against multiple inputs.

vr batch --verifier tau2.retail.order_cancelled --input batch.jsonl

vr list

List available verifiers.

vr list
vr list --tier HARD
vr list --domain code_quality

vr test

Run fixtures against a local verifier (for authoring).

vr test --verifier my_org.domain.verifier_name
vr test --verifier my_org.domain.verifier_name --type adversarial

vr export

Export results to training formats.

vr export --run run_abc123 --format trl --output rewards.jsonl