End-to-End Walkthrough

This guide walks through a complete verification pipeline - from installing the SDK to exporting rewards for RL training.

1. Install

pip install vrdev

2. Run a Single Verifier

from vrdev import get_verifier, VerifierInput

v = get_verifier("vr/code.python.tests_pass")
result = v.verify(VerifierInput(
    completions=["def add(a, b): return a + b"],
    ground_truth={"repo": ".", "test_cmd": "pytest tests/"},
))

print(result[0].passed)    # True
print(result[0].score)     # 1.0
print(result[0].evidence)  # {"stdout": "...", "exit_code": 0}

3. Compose a Pipeline

Chain HARD and SOFT verifiers. HARD verifiers act as gates - if they fail, SOFT scores are discarded via fail_closed.

from vrdev import compose
from vrdev.core.types import PolicyMode

pipeline = compose(
    [
        get_verifier("vr/code.python.lint_ruff"),    # HARD gate
        get_verifier("vr/code.python.tests_pass"),   # HARD gate
    ],
    policy_mode=PolicyMode.FAIL_CLOSED,
)

result = pipeline.verify(VerifierInput(
    completions=["def add(a, b): return a + b"],
    ground_truth={"repo": ".", "test_cmd": "pytest"},
))

print(result[0].passed)        # True only if ALL gates pass
print(result[0].score)         # Aggregated score
print(result[0].breakdown)     # Per-verifier breakdown

4. Use Adversarial Fixtures

Every verifier ships with adversarial fixtures that test edge cases.

vr test --verifier code.python.tests_pass --type adversarial

Adversarial fixtures include cases like empty submissions, unicode manipulation, path traversal attempts, and outputs that look correct but aren't.

5. Export for RL Training

from vrdev import export_to_trl

# Verify a batch of episodes
results = [pipeline.verify(episode) for episode in episodes]

# Export to TRL reward format
export_to_trl(results, output="rewards.jsonl")

The exported file contains one JSON line per episode with score, passed, and evidence_hash fields - ready to use as rewards in TRL, VERL, or OpenClaw training loops.

6. Verify via the Hosted API

If you prefer HTTP over local execution:

curl -X POST https://api.vr.dev/v1/verify \
  -H "Authorization: Bearer vr_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "verifier_id": "vr/code.python.tests_pass",
    "completions": ["def add(a, b): return a + b"],
    "ground_truth": {"repo": ".", "test_cmd": "pytest"}
  }'

The hosted API adds evidence anchoring, tamper-proof audit trails, and team dashboards on top of the same verification logic.