Bring Your Own State (BYOS)

Many verifiers need to check system state: a database row, an HTTP response, a file on disk. By default they query the system directly, but the pre_result pattern lets you inject pre-fetched state for testing, replay, or custom backends.

The Problem

Running verifiers against live systems during RL training adds latency, requires credentials, and creates flaky loops. You want deterministic, fast verification without external dependencies.

The pre_result Pattern

Every database, API, and file verifier accepts an optional pre_result dict in ground_truth. When present, the verifier skips the live query and uses your data directly.

Database Example

from vrdev import get_verifier, VerifierInput

v = get_verifier("vr/database.row.exists")

# Live query (requires a real database)
result = v.verify(VerifierInput(
    completions=["I inserted the user record"],
    ground_truth={
        "connection_string": "sqlite:///app.db",
        "table": "users",
        "match_columns": {"email": "alice@example.com"},
    },
))

# BYOS: inject your own state (no database needed)
result = v.verify(VerifierInput(
    completions=["I inserted the user record"],
    ground_truth={
        "table": "users",
        "match_columns": {"email": "alice@example.com"},
        "pre_result": {"exists": true},
    },
))

API Example

v = get_verifier("vr/api.http.status_ok")

# BYOS: inject a pre-fetched HTTP response
result = v.verify(VerifierInput(
    completions=["I called the API"],
    ground_truth={
        "expected_status": 200,
        "pre_result": {"status_code": 200},
    },
))

HTTP Response Body

v = get_verifier("vr/api.http.response_matches")

result = v.verify(VerifierInput(
    completions=["API returns the order"],
    ground_truth={
        "expected_substrings": ["order_id", "confirmed"],
        "pre_result": {"body": '{"order_id": "ORD-42", "status": "confirmed"}'},
    },
))

pre_result Schemas

Each verifier domain has its own pre_result shape:

| Verifier | pre_result Shape | |----------|-----------------| | database.row.exists | {"exists": bool} | | database.row.updated | {"row": {col: val, ...}} | | database.table.row_count | {"count": int} | | api.http.status_ok | {"status_code": int} | | api.http.response_matches | {"body": str} | | api.http.header_present | {"headers": {name: val}} |

When to Use BYOS

  • RL training loops: Inject state from episode logs for deterministic reward computation
  • Unit tests: Test verifier logic without standing up databases or HTTP servers
  • Replay: Re-verify historical episodes from stored evidence
  • Custom backends: Query your own systems and feed results to standard verifiers

Composing with BYOS

BYOS works seamlessly with the composition engine:

from vrdev import get_verifier, compose, VerifierInput
from vrdev.core.types import PolicyMode

chain = compose(
    [get_verifier("vr/database.row.updated"),
     get_verifier("vr/api.http.status_ok")],
    policy_mode=PolicyMode.FAIL_CLOSED,
)

result = chain.verify(VerifierInput(
    completions=["Order cancelled via API"],
    ground_truth={
        "table": "orders",
        "match_columns": {"id": 42},
        "expected_values": {"status": "cancelled"},
        "pre_result": {"row": {"status": "cancelled"}},
    },
))
← ExamplesTrust Model →
Bring Your Own State (BYOS) | vr.dev Docs