SDK: runner and manifest

The harness that runs a workflow, the objects that describe one on disk, and the comparison that turns method outputs into numbers. The platform calls the same functions inside its sandbox. Back to the SDK reference.

qsolve.runner

from qsolve.runner import RunOutput, estimate_run, run_workflow, validate_input

__all__ of the module: RunOutput, WorkflowError, estimate_run, run_workflow, validate_input. estimate_run and run_workflow are also exported from qsolve.

run_workflow

run_workflow(
    workflow_dir: str,
    input: dict[str, Any],
    *,
    target: str = "sim-cpu",
    params: dict[str, Any] | None = None,
    seed: int = 0,
    backend: Backend | None = None,
    commit_sha: str | None = None,
    image_digest: str | None = None,
    on_event: Sink | None = None,
) -> RunOutput

Runs every method the manifest declares on one input, measures everything and builds the result. This is what qsolve run and the platform call. Who calls it: the CLI, the sandbox entry point, run_benchmarks, and your own tests.

Returns a RunOutput.

Raises WorkflowError for: invalid parameters (unknown parameters: [...], parameter 'x' must be <= 6.0), input that does not match the schema (Invalid input at <path>: <message>), an unusable target or backend (unknown target, no QSOLVE_IBM_TOKEN, missing ibm extra, unknown fake device, noise scale outside 1 to 10), a class that cannot be imported or is not a Workflow subclass, a constructor failure, an input above the envelope, a quantum or reference method that produced no value (a reference whose answer() fails or breaks the output schema counts as no value), a failing answer(), or an answer that breaks the output schema (Workflow produced an invalid answer: ...). Also raises FileNotFoundError, ValueError or pydantic.ValidationError from load_workflow when the manifest or schemas are invalid (the CLI turns those into a message for validate, test, run and estimate). Exceptions that are not WorkflowError raised inside your quantum or reference method propagate, with a note raised in the quantum method '<id>'.

Side effects. Imports your workflow package (and, to keep two workflows with the same package name src apart, removes stale modules of that name from sys.modules and puts the workflow root first on sys.path). Limits BLAS to one thread for the duration of the run so summation order is fixed and the same seed gives bit-identical results on your laptop and in the sandbox. Sends circuits to the backend (on ibm: targets that spends your own quota). Emits events: {"type": "stage", "stage": "build"}, then {"type": "stage", "stage": "run", "device": ..., "target": ...}, then {"type": "stage", "stage": "compare"}, plus whatever your methods emit.

What it computes.

from qsolve import run_workflow

out = run_workflow(".", {"Q": [[-1, 2], [2, -1]]}, params={"shots": 500}, seed=1,
                   on_event=lambda e: print(e["type"], e.get("stage")))
print(out.result.metrics.matches, out.provenance.result_hash)

Gotchas. target is not checked until after parameters and input validate, so a bad target with a bad input reports the input error first. The first failure wins; nothing is partial. Do not call it from inside a workflow.

estimate_run

estimate_run(
    workflow_dir: str,
    input: dict[str, Any],
    *,
    target: str = "sim-cpu",
    params: dict[str, Any] | None = None,
) -> Estimate

Validates and sizes a run without executing any method. Used for price quotes and credit holds, and by qsolve estimate. It loads the manifest, resolves parameters, validates the input, builds your workflow class (so __init__ and size run, and the envelope is enforced) and calls estimate(target).

There is no seed or on_event parameter. The CLI exposes no --param, so qsolve estimate uses the manifest defaults; call the function directly to estimate with overrides.

validate_input

validate_input(wf: LoadedWorkflow, input: dict[str, Any]) -> None

Checks input against wf.input_schema with jsonschema.validate. Returns None when valid. Raises WorkflowError("Invalid input at <dotted.path>: <message>"), where the path is <root> for a top-level problem. Called by run_workflow, estimate_run, qsolve validate (for the examples) and run_benchmarks. If the schema itself is invalid, jsonschema.SchemaError propagates; qsolve lint reports that as E020.

RunOutput

class RunOutput(BaseModel):
    result: Result
    provenance: Provenance

What run_workflow returns. out.model_dump(mode="json") is exactly what qsolve run --out writes and what the sandbox sends as its result frame. Fields are described in Results and events.

Other module-level names

Private helpers (underscore names such as _instantiate, _context, _hash, _stable, _plain, _source, _make_importable) are not part of the API.

qsolve.manifest

from qsolve.manifest import Manifest, load_workflow, LoadedWorkflow

The parsed form of qsolve.yaml. Every model rejects unknown keys (extra="forbid"). The meaning of each manifest field, with examples, is in The manifest and Parameters; this section is the Python surface.

load_workflow

load_workflow(root: str | Path) -> LoadedWorkflow

Reads qsolve.yaml in root, validates it and reads both JSON Schema files. Returns a LoadedWorkflow. Raises:

Side effect: reads files only. It does not import your code or validate the schemas as JSON Schema.

LoadedWorkflow

Manifest

Sub-models: SchemaRef(schema_path, examples), Parameter(name, type, default, label, description, min, max, choices), ObjectiveDecl, MetricDecl, MethodDecl(id, role, title<=80, what<=500, strength<=300, limit<=300), Envelope, Targets, Resources, Runtime. MethodDecl.id follows the name rule (lowercase letters, digits, dashes, 2 to 63 characters).

Manifest.resolve_params

resolve_params(self, overrides: dict[str, Any] | None = None) -> dict[str, Any]

Returns every declared parameter: the override if given, otherwise the default, each passed through Parameter.check. Raises ValueError for unknown parameters: [...] and for any value of the wrong type or outside its bounds. Called by run_workflow and estimate_run, which convert the ValueError to WorkflowError.

Parameter.check

check(self, value: Any) -> Any

Returns value unchanged or raises ValueError. Rules: int rejects bools and non-ints; float accepts finite ints and floats but not bools, NaN or infinity; bool needs a real bool; str needs a string; enum needs a value in choices (a bool never matches the int 1); min and max apply to int and float. A Parameter itself is rejected at load when min > max, when an enum has no choices, or when min or max is NaN or infinite. Messages look like parameter 'layers' must be <= 6.0 (bounds are stored as floats, so an integer bound prints with .0).

qsolve.comparison

from qsolve.comparison import headline, implementation, run_methods

Runs the declared methods and computes the numbers. run_workflow calls it; you do not need to.

implementation

implementation(wf: Workflow, decl: MethodDecl) -> Any

The bound method of wf that implements decl: quantum for role quantum, reference for role reference, classical_<id with - replaced by _> for role classical. Raises WorkflowError("method 'x' is declared in the manifest but the workflow has no name method") when it is missing or not callable.

run_methods

run_methods(
    wf: Workflow,
    decls: list[MethodDecl],
    *,
    quantum_ctx: Context,
    classical_ctx: Context,
    recorder: Recorder,
) -> tuple[list[MethodResult], dict[str, Outcome]]

Runs every declared method, reference and classical ones first and the quantum method last, each with its own context, and returns a MethodResult per declaration (in manifest order) and the Outcome of each method that produced one (by id). For every method it sets recorder.method, times it, and:

• a MethodUnavailable makes it unavailable with your message;
• a classical method that raises anything else is unavailable with failed: <Type>: <msg>; a quantum or reference method that raises WorkflowError re-raises it, and any other exception re-raises with a note;
• a non-quantum method that sent circuits is unavailable (it sent circuits to the quantum backend, which only the quantum method may do);
• a non-finite objective or error bar makes it unavailable;
• otherwise evaluate recomputes the value (verified="recomputed") or, with no solution, the reported value is kept (verified="reported"); a difference between claimed and recomputed values larger than 1e-6 * max(1, |value|) is stored in claimed;
• answer(outcome) is attempted. For a reference or classical method, a failing answer() or an answer that breaks the output schema (passed as output_schema, which run_workflow supplies) makes that method unavailable with the reason (answer() failed: ... or its answer breaks the output schema: ...); the run goes on. For the quantum method a failure leaves answer=None here and the runner reports it (WorkflowError).

After the loop it fills vs_reference = (value - ref) / max(|ref|, 1e-12) for each method, and for sampling methods p_reference, valid_fraction and top_outcomes (the five most probable, each labelled by its label or its JSON, cut at 60 characters). p_reference counts samples whose recomputed value is at most the reference value (with a 1e-9 relative slack).

headline

headline(decl: MetricDecl, methods: list[MethodResult], quantum: Outcome) -> Metrics

The accuracy against the reference with one of three generic formulas, scale = max(abs(reference), 1e-12):

accuracy is clipped to 0 to 1. Raises ValueError if the quantum or reference method has no value. See Benchmarks for how a workflow author chooses a kind.