A forecasting workflow
your agent can follow.

The workbench

A process an agent can resume.

The agent researches the world and supplies judgments. Vorhersage stores the state, supplies the next task, validates its inputs, performs declared calculations, and preserves the resulting forecast.

Start here

Give your agent the forecasting job.

Vorhersage is a CLI an agent uses while it researches and reasons. You give the agent a forecasting question; it defines the event, collects evidence, writes the required JSON inputs, and follows the tasks returned by the tool.

The agent needs the installed CLI and its own research and model tools. Vorhersage supplies the workflow and records; it does not itself turn a natural-language prompt into a researched forecast.

git clone https://github.com/expectedparrot/vorhersage.git
cd vorhersage
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
vorhersage version

To see a complete run without an agent or API keys, run the existing fictional factory example from the repository root:

python examples/factory/walkthrough.py ./factory-demo
cd factory-demo
vorhersage report --question factory_east --format markdown

Use a new destination directory. Return to the repository root before starting the separate study in step 1. This example includes two questions, a 50% baseline, a revised forecast, and outcome scoring. Its authored probabilities test the software, not forecasting skill.

The following sections explain the native commands an agent uses. JSON filenames are files the agent writes; RUN_ID, PACKET_ID, and other uppercase identifiers stand for values returned by the CLI. The agent supplies actual timestamps and keeps track of these identifiers.

01 / Define the event

Create a study and register the question.

vorhersage init ./factory-study --name "Factory forecast"
cd factory-study

The current directory selects the project for every command below. Use --project PATH only when working with a different project. Next the agent writes question.json with the deadline and exact YES, NO, and void criteria, then registers it:

vorhersage schema question
vorhersage question add --from question.json

{
  "id": "factory_east",
  "text": "Will fictional factory East ship by September 12, 2026?",
  "yes": "A qualifying shipment is recorded by the deadline.",
  "no": "No qualifying shipment occurs by the deadline.",
  "void": "The fictional event is withdrawn.",
  "event_deadline": "2026-09-12T23:59:59Z",
  "resolve_after": "2026-09-12T23:59:59Z",
  "resolution_source": "urn:vorhersage:fictional:dispatch",
  "event_group": "factory_east",
  "domain": "operations",
  "profile": "general",
  "kind": "simulation"
}

Result: question_id: "factory_east", version: 1. The general profile will require research on base rates, current state, actors and process, and contrary evidence.

This guide uses the fictional factory from the repository examples. For an actual forecast, the agent uses kind: "real", a future deadline, and an authoritative resolution source. Rewording a registered question creates a new version.

02 / Preserve the evidence

Save what the research actually found.

The agent researches the question using its available tools and writes a source-linked bundle. Vorhersage captures that evidence so later reasoning can point to specific findings.

vorhersage research capture --from evidence.json
vorhersage packet audit PACKET_ID

{
  "sources": [
    {
      "id": "initial",
      "url": "urn:vorhersage:fictional:initial",
      "title": "Fictional factory status",
      "excerpt": "Preparation is underway; dispatch remains uncertain.",
      "retrieved_at": "2026-09-11T11:00:00Z"
    }
  ],
  "findings": [
    {
      "id": "readiness",
      "claim": "Preparation is underway; dispatch remains uncertain.",
      "claim_type": "observation",
      "source_ids": [
        "initial"
      ]
    }
  ],
  "limitations": [
    "Entirely fictional example evidence."
  ]
}

Result: a packet ID and a record ID for each finding. The agent puts those references in its later task responses:

{"packet_id": "PACKET_ID", "record_id": "readiness"}

For existing Epiq research, epiq freeze creates the same kind of portable packet from selected cells. See the Epiq selection and capture formats.

Source counts are not confidence multipliers. The agent should identify copied reporting, contradictions, and missing evidence. Packet hashes preserve content identity; they do not establish truth.

03 / Run the process

The operating loop is next → submit.

The agent writes run.json to identify the question, forecaster, mode, information cutoff, and budget. Starting the run returns data.run_id.

vorhersage run start --from run.json
vorhersage next --run RUN_ID

{
  "question_id": "factory_east",
  "question_version": 1,
  "forecaster": "agent:factory",
  "method": "researched judgment",
  "mode": "simulation",
  "information_as_of": "2026-09-11T12:00:00Z",
  "research_status": "completed",
  "max_searches": 0,
  "max_extra_tasks": 1
}

next returns the current task, the schema for its response, the available context, and the remaining budget. The first task is the prior. A shortened response looks like this:

{
  "data": {
    "disposition": "actionable",
    "revision": 0,
    "task": {"id": "TASK_ID", "kind": "prior"}
  }
}

The agent writes response.json using that task ID and revision, then submits it:

vorhersage submit --run RUN_ID --from response.json
vorhersage next --run RUN_ID

{
  "task_id": "TASK_ID",
  "expected_revision": 0,
  "idempotency_key": "prior-submission-1",
  "payload": {
    "method": "judgment",
    "probability": 0.5,
    "rationale": "An assumed neutral starting point for this fictional example.",
    "limitations": [
      "Not an empirical base rate or a blind prior."
    ],
    "evidence_refs": []
  },
  "usage": {
    "searches": 0,
    "cost_usd": 0,
    "model_calls": 0
  }
}

Result: the response is accepted, the revision advances, and next returns the drivers task. The agent repeats the same two commands, writing a new response for each task. It does not need shell wrappers or a custom loop to operate the CLI.

next is read-only. A replacement agent can inspect run list and continue from the same task. Identical submissions are idempotent; stale revisions are rejected.

04 / Assess and issue

The agent supplies the reasoning at every step.

1. PriorA labeled starting judgment or an empirical reference class.
2. DriversMechanisms and concrete paths to both YES and NO.
3. ResearchA separate response for each required domain, with evidence or explained unknowns.
4. AssessmentA probability, its rationale, limitations, and supporting references.
5. ReviewObjections that the probability is too high and too low; retain, revise, or request research.
6. IssueA stopping reason, review time, and observable update triggers.

In the factory example, the agent’s assessment is 40%. This is the payload inside its submission; the task ID and revision still come from next.

{
  "method": "judgment",
  "probability": 0.4,
  "rationale": "Preparation is unfinished; the fictional estimate allows a substantial chance of delay.",
  "limitations": [
    "Authored example probability; not fitted from data."
  ],
  "evidence_refs": [
    {
      "packet_id": "PACKET_ID",
      "record_id": "readiness"
    }
  ]
}

Vorhersage validates the response and performs declared calculations. It does not generate the rationale or decide that the correct estimate is 40%. The agent must do that work. For a decomposition or scenario mixture, the agent supplies the inputs and the package computes their result.

After a review that challenges the estimate in both directions, the agent submits the issue task through the same submit command.

{
  "stopping_reason": "The available evidence and objections have been reviewed.",
  "review_at": "2026-09-12T09:00:00Z",
  "triggers": [
    {
      "description": "A new report about factory readiness.",
      "evidence_refs": []
    }
  ]
}

vorhersage report --question factory_east --format markdown
vorhersage forecast show FORECAST_ID

Result: a 40% forecast with its question version, evidence, research coverage, reasoning, and input hashes. next returns waiting and the forecast ID. Issuance completes the run; it does not resolve the event.

05 / Revisit the forecast

New evidence starts a linked revision.

The factory example receives a new readiness report. The agent captures it, registers a review signal, and starts a new run that names the original forecast in previous_forecast_id.

vorhersage research capture --from update-evidence.json
vorhersage signal --from signal.json
vorhersage monitor
vorhersage run start --from revision.json

{
  "question_id": "factory_east",
  "question_version": 1,
  "forecaster": "agent:factory",
  "method": "researched judgment",
  "mode": "simulation",
  "information_as_of": "2026-09-11T15:00:00Z",
  "research_status": "completed",
  "max_searches": 0,
  "max_extra_tasks": 1,
  "previous_forecast_id": "FORECAST_ID"
}

The revision retains the same question version, forecaster, and mode. The agent follows next and submit again, reconsidering the evidence and objections. In this fictional example it raises the estimate to 75%.

Result: the report contains both 40% and 75%, with the new forecast linked to its predecessor. Neither the signal nor the new source automatically changes the probability.

For repeated checks on a real event, configure a watch with an Epiq source or research worker and optionally an agent worker. See the watch configuration and worker protocol.

06 / Resolve and score

Record what happened, then evaluate.

Once outcome evidence is available, the agent captures it and writes a resolution with the exact question version, outcome, evidence references, and the time the outcome became known.

vorhersage research capture --from outcome-evidence.json
vorhersage resolve --from resolution.json
vorhersage evaluate --from evaluation.json
vorhersage doctor

{
  "question_id": "factory_east",
  "question_version": 1,
  "outcome": "yes",
  "reason": "The fictional dispatch record confirms a qualifying shipment.",
  "known_at": "2026-09-12T12:00:00Z",
  "evidence_refs": [
    {
      "packet_id": "OUTCOME_PACKET_ID",
      "record_id": "shipment_occurred"
    }
  ],
  "previous_resolution_id": null,
  "idempotency_key": "resolve-factory-east"
}

{
  "question_versions": [
    {
      "question_id": "factory_east",
      "version": 1
    }
  ],
  "forecasters": [
    "agent:factory"
  ],
  "cutoff": "2026-09-12T12:00:00Z",
  "resolution_as_of": "2026-09-12T12:05:00Z",
  "mode": "simulation"
}

For a YES outcome, the revised 75% forecast has Brier loss (0.75 − 1)² = 0.0625. The ordinary evaluator selects the latest eligible forecast before the cutoff and outcome knowledge; the original 40% remains in the history.

The complete factory example adds a second question and a 50% baseline. Its final agent probabilities are 75% for East (YES) and 30% for West (NO), producing mean Brier loss 0.07625, versus 0.25 for the baseline.

Lower Brier loss is better. Inspect the matched cohort and exclusions before comparing methods. These authored fictional outcomes verify the workflow and arithmetic; they do not demonstrate forecasting skill.

07 / Compare procedures

Run the same questions through different methods.

The method-comparison example uses the same factory setting. It registers direct judgment and conditional decomposition, then runs two questions × two methods × two repetitions: eight trials. Each question’s evidence is shared across methods. Run this example from the repository root, using a new output directory.

python examples/method_comparison/walkthrough.py ./method-demo
cd method-demo
vorhersage method list
vorhersage experiment list

Inspect the saved files: started.json contains eight trial IDs; partial.json records the pause after three worker tasks; completed.json shows all eight forecasts; evaluation.json preserves method scores and the underlying selected forecasts. The fictional scores do not establish a better procedure.

1. MethodFreeze instructions, task prompts, calculation type, research domains, worker configuration, and budget.
2. ExperimentPin question versions, method IDs, evidence packets, cutoffs, and repetitions.
3. RunExecute bounded rounds. Resume with the same experiment ID.
4. EvaluateAverage repeated Brier losses within question, then compare methods on their common cohort.

vorhersage schema method
vorhersage method add --from direct.json
vorhersage method add --from outside-view.json
vorhersage schema experiment
vorhersage experiment add --from experiment.json
vorhersage experiment start EXPERIMENT_ID
vorhersage experiment run EXPERIMENT_ID --max-tasks 20
vorhersage experiment status EXPERIMENT_ID
# After recording outcomes, supply your resolution snapshot timestamp:
vorhersage experiment evaluate EXPERIMENT_ID \
  --resolution-as-of RESOLUTION_TIMESTAMP

Complete method and experiment schemas, worker protocol, and scoring rules →

Optional / Inspect assumptions

See which assumptions move the answer.

A scenario mixture combines mutually exclusive possibilities. The agent defines the scenarios and their probabilities; the tool checks the weights and computes the result. No domain-specific model is required.

The complementary scenario always receives the remaining weight. These inputs are illustrative judgments, not measured frequencies. The CLI also calculates sensitivity bounds when ranges are supplied.

vorhersage scenario --from ../examples/workbench_extensions/mixture.json

The path above assumes a study directory directly inside the repository, as in this guide. To use the calculation in a workflow, choose method: "scenario_mixture" for the assessment. Simple judgments, nested conditional paths, and explicit ensembles remain available. Read the assumptions and sensitivity rules →

Reference / Discover the interface

JSON in. Reviewable records out.

Commands use the current directory as the project; an explicit --project PATH overrides it. Input file paths are relative to your current directory. Use vorhersage guide, vorhersage schema NAME, and command-level --help to discover the interface. Ordinary output is a JSON envelope; errors go to stderr with a nonzero exit code. --from - accepts stdin.

Project state lives in .vorhersage/state.sqlite. Evidence packets, issued forecasts, and task results retain their history. The package is intended for small portfolios; large-scale performance has not been established.

Examples / Continue exploring

Explore the records behind the numbers.

The Listen Labs example shows why announcement and completion need separate contracts. Historical replays also require an explicit contamination assessment. Examples are dated records, not continuously updated recommendations. See each example’s mode, information cutoff, and limitations.