Matching that listens
to both sides.

01 · The ideaOne market. Two sets of preferences.

Roth is a Python package and command-line tool for organizing a two-sided matching process—from participant lists to assignments you can inspect and reproduce.

The organizer supplies two lists, the profiles each side will see, and the market's rules. Roth helps collect preferences, records which candidates are acceptable, and computes one-to-one matches. Each internship opening has one place; each student can receive one internship.

1. Define the marketAdd students, openings, profiles, and eligibility rules.
2. Collect preferencesAsk people to rank, or score candidates on their behalf.
3. Freeze and matchClose the active cohort and run deferred acceptance.
4. Inspect the outcomeRead assignments, preference statistics, and coverage.

The two sides can use different collection methods. They still meet at the same matching engine, with a saved record of where each preference came from.

Start a coding agent with the right question

roth agent next

This read-only entry point helps the calling agent identify the scenario before choosing a method. Two distinct sides with one partner each use deferred acceptance. Forming student teams and assigning each team its own project uses joint integer optimization. Many-to-one capacities use the same deferred-acceptance workflow with individual rankings. Reviewer/submission assignment uses coverage and workload optimization. General stable many-to-many matching, roommate matching, and repeated projects require other formulations.

roth --project study agent next --scenario one-to-one
roth --project mentorship agent next --scenario many-to-one
roth --project classroom agent next --scenario teams

In a fresh directory, the response gives questions about rosters, preferences, and organizer rules. With existing files or saved state, it explains the next applicable step and returns exact command arguments. The agent resolves only missing questions, follows the relevant commands within your authorization, and runs the returned data.rerun command after each stage.

The guide distinguishes existing ranking files, human surveys, delegated scoring, required confirmations, and completed reports. Team results are checked against their input hash and independently recomputed scores; changed inputs get a new output directory. Read the agent guidance contract →

02 · The worked exampleA summer internship program

Imagine a university placing students into openings across ten fictional labs. Some students value close mentoring; others care more about the work itself, compensation, or remote arrangements. Employers consider skills, project experience, interests, and availability.

A profile describes what someone offers. Preference instructions describe what they seek. Roth keeps those separate, along with contact information and private rankings.

Where do these rankings come from? The example generator shuffles candidate IDs with a fixed random seed (17) and marks some options unacceptable. These are synthetic inputs, independent of the profile descriptions above. No person or LLM produced them. Importing these same lists reproduces the assignments below.

The actual input: two ordered lists

Each participant supplies a list from best to worst, plus explicit unacceptable options. Here are two of the 22 records used in this run. Lab numbers identify distinct single-slot openings.

Alex ranks Lab 3 seventh; Lab 3 ranks Alex second. Those two positions become 7 / 2 in the matrix. The lists are the inputs; the highlighted assignment is the computed outcome.

[
  {
    "participant_id": "s001",
    "ranking": ["i008", "i004", "i001", "i006", "i002", "i010", "i003", "i005"],
    "unacceptable": ["i007", "i009"],
    "evaluated": ["i008", "i004", "i001", "i006", "i002", "i010", "i003", "i005", "i007", "i009"],
    "complete": true,
    "source": "synthetic",
    "confirmed": false
  },
  {
    "participant_id": "i003",
    "ranking": ["s003", "s001", "s005", "s004", "s006", "s012", "s007", "s010", "s002", "s008"],
    "unacceptable": ["s011", "s009"],
    "evaluated": ["s003", "s001", "s005", "s004", "s006", "s012", "s007", "s010", "s002", "s008", "s011", "s009"],
    "complete": true,
    "source": "synthetic",
    "confirmed": false
  }
]

All rankings, with realized matches

Read a row to see a student’s preferences across openings. Read a column to see how an employer ranks each student. Each cell is student rank / employer rank; 1 is best on either side.

Why doesn’t Alex get Lab 8, their first choice? Lab 8 ranks Alex eighth and Blair first. The stable outcome assigns Blair to Lab 8 and Alex to Lab 3. A high ranking from one side alone does not secure a match.

Two students remain unmatched. Finley 6 exhausts their acceptable options during matching. Lane 12 explicitly rejects all ten openings—the × entries across Lane’s row are submitted choices, not missing answers.

This matrix is computed from the downloadable preference file using student-proposing deferred acceptance. The commands below reproduce it without model calls or survey invitations.

03 · The methodWhat makes a match stable?

A blocking pair is a student and an internship that would both prefer to be matched with one another rather than keep their current assignments. A stable matching has no such pair, relative to the preferences and candidates considered.

With students proposing, deferred acceptance works like this:

1. Students apply in preference order. Each unmatched student approaches their most-preferred opening that has not yet rejected them.
2. Openings hold their preferred acceptable applicant. An employer can replace a tentative hold with an applicant it ranks higher. Other applicants are rejected.
3. Rejected students try their next option. The process continues until nobody has another proposal to make. Tentative holds then become final assignments.

Someone can remain unmatched because they ran out of acceptable partners. An opening can also remain empty. Roth never forces a match that either side rejected.

For the economic background, see Alvin Roth's Nobel lecture on market design. Roth supports strict one-to-one and many-to-one rankings with responsive preferences; see the capacity example.

04 · Get startedRun the example locally

From a checkout of this package, install Roth with Python 3.11 or newer. The core demo runs locally and needs no EDSL account.

# From the Roth source directory
python -m pip install -e .

1. Create the market and inspect the input files

roth example create internship-study
roth --project internship-study status

This initializes 22 participants and writes two files. The status still reports completed: 0: the preference file exists, but its rows have not yet been imported.

2. Import the rankings

roth --project internship-study preferences import \
  internship-study/preferences.json
roth --project internship-study status
roth --project internship-study preferences show

The import reads the JSON array, checks participant and candidate IDs, checks for duplicate or contradictory choices, and records each participant's ordered list. It preserves the submitted order. The import reports imported: 22 and unchanged: 0; status now shows completed: 22 and an empty missing list. preferences show prints the active records for inspection.

This step imports explicit rankings from the file. Natural-language descriptions in market.json are only used to infer rankings when you separately run the delegated scoring workflow. With people, the survey response importer produces these preference records instead.

3. Freeze those inputs and compute the assignments

roth --project internship-study preferences freeze --name main
roth --project internship-study match --snapshot main --name main
roth --project internship-study report --run main \
  --output internship-study/report

Freeze records the exact cohort and preference lists used for a run. The matching command runs student-proposing deferred acceptance on that snapshot, producing the ten highlighted assignments in the matrix. Later answers or revisions produce a new snapshot, so you can reproduce the earlier assignments.

roth demo internship-demo

roth --project my-study init --left students.csv --right internships.csv
# Or import the full definition, including organizer policies:
roth --project another-study init market.json

05 · Human preferencesAsk both sides to rank

For human participation, install the fielding extra and generate personalized surveys. Each student sees internship profiles; each employer representative sees students for their opening.

python -m pip install -e '.[fielding]'
roth --project internship-study field build --name rankings \
  --output internship-study/rankings

The participant ranks the displayed options alongside Remain unmatched. Candidates placed below that outside option are unacceptable, unless the participant separately marks them as impossible to assess.

From preview to email invitation

Review the generated preview.html, then use the commands in handoff.json to create the surveys through EDSL Humanize. Register the returned survey identifiers in Roth. Registration provides the next commands for email invitations, delivery status, and response retrieval.

The organizer supplies real email addresses for recruitment. Roth prepares the survey and recipient mapping; external ep commands publish it and send the invitations. The fictional example contains no deliverable email addresses.

# Replace HUMAN_SURVEY_UUID with the identifier returned by Humanize.
roth --project internship-study field register --field rankings \
  --package rankings_s001_1 --uuid HUMAN_SURVEY_UUID

ep humanize responses HUMAN_SURVEY_UUID --output student-results.ep
roth --project internship-study field import student-results.ep \
  --name rankings --edsl
roth --project internship-study field status --name rankings

Close the cohort deliberately

Final matching requires completed preferences for everyone in the active cohort. At the deadline, the organizer can explicitly exclude nonrespondents from a new snapshot. Roth records that reduced cohort so the result cannot be confused with a run covering every invitee.

Native EDSL package and response round trips have been tested locally. Hosted survey presentation and email delivery still need a live pilot before recruitment.

06 · Delegated preferencesDescribe the tradeoffs. Let the model rank.

Reviewing every possible partner becomes burdensome as a market grows. In delegated mode, participants explain their preferences in natural language and the model evaluates candidate profiles on their behalf.

“I want hands-on data analysis experience and strong mentoring. Remote work would help, but I would consider an on-site role if the learning opportunity were better. Pay matters less to me than the work.”

An employer provides its own instructions for an opening. Roth retrieves promising candidates, scores them against the relevant instructions, and records an explanation and acceptability judgment for each evaluation.

The organizer can choose a different policy for each side. Explicit human rankings are preserved when model scores are applied.

Separate evaluation from matching

The model produces preferences first. Deferred acceptance then runs on a fixed ordering. This prevents a candidate's position from changing because the model gave a different answer halfway through matching.

roth example create delegated-study
roth --project delegated-study score plan --name baseline \
  --model YOUR_MODEL --k 4 --budget 200 --output delegated-study/baseline

# Inspect the plan and cost; execute the generated EDSL handoff.
roth --project delegated-study score import delegated-study/results.ep \
  --name baseline --edsl
roth --project delegated-study score apply --name baseline

07 · Optional preference checkWould the model choose what you would?

A plausible explanation does not establish that the model understood someone's preferences. Roth can generate a short A-versus-B survey and compare the participant's choices with predictions saved in advance.

Questions combine representative pairs with revealing comparisons, such as options the model scores similarly. The displayed A/B order is randomized, and participants do not see the model's prediction while answering.

The organizer can treat the benchmark as advisory or require a minimum level of agreement before using delegated preferences. Reports show answer counts, disagreements, ties, and acceptability errors for each participant, with targeted and representative questions kept separate.

Refining and evaluating are different steps. Answers used to improve the model cannot also serve as fresh evidence of its accuracy. Once held-out answers are revealed, another prediction round needs a new benchmark.

Explicit human choices also constrain the final inferred ranking. Preferring A to B does not make either acceptable: a participant can reject both. Conflicting comparisons are flagged for clarification.

08 · When there are too many optionsBudget attention as well as model calls

For people: screen, then rank across batches

The default direct-ranking budget is 15 candidates. For larger lists, generate manageable screening batches, collect acceptability judgments, then ask for a single ranking across the accepted candidates. Separate batch rankings alone would leave their relative order unresolved.

roth --project internship-study field build --kind screening \
  --name screening --batch-size 10 --output internship-study/screening
# Import the screening responses, then build the final ranking.
roth --project internship-study field build --name final-ranking \
  --output internship-study/final-ranking

If too many acceptable candidates remain, the organizer can raise the ranking budget explicitly or field a retrieved shortlist. Follow-up rounds can expand the candidates considered.

For models: retrieve, score, and expand

Roth's initial retrieval method matches words in preference descriptions to candidate profiles. It retrieves from both directions and combines the proposed pairs. When both sides delegate, each retained pair receives two evaluations: how the student values the opening, and how the employer values the student.

Matching instructions, profiles, model settings, and calibration evidence allow a previous score to be reused. Expansion adds candidates while retaining earlier work. Sampling omitted candidates provides an additional check on what retrieval may have missed.

Roth reports that boundary with each result: which participants were active, which candidates were considered, and whether the preferences were human, delegated, or synthetic.

09 · InterpretationLook beyond the number of matches

A useful outcome explains who matched, who did not, and how the assignments relate to each side's preferences. Here is a compact reading of the complete 12-student example:

These statistics describe different things. Filling every opening does not mean every student received a highly ranked internship. Comparing the reverse proposing orientation helps reveal how the mechanism affects the allocation.

Ranks are ordinal. A first-to-second-choice difference does not have a common value across people, and an LLM score is not automatically comparable across participants. Roth does not present these numbers as a measure of total welfare.

Keep the outcome reproducible

Each run retains its preference snapshot, proposing side, assignments, and proposal trace. Local history under .roth/ preserves earlier revisions. Exports include an organizer report, match and preference files, and individual summaries that reveal only the assigned partner's public profile.

10 · A different assignment problemForm teams and assign their projects together

Students care about both the project and the people they will work with. Roth can take both sets of rankings and solve the team formation and project assignment jointly.

The organizer sets a target team size, whether smaller or larger teams are allowed, and the weight on teammate preferences. Each student joins one team. Each used project hosts one team. The solver first minimizes deviation from the target size, then maximizes the total weighted Borda score.

Start with the actual ranking files

python -m pip install -e '.[teams]'
roth teams example classroom
roth teams validate classroom/market.json \
  --preferences classroom/preferences.json
roth teams solve classroom/market.json \
  --preferences classroom/preferences.json \
  --output classroom/report --time-limit 60

teams example writes inputs without solving. The market file defines 12 students, five projects, a target size of three, and allowed sizes of two through four. The preference file contains each student's project and teammate rankings, plus explicit exclusions. All rankings in this example are hand-built fictional inputs.

For five projects, first through fifth place earns 4, 3, 2, 1, and 0 Borda points. For eleven potential teammates, first place earns 10 points. Unlisted options earn zero, but remain distinguishable from exclusions. The category scores are normalized using the full option counts and target team size before applying the organizer's weight.

A concrete outcome, including its compromises

With equal project and teammate weights, the optimum forms four teams of three. Alex, Blair, and Casey work on the campus climate dashboard; Devon, Emery, and Finley study housing; Gray, Harper, and Indigo work on transport; Jules, Kai, and Lane tackle food waste. The library project is unused.

Casey receives an unranked project. Climate earns Casey zero project points, but their team includes both top-ranked teammates. That is an allowed outcome under this aggregate objective, and the report makes it visible. An explicit project exclusion would prohibit the assignment.

What changes when we change the weight?

These are actual solved variants of the same synthetic inputs, each with four teams of three. The table describes the saved optima; tied solutions can have different individual assignments. Inspect the runs, or reproduce them with python scripts/team_sensitivity.py --output new-sensitivity.json.

This mode uses file inputs and frozen report directories, independently of the stable-matching project store. SciPy/HiGHS is an optional local solver dependency. Humanize assessment and ranking surveys are available through teams field. Skill constraints and repeated projects remain future extensions.

11 · Several partners, one rankingEmployees and mentors

Many mentors can work with more than one employee. Keep one mentor ID and one ranking, and give that mentor a capacity. Roth assigns each employee at most one mentor and each mentor up to their capacity. Either side may propose.

The preferences are supplied explicitly. Inspect the roster and capacities and each person's ranked list, then import them before matching:

roth example mentorship mentorship-demo
roth --project mentorship-demo preferences import mentorship-demo/preferences.json
roth --project mentorship-demo preferences freeze --name main
roth --project mentorship-demo match --snapshot main --name main
roth --project mentorship-demo report --run main --output mentorship-demo/report

The fixture fills all 30 slots. Open the worked report to see every assigned mentee, mentor capacity utilization, and the matrix of both sides' rankings with matches marked. The JSON output records vacancies separately from people with no assignment.

Capacity defaults to one, accepts zero for a closed position, and can exceed one on at most one side. It is an upper bound; unmatched people and vacant slots are allowed. The algorithm retains the best acceptable proposals up to capacity and replaces a lower-ranked incumbent when a better proposal arrives. The verifier checks both capacity violations and blocking pairs.

Human surveys, model delegation, and confirmation use the same workflow. With 30 candidates per mentor, start human elicitation with screening before a cross-batch ranking. roth --project mentorship-demo agent next --collection human provides guidance. This example's synthetic rankings bypass human fielding explicitly.

For more detail, see the many-to-one guide and Python API. The standard vacancy and replacement rule is illustrated in NRMP's matching explanation.

12 · Reviews with coverageWho reviews whose work?

A class has 12 individually authored essays. Each student should review three essays, and each essay should receive three reviews. Students cannot review their own work or an essay by a member of their existing team. Roth enforces those rules and chooses assignments using reviewer preferences.

There are two explicit files: reviewers, authorship, teams, and rules, and completed preference records. This is Student 1's actual record in the fictional fixture:

{
  "reviewer_id": "s01",
  "ranking": [
    "p05",
    "p09",
    "p12",
    "p08",
    "p10",
    "p06",
    "p07",
    "p11"
  ],
  "unacceptable": [],
  "complete": true,
  "source": "synthetic"
}

The solver reads that preference file through --preferences. The example does not infer preferences or send surveys:

python -m pip install -e '.[reviews]'
roth reviews example classroom-reviews
roth reviews validate classroom-reviews/market.json --preferences classroom-reviews/preferences.json
roth reviews solve classroom-reviews/market.json --preferences classroom-reviews/preferences.json --output classroom-reviews/report

The installation command is for a checkout. The optional reviews extra uses SciPy/HiGHS, the same solver dependency as team assignment.

Open the worked report and assignment matrix: it contains 36 reviews, every requested count is met, and 29 assignments are in the reviewer's top three. Each matrix row shows a student's rankings; columns show essays; green checkmarks show realized assignments. Forbidden edges are marked ×.

Organizers can set different workload bounds and review counts, use explicit 0–100 expertise scores, or deliberately request a feasible allocation without a preference objective. Wider workload bounds allow unequal loads. A time-limited feasible result is labeled separately from a proven optimum.

This mode uses file inputs, immutable report directories, and an independent assignment verifier. It makes no stability or strategy-proofness claim. Humanize assessment, ranking, and expertise surveys are available through reviews field; delegated review scoring remains future work. roth --project classroom-reviews agent next guides the calling agent through inputs, solving, and result review. See the full data contract and formulation.

Collect human team and review preferences

Start from the market roster and a private contact map. Roth builds personalized Humanize packages, tracks returned responses, and exports the completed preferences into the existing solver workflow. The agent can lead the process:

roth --project classroom agent next --scenario teams --collection human --contacts contacts.json
roth --project classroom-reviews agent next --scenario reviews --collection human --contacts contacts.json

First, respondents assess options in manageable batches: select for ranking, state no preference/insufficient information, or declare a hard exclusion. Then they rank the selected options across batches. Review expertise surveys instead use 0–100 ratings in increments of 10. Missing responses block export; revising assessments invalidates old rankings.

The offline synthetic pilot exercised native EDSL survey/Results round trips for all 84 packages: 60 for team preferences and 24 for peer review. Both resulting optimizations reached proven optima. No real participants were recruited, no invitations were sent, and no model inference was run. A real cohort pilot remains pending.

Inspect the team assessment preview, review assessment preview, and complete collection workflow. Publication and delivery use explicit, reviewable ep humanize handoffs. Collection files contain private contacts and individual answers.

Compare organizer what-if scenarios

Use a saved matching, team, or review report as the baseline. Change capacities, team sizes, available projects, or review workloads and inspect who receives different assignments and whose preference outcomes rise or fall.

roth compare classroom/report --scenarios scenarios.json --output classroom/comparison

For the classroom team example, a scenario file can contain:

[
  {"name": "allow-four", "config": {"allow_larger": true, "max_size": 4}},
  {"name": "close-climate", "close_projects": ["climate"]},
  {"name": "project-priority", "config": {"social_weight": 0.25}}
]

Every scenario branches from the same original run. The static report shows a before/after assignment matrix, teammate changes, original ranks, individual score changes, workloads, and solver statuses. Frozen preferences and the baseline scoring scale make the comparison interpretable. Closing a project preserves its original rank; a larger review workload is labeled separately from a preference gain.

Inspect the synthetic comparison report · Inputs, commands, and interpretation

13 · At your deskThe workflow reference

Start with roth agent next to identify the scenario and current stage. Use roth guide for the full workflow. The CLI returns structured output for an assistant; this guide explains the decisions behind those commands.

For exact input schemas and the full fielding sequence, continue with the data contracts and delegation and benchmark walkthrough.

This page describes Roth 0.6.0. Stable matching, joint team optimization, peer-review assignment, and the shared one-to-one/many-to-one survey workflow are implemented. Real recruitment and real-model preference quality require a pilot in the intended market.