Green · worked MaxDiff study

Rank roadmap priorities with best–worst choices.

This tutorial follows one complete run: eight product priorities, a balanced MaxDiff design, six EDSL agents, human and model fielding routes, and a pooled Bradley–Terry estimate.

The commands and outputs below were captured from the study in examples/maxdiff-roadmap. The small synthetic panel demonstrates the workflow; it is not evidence about a real customer population.

1. Initialize a MaxDiff study

--type maxdiff changes Green’s definition and analysis vocabulary from attributes and profiles to items and best–worst tasks.

green --project-dir examples/maxdiff-roadmap init --type maxdiff \
  --description "Priorities for an AI research and survey platform"
Show captured output
{
  "status": "ok",
  "data": {"project": {"study_type": "maxdiff"}},
  "next_steps": [{"command": "green item add ..."}]
}

Every Green command emits one versioned JSON envelope. The useful fields for an agent are data, artifacts, warnings, errors, and next_steps.

2. Add the alternatives to compare

Item IDs are durable analysis keys. Labels are the text respondents see.

green --project-dir examples/maxdiff-roadmap item add priorities \
  faster_analysis stronger_privacy better_integrations lower_price \
  easier_setup richer_exports human_interviews synthetic_panels \
  --label faster_analysis="Faster analysis turnaround" \
  --label stronger_privacy="Stronger privacy controls" \
  --label better_integrations="More data integrations" \
  --label lower_price="Lower subscription price" \
  --label easier_setup="Easier study setup" \
  --label richer_exports="Richer reports and exports" \
  --label human_interviews="Built-in human interviews" \
  --label synthetic_panels="Higher-quality synthetic panels"
Show item summary
{"status":"ok","data":{"item_list":"priorities","n_items":8}}

3. Define the six respondents explicitly

Green does not generate a panel from the study description. These are the exact six agent records used in the captured run:

green --project-dir examples/maxdiff-roadmap agent-list create panel
green --project-dir examples/maxdiff-roadmap agent add panel --id r001 \
  --persona "Enterprise research operations lead prioritizing governance, privacy, integrations, and reproducibility" \
  --trait segment=enterprise_ops
green --project-dir examples/maxdiff-roadmap agent add panel --id r002 \
  --persona "Startup product manager prioritizing speed, easy setup, and affordable experimentation" \
  --trait segment=startup_pm
green --project-dir examples/maxdiff-roadmap agent add panel --id r003 \
  --persona "Academic social scientist prioritizing methodological transparency, exports, and respondent quality" \
  --trait segment=academic
green --project-dir examples/maxdiff-roadmap agent add panel --id r004 \
  --persona "Consumer insights director balancing human interviews, fast synthesis, and executive-ready reporting" \
  --trait segment=insights_director
green --project-dir examples/maxdiff-roadmap agent add panel --id r005 \
  --persona "Data engineer prioritizing APIs, integrations, privacy controls, and structured exports" \
  --trait segment=data_engineering
green --project-dir examples/maxdiff-roadmap agent add panel --id r006 \
  --persona "Independent researcher prioritizing low price, simple setup, and access to synthetic respondents" \
  --trait segment=independent
green --project-dir examples/maxdiff-roadmap agent-list show panel
Show panel contract
agent_count: 6
stable IDs: r001 … r006
trait key: segment
persona source: explicit --persona text above

The panel deliberately spans six viewpoints to exercise the workflow. It is not a random or representative customer sample.

4. Generate and inspect the design

Each respondent sees ten sets of four items. Two versions broaden pair coverage without changing the task burden for an individual respondent.

green --project-dir examples/maxdiff-roadmap design generate \
  --name main --method balanced --tasks-per-respondent 10 \
  --alternatives 4 --versions 2 --seed 2026
Show balance diagnostics
{
  "schema": "maxdiff-design-v1",
  "balance": {
    "valid": true,
    "exposure_range": [10, 10],
    "pair_frequency_range": [3, 5]
  }
  

Every item appears exactly ten times across the design. Each pair appears together three to five times. The saved seed makes the 80 design rows reproducible.

5. Build native EDSL packages

green --project-dir examples/maxdiff-roadmap field build \
  --design main --job-name roadmap_piped --agent-list panel \
  --output-dir examples/maxdiff-roadmap
Show generated artifacts
{
  "status": "ok",
  "artifacts": {
    "survey": "examples/maxdiff-roadmap/survey.ep",
    "agent_list": "examples/maxdiff-roadmap/agent_list.ep",
    "jobs": "examples/maxdiff-roadmap/jobs.ep",
    "mapping": "examples/maxdiff-roadmap/question-map.json"
  }
}

The files are native, git-backed EDSL objects rather than generated Python. Green also writes the mapping used to normalize EDSL answers back into item-level response rows.

The worst question pipes in the answer to the best question:

Which item is best?

You selected {{ best_v1_t1.answer }} as best.
Which different item is worst?
Do not select the item you chose as best.

The reference makes EDSL administer the best question first and puts its answer into the follow-up prompt.

6. Inspect agents, questions, and cost before execution

ep inspect examples/maxdiff-roadmap/jobs.ep
ep inspect examples/maxdiff-roadmap/agent_list.ep
ep surveys questions examples/maxdiff-roadmap/survey.ep
ep jobs cost examples/maxdiff-roadmap/jobs.ep --model gpt-5.4-mini
green plan field --design main --job-name roadmap_piped
Show the reviewed plan
{
  "respondents": 6,
  "respondent_tasks": 120,
  "questions_per_task": 2,
  "response_rows": 480,
  "external_model_calls": 240,
  "estimated_cost_usd": 0.186,
  "requires_user_approval": true
}
Execution boundary. Building, inspecting, and pricing the package are local or read-only actions. The following ep run was executed only after the user reviewed and approved the plan.
ep run examples/maxdiff-roadmap/jobs.ep --model gpt-5.4-mini \
  --output examples/maxdiff-roadmap/results.ep
Show execution outcome
{
  "completed": 6,
  "failed": 0,
  "results_uuid": "20401842-a04d-4f6e-833f-ad4825a1d63d"
}

7. Administer the same survey to humans

Humanize renders the saved Survey as a respondent-facing web study. Preview mode does not save answers; the created survey has a durable UUID and respondent URL.

ep humanize preview \
  --survey examples/maxdiff-roadmap/survey.ep \
  --schema examples/maxdiff-roadmap/humanize-schema.json

ep humanize create \
  --survey examples/maxdiff-roadmap/survey.ep \
  --schema examples/maxdiff-roadmap/humanize-schema.json \
  --name "Green MaxDiff roadmap tutorial"

green field register roadmap_piped \
  --human-survey a49f8dde-e9a6-4cf9-bf97-a57fa826f9dd
Show Humanize creation output
{
  "name": "Green MaxDiff roadmap tutorial",
  "uuid": "a49f8dde-e9a6-4cf9-bf97-a57fa826f9dd",
  "respondent_url": "https://www.expectedparrot.com/respond/human-surveys/a49f8dde-e9a6-4cf9-bf97-a57fa826f9dd",
  "n_responses": 0
}
Humanize preview showing the first MaxDiff best-choice question with four roadmap priorities
The live respondent preview. The next screen pipes the selected best item into the worst-choice question and instructs the respondent to choose a different item.

After human fielding, fetch a typed Results package and import it through the same Green contract used for model responses:

ep humanize responses a49f8dde-e9a6-4cf9-bf97-a57fa826f9dd \
  --output examples/maxdiff-roadmap/human-results.ep
green data import-edsl examples/maxdiff-roadmap/human-results.ep \
  --design main --job-name roadmap_piped --name human_field_run

No human responses were fabricated for this tutorial; the screenshot uses ?save_response=false.

8. Import model responses and inspect quality

green data import-edsl examples/maxdiff-roadmap/results.ep \
  --design main --job-name roadmap_piped --name corrected_field_run
green data quality corrected_field_run
green next
Show quality outcome
{
  "phase": "data-invalid",
  "counts": {
    "respondents": 6,
    "tasks_observed": 120,
    "tasks_complete": 120,
    "tasks_invalid": 0,
    "rows": 480
  }
}

All 120 respondent-tasks contain exactly one best item, exactly one worst item, and distinct selections. No task-level override or exclusion policy is needed.

9. Estimate the pooled model

green plan estimate --data corrected_field_run --method mnl \
  --name roadmap_maxdiff
green estimate mnl --data corrected_field_run --name roadmap_maxdiff
Show model diagnostics
{
  "method": "maxdiff-paired-mnl",
  "n_response_rows": 468,
  "n_tasks": 120,
  "n_comparisons": 600,
  "log_likelihood": -355.9876,
  "converged": true,
  "coverage": {"tasks_used":120,"tasks_excluded":0,"quality_valid_before_fit":true}
}

Green converts each best–worst answer into five implied pairwise wins and fits pooled, zero-centered Bradley–Terry utilities. Standard errors come from the optimizer’s covariance estimate.

10. Read the ranking

green report maxdiff roadmap_maxdiff
Show captured ranking
rank  item                              utility    SE
1     Stronger privacy controls          0.710   0.154
2     Easier study setup                  0.493   0.161
3     Richer reports and exports          0.447   0.157
4     Faster analysis turnaround          0.427   0.159
5     More data integrations              0.157   0.157
6     Lower subscription price           -0.370   0.150
7     Higher-quality synthetic panels    -0.696   0.155
8     Built-in human interviews          -1.169   0.166
MaxDiff utilities with 95 percent confidence intervalsStronger privacy controls has the highest estimated utility. Easier setup, richer exports, and faster analysis form a close middle-high group. Human interviews has the lowest utility. Zero-centered utility · dot = estimate · line = 95% CI −1.6−1.0−0.500.51.0 Stronger privacy controls Easier study setup Richer reports and exports Faster analysis turnaround More data integrations Lower subscription price Higher-quality synthetic panels Built-in human interviews
Intervals describe uncertainty in each item’s utility relative to the zero-centered scale. Overlap is a warning against over-reading adjacent ranks; a formal contrast requires the covariance between item estimates.

The utility plot is the statistically faithful view. For a planning conversation, the same estimates can be exponentiated and normalized into relative preference shares that sum to 100 percent:

PriorityPreference shareBest − worst count
Stronger privacy controls21.6%+21
Easier study setup17.4%+13
Richer reports and exports16.6%+12
Faster analysis turnaround16.2%+11
More data integrations12.4%+4
Lower subscription price7.3%−10
Higher-quality synthetic panels5.3%−19
Built-in human interviews3.3%−32
How to present this to a decision-maker. Use preference shares for relative allocation and the dot-and-whisker plot for uncertainty. Call them model-derived priority shares, not respondent percentages or forecast market shares. The best-minus-worst counts are an intuitive descriptive check, while the MNL utilities are the inferential result.

The utilities are relative and sum to zero. In this demonstration panel, privacy leads; setup, exports, and analysis speed form a less clearly separated second group. With only six synthetic personas, the result shows that the pipeline works—not what a market believes.

For heterogeneity, the same normalized responses also support respondent-level shrinkage and latent preference classes:

green estimate hb --data corrected_field_run --name roadmap_hb \
  --iterations 1000 --seed 42
green estimate latent-class --data corrected_field_run \
  --name roadmap_classes --classes auto --seed 42

The empirical-Bayes estimator shrinks each respondent’s item utilities toward the pooled result. The latent-class estimator selects among one to four Bradley–Terry mixtures by BIC and saves class membership probabilities. With six example personas, these are workflow demonstrations rather than credible segmentation.

Persist the decision view, its structured data, provenance, and plot together:

green report save --name roadmap_decision \
  --kind maxdiff --input roadmap_maxdiff
Show saved artifacts
{
  "report": ".green/reports/roadmap_decision/report.md",
  "html": ".green/reports/roadmap_decision/report.html",
  "plot": ".green/reports/roadmap_decision/plot.svg",
  "data": ".green/reports/roadmap_decision/data.json"
}

Reproduce and audit

The concise audit record is checked in as results-summary.json. Green’s durable provenance is under .green/; this tutorial places public EDSL objects and Results explicitly under examples/maxdiff-roadmap/.

green --project-dir examples/maxdiff-roadmap next
Show final state
{"phase":"verification-ready","blockers":[],"recommended":{"command":"green report maxdiff roadmap_maxdiff"}}