Live site · Research report (plain-language, non-technical)
An interactive simulation of multi-party diplomatic negotiation, developed under Ethical Tech CoLab. Participants (human or AI) role-play state actors working through a live geopolitical crisis, and each session produces a structured transcript and analysis.
Each scenario is a multi-party negotiation replayed in the diplomacy table. Every delegation is driven by its own confidential privileged instructions, and each run produces a full transcript, tactic analysis, and a convener report.
▶ Live site: https://ethical-tech-colab.github.io/diplomatic-simulator/ · Watch a live negotiation (delegations take the floor one by one, streamed as a live feed) · Interactive dashboard (parties' thoughts, proposals, detector hits, scoreboard — pick any scenario) · Monte Carlo analysis (outcome distributions across ~20 randomized-condition trials per scenario)
| Scenario | Parties | Report page | Interactive | Focus |
|---|---|---|---|---|
| Arctic | 7 | report | table | Continental-shelf claims, NWP/NSR sea-lanes, Svalbard, Arctic governance |
| Central Asia — Fergana Valley | 7 | report | table | Disputed borders, Syr Darya water, hydropower, great-power influence |
| Cyprus — Reunification | 6 | report | table | Federation vs. two-state, territory, property, security, EEZ gas |
| South China Sea | 9 | report | table | Competing claims, UNCLOS, freedom of navigation, code of conduct |
| Korea — Six-Party Talks | 6 | report | table | Denuclearization vs. mutual freeze, THAAD, sanctions & humanitarian aid, peace treaty, LWR/KEDO |
| Jammu & Kashmir | 7 | report | table | Sovereignty & the Line of Control, Article 370 and the unheld plebiscite, human rights investigation, 950,000 displaced |
| Iran–US Strait of Hormuz | 2 | report | — | Original session report (ceasefire / Hormuz closure crisis) |
Each report page is a standalone, readable write-up (overview, convener report, scoreboard, full transcript); the interactive table replays the same session with a scenario picker.
▶ Methodology & AI use — a transparent account of the pipeline, exactly how AI agents role-play each delegation (and what they do not do), the model used, and the limitations. All negotiation content is generated by Claude Opus 4.8 agents and is notional/educational only.
- Diplomacy Table Demo — standalone interactive negotiation table (pick a scenario from the dropdown)
sim/— the simulation toolchain and per-scenario run artifacts (profiles, transcripts, analyses)
Working prototype. Negotiation statements are generated by AI agents role-playing each
delegation from its privileged instructions; scenarios are notional, for educational use
only. Search-engine indexing is disabled via robots.txt and per-page noindex tags.
Adapted from the original Nova Fellowship portfolio work. See individual files for prior attribution.
The full independent academic peer review of this report is in PEER-REVIEW.md (also available as Word under peer-review/).
Recommendation: Major revisions
What the review found:
- The deepest limitation, that one model plays every side, is acknowledged but never tested, though the test is cheap and available. — Addressed: the Korea mixed-model run is done, committed and audited. Its results are reported in the paper at 11.1a, including one that is adverse to the paper's own headline.
The multi-model run has been executed for Korea and audited. A controlled A/B against the isolated single-model baseline (93b2941) holds scenario, profiles, brief, prompts, rounds and information isolation constant and varies only the model behind each seat — DPRK and USA on Opus 4.8, China and Russia on Sonnet, ROK and Japan on Haiku 4.5. Design, all three round transcripts and the comparative audit are committed under sim/analysis/mixed-model/.
What it found, in short. The coarse outcomes replicated: substituting four of six seats across two model tiers did not change whether the freeze was accepted, whether sanctions moved, or whether a settlement was reached. Three finer outcomes did not replicate. But the audit's central result is adverse to the paper rather than reassuring about it — the "zero comprehensive settlements" result is over-determined by profile design. The DPRK's maximum authorised concession and the trilateral bloc's minimum authorised demand are logically disjoint, so no role-faithful agent of any tier can settle this scenario. That is a property of the instrument, and the paper now carries it as a limitation (S8.6, S11.1a) rather than as a finding about diplomacy.
Three cautions travel with it, all recorded in findings.md: one headline outcome moved between runs at a seat whose model never changed, which is run-to-run variance rather than a treatment effect; the all-Opus baseline contains the same error families as the mixed run, so reporting only weak-model errors would be selecting on the hypothesis; and model and role are perfectly confounded in this assignment, since both Haiku seats were the allied non-US delegations and both Sonnet seats the step-by-step bloc. n=1 per condition. Rotating tiers across seats, replicating to turn instances into rates, blind re-coding of the red-line audit, and a scenario whose concession sets actually overlap are the next steps.
What has also been done is the reviewer's stated fallback — making the conditionality travel with the numbers instead of sitting in a limitations section readers reach last:
- S5.4, S6, and S8.6 each carry a "conditional on a single model" banner at the point the results are presented, stating that satisfaction scores, red-line survival rates, deal-type distributions, and coalition frequencies describe how one model behaves playing every side, not how negotiations behave.
- S11.1a documents the mixed-model run's design and model assignment and now reports the audit in full, including what it does not license: no rate, frequency or significance claim about model tiers, no separation of model from seat, and no claim that the all-Opus baseline is a clean reference.
- S8.6 reframes the "0 comprehensive settlements" result at the point it is stated. The review's worry was that it might be an artefact of the model's disposition; the audit found something else, that it is fixed by the profiles themselves, so it measures the scenario pack rather than the negotiation. The same check has not been run on the other five scenarios.
- S13.1 is softened: the conclusion now opens by saying the negotiation results should not yet be read as findings about negotiation, and confines the contribution to the methodological discipline, which is what the prototype actually demonstrates.
- Small-n percentages ("38% of runs" = 3 of 8) contradict the paper's own caveat against quoting such figures. — Fixed. The Monte Carlo layer was topped up from eight trials to twenty, and S8.6 now reports counts rather than percentages throughout (China's Arctic red line held in nine runs of twenty, the United States' in fifteen), with a sentence saying why a percentage over twenty runs misleads.
Noted strength: Radical, self-documenting transparency, including disabling search indexing so AI-invented statements cannot circulate as reporting.