Skip to content

[I-02] Debate Result Diff — DebateResult.Diff(other) side-by-side Markdown comparison #15

Description

@techbuzzz

Overview

Version: v10.4.0 Intelligence & DX
Effort: M · 3–5 days
Branch: feature/i02-debate-diff

Add DebateResult.Diff(DebateResult other) that produces a structured comparison between two debate runs, useful for A/B testing different council configurations, strategies, or prompts.


Use Cases

  • Compare StandardDebate vs CritiqueDebate on the same topic
  • Compare council with 3 members vs 5 members
  • Compare GPT-4o vs Claude 3.5 as Chairman
  • Regression testing: same council, two versions of Delibera
  • Benchmark mode: run N variants, diff all against baseline

Design

API

public record DebateDiff
{
    // Metadata
    public DebateResult Left { get; init; }
    public DebateResult Right { get; init; }

    // Structural diff
    public int RoundCountDelta { get; init; }           // Right.Rounds - Left.Rounds
    public TimeSpan DurationDelta { get; init; }        // Right.Duration - Left.Duration
    public int TokenDelta { get; init; }                // Right.TotalTokens - Left.TotalTokens

    // Content diff
    public double VerdictSimilarity { get; init; }      // 0.0–1.0 cosine/Levenshtein similarity
    public string VerdictDiffMarkdown { get; init; }    // Word-level inline diff
    public IReadOnlyList<RoundDiff> RoundDiffs { get; init; }

    // Voting diff (if both used voting)
    public VotingTallyDiff? VotingDiff { get; init; }

    // Output formats
    public string ToMarkdown();
    public string ToHtml();
    public Task SaveToMarkdownAsync(string path, CancellationToken ct = default);
    public Task SaveToHtmlAsync(string path, CancellationToken ct = default);
}

public record RoundDiff
{
    public int RoundNumber { get; init; }
    public string Role { get; init; }
    public string LeftResponse { get; init; }
    public string RightResponse { get; init; }
    public double Similarity { get; init; }
    public string InlineDiffMarkdown { get; init; }     // Added/removed words highlighted
}

// DebateResult extension:
public static DebateDiff Diff(this DebateResult left, DebateResult right);

Diff Algorithm

  • Verdict-level: Word tokenization → LCS diff → inline **added** / removed Markdown
  • Round-level: Match rounds by Role + round number, produce per-turn diff
  • Similarity score: Normalized Levenshtein distance (0 = identical, 1 = completely different)
  • No external ML embedding required — pure string diff, keep it dependency-free

Markdown Output Format

## Debate Diff
**Left**: debate-abc123 | GPT-4o Chairman | StandardDebate | 3 rounds
**Right**: debate-def456 | Claude-3.5 Chairman | CritiqueDebate | 4 rounds

### Summary
| Metric | Left | Right | Delta |
|--------|------|-------|-------|
| Rounds | 3 | 4 | +1 |
| Duration | 12.4s | 18.1s | +5.7s |
| Tokens | 4,210 | 6,830 | +2,620 |
| Verdict similarity ||| 72% |

### Verdict Diff
...The council **recommends** ~~suggests~~ adopting **event sourcing** for audit trail...

### Round-by-Round
#### Round 1 — Architect
> Similarity: 84%
...(inline diff)...

Tasks

  • Implement DebateDiff record
  • Implement RoundDiff record
  • Implement VotingTallyDiff record
  • Implement word-level LCS diff algorithm
  • Implement Diff() extension method on DebateResult
  • Implement ToMarkdown() for DebateDiff
  • Implement ToHtml() for DebateDiff (styled, side-by-side columns)
  • Add DebateResultDiffExample.cs to Delibera.Examples
  • CLI integration: delibera compare <id1> <id2> command (I-03 dependency)
  • Unit tests for diff algorithm edge cases (empty, identical, completely different)
  • Integration test: run two debates, diff them, assert structure

Acceptance Criteria

  • result1.Diff(result2) returns a populated DebateDiff
  • ToMarkdown() produces valid, readable Markdown with summary table
  • ToHtml() produces styled HTML with side-by-side comparison
  • Similarity score is in [0.0, 1.0] and reflects actual textual distance
  • Works on debates with different numbers of rounds (graceful partial match)
  • No new package dependencies introduced

Labels

enhancement, v10.4.0, DX

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions