Eval stage skill

[savitharaghunathan]

Part of #54.

Eval

Evaluates migration quality and extracts learned patterns for future runs.

Consumes	Migrated source (post-verify), PLAN.md, .konveyor/questionnaire.json, all .konveyor/results.json entries
Produces	.konveyor/eval.json (quality scores, learned patterns), .konveyor/results.json entry
Image	agent-verify-{lang} or dedicated eval image
Mode	Non-interactive

Feedback loop

In non-interactive mode, the questionnaire stage records LLM reasoning for its autonomous choices. Eval scores whether those choices led to a good migration — this data feeds back to migration intelligence for future runs. Potential integration point with solution server.

Expected artifact

.konveyor/eval.json:

{
  "qualityScore": 0.85,
  "buildPassed": true,
  "testsPassing": "38/40",
  "fixIterations": 2,
  "questionnaireScores": {
    "targetFramework": "good",
    "scope": "good"
  },
  "learnedPatterns": [
    {"pattern": "weblogic-timer → quarkus-scheduler", "outcome": "succeeded", "notes": "clean conversion"}
  ]
}

Tasks

• Define eval.json schema
• Create skills/eval/SKILL.md
• Define quality scoring criteria (build pass, test pass rate, fix iterations, diff size)
• Define learned-patterns format and feedback mechanism
• Score non-interactive questionnaire choices against migration outcome