Skip to content

qt resume on a run that used a demo model fails #16

Description

@arschles

Below is an example qt resume failure:

	qt show 8
	Run 8
		eval: pubmedqa
		status: failed
		created: 2026-06-23T03:21:45.171832Z
		duration: 80.673s
		input: {"model":"demo-builtin","num_samples":1000}
		output: -
		error: invalid builtin input JSON: invalid value: string "demo-builtin", expected expected format "provider:model_id" at line 1 column 43
		
	Aggregated Metrics No metrics found.
	
	
	Run 'qt show 8 --json' for sample-level details.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions