WTF — What's This From?

Snap it. Know its story.

WTF turns one photo into a source-checked visual history microsite. It identifies the most specific defensible conceptual subject without inventing the photographed physical instance's personal provenance, researches that scope, creates editorial historical reconstructions, narrates a compact explainer, and keeps a verifiable receipt for every artifact.

The microsite is the primary output. The mini-documentary is its cinematic companion.

Try it: live app · finished KAPLA story · public story directory · hackathon submission package

Why it can win

• Immediately understandable: take a photo; get the story behind what it is.
• Specific without pretending: a readable book becomes that exact work; a generic television becomes television history; neither invents the physical copy's chain of custody.
• Generative media with a point: story plates and narration make research memorable instead of decorative.
• B2 is part of trust: the source photo, generated images, narration, final video, story JSON, and hash-bound manifests live together in durable storage.
• Visible receipts: the finished microsite shows citations, uncertainty, SHA-256 hashes, and manifest verification.

Judge map

Criterion	What to inspect
Real-World Utility	Upload one photo, then watch the first sourced facts appear before the heavier media finishes.
Production Readiness	Persistent async jobs, refresh-safe recovery, confirmation when models disagree, native sharing, and the public directory.
B2 Storage/Data Orchestration	Open a story's Technical information to inspect stored artifacts, SHA-256 hashes, manifest URIs, and verification.
Use of Genblaze	Open the Genblaze pipeline section to see multi-provider inspection, research, parallel image branches, evaluation/retries, narration, ingestion, and manifests.

The demo flow

1. Photograph or upload an object.
2. OpenAI and Google independently build a specificity ladder from the exact supported subject toward broader fallbacks.
3. If they disagree, the user confirms the subject before expensive media generation begins.
4. WTF researches the narrowest supportable rung, widening only when the available evidence requires it.
5. Genblaze fans out four labeled reconstruction branches across OpenAI and Google, evaluates each result, and retries weak plates while narration runs in parallel.
6. The app publishes a visual story with a timeline, citations, provenance, and a zero-generative-video-cost FFmpeg mini-documentary.
7. Every finished story joins the public story directory.

Stack

Layer	Role
web/	Vinext/React interface and shareable visual story
backend/	FastAPI pipeline, research, rendering, and story API
Genblaze	Multi-provider workflow, AgentLoop evaluation/retries, parallel media branches, ingestion, and manifests
Backblaze B2	Durable originals, generated media, JSON records, and provenance manifests
FFmpeg	Lightweight, dependable assembly of the narrated mini-documentary

Current provider roles are deliberately cost-shaped: gpt-4.1-mini and gemini-3.6-flash handle identification and verification; gpt-image-1-mini and gemini-3.1-flash-image split the reconstruction plates; tts-1-hd narrates; and FFmpeg supplies motion without a per-second video-model bill.

flowchart LR
    A[One photo] --> B[OpenAI + Gemini inspection]
    B --> C[Specificity ladder]
    C --> D[Research + verification]
    D --> E[Parallel image branches]
    D --> F[Narration]
    E --> G[Evaluator + retries]
    F --> H[FFmpeg mini-film]
    G --> I[Genblaze manifests]
    H --> I
    I --> J[(Backblaze B2)]
    J --> K[Progressive story microsite]

Loading

Local setup

Requirements: Node 22+, Python 3.11+, and FFmpeg.

python3 -m venv .venv
source .venv/bin/activate
pip install -r backend/requirements.txt
PYTHONPATH=backend uvicorn app.main:app --reload --port 8000

In a second terminal:

cd web
npm install
npm run dev

Open http://localhost:3000.

Environment

Copy .env.example to .env and configure:

OPENAI_API_KEY=...
GOOGLE_API_KEY=...             # Gemini identification, verification, and image branches
B2_KEY_ID=...                 # non-master application key ID
B2_APPLICATION_KEY=...        # secret; B2_APP_KEY also works
B2_BUCKET=wtf-hackathon
B2_REGION=eu-central-003
WTF_PUBLIC_BASE_URL=http://localhost:8000

Backblaze's S3-compatible API does not accept the automatically created master key. Create a non-master application key restricted to wtf-hackathon, enable “Allow List All Bucket Names,” and grant listBuckets, readFiles, and writeFiles. No delete capability is needed. See Backblaze's S3 application-key guide.

Without all B2 variables, WTF runs in local demo storage. Set WTF_DEMO_MODE=1 to use the fixed mirror story while still exercising image generation, narration, and rendering.

API

• GET /api/health — pipeline and storage readiness
• POST /api/story-jobs — start a persistent async workflow with multipart photo and optional context; every finished story joins the public directory
• GET /api/story-jobs/{id} — poll live Genblaze events and job state
• POST /api/story-jobs/{id}/confirm — choose an identification when providers disagree
• POST /api/stories — synchronous compatibility endpoint
• GET /api/stories — list every finished story in the public directory
• GET /api/stories/{id} — retrieve a visual story by its unlisted or public ID

The source image types accepted are JPEG, PNG, and WebP up to 12 MB.

Production

WTF is live at wtf.assistedeverything.com.

Coolify runs the repository as one custom Compose service with three containers: the Vinext frontend, the FastAPI/Genblaze/FFmpeg backend, and an Nginx same-origin gateway. A persistent Docker volume keeps locally served stories and media across container replacements, while the durable artifacts and provenance manifests are also written to the private Backblaze B2 bucket. Cloudflare Tunnel publishes the gateway without exposing its host port directly.

Deployment details and the update procedure are recorded in .agents/skills/coolify/references/wtf.md.