Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ on:
branches: [dev, main]
pull_request:

permissions:
contents: read

jobs:
verify:
runs-on: macos-latest
Expand Down
47 changes: 45 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,49 @@ What makes it different is the **Edit Agent**: a chat panel that sits beside the

Nothing is uploaded on its own. Model providers are opt-in, connected one at a time with your own API key or sign-in, and the app works with none of them connected.

## Architecture

```mermaid
flowchart LR
Creator[Creator]

subgraph Renderer["Renderer — React UI"]
Editor["Timeline · Program Monitor"]
Agent["Edit Agent · approval UI"]
Studios["Voice · Image · Video studios"]
end

Bridge["Preload — typed window.videoTool bridge"]

subgraph Main["Electron main process"]
Policy["Validation · approval · provider policy"]
Projects["Local projects · assets · chats"]
Jobs["FFmpeg export · AI job manager"]
Secrets["safeStorage · OAuth tokens"]
Tools["TypeMCP tool surface"]
end

subgraph Shared["Shared editing core"]
Timeline["Timeline rules · composition · validation"]
Planning["Shot planning · cost estimation"]
Contracts["IPC · provider contracts"]
end

Local[("User-controlled local files")]
Providers["Connected providers\nonly after explicit approval"]

Creator --> Renderer --> Bridge --> Main
Renderer <--> Shared
Main <--> Shared
Main <--> Local
Jobs --> Providers
Tools --> Policy
```

The **renderer** collects intent and renders editor state; it never receives raw IPC, FFmpeg execution paths or arguments, or stored provider credentials and OAuth tokens. The **preload** layer exposes only the typed `window.videoTool` bridge. The **main process** owns local projects, secrets, job lifecycle, local FFmpeg execution, and the TypeMCP tool surface. Editing rules, composition, validation, and generation planning live in the portable **shared core**, which desktop and mobile use together.

Project folders, imports, generated results, chats, and exports remain local. A connected provider is contacted only for an operation you explicitly start: in a generation studio, that is the visible **Generate** action; for an agent-initiated mutation or job, the Edit Agent asks for approval before execution. The Program Monitor is a best-effort review surface; local FFmpeg MP4 export is the authoritative saved output.

## The workspace

Open a folder and you land in the workspace. One tab strip switches between editing and the two generation studios; the agent chat stays docked beside all three.
Expand Down Expand Up @@ -127,7 +170,7 @@ The provider and model registry is generated from a snapshot of the [models.dev]
- **OpenAI**: two login methods on one provider — an API key, or a ChatGPT sign-in (PKCE OAuth) for the model set that backend serves. Tokens stay in main-process safe storage; the renderer only learns whether you are connected.
- **Generation**: 17 runnable video models across Google Veo, OpenAI Sora, Runway and Luma — Runway alone fronts Seedance, Veo 3.1, HappyHorse and Gemini Omni Flash on one key. Eight image models and seven voices. Providers without a real adapter stay listed but honestly unavailable rather than pretending to work, and every model says which it is.

API keys are written to Electron `safeStorage` in the main process and never reach the renderer.
A provider API key is entered in Settings and sent once through the typed bridge to Electron `safeStorage`; stored provider credentials are never returned to the renderer.

## Quick start

Expand Down Expand Up @@ -171,7 +214,7 @@ Projects are folders you choose. Assets, chat history, and generated results are
VIDEO_TOOL_RECORDINGS_DIR=/absolute/path/to/recordings npm run dev
```

The renderer talks to the main process through a narrow typed `window.videoTool` bridge. Raw `ipcRenderer`, filesystem paths, FFmpeg arguments, API keys, and OAuth tokens stay outside it a picked reference image, for example, crosses as bytes, never as a path.
The renderer talks to the main process through a narrow typed `window.videoTool` bridge. Raw `ipcRenderer`, FFmpeg executable paths and arguments, stored credentials, and OAuth tokens stay outside it. Some safe display paths and an API key entered in Settings cross through explicit typed operations; a picked reference image, for example, crosses as bytes, never as a path.

- **No account, no telemetry.** No analytics, crash reporting, or usage tracking.
- **No background network calls.** The app talks to a provider only when you ask it to, using a provider you connected.
Expand Down
10 changes: 7 additions & 3 deletions docs/hybrid-ai-editor-direction.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# OpenScene hybrid AI editor direction

**Status:** Approved future product direction; no AI provider integration is currently shipped
**Status:** Historical direction plus future assisted-editing constraints. Provider-backed voice, image, video generation, and the approval-gated Edit Agent are now released; the AI-assisted editing proposals in §3.1 remain future work.
**Date:** 2026-07-23
**Issue:** #12

> **Current capability source:** [the root README](../README.md) is the public current-capability and release boundary. This document preserves the initial hybrid-AI direction and defines constraints for capabilities that remain unimplemented.

## 1. Decision

OpenScene will evolve into an open-source **hybrid AI video editor**. The local timeline remains the primary workspace and system of record. AI supports the creator with proposed edits and optional generated assets; it does not replace human review, local project ownership, or the existing local export path.
Expand Down Expand Up @@ -125,9 +127,11 @@ Accounts, billing, analytics, crash reporting, cloud project sync, and hidden ne

## 7. Release and public-copy boundary

The current release provides local selected-window capture, local project/timeline editing, local MP4 export, and user-configured local Qwen TTS. It does **not** currently provide AI-assisted editing, AI generation, or connected external AI services.
The initial July MVP described local selected-window capture, local project/timeline editing, and local MP4 export. The current release boundary is broader: the approval-gated Edit Agent and provider-backed voice, image, and video generation are shipped. See [the root README](../README.md) for provider-specific availability and platform limits.

The following capability groups remain future-facing until a separately reviewed implementation ships: AI-assisted edit suggestions (cuts, highlight sequences, reframing, captions), automated acceptance of suggestions into a saved timeline, cloud project sync, hosted rendering, accounts, analytics, and hidden network activity.

Until a separately reviewed implementation ships, README, renderer, product site, app metadata, and marketing materials must use future-facing language such as “planned”, “in development”, or “designed for” when mentioning these capabilities. They must not claim automated edits, generated assets, provider calls, or remote processing are available today.
For every shipped or future capability, README, renderer, product site, app metadata, and marketing materials must state whether processing is local or provider-connected, and must not present a proposed capability as current behavior.

## 8. Implementation sequence

Expand Down
8 changes: 5 additions & 3 deletions docs/planning.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# OpenScene Product Plan
# OpenScene Product Plan — Historical MVP Baseline

Status: Draft
> **Current capability source:** [the root README](../README.md) is the current product and release boundary. This document records the 2026-07-20 MVP baseline and its original implementation plan; its former “future” provider and voice statements are not a claim about the current `dev` branch.

Status: Historical planning baseline
Created: 2026-07-20
Audience: product decision-makers, implementers, and security reviewers

Expand All @@ -10,7 +12,7 @@ OpenScene is an Electron video production app for selecting one desktop window,

The user should be able to create work demos, lessons, product walkthroughs, and short presentation videos without setting up a full broadcast tool. The app should reduce accidental privacy exposure by recording a selected window instead of the entire screen.

Longer term, OpenScene will evolve into a hybrid AI video editor: the local recording and editing flow stays primary, while users can optionally use local models or explicitly selected external AI services for assisted editing and media generation. The future direction, data boundary, consent requirements, shared job architecture, and release-copy rules are defined in [`hybrid-ai-editor-direction.md`](hybrid-ai-editor-direction.md). Gemini Veo, OpenAI Sora, and ElevenLabs remain possible future provider targets only; they are not implemented in the current MVP.
The longer-term direction is still a hybrid AI editor: local recording and editing remain primary while optional connected services support the currently shipped generation workflows and future assisted-editing workflows. The historical direction, data boundary, consent requirements, shared job architecture, and remaining future constraints are in [`hybrid-ai-editor-direction.md`](hybrid-ai-editor-direction.md). For what is released on the current branch, including provider-specific availability, use [the root README](../README.md).

## 2. Core Users

Expand Down
128 changes: 128 additions & 0 deletions docs/planning/2026-08-03-issue-197-trust-planning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Issue 197: Trust & Planning Implementation Plan

> **For Hermes:** Use subagent-driven-development skill to implement this plan task-by-task.

**Goal:** Document OpenScene's real security/process architecture, ensure legal shot planning is exact whenever possible, and remove the current MCP production audit finding.

**Architecture:** README and historical planning documents name the renderer/preload/main/shared/local/provider boundaries. A bounded dynamic-programming planner selects the optimal legal shot sequence before the existing tool surfaces it. The direct TypeMCP version update regenerates npm's lockfile.

**Tech Stack:** TypeScript, Vitest, npm, Electron/Vite, Mermaid Markdown.

---

### Task 1: Lock the exact-first storyboard contract in tests

**Objective:** Demonstrate that the existing greedy planner unnecessarily rounds a representable request.

**Files:**
- Modify: `tests/videoStoryboardPlan.test.ts`

**Step 1: Write failing test**

Add a case for Google Gemini 10 seconds that expects `[6, 4]`, total `10`, and no `roundedFrom`.

**Step 2: Run test to verify failure**

Run: `npm test -- tests/videoStoryboardPlan.test.ts --reporter=dot`

Expected: FAIL because the current longest-first loop returns 12 seconds (`[8, 4]`).

**Step 3: Commit**

Do not commit until the implementation and tests are green as one coherent feature slice.

### Task 2: Implement bounded exact-first legal-duration planning

**Objective:** Select the optimal legal duration sequence without exceeding the 24-shot cap.

**Files:**
- Modify: `src/shared/videoStoryboardPlan.ts`
- Test: `tests/videoStoryboardPlan.test.ts`

**Step 1: Implement minimal planner**

Enumerate totals reachable by legal durations with at most `MAX_PLANNED_SHOTS`. Select an exact plan first; otherwise select the smallest absolute distance. Compare ties by fewer shots and lexicographically longer earlier durations.

**Step 2: Run focused tests**

Run: `npm test -- tests/videoStoryboardPlan.test.ts --reporter=dot`

Expected: PASS.

**Step 3: Run affected checks**

Run: `npm run typecheck && npm test -- --reporter=dot`

Expected: PASS.

### Task 3: Update TypeMCP and validate production dependency safety

**Objective:** Remove the current `@hono/node-server` moderate audit finding through a direct supported dependency upgrade.

**Files:**
- Modify: `package.json`
- Modify: `package-lock.json`

**Step 1: Upgrade**

Run: `npm install @theorvane/type-mcp@0.3.1 --save-exact`.

**Step 2: Verify resolution**

Run: `npm ls @theorvane/type-mcp @modelcontextprotocol/sdk @hono/node-server --all && npm audit --omit=dev --audit-level=moderate`

Expected: TypeMCP 0.3.1; no moderate-or-higher production audit result.

**Step 3: Run affected checks**

Run: `npm run typecheck && npm test -- --reporter=dot && npm run build`

Expected: PASS.

### Task 4: Publish architecture and capability-boundary documentation

**Objective:** Make the current implementation legible and prevent stale future-only claims.

**Files:**
- Modify: `README.md`
- Modify: `docs/planning.md`
- Modify: `docs/hybrid-ai-editor-direction.md`

**Step 1: Add README architecture section**

Add the approved Mermaid diagram after the product explanation and explain each process ownership and local/connected boundary.

**Step 2: Reconcile historical documents**

Mark the planning document's state accurately and add a current-capability pointer. Update the hybrid direction's status/release boundary to distinguish released generation/agent capabilities from still-future assisted-editing capabilities.

**Step 3: Verify source documentation contract**

Run: `npm test -- tests/openSceneBrandingSource.test.ts --reporter=dot && git diff --check`

Expected: PASS.

### Task 5: Complete cross-surface verification and delivery checkpoint

**Objective:** Prove the shared-core change and release documentation are ready for review.

**Files:**
- Review: all changed files

**Step 1: Run full verification**

Run:

```bash
npm test -- --reporter=dot
npm run typecheck
npm run build
npm --prefix mobile run typecheck
npm audit --omit=dev --audit-level=moderate
git diff --check
git status --short --branch
```

**Step 2: Commit and push**

Create one conventional commit, push `feat/197-trust-planning`, then open a PR against `dev` with `Closes #197`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# OpenScene Trust & Planning Design

**Issue:** #197
**Status:** Approved for implementation
**Date:** 2026-08-03

## Goal

Make OpenScene's public architecture and capability boundaries easy to understand, prevent valid requested video durations from being unnecessarily changed, and remove the known moderate MCP transitive dependency advisory.

## Scope

1. Add an architecture section to the root README, using a Mermaid diagram and concise ownership/boundary explanation.
2. Reconcile outdated planning and AI-direction documents with the capabilities that are actually implemented in the current product.
3. Replace the storyboard planner's greedy duration fill with an exact-first search over a bounded legal-duration set.
4. Upgrade `@theorvane/type-mcp` from `0.2.0` to `0.3.1`, bringing its MCP SDK dependency to a version that resolves the reported `@hono/node-server` advisory.

## Architecture documentation

The README diagram must show these real boundaries:

- **Renderer:** React UI, editor, generation studios, agent approval UI.
- **Preload:** narrow typed `window.videoTool` bridge; no raw IPC.
- **Electron main process:** local project/assets/chats, credential storage, provider calls, FFmpeg export, jobs, and TypeMCP tools.
- **Shared core:** portable timeline/composition/validation/planning/contracts consumed by desktop and mobile.
- **Local data:** project folders and Electron user data remain local.
- **Connected providers:** contacted only for an explicitly chosen/approved operation.

Program Monitor remains best-effort preview; local FFmpeg output remains authoritative.

## Documentation truth model

`README.md` is the public current-capability entry point. `docs/planning.md` and `docs/hybrid-ai-editor-direction.md` preserve historical decisions but must not contradict the current implementation. They will identify their historical/future content and link to README for current capability status.

## Storyboard planning contract

`planVideoStoryboard({ totalSeconds, providerId })` has a maximum of 24 shots and chooses only that provider's listed legal durations.

1. Normalize the request to the existing nearest positive integer policy and bounded maximum duration.
2. Search feasible duration combinations within the shot cap.
3. Prefer an exact total.
4. Among equally exact plans, prefer fewer shots; for ties, prefer longer earlier shots.
5. If no exact plan exists, choose the legal result with the smallest absolute distance from the requested duration. Tie-break toward fewer shots, then longer earlier shots.
6. Preserve sequential `startSeconds`, all legal-duration guarantees, continuity keys, and disclosed `roundedFrom` when the selected total differs from the requested total.

This makes a 10-second Google Gemini request use `[6, 4]` rather than the greedy `[8, 4]` (12 seconds), while an unrepresentable Sora 10-second request remains visibly rounded.

## Dependency security

The package-lock will be regenerated through npm after updating the direct TypeMCP dependency. The verification target is a clean `npm audit --omit=dev --audit-level=moderate` result, not a hand-edited lockfile.

## Non-goals

- New provider adapters or network calls.
- Automated timeline changes without approval.
- Cloud project storage, accounts, analytics, or UI redesign.
- Changes to persisted compatibility identifiers.

## Verification

- Focused planner and cost-gate tests show the new exact-first behavior.
- Full root tests, typecheck, build, and `git diff --check` pass.
- `mobile/` typecheck passes because the shared planning contract stays platform-neutral.
- Production dependency audit passes at the requested moderate threshold.
Loading
Loading