[FEATURE] support shadow run / preview

[georgeh0]

From users from Discord:

One feature I’d genuinely love to see is a shadow run/preview mode. When someone changes chunking or something, can we somehow show what would actually change before anything is written?
This would reduce the breaking of pipelines in a sense

We want to support an option for update API and CLI subcommand to enable preview run. During preview run, all Actions will be:

• For API: emitted to a callback
• For CLI: printed out

instead of being applied to the real target.

---

❤️ Contributors, please refer to 📙Contributing Guide.
Unless the PR can be sent immediately (e.g. just a few lines of code), we recommend you to leave a comment on the issue like I'm working on it or Can I work on this issue? to avoid duplicating work. Our Discord server is always open and friendly.

[MrAnayDongre]

Hey @georgeh0
would love to take part in this, can we work together on it?

[georgeh0]

Hi @MrAnayDongre , I've not started yet. Feel free to give a try! Thanks for your contribution!

[MrAnayDongre]

I think the cleanest design is a Terraform-style plan split for CocoIndex.
update --preview should run the normal update path far enough to compute the target-state diff, then swap the final executor with a preview sink that emits structured actions instead of writing to real targets.
API can expose preview=True plus an action callback, and CLI can print CRUD attachment changes from the same action stream.
That keeps preview, and real apply on the exact same planning logic.
If this direction sounds right, I can turn it into a concrete GitHub implementation plan next, or do you have any other approach in mind, @georgeh0

[georgeh0]

Hi @MrAnayDongre, yes, this matches exactly what's in my mind. Thanks!

[MrAnayDongre]

@georgeh0 I’ll implement this as a shared plan/apply split on top of the existing update path.

The first pass I’m planning is:

• keep planning logic shared with normal update
• add a preview execution mode that emits structured target actions instead of applying them
• expose that through both the Python API and cocoindex update --preview

The concrete scope I’m planning is to extend App.update(...) / App.update_blocking(...) in python/cocoindex/_internal/app.py, add cocoindex update --preview in python/cocoindex/cli.py, and thread the preview options through rust/py/src/app.rs into the Rust core update flow. The important part is that preview will not get a separate planner. It will reuse the normal diff/planning logic and swap the final executor with a preview sink.

For the first pass, preview will emit the same structured target actions that a real apply would execute. On the API side, those actions will go to a callback. On the CLI side, they’ll be printed from the same action stream. I’ll also keep the tests focused on parity between preview and apply, and make sure preview proves it does not write to real targets.

[georgeh0]

@georgeh0 I’ll implement this as a shared plan/apply split on top of the existing update path.

The first pass I’m planning is:

• keep planning logic shared with normal update
• add a preview execution mode that emits structured target actions instead of applying them
• expose that through both the Python API and cocoindex update --preview

The concrete scope I’m planning is to extend App.update(...) / App.update_blocking(...) in python/cocoindex/_internal/app.py, add cocoindex update --preview in python/cocoindex/cli.py, and thread the preview options through rust/py/src/app.rs into the Rust core update flow. The important part is that preview will not get a separate planner. It will reuse the normal diff/planning logic and swap the final executor with a preview sink.

For the first pass, preview will emit the same structured target actions that a real apply would execute. On the API side, those actions will go to a callback. On the CLI side, they’ll be printed from the same action stream. I’ll also keep the tests focused on parity between preview and apply, and make sure preview proves it does not write to real targets.

This looks reasonable to me. Thanks!