Frame3D AI

A web-based experimental project for generating and exploring immersive 3D virtual tours from smartphone video using Gaussian Splatting and Three.js.

This repository is being released as an open learning resource. It is no longer actively maintained, but the code is provided so others can study, run, and experiment with browser-based 3D reconstruction and navigation systems.

---

⚠️ Project Status

Archived / Experimental

This project is no longer under active development. It is shared publicly for educational and experimental use.

Expect parts of the codebase to reflect prototype-level decisions and evolving architecture.

---

Overview

Frame3D AI explores how real-world video capture can be turned into interactive 3D environments that run directly in the browser.

Key ideas demonstrated:

• Rendering Gaussian Splat scenes in real time
• Navigating reconstructed 3D spaces using custom controls
• Integrating 3D rendering into a modern Next.js application
• Structuring a hybrid app (marketing UI + 3D viewer)

---

Tech Stack

• Next.js 14 – Application framework
• React 18 – UI layer
• Three.js – Web-based 3D rendering
• Gaussian Splatting renderer – Scene visualization engine
• Custom Fly Controls – Camera navigation system
• CSS Modules – Styling

---

Project Structure

src/
├── app/
│   ├── components/
│   ├── styles/
│   ├── virtual-tours/
│   │   └── [tour-id]/
│   ├── layout.jsx
│   └── page.jsx
│
├── libs/
│   ├── FlyControls.js
│   └── gaussian-splats-3d.module.js
│
├── public/
│   Contains:
│   - Images and videos
│   - Marketing assets
│   - Gaussian Splat (.splat) files

Getting Started

1. Clone the Repository

git clone https://github.com/Justus03/Frame3D-Prototype.git
cd Frame3D-Prototype

---

2. Install Dependencies

npm install

---

3. Run Development Server

npm run dev

Then open:

http://localhost:3000

---

Running the 3D Virtual Tours

Once the app is running:

• Navigate to the virtual tour route in the app
• Ensure required .splat assets are placed inside public/
• Use keyboard or mouse controls to explore scenes

---

Controls

Desktop

Action	Key
Move Forward	W
Move Backward	S
Move Left	A
Move Right	D
Look Around	Arrow Keys
Roll Left	Q
Roll Right	E

---

Mobile

• Drag to rotate camera
• Swipe to move through the environment
• Touch gestures control navigation depending on viewport

---

Experimentation Notes

This project is best treated as a sandbox for exploring:

1. Gaussian Splat Rendering in the Browser

The renderer loads and displays splat-based 3D reconstructions in real time.

2. Custom Camera Systems

The navigation system is fully custom and can be extended or replaced.

3. Next.js + 3D Hybrid Architecture

The app demonstrates how to combine:

• Standard web UI (React)
• High-performance WebGL rendering (Three.js)

Final Note

This project represents an early exploration into browser-native immersive 3D experiences. It is shared in its current state so others can learn from it, improve upon it, or use it as a starting point for their own work.

Acknowledgements

This project would not have been possible without the work of the open-source community in neural rendering and Gaussian Splatting.

Special thanks to:

• Nerfstudio — for providing an excellent framework for training, processing, and experimenting with neural radiance fields and Gaussian Splatting pipelines. https://www.nerf.studio/

• mkkellogg — for the highly useful GaussianSplats3D web renderer, which made browser-based visualization of Gaussian Splat scenes practical and accessible. https://github.com/mkkellogg/GaussianSplats3D

These tools were instrumental in enabling the workflow used in this project, from scene generation to real-time browser rendering.