Video Depth Slicer - Blender Plugin

Transform video footage into animated 3D geometry using AI-powered depth estimation.

Overview

The Video Depth Slicer plugin is a Blender-native addon that enables users to transform standard video footage into animated 3D geometry with minimal effort. The plugin leverages the UniK3D AI model for depth estimation to generate dynamic, animated depth maps from any video input, which are then used to displace geometry within Blender.

Features

• One-Click Video-to-3D Conversion: Import a video file and generate animated depth maps and 3D geometry with a single click
• Cloud-Based Processing: Core processing handled in the cloud, requiring no local setup or powerful hardware
• Optional Local Mode: For users with compatible NVIDIA GPUs, local processing mode for faster/offline generation
• Animated Displacement: Animates 3D displacement of geometry based on video's depth information
• Text-to-Image Generator: Bonus feature for creating images from textual prompts

Installation

1. Download the plugin as a .zip file
2. In Blender, go to Edit > Preferences > Add-ons
3. Click "Install..." and select the downloaded .zip file
4. Enable the "Video Depth Slicer" addon

Usage

1. Import Video: Use the "Import Video" button to select your video file
2. Generate Depth Maps: Click "Generate Depth Maps" to process the video
3. Select Target Mesh: Select the mesh object you want to displace
4. Apply Displacement: Click "Apply Displacement" to create animated 3D geometry

Requirements

For Cloud Processing

• Internet connection
• Valid API key for the depth estimation service

For Local Processing (Optional)

• NVIDIA GPU with 6GB+ VRAM
• CUDA-compatible drivers
• PyTorch with CUDA support

Development

Project Structure

depth_blender_plugin_v1/
├── __init__.py              # Main plugin file
├── operators/               # Blender operators
│   ├── __init__.py
│   ├── video_import.py
│   ├── depth_generation.py
│   └── displacement.py
├── ui/                      # UI panels
│   ├── __init__.py
│   └── main_panel.py
├── utils/                   # Utility functions
│   ├── __init__.py
│   ├── video_processing.py
│   ├── depth_processing.py
│   └── cloud_api.py
├── models/                  # AI model implementations
│   ├── __init__.py
│   └── unik3d_local.py
└── resources/               # Additional resources
    └── __init__.py

Dependencies

• bpy (Blender Python API)
• opencv-python (for video processing)
• requests (for cloud API communication)
• torch (for local processing, optional)
• numpy (for array operations)

License

[Add your license here]

Credits

This plugin utilizes the UniK3D model for depth estimation:

• Repository: https://github.com/lpiccinelli-eth/UniK3D

Support

[Add support information here]