A sophisticated software library designed to automatically adjust camera exposure for optimal image brightness on mobile devices. This system integrates various inputs, algorithms, and features to achieve accurate and pleasing exposure across diverse scenes.
This code accompanies Chapter 8 of "Mobile Computational Photography".
- 📖 CHAPTER_8_GUIDE.md - Complete learning roadmap and book mapping
- 📁 examples/ - Progressive learning examples (start here!)
- 📋 CODE_ORGANIZATION.md - How code is organized for learning
- 🚀 HOW_TO_RUN.md - Quick start guide
Quick Start for Students:
# From AutoExposure directory
cd AutoExposure
source ../virtual_env/bin/activate
cd examples
# Option 1: Run all examples automatically (recommended)
python run_all_examples.py
# Option 2: Run examples individually
python 01_basic_exposure.py # Start here!
python 02_image_statistics.py # Then this...
# ... progress through all examplesThe Auto Exposure System is a comprehensive solution that processes RAW camera data, image statistics, scene information, and sensor data to automatically determine optimal exposure settings. It provides real-time exposure control with advanced features like face detection, motion analysis, and flicker detection.
Key Features:
- RAW10 Bayer data support with automatic unpacking
- Real-time exposure calculation (33+ fps)
- Scene-aware exposure adjustment
- Flicker detection and mitigation
- APEX value calculation for EXIF metadata
- Comprehensive analysis output with visualizations
RAW10 Bayer Input → Unpack (10-bit to 8-bit) → Demosaic (Bayer to RGB)
↓
Extract Image Statistics
(64x48 grid, Histogram)
↓
┌───────────────────┐
│ Spatial Analysis │
│ Histogram Analysis│
│ Scene Analysis │
└───────────────────┘
↓
Calculate TES (Total Exposure Sensitivity)
↓
Distribute Exposure (Time vs Gain)
↓
Flicker Detection & Adjustment
↓
Output: Exposure Settings + APEX Values
1. Inputs:
- RAW10 Bayer data from camera sensor
- Image Statistics: 64x48 Bayer grid, histograms, saturated pixel counts
- Scene Information: Scene type, face ROIs, motion level
- Sensor Info: Current exposure time, analog/digital gains, ISO, aperture
2. Processing Modules:
- SpatialFrameAnalyzer: Center-weighted metering, exposure zone distribution
- HistogramAnalyzer: Dynamic range, exposure bias, entropy calculation
- SceneAnalyzer: Scene classification, face detection, motion estimation
- ExposureController: TES calculation and exposure distribution
- FlickerDetector: 50Hz/60Hz flicker detection and mitigation
3. Outputs:
- Exposure Time (seconds)
- Analog Gain (1.0x - 16.0x)
- Digital Gain (typically 1.0x)
- APEX Values: Bv (Brightness), Av (Aperture), Tv (Time), Sv (Sensitivity)
- Python 3.7 or higher
- Virtual environment (recommended)
# Navigate to AutoExposure directory
cd AutoExposure
# Activate virtual environment (one directory above)
source ../virtual_env/bin/activate
# Install required packages
pip install -r requirements.txtDependencies:
numpy>=1.21.0- Numerical computationsopencv-python>=4.5.0- Image processing and demosaicingmatplotlib>=3.3.0- Visualization and charting
The test suite processes RAW10 Bayer camera data and generates comprehensive analysis:
# Navigate to AutoExposure directory
cd AutoExposure
# Activate virtual environment (one directory above)
source ../virtual_env/bin/activate
# Run the test
python tests/test_ae_system.pyOutput:
ae_analysis_output/- Annotated images with exposure overlaysae_test_results.png- Comparative analysis charts- Console output with detailed exposure metrics
from core.auto_exposure_system import AutoExposureSystem, SensorInfo
import cv2
# Initialize the system
ae_system = AutoExposureSystem()
# Configure current sensor information
sensor_info = SensorInfo(
exposure_time=1/60, # Current exposure time (seconds)
analog_gain=2.0, # Current analog gain
digital_gain=1.0, # Current digital gain
iso=200, # ISO value
aperture=2.8 # F-number
)
# Load and process an image
image = cv2.imread('your_image.jpg')
result = ae_system.process_frame(image, sensor_info)
# Access exposure recommendations
print(f"Recommended Exposure Time: {result.exposure_time:.6f}s")
print(f"Recommended Analog Gain: {result.analog_gain:.2f}x")
print(f"Total Exposure Sensitivity: {result.total_exposure_sensitivity:.2f}")
print(f"Brightness Value (Bv): {result.brightness_value:.2f}")import numpy as np
from tests.test_ae_system import load_raw10_image
# Load RAW10 file from data folder
raw_bayer = load_raw10_image('data/your_raw_file.raw', width=4032, height=756)
# Convert Bayer to RGB
import cv2
image_rgb = cv2.cvtColor(raw_bayer, cv2.COLOR_BayerBG2BGR)
# Process with AE system
result = ae_system.process_frame(image_rgb, sensor_info)# Customize exposure controller parameters
ae_system.exposure_controller.target_brightness = 140 # Default: 128
ae_system.exposure_controller.min_exposure_time = 1/8000 # 0.125ms
ae_system.exposure_controller.max_exposure_time = 1/30 # 33ms
ae_system.exposure_controller.min_gain = 1.0
ae_system.exposure_controller.max_gain = 16.0
# Customize flicker detection
ae_system.flicker_detector.flicker_frequencies = [50, 60] # Hz
ae_system.flicker_detector.detection_threshold = 0.1
# Process frame with custom settings
result = ae_system.process_frame(image, sensor_info)Analyzes spatial brightness distribution across the image using a 64x48 grid for efficient computation.
Core Algorithm:
class SpatialFrameAnalyzer:
def __init__(self, grid_size=(64, 48)):
self.grid_size = grid_sizeCenter-Weighted Metering Implementation:
The center-weighted metering uses a Gaussian-like weight distribution where the center of the image has maximum weight (1.0) and decreases towards the edges (minimum 0.1):
def _calculate_center_weight(self, stats: np.ndarray) -> float:
h, w = stats.shape
center_h, center_w = h // 2, w // 2
# Create distance map from center
y, x = np.ogrid[:h, :w]
center_dist = np.sqrt((x - center_w)**2 + (y - center_h)**2)
max_dist = np.sqrt(center_w**2 + center_h**2)
# Weight decreases linearly with distance
weight_mask = 1.0 - (center_dist / max_dist)
weight_mask = np.maximum(weight_mask, 0.1) # Minimum 10% weight
return np.average(stats, weights=weight_mask)Exposure Zone Classification:
Pixels are classified into three zones based on standard deviation:
# Highlights: brightness > mean + std
highlights = np.sum(bayer_stats > brightness_mean + brightness_std)
# Shadows: brightness < mean - std
shadows = np.sum(bayer_stats < brightness_mean - brightness_std)
# Midtones: everything else
midtones = total_pixels - highlights - shadowsOutput Structure:
{
'center_weight': 126.85, # Center-weighted brightness [0-255]
'brightness_mean': 126.18, # Average brightness [0-255]
'brightness_std': 74.17, # Standard deviation
'highlights_ratio': 0.21, # Fraction of highlight pixels
'shadows_ratio': 0.21, # Fraction of shadow pixels
'midtones_ratio': 0.57 # Fraction of midtone pixels
}Analyzes the image histogram (256 bins) to understand light distribution and dynamic range.
Histogram Entropy Calculation:
Shannon entropy measures information content and scene complexity:
def _calculate_entropy(self, hist: np.ndarray) -> float:
"""
Entropy = -Σ(p(i) × log₂(p(i)))
Higher entropy = more detail/texture in the image
"""
hist = hist[hist > 0] # Remove zero bins
return -np.sum(hist * np.log2(hist))Percentile-Based Analysis:
Uses cumulative distribution function to find key percentiles:
def analyze_histogram(self, histogram: np.ndarray) -> Dict[str, float]:
# Normalize histogram to probability distribution
hist_norm = histogram / np.sum(histogram)
# Calculate CDF
cumsum = np.cumsum(hist_norm)
# Find percentiles
p5 = np.argmax(cumsum >= 0.05) # 5th percentile
p50 = np.argmax(cumsum >= 0.50) # Median
p95 = np.argmax(cumsum >= 0.95) # 95th percentile
# Dynamic range: spread between 5th and 95th percentiles
dynamic_range = p95 - p5
# Exposure bias: how far median is from middle gray (128)
exposure_bias = (p50 - 128) / 128.0Interpretation:
- Dynamic Range > 200: High contrast scene, consider HDR
- Exposure Bias > 0: Image trending bright
- Exposure Bias < 0: Image trending dark
- Entropy > 7.5: High detail/texture
- Entropy < 5: Low detail/smooth regions
Output Structure:
{
'p5': 14, # 5th percentile brightness
'p25': 61, # 25th percentile
'p50': 119, # Median brightness
'p75': 190, # 75th percentile
'p95': 241, # 95th percentile
'dynamic_range': 227, # p95 - p5
'exposure_bias': -0.07, # (median - 128) / 128
'histogram_entropy': 7.73 # Shannon entropy
}Classifies scene type and detects faces for intelligent exposure adjustment.
Scene Classification Algorithm:
def _classify_scene(self, image, spatial_stats):
brightness_mean = spatial_stats['brightness_mean']
brightness_std = spatial_stats['brightness_std']
# Decision tree based on brightness characteristics
if brightness_mean < 80:
return SceneType.LOW_LIGHT, 0.8
elif brightness_mean > 200:
return SceneType.HIGH_CONTRAST, 0.7
elif brightness_std > 50:
return SceneType.BACKLIT, 0.6
else:
return SceneType.OUTDOOR, 0.5Scene Type Exposure Adjustments:
| Scene Type | Target Brightness Multiplier | Rationale |
|---|---|---|
| LOW_LIGHT | 0.8× (102) | Prevent blown highlights in dark scenes |
| HIGH_CONTRAST | 1.2× (154) | Boost exposure for very bright scenes |
| BACKLIT | 1.1× (141) | Compensate for backlight |
| OUTDOOR | 1.0× (128) | Standard exposure |
| INDOOR | 0.9× (115) | Slightly darker for indoor lighting |
| FACE_DETECTED | 1.1× (141) | Brighten faces |
Face Detection:
Uses OpenCV's Haar Cascade classifier:
def _detect_faces(self, image: np.ndarray) -> List[Tuple[int, int, int, int]]:
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
faces = self.face_cascade.detectMultiScale(
gray,
scaleFactor=1.1, # 10% scale reduction per pyramid level
minNeighbors=4 # Minimum rectangles to keep detection
)
return [(x, y, w, h) for x, y, w, h in faces]Motion Estimation:
Simplified edge-based motion proxy (in real implementation, use optical flow):
def _estimate_motion(self, image: np.ndarray) -> float:
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray, 50, 150)
motion_level = np.sum(edges) / (edges.shape[0] * edges.shape[1])
return min(motion_level, 1.0) # Normalize to [0, 1]Calculates Total Exposure Sensitivity (TES) and distributes it between exposure time and gain.
Total Exposure Sensitivity (TES) Calculation:
def calculate_total_exposure_sensitivity(self, current_stats, scene_info):
current_brightness = current_stats.brightness_mean
# Adjust target based on scene type
target = self._adjust_target_for_scene(128, scene_info)
# Calculate exposure compensation ratio
tes = target / max(current_brightness, 1.0)
# Boost for faces (20% increase)
if scene_info.face_rois:
tes *= 1.2
return tesExposure Distribution Algorithm:
Balances between motion blur (short exposure) and noise (low gain):
def distribute_exposure(self, tes, sensor_info, motion_level):
# Motion factor: 0.5 (high motion) to 1.0 (no motion)
motion_factor = 1.0 - (motion_level * 0.5)
# Calculate exposure time based on motion
# High motion → shorter exposure
# Low motion → longer exposure
time_range = self.max_exposure_time - self.min_exposure_time
optimal_time = self.min_exposure_time + time_range * motion_factor
# Calculate required gain to achieve TES
required_gain = tes / optimal_time
# Clamp to valid ranges
exposure_time = np.clip(optimal_time,
self.min_exposure_time, # 1/10000s = 0.1ms
self.max_exposure_time) # 1/15s = 66ms
analog_gain = np.clip(required_gain,
self.min_gain, # 1.0x
self.max_gain) # 16.0x
digital_gain = 1.0 # Keep at 1.0 for best image quality
return exposure_time, analog_gain, digital_gainExample Calculation:
Given:
- Current brightness: 36 (very dark)
- Target brightness: 128 (middle gray)
- Motion level: 0.3 (low motion)
Step 1: Calculate TES
TES = 128 / 36 = 3.56
Step 2: Calculate motion factor
motion_factor = 1.0 - (0.3 × 0.5) = 0.85
Step 3: Calculate optimal exposure time
optimal_time = 0.0001 + (0.0666 - 0.0001) × 0.85 = 0.0567s ≈ 56.7ms
Step 4: Calculate required gain
required_gain = 3.56 / 0.0567 = 62.8x
Step 5: Clamp values
exposure_time = min(0.0567, 0.0666) = 0.0567s
analog_gain = min(62.8, 16.0) = 16.0x (maxed out)
Result: 56.7ms exposure, 16x gain
Exposure Limits:
self.min_exposure_time = 1/10000 # 0.1ms (for bright scenes/action)
self.max_exposure_time = 1/15 # 66ms (for low light)
self.min_gain = 1.0 # No amplification
self.max_gain = 16.0 # Maximum before excessive noisePrevents banding artifacts caused by AC-powered artificial lighting.
Flicker Fundamentals:
AC lighting flickers at 2× the power frequency:
- 50Hz AC → 100Hz flicker (10ms period)
- 60Hz AC → 120Hz flicker (8.33ms period)
Detection Algorithm:
def detect_flicker(self, exposure_time: float) -> bool:
for freq in [50, 60]: # Common AC frequencies
period = 1.0 / freq
# Check if exposure time is close to a flicker period
if abs(exposure_time - period) < 0.1: # 100ms threshold
return True
return FalseAdjustment Strategy:
def adjust_for_flicker(self, exposure_time: float) -> float:
for freq in [50, 60]:
period = 1.0 / freq
if abs(exposure_time - period) < 0.1:
# Adjust to avoid flicker period
if exposure_time < period:
return period * 0.5 # Use half period
else:
return period * 1.5 # Use 1.5× period
return exposure_time # No adjustment neededAnti-Flicker Exposure Times:
For 60Hz lighting:
- Safe times: 8.33ms, 16.67ms, 25ms, 33.33ms (multiples of 1/120s)
For 50Hz lighting:
- Safe times: 10ms, 20ms, 30ms, 40ms (multiples of 1/100s)
Main coordinator that orchestrates all components.
Processing Pipeline:
def process_frame(self, image, sensor_info):
# 1. Extract image statistics
image_stats = self._extract_image_statistics(image)
# 2. Spatial analysis
spatial_stats = self.spatial_analyzer.analyze_spatial_distribution(
image_stats.bayer_stats
)
# 3. Histogram analysis
histogram_stats = self.histogram_analyzer.analyze_histogram(
image_stats.histogram
)
# 4. Scene analysis
scene_info = self.scene_analyzer.analyze_scene(image, spatial_stats)
# 5. Calculate TES
tes = self.exposure_controller.calculate_total_exposure_sensitivity(
image_stats, scene_info
)
# 6. Distribute exposure
exposure_time, analog_gain, digital_gain = \
self.exposure_controller.distribute_exposure(
tes, sensor_info, scene_info.motion_level
)
# 7. Flicker detection and adjustment
if self.flicker_detector.detect_flicker(exposure_time):
exposure_time = self.flicker_detector.adjust_for_flicker(exposure_time)
# 8. Calculate APEX values
apex_values = self._calculate_apex_values(
exposure_time, analog_gain, sensor_info.aperture
)
# 9. Create result
result = ExposureResult(
exposure_time=exposure_time,
analog_gain=analog_gain,
digital_gain=digital_gain,
total_exposure_sensitivity=tes,
**apex_values
)
# 10. Update system state (for temporal consistency)
self._update_system_state(image, result)
return resultAPEX Value Calculation:
def _calculate_apex_values(self, exposure_time, gain, aperture):
import math
# Brightness Value: Bv = log₂(exposure × gain × aperture)
bv = math.log2(exposure_time * gain * aperture)
# Aperture Value: Av = log₂(f-number²)
av = math.log2(aperture * aperture)
# Time Value: Tv = -log₂(exposure_time)
# Larger Tv = faster shutter
tv = -math.log2(exposure_time)
# Sensitivity Value: Sv = log₂(ISO)
# Using gain × 100 as ISO equivalent
sv = math.log2(gain * 100)
return {
'bv': bv, # Brightness
'av': av, # Aperture
'tv': tv, # Time/Shutter
'sv': sv # Sensitivity/ISO
}Temporal Consistency:
Maintains exposure history for smooth transitions:
def _update_system_state(self, image, result):
self.previous_frame = image.copy()
self.exposure_history.append(result)
# Keep only recent history (10 frames)
if len(self.exposure_history) > 10:
self.exposure_history.pop(0)
# Can be used for:
# - Temporal smoothing (average recent exposures)
# - Change detection (sudden scene changes)
# - Motion estimation (frame-to-frame comparison)The system calculates standard APEX values for EXIF metadata:
| Value | Formula | Description |
|---|---|---|
| Bv (Brightness) | log₂(exposure × gain × aperture) | Scene brightness |
| Av (Aperture) | log₂(f-number²) | Aperture setting |
| Tv (Time) | -log₂(exposure_time) | Shutter speed |
| Sv (Sensitivity) | log₂(gain × 100) | ISO sensitivity |
Real-world test with RAW10 Bayer data (4032x756):
| Metric | Value |
|---|---|
| Input Format | RAW10 Bayer (BayerBG pattern) |
| Exposure Time | 30ms (33.3 fps) |
| Analog Gain | 16.00x |
| Digital Gain | 1.00x |
| TES | 3.53 |
| Scene Type | Backlit |
| Brightness (Bv) | 0.43 |
| Dynamic Range | 227 levels |
| Exposure Bias | -0.07 (slight underexposure) |
The system supports RAW10 format used by mobile camera sensors:
Format Specification:
- 10-bit packed Bayer data
- 4 pixels stored in 5 bytes
- Unpacking:
[P0[9:2]][P1[9:2]][P2[9:2]][P3[9:2]][P3[1:0]P2[1:0]P1[1:0]P0[1:0]]
Supported Bayer Patterns:
- BayerBG (default)
- BayerRG, BayerGB, BayerGR (configurable in code)
- Processing Speed: Real-time (33+ fps)
- Spatial Grid: 64x48 (efficient computation)
- Histogram Bins: 256
- Memory: Low overhead with frame history
- Latency: < 1 frame delay
AutoExposure/
├── README.md # This file - main documentation
├── requirements.txt # Python dependencies
│
├── doc/ # Documentation folder
│ ├── CHAPTER_8_GUIDE.md # Learning roadmap for Chapter 8
│ ├── CODE_ORGANIZATION.md # Code organization guide
│ └── HOW_TO_RUN.md # Quick start guide
│
├── examples/ # Progressive learning examples
│ ├── README.md # Examples guide
│ ├── 01_basic_exposure.py # Level 1: Core concepts
│ ├── 02_image_statistics.py # Level 1: Statistics extraction
│ └── ... # More examples (Levels 2-7)
│
├── core/ # Production-ready implementation
│ └── auto_exposure_system.py # Complete AE system (main implementation)
│
├── data/ # Input data folder
│ └── 20250929_*.raw # RAW10 camera data files
│
├── tests/ # Test suite
│ ├── README.md # Test documentation
│ └── test_ae_system.py # Complete test with RAW10 loader
├── ae_test_results.png # Analysis charts
└── ae_analysis_output/ # Annotated images
└── raw_bayer_4032x756_analyzed.png
from core.auto_exposure_system import SceneType, SceneAnalyzer
# Extend SceneType enum
SceneType.NIGHT_PORTRAIT = "night_portrait"
# Custom scene analyzer
class CustomSceneAnalyzer(SceneAnalyzer):
def _classify_scene(self, image, spatial_stats):
# Your custom classification logic
if self._detect_faces(image) and spatial_stats['brightness_mean'] < 60:
return SceneType.NIGHT_PORTRAIT, 0.9
return super()._classify_scene(image, spatial_stats)from core.auto_exposure_system import ExposureController
class CustomExposureController(ExposureController):
def calculate_total_exposure_sensitivity(self, current_stats, scene_info):
# Custom TES calculation
base_tes = super().calculate_total_exposure_sensitivity(current_stats, scene_info)
# Apply custom adjustments
if scene_info.scene_type == SceneType.NIGHT_PORTRAIT:
base_tes *= 1.5 # Boost exposure for night portraits
return base_tesSolution: Adjust target brightness
ae_system.exposure_controller.target_brightness = 140 # Increase for brighter
ae_system.exposure_controller.target_brightness = 110 # Decrease for darkerSolution: Prioritize faster shutter speed
ae_system.exposure_controller.max_exposure_time = 1/60 # Limit to 16msSolution: Allow longer exposure times
ae_system.exposure_controller.max_exposure_time = 1/15 # Allow up to 66msSolution: Flicker detection is automatic, but you can adjust:
ae_system.flicker_detector.flicker_frequencies = [50, 60, 100]
ae_system.flicker_detector.detection_threshold = 0.15We welcome contributions! Please follow these guidelines:
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Make your changes
- Add tests for new functionality
- Ensure all tests pass
- Update documentation
- Submit a pull request
- Multi-frame HDR exposure bracketing
- Learning-based scene classification with ML models
- Temporal motion analysis across frames
- Advanced face-aware metering with landmarks
- Support for additional RAW formats (RAW12, RAW16)
- GPU acceleration for real-time processing
- Integration with ISP pipelines
- APEX (Additive System of Photographic Exposure) standard
- OpenCV documentation: https://docs.opencv.org/
- Bayer filter demosaicing algorithms
- Computer vision and computational photography principles
This project is licensed under the MIT License.
If you use this system in your research or project, please cite:
Auto Exposure System for Mobile Computational Photography
Implementation based on modern camera ISP principles
https://github.com/your-repo/AutoExposure
For questions, issues, or suggestions, please open an issue on the repository.
Note: All vendor-specific references have been removed. This is an independent implementation of auto exposure algorithms based on industry-standard practices.