📟 PanelBridge

Non-Intrusive Industrial Telemetry Broadcasting System

Securely bridge a hardware-locked factory control panel to mobile devices — without touching the master CPU.

---

🚀 The Problem Statement

Operational Context

In a heavy manufacturing plant, a single master control room screen displays live telemetry across the entire factory floor — including machine temperatures, conveyor belt speeds, raw material flow rates, and active line fault alerts. This screen is the single source of truth for plant health.

The Communication Bottleneck

Senior floor managers frequently needed real-time system status, but were physically separated from the control room. The only available mechanism was manual radio or phone calls to the control room operator — interrupting both parties, creating communication delays, and introducing operational friction during time-sensitive situations.

The Hard Constraint

The most critical constraint of the project was absolute: the master CPU driving the control panel could not be modified in any way. Specifically:

• ❌ No third-party software may be installed on the control unit
• ❌ No network interface or remote access may be configured
• ❌ No changes to the operating system or running processes are permitted

This constraint existed for legitimate reasons — plant stability and industrial cybersecurity policy. Any solution had to be entirely non-intrusive to the master system.

---

💡 Architecture & Workflow

PanelBridge solves this with a hardware-bridged, air-gapped capture pipeline — the master CPU is never touched, networked, or modified in any way.

graph TD;
    A["Master Control CPU <br> (LOCKED) <br> [NO MODIFICATION]"]
    B["HDMI Capture Card (Secondary Unit) <br> Python Capture Script (Screen.py)"]
    C["Node.js Backend Distribution Server <br> (Express + WebSocket)"]
    D["Manager Phone A <br> (Flutter)"]
    E["Manager Phone B <br> (Flutter)"]

    A -- "HDMI Cable <br> (Passive Video Out)" --> B
    B -- "HTTP POST <br> (JPEG Frame)" --> C
    C -- "WebSocket <br> (ws://)" --> D
    C -- "WebSocket <br> (ws://)" --> E

    style A fill:#ffebee,stroke:#c62828,stroke-width:2px,color:#000000
    style B fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000000
    style C fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px,color:#000000
    style D fill:#fff8e1,stroke:#f57f17,stroke-width:2px,color:#000000
    style E fill:#fff8e1,stroke:#f57f17,stroke-width:2px,color:#000000

Loading

Step-by-Step Workflow

1. Hardware Bridge (HDMI Capture Card)

An HDMI capture card is connected between the output port of the master control CPU and a completely separate secondary processing unit. This is a passive hardware tap — no signal is injected, no data is written back, and the control panel continues operating normally.

2. Conditional Python Capture (hardware-client/Screen.py)

A Python script running on the secondary unit polls the backend server via HTTP to check whether any authenticated mobile client is currently connected. Frames are captured from the HDMI input only when actively requested, preventing unnecessary processing and bandwidth usage when no managers are viewing. The script uses OpenCV for device enumeration and frame acquisition, and Pillow for compression before upload.

There are two variants:

• Screen.py — Windows-compatible, using DirectShow (CAP_DSHOW) for HDMI device selection via an interactive CLI prompt.
• Linux_screen.py — Linux-compatible, with automatic /dev/video* enumeration that skips internal webcams and auto-selects an HDMI capture card by resolution threshold.

3. Node.js Distribution Server (backend/index.js)

The backend is a lightweight, stateless Node.js server that acts as the central frame relay and connection manager. It handles:

• Frame ingestion via HTTP multipart upload from the Python client
• Real-time broadcast of incoming frames to all connected WebSocket clients
• Client session management, including live viewer count tracking
• Password-protected access with a separate admin credential system for password rotation
• Camera status propagation — when the capture script goes offline, a camera_offline.png placeholder is broadcast to all clients automatically

4. Flutter Mobile Client (mobile-app/)

Managers install the Flutter application on their Android devices. On launch, the app authenticates against the backend via a password dialog, then opens a persistent WebSocket connection to receive the live frame stream. The UI renders incoming JPEG frames in near-real-time with support for pinch-to-zoom, pause/resume, screen sharing, dark/light mode, and automatic reconnection on network loss.

---

Images

---

✨ Key Features

Feature	Description
Non-Intrusive Hardware Tap	The master CPU is never networked, modified, or aware of the system. Capture is achieved entirely via a passive HDMI output connection.
Conditional Frame Capture	The Python client captures and uploads frames only when authenticated clients are actively connected. When no one is viewing, the capture loop is idle, conserving compute and bandwidth.
Password-Protected Access	All WebSocket connections require prior HTTP authentication. A separate admin credential allows the stream password to be rotated at runtime without restarting the server.
Live Concurrent Viewer Tracking	The mobile app displays the real-time count of all active WebSocket connections, giving each manager awareness of who else is currently viewing the dashboard.
Camera Offline Detection	If the hardware capture script disconnects, the server immediately broadcasts a camera_offline status and a placeholder image to all connected clients.
Automatic Reconnection	The Flutter client implements a debounced reconnection strategy with a safety lock to prevent rapid-retry loops on network interruption.
Cross-Platform Mobile Access	Compiled for Android. iOS support can be enabled in pubspec.yaml with a configuration change.
Pinch-to-Zoom Viewer	The stream viewer supports InteractiveViewer with up to 4× magnification for examining fine detail on the telemetry panel.
Pause & Share	Managers can freeze the current frame for inspection and share it directly via the native device share sheet, with the capture timestamp embedded.
Light / Dark Mode	Full Material 3 theming with a user-toggleable dark mode.
Lifecycle-Aware Connection	The WebSocket connection is automatically torn down when the app is backgrounded and re-established on foreground resume, ensuring accurate viewer counts on the server.

---

🛠 Tech Stack

Hardware

Component	Role
HDMI Capture Card	Passively taps the video output of the master control CPU without any modification to the source machine. Acts as the physical air-gap bridge.
Secondary Processing Unit	An isolated machine (Windows or Linux) that hosts the Python capture script. Has no direct connection to the master CPU beyond the HDMI cable.

Software

hardware-client/ — Python Capture Agent

Library	Purpose
opencv-python	HDMI video device enumeration, frame capture, and resolution handling
Pillow	Frame conversion from BGR to RGB and JPEG compression before upload
requests	HTTP communication with the Node.js backend (frame upload, status polling, video status notification)
logging / RotatingFileHandler	Persistent, size-bounded log file (client.log) for capture diagnostics

backend/ — Node.js Distribution Server

Library	Purpose
express	HTTP server for REST API endpoints (auth, frame upload, status)
ws	WebSocket server for real-time frame broadcasting to mobile clients
multer	Multipart form-data parsing for incoming JPEG frame uploads
cors	Cross-origin request handling for local network clients

mobile-app/ — Flutter Mobile Application

Package	Purpose
web_socket_channel	WebSocket client for receiving live frames from the backend
http	REST API calls for authentication and camera status polling
connectivity_plus	Network state monitoring for triggering reconnection logic
share_plus	Native share sheet integration for frame sharing
path_provider	Temporary file storage for frame export before sharing

---

📈 Business Impact & Beta Testing

PanelBridge was deployed in a live production beta test on an active factory floor, operating under real manufacturing conditions.

Outcomes

• ✅ Eliminated manual radio checks entirely. Senior floor managers could view live telemetry on their mobile devices on demand, without interrupting the control room operator.
• ✅ Demonstrated non-intrusive deployment. The master control CPU continued operating without any modification, change, or awareness of the system throughout the beta — validating the core architectural premise.
• ✅ Received strong approval from senior floor management. The proof-of-concept was formally validated by management as a viable approach for non-intrusive mobile telemetry in industrial environments.
• ✅ Zero operational disruption. The capture pipeline and mobile distribution ran concurrently with live factory operations without interfering with any existing control room systems.

System Performance Observations (Beta)

• Frame delivery latency was acceptable for monitoring workloads (non-real-time telemetry observation)
• Conditional capture logic demonstrably reduced network and CPU load during inactive periods
• Automatic reconnection handling operated correctly across WiFi interruptions during floor mobility

---

⚙️ Installation & Setup

Prerequisites

Component	Requirement
Python	3.8 or higher
Node.js	18.x or higher
Flutter SDK	3.x (Dart SDK ^3.10.4)
HDMI Capture Card	Any capture card that Supports Windows (DirectShow) ans/or Linux
Network	Secondary unit, backend server, and mobile devices must be on the same local network or backend server must be online properly and secondary unit must have internet access

---

1. Backend Server Setup

cd backend
npm install

Before starting, create the credential files in the backend/ directory:

# The password clients use to authenticate
echo "your_stream_password" > password.txt

# The admin password used to rotate the stream password at runtime
echo "your_admin_password" > admin_password.txt

Start the server:

npm start
# ✅ Server running on port 8000

The server binds to 0.0.0.0:8000 and is accessible to all devices on the local network.

---

2. Hardware Capture Client Setup

cd hardware-client
python -m venv venv

# Windows
venv\Scripts\activate

# Linux / macOS
source venv/bin/activate

pip install -r requirements.txt

Configure the server IP by editing the SERVER variable at the top of the relevant script:

# hardware-client/Screen.py  (Windows)
SERVER = "http://192.168.x.x:8000"

# hardware-client/Linux_screen.py  (Linux)
SERVER = "http://192.168.x.x:8000"

Start the capture agent:

# Windows — interactive device selection prompt
python Screen.py

# Linux — automatic HDMI capture card detection
python Linux_screen.py

The script will enumerate available video devices, allow you to select the HDMI capture card index (Windows), and begin the polling loop.

---

3. Mobile App Setup

Configure the server IP in mobile-app/lib/main.dart:

final String serverIp = "192.168.x.x:8000";

Install Flutter dependencies and run:

cd mobile-app
flutter pub get

# Run on a connected Android device
flutter run

# Build a release APK
flutter build apk --release

Note: To enable iOS support, set ios: true in the flutter_launcher_icons section of pubspec.yaml and run flutter build ios.

---

🌐 Network & Deployment Topology

PanelBridge is built to operate entirely within an air-gapped Local Area Network (LAN), requiring zero external internet connectivity or cloud dependencies. This architectural choice ensures maximum data privacy and operational security on the factory floor.

While the initial production beta test was executed using a cloud-hosted Node.js instance for rapid evaluation, the architecture natively relies on local network addressing:

• Node.js Backend Server: Serves as the central data broker listening on port 8000. For production environments, this host machine should be assigned a Static IP address (or mapped via a local DNS entry) to maintain a reliable connection gateway.

• Hardware Client (Python): Operates on a static local IP within the same subnet, securely pushing captured JPEG frames to the server via local HTTP requests.

• Mobile Clients (Flutter): Connect dynamically via standard DHCP assignment across factory Wi-Fi access points, maintaining a persistent state connection to the backend through WebSockets (ws://).

graph LR
    A["Secondary Unit (Python) ══════════════════════ <br> Network: Local Static IP"]
    B["Node.js Backend Server ══════════════════════ <br> Static IP | Port: 8000"]
    C["Mobile Devices (Flutter) ══════════════════════ <br> Network: Dynamic DHCP"]

    A <-- "HTTP POST" --> B
    C <-- "WebSockets" --> B

    style A fill:#e3f2fd,stroke:#1565c0,stroke-width:2px,color:#000000
    style B fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px,color:#000000
    style C fill:#fff8e1,stroke:#f57f17,stroke-width:2px,color:#000000

Loading

---

Repository Structure

PanelBridge/
├── backend/                    # Node.js distribution server
│   ├── index.js                # Main server — Express + WebSocket
│   ├── package.json
│   ├── camera_offline.png      # Placeholder broadcast when capture is offline
│   └── .gitignore
│
├── hardware-client/            # Python HDMI capture agent
│   ├── Screen.py               # Windows — DirectShow capture with device selection
│   ├── Linux_screen.py         # Linux — automatic V4L2 HDMI device detection
│   ├── requirements.txt
│   └── .gitignore
│
└── mobile-app/                 # Flutter Android application
    ├── lib/
    │   ├── main.dart           # App entry point & dependency wiring
    │   ├── core/
    │   │   └── theme/
    │   │       └── app_theme.dart
    │   └── features/
    │       ├── auth/
    │       │   ├── auth_service.dart    # Authentication API client
    │       │   └── auth_dialog.dart     # Password entry & rotation UI
    │       └── telemetry/
    │           ├── telemetry_service.dart      # WebSocket & status polling
    │           ├── live_viewer_screen.dart     # Main stream viewer screen
    │           └── widgets/
    │               ├── image_viewer.dart       # Zoomable live frame display
    │               └── status_banner.dart      # Camera status & viewer count
    ├── pubspec.yaml
    └── .gitignore

---

PanelBridge — Industrial telemetry on mobile, without touching the machine that runs the plant.

Built with a hardware-first approach to solve a real operational problem under strict industrial constraints.