Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,12 @@
# C/C++ build artifacts
build/
cmake-build-*/
CMakeCache.txt
CMakeFiles/
*.o
*.a
*.obj

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
Expand Down
53 changes: 53 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
cmake_minimum_required(VERSION 3.16)
project(videoanalytics LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release)
endif()

find_package(OpenCV REQUIRED)
find_package(Threads REQUIRED)
find_package(nlohmann_json QUIET)

# ZeroMQ (libzmq). Prefer pkg-config, fall back to plain library search.
find_package(PkgConfig QUIET)
if(PkgConfig_FOUND)
pkg_check_modules(ZMQ QUIET libzmq)
endif()
if(NOT ZMQ_FOUND)
find_path(ZMQ_INCLUDE_DIRS zmq.h)
find_library(ZMQ_LIBRARIES NAMES zmq libzmq)
endif()
if(NOT ZMQ_LIBRARIES)
message(FATAL_ERROR "libzmq not found. Install libzmq3-dev (or equivalent).")
endif()

add_library(va_core
src/config.cpp
src/camera.cpp
src/broker.cpp
src/worker.cpp
src/gui.cpp
src/psnr.cpp
src/detector.cpp
)
target_include_directories(va_core PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}/src
${OpenCV_INCLUDE_DIRS}
${ZMQ_INCLUDE_DIRS}
)
target_link_libraries(va_core PUBLIC
${OpenCV_LIBS}
${ZMQ_LIBRARIES}
Threads::Threads
)
if(nlohmann_json_FOUND)
target_link_libraries(va_core PUBLIC nlohmann_json::nlohmann_json)
endif()

add_executable(videoanalytics src/main.cpp)
target_link_libraries(videoanalytics PRIVATE va_core)
113 changes: 89 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,52 +1,117 @@
# videoanalytics

This project is about videosources emergency detection made with ZeroMQ infrastructure
This project is about videosources emergency detection made with ZeroMQ infrastructure.

This is the C++ port of the project (originally written in Python). The
architecture is unchanged: `cameras -> broker -> workers -> gui`, all connected
over ZeroMQ.

## Used technologies
- ZeroMQ
- YOLOv8
- ZeroMQ (libzmq / cppzmq)
- YOLOv8 (via OpenCV DNN, ONNX models)
- PSNR
- OpenCV
- Multiprocessing & Threading
- C++17 threads

## Architecture

```
cameras (PUB) --> broker (SUB -> PUSH) --> workers (PULL -> PUSH) --> gui (PULL)
```

- **cameras** – one thread per source reads a video and publishes JPEG frames
tagged with a 2-byte source id.
- **broker** – subscribes to every source and routes each frame to the worker
whose index matches the source id.
- **workers** – decode the frame, run the fire and forklift detectors plus PSNR
concurrently, and forward `[address, jpeg, detections_json, psnr]`.
- **gui** – renders the latest annotated frame of every source on a grid with
bounding boxes and `WARNING` / `FIRE` overlays.

## Prerequisites

- Torch with CUDA
- Python3.12
- A C++17 compiler
- CMake >= 3.16
- OpenCV 4 (with the `dnn` module)
- ZeroMQ (`libzmq`) and the `cppzmq` header (`zmq.hpp`)
- nlohmann/json

On Debian/Ubuntu:

```bash
sudo apt-get install -y build-essential cmake libzmq3-dev libopencv-dev nlohmann-json3-dev pkg-config
```

## Models

The original project shipped Ultralytics PyTorch checkpoints (`.pt`). The C++
port runs the exported ONNX versions through OpenCV's DNN module. Export them
once with Ultralytics:

```bash
yolo export model=forklift_8s.pt format=onnx
yolo export model=fire.pt format=onnx
```

Place the resulting `.onnx` files under `./models` and point `config.json` at
them. Class names default to `forklift` and `fire`; adjust them in
`src/worker.cpp` if your models use different labels.

## Preparations

- `pip install -r requirements.txt`
- Download all necessary files from [Google Drive](https://drive.google.com/drive/u/0/folders/1OI_XtRNcwbm-JvojeKGR_x1SE1GuonQq) :
- (optional) place test videos & meanframes to `./videos` folder
- place YOLO models to `./models` folder
- set up `config.json` with actual files locations, ip addresses & ports, psnr threshold
- Download the necessary media/models and place test videos & meanframes in
`./videos` and YOLO ONNX models in `./models`.
- Set up `config.json` with actual file locations, ip addresses & ports, and the
psnr threshold.

Notes:
> meanframes must have same resolution as original videos, '.jpg' hardcoded <BR>
> meanframes were created on "normal" parts of videos with `psnr.py`
> meanframes must have the same resolution as the original videos, '.jpg' hardcoded <BR>
> meanframes are created on "normal" parts of videos with the `psnr` subcommand

## Meanframes for PSNR on custom videos
Create meanframes for PSNR function:
- Prepare "normal" videofragment (not containing emergency situations) in video editor
- Use `psnr.py`:
- `python psnr.py <videopath>`
- Meanframe will be created in same directory with same filename in '.jpg' format
## Build

```bash
mkdir -p build && cd build
cmake ..
make -j
```

This produces a single `videoanalytics` executable.

## Usage
After you've set up config.json, just:

Run the full pipeline (broker + workers + cameras + gui):

```bash
python run.py
./build/videoanalytics
```
`Ctrl+C` to stop

`ESC` (in the GUI window) to stop.

Individual components can also be launched separately:

```bash
./build/videoanalytics broker
./build/videoanalytics workers
./build/videoanalytics cameras
./build/videoanalytics gui
```

## Meanframes for PSNR on custom videos
Create meanframes for the PSNR function:
- Prepare a "normal" video fragment (not containing emergency situations).
- Run the meanframe tool:
- `./build/videoanalytics psnr <videopath>`
- The meanframe is created in the same directory with the same filename in
'.jpg' format.

## TODOs
- create meanframes automatically
- train models on another datasets
- add resolution to config
- Ctrl+C handler
- Auto scale processes
- Auto scale processes/threads
- Test on realtime videosources

## Example
![example](./media/example_videoanalytics.gif)
![example](./media/example_videoanalytics.gif)
26 changes: 0 additions & 26 deletions broker.py

This file was deleted.

45 changes: 0 additions & 45 deletions cameras/cam.py

This file was deleted.

57 changes: 0 additions & 57 deletions colors.py

This file was deleted.

4 changes: 2 additions & 2 deletions config.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@
"tcp://127.0.0.1:5565"
],
"models": {
"fire": "./models/fire.pt",
"forklift": "./models/forklift_8s.pt"
"fire": "./models/fire.onnx",
"forklift": "./models/forklift_8s.onnx"
},
"gui": "tcp://127.0.0.1:5562",
"psnr_threshold": 25
Expand Down
5 changes: 0 additions & 5 deletions config.py

This file was deleted.

Loading