diff --git a/.github/README.md b/.github/README.md new file mode 100644 index 000000000..f97d25ffb --- /dev/null +++ b/.github/README.md @@ -0,0 +1,469 @@ +# tr69hostif — TR-069 Host Interface Manager + +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) +[![Version](https://img.shields.io/badge/version-1.3.6-green.svg)](CHANGELOG.md) + +## Overview + +`tr69hostif` is the **TR-069 Host Interface Manager** for RDK-based devices. It acts as the central broker between remote management infrastructure (ACS, WebPA/Parodus) and the TR-181 data model implemented on the device. Any component that needs to read or write TR-181 parameters — including the CWMP stack, WebPA gateway, RFC override system, and SNMP bridge — routes its requests through `tr69hostif`. + +The daemon runs as a persistent systemd service, initializes all TR-181 profile handlers at startup, and then services get/set requests over multiple IPC channels simultaneously. + +## Architecture + +### High-Level Component Diagram + +```mermaid +graph TB + subgraph Remote["Remote Callers"] + ACS[ACS / CWMP Stack] + WebPA[WebPA / parodus] + SNMP[SNMP Manager] + RBUS[RBUS Clients] + end + + subgraph tr69hostif["tr69hostif Daemon"] + IARM[IARM-Bus IPC Handler] + JSON[JSON Request Handler\nPort 10999] + RBUS_P[RBUS DML Provider] + PAR[Parodus PAL\nlibpd] + UPD[Update Handler\nValue Change Events] + MSG[Message Dispatcher\nhostIf_msgHandler] + + subgraph Profiles["TR-181 Profile Handlers"] + DEV[DeviceInfo] + WIFI[WiFi] + ETH[Ethernet] + IP[IP] + MOCA[MoCA] + TIME[Time] + DHCP[DHCPv4] + STBS[STBService\nDS Profile] + STOR[StorageService] + INTF[InterfaceStack] + SNMPA[SNMP Adapter] + end + + subgraph RFC["RFC / Bootstrap"] + RFC_S[RFC Store\nXRFCStorage] + BS_S[Bootstrap Store\nXBSStore] + end + end + + ACS -->|IARM RPC| IARM + SNMP -->|IARM RPC| IARM + WebPA-->|msgpack/WRP| PAR + RBUS -->|rbus API| RBUS_P + JSON -->|HTTP JSON| MSG + + IARM --> MSG + PAR --> MSG + RBUS_P --> MSG + MSG --> Profiles + MSG --> RFC + UPD -->|ValueChanged| IARM + UPD -->|ValueChanged| PAR +``` + +### Request Flow + +```mermaid +sequenceDiagram + participant Caller as Caller (IARM/RBUS/WebPA) + participant MSG as Message Dispatcher + participant PROF as Profile Handler + participant HAL as Platform HAL / OS + + Caller->>MSG: Get/Set paramName + value + MSG->>MSG: Route by prefix (mgrlist.conf) + MSG->>PROF: handler->handleGetMsg() / handleSetMsg() + PROF->>HAL: Read device state / write config + HAL-->>PROF: Raw value + PROF-->>MSG: Populated HOSTIF_MsgData_t + MSG-->>Caller: Response + faultCode +``` + +### Startup Sequence + +```mermaid +sequenceDiagram + participant main as main() + participant CFG as ConfigManager + participant IARM as IARM-Bus + participant DM as DataModel XML + participant THR as Threads + + main->>CFG: hostIf_initalize_ConfigManger() + main->>IARM: hostIf_IARM_IF_Start() + main->>DM: mergeDataModel() + loadDataModel() + main->>THR: json_if_handler_thread (GLib) + main->>THR: http_server_thread (optional, legacy RFC) + main->>THR: updateHandler::Init() (value-change polling) + main->>THR: libpd_client_mgr() (Parodus, if enabled) + main->>THR: initWebConfigTask() (WebConfig, if enabled) + main->>main: init_rbus_dml_provider() + main->>main: sd_notify(READY=1) + main->>main: g_main_loop_run() +``` + +## Key Components + +### Core Daemon (`src/hostif/src/`) + +| File | Purpose | +|------|---------| +| `hostIf_main.cpp` | `main()` entry point: argument parsing, signal handling, thread lifecycle, GLib main loop | +| `hostIf_utils.cpp` | Utility helpers: type conversion, reset state machine, gateway connectivity | +| `IniFile.cpp` | INI file parser used by RFC and Bootstrap stores | + +### Request Handlers (`src/hostif/handlers/`) + +| Handler | IARM Bus Manager Token | TR-181 Subtree | +|---------|----------------------|----------------| +| `hostIf_DeviceClient_ReqHandler` | `deviceMgr` | `Device.DeviceInfo.*` | +| `hostIf_WiFi_ReqHandler` | `wifiMgr` | `Device.WiFi.*` | +| `hostIf_EthernetClient_ReqHandler` | `ethernetMgr` | `Device.Ethernet.*` | +| `hostIf_IPClient_ReqHandler` | `ipMgr` | `Device.IP.*` | +| `hostIf_MoCAClient_ReqHandler` | `mocaMgr` | `Device.MoCA.*` | +| `hostIf_TimeClient_ReqHandler` | `timeMgr` | `Device.Time.*` | +| `hostIf_DHCPv4Client_ReqHandler` | `dhcpv4Mgr` | `Device.DHCPv4.*` | +| `hostIf_dsClient_ReqHandler` | `dsMgr` | `Device.Services.STBService.*` | +| `hostIf_StorageSrvc_ReqHandler` | `storageSrvcMgr` | `Device.Services.StorageService.*` | +| `hostIf_InterfaceStackClient_ReqHandler` | `intfStackMgr` | `Device.InterfaceStack.*` | +| `hostIf_SNMPClient_ReqHandler` | `snmpAdapterMgr` | `Device.X_RDKCENTRAL-COM.*` (SNMP bridge) | +| `hostIf_rbus_Dml_Provider` | — | Exposes all registered params over RBUS | +| `hostIf_updateHandler` | — | Polls profiles for value changes; publishes IARM events | +| `hostIf_NotificationHandler` | — | Queues value-change notifications to Parodus | + +All handlers inherit from the abstract `msgHandler` base class. The `hostIf_msgHandler.cpp` dispatcher instantiates each handler at startup and routes requests by matching the parameter name prefix against the manager map loaded from `tr69hostIf.conf`. + +### TR-181 Profiles (`src/hostif/profiles/`) + +Each subdirectory implements one or more TR-181 objects. Profiles contain the business logic: they read HAL APIs (IARM Device Settings, wifihal, platform sysfs, etc.) and translate results to/from `HOSTIF_MsgData_t`. + +| Profile Directory | TR-181 Object | Key Dependencies | +|-------------------|---------------|-----------------| +| `DeviceInfo/` | `Device.DeviceInfo` | IARM, rfcapi, rfcdefaults, partners\_defaults.json | +| `wifi/` | `Device.WiFi` | wifihal (libwifi) | +| `Ethernet/` | `Device.Ethernet` | sysfs, IARM | +| `IP/` | `Device.IP` | netlink / sysfs | +| `moca/` | `Device.MoCA` | IARM mocaMgr | +| `Time/` | `Device.Time` | NTP daemon, chrony | +| `DHCPv4/` | `Device.DHCPv4` | udhcpc / dnsmasq | +| `STBService/` | `Device.Services.STBService` | IARM Device Settings (DS) | +| `StorageService/` | `Device.Services.StorageService` | sysfs block devices | +| `InterfaceStack/` | `Device.InterfaceStack` | sysfs | +| `Device/` | `Device.*` (root object) | — | + +### RFC & Bootstrap Subsystem (`src/hostif/profiles/DeviceInfo/`) + +| Class | File | Purpose | +|-------|------|---------| +| `XRFCStorage` | `XrdkCentralComRFC.cpp` | Persists RFC override values in an INI file under `/opt/secure/RFC/` | +| `XBSStore` | `XrdkCentralComBSStore.cpp` | Loads per-partner bootstrap defaults from `partners_defaults.json`; owns the background partner-ID resolution thread | +| `XBSStoreJournal` | `XrdkCentralComBSStoreJournal.cpp` | Append-only journal for bootstrap value changes | + +RFC parameter precedence (highest to lowest): + +``` +RFC Override (/opt/secure/RFC/) > WebPA Set > Bootstrap Default > Firmware Default +``` + +### Parodus / WebPA Client (`src/hostif/parodusClient/pal/`) + +| File | Purpose | +|------|---------| +| `libpd.cpp` | Connects to `parodus` process; manages the recv-wait thread | +| `webpa_adapter.cpp` | Translates libparodus WRP messages to `HOSTIF_MsgData_t` | +| `webpa_parameter.cpp` | GetParam / SetParam over WebPA | +| `webpa_attribute.cpp` | GetAttr / SetAttr over WebPA | +| `webpa_notification.cpp` | Pushes value-change events back to parodus | + +### HTTP Server (`src/hostif/httpserver/`) + +An optional Mongoose-based HTTP server (disabled when `NEW_HTTP_SERVER_DISABLE` is defined or when the Legacy RFC feature flag is active). Provides a local REST endpoint used during RFC migration. Controlled at runtime by `/opt/RFC/.RFC_LegacyRFCEnabled.ini`. + +### SNMP Adapter (`src/hostif/snmpAdapter/`) + +Maps selected `Device.X_RDKCENTRAL-COM.*` parameters to SNMP OIDs defined in `conf/tr181_snmpOID.conf`. Enabled at build time with `--enable-snmp-adapter`. + +## Threading Model + +| Thread | Name | How Created | Purpose | +|--------|------|------------|---------| +| Main | `main` | OS | Init, GLib main loop | +| Shutdown | `shutdown_thread` | `pthread_create` | Waits on semaphore; calls `exit_gracefully()` on signal | +| JSON Handler | `json_if_handler_thread` | `g_thread_try_new` | Services JSON-over-socket requests | +| HTTP Server | `http_server_thread` | `g_thread_try_new` | Optional legacy HTTP RFC endpoint | +| Update Handler | `updateHandler` | `g_thread_try_new` | Polls profiles for value changes; fires IARM / Parodus events | +| Parodus Init | `parodus_init_tid` | `pthread_create` | Connects to parodus daemon, starts recv loop | +| WebConfig | `webconfig_threadId` | `pthread_create` | Handles WebConfig Lite document processing | +| Partner ID | `partnerIdThread` | `std::thread` (inside `XBSStore`) | Resolves partner ID asynchronously at boot | + +### Synchronization + +```c +// Signal → shutdown path +sem_t shutdown_thread_sem; // Main signals shutdown thread +pthread_mutex_t graceful_exit_mutex; // Protects shutdown sequence + +// HTTP server startup handshake +std::mutex mtx_httpServerThreadDone; +std::condition_variable cv_httpServerThreadDone; + +// Bootstrap store +static recursive_mutex XBSStore::mtx; // Guards m_dict cache +static mutex XBSStore::mtx_stopped; +static condition_variable XBSStore::cv; + +// Notification queue (lock-free) +GAsyncQueue* NotificationHandler::notificationQueue; +``` + +**Lock ordering**: No nested lock acquisitions exist across manager threads; each subsystem owns its own mutex. The GLib `GAsyncQueue` is used for the notification path to avoid blocking the update handler. + +## Data Structures + +### `HOSTIF_MsgData_t` — the universal request/response envelope + +```c +typedef struct _HostIf_MsgData_t { + char paramName[4096]; // Full TR-181 parameter path + char paramValue[4096]; // Value as string + char *paramValueLong; // Heap buffer for values > 4096 bytes + char transactionID[256]; // Correlation ID (WebPA / CWMP) + short paramLen; // Byte length of paramValue + short instanceNum; // Object instance number + HostIf_ParamType_t paramtype; // String/Int/Bool/DateTime/ULong + HostIf_ReqType_t reqType; // GET / SET / GETATTRIB / SETATTRIB + faultCode_t faultCode; // TR-069 fault code (0 = success) + HostIf_Source_Type_t requestor; // WEBPA / RFC / IARM / DEFAULT + HostIf_Source_Type_t bsUpdate; // Bootstrap source level + bool isLengthyParam; // true → use paramValueLong +} HOSTIF_MsgData_t; +``` + +### Fault Codes + +| Code | Name | Meaning | +|------|------|---------| +| 0 | `fcNoFault` | Success | +| 9000 | `fcMethodNotSupported` | RPC not implemented | +| 9001 | `fcRequestDenied` | Access denied | +| 9002 | `fcInternalError` | Unexpected internal failure | +| 9003 | `fcInvalidArguments` | Bad arguments | +| 9004 | `fcResourcesExceeded` | Resource limit hit | +| 9005 | `fcInvalidParameterName` | Unknown parameter | +| 9006 | `fcInvalidParameterType` | Type mismatch | +| 9007 | `fcInvalidParameterValue` | Value out of range or invalid | +| 9008 | `fcAttemptToSetaNonWritableParameter` | Read-only parameter | + +## Configuration + +### `conf/tr69hostIf.conf` + +```ini +[HOSTIF_DM_PROFILE_MGR] +Device.DeviceInfo=deviceMgr +Device.Services.STBService=dsMgr +Device.Services.StorageService=storageSrvcMgr +Device.MoCA=mocaMgr +Device.Ethernet=ethernetMgr +Device.IP=ipMgr +Device.Time=timeMgr +Device.WiFi=wifiMgr + +[HOSTIF_JSON_CONFIG] +PORT=10999 + +[HOSTIF_CONFIG] +REBOOT_SCR="/rebootNow.sh -s tr69hostIfReset" +RDK_SCR_PATH=/lib/rdk +NTP_FILE_NAME=/opt/persistent/firstNtpTime +FW_DWN_FILE_PATH=/opt/fwdnldstatus.txt +``` + +The `[HOSTIF_DM_PROFILE_MGR]` section defines which manager handles each TR-181 subtree prefix. The dispatcher matches incoming parameter names against these prefixes to route requests. + +### Runtime Feature Flags (RFC) + +| Path | Feature | +|------|---------| +| `/opt/RFC/.RFC_LegacyRFCEnabled.ini` | Enable legacy HTTP server instead of new HTTP server | +| `/opt/secure/RFC/.RFC_.ini` | General RFC feature toggles (created by `XRFCStorage`) | +| `/opt/debug.ini` | RDK logger configuration | + +### Build-Time Feature Flags (`configure.ac`) + +| Configure Flag | Preprocessor Define | Effect | +|----------------|--------------------|----| +| `--enable-parodus` | `PARODUS_ENABLE` | Enable WebPA/Parodus client | +| `--disable-new-http-server` | `NEW_HTTP_SERVER_DISABLE` | Remove internal HTTP server | +| `--enable-snmp-adapter` | `SNMP_ADAPTER_ENABLED` | Include SNMP OID bridge | +| `--enable-webpa-rfc` | `WEBPA_RFC_ENABLED` | Guard service on RFC flag | +| `--enable-rbus` | *(rbus linkage)* | Enable RBUS DML provider | +| `--enable-t2` | `T2_EVENT_ENABLED` | Telemetry 2.0 markers | +| `--enable-webconfig` | `WEB_CONFIG_ENABLED` | WebConfig multipart support | +| `--enable-webconfig-lite` | `WEBCONFIG_LITE_ENABLE` | WebConfig Lite | +| `--enable-wifi` | `USE_WIFI_PROFILE` | WiFi profile handlers | +| `--enable-moca` | *(moca linkage)* | MoCA profile handlers | + +## Build & Install + +### Prerequisites + +| Dependency | Minimum Version | Notes | +|------------|----------------|-------| +| GCC / G++ | 7+ | C++17 required | +| GLib 2 | 2.32+ | GThread, GMainLoop, GAsyncQueue | +| libcurl | 7.65+ | Used by DeviceInfo utilities | +| IARM Bus | — | RDK platform IPC | +| libparodus | — | Required with `--enable-parodus` | +| rbus | — | Required with `--enable-rbus` | +| safec | — | Safe string functions (`strcpy_s`, etc.) | +| cJSON | — | JSON parsing | +| OpenSSL | 1.1.1+ | TLS for HTTP server | + +### Build Steps + +```bash +# Generate build system +autoreconf -iv + +# Configure (example for a typical RDK broadband build) +./configure \ + --enable-parodus \ + --enable-rbus \ + --enable-wifi \ + --enable-moca \ + --enable-t2 + +# Build +make -j$(nproc) + +# Install +make install +``` + +### Run + +```bash +# Typical invocation (as managed by systemd) +/usr/bin/tr69hostIf -c /etc/tr69hostIf.conf -p 10000 + +# Options +# -c Configuration file path +# -p IARM listen port +# -s HTTP server port (legacy mode only) +# -l Log file path +# -h Show usage +``` + +The provided systemd unit files are: +- `tr69hostif.service` — standard deployment +- `tr69hostif_no_new_http_server.service` — deployment with `NEW_HTTP_SERVER_DISABLE` + +## Testing + +### Unit Tests + +```bash +# Build and run unit tests +./run_ut.sh +``` + +Unit tests live under `src/unittest/` and `src/hostif/**/gtest/`. They use **Google Test** and rely on stub headers under `src/unittest/stubs/` to isolate the daemon from IARM, DS, and other platform dependencies. + +Key test areas: + +| Test Suite | Location | Coverage | +|------------|----------|----------| +| RFC Store | `profiles/DeviceInfo/gtest/` | `XRFCStorage` get/set/clear | +| Bootstrap Store | `profiles/DeviceInfo/gtest/` | `XBSStore` partner loading | +| JSON Handler | `handlers/src/gtest/` | Request parsing and routing | +| IARM Handler | `handlers/src/gtest/` | IARM RPC dispatch | +| IniFile | `src/gtest/` | INI parser correctness | + +### Integration / L2 Tests + +```bash +# Run L2 integration tests (requires Docker) +./run_l2.sh +``` + +L2 tests live under `src/integrationtest/` (configuration fixtures) and `test/functional-tests/` (Behave BDD scenarios). They exercise the full daemon end-to-end against mock IARM and RFC infrastructure. + +## Directory Reference + +``` +tr69hostif/ +├── configure.ac # Autoconf top-level +├── Makefile.am # Top-level Automake +├── conf/ # Runtime configuration +│ ├── tr69hostIf.conf # Manager-to-prefix mapping +│ ├── mgrlist.conf # Manager list +│ ├── tr181_snmpOID.conf # SNMP OID mappings +│ └── rfcdefaults/ +│ └── tr69hostif.ini # RFC default values +├── src/ +│ ├── backgroundrun.c # Helper to run scripts in background +│ └── hostif/ +│ ├── src/ # Core daemon source +│ ├── include/ # Core public headers +│ ├── handlers/ # Request dispatching layer +│ ├── profiles/ # TR-181 object implementations +│ ├── parodusClient/ # WebPA / Parodus PAL +│ ├── httpserver/ # Optional HTTP server +│ └── snmpAdapter/ # SNMP bridge +├── test/ +│ └── functional-tests/ # BDD integration tests (Behave) +└── scripts/ + └── validateDataModel.py # Data model XML validation utility +``` + +## Logging + +tr69hostif uses the RDK Logger (`rdk_debug.h`). Log levels map to standard RDK levels: `FATAL`, `ERROR`, `WARN`, `NOTICE`, `INFO`, `DEBUG`, `TRACE1/2`. + +The log category is `LOG_TR69HOSTIF`. To enable verbose logging at runtime, add the following to `/opt/debug.ini`: + +```ini +LOG.RDK.TR69HOSTIF = DEBUG +``` + +Telemetry 2.0 markers (when `T2_EVENT_ENABLED` is defined) are emitted via `t2_event_s()` / `t2_event_d()` for key lifecycle events. + +## Platform Notes + +### RDKB (Broadband Gateway) +- Uses IARM-Bus for all cross-process communication. +- WiFi parameters delegate to the `wifihal` abstraction layer. +- RFC overrides stored under `/opt/secure/RFC/`. +- Bootstrap defaults loaded from `/etc/partners_defaults.json` or `/opt/partners_defaults.json`. + +### RDKV (Video/STB) +- `RDKV_TR69` compile flag activates STB-specific code paths. +- DS (Device Settings) profile enabled; STBService provides HDMI, FPD, audio, and video object support. +- Base data model file: `/etc/data-model.xml` merged with device-type overlays at startup. + +### General Constraints +- Minimum 64 MB RAM recommended. +- ARMv7 or better CPU. +- GLib 2 event loop required (no bare POSIX event loop replacement). + +## Contributing + +See [CONTRIBUTING.md](CONTRIBUTING.md). All contributions require signing the RDK Contributor License Agreement. + +## License + +Licensed under the [Apache License, Version 2.0](LICENSE). + +Copyright 2016 RDK Management. + +## See Also + +- [CHANGELOG](CHANGELOG.md) — Release history +- [conf/tr69hostIf.conf](conf/tr69hostIf.conf) — Runtime configuration reference +- [run_ut.sh](run_ut.sh) — Unit test runner +- [run_l2.sh](run_l2.sh) — L2 integration test runner diff --git a/.github/agents/embedded-programmer.agent.md b/.github/agents/embedded-programmer.agent.md new file mode 100644 index 000000000..1993bd4a1 --- /dev/null +++ b/.github/agents/embedded-programmer.agent.md @@ -0,0 +1,177 @@ +--- +name: 'Embedded Programming Expert' +description: 'Expert in embedded C++ development with focus on resource constraints, memory safety, and platform independence for tr69hostif / TR-069 host interface systems' +tools: ['codebase', 'search', 'edit', 'runCommands', 'runTests', 'problems', 'web'] +--- + +# Embedded C++ Development Expert + +You are an expert embedded systems C++ developer specializing in resource-constrained environments. You have deep knowledge of: + +- Memory management without garbage collection +- Platform-independent C/C++ programming +- Real-time and embedded systems constraints +- RDK (Reference Design Kit) architecture +- TR-069/TR-181 data model management and CWMP protocol +- tr69hostif architecture: handlers, profiles, WebPA/parodus integration, RFC parameter management + +## Your Expertise + +### Memory Management +- RAII patterns in C using cleanup functions +- Memory pools and custom allocators +- Fragmentation prevention strategies +- Stack vs heap tradeoffs +- Valgrind and memory leak detection + +### Thread Safety and Concurrency +- Lightweight synchronization primitives (atomic operations, simple mutexes) +- Deadlock prevention (lock ordering, timeouts) +- Minimal thread memory configuration (pthread attributes) +- Lock-free patterns for embedded systems +- Thread pool design to prevent fragmentation +- Race condition detection and prevention + +### Resource Optimization +- Minimal CPU usage patterns +- Code size reduction techniques +- Static memory allocation strategies +- Efficient data structures for embedded systems +- Zero-copy techniques + +### Platform Independence +- POSIX compliance +- Endianness handling +- Type size portability (stdint.h) +- Build system abstractions +- Hardware abstraction layers + +### Code Quality +- Static analysis (cppcheck, scan-build) +- Unit testing with gtest/gmock from C +- Coverage analysis +- Defensive programming +- Error handling patterns + +## Your Approach + +### When Reviewing Code +1. Check for memory leaks (every malloc needs a free) +2. Verify error handling (all return values checked) +3. Validate resource cleanup (files, mutexes, etc.) +4. Ensure platform independence (no assumptions) +5. Look for buffer overflows and bounds checking +6. Verify thread safety if multi-threaded +7. Check for proper synchronization (no race conditions, no deadlocks) +8. Validate thread creation uses minimal stack attributes +9. Ensure lock-free patterns used where appropriate + +### When Writing Code +1. Start with function signature and error handling +2. Document ownership and lifetime of pointers +3. Use single exit point pattern for cleanup +4. Add bounds checking and validation +5. Write corresponding tests +6. Run valgrind to verify no leaks + +### When Refactoring +1. Don't change behavior (verify with tests) +2. Reduce memory footprint when possible +3. Improve error handling and logging +4. Extract common patterns into functions +5. Maintain backward compatibility +6. Update tests to match changes + +## Guidelines + +### Memory Safety +- Always check malloc/calloc return values +- Free memory in reverse order of allocation +- Use goto for cleanup in complex error paths +- NULL pointers after free to catch double-free +- Use const for read-only data +- Prefer stack allocation for small, fixed-size data + +### Performance +- Profile before optimizing (measure, don't guess) +- Cache frequently accessed data +- Minimize system calls +- Use atomic operations instead of locks when possible +- Keep critical sections minimal +- Use efficient algorithms (avoid O(n²)) +- Consider memory vs speed tradeoffs +- Know your platform's cache sizes + +### Maintainability +- Follow existing code style +- Use meaningful variable names +- Comment non-obvious logic (why, not what) +- Keep functions small and focused +- Avoid premature optimization +- Write self-documenting code + +### Platform Independence +- Use stdint.h for fixed-width types +- Use stdbool.h for boolean +- Handle endianness explicitly +- Don't assume structure packing +- Use configure checks for platform features +- Abstract platform-specific code + +## Anti-Patterns to Avoid + +```c +// Never assume malloc succeeds +char* buf = malloc(size); +strcpy(buf, input); // Crash if malloc failed! + +// Never ignore return values +fwrite(data, size, 1, file); // Did it succeed? + +// Never use magic numbers +if (size > 1024) { ... } // What is 1024? + +// Never leak on error paths +FILE* f = fopen(path, "r"); +if (error) return -1; // Leaked f! + + +// Never create threads with default stack size +pthread_create(&t, NULL, func, arg); // Wastes 8MB! + +// Never use inconsistent lock ordering +pthread_mutex_lock(&lock_a); +pthread_mutex_lock(&lock_b); // OK in func1 +// But in func2: +pthread_mutex_lock(&lock_b); +pthread_mutex_lock(&lock_a); // DEADLOCK! + +7. Use thread sanitizer for concurrent code +8. Test for race conditions with helgrind +9. Verify no deadlocks under load +// Never use heavy locks for simple operations +pthread_rwlock_wrlock(&lock); +counter++; // Use atomic_int instead! +pthread_rwlock_unlock(&lock); +// Never assume integer sizes +long timestamp; // 32 or 64 bits? +``` + +## Testing Focus + +For every change: +1. Write tests that verify the behavior +2. Run tests under valgrind to catch leaks +3. Verify tests pass on target platform +4. Check code coverage (aim for >80%) +5. Run static analysis tools +6. Test error paths and edge cases + +## Communication Style + +- Be direct and specific +- Explain memory implications +- Point out potential issues proactively +- Suggest platform-independent alternatives +- Reference specific line numbers +- Provide complete, working code examples diff --git a/.github/agents/l2-test-runner.agent.md b/.github/agents/l2-test-runner.agent.md new file mode 100644 index 000000000..e557356e7 --- /dev/null +++ b/.github/agents/l2-test-runner.agent.md @@ -0,0 +1,267 @@ +--- +name: 'L2 Test Runner' +description: 'Runs tr69hostif L2 integration tests in Docker containers, reports failures with root-cause analysis, and identifies untested areas. Prefers locally cached container images; asks before pulling or building new ones.' +tools: ['codebase', 'runCommands', 'search', 'edit', 'problems'] +--- + +# L2 Integration Test Runner + +You are a CI/test-execution specialist for the tr69hostif project. Your job is to run the L2 +functional integration test suite locally using Docker containers, exactly as the GitHub Actions +workflow `.github/workflows/L2-tests.yml` does, interpret results, and guide the developer to fix +any failures. + +## Responsibilities + +1. **Run L2 tests** inside the correct Docker containers on the developer's machine. +2. **Prefer local images** — check `docker images` before pulling anything from GHCR. +3. **Never pull or build images without user confirmation** when a pull is required or when + the local image is incompatible. +4. **Report failures** with a triage summary: failing test, assertion text, likely root cause, + and a suggested fix. +5. **Identify untested areas**: after every run, list functional areas with no L2 test coverage. + +--- + +## Container Images + +| Image name | GHCR path | Purpose | +|------------|-----------|---------| +| `mockxconf` | `ghcr.io/rdkcentral/docker-device-mgt-service-test/mockxconf:latest` | Mock XConf / WebPA server | +| `native-platform` | `ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest` | Build host + test runtime | +| `docker-rdk-ci` | `ghcr.io/rdkcentral/docker-rdk-ci:latest` | Results upload to Automatics | + +Container source: **https://github.com/rdkcentral/docker-device-mgt-service-test** + +--- + +## Workflow + +### Step 1 — Check local Docker images + +```bash +docker images --format "table {{.Repository}}:{{.Tag}}\t{{.ID}}\t{{.CreatedAt}}" | grep -E "mockxconf|native-platform" +``` + +- If **both images exist locally** → proceed directly to Step 3. +- If **one or both are missing** → ask the user: + + > "Image `` is not found locally. Should I pull it from GHCR (`docker pull ...`)? + > If the host architecture is incompatible with the pre-built image, I can also guide you + > to build it from source at https://github.com/rdkcentral/docker-device-mgt-service-test + > (requires your approval)." + + **Do not run `docker pull` or `docker build` without explicit user approval.** + +### Step 2 (conditional) — Authenticate, then pull or build + +Only after user approval. Before pulling, attempt GHCR login automatically using the +`rdkcentral` credentials stored in `~/.netrc`: + +```bash +# Extract token from ~/.netrc for ghcr.io +NETRC_TOKEN=$(awk '/machine ghcr.io/{getline; if ($1=="password") print $2}' ~/.netrc) +NETRC_USER=$(awk '/machine ghcr.io/{getline; if ($1=="login") print $2}' ~/.netrc) + +if [ -n "$NETRC_TOKEN" ]; then + echo "$NETRC_TOKEN" | docker login ghcr.io -u "$NETRC_USER" --password-stdin +else + echo "No ghcr.io entry found in ~/.netrc — login skipped." +fi +``` + +If `docker login` fails (exit code ≠ 0), **stop immediately** and show the user this prompt: + +> **GHCR login failed.** To authenticate manually: +> 1. Create a GitHub Personal Access Token (PAT) with `read:packages` scope at +> https://github.com/settings/tokens +> 2. Add it to `~/.netrc`: +> ``` +> machine ghcr.io +> login +> password +> ``` +> 3. Or log in directly: +> ```bash +> echo "" | docker login ghcr.io -u --password-stdin +> ``` +> Re-run the agent once you have authenticated. + +Do not attempt the pull until login succeeds. + +```bash +docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/mockxconf:latest +docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest +``` + +If the image architecture is incompatible with the host (e.g., `exec format error`), present this +prompt to the user instead of retrying the pull: + +> "The pre-built image is not compatible with your host architecture. +> To build compatible images from source, clone +> https://github.com/rdkcentral/docker-device-mgt-service-test and run: +> ```bash +> docker build -t mockxconf -f Dockerfile.mockxconf . +> docker build -t native-platform -f Dockerfile.native-platform . +> ``` +> Shall I proceed with the build?" + +### Step 3 — Handle existing containers + +First check whether `mockxconf` or `native-platform` containers are already running: + +```bash +docker ps --filter "name=mockxconf" --filter "name=native-platform" --format "table {{.Names}}\t{{.Status}}\t{{.CreatedAt}}" +``` + +If **either container exists** (running or stopped), **always ask the user** before removing it: + +> "Found existing container(s): ``. These may be left over from a +> previous test session. Should I stop and remove them to start a clean run? +> (If you are debugging a previous failure, you may want to keep them.)" + +**Do not run `docker rm` or `docker stop` without explicit user approval.** Proceed to +Step 4 only after confirmation. + +### Step 4 — Start mock XConf container + +```bash +docker run -d --name mockxconf \ + -p 50050:50050 -p 50051:50051 -p 50052:50052 -p 50053:50053 \ + -v "$(pwd)":/mnt/L2_CONTAINER_SHARED_VOLUME \ + mockxconf:latest # use local tag, fall back to ghcr.io/… if pulled +``` + +### Step 5 — Start native-platform container + +```bash +docker run -d --name native-platform \ + --link mockxconf \ + -v "$(pwd)":/mnt/L2_CONTAINER_SHARED_VOLUME \ + native-platform:latest +``` + +### Step 6 — Build and run tests + +Run the build and tests as **two separate `docker exec` calls** so that a build failure +can be detected and reported before the test runner is invoked. + +**6a — Build:** +```bash +docker exec -i native-platform /bin/bash -c \ + "cd /mnt/L2_CONTAINER_SHARED_VOLUME/ && sh cov_build.sh" +``` + +If the build exits with a non-zero code: +1. Capture the last 60 lines of compiler output. +2. Present a **Build Failure Summary**: + + ``` + ## Build Failure Summary + + **Exit code:** + + **First error:** + :: error: + + **Compiler output (last 60 lines):** + + + **Next step:** Fix the compiler error above and re-run the agent. + No further build or test steps will be attempted. + ``` +3. **Stop immediately.** Do not retry the build, do not proceed to Step 6b. + +**6b — Run tests** (only if 6a succeeded): +```bash +docker exec -i native-platform /bin/bash -c \ + "export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/lib/aarch64-linux-gnu:/usr/local/lib && \ + cd /mnt/L2_CONTAINER_SHARED_VOLUME/ && sh run_l2.sh" +``` + +### Step 7 — Collect results + +```bash +docker cp native-platform:/tmp/l2_test_report /tmp/L2_TEST_RESULTS +``` + +### Step 8 — Analyse and report + +Parse JSON reports in `/tmp/L2_TEST_RESULTS/` and produce the outputs described below. + +--- + +## Output Format + +### A. Test Run Summary + +| Suite | Total | Passed | Failed | Errors | +|-------|-------|--------|--------|--------| +| bootup_sequence | N | N | N | N | +| handlers_communications | N | N | N | N | +| deviceip | N | N | N | N | +| webpa | N | N | N | N | + +### B. Failure Analysis (one entry per failed test) + +``` +## FAIL: [.json] + +**Assertion:** + + +**Likely cause:** +<2–3 sentence root-cause hypothesis based on test code and source> + +**Suggested fix:** + +``` + +### C. Untested Functionality + +After each run, audit `src/hostif/` against the test suites and list areas with no L2 coverage. +Always check these areas at minimum: + +| Area | Source path | L2 coverage? | +|------|------------|-------------| +| Bootstrap sequence / daemon startup | `src/hostif/src/hostIf_main.cpp` | ✅ | +| TR-181 Device.IP parameter handlers | `src/hostif/profiles/IP/` | ✅ | +| WebPA/parodus GET/SET request handling | `src/hostif/parodusClient/pal/` | ✅ | +| RFC parameter retrieval and override | `src/hostif/handlers/src/` — rfcapi path | ❌ | +| Device.Time parameter handlers | `src/hostif/profiles/Time/` | ❌ | +| STBService profile handlers | `src/hostif/profiles/STBService/` | ❌ | +| SNMP adapter integration | `src/hostif/snmpAdapter/` | ❌ | +| DeviceInfo firmware update status | `src/hostif/profiles/DeviceInfo/` — fwdnld handlers | partial | +| Ethernet interface handlers | `src/hostif/profiles/Ethernet/` | ❌ | +| moca profile handlers | `src/hostif/profiles/moca/` | ❌ | +| WiFi profile handlers | `src/hostif/profiles/wifi/` | ❌ | + +Update this table with actual results from each run (`✅` / `❌` / `partial`). + +--- + +## Rules and Constraints + +- **Never** run `docker pull` or `docker build` without explicit user approval. +- **Never** remove or stop `mockxconf` or `native-platform` containers without asking the user, + even if they look stale — they may be intentionally kept for debugging. +- **Never** stop or remove any container other than `mockxconf` / `native-platform` under any + circumstances. +- **Never** modify source files as part of a test run — only suggest edits. +- **Always** attempt GHCR login from `~/.netrc` before any `docker pull`; if login fails, show + the credential steps prompt and stop. +- **Always** clean up (`docker rm -f mockxconf native-platform`) at the end of a successful run, + unless the user asks to keep containers for debugging. +- If `build_inside_container.sh` fails: capture output, show the Build Failure Summary, and stop. + **Do not retry the build.** Do not attempt any workaround or source patch. +- If architecture incompatibility is detected, present the build-from-source prompt (see Step 2) + and wait for user approval before doing anything else. + +--- + +## Example Invocations + +- "Run the L2 tests and tell me what failed." +- "Run L2 tests using the images I already have." +- "Which parts of the xconf-client are not covered by L2 tests?" +- "L2 tests failed on `test_xconf_connection_with_empty_url` — what should I fix?" diff --git a/.github/agents/legacy-refactor-specialist.agent.md b/.github/agents/legacy-refactor-specialist.agent.md new file mode 100644 index 000000000..dcd7a39bc --- /dev/null +++ b/.github/agents/legacy-refactor-specialist.agent.md @@ -0,0 +1,263 @@ +--- +name: 'Legacy Code Refactoring Specialist' +description: 'Expert in safely refactoring legacy C/C++ code while preventing regressions and maintaining API compatibility' +tools: ['codebase', 'search', 'edit', 'runCommands', 'runTests', 'problems', 'usages'] +--- + +# Legacy Code Refactoring Specialist + +You are a specialist in working with legacy embedded C/C++ code. You follow Michael Feathers' "Working Effectively with Legacy Code" principles adapted for embedded systems. + +## Your Mission + +Improve code quality, reduce technical debt, and enhance maintainability while: +- **Zero regressions**: All existing tests must continue to pass +- **API stability**: Maintain backward compatibility +- **Resource constraints**: Don't increase memory footprint +- **Production safety**: Code ships to millions of devices + +## Your Process + +### 1. Understand Before Changing +- Read and analyze the existing code thoroughly +- Identify all entry points and dependencies +- Map data flow and control flow +- Document current behavior with tests +- Find all callers using search tools + +### 2. Establish Safety Net +- Write characterization tests for existing behavior +- Run tests before ANY changes +- Use static analysis tools (cppcheck, valgrind) +- Create test coverage baseline +- Document any undefined behavior found + +### 3. Make Changes Incrementally +- One small change at a time +- Run full test suite after each change +- Verify memory usage hasn't increased +- Check for new static analysis warnings +- Commit frequently with clear messages + +### 4. Refactoring Patterns + +#### Extract Function +```c +// BEFORE: Long function with mixed concerns +int process_data(const char* input) { + // 200 lines of code doing multiple things + // Parsing, validation, transformation, storage +} + +// AFTER: Extracted, focused functions +static int validate_input(const char* input); +static int parse_data(const char* input, data_t* out); +static int store_data(const data_t* data); + +int process_data(const char* input) { + data_t data; + + if (validate_input(input) != 0) return -1; + if (parse_data(input, &data) != 0) return -1; + if (store_data(&data) != 0) return -1; + + return 0; +} +``` + +#### Introduce Seam (for testing) +```c +// BEFORE: Hard to test due to tight coupling +void process() { + FILE* f = fopen("/etc/config", "r"); + // ... process file ... + fclose(f); +} + +// AFTER: Dependency injection +typedef struct { + FILE* (*open_file)(const char* path); + // ... other dependencies ... +} dependencies_t; + +void process_with_deps(const dependencies_t* deps) { + FILE* f = deps->open_file("/etc/config"); + // ... process file ... + fclose(f); +} + +// Production code +FILE* real_open(const char* path) { return fopen(path, "r"); } +dependencies_t prod_deps = { .open_file = real_open }; + +void process() { + process_with_deps(&prod_deps); +} + +// Test code can inject mocks +``` + +#### Reduce God Object +```c +// BEFORE: Huge structure with everything +typedef struct { + char config_path[256]; + int config_version; + FILE* log_file; + void* data_buffer; + size_t buffer_size; + // ... 50 more fields ... +} context_t; + +// AFTER: Separate concerns +typedef struct { + char path[256]; + int version; +} config_t; + +typedef struct { + FILE* file; +} logger_t; + +typedef struct { + void* buffer; + size_t size; +} data_buffer_t; + +// Compose only what's needed +typedef struct { + config_t* config; + logger_t* logger; + data_buffer_t* buffer; +} context_t; +``` + +### 5. Memory Optimization Patterns + +#### Replace Heap with Stack +```c +// BEFORE: Unnecessary heap allocation +char* format_message(const char* fmt, ...) { + char* buf = malloc(256); + // ... format into buf ... + return buf; // Caller must free +} + +// AFTER: Use stack (if size is known and reasonable) +#define MSG_MAX_SIZE 256 + +int format_message(char* buf, size_t size, const char* fmt, ...) { + // ... format into buf ... + return strlen(buf); +} + +// Caller: +char msg[MSG_MAX_SIZE]; +format_message(msg, sizeof(msg), "Error: %d", code); +``` + +#### Memory Pool for Frequent Allocations +```c +// BEFORE: Frequent malloc/free causing fragmentation +for (int i = 0; i < 1000; i++) { + event_t* e = malloc(sizeof(event_t)); + process_event(e); + free(e); +} + +// AFTER: Pre-allocated pool +#define EVENT_POOL_SIZE 10 + +typedef struct { + event_t events[EVENT_POOL_SIZE]; + bool used[EVENT_POOL_SIZE]; +} event_pool_t; + +event_t* event_pool_acquire(event_pool_t* pool); +void event_pool_release(event_pool_t* pool, event_t* event); + +// Usage +event_pool_t pool = {0}; +for (int i = 0; i < 1000; i++) { + event_t* e = event_pool_acquire(&pool); + process_event(e); + event_pool_release(&pool, e); +} +``` + +## Regression Prevention + +### Before Any Refactoring +1. Ensure all existing tests pass +2. Run valgrind (no leaks in current code) +3. Measure memory footprint baseline +4. Document current behavior + +### During Refactoring +1. Make one logical change at a time +2. Run tests after EVERY change +3. Use git to create checkpoint commits +4. Monitor memory usage + +### After Refactoring +1. All tests still pass +2. No new memory leaks (valgrind) +3. Memory footprint same or better +4. No new compiler warnings +5. Static analysis clean +6. Code review by human + +## Communication + +### When Proposing Changes +- Explain the problem being solved +- Show before/after comparison +- Highlight safety measures +- Document any risks +- Estimate memory impact + +### When Blocked +- Explain what's preventing progress +- Suggest alternatives +- Ask for clarification on requirements +- Note any missing tests + +### Code Review Focus +- Point out missing error handling +- Identify memory leak risks +- Note API compatibility concerns +- Suggest additional test cases +- Highlight complexity that could be simplified + +## Emergency Procedures + +If tests start failing: +1. **STOP** immediately +2. Review the last change +3. Use git diff to see what changed +4. Revert if cause isn't obvious +5. Fix the issue before continuing + +If memory leaks detected: +1. **STOP** the refactoring +2. Run valgrind to identify leak +3. Fix the leak +4. Verify fix with valgrind +5. Resume refactoring + +If API breaks: +1. **REVERT** the breaking change +2. Find alternative approach +3. Use wrapper functions if needed +4. Maintain old API alongside new + +## Success Criteria + +You've succeeded when: +- All tests pass +- No memory leaks (valgrind clean) +- Code is more maintainable +- No API breaks +- Memory footprint same or improved +- Complexity metrics improved +- Test coverage maintained or improved diff --git a/.github/instructions/build-system.instructions.md b/.github/instructions/build-system.instructions.md new file mode 100644 index 000000000..17121156d --- /dev/null +++ b/.github/instructions/build-system.instructions.md @@ -0,0 +1,137 @@ +--- +applyTo: "**/Makefile.am,**/configure.ac,**/*.ac,**/*.mk" +--- + +# Build System Standards (Autotools) + +## Autotools Best Practices + +### configure.ac +- Check for required headers and functions +- Provide clear error messages for missing dependencies +- Support cross-compilation +- Allow feature toggles + +```autoconf +# GOOD: Check for required features +AC_CHECK_HEADERS([pthread.h], [], + [AC_MSG_ERROR([pthread.h is required])]) + +AC_CHECK_LIB([pthread], [pthread_create], [], + [AC_MSG_ERROR([pthread library is required])]) + +# GOOD: Optional features with clear naming +AC_ARG_ENABLE([gtest], + AS_HELP_STRING([--enable-gtest], [Enable Google Test support]), + [enable_gtest=$enableval], + [enable_gtest=no]) + +AM_CONDITIONAL([WITH_GTEST_SUPPORT], [test "x$enable_gtest" = "xyes"]) +``` + +### Makefile.am +- Use non-recursive makefiles when possible +- Minimize intermediate libraries +- Support parallel builds +- Link only what's needed + +```makefile +# GOOD: Minimal linking +bin_PROGRAMS = tr69hostif + +tr69hostif_SOURCES = src/hostif/src/hostIf_main.cpp +tr69hostif_CXXFLAGS = -DFEATURE_SUPPORT_RDKLOG +tr69hostif_LDADD = \ + $(top_builddir)/src/hostif/handlers/libhandlers.la \ + $(top_builddir)/src/hostif/profiles/libprofiles.la \ + -lpthread -ldl + +# GOOD: Conditional compilation +if WITH_GTEST_SUPPORT +SUBDIRS += src/unittest +endif +``` + +## Cross-Compilation Support + +### Platform Detection +```autoconf +# Support different target platforms +case "$host" in + *-linux*) + AC_DEFINE([PLATFORM_LINUX], [1], [Linux platform]) + ;; + *-arm*) + AC_DEFINE([PLATFORM_ARM], [1], [ARM platform]) + ;; +esac +``` + +### Compiler Flags +```makefile +# Platform-specific optimizations +if TARGET_ARM +AM_CFLAGS += -march=armv7-a -mfpu=neon +endif + +# Debug vs Release +if DEBUG_BUILD +AM_CFLAGS += -g -O0 -DDEBUG +else +AM_CFLAGS += -O2 -DNDEBUG +endif +``` + +## Dependency Management + +### Package Config +```autoconf +# Use pkg-config for external dependencies +PKG_CHECK_MODULES([DBUS], [dbus-1 >= 1.6]) +AC_SUBST([DBUS_CFLAGS]) +AC_SUBST([DBUS_LIBS]) +``` + +### Header Organization +```makefile +# Include paths +AM_CPPFLAGS = -I$(top_srcdir)/src/hostif/include \ + -I$(top_srcdir)/src/hostif/handlers/include \ + -I$(top_srcdir)/src/hostif/profiles \ + $(DBUS_CFLAGS) +``` + +## Build Performance + +### Parallel Builds +- Support `make -j` +- Avoid circular dependencies +- Use order-only prerequisites when appropriate + +### Incremental Builds +- Proper dependency tracking +- Don't force full rebuilds unless necessary +- Use libtool for shared libraries + +## Testing Integration + +```makefile +# Test targets +check-local: + @echo "Running memory leak tests..." + @for test in $(TESTS); do \ + valgrind --leak-check=full \ + --error-exitcode=1 \ + ./$$test || exit 1; \ + done + +# Code coverage +if ENABLE_COVERAGE +AM_CFLAGS += --coverage +AM_LDFLAGS += --coverage +endif + +coverage: check + $(LCOV) --capture --directory . --output-file coverage.info + $(GENHTML) coverage.info --output-directory coverage +``` diff --git a/.github/instructions/c-embedded.instructions.md b/.github/instructions/c-embedded.instructions.md new file mode 100644 index 000000000..1ef2a9812 --- /dev/null +++ b/.github/instructions/c-embedded.instructions.md @@ -0,0 +1,693 @@ +--- +applyTo: "**/*.c,**/*.h" +--- + +# C Programming Standards for Embedded Systems + +## Memory Management + +### Allocation Rules +- **Prefer stack allocation** for fixed-size, short-lived data +- **Use malloc/free** only when necessary; always pair them +- **Check all allocations**: Never assume malloc succeeds +- **Free in reverse order** of allocation to reduce fragmentation +- **Use memory pools** for frequent same-size allocations +- **Zero memory after free** to catch use-after-free bugs in debug builds + +```c +// GOOD: Stack allocation for fixed-size data +char buffer[256]; + +// GOOD: Checked heap allocation with cleanup +char* data = malloc(size); +if (!data) { + log_error("Failed to allocate %zu bytes", size); + return ERR_NO_MEMORY; +} +// ... use data ... +free(data); +data = NULL; // Prevent double-free + +// BAD: Unchecked allocation +char* data = malloc(size); +strcpy(data, input); // Crash if malloc failed +``` + +### Memory Leak Prevention +- Every function that allocates must document ownership transfer +- Use goto for single exit point in complex error handling +- Implement cleanup functions for complex structures +- Use valgrind regularly during development + +```c +// GOOD: Single exit point with cleanup +int process_data(const char* input) { + int ret = 0; + char* buffer = NULL; + FILE* file = NULL; + + buffer = malloc(BUFFER_SIZE); + if (!buffer) { + ret = ERR_NO_MEMORY; + goto cleanup; + } + + file = fopen(input, "r"); + if (!file) { + ret = ERR_FILE_OPEN; + goto cleanup; + } + + // ... processing ... + +cleanup: + free(buffer); + if (file) fclose(file); + return ret; +} +``` + +## Resource Constraints + +### Code Size Optimization +- Avoid inline functions unless proven beneficial +- Share common code paths +- Use function pointers for conditional logic in tables +- Strip debug symbols in release builds + +### CPU Optimization +- Minimize system calls +- Cache frequently accessed data +- Use efficient algorithms (prefer O(n) over O(n²)) +- Avoid floating point on devices without FPU +- Profile before optimizing (don't guess) + +### Memory Optimization +- Use bitfields for boolean flags +- Pack structures to minimize padding +- Use const for read-only data (goes in .rodata) +- Prefer static buffers with maximum sizes when bounds are known +- Implement object pools for frequently created/destroyed objects + +```c +// GOOD: Packed structure +typedef struct __attribute__((packed)) { + uint8_t flags; + uint16_t id; + uint32_t timestamp; + char name[32]; +} telemetry_event_t; + +// GOOD: Const data in .rodata +static const char* const ERROR_MESSAGES[] = { + "Success", + "Out of memory", + "Invalid parameter", + // ... +}; +``` + +## Platform Independence + +### Never Assume +- Pointer size (use uintptr_t for pointer arithmetic) +- Byte order (use htonl/ntohl for network data) +- Structure packing (use __attribute__((packed)) or #pragma pack) +- Integer sizes (use int32_t, uint64_t from stdint.h) +- Boolean type (use stdbool.h) + +```c +// GOOD: Platform-independent types +#include +#include + +typedef struct { + uint32_t id; // Always 32 bits + uint64_t timestamp; // Always 64 bits + bool enabled; // Standard boolean +} config_t; + +// GOOD: Endianness handling +uint32_t network_value = htonl(host_value); + +// BAD: Assumptions +int id; // Size varies by platform +long timestamp; // 32 or 64 bits depending on platform +``` + +### Abstraction Layers +- Use platform abstraction for OS-specific code +- Isolate hardware dependencies +- Use configure.ac to detect platform capabilities + +## Error Handling + +### Return Value Convention +- Return 0 for success, negative for errors +- Use errno for system call failures +- Define error codes in header files +- Never ignore return values + +```c +// GOOD: Consistent error handling +typedef enum { + T2ERROR_SUCCESS = 0, + T2ERROR_FAILURE = -1, + T2ERROR_INVALID_PARAM = -2, + T2ERROR_NO_MEMORY = -3, + T2ERROR_TIMEOUT = -4 +} T2ERROR; + +T2ERROR init_telemetry() { + if (!validate_config()) { + return T2ERROR_INVALID_PARAM; + } + + if (allocate_resources() != 0) { + return T2ERROR_NO_MEMORY; + } + + return T2ERROR_SUCCESS; +} +``` + +### Logging +- Use severity levels appropriately +- Log errors with context (function, line, errno) +- Avoid logging in hot paths +- Make logging configurable at runtime +- Never log sensitive data + +```c +// GOOD: Contextual error logging +if (ret != 0) { + T2Error("%s:%d Failed to initialize: %s (errno=%d)", + __FUNCTION__, __LINE__, strerror(errno), errno); + return T2ERROR_FAILURE; +} +``` + +## Thread Safety and Concurrency + +### Critical Principles + +- **Minimize synchronization overhead**: Use lightweight primitives +- **Prevent deadlocks**: Establish lock ordering, use timeouts +- **Avoid memory fragmentation**: Configure thread stack sizes appropriately +- **Reduce contention**: Design for lock-free patterns where possible +- **Document thread safety**: Mark functions as thread-safe or not + +### Thread Creation with Minimal Memory + +Always create threads with attributes that specify required memory: + +```c +// GOOD: Thread with minimal stack size +#include + +#define THREAD_STACK_SIZE (64 * 1024) // 64KB instead of default (often 8MB) + +pthread_t thread; +pthread_attr_t attr; + +// Initialize attributes +pthread_attr_init(&attr); + +// Set minimal stack size (reduces memory fragmentation) +pthread_attr_setstacksize(&attr, THREAD_STACK_SIZE); + +// Detached threads free resources immediately when done +pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED); + +// Create thread +int ret = pthread_create(&thread, &attr, thread_function, arg); +if (ret != 0) { + T2Error("Failed to create thread: %s", strerror(ret)); + pthread_attr_destroy(&attr); + return T2ERROR_FAILURE; +} + +// Clean up attributes +pthread_attr_destroy(&attr); + +// BAD: Default thread (wastes memory) +pthread_create(&thread, NULL, thread_function, arg); // Uses 8MB stack! +``` + +### Lightweight Synchronization + +Prefer lightweight synchronization primitives to avoid deadlocks and overhead: + +```c +// GOOD: Simple mutex with minimal overhead +typedef struct { + pthread_mutex_t lock; + int counter; +} thread_safe_counter_t; + +int init_counter(thread_safe_counter_t* c) { + // Use default attributes (lightest weight) + pthread_mutex_init(&c->lock, NULL); + c->counter = 0; + return 0; +} + +void increment_counter(thread_safe_counter_t* c) { + pthread_mutex_lock(&c->lock); + c->counter++; + pthread_mutex_unlock(&c->lock); +} + +void cleanup_counter(thread_safe_counter_t* c) { + pthread_mutex_destroy(&c->lock); +} + +// GOOD: Use atomic operations when possible (no locks needed) +#include + +typedef struct { + atomic_int counter; // Lock-free! +} lockfree_counter_t; + +void increment_lockfree(lockfree_counter_t* c) { + atomic_fetch_add(&c->counter, 1); // No mutex overhead +} +``` + +### Deadlock Prevention + +Follow strict rules to prevent deadlocks: + +```c +// GOOD: Consistent lock ordering +typedef struct { + pthread_mutex_t lock_a; + pthread_mutex_t lock_b; + // ... data ... +} resource_t; + +// RULE: Always acquire locks in alphabetical order (a, then b) +void multi_lock_operation(resource_t* r) { + pthread_mutex_lock(&r->lock_a); // First: lock_a + pthread_mutex_lock(&r->lock_b); // Second: lock_b + + // ... critical section ... + + pthread_mutex_unlock(&r->lock_b); // Release in reverse order + pthread_mutex_unlock(&r->lock_a); +} + +// GOOD: Use trylock with timeout to avoid indefinite blocking +#include + +int safe_lock_with_timeout(pthread_mutex_t* lock, int timeout_ms) { + struct timespec ts; + clock_gettime(CLOCK_REALTIME, &ts); + ts.tv_sec += timeout_ms / 1000; + ts.tv_nsec += (timeout_ms % 1000) * 1000000; + + int ret = pthread_mutex_timedlock(lock, &ts); + if (ret == ETIMEDOUT) { + T2Error("Lock timeout - potential deadlock detected"); + return -1; + } + return ret; +} + +// BAD: Different lock order in different functions (DEADLOCK RISK!) +void bad_function_1(resource_t* r) { + pthread_mutex_lock(&r->lock_a); + pthread_mutex_lock(&r->lock_b); // Order: a, b + // ... +} + +void bad_function_2(resource_t* r) { + pthread_mutex_lock(&r->lock_b); + pthread_mutex_lock(&r->lock_a); // Order: b, a - DEADLOCK! + // ... +} +``` + +### Avoid Heavy Synchronization + +Heavy synchronization causes performance issues and fragmentation: + +```c +// BAD: Reader-writer lock for simple counter (overkill) +pthread_rwlock_t heavy_lock; +int counter; + +void heavy_increment() { + pthread_rwlock_wrlock(&heavy_lock); // Too heavy! + counter++; + pthread_rwlock_unlock(&heavy_lock); +} + +// GOOD: Use appropriate synchronization level +atomic_int light_counter; // Lock-free for simple operations + +void light_increment() { + atomic_fetch_add(&light_counter, 1); // No lock overhead +} + +// BAD: Fine-grained locking everywhere (lock thrashing) +typedef struct { + pthread_mutex_t lock; + int value; +} each_field_locked_t; // Don't do this! + +// GOOD: Coarse-grained locking for related data +typedef struct { + pthread_mutex_t lock; + int value_a; + int value_b; + int value_c; // All protected by one lock +} properly_locked_t; +``` + +### Lock-Free Patterns + +Use lock-free patterns to avoid synchronization overhead: + +```c +// GOOD: Lock-free flag +#include + +typedef struct { + atomic_bool shutdown_requested; +} thread_control_t; + +void request_shutdown(thread_control_t* ctrl) { + atomic_store(&ctrl->shutdown_requested, true); +} + +bool should_shutdown(thread_control_t* ctrl) { + return atomic_load(&ctrl->shutdown_requested); +} + +// GOOD: Lock-free queue for single producer, single consumer +typedef struct { + atomic_int read_index; + atomic_int write_index; + void* buffer[256]; +} spsc_queue_t; + +bool spsc_enqueue(spsc_queue_t* q, void* item) { + int write = atomic_load(&q->write_index); + int next_write = (write + 1) % 256; + + if (next_write == atomic_load(&q->read_index)) { + return false; // Queue full + } + + q->buffer[write] = item; + atomic_store(&q->write_index, next_write); + return true; +} +``` + +### Minimize Critical Sections + +Keep locked sections as short as possible: + +```c +// BAD: Long critical section +void bad_process(data_t* shared) { + pthread_mutex_lock(&shared->lock); + + // Heavy computation while holding lock (BAD!) + for (int i = 0; i < 1000000; i++) { + compute_something(); + } + + shared->value = result; + pthread_mutex_unlock(&shared->lock); +} + +// GOOD: Minimal critical section +void good_process(data_t* shared) { + // Do heavy computation WITHOUT lock + int result = 0; + for (int i = 0; i < 1000000; i++) { + result += compute_something(); + } + + // Lock only for the update + pthread_mutex_lock(&shared->lock); + shared->value = result; + pthread_mutex_unlock(&shared->lock); +} +``` + +### Thread-Safe Initialization + +Use pthread_once for thread-safe initialization: + +```c +// GOOD: Thread-safe singleton initialization +static pthread_once_t init_once = PTHREAD_ONCE_INIT; +static config_t* global_config = NULL; + +static void init_config_once(void) { + global_config = malloc(sizeof(config_t)); + // ... initialize config ... +} + +config_t* get_config(void) { + pthread_once(&init_once, init_config_once); + return global_config; +} + +// BAD: Double-checked locking (broken in C without memory barriers) +static pthread_mutex_t init_lock; +static config_t* config = NULL; + +config_t* bad_get_config(void) { + if (config == NULL) { // First check (no lock) + pthread_mutex_lock(&init_lock); + if (config == NULL) { // Second check + config = malloc(sizeof(config_t)); // Race condition! + } + pthread_mutex_unlock(&init_lock); + } + return config; +} +``` + +### Thread Safety Documentation + +Always document thread safety expectations: + +```c +// GOOD: Clear thread safety documentation + +/** + * Process telemetry event + * @param event Event to process + * @return 0 on success, negative on error + * + * Thread Safety: This function is thread-safe and may be called + * from multiple threads concurrently. + */ +int process_event(const event_t* event) { + // Uses internal locking +} + +/** + * Initialize event processor + * @return 0 on success, negative on error + * + * Thread Safety: NOT thread-safe. Must be called once during + * initialization before any worker threads start. + */ +int init_event_processor(void) { + // No locking - initialization only +} + +/** + * Get current statistics + * @param stats Output buffer for statistics + * + * Thread Safety: Caller must hold stats_lock before calling. + * Use get_stats_safe() for automatic locking. + */ +void get_stats_unlocked(stats_t* stats) { + // Assumes caller holds lock +} +``` + +### Memory Fragmentation Prevention + +Configure thread pools to prevent fragmentation: + +```c +// GOOD: Thread pool with pre-allocated threads +#define THREAD_POOL_SIZE 4 +#define WORK_QUEUE_SIZE 256 + +typedef struct { + pthread_t threads[THREAD_POOL_SIZE]; + pthread_attr_t thread_attr; + // ... work queue ... +} thread_pool_t; + +int init_thread_pool(thread_pool_t* pool) { + // Configure thread attributes once + pthread_attr_init(&pool->thread_attr); + pthread_attr_setstacksize(&pool->thread_attr, THREAD_STACK_SIZE); + pthread_attr_setdetachstate(&pool->thread_attr, PTHREAD_CREATE_JOINABLE); + + // Create fixed number of threads (no dynamic allocation) + for (int i = 0; i < THREAD_POOL_SIZE; i++) { + int ret = pthread_create(&pool->threads[i], &pool->thread_attr, + worker_thread, pool); + if (ret != 0) { + // Cleanup already created threads + cleanup_partial_pool(pool, i); + return -1; + } + } + + return 0; +} + +// BAD: Creating threads dynamically (causes fragmentation) +void bad_handle_request(request_t* req) { + pthread_t thread; + pthread_create(&thread, NULL, handle_one_request, req); + pthread_detach(thread); // New thread for each request! +} +``` + +### Testing Thread Safety + +```c +// GOOD: Test for race conditions +#include + +TEST(ThreadSafety, ConcurrentIncrement) { + thread_safe_counter_t counter = {0}; + init_counter(&counter); + + const int NUM_THREADS = 10; + const int INCREMENTS_PER_THREAD = 1000; + pthread_t threads[NUM_THREADS]; + + // Create multiple threads + for (int i = 0; i < NUM_THREADS; i++) { + pthread_create(&threads[i], NULL, + increment_n_times, &counter); + } + + // Wait for all threads + for (int i = 0; i < NUM_THREADS; i++) { + pthread_join(threads[i], NULL); + } + + // Verify no race conditions + EXPECT_EQ(counter.counter, NUM_THREADS * INCREMENTS_PER_THREAD); + + cleanup_counter(&counter); +} +``` + +### Static Analysis for Concurrency + +```bash +# Use thread sanitizer to detect race conditions +gcc -g -fsanitize=thread source.c -o program +./program + +# Use helgrind (valgrind) to detect synchronization issues +valgrind --tool=helgrind ./program + +# Check for deadlocks +valgrind --tool=helgrind --track-lockorders=yes ./program +``` + +## Code Style + +### Naming Conventions +- Functions: `snake_case` (e.g., `init_telemetry`) +- Types: `snake_case_t` (e.g., `telemetry_event_t`) +- Macros/Constants: `UPPER_SNAKE_CASE` (e.g., `MAX_BUFFER_SIZE`) +- Global variables: `g_` prefix (avoid when possible) +- Static variables: `s_` prefix + +### File Organization +- One .c file per module +- Corresponding .h file for public interface +- Internal functions marked static +- Header guards in all .h files + +```c +// GOOD: header guard +#ifndef TELEMETRY_INTERNAL_H +#define TELEMETRY_INTERNAL_H + +// ... declarations ... + +#endif /* TELEMETRY_INTERNAL_H */ +``` + +## Testing Requirements + +### Unit Tests +- Test all public functions +- Test error paths and edge cases +- Use mocks for external dependencies +- Verify resource cleanup (no leaks) +- Run tests under valgrind + +### Memory Testing +```bash +# Run with memory checking +valgrind --leak-check=full --show-leak-kinds=all \ + --track-origins=yes ./test_binary + +# Static analysis +cppcheck --enable=all --inconclusive source/ +``` + +## Anti-Patterns to Avoid + +```c +// BAD: Magic numbers +if (size > 1024) { ... } + +// GOOD: Named constants +#define MAX_PACKET_SIZE 1024 +if (size > MAX_PACKET_SIZE) { ... } + +// BAD: Unchecked allocation +char* buf = malloc(size); +strcpy(buf, input); + +// GOOD: Checked with cleanup +char* buf = malloc(size); +if (!buf) return ERR_NO_MEMORY; +strncpy(buf, input, size - 1); +buf[size - 1] = '\0'; + +// BAD: Memory leak in error path +FILE* f = fopen(path, "r"); +if (condition) return -1; // Leaked f +fclose(f); + +// GOOD: Cleanup on all paths +FILE* f = fopen(path, "r"); +if (!f) return -1; +if (condition) { + fclose(f); + return -1; +} +fclose(f); +return 0; +``` + +## References + +- Project follows RDK coding standards +- See `src/hostif/include/` for tr69hostif API header documentation +- Review existing code in `src/hostif/` for patterns +- Check `src/unittest/` directory for testing examples diff --git a/.github/instructions/cpp-testing.instructions.md b/.github/instructions/cpp-testing.instructions.md new file mode 100644 index 000000000..0e1bcf82b --- /dev/null +++ b/.github/instructions/cpp-testing.instructions.md @@ -0,0 +1,178 @@ +--- +applyTo: "src/unittest/**/*.cpp,src/unittest/**/*.h,src/hostif/**/gtest/**/*.cpp,src/hostif/**/gtest/**/*.h" +--- + +# C++ Testing Standards (Google Test) + +## Test Framework + +Use Google Test (gtest) and Google Mock (gmock) for all C++ test code. + +## Test Organization + +### File Structure +- One test file per source file: `foo.c` → `test/FooTest.cpp` +- Test fixtures for complex setups +- Mocks in separate files when reusable + +```cpp +// GOOD: Test file structure +// filepath: src/unittest/hostIf_utils_Test.cpp + +extern "C" { +#include "hostIf_utils.h" +#include "IniFile.h" +} + +#include +#include + +class HostIfUtilsTest : public ::testing::Test { +protected: + void SetUp() override { + // Initialize test resources + } + + void TearDown() override { + // Clean up test resources + } +}; + +TEST_F(HostIfUtilsTest, IniFileReadWriteRoundTrip) { + IniFile ini; + ini.load("/tmp/test.ini"); + // verify read back value matches written value + ASSERT_EQ(ini.get("key"), "value"); +} +``` + +## Testing Patterns + +### Test C Code from C++ +- Wrap C headers in `extern "C"` blocks +- Use RAII in tests for automatic cleanup +- Mock C functions using gmock when needed + +```cpp +extern "C" { +#include "hostIf_main.h" +#include "hostIf_tr69ReqHandler.h" +} + +#include + +class HostIfHandlerTest : public ::testing::Test { +protected: + void SetUp() override { + // Initialize handler stubs + } + + void TearDown() override { + // Clean up + } +}; + +TEST_F(HostIfHandlerTest, GetParamValueReturnsExpected) { + HOSTIF_MsgData_t msgData = {}; + strncpy(msgData.paramName, "Device.DeviceInfo.Manufacturer", sizeof(msgData.paramName) - 1); + msgData.reqType = HOSTIF_GET; + // verify handler returns success and populates paramValue +} +``` + +### Memory Leak Testing +- All tests must pass valgrind +- Use RAII wrappers for C resources +- Verify cleanup in TearDown + +```cpp +// GOOD: RAII wrapper for C resource +class FileHandle { + FILE* file_; +public: + explicit FileHandle(const char* path, const char* mode) + : file_(fopen(path, mode)) {} + + ~FileHandle() { + if (file_) fclose(file_); + } + + FILE* get() const { return file_; } + bool valid() const { return file_ != nullptr; } +}; + +TEST(FileTest, ReadConfig) { + FileHandle file("/tmp/config.json", "r"); + ASSERT_TRUE(file.valid()); + // file automatically closed when test exits +} +``` + +### Mocking External Dependencies + +```cpp +// GOOD: Mock for handler dependencies +class MockIniFile { +public: + MOCK_METHOD(std::string, get, (const std::string& key)); + MOCK_METHOD(bool, set, (const std::string& key, const std::string& value)); +}; + +TEST(HandlerTest, GetParamUsesIniFile) { + MockIniFile mock; + + EXPECT_CALL(mock, get("Device.DeviceInfo.Manufacturer")) + .WillOnce(testing::Return("TestVendor")); + + std::string result = mock.get("Device.DeviceInfo.Manufacturer"); + EXPECT_EQ("TestVendor", result); +} +``` + +## Test Quality Standards + +### Coverage Requirements +- All public functions must have tests +- Test both success and failure paths +- Test boundary conditions +- Test error handling + +### Test Naming +```cpp +// Pattern: TEST(ComponentName, BehaviorBeingTested) + +TEST(Vector, CreateReturnsNonNull) { ... } +TEST(Vector, DestroyHandlesNull) { ... } +TEST(Vector, PushIncrementsSize) { ... } +TEST(Utils, ParseConfigInvalidJson) { ... } +``` + +### Assertions +- Use `ASSERT_*` when test can't continue after failure +- Use `EXPECT_*` when subsequent checks are still valuable +- Provide helpful failure messages + +```cpp +// GOOD: Informative assertions +ASSERT_NE(nullptr, ptr) << "Failed to allocate " << size << " bytes"; +EXPECT_EQ(expected, actual) << "Mismatch at index " << i; +EXPECT_TRUE(condition) << "Context: " << debug_info; +``` + +## Running Tests + +### Build Tests +```bash +./configure --enable-gtest +make check +``` + +### Memory Checking +```bash +valgrind --leak-check=full --show-leak-kinds=all \ + ./src/unittest/tr69hostif_gtest +``` + +### Test Output +- Use `GTEST_OUTPUT=xml:results.xml` for CI integration +- Check return code: 0 = all passed diff --git a/.github/instructions/shell-scripts.instructions.md b/.github/instructions/shell-scripts.instructions.md new file mode 100644 index 000000000..a25a2c69b --- /dev/null +++ b/.github/instructions/shell-scripts.instructions.md @@ -0,0 +1,179 @@ +--- +applyTo: "**/*.sh" +--- + +# Shell Script Standards for Embedded Systems + +## Platform Independence + +### Use POSIX Shell +- Use `#!/bin/sh` not `#!/bin/bash` +- Avoid bashisms (use shellcheck to verify) +- Test on busybox ash (common in embedded) + +```bash +#!/bin/sh +# GOOD: POSIX compliant + +# BAD: Bash-specific +if [[ $var == "value" ]]; then # Use [ ] instead + array=(1 2 3) # Arrays not in POSIX +fi + +# GOOD: POSIX compliant +if [ "$var" = "value" ]; then + set -- 1 2 3 # Use positional parameters +fi +``` + +## Resource Awareness + +### Minimize Process Spawning +- Use shell builtins when possible +- Avoid pipes when not necessary +- Batch operations to reduce forks + +```bash +# BAD: Multiple processes +cat file | grep pattern | wc -l + +# GOOD: Fewer processes +grep -c pattern file + +# BAD: Loop with external commands +for file in *.txt; do + cat "$file" >> output +done + +# GOOD: Single cat invocation +cat *.txt > output +``` + +### Memory Usage +- Avoid reading entire files into variables +- Process streams line by line +- Clean up temporary files + +```bash +# BAD: Loads entire file into memory +content=$(cat large_file.log) +echo "$content" | grep ERROR + +# GOOD: Stream processing +grep ERROR large_file.log + +# GOOD: Line-by-line processing +while IFS= read -r line; do + process_line "$line" +done < large_file.log +``` + +## Error Handling + +### Always Check Exit Codes +```bash +# GOOD: Check critical operations +if ! mkdir -p /tmp/telemetry; then + logger -t telemetry "ERROR: Failed to create directory" + exit 1 +fi + +# GOOD: Use set -e for fail-fast +set -e # Exit on any error +set -u # Exit on undefined variable +set -o pipefail # Catch errors in pipes + +# GOOD: Trap for cleanup +cleanup() { + rm -f "$TEMP_FILE" +} +trap cleanup EXIT INT TERM + +TEMP_FILE=$(mktemp) +# ... use temp file ... +# cleanup happens automatically +``` + +## Script Quality + +### Defensive Programming +```bash +# GOOD: Quote all variables +rm -f "$file_path" # Not: rm -f $file_path + +# GOOD: Use -- to separate options from arguments +grep -r -- "$pattern" "$directory" + +# GOOD: Check variable is set +: "${CONFIG_FILE:?CONFIG_FILE must be set}" + +# GOOD: Validate inputs +if [ -z "$1" ]; then + echo "Usage: $0 " >&2 + exit 1 +fi +``` + +### Logging +```bash +# Use logger for syslog integration +log_info() { + logger -t telemetry -p user.info "$*" +} + +log_error() { + logger -t telemetry -p user.error "$*" + echo "ERROR: $*" >&2 +} + +# Usage +log_info "Starting telemetry collection" +if ! start_service; then + log_error "Failed to start service" + exit 1 +fi +``` + +## Testing Scripts + +### Use shellcheck +```bash +# Run shellcheck on all scripts +shellcheck script.sh + +# In CI +find . -name "*.sh" -exec shellcheck {} + +``` + +### Test on Target Platform +- Test on actual embedded device or emulator +- Verify with busybox tools +- Check resource usage (memory, CPU) + +## Anti-Patterns + +```bash +# BAD: Unquoted variables +for file in $FILES; do # Word splitting! + +# GOOD: Quoted +for file in "$FILES"; do + +# BAD: Parsing ls output +for file in $(ls *.txt); do + +# GOOD: Use glob +for file in *.txt; do + +# BAD: Useless use of cat +cat file | grep pattern + +# GOOD: grep can read files +grep pattern file + +# BAD: Not checking if file exists +rm /tmp/file # Error if doesn't exist + +# GOOD: Check or use -f +rm -f /tmp/file # Or: [ -f /tmp/file ] && rm /tmp/file +``` diff --git a/.github/skills/memory-safety-analyzer/SKILL.md b/.github/skills/memory-safety-analyzer/SKILL.md new file mode 100644 index 000000000..5d2d9b293 --- /dev/null +++ b/.github/skills/memory-safety-analyzer/SKILL.md @@ -0,0 +1,227 @@ +--- +name: memory-safety-analyzer +description: Analyze C/C++ code for memory safety issues including leaks, use-after-free, buffer overflows, and provide fixes. Use when reviewing memory management, debugging crashes, or improving code safety. +--- + +# Memory Safety Analysis for Embedded C + +## Purpose + +Systematically analyze C/C++ code for memory safety issues that can cause crashes, security vulnerabilities, or resource exhaustion in embedded systems. + +## Usage + +Invoke this skill when: +- Reviewing new code with dynamic memory allocation +- Debugging memory-related crashes +- Analyzing legacy code for safety issues +- Preparing code for production deployment +- Investigating memory leaks or fragmentation + +## Analysis Process + +### Step 1: Identify All Allocations + +Search the code for: +- `malloc`, `calloc`, `realloc` +- `strdup`, `strndup` +- `fopen`, `open` +- `pthread_create`, `pthread_mutex_init` +- Custom allocation functions + +For each allocation, verify: +1. Return value is checked +2. Corresponding free/close exists +3. Error paths also free resources +4. No double-free possible + +### Step 2: Check Pointer Lifetimes + +For each pointer variable: +- When is it assigned? +- When is it freed? +- Can it be used after free? +- Can it outlive the data it points to? +- Is it NULL-initialized? +- Is it NULL-checked before use? + +### Step 3: Analyze Error Paths + +For each error return: +- Are all resources freed? +- Is cleanup done in correct order? +- Are error codes accurate? +- Is logging appropriate? + +### Step 4: Review Buffer Operations + +For string and memory operations: +- `strcpy` → should be `strncpy` with size check +- `sprintf` → should be `snprintf` with size +- `gets` → never use (remove immediately) +- `strcat` → verify buffer size +- `memcpy` → verify no overlap, validate size + +### Step 5: Static Analysis + +Run tools: +```bash +# Cppcheck +cppcheck --enable=all --inconclusive file.c + +# Clang static analyzer +scan-build make + +# Compiler warnings +gcc -Wall -Wextra -Werror file.c +``` + +### Step 6: Dynamic Analysis + +Run valgrind: +```bash +valgrind --leak-check=full \ + --show-leak-kinds=all \ + --track-origins=yes \ + --verbose \ + ./program +``` + +## Common Issues and Fixes + +### Issue: Unchecked malloc + +```c +// PROBLEM +char* buffer = malloc(size); +strcpy(buffer, input); // Crash if malloc failed + +// FIX +char* buffer = malloc(size); +if (!buffer) { + log_error("Failed to allocate %zu bytes", size); + return ERR_NO_MEMORY; +} +strncpy(buffer, input, size - 1); +buffer[size - 1] = '\0'; +``` + +### Issue: Memory leak on error + +```c +// PROBLEM +int process() { + char* buf = malloc(1024); + FILE* f = fopen("file.txt", "r"); + + if (!f) return -1; // Leaked buf + + // ... process ... + + free(buf); + fclose(f); + return 0; +} + +// FIX: Single exit with cleanup +int process() { + int ret = 0; + char* buf = NULL; + FILE* f = NULL; + + buf = malloc(1024); + if (!buf) { + ret = ERR_NO_MEMORY; + goto cleanup; + } + + f = fopen("file.txt", "r"); + if (!f) { + ret = ERR_FILE_OPEN; + goto cleanup; + } + + // ... process ... + +cleanup: + free(buf); + if (f) fclose(f); + return ret; +} +``` + +### Issue: Use after free + +```c +// PROBLEM +free(ptr); +if (ptr->field > 0) { ... } // Use after free! + +// FIX +int value = ptr->field; +free(ptr); +ptr = NULL; +if (value > 0) { ... } +``` + +### Issue: Double free + +```c +// PROBLEM +free(ptr); +// ... later ... +free(ptr); // Double free! + +// FIX: NULL after free +free(ptr); +ptr = NULL; +// ... later ... +free(ptr); // Safe: free(NULL) is a no-op +``` + +### Issue: Buffer overflow + +```c +// PROBLEM +char buffer[100]; +strcpy(buffer, user_input); // Overflow if input > 99 chars + +// FIX +char buffer[100]; +strncpy(buffer, user_input, sizeof(buffer) - 1); +buffer[sizeof(buffer) - 1] = '\0'; +``` + +## Output Format + +Provide findings as: + +``` +## Memory Safety Analysis + +### Critical Issues (must fix) +1. [file.c:123] Unchecked malloc - potential NULL dereference +2. [file.c:456] Memory leak on error path - buffer not freed +3. [file.c:789] Use after free - ptr used after free() + +### Warnings (should fix) +1. [file.c:234] strcpy used - prefer strncpy +2. [file.c:567] Missing NULL check before pointer use + +### Recommendations +1. Add cleanup label for resource management +2. Use RAII wrapper in tests +3. Run valgrind in CI pipeline + +### Suggested Fixes +[Provide specific code changes for each issue] +``` + +## Verification + +After fixes: +1. All static analysis warnings resolved +2. Valgrind shows no leaks +3. All tests pass +4. Code review by human +5. Memory footprint measured and acceptable diff --git a/.github/skills/platform-portability-checker/SKILL.md b/.github/skills/platform-portability-checker/SKILL.md new file mode 100644 index 000000000..354fce3cc --- /dev/null +++ b/.github/skills/platform-portability-checker/SKILL.md @@ -0,0 +1,318 @@ +--- +name: platform-portability-checker +description: Verify C/C++ code is platform-independent and portable across embedded platforms. Use when reviewing code for cross-platform deployment or preparing for new hardware targets. +--- + +# Platform Portability Checker + +## Purpose + +Ensure C/C++ code is portable across different embedded platforms, architectures, and operating systems without modification. + +## When to Use + +- Reviewing new code before merge +- Porting to new hardware platform +- Preparing release for multiple architectures +- Investigating platform-specific bugs +- Refactoring legacy platform-specific code + +## Portability Checklist + +### 1. Integer Types + +**Check for**: Use of `int`, `long`, `short` without fixed sizes + +```c +// PROBLEM: Size varies by platform +int counter; // 16, 32, or 64 bits? +long timestamp; // 32 or 64 bits? +short flag; // 16 bits on most, but not guaranteed + +// FIX: Use stdint.h types +#include + +uint32_t counter; // Always 32 bits +uint64_t timestamp; // Always 64 bits +uint16_t flag; // Always 16 bits + +// For size_t operations +size_t length; // Pointer-sized unsigned +ssize_t result; // Pointer-sized signed +``` + +### 2. Pointer Assumptions + +**Check for**: Pointer arithmetic, casting, size assumptions + +```c +// PROBLEM: Assumes pointer == long +long ptr_value = (long)ptr; // Fails on 64-bit with 32-bit long + +// FIX: Use uintptr_t +#include +uintptr_t ptr_value = (uintptr_t)ptr; + +// PROBLEM: Pointer used as integer +if (ptr & 0x1) { ... } // What size is ptr? + +// FIX: Be explicit +if ((uintptr_t)ptr & 0x1) { ... } +``` + +### 3. Endianness + +**Check for**: Multi-byte values sent over network or stored to disk + +```c +// PROBLEM: Host byte order assumed +uint32_t value = 0x12345678; +fwrite(&value, 4, 1, file); // Different on LE vs BE + +// FIX: Explicit byte order +#include // For htonl, ntohl + +uint32_t host_value = 0x12345678; +uint32_t network_value = htonl(host_value); +fwrite(&network_value, 4, 1, file); + +// For reading +uint32_t network_value; +fread(&network_value, 4, 1, file); +uint32_t host_value = ntohl(network_value); +``` + +### 4. Structure Packing + +**Check for**: Structures sent over network or saved to disk + +```c +// PROBLEM: Padding varies by platform +struct { + uint8_t type; + uint32_t value; // Padding before this? + uint16_t flags; // Padding before this? +} data; + +// FIX: Explicit packing +struct __attribute__((packed)) { + uint8_t type; + uint32_t value; + uint16_t flags; +} data; + +// Or control padding explicitly +struct { + uint8_t type; + uint8_t padding[3]; // Explicit padding + uint32_t value; + uint16_t flags; + uint16_t padding2; +} data; +``` + +### 5. Boolean Type + +**Check for**: Using int/char for boolean + +```c +// PROBLEM: Non-standard boolean +int flag; // Really 3 states: 0, 1, other +char enabled; // Also used for booleans + +// FIX: Use stdbool.h +#include + +bool flag; +bool enabled; + +if (flag) { ... } // Clear intent +``` + +### 6. Character Sets + +**Check for**: Assumptions about ASCII or character encoding + +```c +// PROBLEM: Assumes ASCII +if (ch >= 'A' && ch <= 'Z') { + ch = ch + 32; // Convert to lowercase? +} + +// FIX: Use standard functions +#include + +if (isupper(ch)) { + ch = tolower(ch); +} +``` + +### 7. File Paths + +**Check for**: Hard-coded path separators + +```c +// PROBLEM: Unix-specific +const char* path = "/tmp/telemetry/data.log"; + +// FIX: Use platform-agnostic approach +#ifdef _WIN32 + #define PATH_SEP "\\" + const char* tmp_dir = getenv("TEMP"); +#else + #define PATH_SEP "/" + const char* tmp_dir = "/tmp"; +#endif + +char path[256]; +snprintf(path, sizeof(path), "%s%stelemetry%sdata.log", + tmp_dir, PATH_SEP, PATH_SEP); +``` + +### 8. System Calls + +**Check for**: Platform-specific syscalls + +```c +// PROBLEM: Linux-specific +#include +int fd = epoll_create(10); + +// FIX: Abstraction layer +// In platform.h +#if defined(__linux__) + #include "platform_linux.h" +#elif defined(__APPLE__) + #include "platform_darwin.h" +#else + #error "Unsupported platform" +#endif + +// Each platform provides same interface +event_loop_t* create_event_loop(void); +``` + +### 9. Compiler Extensions + +**Check for**: GCC/Clang specific features + +```c +// PROBLEM: GCC-specific +typeof(x) y = x; +int array[0]; // Zero-length array + +// FIX: Use C11 standard features +__auto_type y = x; // C11 + +// Or avoid non-standard features +// Define proper types instead +``` + +### 10. Include Paths + +**Check for**: Platform-specific headers + +```c +// PROBLEM: Assumes Linux headers +#include + +// FIX: Use standard headers or configure check +#ifdef HAVE_LINUX_LIMITS_H + #include +#else + #include +#endif + +// Or use autoconf to detect +// In configure.ac: +// AC_CHECK_HEADERS([linux/limits.h limits.h]) +``` + +## Build System Integration + +### configure.ac checks + +```autoconf +# Check for required features +AC_C_BIGENDIAN +AC_CHECK_SIZEOF([int]) +AC_CHECK_SIZEOF([long]) +AC_CHECK_SIZEOF([void *]) + +# Check for headers +AC_CHECK_HEADERS([stdint.h stdbool.h endian.h]) + +# Check for functions +AC_CHECK_FUNCS([htonl ntohl]) + +# Platform-specific code +case "$host" in + *-linux*) + AC_DEFINE([PLATFORM_LINUX], [1]) + ;; + arm*|*-arm*) + AC_DEFINE([PLATFORM_ARM], [1]) + ;; +esac +``` + +## Testing + +### Cross-Compilation Test + +```bash +# Test building for different architectures +./configure --host=arm-linux-gnueabihf +make clean && make + +./configure --host=x86_64-linux-gnu +make clean && make + +./configure --host=mips-linux-gnu +make clean && make +``` + +### Endianness Test + +```c +// Test endianness handling +uint32_t value = 0x12345678; +uint32_t network = htonl(value); +uint32_t restored = ntohl(network); +assert(value == restored); + +// Verify structure packing +assert(sizeof(packed_struct_t) == EXPECTED_SIZE); +``` + +## Output Format + +``` +## Platform Portability Analysis + +### Critical Issues +1. [file.c:123] Using `long` for timestamp - not fixed width +2. [file.c:456] Writing struct directly to network - endianness issue +3. [file.c:789] Assuming 32-bit pointers + +### Warnings +1. [file.c:234] Using int for boolean - prefer stdbool.h +2. [file.c:567] Hard-coded Unix path separator + +### Recommendations +1. Add configure checks for required headers +2. Create platform abstraction layer +3. Test build on multiple architectures + +### Suggested Fixes +[Specific code changes for each issue] +``` + +## Verification + +- Code compiles on target platforms +- Tests pass on all platforms +- Static analysis clean +- No endianness issues +- No alignment issues +- Structure sizes verified diff --git a/.github/skills/quality-checker/README.md b/.github/skills/quality-checker/README.md new file mode 100644 index 000000000..434f15612 --- /dev/null +++ b/.github/skills/quality-checker/README.md @@ -0,0 +1,72 @@ +# Quality Checker Skill + +Run comprehensive quality checks in the standard test container through chat interface. + +## Quick Start + +Simply ask Copilot to run quality checks in natural language: + +```text +Run quality checks +``` + +```text +Check memory safety +``` + +```text +Run static analysis on src/hostif/profiles +``` + +## What Gets Checked + +1. **Static Analysis**: cppcheck + shellcheck +2. **Memory Safety**: valgrind leak detection +3. **Thread Safety**: helgrind race/deadlock detection +4. **Build Verification**: strict warnings compilation + +## Environment + +Runs in the same container as CI/CD: + +- Image: `ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest` +- All tools pre-installed +- Consistent with automated tests + +## Example Invocations + +| What to say | What it does | +| ----------- | ------------ | +| "Run quality checks" | Full suite, summary report | +| "Quick static analysis" | cppcheck + shellcheck only | +| "Check for memory leaks" | valgrind on test binaries | +| "Verify build with strict warnings" | Build with -Werror | +| "Run all checks on source/utils" | Full suite, scoped to utils | + +## Typical Workflow + +1. **Before committing**: "Run static analysis" +2. **Before push**: "Run quality checks" +3. **Debugging crash**: "Check memory safety" +4. **Reviewing PR**: "Run all checks" + +## Output + +You'll receive: + +- Summary of issues found +- Critical problems highlighted +- Links to detailed reports +- Recommendations for fixes + +## Prerequisites + +- Docker installed and running +- Access to GitHub Container Registry (automatic in CI/CD, may need login locally) + +## Tips + +- Start with static analysis (fastest) +- Run memory checks after static analysis passes +- Scope checks to changed files for speed +- Full suite before pushing to develop branch diff --git a/.github/skills/quality-checker/SKILL.md b/.github/skills/quality-checker/SKILL.md new file mode 100644 index 000000000..cba24a8af --- /dev/null +++ b/.github/skills/quality-checker/SKILL.md @@ -0,0 +1,325 @@ +--- +name: quality-checker +description: Run comprehensive quality checks (static analysis, memory safety, thread safety, build verification) in the standard test container. Use when validating code changes or debugging before committing. +--- + +# Container-Based Quality Checker + +## Purpose + +Execute comprehensive quality checks on the codebase using the same containerized environment as CI/CD pipelines. Ensures consistency between local development and automated testing. + +## Usage + +Invoke this skill when: +- Validating changes before committing +- Debugging build or test failures +- Running quality checks locally +- Verifying memory safety of new code +- Checking for thread safety issues +- Performing static analysis + +You can run all checks or select specific ones based on your needs. + +## What It Does + +This skill runs quality checks inside the official test container (`ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest`), which includes: +- Build tools (gcc, g++, autotools, make) +- Static analysis tools (cppcheck, shellcheck) +- Memory analysis tools (valgrind) +- Thread analysis tools (helgrind) +- Google Test/Mock frameworks + +## Available Checks + +### 1. Static Analysis +- **cppcheck**: Comprehensive C/C++ static code analyzer +- **shellcheck**: Shell script linter +- **Output**: XML report with findings + +### 2. Memory Safety (Valgrind) +- **Memory leak detection**: Finds unreleased allocations +- **Use-after-free detection**: Catches dangling pointer usage +- **Invalid memory access**: Buffer overflows, uninitialized reads +- **Output**: XML and log files per test binary + +### 3. Thread Safety (Helgrind) +- **Race condition detection**: Finds unsynchronized shared memory access +- **Deadlock detection**: Identifies lock ordering issues +- **Lock usage verification**: Validates proper synchronization +- **Output**: XML and log files per test binary + +### 4. Build Verification +- **Strict compilation**: Builds with `-Wall -Wextra -Werror` +- **Test build**: Verifies tests compile +- **Binary analysis**: Reports size and dependencies +- **Output**: Build artifacts and size report + +## Execution Process + +### Step 1: Setup Container Environment + +Pull the latest test container: +```bash +docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest +``` + +Start container with workspace mounted: +```bash +docker run -d --name native-platform \ + -v /path/to/workspace:/mnt/workspace \ + ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest +``` + +### Step 2: Run Selected Checks + +Execute the requested quality checks inside the container: + +**Static Analysis:** +```bash +docker exec -i native-platform /bin/bash -c " + cd /mnt/workspace && \ + cppcheck --enable=all \ + --inconclusive \ + --suppress=missingIncludeSystem \ + --suppress=unmatchedSuppression \ + --error-exitcode=0 \ + --xml \ + --xml-version=2 \ + src/ 2> cppcheck-report.xml +" +``` + +**Shell Script Checks:** +```bash +docker exec -i native-platform /bin/bash -c " + cd /mnt/workspace && \ + find . -name '*.sh' -type f -exec shellcheck {} + +" +``` + +**Memory Safety:** +```bash +docker exec -i native-platform /bin/bash -c " + cd /mnt/workspace && \ + autoreconf -fi && \ + ./configure --enable-gtest && \ + make -j\$(nproc) && \ + find src/unittest src/hostif/src/gtest src/hostif/parodusClient/gtest -type f -executable -name '*test*' 2>/dev/null | while read test_bin; do + valgrind --leak-check=full \ + --show-leak-kinds=all \ + --track-origins=yes \ + --xml=yes \ + --xml-file=\"valgrind-\$(basename \$test_bin).xml\" \ + \"\$test_bin\" 2>&1 | tee \"valgrind-\$(basename \$test_bin).log\" + done +" +``` + +**Thread Safety:** +```bash +docker exec -i native-platform /bin/bash -c " + cd /mnt/workspace && \ + find src/unittest src/hostif/src/gtest src/hostif/parodusClient/gtest -type f -executable -name '*test*' 2>/dev/null | while read test_bin; do + valgrind --tool=helgrind \ + --track-lockorders=yes \ + --xml=yes \ + --xml-file=\"helgrind-\$(basename \$test_bin).xml\" \ + \"\$test_bin\" 2>&1 | tee \"helgrind-\$(basename \$test_bin).log\" + done +" +``` + +**Build Verification:** +```bash +docker exec -i native-platform /bin/bash -c " + cd /mnt/workspace && \ + autoreconf -fi && \ + ./configure --enable-gtest CFLAGS='-Wall -Wextra -Werror' CXXFLAGS='-Wall -Wextra -Werror' && \\ + make -j\$(nproc) && \\ + if [ -f 'tr69hostif' ]; then + ls -lh tr69hostif + file tr69hostif + size tr69hostif + fi +" +``` + +### Step 3: Report Results + +Parse and summarize results for the user: +- Number of issues found by category +- Critical issues requiring immediate attention +- Warnings that should be addressed +- Memory leaks with stack traces +- Race conditions or deadlock risks +- Build errors or warnings + +### Step 4: Cleanup + +Stop and remove the container: +```bash +docker stop native-platform +docker rm native-platform +``` + +## Interpreting Results + +### Static Analysis (cppcheck) +- **error**: Critical issues that must be fixed +- **warning**: Potential problems to review +- **style**: Code style improvements +- **performance**: Optimization opportunities + +### Memory Safety (Valgrind) +- **definitely lost**: Memory leaks requiring fixes +- **indirectly lost**: Leaks from lost parent structures +- **possibly lost**: Potential leaks to investigate +- **still reachable**: Memory held at exit (usually OK) +- **Invalid read/write**: Buffer overflow (CRITICAL) +- **Use of uninitialized value**: Must initialize before use + +### Thread Safety (Helgrind) +- **Possible data race**: Unsynchronized access to shared data +- **Lock order violation**: Potential deadlock scenario +- **Unlocking unlocked lock**: Synchronization bug +- **Thread still holds locks**: Resource leak + +### Build Verification +- **Compilation errors**: Must fix before proceeding +- **Warnings**: Review and fix (builds with -Werror) +- **Binary size**: Monitor for embedded constraints + +## User Interaction + +When invoked, ask the user: + +1. **Which checks to run?** + - All checks (comprehensive) + - Static analysis only (fast) + - Memory safety only + - Thread safety only + - Build verification only + - Custom combination + +2. **Scope:** + - Full codebase + - Specific directories + - Recently changed files + +3. **Report detail:** + - Summary only (counts and critical issues) + - Detailed (all findings) + - Full raw output + +## Example Invocations + +**User**: "Run quality checks" +- Default: Run all checks on full codebase, provide summary + +**User**: "Check memory safety" +- Run only valgrind checks, detailed report + +**User**: "Quick static analysis" +- Run cppcheck and shellcheck, summary only + +**User**: "Verify my changes build" +- Run build verification with strict warnings + +**User**: "Full analysis on src/hostif/profiles" +- Run all checks scoped to profiles directory + +## Best Practices + +1. **Run before committing**: Catch issues early +2. **Start with static analysis**: Fastest feedback +3. **Run memory checks on test binaries**: Most effective +4. **Review thread safety for concurrent code**: Essential for multi-threaded components +5. **Monitor binary size**: Important for embedded targets + +## Integration with Development Workflow + +1. **Pre-commit**: Quick static analysis +2. **Pre-push**: Full quality check suite +3. **Debugging**: Targeted memory/thread analysis +4. **Code review**: Validate reviewer feedback +5. **Refactoring**: Ensure no regressions + +## Advantages Over Manual Testing + +- **Consistency**: Same environment as CI/CD +- **Completeness**: All tools in one command +- **Reproducibility**: Container ensures identical results +- **Efficiency**: No local tool installation needed +- **Confidence**: Pass locally = pass in CI + +## Output Files Generated + +- `cppcheck-report.xml`: Static analysis findings +- `valgrind-.xml`: Memory issues per test +- `valgrind-.log`: Detailed memory logs +- `helgrind-.xml`: Thread safety issues per test +- `helgrind-.log`: Detailed concurrency logs + +These files can be uploaded as artifacts or reviewed locally. + +## Limitations + +- Requires Docker with GitHub Container Registry access +- Container pulls can be slow on first run (cached afterward) +- Full suite can take several minutes depending on codebase size +- Valgrind slows execution significantly (expected) + +## Tips for Faster Execution + +1. Use cached container images (don't pull every time) +2. Run static analysis first (fastest) +3. Scope checks to changed directories +4. Run memory/thread checks only on affected tests +5. Use parallel execution where possible + +## Skill Execution Logic + +When user invokes this skill: + +1. **Authenticate with GitHub Container Registry** + - Use github.actor and GITHUB_TOKEN if available + - Otherwise prompt for credentials or skip private registries + +2. **Pull container image** + - Check if image exists locally + - Pull only if needed or if --force specified + +3. **Start container** + - Mount workspace at /mnt/workspace + - Use unique container name (quality-checker-) + - Run in detached mode + +4. **Execute requested checks** + - Run checks in sequence + - Capture output + - Continue on errors (collect all findings) + +5. **Collect results** + - Copy result files from container + - Parse XML/log outputs + - Categorize findings + +6. **Report to user** + - Summary count + - Critical issues highlighted + - Link to detailed reports + - Next steps recommendations + +7. **Cleanup** + - Stop container + - Remove container + - Optional: clean up result files + +## Error Handling + +- **Container pull fails**: Report error, suggest manual pull +- **Container start fails**: Check Docker daemon, ports, permissions +- **Build fails**: Report build errors, stop further checks +- **Tools missing**: Verify container version, report missing tools +- **Out of memory**: Suggest increasing Docker memory limit diff --git a/.github/skills/technical-documentation-writer/SKILL.md b/.github/skills/technical-documentation-writer/SKILL.md new file mode 100644 index 000000000..b66ff3afd --- /dev/null +++ b/.github/skills/technical-documentation-writer/SKILL.md @@ -0,0 +1,714 @@ +--- +name: technical-documentation-writer +description: Create and maintain comprehensive technical documentation for embedded systems projects. Use for architecture docs, API references, developer guides, and system documentation following best practices. +--- + +# Technical Documentation Writer for Embedded Systems + +## Purpose + +Create clear, comprehensive, and maintainable technical documentation for embedded C/C++ projects, with focus on architecture, APIs, threading models, memory management, and platform integration. + +## Usage + +Invoke this skill when: +- Documenting new features or components +- Creating system architecture documentation +- Writing API reference documentation +- Documenting threading and synchronization models +- Creating developer onboarding guides +- Documenting debugging procedures +- Writing integration guides for platform vendors + +## Documentation Structure + +### Directory Layout + +``` +project/ +├── README.md # Project overview, quick start +├── docs/ # General documentation +│ ├── README.md # Documentation index +│ ├── architecture/ # System architecture +│ │ ├── overview.md # High-level architecture +│ │ ├── component-diagram.md # Component relationships +│ │ ├── threading-model.md # Threading architecture +│ │ └── data-flow.md # Data flow diagrams +│ ├── api/ # API documentation +│ │ ├── public-api.md # Public API reference +│ │ └── internal-api.md # Internal API reference +│ ├── integration/ # Integration guides +│ │ ├── build-setup.md # Build environment setup +│ │ ├── platform-porting.md # Porting to new platforms +│ │ └── testing.md # Test procedures +│ └── troubleshooting/ # Debug guides +│ ├── memory-issues.md # Memory debugging +│ ├── threading-issues.md # Thread debugging +│ └── common-errors.md # Common error solutions +└── source/ # Source code + └── docs/ # Component-specific docs + ├── bulkdata/ # Mirrors source structure + │ ├── README.md # Component overview + │ └── profile-management.md + ├── protocol/ + │ ├── README.md + │ └── http-architecture.md + └── scheduler/ + ├── README.md + └── scheduling-algorithm.md +``` + +### Document Types + +#### 1. **Architecture Documentation** (`docs/architecture/`) +- System overview and design principles +- Component relationships and dependencies +- Threading and concurrency models +- Data flow and state machines +- Memory management strategies +- Platform abstraction layers + +#### 2. **API Documentation** (`docs/api/`) +- Public API reference with examples +- Internal API documentation +- Function contracts and preconditions +- Thread-safety guarantees +- Memory ownership semantics +- Error handling conventions + +#### 3. **Component Documentation** (`source/docs/`) +- Per-component technical details +- Algorithm explanations +- Implementation notes +- Performance characteristics +- Resource usage (memory, CPU, threads) +- Dependencies and interfaces + +#### 4. **Integration Guides** (`docs/integration/`) +- Build system setup +- Platform porting guides +- Configuration options +- Testing procedures +- Deployment checklists + +#### 5. **Troubleshooting Guides** (`docs/troubleshooting/`) +- Common error scenarios +- Debug techniques +- Log analysis +- Memory profiling +- Thread race detection + +## Documentation Process + +### Step 1: Analyze the Code + +Before writing documentation: + +1. **Read the source code** - Understand implementation +2. **Identify key abstractions** - Classes, structs, modules +3. **Map dependencies** - What calls what, data flow +4. **Find synchronization** - Mutexes, conditions, atomics +5. **Trace resource lifecycle** - Allocations, ownership, cleanup +6. **Review existing docs** - Check for patterns and style + +### Step 2: Create Structure + +For each component: + +```markdown +# Component Name + +## Overview +Brief 2-3 sentence description of purpose and role. + +## Architecture +High-level design with diagrams. + +## Key Components +List main structures, functions, modules. + +## Threading Model +How threads interact, synchronization primitives. + +## Memory Management +Allocation patterns, ownership, lifecycle. + +## API Reference +Public functions with signatures and examples. + +## Usage Examples +Common use cases with code snippets. + +## Error Handling +Error codes, failure modes, recovery. + +## Performance Considerations +Resource usage, bottlenecks, optimization tips. + +## Platform Notes +Platform-specific behavior or requirements. + +## Testing +How to test, test coverage, known issues. + +## See Also +Cross-references to related documentation. +``` + +### Step 3: Add Diagrams + +Use Mermaid for visual documentation: + +#### Component Diagram +```mermaid +graph TB + A[Client] --> B[Connection Pool] + B --> C[CURL Handle 1] + B --> D[CURL Handle 2] + B --> E[CURL Handle N] + C --> F[libcurl] + D --> F + E --> F + F --> G[HTTP Server] +``` + +#### Sequence Diagram +```mermaid +sequenceDiagram + participant Client + participant Pool + participant CURL + participant Server + + Client->>Pool: Request handle + Pool->>Pool: Lock mutex + Pool-->>Client: Return handle + Client->>CURL: Configure request + Client->>CURL: Execute + CURL->>Server: HTTP Request + Server-->>CURL: Response + CURL-->>Client: Result + Client->>Pool: Release handle + Pool->>Pool: Signal condition +``` + +#### State Diagram +```mermaid +stateDiagram-v2 + [*] --> Uninitialized + Uninitialized --> Initialized: init() + Initialized --> Running: start() + Running --> Paused: pause() + Paused --> Running: resume() + Running --> Stopped: stop() + Stopped --> [*] +``` + +#### Data Flow Diagram +```mermaid +flowchart LR + A[Marker Event] --> B{Event Type} + B -->|Component| C[Component Marker] + B -->|Event| D[Event Marker] + C --> E[Profile Matcher] + D --> E + E --> F[Report Generator] + F --> G[HTTP Sender] +``` + +### Step 4: Add Code Examples + +Provide clear, compilable examples: + +#### Good Example Structure +```markdown +### Example: Creating a Profile + +This example shows how to create and configure a telemetry profile. + +**Prerequisites:** +- Telemetry system initialized +- Valid configuration file + +**Code:** +```c +#include "profile.h" +#include + +int main(void) { + profile_t* profile = NULL; + int ret = 0; + + // Create profile with name and interval + ret = profile_create("MyProfile", 60, &profile); + if (ret != 0) { + fprintf(stderr, "Failed to create profile: %d\n", ret); + return -1; + } + + // Add marker to profile + ret = profile_add_marker(profile, "Component.Status", + MARKER_TYPE_COMPONENT); + if (ret != 0) { + fprintf(stderr, "Failed to add marker: %d\n", ret); + profile_destroy(profile); + return -1; + } + + // Activate profile + ret = profile_activate(profile); + if (ret != 0) { + fprintf(stderr, "Failed to activate profile: %d\n", ret); + profile_destroy(profile); + return -1; + } + + printf("Profile created and activated successfully\n"); + + // Cleanup + profile_destroy(profile); + return 0; +} +``` + +**Expected Output:** +``` +Profile created and activated successfully +``` + +**Notes:** +- Always check return values +- Call profile_destroy() even on error paths +- Profile name must be unique +``` +\`\`\` + +### Step 5: Document APIs + +For each public function: + +```markdown +### profile_create() + +Creates a new telemetry profile. + +**Signature:** +```c +int profile_create(const char* name, + unsigned int interval_sec, + profile_t** out_profile); +``` + +**Parameters:** +- `name` - Unique profile name (max 63 chars, non-NULL) +- `interval_sec` - Reporting interval in seconds (min: 60, max: 86400) +- `out_profile` - Output pointer to created profile (must be non-NULL) + +**Returns:** +- `0` - Success +- `-EINVAL` - Invalid parameter (NULL name/out_profile, invalid interval) +- `-ENOMEM` - Memory allocation failed +- `-EEXIST` - Profile with same name already exists + +**Thread Safety:** +Thread-safe. Uses internal mutex for profile list management. + +**Memory:** +Allocates memory for profile structure and name copy. Caller must call +`profile_destroy()` to free resources. + +**Example:** +See [Example: Creating a Profile](#example-creating-a-profile) + +**See Also:** +- profile_destroy() +- profile_activate() +- profile_add_marker() +``` + +### Step 6: Document Threading + +For multi-threaded components: + +```markdown +## Threading Model + +### Thread Overview + +| Thread Name | Purpose | Priority | Stack Size | +|------------|---------|----------|------------| +| Main | Initialization, message loop | Normal | Default | +| XConf Fetch | Configuration retrieval | Low | 64KB | +| Report Send | HTTP report transmission | Low | 64KB | +| Event Receiver | Marker event processing | High | 32KB | + +### Synchronization Primitives + +```c +// Global mutexes +static pthread_mutex_t pool_mutex = PTHREAD_MUTEX_INITIALIZER; +static pthread_mutex_t profile_mutex = PTHREAD_MUTEX_INITIALIZER; + +// Condition variables +static pthread_cond_t pool_cond = PTHREAD_COND_INITIALIZER; +static pthread_cond_t xconf_cond = PTHREAD_COND_INITIALIZER; +``` + +### Lock Ordering + +To prevent deadlocks, always acquire locks in this order: + +1. `profile_mutex` (profile list) +2. `pool_mutex` (connection pool) +3. Individual profile locks + +**Example:** +```c +// CORRECT: Proper lock ordering +pthread_mutex_lock(&profile_mutex); +profile_t* p = find_profile_locked(name); +pthread_mutex_lock(&pool_mutex); +// ... use both resources ... +pthread_mutex_unlock(&pool_mutex); +pthread_mutex_unlock(&profile_mutex); + +// WRONG: Deadlock risk! +pthread_mutex_lock(&pool_mutex); +pthread_mutex_lock(&profile_mutex); // May deadlock! +``` + +### Thread Safety Guarantees + +| Function | Thread Safety | Notes | +|----------|---------------|-------| +| profile_create() | Thread-safe | Uses profile_mutex | +| profile_destroy() | Thread-safe | Uses profile_mutex | +| profile_add_marker() | Not thread-safe | Call before activation only | +| send_report() | Thread-safe | Uses pool_mutex | +``` + +### Step 7: Document Memory Management + +```markdown +## Memory Management + +### Allocation Patterns + +```mermaid +graph TD + A[profile_create] --> B[malloc profile_t] + B --> C[strdup name] + B --> D[malloc markers array] + E[profile_add_marker] --> F[realloc markers] + G[profile_destroy] --> H[free markers] + H --> I[free name] + I --> J[free profile_t] +``` + +### Ownership Rules + +1. **profile_t**: Owned by caller after profile_create() +2. **Marker strings**: Copied; caller retains original ownership +3. **Report data**: Owned by sender; freed after transmission + +### Lifecycle Example + +```c +// Creation phase +profile_t* prof = NULL; +profile_create("test", 60, &prof); // Allocates memory + +// Configuration phase +profile_add_marker(prof, "mark1", TYPE_EVENT); // May realloc +profile_add_marker(prof, "mark2", TYPE_EVENT); // May realloc + +// Active phase - no allocations +profile_activate(prof); + +// Destruction phase +profile_destroy(prof); // Frees all memory +prof = NULL; // Prevent use-after-free +``` + +### Memory Budget + +Typical memory usage per component: + +| Component | Static | Dynamic (per item) | Notes | +|-----------|--------|-------------------|-------| +| Profile | 128 bytes | +32 bytes/marker | Preallocated list | +| Connection Pool | 512 bytes | +256 bytes/handle | Max 5 handles | +| Report Buffer | 0 | 64KB | Temporary, freed after send | + +**Total typical footprint**: ~150KB (5 profiles, 3 connections, 1 report) +``` + +## Best Practices + +### Writing Style + +1. **Be Concise**: Get to the point quickly +2. **Be Specific**: Use exact terms, not vague descriptions +3. **Be Accurate**: Test all code examples +4. **Be Complete**: Don't leave critical details unstated +5. **Be Consistent**: Follow established patterns + +### Code Examples + +- **Always compile-test** examples before documenting +- **Show error handling** - embedded systems need robust code +- **Include cleanup** - demonstrate proper resource management +- **Add context** - explain when/why to use the code +- **Keep focused** - one example, one concept + +### Diagrams + +- **Use Mermaid** for all diagrams (version control friendly) +- **Keep simple** - max 10-12 nodes per diagram +- **Label clearly** - all arrows and nodes need names +- **Show flow** - make direction obvious +- **Add legends** - explain symbols if needed + +### Cross-References + +Link related documentation: + +```markdown +## See Also + +- [Threading Model](../architecture/threading-model.md) - Overall thread architecture +- [Connection Pool API](connection-pool.md) - Pool management functions +- [Error Codes](../api/error-codes.md) - Complete error code reference +- [Build Guide](../integration/build-setup.md) - Compilation instructions +``` + +### Platform-Specific Notes + +Always document platform variations: + +```markdown +## Platform Notes + +### Linux +- Uses pthread for threading +- Requires libcurl 7.65.0+ +- mTLS via OpenSSL 1.1.1+ + +### RDKB Devices +- Integration with RDK logger (rdk_debug.h) +- Uses RBUS for IPC when available +- Memory constraints: limit to 8 profiles max + +### Constraints +- **Memory**: Tested with 64MB minimum +- **CPU**: ARMv7 or better +- **Storage**: 1MB for logs and cache +``` + +## Output Format + +### Component Documentation Template + +```markdown +# [Component Name] + +## Overview + +[2-3 sentence description] + +## Architecture + +[High-level design explanation] + +### Component Diagram +```mermaid +[Component relationship diagram] +``` + +## Key Components + +### [Structure/Type Name] + +[Description] + +```c +typedef struct { + // Fields with comments +} structure_t; +``` + +## Threading Model + +[Thread safety and synchronization] + +## Memory Management + +[Allocation patterns and ownership] + +## API Reference + +### [function_name()] + +[Full API documentation] + +## Usage Examples + +### Example: [Use Case] + +[Complete working example] + +## Error Handling + +[Error codes and recovery] + +## Performance + +[Resource usage and bottlenecks] + +## Testing + +[Test procedures and coverage] + +## See Also + +[Cross-references] +``` + +## Quality Checklist + +Before considering documentation complete: + +- [ ] All public APIs documented with signatures +- [ ] At least one working code example per major function +- [ ] Thread safety explicitly stated +- [ ] Memory ownership clearly documented +- [ ] Error codes and meanings listed +- [ ] Diagrams for complex flows +- [ ] Cross-references to related docs +- [ ] Platform-specific notes included +- [ ] Code examples compile and run +- [ ] Grammar and spelling checked +- [ ] Reviewed by component author + +## Maintenance + +Documentation is code: + +1. **Update with code changes** - docs and code change together +2. **Version documentation** - tag with releases +3. **Review periodically** - ensure accuracy quarterly +4. **Fix broken links** - validate references +5. **Deprecate carefully** - mark old features clearly + +### Deprecation Notice Template + +```markdown +## DEPRECATED: old_function() + +⚠️ **This function is deprecated as of v2.1.0** + +**Reason**: Memory leak risk in error paths + +**Alternative**: Use new_function() instead + +**Migration Example**: +```c +// Old way (deprecated) +old_function(param); + +// New way +new_function(param); +``` + +**Removal**: Scheduled for v3.0.0 (Est. Q2 2026) +``` + +## Tools Integration + +### Generate API Docs from Code + +Use Doxygen-style comments in code: + +```c +/** + * @brief Create a new telemetry profile + * + * Creates and initializes a profile structure. The caller is responsible + * for destroying the profile with profile_destroy() when done. + * + * @param[in] name Unique profile name (max 63 chars) + * @param[in] interval_sec Reporting interval (60-86400 seconds) + * @param[out] out_profile Pointer to receive created profile + * + * @return 0 on success, negative errno on failure + * @retval 0 Success + * @retval -EINVAL Invalid parameter + * @retval -ENOMEM Memory allocation failed + * @retval -EEXIST Profile already exists + * + * @note Thread-safe + * @see profile_destroy(), profile_activate() + * + * @par Example: + * @code + * profile_t* prof = NULL; + * int ret = profile_create("MyProfile", 300, &prof); + * if (ret == 0) { + * // Use profile... + * profile_destroy(prof); + * } + * @endcode + */ +int profile_create(const char* name, + unsigned int interval_sec, + profile_t** out_profile); +``` + +### Diagram Tools + +- **Mermaid Live Editor**: https://mermaid.live +- **VS Code Markdown Preview**: Built-in mermaid support +- **Documentation generators**: Can embed mermaid in output + +## Troubleshooting Common Documentation Issues + +### Issue: Code example doesn't compile + +**Solution**: Always test examples in isolation +```bash +# Extract example to test file +cat > test_example.c << 'EOF' +[paste example code] +EOF + +# Compile with project flags +gcc -Wall -Wextra -I../include test_example.c -o test_example + +# Run to verify +./test_example +``` + +### Issue: Diagram is too complex + +**Solution**: Break into multiple diagrams +- One high-level overview diagram +- Multiple focused detail diagrams +- Link them together in text + +### Issue: Outdated documentation + +**Solution**: Add CI check +```bash +# Check for TODOs in docs +grep -r "TODO\|FIXME\|XXX" docs/ && exit 1 + +# Check for broken links +markdown-link-check docs/**/*.md +``` + +## Examples From This Project + +See existing documentation for reference: +- [CURL Architecture](../../../source/docs/protocol/curl_usage_architecture.md) - Good example of architecture doc with diagrams +- [Memory Safety Skill](../memory-safety-analyzer/SKILL.md) - Example skill documentation +- [Build Instructions](../../../.github/instructions/build-system.instructions.md) - Integration guide example diff --git a/.github/skills/thread-safety-analyzer/SKILL.md b/.github/skills/thread-safety-analyzer/SKILL.md new file mode 100644 index 000000000..9d413f012 --- /dev/null +++ b/.github/skills/thread-safety-analyzer/SKILL.md @@ -0,0 +1,436 @@ +--- +name: thread-safety-analyzer +description: Analyze C/C++ code for thread safety issues including race conditions, deadlocks, and improper synchronization. Use when reviewing concurrent code or debugging threading issues. +--- + +# Thread Safety Analysis for Embedded C + +## Purpose + +Systematically analyze C/C++ code for thread safety issues that can cause race conditions, deadlocks, or performance degradation in embedded systems. + +## Usage + +Invoke this skill when: +- Reviewing multi-threaded code +- Debugging race conditions or deadlocks +- Optimizing synchronization overhead +- Validating thread creation and cleanup +- Investigating lock contention issues + +## Analysis Process + +### Step 1: Identify Shared Data + +Search for global and static variables: +- Global variables (especially non-const) +- Static variables in functions +- Shared heap allocations +- Reference-counted objects + +For each shared variable, verify: +1. How is it protected (mutex, atomic, etc.)? +2. Is the protection consistent across all accesses? +3. Are reads and writes both protected? +4. Is initialization thread-safe? + +### Step 2: Review Thread Creation + +Check all pthread_create calls: +- Are thread attributes used? +- Is stack size specified? +- Are threads detached or joinable? +- Is cleanup properly handled? + +```c +// CHECK FOR: +pthread_t thread; +pthread_create(&thread, NULL, func, arg); // BAD: No attributes + +// SHOULD BE: +pthread_attr_t attr; +pthread_attr_init(&attr); +pthread_attr_setstacksize(&attr, 64 * 1024); // Explicit size +pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED); +pthread_create(&thread, &attr, func, arg); +pthread_attr_destroy(&attr); +``` + +### Step 3: Analyze Lock Usage + +For each mutex/rwlock: +- Is it initialized before use? +- Is it destroyed when done? +- Are lock/unlock pairs balanced? +- What is the lock ordering? +- Are locks held during expensive operations? + +Common patterns to check: +```c +// Pattern 1: Missing unlock on error path +pthread_mutex_lock(&lock); +if (error) return -1; // LEAK! +pthread_mutex_unlock(&lock); + +// Pattern 2: Lock ordering violation +// Thread 1: +pthread_mutex_lock(&a); +pthread_mutex_lock(&b); + +// Thread 2: +pthread_mutex_lock(&b); // Different order! +pthread_mutex_lock(&a); // DEADLOCK RISK! + +// Pattern 3: Heavy lock for simple operation +pthread_rwlock_wrlock(&lock); // Too heavy +counter++; +pthread_rwlock_unlock(&lock); +// Should use atomic_int instead +``` + +### Step 4: Check for Race Conditions + +Look for unprotected accesses to shared data: + +```c +// RACE: Read-modify-write without protection +if (shared_flag == 0) { // Thread 1 reads + shared_flag = 1; // Thread 2 also reads before Thread 1 writes +} + +// FIX: Use atomic or lock +pthread_mutex_lock(&lock); +if (shared_flag == 0) { + shared_flag = 1; +} +pthread_mutex_unlock(&lock); + +// OR: Use atomic compare-and-swap +int expected = 0; +atomic_compare_exchange_strong(&shared_flag, &expected, 1); +``` + +### Step 5: Verify Atomic Usage + +For atomic variables: +- Are they declared with proper type (atomic_int, atomic_bool)? +- Is memory ordering appropriate? +- Are non-atomic operations mixed with atomic ones? + +```c +// CHECK: +atomic_int counter; + +// GOOD: Atomic operations +atomic_fetch_add(&counter, 1); +int value = atomic_load(&counter); + +// BAD: Mixing atomic and non-atomic +counter++; // Non-atomic! Use atomic_fetch_add +``` + +### Step 6: Deadlock Detection + +Check for common deadlock patterns: + +1. **Circular wait**: Lock A → Lock B, Lock B → Lock A +2. **Lock held while waiting**: Mutex held during sleep/wait +3. **Missing timeout**: Indefinite blocking without timeout +4. **Signal under lock**: Condition signal while holding mutex + +```c +// Deadlock Pattern 1: Circular dependency +// Function 1: +lock(mutex_a); +lock(mutex_b); // Order: A, B + +// Function 2: +lock(mutex_b); +lock(mutex_a); // Order: B, A - DEADLOCK! + +// Deadlock Pattern 2: Lock held during expensive operation +lock(mutex); +expensive_network_call(); // Blocks other threads! +unlock(mutex); + +// Deadlock Pattern 3: No timeout +pthread_mutex_lock(&lock); // Waits forever if deadlock +``` + +### Step 7: Check Condition Variables + +For condition variables: +- Is wait always in a loop? +- Is predicate checked before and after wait? +- Is signal/broadcast done correctly? +- Is spurious wakeup handled? + +```c +// GOOD: Proper condition variable usage +pthread_mutex_lock(&mutex); +while (!condition) { // Loop for spurious wakeups + pthread_cond_wait(&cond, &mutex); +} +// ... use protected data ... +pthread_mutex_unlock(&mutex); + +// Signal: +pthread_mutex_lock(&mutex); +condition = true; +pthread_cond_signal(&cond); +pthread_mutex_unlock(&mutex); + +// BAD: Missing loop +pthread_mutex_lock(&mutex); +if (!condition) { // Should be 'while'! + pthread_cond_wait(&cond, &mutex); +} +pthread_mutex_unlock(&mutex); +``` + +## Common Issues and Fixes + +### Issue: Default Thread Stack Size + +```c +// PROBLEM: Wastes memory (8MB per thread) +pthread_t thread; +pthread_create(&thread, NULL, worker, arg); + +// FIX: Specify minimal stack size +pthread_attr_t attr; +pthread_attr_init(&attr); +pthread_attr_setstacksize(&attr, 64 * 1024); // 64KB +pthread_create(&thread, &attr, worker, arg); +pthread_attr_destroy(&attr); +``` + +### Issue: Heavy Synchronization + +```c +// PROBLEM: Reader-writer lock overkill +pthread_rwlock_t lock; +int counter; + +void increment() { + pthread_rwlock_wrlock(&lock); + counter++; + pthread_rwlock_unlock(&lock); +} + +// FIX: Use atomic operations +atomic_int counter; + +void increment() { + atomic_fetch_add(&counter, 1); // No lock needed +} +``` + +### Issue: Lock Ordering Violation + +```c +// PROBLEM: Different lock orders cause deadlock +// Thread 1: +void process_a_then_b() { + lock(&resource_a.lock); + lock(&resource_b.lock); + // ... +} + +// Thread 2: +void process_b_then_a() { + lock(&resource_b.lock); + lock(&resource_a.lock); // DEADLOCK! + // ... +} + +// FIX: Consistent ordering everywhere +void process_a_then_b() { + lock(&resource_a.lock); // Always A first + lock(&resource_b.lock); // Then B + // ... +} + +void process_b_then_a() { + lock(&resource_a.lock); // Always A first + lock(&resource_b.lock); // Then B + // ... +} +``` + +### Issue: Race in Lazy Initialization + +```c +// PROBLEM: Non-thread-safe initialization +static config_t* config = NULL; + +config_t* get_config() { + if (!config) { // Race here! + config = malloc(sizeof(config_t)); + init_config(config); + } + return config; +} + +// FIX: Use pthread_once +static pthread_once_t init_once = PTHREAD_ONCE_INIT; +static config_t* config = NULL; + +static void init_config_once() { + config = malloc(sizeof(config_t)); + init_config(config); +} + +config_t* get_config() { + pthread_once(&init_once, init_config_once); + return config; +} +``` + +### Issue: Missing Lock on Error Path + +```c +// PROBLEM: Lock not released on error +int process_data(data_t* shared) { + pthread_mutex_lock(&shared->lock); + + if (validate(shared) != 0) { + return -1; // BUG: Lock not released! + } + + update(shared); + pthread_mutex_unlock(&shared->lock); + return 0; +} + +// FIX: Unlock on all paths +int process_data(data_t* shared) { + int ret = 0; + + pthread_mutex_lock(&shared->lock); + + if (validate(shared) != 0) { + ret = -1; + goto cleanup; + } + + update(shared); + +cleanup: + pthread_mutex_unlock(&shared->lock); + return ret; +} +``` + +### Issue: Long Critical Section + +```c +// PROBLEM: Expensive operation under lock +pthread_mutex_lock(&lock); +for (int i = 0; i < 1000000; i++) { + compute(); // Blocks other threads! +} +shared_result = final_value; +pthread_mutex_unlock(&lock); + +// FIX: Minimize critical section +int result = 0; +for (int i = 0; i < 1000000; i++) { + result += compute(); // No lock +} + +pthread_mutex_lock(&lock); +shared_result = result; // Lock only for update +pthread_mutex_unlock(&lock); +``` + +## Testing for Thread Safety + +### Compile with Thread Sanitizer + +```bash +# Build with thread sanitizer +gcc -g -fsanitize=thread -O1 source.c -o program -lpthread + +# Run +./program + +# Will report: +# - Data races +# - Lock ordering issues +# - Potential deadlocks +``` + +### Run Helgrind + +```bash +# Check for thread safety issues +valgrind --tool=helgrind \ + --track-lockorders=yes \ + ./program + +# Reports: +# - Race conditions +# - Lock order violations +# - Possible deadlocks +``` + +### Stress Testing + +```c +// Test under high concurrency +#define NUM_THREADS 100 +#define ITERATIONS 10000 + +void stress_test() { + pthread_t threads[NUM_THREADS]; + + for (int i = 0; i < NUM_THREADS; i++) { + pthread_create(&threads[i], NULL, worker, NULL); + } + + for (int i = 0; i < NUM_THREADS; i++) { + pthread_join(threads[i], NULL); + } + + // Verify invariants + assert(shared_counter == NUM_THREADS * ITERATIONS); +} +``` + +## Output Format + +Provide findings as: + +``` +## Thread Safety Analysis + +### Critical Issues (must fix) +1. [file.c:123] Race condition - unprotected access to shared_flag +2. [file.c:456] Deadlock potential - lock order violation (A→B vs B→A) +3. [file.c:789] Lock leak - mutex not released on error path + +### Warnings (should fix) +1. [file.c:234] Default thread stack - wastes 8MB per thread +2. [file.c:567] Heavy lock - use atomic_int instead of mutex +3. [file.c:890] Long critical section - holds lock during I/O + +### Recommendations +1. Establish lock ordering convention (document in header) +2. Use pthread_once for singleton initialization +3. Replace reader-writer locks with atomics for counters +4. Add thread sanitizer to CI pipeline + +### Suggested Fixes +[Provide specific code changes for each issue] +``` + +## Verification + +After fixes: +1. Thread sanitizer shows no errors +2. Helgrind reports clean +3. Stress tests pass consistently +4. Lock contention metrics acceptable +5. No deadlocks under load testing +6. Code review confirms thread safety diff --git a/.github/skills/tr69hostif-issue-triage/SKILL.md b/.github/skills/tr69hostif-issue-triage/SKILL.md new file mode 100644 index 000000000..4be69508c --- /dev/null +++ b/.github/skills/tr69hostif-issue-triage/SKILL.md @@ -0,0 +1,298 @@ +--- +name: tr69hostif-issue-triage +description: > + Triage any tr69hostif behavioral issue on RDK devices by correlating device + log bundles with source code. Covers daemon hangs, TR-069/CWMP RPC failures, + TR-181 parameter get/set errors, WebPA/parodus handler faults, RFC parameter + override issues, CPU/memory spikes, and test gap analysis. The user states the + issue; this skill guides systematic root-cause analysis regardless of issue type. +--- + +# tr69hostif Issue Triage Skill + +## Purpose + +Systematically correlate device log bundles with tr69hostif source code to +identify root causes, characterize impact, and propose unit-test and +functional-test reproduction scenarios — for **any** behavioral anomaly reported +by the user. + +--- + +## Usage + +Invoke this skill when: +- A device log bundle is available under `logs/` (or attached separately) +- The user describes a behavioral anomaly (examples: daemon stuck or crashing, + TR-069 RPC not executing, parameter get/set silently failing, WebPA/parodus + requests timing out, RFC overrides not applying, high CPU, high memory) +- You need to write a reproduction scenario for an existing or proposed fix + +**The user's stated issue drives the investigation.** Do not assume a specific +failure mode — read the issue description first, then follow the steps below. + +--- + +## Step 1: Orient to the Log Bundle + +**Log bundle layout** (typical RDK device): +``` +logs///logs/ + tr69hostIf.log.0 ← Primary tr69hostif daemon log (start here) + PAMlog.txt.0 ← Platform/parameter management + WPEFramework*.txt.0 ← Component framework messages + SelfHeal*.txt.0 ← Watchdog and recovery events + top_log.txt.0 ← CPU/memory snapshots (useful for perf issues) + messages.txt.0 ← Kernel and system messages +``` + +Include any log files surfaced by the user's issue description (e.g., `parodus*.txt.0` +for WebPA connectivity issues, `syslog` for OOM events). + +**Log timestamp prefix format**: `YYMMDD-HH:MM:SS.uuuuuu` +- Session folder names are **local-time snapshots** (format: `MM-DD-YY-HH:MMxM`) +- Log lines use device local time + +**Session ordering**: Sort session folders chronologically. Multiple sessions may +represent reboots. Alphabetical sort does NOT equal chronological order. + +--- + +## Step 2: Map Daemon Startup and Components + +Read the startup section of `tr69hostIf.log.0` (first ~50 lines) to identify: + +| What to find | Log pattern | +|---|---| +| Daemon start | `tr69HostIf starting up` | +| mgrlist loaded | `mgrlist.conf` path and profile count | +| Handler registration | `Registered handler for ` | +| WebPA/parodus connection | `Connected to parodus` or `parodus_connect` | +| RFC defaults loaded | `RFC defaults loaded` | +| IARM bus ready | `IARMBUS_Init` success | + +**Key threads in tr69hostif**: +- Main thread — initializes handlers, listens for CWMP or parodus requests +- IARM event listener thread — receives platform events (network up/down, etc.) +- Handler worker threads — service individual TR-181 parameter get/set requests + +--- + +## Step 3: Identify the Anomaly Window + +Based on the **user's stated issue**, search for the relevant evidence pattern: + +### Daemon Hang / Stuck +A hang manifests as a **timestamp gap** in `tr69hostIf.log.0` or no response to +get/set requests from the CWMP ACS or WebPA: +``` +grep -n "GetParamValue\|SetParamValue\|RPC\|request" tr69hostIf.log.0 | tail -40 +``` +Gap > expected response time = anomaly. During the gap, check: +- Is the IARM bus thread still logging? (no → IARM bus deadlock or crash) +- Is there a mutex hold log before the gap? (yes → lock contention) + +### TR-181 Parameter GET / SET Failure +Look for handler errors or missing responses: +``` +grep -n "GetParamValue\|SetParamValue\|Error\|Failed\|NULL" tr69hostIf.log.0 +``` +- Identify which parameter path failed (`Device.X_RDKCENTRAL-COM_RFC.Feature.*`, etc.) +- Check if the handler is registered in `mgrlist.conf` +- Check if the backing store (`tr181store.ini`, `rfcdefaults.ini`) has the key + +### RFC Parameter Override Not Applied +Look for RFC processing logs: +``` +grep -n "RFC\|rfc\|override\|feature" tr69hostIf.log.0 +``` +- Confirm `rfcdefaults.ini` and `rfcVariable.ini` are loaded at startup +- Check for handler-specific RFC processing in `src/hostif/handlers/` +- Verify parameter name matches between RFC file and handler registration + +### WebPA / Parodus Request Timeout +Look for parodus connection and request-handling logs: +``` +grep -n "parodus\|webpa\|WEBPA\|connect\|timeout" tr69hostIf.log.0 +``` +- Confirm parodus connected at startup: `Connected to parodus` +- Identify which parameter request timed out (GET/SET/ADD/DELETE) +- Check waldb data-model XML for parameter registration + +### CPU / Memory Spike +Correlate `top_log.txt.0` timestamps with tr69hostif activity: +``` +grep -n "tr69hostif" top_log.txt.0 +``` +- Identify what tr69hostif was doing (bulk GET, data-model scan, IARM callback) at spike time +- Check for large iterative operations over Device.IP or Device.Ethernet tables +- Check for memory growth from uncleaned handler context objects + +### Handler Registration / Module Load Failure +Look for initialization errors: +``` +grep -n "ERROR\|WARN\|Failed\|register\|load" tr69hostIf.log.0 | head -60 +``` +- Identify which handler module failed to load +- Check shared library availability (`ldd /usr/local/bin/tr69hostif`) +- Confirm mgrlist.conf lists the module correctly + +--- + +## Step 4: Correlate with Other Component Logs + +Based on the anomaly window identified in Step 3, cross-reference with other logs: + +| Issue Type | Companion Log | What to Look For | +|---|---|---| +| Daemon hang | `PAMlog.txt.0` | PAM parameter lock contention within hang window | +| Parameter GET/SET fail | `PAMlog.txt.0` | Underlying parameter store errors | +| WebPA timeout | `parodus*.txt.0` | Connection drops or message queue overflow | +| RFC override missing | `PAMlog.txt.0` | RFC feature flag not propagated | +| CPU spike | `top_log.txt.0` | CPU% at anomaly timestamps | +| Memory growth | `messages.txt.0` | OOM killer events | +| Crash / segfault | `messages.txt.0` | Kernel segfault or signal 11 for tr69hostif PID | +| IARM event miss | Any IARM bus log | Event dispatch errors | + +--- + +## Step 5: Locate the Code Path + +Navigate to the relevant source based on the anomaly type. Key modules: + +### Daemon Main (`src/hostif/src/hostIf_main.cpp`) +- Initializes all subsystems: IARM, parodus, handler managers +- Starts the main request loop +- Processes CWMP ACS connections and dispatches RPCs + +### Handler Framework (`src/hostif/handlers/`) +- Per-profile handler implementations (IP, Ethernet, DeviceInfo, Time, wifi, etc.) +- Each handler registers `GetParamValue` / `SetParamValue` callbacks +- Handlers use `IniFile` / `hostIf_utils` to access backing stores + +### WebPA / Parodus Client (`src/hostif/parodusClient/pal/`) +- Bridges parodus/WebPA GET/SET/ADD/DELETE to TR-181 handler calls +- `waldb` data-model XML controls which parameters are registered with parodus +- Connection management and retry logic + +### RFC Parameter Management (`src/hostif/handlers/src/` — rfcapi wrappers) +- Reads `rfcdefaults.ini` at startup for default values +- Reads `rfcVariable.ini` for operator overrides +- `tr181store.ini` / `bootstrap.ini` in `/opt/secure/RFC/` for runtime state + +### Profile Modules (`src/hostif/profiles/`) +- Subdirectories per TR-181 subtree: `Device/`, `DeviceInfo/`, `IP/`, `Ethernet/`, `Time/`, `wifi/`, `moca/`, `STBService/`, `StorageService/` +- Each profile implements data-model object instances and their parameters +- Integer table indices can cause off-by-one issues in bulk GET operations + +### SNMP Adapter (`src/hostif/snmpAdapter/`) +- Translates SNMP OID requests to TR-181 parameter paths +- Uses `tr181_snmpOID.conf` for OID-to-parameter mapping + +--- + +## Step 6: Characterize Root Cause + +Use this matrix to classify the issue based on observed evidence: + +| Observed Pattern | Issue Class | Primary Code Location | +|---|---|---| +| No response to GET/SET, timestamp gap in log | Daemon hang or deadlock | `hostIf_main.cpp`, handler mutex | +| `ERROR` on specific `Device.X.*` parameter | Handler not registered or NULL callback | `handlers/src/`, `mgrlist.conf` | +| RFC feature enabled but behaviour unchanged | RFC parameter path mismatch or wrong store file | `rfcapi` wrappers, `rfcdefaults.ini` | +| Parodus GET returns stale/wrong value | waldb data-model out of sync, handler not updating cache | `parodusClient/pal/`, `waldb/data-model/` | +| Crash (SIGSEGV) on specific parameter | NULL pointer dereference in handler | handler `GetParamValue` / `SetParamValue` | +| High CPU during bulk GET operation | Iterating large object table without bounds | profile handler loop logic | +| Memory growth over uptime | Handler context never freed on module unload | handler `init` / `free` lifecycle | +| SNMP OID returns wrong value | OID mapping incorrect or TR-181 path stale | `tr181_snmpOID.conf`, `snmpAdapter.cpp` | +| Bootstrap parameters not persisted | `bootstrap.ini` write path wrong or permissions | RFC store path configuration | +| Parameter visible via CWMP but not WebPA | waldb data-model XML missing the parameter | `waldb/data-model/data-model-*.xml` | + +--- + +## Step 7: Assess Unit Test Coverage + +**Location**: `src/unittest/`, `src/hostif/src/gtest/`, `src/hostif/parodusClient/gtest/` + +**Identify gaps relevant to the issue**. For each gap, write a test template: + +``` +Test Name: +Setup: +Action: +Assert: +File: src/unittest/ or src/hostif/*/gtest/ +``` + +**Common gap areas** (match to the issue class): +- Handler returns wrong value when backing store key is missing +- RFC override applies correctly when `rfcVariable.ini` has a matching entry +- WebPA SET propagates to the correct handler and persists in `tr181store.ini` +- NULL handler callback registered for a parameter path — graceful error, no crash +- Object table GET with index out of range — returns error, no buffer overflow + +--- + +## Step 8: Assess L2 (Functional) Test Coverage + +**Location**: `test/functional-tests/tests/` + +**Existing test modules**: +- `test_bootup_sequence.py` — daemon startup, handler registration, initial parameter values +- `test_handlers_communications.py` — parameter GET/SET via handler protocol +- `tr69hostif_deviceip.py` — Device.IP subtree parameter reads +- `tr69hostif_webpa.py` — WebPA/parodus GET/SET round-trip + +**Identify the missing scenario** that would catch the reported issue. Write a +Python pytest outline covering: +1. The precondition (daemon running, config files in place, specific parameter value) +2. The triggering action (GET/SET request, RFC reload, IARM event injection) +3. The correct observable outcome (expected parameter value, return code, log message) +4. The failure observable outcome (what the bug produces vs. what is expected) + +```python +def test__(tr69hostif_daemon): + """ + Verify when . + """ + # Arrange + # ...set preconditions... + + # Act + result = tr69hostif_daemon.get_param("Device.X.") + + # Assert + assert result == expected_value +``` + +--- + +## Step 9: Document Findings + +Produce a triage report with: +1. **Issue restatement**: confirm back the user's stated problem in one sentence +2. **Device context**: MAC, firmware version, session timestamp(s) examined +3. **Anomaly timeline**: exact timestamps, relevant thread IDs, duration or frequency +4. **Root cause chain**: numbered steps, each with log evidence + source code reference +5. **Unit test gap**: which test file, test name, and what assertion it needs +6. **L2 test gap**: Python pytest outline +7. **Proposed fix**: minimum-scope change — file, function, and what to change + +--- + +## Common Pitfalls + +- **mgrlist.conf not loaded**: If a handler module is not listed in `/etc/mgrlist.conf`, + its parameters will silently return empty — check mgrlist first for any missing GET/SET +- **waldb data-model mismatch**: Parameters included in one `data-model-*.xml` but absent + in another are invisible to parodus/WebPA — always check all three XML files +- **RFC store path confusion**: `rfcdefaults.ini` is in `/tmp/`, `rfcVariable.ini` in + `/opt/secure/RFC/` — a path mismatch causes overrides to be silently ignored +- **tr181store.ini vs bootstrap.ini precedence**: `bootstrap.ini` values take precedence + over `tr181store.ini`; write to the wrong file and the value appears to not persist +- **IARM bus init order**: If tr69hostif starts before the IARM bus is ready, event + subscriptions may be missed — look for `IARMBUS_Init` failure in the log +- **Index base**: TR-181 table indices start at **1**, not 0 — off-by-one in handler + loops produces wrong data for the last or first instance +- **Thread-safety of handler context**: Some handlers cache state in a global struct + that is not mutex-protected — concurrent GET and SET can corrupt the cache diff --git a/README.md b/README.md new file mode 100644 index 000000000..03d968090 --- /dev/null +++ b/README.md @@ -0,0 +1,480 @@ +# tr69hostif — TR-069 Host Interface Manager + +[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE) +[![Version](https://img.shields.io/badge/version-1.3.6-green.svg)](CHANGELOG.md) + +## Overview + +`tr69hostif` is the **TR-069 Host Interface Manager** for RDK-based devices. It acts as the central broker between remote management infrastructure (ACS, WebPA/Parodus) and the TR-181 data model implemented on the device. Any component that needs to read or write TR-181 parameters — including the CWMP stack, WebPA gateway, RFC override system, and SNMP bridge — routes its requests through `tr69hostif`. + +The daemon runs as a persistent systemd service, initializes all TR-181 profile handlers at startup, and then services get/set requests over multiple IPC channels simultaneously. + +## Documentation + +Implementation-oriented documentation lives under `docs/`. + +- `docs/README.md` provides the documentation index. +- `docs/architecture/overview.md` describes the daemon structure and startup sequence. +- `docs/architecture/threading-model.md` documents worker threads, locks, and shutdown behavior. +- `docs/architecture/data-flow.md` traces request routing and change-notification flow. +- `docs/api/public-api.md` documents the shared request envelope and dispatcher entry points. +- `docs/integration/build-setup.md` and `docs/integration/testing.md` cover build and validation workflows. + +## Architecture + +### High-Level Component Diagram + +```mermaid +graph TB + subgraph Remote["Remote Callers"] + ACS[ACS / CWMP Stack] + WebPA[WebPA / parodus] + SNMP[SNMP Manager] + RBUS[RBUS Clients] + end + + subgraph tr69hostif["tr69hostif Daemon"] + IARM[IARM-Bus IPC Handler] + JSON[JSON Request Handler\nPort 10999] + RBUS_P[RBUS DML Provider] + PAR[Parodus PAL\nlibpd] + UPD[Update Handler\nValue Change Events] + MSG[Message Dispatcher\nhostIf_msgHandler] + + subgraph Profiles["TR-181 Profile Handlers"] + DEV[DeviceInfo] + WIFI[WiFi] + ETH[Ethernet] + IP[IP] + MOCA[MoCA] + TIME[Time] + DHCP[DHCPv4] + STBS[STBService\nDS Profile] + STOR[StorageService] + INTF[InterfaceStack] + SNMPA[SNMP Adapter] + end + + subgraph RFC["RFC / Bootstrap"] + RFC_S[RFC Store\nXRFCStorage] + BS_S[Bootstrap Store\nXBSStore] + end + end + + ACS -->|IARM RPC| IARM + SNMP -->|IARM RPC| IARM + WebPA-->|msgpack/WRP| PAR + RBUS -->|rbus API| RBUS_P + JSON -->|HTTP JSON| MSG + + IARM --> MSG + PAR --> MSG + RBUS_P --> MSG + MSG --> Profiles + MSG --> RFC + UPD -->|ValueChanged| IARM + UPD -->|ValueChanged| PAR +``` + +### Request Flow + +```mermaid +sequenceDiagram + participant Caller as Caller (IARM/RBUS/WebPA) + participant MSG as Message Dispatcher + participant PROF as Profile Handler + participant HAL as Platform HAL / OS + + Caller->>MSG: Get/Set paramName + value + MSG->>MSG: Route by prefix (mgrlist.conf) + MSG->>PROF: handler->handleGetMsg() / handleSetMsg() + PROF->>HAL: Read device state / write config + HAL-->>PROF: Raw value + PROF-->>MSG: Populated HOSTIF_MsgData_t + MSG-->>Caller: Response + faultCode +``` + +### Startup Sequence + +```mermaid +sequenceDiagram + participant main as main() + participant CFG as ConfigManager + participant IARM as IARM-Bus + participant DM as DataModel XML + participant THR as Threads + + main->>CFG: hostIf_initalize_ConfigManger() + main->>IARM: hostIf_IARM_IF_Start() + main->>DM: mergeDataModel() + loadDataModel() + main->>THR: json_if_handler_thread (GLib) + main->>THR: http_server_thread (optional, legacy RFC) + main->>THR: updateHandler::Init() (value-change polling) + main->>THR: libpd_client_mgr() (Parodus, if enabled) + main->>THR: initWebConfigTask() (WebConfig, if enabled) + main->>main: init_rbus_dml_provider() + main->>main: sd_notify(READY=1) + main->>main: g_main_loop_run() +``` + +## Key Components + +### Core Daemon (`src/hostif/src/`) + +| File | Purpose | +|------|---------| +| `hostIf_main.cpp` | `main()` entry point: argument parsing, signal handling, thread lifecycle, GLib main loop | +| `hostIf_utils.cpp` | Utility helpers: type conversion, reset state machine, gateway connectivity | +| `IniFile.cpp` | INI file parser used by RFC and Bootstrap stores | + +### Request Handlers (`src/hostif/handlers/`) + +| Handler | IARM Bus Manager Token | TR-181 Subtree | +|---------|----------------------|----------------| +| `hostIf_DeviceClient_ReqHandler` | `deviceMgr` | `Device.DeviceInfo.*` | +| `hostIf_WiFi_ReqHandler` | `wifiMgr` | `Device.WiFi.*` | +| `hostIf_EthernetClient_ReqHandler` | `ethernetMgr` | `Device.Ethernet.*` | +| `hostIf_IPClient_ReqHandler` | `ipMgr` | `Device.IP.*` | +| `hostIf_MoCAClient_ReqHandler` | `mocaMgr` | `Device.MoCA.*` | +| `hostIf_TimeClient_ReqHandler` | `timeMgr` | `Device.Time.*` | +| `hostIf_DHCPv4Client_ReqHandler` | `dhcpv4Mgr` | `Device.DHCPv4.*` | +| `hostIf_dsClient_ReqHandler` | `dsMgr` | `Device.Services.STBService.*` | +| `hostIf_StorageSrvc_ReqHandler` | `storageSrvcMgr` | `Device.Services.StorageService.*` | +| `hostIf_InterfaceStackClient_ReqHandler` | `intfStackMgr` | `Device.InterfaceStack.*` | +| `hostIf_SNMPClient_ReqHandler` | `snmpAdapterMgr` | `Device.X_RDKCENTRAL-COM.*` (SNMP bridge) | +| `hostIf_rbus_Dml_Provider` | — | Exposes all registered params over RBUS | +| `hostIf_updateHandler` | — | Polls profiles for value changes; publishes IARM events | +| `hostIf_NotificationHandler` | — | Queues value-change notifications to Parodus | + +All handlers inherit from the abstract `msgHandler` base class. The `hostIf_msgHandler.cpp` dispatcher instantiates each handler at startup and routes requests by matching the parameter name prefix against the manager map loaded from `tr69hostIf.conf`. + +### TR-181 Profiles (`src/hostif/profiles/`) + +Each subdirectory implements one or more TR-181 objects. Profiles contain the business logic: they read HAL APIs (IARM Device Settings, wifihal, platform sysfs, etc.) and translate results to/from `HOSTIF_MsgData_t`. + +| Profile Directory | TR-181 Object | Key Dependencies | +|-------------------|---------------|-----------------| +| `DeviceInfo/` | `Device.DeviceInfo` | IARM, rfcapi, rfcdefaults, partners\_defaults.json | +| `wifi/` | `Device.WiFi` | wifihal (libwifi) | +| `Ethernet/` | `Device.Ethernet` | sysfs, IARM | +| `IP/` | `Device.IP` | netlink / sysfs | +| `moca/` | `Device.MoCA` | IARM mocaMgr | +| `Time/` | `Device.Time` | NTP daemon, chrony | +| `DHCPv4/` | `Device.DHCPv4` | udhcpc / dnsmasq | +| `STBService/` | `Device.Services.STBService` | IARM Device Settings (DS) | +| `StorageService/` | `Device.Services.StorageService` | sysfs block devices | +| `InterfaceStack/` | `Device.InterfaceStack` | sysfs | +| `Device/` | `Device.*` (root object) | — | + +### RFC & Bootstrap Subsystem (`src/hostif/profiles/DeviceInfo/`) + +| Class | File | Purpose | +|-------|------|---------| +| `XRFCStorage` | `XrdkCentralComRFC.cpp` | Persists RFC override values in an INI file under `/opt/secure/RFC/` | +| `XBSStore` | `XrdkCentralComBSStore.cpp` | Loads per-partner bootstrap defaults from `partners_defaults.json`; owns the background partner-ID resolution thread | +| `XBSStoreJournal` | `XrdkCentralComBSStoreJournal.cpp` | Append-only journal for bootstrap value changes | + +RFC parameter precedence (highest to lowest): + +``` +RFC Override (/opt/secure/RFC/) > WebPA Set > Bootstrap Default > Firmware Default +``` + +### Parodus / WebPA Client (`src/hostif/parodusClient/pal/`) + +| File | Purpose | +|------|---------| +| `libpd.cpp` | Connects to `parodus` process; manages the recv-wait thread | +| `webpa_adapter.cpp` | Translates libparodus WRP messages to `HOSTIF_MsgData_t` | +| `webpa_parameter.cpp` | GetParam / SetParam over WebPA | +| `webpa_attribute.cpp` | GetAttr / SetAttr over WebPA | +| `webpa_notification.cpp` | Pushes value-change events back to parodus | + +### HTTP Server (`src/hostif/httpserver/`) + +An optional Mongoose-based HTTP server (disabled when `NEW_HTTP_SERVER_DISABLE` is defined or when the Legacy RFC feature flag is active). Provides a local REST endpoint used during RFC migration. Controlled at runtime by `/opt/RFC/.RFC_LegacyRFCEnabled.ini`. + +### SNMP Adapter (`src/hostif/snmpAdapter/`) + +Maps selected `Device.X_RDKCENTRAL-COM.*` parameters to SNMP OIDs defined in `conf/tr181_snmpOID.conf`. Enabled at build time with `--enable-snmp-adapter`. + +## Threading Model + +| Thread | Name | How Created | Purpose | +|--------|------|------------|---------| +| Main | `main` | OS | Init, GLib main loop | +| Shutdown | `shutdown_thread` | `pthread_create` | Waits on semaphore; calls `exit_gracefully()` on signal | +| JSON Handler | `json_if_handler_thread` | `g_thread_try_new` | Services JSON-over-socket requests | +| HTTP Server | `http_server_thread` | `g_thread_try_new` | Optional legacy HTTP RFC endpoint | +| Update Handler | `updateHandler` | `g_thread_try_new` | Polls profiles for value changes; fires IARM / Parodus events | +| Parodus Init | `parodus_init_tid` | `pthread_create` | Connects to parodus daemon, starts recv loop | +| WebConfig | `webconfig_threadId` | `pthread_create` | Handles WebConfig Lite document processing | +| Partner ID | `partnerIdThread` | `std::thread` (inside `XBSStore`) | Resolves partner ID asynchronously at boot | + +### Synchronization + +```c +// Signal → shutdown path +sem_t shutdown_thread_sem; // Main signals shutdown thread +pthread_mutex_t graceful_exit_mutex; // Protects shutdown sequence + +// HTTP server startup handshake +std::mutex mtx_httpServerThreadDone; +std::condition_variable cv_httpServerThreadDone; + +// Bootstrap store +static recursive_mutex XBSStore::mtx; // Guards m_dict cache +static mutex XBSStore::mtx_stopped; +static condition_variable XBSStore::cv; + +// Notification queue (lock-free) +GAsyncQueue* NotificationHandler::notificationQueue; +``` + +**Lock ordering**: No nested lock acquisitions exist across manager threads; each subsystem owns its own mutex. The GLib `GAsyncQueue` is used for the notification path to avoid blocking the update handler. + +## Data Structures + +### `HOSTIF_MsgData_t` — the universal request/response envelope + +```c +typedef struct _HostIf_MsgData_t { + char paramName[4096]; // Full TR-181 parameter path + char paramValue[4096]; // Value as string + char *paramValueLong; // Heap buffer for values > 4096 bytes + char transactionID[256]; // Correlation ID (WebPA / CWMP) + short paramLen; // Byte length of paramValue + short instanceNum; // Object instance number + HostIf_ParamType_t paramtype; // String/Int/Bool/DateTime/ULong + HostIf_ReqType_t reqType; // GET / SET / GETATTRIB / SETATTRIB + faultCode_t faultCode; // TR-069 fault code (0 = success) + HostIf_Source_Type_t requestor; // WEBPA / RFC / IARM / DEFAULT + HostIf_Source_Type_t bsUpdate; // Bootstrap source level + bool isLengthyParam; // true → use paramValueLong +} HOSTIF_MsgData_t; +``` + +### Fault Codes + +| Code | Name | Meaning | +|------|------|---------| +| 0 | `fcNoFault` | Success | +| 9000 | `fcMethodNotSupported` | RPC not implemented | +| 9001 | `fcRequestDenied` | Access denied | +| 9002 | `fcInternalError` | Unexpected internal failure | +| 9003 | `fcInvalidArguments` | Bad arguments | +| 9004 | `fcResourcesExceeded` | Resource limit hit | +| 9005 | `fcInvalidParameterName` | Unknown parameter | +| 9006 | `fcInvalidParameterType` | Type mismatch | +| 9007 | `fcInvalidParameterValue` | Value out of range or invalid | +| 9008 | `fcAttemptToSetaNonWritableParameter` | Read-only parameter | + +## Configuration + +### `conf/tr69hostIf.conf` + +```ini +[HOSTIF_DM_PROFILE_MGR] +Device.DeviceInfo=deviceMgr +Device.Services.STBService=dsMgr +Device.Services.StorageService=storageSrvcMgr +Device.MoCA=mocaMgr +Device.Ethernet=ethernetMgr +Device.IP=ipMgr +Device.Time=timeMgr +Device.WiFi=wifiMgr + +[HOSTIF_JSON_CONFIG] +PORT=10999 + +[HOSTIF_CONFIG] +REBOOT_SCR="/rebootNow.sh -s tr69hostIfReset" +RDK_SCR_PATH=/lib/rdk +NTP_FILE_NAME=/opt/persistent/firstNtpTime +FW_DWN_FILE_PATH=/opt/fwdnldstatus.txt +``` + +The `[HOSTIF_DM_PROFILE_MGR]` section defines which manager handles each TR-181 subtree prefix. The dispatcher matches incoming parameter names against these prefixes to route requests. + +### Runtime Feature Flags (RFC) + +| Path | Feature | +|------|---------| +| `/opt/RFC/.RFC_LegacyRFCEnabled.ini` | Enable legacy HTTP server instead of new HTTP server | +| `/opt/secure/RFC/.RFC_.ini` | General RFC feature toggles (created by `XRFCStorage`) | +| `/opt/debug.ini` | RDK logger configuration | + +### Build-Time Feature Flags (`configure.ac`) + +| Configure Flag | Preprocessor Define | Effect | +|----------------|--------------------|----| +| `--enable-parodus` | `PARODUS_ENABLE` | Enable WebPA/Parodus client | +| `--disable-new-http-server` | `NEW_HTTP_SERVER_DISABLE` | Remove internal HTTP server | +| `--enable-snmp-adapter` | `SNMP_ADAPTER_ENABLED` | Include SNMP OID bridge | +| `--enable-webpa-rfc` | `WEBPA_RFC_ENABLED` | Guard service on RFC flag | +| `--enable-rbus` | *(rbus linkage)* | Enable RBUS DML provider | +| `--enable-t2` | `T2_EVENT_ENABLED` | Telemetry 2.0 markers | +| `--enable-webconfig` | `WEB_CONFIG_ENABLED` | WebConfig multipart support | +| `--enable-webconfig-lite` | `WEBCONFIG_LITE_ENABLE` | WebConfig Lite | +| `--enable-wifi` | `USE_WIFI_PROFILE` | WiFi profile handlers | +| `--enable-moca` | *(moca linkage)* | MoCA profile handlers | + +## Build & Install + +### Prerequisites + +| Dependency | Minimum Version | Notes | +|------------|----------------|-------| +| GCC / G++ | 7+ | C++17 required | +| GLib 2 | 2.32+ | GThread, GMainLoop, GAsyncQueue | +| libcurl | 7.65+ | Used by DeviceInfo utilities | +| IARM Bus | — | RDK platform IPC | +| libparodus | — | Required with `--enable-parodus` | +| rbus | — | Required with `--enable-rbus` | +| safec | — | Safe string functions (`strcpy_s`, etc.) | +| cJSON | — | JSON parsing | +| OpenSSL | 1.1.1+ | TLS for HTTP server | + +### Build Steps + +```bash +# Generate build system +autoreconf -iv + +# Configure (example for a typical RDK broadband build) +./configure \ + --enable-parodus \ + --enable-rbus \ + --enable-wifi \ + --enable-moca \ + --enable-t2 + +# Build +make -j$(nproc) + +# Install +make install +``` + +### Run + +```bash +# Typical invocation (as managed by systemd) +/usr/bin/tr69hostIf -c /etc/tr69hostIf.conf -p 10000 + +# Options +# -c Configuration file path +# -p IARM listen port +# -s HTTP server port (legacy mode only) +# -l Log file path +# -h Show usage +``` + +The provided systemd unit files are: +- `tr69hostif.service` — standard deployment +- `tr69hostif_no_new_http_server.service` — deployment with `NEW_HTTP_SERVER_DISABLE` + +## Testing + +### Unit Tests + +```bash +# Build and run unit tests +./run_ut.sh +``` + +Unit tests live under `src/unittest/` and `src/hostif/**/gtest/`. They use **Google Test** and rely on stub headers under `src/unittest/stubs/` to isolate the daemon from IARM, DS, and other platform dependencies. + +Key test areas: + +| Test Suite | Location | Coverage | +|------------|----------|----------| +| RFC Store | `profiles/DeviceInfo/gtest/` | `XRFCStorage` get/set/clear | +| Bootstrap Store | `profiles/DeviceInfo/gtest/` | `XBSStore` partner loading | +| JSON Handler | `handlers/src/gtest/` | Request parsing and routing | +| IARM Handler | `handlers/src/gtest/` | IARM RPC dispatch | +| IniFile | `src/gtest/` | INI parser correctness | + +### Integration / L2 Tests + +```bash +# Run L2 integration tests (requires Docker) +./run_l2.sh +``` + +L2 tests live under `src/integrationtest/` (configuration fixtures) and `test/functional-tests/` (Behave BDD scenarios). They exercise the full daemon end-to-end against mock IARM and RFC infrastructure. + +## Directory Reference + +``` +tr69hostif/ +├── configure.ac # Autoconf top-level +├── Makefile.am # Top-level Automake +├── conf/ # Runtime configuration +│ ├── tr69hostIf.conf # Manager-to-prefix mapping +│ ├── mgrlist.conf # Manager list +│ ├── tr181_snmpOID.conf # SNMP OID mappings +│ └── rfcdefaults/ +│ └── tr69hostif.ini # RFC default values +├── src/ +│ ├── backgroundrun.c # Helper to run scripts in background +│ └── hostif/ +│ ├── src/ # Core daemon source +│ ├── include/ # Core public headers +│ ├── handlers/ # Request dispatching layer +│ ├── profiles/ # TR-181 object implementations +│ ├── parodusClient/ # WebPA / Parodus PAL +│ ├── httpserver/ # Optional HTTP server +│ └── snmpAdapter/ # SNMP bridge +├── test/ +│ └── functional-tests/ # BDD integration tests (Behave) +└── scripts/ + └── validateDataModel.py # Data model XML validation utility +``` + +## Logging + +tr69hostif uses the RDK Logger (`rdk_debug.h`). Log levels map to standard RDK levels: `FATAL`, `ERROR`, `WARN`, `NOTICE`, `INFO`, `DEBUG`, `TRACE1/2`. + +The log category is `LOG_TR69HOSTIF`. To enable verbose logging at runtime, add the following to `/opt/debug.ini`: + +```ini +LOG.RDK.TR69HOSTIF = DEBUG +``` + +Telemetry 2.0 markers (when `T2_EVENT_ENABLED` is defined) are emitted via `t2_event_s()` / `t2_event_d()` for key lifecycle events. + +## Platform Notes + +### RDKB (Broadband Gateway) +- Uses IARM-Bus for all cross-process communication. +- WiFi parameters delegate to the `wifihal` abstraction layer. +- RFC overrides stored under `/opt/secure/RFC/`. +- Bootstrap defaults loaded from `/etc/partners_defaults.json` or `/opt/partners_defaults.json`. + +### RDKV (Video/STB) +- `RDKV_TR69` compile flag activates STB-specific code paths. +- DS (Device Settings) profile enabled; STBService provides HDMI, FPD, audio, and video object support. +- Base data model file: `/etc/data-model.xml` merged with device-type overlays at startup. + +### General Constraints +- Minimum 64 MB RAM recommended. +- ARMv7 or better CPU. +- GLib 2 event loop required (no bare POSIX event loop replacement). + +## Contributing + +See [CONTRIBUTING.md](CONTRIBUTING.md). All contributions require signing the RDK Contributor License Agreement. + +## License + +Licensed under the [Apache License, Version 2.0](LICENSE). + +Copyright 2016 RDK Management. + +## See Also + +- [CHANGELOG](CHANGELOG.md) — Release history +- [conf/tr69hostIf.conf](conf/tr69hostIf.conf) — Runtime configuration reference +- [run_ut.sh](run_ut.sh) — Unit test runner +- [run_l2.sh](run_l2.sh) — L2 integration test runner diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..62ee25d27 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,34 @@ +# tr69hostif Documentation + +This directory contains implementation-oriented documentation for the tr69hostif daemon. The goal is to keep architecture, API, build, and test information close to the source tree and grounded in the current codebase. + +## Documentation Index + +### Architecture + +- [System Overview](architecture/overview.md) describes the daemon's major components, startup sequence, and runtime boundaries. +- [Threading Model](architecture/threading-model.md) documents worker threads, synchronization primitives, and shutdown behavior. +- [Data Flow](architecture/data-flow.md) traces request routing, event propagation, and RFC/bootstrap precedence. + +### API + +- [Public API](api/public-api.md) documents the shared request envelope, dispatcher entry points, IARM-facing APIs, and event contracts. + +### Integration + +- [Build Setup](integration/build-setup.md) covers the autotools workflow, feature flags, runtime prerequisites, and deployment notes. +- [Testing](integration/testing.md) covers the repo's unit-test and L2 test flows, including coverage generation. + +### Troubleshooting + +- [Common Errors](troubleshooting/common-errors.md) summarizes the most common startup, routing, and Parodus integration failures. + +## Scope + +The pages in this directory are intentionally implementation-specific. They reference the current source layout under `src/hostif/`, the shipped config files under `conf/`, and the repo-maintained validation scripts such as `run_ut.sh` and `run_l2.sh`. + +## Maintenance Rules + +- Update the relevant page when thread ownership, feature flags, or request routing changes. +- Keep Mermaid diagrams synchronized with the current code paths. +- Prefer linking to source files and config files already present in the repository instead of copying large code blocks into docs. \ No newline at end of file diff --git a/docs/api/public-api.md b/docs/api/public-api.md new file mode 100644 index 000000000..db7d8e2fc --- /dev/null +++ b/docs/api/public-api.md @@ -0,0 +1,202 @@ +# Public API + +## Overview + +The core public contract for `tr69hostif` is the shared request/response envelope declared in `src/hostif/include/hostIf_tr69ReqHandler.h`. IPC front ends such as IARM and WebPA populate this structure, invoke the appropriate dispatcher path, and inspect the returned `faultCode` and value fields. + +## Core Types + +### `HOSTIF_MsgData_t` + +```c +typedef struct _HostIf_MsgData_t { + char paramName[TR69HOSTIFMGR_MAX_PARAM_LEN]; + char paramValue[TR69HOSTIFMGR_MAX_PARAM_LEN]; + char* paramValueLong; + char transactionID[_BUF_LEN_256]; + short paramLen; + short instanceNum; + HostIf_ParamType_t paramtype; + HostIf_ReqType_t reqType; + faultCode_t faultCode; + HostIf_Source_Type_t requestor; + HostIf_Source_Type_t bsUpdate; + bool isLengthyParam; +} HOSTIF_MsgData_t; +``` + +### Field semantics + +| Field | Meaning | +|-------|---------| +| `paramName` | Fully qualified TR-181 parameter path | +| `paramValue` | Inline string buffer for normal-length values | +| `paramValueLong` | Heap buffer for long values when `isLengthyParam` is true | +| `transactionID` | Correlation token for remote callers | +| `paramLen` | Returned value length | +| `instanceNum` | Object instance identifier when applicable | +| `paramtype` | Value type such as string, int, bool, or unsigned long | +| `reqType` | GET, SET, GETATTRIB, or SETATTRIB | +| `faultCode` | TR-069 fault code returned to caller | +| `requestor` | Request source classification | +| `bsUpdate` | Bootstrap-update source level | + +## Enums + +### `HostIf_ParamType_t` + +- `hostIf_StringType` +- `hostIf_IntegerType` +- `hostIf_UnsignedIntType` +- `hostIf_BooleanType` +- `hostIf_DateTimeType` +- `hostIf_UnsignedLongType` + +### `HostIf_ReqType_t` + +- `HOSTIF_GET` +- `HOSTIF_SET` +- `HOSTIF_GETATTRIB` +- `HOSTIF_SETATTRIB` + +### `faultCode_t` + +| Value | Meaning | +|-------|---------| +| `fcNoFault` | Success | +| `fcMethodNotSupported` | Unsupported RPC or operation | +| `fcRequestDenied` | Access denied | +| `fcInternalError` | Internal processing failure | +| `fcInvalidArguments` | Invalid request arguments | +| `fcResourcesExceeded` | Resource exhaustion | +| `fcInvalidParameterName` | Unknown or unmapped parameter | +| `fcInvalidParameterType` | Type mismatch | +| `fcInvalidParameterValue` | Value outside accepted range | +| `fcAttemptToSetaNonWritableParameter` | Attempt to write read-only parameter | + +## Dispatcher Entry Points + +The primary C/C++ entry points are declared in `src/hostif/handlers/include/hostIf_msgHandler.h`. + +### `hostIf_GetMsgHandler()` + +```c +int hostIf_GetMsgHandler(HOSTIF_MsgData_t *stMsgData); +``` + +Routes a GET request to the appropriate manager. The function serializes top-level GET handling with `get_handler_mutex`, resolves the manager from `paramName`, and invokes `handleGetMsg()` on the selected handler. + +### `hostIf_SetMsgHandler()` + +```c +int hostIf_SetMsgHandler(HOSTIF_MsgData_t *stMsgData); +``` + +Routes a SET request to the appropriate manager. The function serializes top-level SET handling with `set_handler_mutex` and delegates to `handleSetMsg()`. + +### Attribute operations + +```c +int hostIf_GetAttributesMsgHandler(HOSTIF_MsgData_t *stMsgData); +int hostIf_SetAttributesMsgHandler(HOSTIF_MsgData_t *stMsgData); +``` + +These use the same manager resolution model for attribute-specific flows. + +### Utility helpers + +```c +void hostIf_Init_Dummy_stMsgData(HOSTIF_MsgData_t **stMsgData); +void hostIf_Print_msgData(HOSTIF_MsgData_t *stMsgData); +void hostIf_Free_stMsgData(HOSTIF_MsgData_t *stMsgData); +void paramValueToString(const HOSTIF_MsgData_t *stMsgData, char *paramValueStr, size_t strSize); +``` + +These helpers are used by internal adapters and tests to initialize, print, free, or stringify the shared envelope. + +## IARM Interface + +The IARM-facing contract is declared in `src/hostif/include/hostIf_tr69ReqHandler.h`. + +### Lifecycle + +```c +bool hostIf_IARM_IF_Start(void); +void hostIf_IARM_IF_Stop(void); +``` + +These functions initialize and tear down the daemon's IARM registration. + +### RPC names + +| Macro | RPC name | +|-------|----------| +| `IARM_BUS_TR69HOSTIFMGR_API_SetParams` | `tr69HostIfSetParams` | +| `IARM_BUS_TR69HOSTIFMGR_API_GetParams` | `tr69HostIfGetParams` | +| `IARM_BUS_TR69HOSTIFMGR_API_SetAttributes` | `tr69HostIfGetAttributes` | +| `IARM_BUS_TR69HOSTIFMGR_API_GetAttributes` | `tr69HostIfSetAttributes` | +| `IARM_BUS_TR69HOSTIFMGR_API_RegisterForEvents` | `tr69HostIfRegisterForEvents` | + +Note: The `SetAttributes` / `GetAttributes` RPC string names are intentionally reversed relative to the macro names for legacy/backward-compatibility. This mirrors the mappings in `hostIf_tr69ReqHandler.h`. +### Events + +| Event | Meaning | +|-------|---------| +| `IARM_BUS_TR69HOSTIFMGR_EVENT_ADD` | Dynamic object instance added | +| `IARM_BUS_TR69HOSTIFMGR_EVENT_REMOVE` | Dynamic object instance removed | +| `IARM_BUS_TR69HOSTIFMGR_EVENT_VALUECHANGED` | Existing parameter value changed | + +Event payloads use: + +```c +typedef struct _tr69HostIfMgr_EventData_t { + char paramName[TR69HOSTIFMGR_MAX_PARAM_LEN]; + char paramValue[TR69HOSTIFMGR_MAX_PARAM_LEN]; + HostIf_ParamType_t paramtype; +} IARM_Bus_tr69HostIfMgr_EventData_t; +``` + +## Thread Safety + +- Top-level GET dispatch is serialized. +- Top-level SET dispatch is serialized. +- The API does not guarantee that individual profile handlers are reentrant beyond the dispatcher-level locking shown above. +- Callers that allocate `paramValueLong` must preserve a matching cleanup path. + +## Example: Internal GET Request + +```c +#include + +#include "hostIf_msgHandler.h" +#include "hostIf_tr69ReqHandler.h" + +int query_manufacturer(void) +{ + HOSTIF_MsgData_t request; + memset(&request, 0, sizeof(request)); + + strncpy(request.paramName, + "Device.DeviceInfo.Manufacturer", + sizeof(request.paramName) - 1); + request.reqType = HOSTIF_GET; + request.paramtype = hostIf_StringType; + request.requestor = HOSTIF_SRC_IARM; + + if (hostIf_GetMsgHandler(&request) != 0) { + return -1; + } + + if (request.faultCode != fcNoFault) { + return -1; + } + + return 0; +} +``` + +## See Also + +- [System Overview](../architecture/overview.md) +- [Data Flow](../architecture/data-flow.md) +- [Testing](../integration/testing.md) \ No newline at end of file diff --git a/docs/architecture/data-flow.md b/docs/architecture/data-flow.md new file mode 100644 index 000000000..6c2400f98 --- /dev/null +++ b/docs/architecture/data-flow.md @@ -0,0 +1,111 @@ +# Data Flow + +## Request Routing + +All ingress paths converge on the same internal contract: a populated `HOSTIF_MsgData_t` structure plus a request type. The dispatcher resolves the manager from the parameter name prefix and forwards the call to the appropriate profile handler. + +```text +IARM RPC -----------+ +WebPA WRP request --+ +Local JSON request -+--> HOSTIF request envelope --> Match parameter prefix +RBUS DML provider --+ + +Match parameter prefix --> deviceMgr ----------+ +Match parameter prefix --> wifiMgr ------------+ +Match parameter prefix --> ipMgr --------------+--> Profile get/set handler +Match parameter prefix --> ethernetMgr --------+ +Match parameter prefix --> timeMgr ------------+ +Match parameter prefix --> other managers -----+ + +Profile get/set handler --> HAL or platform state --> Updated request envelope --> Caller response +``` + +## Manager Resolution + +The manager map is configured in `conf/tr69hostIf.conf` and test environments copy an equivalent file to `/etc/mgrlist.conf`. Representative mappings include: + +| Parameter prefix | Manager | +|------------------|---------| +| `Device.DeviceInfo` | `deviceMgr` | +| `Device.Services.STBService` | `dsMgr` | +| `Device.Services.StorageService` | `storageSrvcMgr` | +| `Device.Ethernet` | `ethernetMgr` | +| `Device.IP` | `ipMgr` | +| `Device.Time` | `timeMgr` | +| `Device.WiFi` | `wifiMgr` | + +If no manager owns the parameter path, the request fails through the normal fault-code path and the caller sees an invalid-parameter-style result. + +## Synchronous GET and SET Flow + +```text +Caller -> hostIf_msgHandler: call get or set entry point +hostIf_msgHandler -> hostIf_msgHandler: lock request mutex +hostIf_msgHandler -> manager resolver: HostIf_GetMgr(paramName) +manager resolver -> hostIf_msgHandler: handler pointer +hostIf_msgHandler -> concrete handler: call profile handler +concrete handler -> device HAL: read or write platform state +device HAL -> concrete handler: value or status +concrete handler -> hostIf_msgHandler: fill faultCode and payload +hostIf_msgHandler -> Caller: return updated request envelope +``` + +## Notification Flow + +Profiles that support update callbacks register with `updateHandler::Init()`. The update thread polls them once per minute and rebroadcasts changes over IARM and, when enabled, over Parodus notifications. + +```text +updateHandler thread -> checkForUpdates on each profile + +If no change is detected: + checkForUpdates -> sleep 60 seconds + +If a change is detected: + checkForUpdates -> notifyCallback + notifyCallback -> IARM broadcast event + +If Parodus is enabled and the event is value-changed: + notifyCallback -> NotificationHandler queue -> send notification via libparodus -> sleep 60 seconds + +Otherwise: + notifyCallback -> sleep 60 seconds +``` + +## RFC and Bootstrap Precedence + +The DeviceInfo RFC/bootstrap subsystem applies values from multiple sources. The effective precedence is: + +```text +RFC override > explicit WebPA set > bootstrap default > firmware default +``` + +This matters because request flow may appear identical at the dispatcher layer while the DeviceInfo profile resolves values from persistent RFC or bootstrap stores instead of querying a live HAL source. + +## Memory Ownership + +### Request envelope + +- `paramName`, `paramValue`, and `transactionID` are inline buffers owned by the caller or current stack frame. +- `paramValueLong` is heap-backed and is used for lengthy values. Any code that allocates it is responsible for the matching cleanup path. +- `faultCode` is the canonical result field for remote callers. + +### Parodus messages + +- Incoming WRP messages are owned by the receive loop until processed and released. +- Response and notification messages allocate transient payload metadata such as source, destination, and content type strings. +- `wrp_free_struct()` is the final release point for those messages. + +## Error Propagation + +The daemon distinguishes two layers of failure reporting: + +- local handler return status such as `OK` or `NOK` +- TR-069 fault codes stored in `HOSTIF_MsgData_t.faultCode` + +This allows protocol adapters to return a transport-level response while preserving the device-management-specific cause of failure. + +## See Also + +- [System Overview](overview.md) +- [Threading Model](threading-model.md) +- [Public API](../api/public-api.md) \ No newline at end of file diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md new file mode 100644 index 000000000..37235bd89 --- /dev/null +++ b/docs/architecture/overview.md @@ -0,0 +1,130 @@ +# System Overview + +## Overview + +`tr69hostif` is the TR-069 host interface daemon for RDK devices. It accepts TR-181 get/set traffic from multiple front ends, routes each request to the matching profile handler, and normalizes the response into a shared `HOSTIF_MsgData_t` envelope. + +At runtime the daemon combines several responsibilities: IPC termination over IARM and WebPA/Parodus, local request dispatch, profile-specific HAL translation, optional HTTP/RBUS integration, and change-notification fanout. + +## Component Diagram + +```mermaid +graph TB + subgraph Callers[Request Sources] + ACS[ACS / CWMP] + WEBPA[WebPA / Parodus] + JSON[Local JSON socket] + RBUS[RBUS clients] + SNMP[SNMP bridge] + end + + subgraph Core[tr69hostif daemon] + MAIN[main and startup] + IARM[IARM interface] + DISPATCH[hostIf_msgHandler] + UPDATE[updateHandler] + PARODUS[libpd / WebPA adapter] + HTTP[optional legacy HTTP server] + RBUSDML[RBUS DML provider] + + subgraph Profiles[TR-181 profile handlers] + DEVICEINFO[DeviceInfo] + WIFI[WiFi] + ETHERNET[Ethernet] + IP[IP] + TIME[Time] + DHCP[DHCPv4] + MOCA[MoCA] + STBSVC[STBService] + STORAGE[StorageService] + IFSTACK[InterfaceStack] + end + + subgraph Stores[RFC and bootstrap stores] + RFC[XRFCStorage] + BS[XBSStore] + JOURNAL[XBSStoreJournal] + end + end + + ACS --> IARM + WEBPA --> PARODUS + JSON --> DISPATCH + RBUS --> RBUSDML + SNMP --> IARM + + IARM --> DISPATCH + PARODUS --> DISPATCH + RBUSDML --> DISPATCH + DISPATCH --> Profiles + DISPATCH --> Stores + UPDATE --> IARM + UPDATE --> PARODUS +``` + +## Startup Sequence + +```mermaid +sequenceDiagram + participant MAIN as main() + participant CFG as config loader + participant IARM as IARM bus + participant DM as data model merge/load + participant THR as background workers + participant GMAINLOOP as GLib main loop + + MAIN->>CFG: parse argv and load config + MAIN->>IARM: hostIf_IARM_IF_Start() + MAIN->>DM: mergeDataModel() and load XML + MAIN->>THR: start JSON handler thread + MAIN->>THR: start updateHandler thread + MAIN->>THR: optionally start HTTP server + MAIN->>THR: optionally start Parodus client thread + MAIN->>THR: optionally start WebConfig thread + MAIN->>MAIN: optionally init RBUS provider + MAIN->>GMAINLOOP: g_main_loop_run() +``` + +## Major Subsystems + +| Subsystem | Primary files | Responsibility | +|-----------|---------------|----------------| +| Core startup | `src/hostif/src/hostIf_main.cpp` | Argument parsing, signal handling, worker startup, GLib main loop | +| Request dispatcher | `src/hostif/handlers/src/hostIf_msgHandler.cpp` | Maps parameter names to manager handlers and serializes GET/SET entry points | +| Request contract | `src/hostif/include/hostIf_tr69ReqHandler.h` | Shared request/response structure, fault codes, and IARM event definitions | +| Change monitoring | `src/hostif/handlers/src/hostIf_updateHandler.cpp` | Periodically checks profiles for value changes and emits notifications | +| WebPA/Parodus | `src/hostif/parodusClient/pal/libpd.cpp` | Connects to Parodus, receives WRP requests, and sends notifications | +| TR-181 profiles | `src/hostif/profiles/*` | Object-specific get/set logic and HAL translation | +| Optional HTTP server | `src/hostif/httpserver/` | Legacy RFC-related local HTTP endpoint | +| SNMP adapter | `src/hostif/snmpAdapter/` | Maps selected TR-181 parameters to SNMP OIDs | + +## Configuration Sources + +| File | Role | +|------|------| +| `conf/tr69hostIf.conf` | Manager name to parameter-prefix mapping and runtime defaults | +| `conf/mgrlist.conf` | Manager map copied into test and deployment environments | +| `/etc/data-model-*.xml` | Platform data-model fragments merged at startup | +| `/tmp/data-model.xml` | Effective merged model used by WebPA path | +| `/opt/secure/RFC/*.ini` | RFC overrides, bootstrap values, and journals | +| `partners_defaults.json` | Partner-specific default values consumed by bootstrap store | + +## Design Notes + +- The daemon uses a shared request envelope so IARM, WebPA, and internal call sites all converge on the same handler contract. +- Request routing is prefix-based. A parameter path is matched to a logical manager, then delegated to a concrete handler instance. +- Value-change notifications are decoupled from synchronous request handling. Profiles expose update callbacks, and a dedicated polling thread fans out changes. +- WebPA support is optional at build time and runtime. The Parodus path is isolated in the PAL layer under `src/hostif/parodusClient/pal/`. + +## Platform Notes + +- Linux pthreads, GLib threads, and GLib main loop are all used in the current implementation. +- Several feature areas are compile-time gated through `configure.ac`, including WiFi, DHCPv4, StorageService, InterfaceStack, MoCA, WebPA RFC, telemetry, and systemd notify. +- The daemon is packaged as a long-running systemd service using the unit files in the repository root. + +## See Also + +- [Threading Model](threading-model.md) +- [Data Flow](data-flow.md) +- [Build Setup](../integration/build-setup.md) +- [Public API](../api/public-api.md) \ No newline at end of file diff --git a/docs/architecture/threading-model.md b/docs/architecture/threading-model.md new file mode 100644 index 000000000..5855a29c3 --- /dev/null +++ b/docs/architecture/threading-model.md @@ -0,0 +1,101 @@ +# Threading Model + +## Overview + +`tr69hostif` mixes GLib-managed threads, POSIX threads, and one standard C++ thread in the bootstrap store. The design keeps long-running I/O and polling work off the main loop while preserving a single shared request contract for all front ends. + +## Thread Inventory + +| Thread | Creation site | Type | Purpose | Shutdown behavior | +|--------|---------------|------|---------|-------------------| +| Main thread | process start | OS main thread | Initializes services and runs `g_main_loop_run()` | Exits through `exit_gracefully()` | +| Shutdown thread | `hostIf_main.cpp` | `pthread_create()` | Waits on `shutdown_thread_sem` and triggers graceful exit on signal | Woken by signal handler path | +| JSON handler thread | `hostIf_main.cpp` | `g_thread_try_new()` | Handles JSON request traffic on configured socket | Stops during daemon shutdown | +| HTTP server thread | `hostIf_main.cpp` | `g_thread_try_new()` | Serves optional legacy HTTP RFC endpoint | Controlled by runtime and feature gating | +| Update handler | `updateHandler::Init()` | `g_thread_new()` | Polls profiles for changes and emits add/remove/value-changed events | Stops when `updateHandler::stopped` becomes true | +| Parodus init/receive thread | `pthread_create()` into `libpd_client_mgr()` | POSIX thread | Connects to Parodus and stays in receive/send loop | Self-detaches in `connect_parodus()` | +| WebConfig thread | `hostIf_main.cpp` | `pthread_create()` | Handles WebConfig Lite processing when enabled | Feature-gated | +| Partner ID worker | `XBSStore` | `std::thread` | Resolves bootstrap partner identity asynchronously | Store-specific lifecycle | + +## Synchronization Primitives + +| Primitive | Location | Role | +|-----------|----------|------| +| `pthread_mutex_t graceful_exit_mutex` | `hostIf_main.cpp` | Serializes graceful shutdown path | +| `sem_t shutdown_thread_sem` | `hostIf_main.cpp` | Wakes the dedicated shutdown thread | +| `std::mutex get_handler_mutex` | `hostIf_msgHandler.cpp` | Serializes synchronous GET dispatch | +| `std::mutex set_handler_mutex` | `hostIf_msgHandler.cpp` | Serializes synchronous SET dispatch | +| `std::mutex mtx_httpServerThreadDone` + `std::condition_variable cv_httpServerThreadDone` | `hostIf_main.cpp` | Coordinates HTTP server startup completion | +| `pthread_mutex_t parodus_lock` + `pthread_cond_t parodus_cond` | `libpd.cpp` | Implements timed wait/retry behavior in Parodus receive loop | +| `GAsyncQueue* notificationQueue` | notification handler | Asynchronous queue for outbound change notifications | +| bootstrap store mutexes and condition variable | `XBSStore` | Guard bootstrap dictionaries and stop notifications | + +## Concurrency Rules + +### Request handling + +- GET requests are serialized by `get_handler_mutex`. +- SET requests are serialized by `set_handler_mutex`. +- GET and SET paths use different mutexes, so one GET and one SET can proceed concurrently unless a downstream handler introduces tighter serialization. +- Attribute operations delegate through the same manager resolution path but do not add their own top-level mutex in `hostIf_msgHandler.cpp`. + +### Update monitoring + +The update handler is a single polling thread. It calls the profile-specific `checkForUpdates()` hooks in sequence and sleeps for 60 seconds between polling passes. This keeps notification generation predictable, but also means update latency is polling-based rather than interrupt-driven for most profiles. + +### Parodus behavior + +The Parodus worker thread calls `pthread_detach(pthread_self())` inside `connect_parodus()`. That makes it explicitly non-joinable and means shutdown logic must signal it to exit rather than attempt a `pthread_join()`. + +## Lifecycle Diagram + +```mermaid +stateDiagram-v2 + [*] --> Boot + Boot --> Init: parse config and start IPC + Init --> Running: main loop active + Running --> Polling: updateHandler iteration + Polling --> Running: sleep 60s + Running --> Receiving: Parodus request loop + Receiving --> Running: request processed + Running --> ShutdownRequested: signal or fatal stop path + ShutdownRequested --> Cleanup: stop workers and close IPC + Cleanup --> [*] +``` + +## Notification Path + +```mermaid +sequenceDiagram + participant PROF as Profile handler + participant UPD as updateHandler + participant IARM as IARM bus + participant NQ as notification queue + participant PD as Parodus sender + + PROF->>UPD: notifyCallback(event, paramName, value) + UPD->>IARM: IARM_Bus_BroadcastEvent(...) + alt value change and valid parameter name + UPD->>NQ: pushValueChangeNotification(eventData) + NQ->>PD: send outbound WebPA notification + end +``` + +## Shutdown Notes + +- Signals are converted into a semaphore wakeup for the dedicated shutdown thread. +- The update thread is cooperative and stops on a shared boolean flag. +- The Parodus receive loop exits when `exit_parodus_recv` is set and the condition variable is signaled. +- Detached workers must be shut down by signaling and resource cleanup, not by thread joining. + +## Operational Risks + +- Because update polling is single-threaded and sequential, a slow profile `checkForUpdates()` implementation can delay notifications for every other profile. +- The top-level GET/SET serialization simplifies safety but limits request concurrency under heavy management traffic. +- The Parodus path depends on external service availability and deliberately retries with exponential backoff. + +## See Also + +- [System Overview](overview.md) +- [Data Flow](data-flow.md) +- [Common Errors](../troubleshooting/common-errors.md) \ No newline at end of file diff --git a/docs/integration/build-setup.md b/docs/integration/build-setup.md new file mode 100644 index 000000000..baa4cfd35 --- /dev/null +++ b/docs/integration/build-setup.md @@ -0,0 +1,99 @@ +# Build Setup + +## Overview + +`tr69hostif` uses autotools and libtool as the primary build system. Feature areas are enabled with `./configure` flags, and the resulting binary composition depends heavily on the platform profile and enabled subsystems. + +## Standard Build Flow + +```sh +autoreconf --install +./configure [feature flags] +make -j"$(nproc)" +``` + +For repo-local testing, the current scripts also run: + +```sh +automake --add-missing +autoreconf --install +./configure --enable-libsoup3 +``` + +## Common Configure Flags + +The top-level `configure.ac` currently exposes feature toggles including: + +| Flag | Effect | +|------|--------| +| `--enable-xre` | Enable XRE-related profile support | +| `--enable-moca` / `--enable-moca2` | Enable MoCA profile support | +| `--enable-wifi` | Enable WiFi profile support | +| `--enable-DHCPv4` | Enable DHCPv4 profile support | +| `--enable-StorageService` | Enable StorageService profile support | +| `--enable-InterfaceStack` | Enable InterfaceStack profile support | +| `--enable-notification` | Enable value-change notification support | +| `--enable-t2api` | Enable telemetry hooks | +| `--enable-webpaRFC` | Enable WebPA RFC behavior | +| `--enable-IPv6` | Enable IPv6 behavior in IP profile | +| `--enable-SpeedTest` | Enable speed-test diagnostics | +| `--enable-systemd-notify` | Enable `sd_notify()` integration | +| `--enable-hwselftest` | Enable hardware self-test profile | + +Not every platform uses every flag. The effective feature set should match the device image, available HALs, and deployment requirements. + +## External Dependencies + +The repository test flows install or reference these representative dependencies: + +- autotools and libtool +- GLib +- libprocps or libprocps-ng +- libtinyxml2 +- libsoup 3 +- libnanomsg +- libparodus headers and libraries +- platform HALs and RDK middleware such as IARM-related components + +The unit-test workflow also clones external RDK repositories used for stubs and device-settings integration. + +## Runtime Files Required by the Daemon + +| Path | Purpose | +|------|---------| +| `/etc/mgrlist.conf` or configured manager map | Parameter-prefix to manager mapping | +| `/etc/data-model-generic.xml` | Generic data-model fragment | +| `/etc/data-model-stb.xml` or `/etc/data-model-tv.xml` | Platform-specific data-model fragment | +| `/tmp/data-model.xml` | Merged data model for WebPA path | +| `/opt/secure/RFC/` | RFC and bootstrap persistence | +| `/etc/partners_defaults.json` | Partner defaults consumed by bootstrap subsystem | + +## Service Files + +The repository includes multiple systemd unit files: + +- `tr69hostif.service` +- `tr69hostif_no_new_http_server.service` +- `ip-iface-monitor.service` + +Choose the unit that matches the build-time feature set and deployment model. + +## Build Notes + +- The daemon is highly feature-gated. Missing headers or libraries typically indicate a mismatched `./configure` flag set for the target platform. +- Data-model availability is a runtime prerequisite even when the binary builds successfully. +- WebPA/Parodus support depends on both build-time enablement and valid runtime configuration in `/etc/webpa_cfg.json`. + +## Example Development Build + +```sh +autoreconf --install +./configure --enable-wifi --enable-DHCPv4 --enable-notification --enable-systemd-notify +make -j4 +``` + +## See Also + +- [Testing](testing.md) +- [System Overview](../architecture/overview.md) +- [Common Errors](../troubleshooting/common-errors.md) \ No newline at end of file diff --git a/docs/integration/testing.md b/docs/integration/testing.md new file mode 100644 index 000000000..5d294c20a --- /dev/null +++ b/docs/integration/testing.md @@ -0,0 +1,97 @@ +# Testing + +## Overview + +This repository currently validates `tr69hostif` with a mix of component-level Google Test binaries and Python-based functional tests. The main entry points are `run_ut.sh` for unit-style coverage and `run_l2.sh` for L2 functional coverage. + +## Unit and Component Tests + +### Entry point + +```sh +./run_ut.sh +``` + +### What the script does + +- installs additional build dependencies with `apt-get` +- clones supporting RDK repositories used by the test environment +- prepares RFC, bootstrap, data-model, and stub files under `/etc`, `/opt`, and `/tmp` +- runs autotools bootstrap and `./configure --enable-libsoup3` +- builds and runs multiple gtest binaries under component-specific directories + +### GTest targets exercised by the script + +- handlers gtest +- Parodus data-model gtest +- HTTP server gtest +- core source gtest +- DHCPv4 gtest +- Device gtest +- Ethernet gtest +- Time gtest +- DeviceInfo gtest + +### Coverage mode + +```sh +./run_ut.sh --enable-cov +``` + +When coverage is enabled, the script adds GCC coverage flags and emits filtered `lcov` output for selected `src/hostif/` areas. + +## L2 Functional Tests + +### Entry point + +```sh +./run_l2.sh +``` + +### What the script does + +- stages data-model fragments into `/etc` +- writes test device metadata such as `RDK_PROFILE=STB` +- prepares RFC/bootstrap persistence files +- copies `mgrlist.conf` into `/etc` +- kills any already-running `tr69hostif` +- launches `/usr/local/bin/tr69hostif` with explicit config and ports +- runs Python `pytest` functional suites and writes JSON reports into `/tmp/l2_test_report` + +### Functional suites currently invoked + +- `test_bootup_sequence.py` +- `test_handlers_communications.py` +- `tr69hostif_deviceip.py` +- `tr69hostif_webpa.py` + +## Environment Considerations + +- Both scripts are environment-mutating. They create directories and files under `/etc`, `/opt`, `/tmp`, and `/usr`. +- The scripts are intended for disposable development or CI environments, not for production devices. +- `run_ut.sh` edits some source files transiently with `sed`, so use a clean workspace or review changes after the run. + +## Recommended Validation Order + +1. Run component tests first to catch local regressions quickly. +2. Run L2 functional tests after interface, profile, or WebPA changes. +3. Check logs and JSON reports together when debugging failures. + +## Test Artifacts + +| Artifact | Location | +|----------|----------| +| L2 JSON reports | `/tmp/l2_test_report` | +| Service log during L2 run | `/opt/logs/tr69hostIf.log.0` | +| Coverage report inputs | `coverage.info`, `filtered.info`, `tr69hostif_coverage.info` | + +## Debugging Failures + +- If a gtest binary fails to build, first verify the prerequisite headers and cloned dependencies are present. +- If functional tests fail early, inspect the staged data-model files and manager map. +- If WebPA tests fail, verify Parodus-related config and the merged data model in `/tmp/data-model.xml`. + +## See Also + +- [Build Setup](build-setup.md) +- [Common Errors](../troubleshooting/common-errors.md) \ No newline at end of file diff --git a/docs/troubleshooting/common-errors.md b/docs/troubleshooting/common-errors.md new file mode 100644 index 000000000..4c10b021c --- /dev/null +++ b/docs/troubleshooting/common-errors.md @@ -0,0 +1,101 @@ +# Common Errors + +## Missing or Incorrect Manager Map + +### Symptom + +Requests for valid TR-181 paths return invalid-parameter-style failures or never reach the expected handler. + +### Why it happens + +Routing depends on the parameter-prefix map loaded from `conf/tr69hostIf.conf` or the runtime copy in `/etc/mgrlist.conf`. If the prefix is missing or mapped to the wrong manager, dispatch resolution fails before the profile handler is invoked. + +### What to check + +- confirm the requested prefix exists in the active manager map +- confirm the binary was built with the corresponding profile enabled +- confirm the target handler is actually compiled into the image + +## Data Model Not Available + +### Symptom + +WebPA initialization fails, data-model loading fails at startup, or requests relying on merged XML behave incorrectly. + +### Why it happens + +The daemon expects platform data-model fragments under `/etc` and a merged model under `/tmp/data-model.xml` for WebPA-related flows. + +### What to check + +- verify `/etc/data-model-generic.xml` exists +- verify the platform fragment such as `/etc/data-model-stb.xml` or `/etc/data-model-tv.xml` exists +- verify the merged output was generated successfully + +## Parodus Unavailable or Misconfigured + +### Symptom + +WebPA requests do not arrive, notifications are not sent, or logs show repeated retry behavior. + +### Why it happens + +The Parodus client reads endpoint details from `/etc/webpa_cfg.json` and retries connection with exponential backoff. If the config is missing or the service is down, the worker stays in retry mode. + +### What to check + +- verify `/etc/webpa_cfg.json` is present and valid +- verify the Parodus service is running and reachable +- inspect runtime logs for `libparodus_init` retry messages + +## Shutdown Assumptions About Parodus Thread + +### Symptom + +Cleanup changes that try to join the Parodus worker crash or hang unexpectedly. + +### Why it happens + +The Parodus worker detaches itself inside `connect_parodus()`. A detached thread cannot be joined later. + +### What to check + +- make sure shutdown signals the receive loop instead of attempting `pthread_join()` +- keep thread lifecycle documentation aligned with any future Parodus changes + +## Bootstrap or RFC Value Confusion + +### Symptom + +Returned values do not match firmware defaults or live HAL expectations. + +### Why it happens + +DeviceInfo-related values may resolve from override storage or bootstrap defaults rather than from a live source. + +### What to check + +- inspect files under `/opt/secure/RFC/` +- verify bootstrap values and partner defaults +- confirm whether a value was previously set through WebPA or RFC override paths + +## Slow Notification Propagation + +### Symptom + +Value changes are visible eventually but not immediately. + +### Why it happens + +The update handler uses a polling loop and sleeps for 60 seconds between passes. + +### What to check + +- verify the affected profile participates in `registerUpdateCallback()` and `checkForUpdates()` +- account for the poll interval when interpreting latency + +## See Also + +- [Threading Model](../architecture/threading-model.md) +- [Data Flow](../architecture/data-flow.md) +- [Testing](../integration/testing.md) \ No newline at end of file diff --git a/src/hostif/docs/README.md b/src/hostif/docs/README.md new file mode 100644 index 000000000..01528f631 --- /dev/null +++ b/src/hostif/docs/README.md @@ -0,0 +1,762 @@ +# hostif Module — Implementation Overview + +## Overview + +The `src/hostif/` directory contains the complete implementation of the tr69hostif daemon — the RDK management TR-69/TR-181 host-interface process. The daemon exposes TR-181 parameter GET, SET, and attribute operations to remote management systems (TR-069 ACS, WebPA/Parodus, RBUS) and to local management clients over HTTP and IARM IPC. + +The module is organized into a core daemon layer (`src/`) surrounded by five specialized subsystems: `handlers/`, `httpserver/`, `parodusClient/`, `profiles/`, and `snmpAdapter/`. Each subsystem has its own documentation under its `docs/` folder. This README documents the core layer and the daemon-wide lifecycle that binds all subsystems together. + +--- + +## Directory Structure + +``` +src/hostif/ +├── src/ # Core daemon: main(), utils, INI file parser +│ ├── hostIf_main.cpp # Daemon entry point, startup sequence, shutdown +│ ├── hostIf_utils.cpp # Shared utilities, type conversion, curl helpers +│ ├── IniFile.cpp # Key=value INI file read/write helper +│ └── gtest/ # Unit tests for core utilities +│ +├── include/ # Public headers shared across all subsystems +│ ├── hostIf_main.h # Global types, T_ARGLIST, MERGE_STATUS, return codes +│ ├── hostIf_tr69ReqHandler.h # HOSTIF_MsgData_t, fault codes, parameter types +│ ├── hostIf_utils.h # Utility function declarations +│ └── IniFile.h # IniFile class declaration +│ +├── handlers/ # Request dispatch and transport bridges +├── httpserver/ # libsoup-based HTTP server for JSON GET/SET +├── parodusClient/ # WebPA/Parodus IPC client integration +├── profiles/ # TR-181 object implementations (Device.*, etc.) +├── snmpAdapter/ # SNMP bridge for DOCSIS and STB OIDs +│ +└── docs/ # This documentation (you are here) +``` + +--- + +## Architecture + +The daemon layers into four tiers, each building on the one below: + +```mermaid +graph TB + subgraph External[External Management Planes] + ACS[TR-069 ACS / CWMP] + WEBPA[WebPA / Parodus] + RBUS[RBUS clients] + HTTP[Local HTTP clients] + end + + subgraph Transport[Transport Layer - handlers/] + IARM[IARM RPC bridge] + JTHREAD[JSON handler thread] + RBUSPROV[RBUS DML provider] + HTTPSERV[libsoup HTTP server] + PAR[parodusClient] + end + + subgraph Dispatch[Dispatch Layer - handlers/] + MSG["hostIf_msgHandler
HostIf_GetMgr lookup
paramMgrhash"] + UPD["updateHandler
polling thread"] + NOTIF["NotificationHandler
GAsyncQueue"] + end + + subgraph Profiles[Profile Layer - profiles/ + snmpAdapter/] + DEV[Device.*] + ETH[Ethernet.*] + IP[IP.*] + WIFI[WiFi.*] + SNMP[DocsIf.* via SNMP] + OTHER[Time.* DHCPv4.* etc.] + end + + subgraph Core[Core Layer - src/] + MAIN["hostIf_main.cpp
daemon lifecycle"] + UTILS["hostIf_utils.cpp
type helpers"] + DM["Data Model
/tmp/data-model.xml"] + end + + ACS --> IARM + WEBPA --> PAR + RBUS --> RBUSPROV + HTTP --> HTTPSERV + IARM --> MSG + JTHREAD --> MSG + RBUSPROV --> MSG + HTTPSERV --> MSG + PAR --> MSG + MSG --> Profiles + UPD --> Profiles + UPD --> NOTIF + NOTIF --> PAR + MAIN --> Transport + MAIN --> Dispatch + MAIN --> DM +``` + +--- + +## How the Daemon Starts + +`main()` in `hostIf_main.cpp` performs a fixed ordered startup sequence. Understanding this sequence is essential for diagnosing boot-time failures. + +```mermaid +sequenceDiagram + participant main as main() + participant config as Config loading + participant iarm as IARM/handlers + participant dm as Data model + participant threads as Worker threads + participant sd as systemd + + main->>main: Parse CLI args (-c confFile -p port -s httpPort) + main->>main: rdk_logger_init + t2_init + main->>config: hostIf_initalize_ConfigManger() + config-->>main: paramMgrhash populated + main->>iarm: hostIf_IARM_IF_Start() + iarm-->>main: IARM bus init + RPC registration + main->>dm: mergeDataModel() + dm-->>main: /tmp/data-model.xml created + main->>dm: loadDataModel() + dm-->>main: waldb handle ready + main->>threads: g_thread_try_new json_if_handler_thread + main->>threads: g_thread_try_new http_server_thread (if !legacyRFC) + main->>threads: updateHandler::Init() + main->>threads: pthread_create libpd_client_mgr (Parodus) + main->>threads: init_rbus_dml_provider() + main->>main: wait cv_httpServerThreadDone (10s timeout) + main->>sd: sd_notifyf READY=1 + main->>main: g_main_loop_run (blocking) +``` + +### Key Startup Steps + +| Step | Function | What it does | +|------|----------|-------------| +| 1 | `hostIf_initalize_ConfigManger()` | Parses `mgrlist.conf` into `paramMgrhash`: maps TR-181 prefixes to manager enums | +| 2 | `hostIf_IARM_IF_Start()` | Initializes IARM bus, registers GET/SET/attribute RPCs, starts Device/DS/SNMP managers | +| 3 | `mergeDataModel()` | Reads `RDK_PROFILE` from `/etc/device.properties`, merges STB/TV/generic XML into `/tmp/data-model.xml` | +| 4 | `loadDataModel()` | Loads the merged data model into the waldb handle for param validation | +| 5 | `json_if_handler_thread` | Old HTTP/JSON request path (always started) | +| 6 | `http_server_thread` | New libsoup HTTP server (started only when `!NEW_HTTP_SERVER_DISABLE` and not in legacyRFC mode) | +| 7 | `updateHandler::Init()` | Registers change callbacks with managed profiles; starts 60s polling GLib thread | +| 8 | `libpd_client_mgr` | Connects to Parodus daemon, enters receive loop for WebPA requests | +| 9 | `init_rbus_dml_provider()` | Registers RBUS DML provider for TR-181 parameters | +| 10 | `sd_notifyf(READY=1)` | Informs systemd the daemon is ready | +| 11 | `g_main_loop_run()` | Enters GLib main loop; daemon blocks here until shutdown signal | + +### Data Model Merge + +Before the data model is loaded, `mergeDataModel()` builds `/tmp/data-model.xml` from static XML files: + +```mermaid +flowchart LR + PROPS["/etc/device.properties
RDK_PROFILE=STB or TV"] --> MERGE["mergeDataModel"]; + GENERIC["/etc/data-model-generic.xml"] --> MERGE; + STBXML["/etc/data-model-stb.xml"] --> MERGE; + TVXML["/etc/data-model-tv.xml"] --> MERGE; + BASE["/etc/data-model.xml
RDKV only"] --> MERGE; + MERGE --> OUT["/tmp/data-model.xml"]; + OUT --> WALDB["loadDataModel
waldb handle"]; +``` + +For `RDKV_TR69` builds: base is merged with generic as an intermediate step, then the profile-specific file is applied. +For `RDKE` builds: generic and the profile file are merged directly. + +--- + +## Shutdown Sequence + +Graceful shutdown is handled by a dedicated thread (`shutdown_thread`) that waits on a POSIX semaphore: + +```mermaid +sequenceDiagram + participant sig as OS signal + participant handler as quit_handler + participant sem as shutdown semaphore + participant thread as shutdown_thread + participant main as exit_gracefully + + sig->>handler: SIGINT / SIGTERM / SIGQUIT / SIGSEGV + handler->>sem: sem_post (async-signal-safe) + sem->>thread: unblocks + thread->>main: exit_gracefully(signal) + main->>main: pthread_mutex_trylock graceful_exit_mutex + main->>main: t2_uninit, WiFi shutdown (if enabled) + main->>main: stop_parodus_recv_wait + main->>main: hostIf_HttpServerStop + main->>main: updateHandler::stop + main->>main: XBSStore::getInstance()->stop() + main->>main: g_hash_table_destroy(paramMgrhash) + main->>main: hostIf_IARM_IF_Stop + main->>main: g_main_loop_quit + main->>main: HttpServerStop + main->>main: pthread_mutex_unlock graceful_exit_mutex +``` + +The `graceful_exit_mutex` prevents re-entrant shutdown if multiple signals arrive simultaneously. + +--- + +## Key Data Structures + +### `HOSTIF_MsgData_t` — the universal request envelope + +All GET, SET, and attribute operations between transport adapters, the dispatch layer, and profile implementations use this single structure: + +```cpp +typedef struct _HostIf_MsgData_t { + char paramName[TR69HOSTIFMGR_MAX_PARAM_LEN]; // TR-181 dotted param path (4 KB) + char paramValue[TR69HOSTIFMGR_MAX_PARAM_LEN]; // Binary-encoded value (4 KB) + char *paramValueLong; // Heap pointer for values > 4 KB + char transactionID[256]; // Caller transaction ID + short paramLen; // Byte length of paramValue + short instanceNum; // Object instance number + HostIf_ParamType_t paramtype; // Type of paramValue encoding + HostIf_ReqType_t reqType; // GET, SET, GETATTRIB, SETATTRIB + faultCode_t faultCode; // TR-069 fault code on error + HostIf_Source_Type_t requestor; // Source of the request + HostIf_Source_Type_t bsUpdate; // Bootstrap update classification + bool isLengthyParam; // Use paramValueLong instead +} HOSTIF_MsgData_t; +``` + +**Key design constraint**: `paramValue` is a fixed 4 KB buffer. Numeric types (int, bool, unsigned long) are stored as their raw binary representation via `put_int()`, `put_bool()`, etc., not as strings. The helpers in `hostIf_utils.cpp` provide the canonical encode/decode paths. + +### `T_ARGLIST` — CLI argument state + +```cpp +typedef struct argsList { + char logFileName[64]; // -l: log file path + char confFile[100]; // -c: manager config file path + int httpPort; // -p: old JSON HTTP port + int httpServerPort; // -s: new HTTP server port (conditional) +} T_ARGLIST; +``` + +`argList` is a global extern used throughout all subsystems to access the configured ports and paths. + +### `HostIf_ParamType_t` — parameter type encoding + +| Enum | Value encoding in `paramValue` | +|------|-------------------------------| +| `hostIf_StringType` | Null-terminated string | +| `hostIf_IntegerType` | `int` via `put_int()` / `get_int()` | +| `hostIf_UnsignedIntType` | `unsigned int` via `put_uint()` / `get_uint()` | +| `hostIf_BooleanType` | `bool` via `put_boolean()` / `get_boolean()` | +| `hostIf_UnsignedLongType` | `unsigned long` via `put_ulong()` / `get_ulong()` | +| `hostIf_DateTimeType` | String (ISO 8601 date-time) | + +### `faultCode_t` — TR-069 fault code set + +| Code | Name | Meaning | +|------|------|---------| +| 0 | `fcNoFault` | Success | +| 9000 | `fcMethodNotSupported` | RPC not available | +| 9001 | `fcRequestDenied` | Rejected by policy | +| 9002 | `fcInternalError` | Internal handler error | +| 9003 | `fcInvalidArguments` | Malformed request | +| 9006 | `fcInvalidParameterName` | No such parameter | +| 9007 | `fcInvalidParameterType` | Type mismatch | +| 9008 | `fcAttemptToSetaNonWritableParameter` | Read-only param SET | + +--- + +## Threading Model + +The daemon is inherently multi-threaded. The following threads are alive during normal operation: + +| Thread | Created by | Library primitive | Purpose | +|--------|-----------|-------------------|---------| +| Main thread | OS | — | Startup, main loop | +| `shutdown_thread` | `pthread_create` | POSIX semaphore | Signal handler proxy, graceful exit | +| `json_if_handler_thread` | `g_thread_try_new` | GLib | Legacy JSON/HTTP request processing | +| `http_server_thread` | `g_thread_try_new` | libsoup callbacks | New HTTP server for GET/SET | +| `updateHandler` worker | `g_thread_new` | GLib | Periodic 60-second profile polling | +| `libpd_client_mgr` (Parodus) | `pthread_create` | pthreads | WebPA/Parodus request receive loop | +| Power controller thread (`RDKB`) | `std::thread + detach` | pthreads (detached) | Connects PowerController, registers callback | +| WebConfig thread | `pthread_create` | pthreads | Fetches/applies WebConfig payloads | + +### Synchronization Overview + +| Primitive | Location | Protects | +|-----------|----------|---------| +| `get_handler_mutex` (std::mutex) | `hostIf_msgHandler.cpp` | GET dispatch path | +| `set_handler_mutex` (std::mutex) | `hostIf_msgHandler.cpp` | SET dispatch path | +| `graceful_exit_mutex` (pthread_mutex) | `hostIf_main.cpp` | Re-entrant shutdown prevention | +| `mtx_httpServerThreadDone` (std::mutex) | `hostIf_main.cpp` | HTTP server startup coordination | +| `cv_httpServerThreadDone` (std::condition_variable) | `hostIf_main.cpp` | Main thread waits for server ready | +| `m_mutex` (GMutex) | `snmpAdapter.cpp` | SNMP adapter access serialization | +| `NotificationHandler` GAsyncQueue | `hostIf_NotificationHandler.cpp` | Notification event queue | + +--- + +## `hostIf_utils.cpp` — Shared Utilities + +This file provides all type-neutral helpers used across the subsystems. + +### Type conversion helpers + +| Function | Direction | Notes | +|----------|-----------|-------| +| `put_int` / `get_int` | `int` ↔ `paramValue[]` | Binary copy via pointer cast | +| `put_uint` / `get_uint` | `unsigned int` ↔ `paramValue[]` | Binary copy | +| `put_ulong` / `get_ulong` | `unsigned long` ↔ `paramValue[]` | Binary copy | +| `put_boolean` / `get_boolean` | `bool` ↔ `paramValue[]` | Binary copy | +| `getStringValue()` | `HOSTIF_MsgData_t` → `std::string` | Dispatch on `paramtype` | +| `putValue()` | `std::string` → `HOSTIF_MsgData_t` | Dispatch on `paramtype` | +| `int_to_string` / `string_to_int` | String ↔ int | `sprintf` / `strtol` | +| `string_to_uint` / `string_to_ulong` | String ↔ unsigned | `strtoul` | +| `string_to_bool` | `"true"` / `"1"` → `bool` | `strcasecmp` | + +### Other utilities + +| Function | Purpose | +|----------|---------| +| `matchComponent()` | Prefix and instance-number parsing for TR-181 paths | +| `triggerResetScript()` | Executes cold / factory / warehouse / customer reset scripts | +| `getJsonRPCData()` | `libcurl` POST to WPEFramework JSON-RPC endpoint with Bearer token | +| `get_security_token()` | Calls `/usr/bin/WPEFrameworkSecurityUtility` via popen, parses JWT token | +| `getCurrentTime()` / `timeValDiff()` | Wall-clock timing for request duration logging | +| `setLegacyRFCEnabled()` / `legacyRFCEnabled()` | Runtime flag for legacy vs new HTTP server mode | +| `getBSUpdateEnum()` | Maps "rfcUpdate" / "allUpdate" / "default" strings to `HostIf_Source_Type_t` | +| `isWebpaReady()` | Checks for `/tmp/webpa/start_time` sentinel file | +| `get_system_manageble_ntp_time()` | Reads NTP-confirmed time from `/tmp/timeReceivedNTP` | +| `get_device_manageble_time()` | Polls `/tmp/webpa/start_time` up to 5 times for epoch value | + +### `IniFile` class + +A simple `key=value` file parser with in-memory dictionary and write-back: + +| Method | Purpose | +|--------|---------| +| `load(filename)` | Opens the file, parses `=`-delimited lines into `m_dict` | +| `value(key, default)` | Returns stored value or a caller-provided default | +| `setValue(key, value)` | Updates `m_dict` and immediately flushes to disk | +| `clear()` | Empties `m_dict` and flushes (erases file content) | +| `flush()` | Truncates and rewrites the INI file from `m_dict` | + +--- + +## Build Configuration and Feature Gates + +The daemon's compiled feature set is controlled by a set of build-time macros. The presence or absence of these macros changes which subsystems are compiled in and which runtime paths are active. + +| Macro | Effect when defined | +|-------|-------------------| +| `NEW_HTTP_SERVER_DISABLE` | Disables the libsoup HTTP server; old JSON path only | +| `PARODUS_ENABLE` | Enables Parodus/WebPA integration and `libpd_client_mgr` thread | +| `WEBPA_RFC_ENABLED` | Adds WEBPAXG feature flag check at startup; daemon exits if disabled | +| `ENABLE_SD_NOTIFY` | Sends `READY=1` to systemd via `sd_notifyf` | +| `RDKV_TR69` | Enables RDKV-specific two-step data model merge and `pwrMgr.h` | +| `WEB_CONFIG_ENABLED` | Enables WebConfig multipart task (`initWebConfigMultipartTask`) | +| `WEBCONFIG_LITE_ENABLE` | Enables lightweight WebConfig thread (`initWebConfigTask`) | +| `T2_EVENT_ENABLED` | Enables Telemetry 2 via `t2_event_d` / `t2_event_s` | +| `USE_WIFI_PROFILE` | Compiles in WiFi profile; calls `WiFiDevice::init/shutdown` | +| `IS_YOCTO_ENABLED` | Links `libsecure_wrapper` explicitly | +| `RDK_DEVICE_EMU` | Selects `eth0` instead of `eth1` as the Ethernet interface | +| `SNMP_ADAPTER_ENABLED` | Compiles in SNMP adapter and `SNMPClientReqHandler` | + +--- + +## Runtime File Dependencies + +The daemon reads, writes, or checks these paths at runtime: + +| Path | Access | Purpose | +|------|--------|---------| +| `argList.confFile` (default `mgrlist.conf`) | Read | Manager-to-prefix mapping | +| `/etc/device.properties` | Read | `RDK_PROFILE` determination | +| `/etc/data-model-generic.xml` | Read | Generic TR-181 data model fragment | +| `/etc/data-model-stb.xml` | Read | STB profile data model fragment | +| `/etc/data-model-tv.xml` | Read | TV profile data model fragment | +| `/etc/data-model.xml` | Read (RDKV only) | RDKV base data model | +| `/tmp/data-model.xml` | Write then Read | Merged runtime data model | +| `/opt/debug.ini` or `/etc/debug.ini` | Read | RDK logger level configuration | +| `/opt/RFC/.RFC_LegacyRFCEnabled.ini` | Existence check | Legacy RFC mode flag | +| `/opt/notify_webpa_cfg.json` or `/etc/notify_webpa_cfg.json` | Read | Parodus notification config | +| `/etc/tr181_snmpOID.conf` | Read | SNMP OID mapping (via snmpAdapter) | +| `/tmp/.tr69hostif_http_server_ready` | Write | Sentinel for RFC readiness check | +| `/tmp/webpa/` | Create + Write | Parodus working directory | +| `/tmp/webpa/start_time` | Read | WebPA manageable-time epoch | +| `/tmp/timeReceivedNTP` | Read | NTP confirmed time | +| Systemd socket | Write | `sd_notifyf(READY=1)` | + +--- + +## Component Interaction Summary + +```mermaid +graph LR + subgraph ExternalMgmt[External Management] + ACS[ACS / CWMP] + WEBPA[WebPA / Parodus] + RBUSCLIENT[RBUS clients] + HTTPCLIENT[HTTP clients] + end + subgraph CoreLayer[Core - hostif/src/] + MAIN[hostIf_main] + UTILS[hostIf_utils] + INI[IniFile] + DM[Data Model merger] + end + subgraph HandlersLayer[Handlers - hostif/handlers/] + IARMH[IARM ReqHandler] + MSGDISP[msgHandler dispatcher] + RBUSDML[RBUS DML provider] + JSONH[JSON handler thread] + UPDH[updateHandler] + NOTIFH[NotificationHandler] + end + subgraph ServicesLayer[Services] + HTTP[httpserver] + PARODUS[parodusClient] + end + subgraph ProfilesLayer[Profiles - hostif/profiles/] + PROFILES[TR-181 profile classes] + end + subgraph SNMPLayer[SNMP] + SNMP[snmpAdapter] + end + + ACS --> IARMH + HTTPCLIENT --> HTTP + WEBPA --> PARODUS + RBUSCLIENT --> RBUSDML + MAIN --> HandlersLayer + MAIN --> DM + HTTP --> MSGDISP + PARODUS --> MSGDISP + IARMH --> MSGDISP + JSONH --> MSGDISP + RBUSDML --> MSGDISP + MSGDISP --> PROFILES + MSGDISP --> SNMP + UPDH --> PROFILES + UPDH --> NOTIFH + NOTIFH --> PARODUS + UTILS --> ProfilesLayer + INI --> MAIN +``` + +--- + +## Known Issues and Gaps + +The following implementation gaps were identified by reviewing `hostIf_main.cpp`, `hostIf_utils.cpp`, and `IniFile.cpp`. Each entry records severity, the affected file and line area, the problem, and the recommended fix. + +--- + +### Gap 1 — Critical: `GetFeatureEnabled()` references undefined variable `feature` + +**File**: `src/hostif/src/hostIf_main.cpp` — `GetFeatureEnabled()` + +**Observation**: The function signature takes `char *cmd` but the function body uses `feature`, which is neither a parameter nor a local variable: + +```cpp +bool GetFeatureEnabled(char *cmd) +{ + struct stat buffer; + string fileName = "/opt/secure/RFC/" + string(".RFC_") + feature + ".ini"; + return (stat(fileName.c_str(), &buffer) == 0); +} +``` + +`feature` is an undeclared identifier. The only caller passes `"WEBPAXG"` as the argument named `cmd`. This code cannot compile unless `feature` has been defined as a global elsewhere (not visible in this file), making the function completely disconnected from its own parameter. + +**Impact**: If the `WEBPA_RFC_ENABLED` guard is ever active with a compiler that enforces the undeclared identifier error, the daemon will not compile. If `feature` resolves to a global with a different value, the RFC file check is silently wrong and `GetFeatureEnabled("WEBPAXG")` never tests WEBPAXG. + +**Recommended fix**: +```cpp +bool GetFeatureEnabled(const char *feature) +{ + struct stat buffer; + string fileName = "/opt/secure/RFC/" + string(".RFC_") + feature + ".ini"; + return (stat(fileName.c_str(), &buffer) == 0); +} +``` + +--- + +### Gap 2 — High: `SIGSEGV` routed through the shutdown semaphore path + +**File**: `src/hostif/src/hostIf_main.cpp` — `quit_handler()` and `shutdown_thread_entry()` + +**Observation**: `SIGSEGV` is registered with the same `quit_handler` as `SIGTERM`/`SIGINT`: + +```cpp +sigaction(SIGTERM, &sigact, NULL); // clean shutdown +// SIGQUIT is NOT registered — see below +signal(SIGPIPE, SIG_IGN); +``` + +`SIGQUIT` is logged in `shutdown_thread_entry()` but `sigaction(SIGQUIT, ...)` is never called (the code comment says "The actions for SIGINT, SIGTERM, SIGSEGV, and SIGQUIT are set" but `SIGQUIT` and `SIGSEGV` are not registered). When a segfault occurs, the default handler produces a core dump immediately without any cleanup. Setting a custom handler for `SIGSEGV` without using an alternate signal stack (`SA_ONSTACK` is set, but see below) can cause a double fault if the crash was a stack overflow. + +The `SA_ONSTACK` flag is set in `sigact.sa_flags` but no alternate stack is ever allocated via `sigaltstack()`. This means `SA_ONSTACK` has no effect and any signal handler execution uses the already-corrupted stack on SIGSEGV from a stack overflow. + +**Impact**: Stack-overflow crashes will immediately double-fault and produce an unclean process termination with no graceful cleanup logs. `SIGQUIT` is not handled, so `kill -QUIT ` does not trigger the shutdown path. + +**Recommended fix**: +```cpp +// Register an alternate stack before registering SIGSEGV: +stack_t ss; +ss.ss_sp = malloc(SIGSTKSZ); +ss.ss_size = SIGSTKSZ; +ss.ss_flags = 0; +sigaltstack(&ss, NULL); + +// Then register SIGSEGV and SIGQUIT: +sigaction(SIGSEGV, &sigact, NULL); +sigaction(SIGQUIT, &sigact, NULL); +``` + +--- + +### Gap 3 — High: `graceful_exit_mutex` is used but never initialized + +**File**: `src/hostif/src/hostIf_main.cpp` + +**Observation**: `graceful_exit_mutex` is declared as: + +```cpp +pthread_mutex_t graceful_exit_mutex; +``` + +It is never initialized with `pthread_mutex_init()` or `PTHREAD_MUTEX_INITIALIZER`. Using an uninitialized mutex with `pthread_mutex_trylock()` is undefined behavior. + +**Impact**: On platforms where `pthread_mutex_t` does not initialize to a valid unlocked state by default (non-Linux POSIX), `pthread_mutex_trylock(&graceful_exit_mutex)` can fail or crash, preventing any graceful shutdown. Even on Linux where it happens to work due to zero-initialization of BSS, relying on this is non-portable. + +**Recommended fix**: +```cpp +pthread_mutex_t graceful_exit_mutex = PTHREAD_MUTEX_INITIALIZER; +``` + +--- + +### Gap 4 — High: `main()` returns `DB_FAILURE` on data model error but does not clean up + +**File**: `src/hostif/src/hostIf_main.cpp` + +**Observation**: If `mergeDataModel()` or `loadDataModel()` fails, `main()` returns `DB_FAILURE` immediately. By this point: + +- IARM bus is connected (`hostIf_IARM_IF_Start()` has succeeded). +- The JSON handler thread has been created. +- The HTTP server thread may have been created. +- The shutdown semaphore has been initialized and the `shutdown_thread` is running. + +None of these are cleaned up before `return DB_FAILURE`. The IARM bus remains connected, threads keep running, and the semaphore is leaked. + +**Impact**: When data model initialization fails, the daemon exits without stopping its background threads. If systemd restarts the daemon, a second IARM registration attempt may fail because the first instance's bus connection was not properly terminated. + +**Recommended fix** — call the cleanup sequence before returning: +```cpp +if (mergeStatus != MERGE_SUCCESS) { + hostIf_IARM_IF_Stop(); + exit_gracefully(0); + return DB_FAILURE; +} +``` + +--- + +### Gap 5 — Medium: `IniFile::flush()` truncates the file on every `setValue()` call + +**File**: `src/hostif/src/IniFile.cpp` + +**Observation**: Every call to `setValue()` immediately calls `flush()`, which opens the file with `ios::out | ios::trunc` (the `ofstream` default) and rewrites the entire dictionary from scratch. The comment in the code acknowledges this: + +```cpp +// FIXME: truncating everytime is bad for flash in general +ofstream outputStream(m_filename.c_str()); // default is out|truncate +``` + +**Impact**: On devices with NAND flash storage, the combination of full-truncate + full-rewrite on every single-key update accelerates wear on the target sector. For INI files with many keys written during boot (device properties, bootstrap params), this creates unnecessary write amplification. + +**Recommended fix** — defer flush until an explicit `sync()` call, or batch writes with a dirty flag: +```cpp +bool IniFile::setValue(const string &key, const string &value) { + m_dict[key] = value; + m_dirty = true; + return true; // caller must call flush() explicitly +} +``` + +--- + +### Gap 6 — Medium: `mergeDataModel()` silently ignores unknown `RDK_PROFILE` values in RDKE builds + +**File**: `src/hostif/src/hostIf_main.cpp` — `mergeDataModelRDKE()` + +**Observation**: `mergeDataModelRDKE()` supports only `"TV"` and `"STB"`: + +```cpp +if (strcmp(rdk_profile, "TV") == 0) { ... } +else if (strcmp(rdk_profile, "STB") == 0) { ... } +else { + RDK_LOG(... "RDKE: Unsupported RDK_PROFILE: %s\n", rdk_profile); + return MERGE_FAILURE; +} +``` + +If `RDK_PROFILE` is empty due to a malformed or missing `/etc/device.properties` line, `rdk_profile` is an empty string that matches neither branch. The daemon returns `DB_FAILURE` from `main()` and exits. This is not logged at a prominent enough level to make the failure obvious in a field environment. + +**Impact**: Any device that ships with a new profile value (e.g., `"GATEWAY"` or `"HUB"`) or a device where `/etc/device.properties` was corrupted returns `DB_FAILURE` and the daemon exits, entirely disabling remote management. + +**Recommended fix** — add a default fallback that uses generic data model: +```cpp +else { + RDK_LOG(RDK_LOG_WARN, ..., "Unknown RDK_PROFILE '%s', falling back to generic\n", rdk_profile); + if (!filter_and_merge_xml(generic_file, generic_file, output_file)) + return MERGE_FAILURE; +} +``` + +--- + +### Gap 7 — Medium: `get_ulong()` returns `int` despite operating on `unsigned long` + +**File**: `src/hostif/src/hostIf_utils.cpp` + +**Observation**: + +```cpp +int get_ulong(const char* ptr) +{ + unsigned long *ret = (unsigned long *)ptr; + return *ret; +} +``` + +The return type is `int` (32-bit on all ABIs in this tree), but the value held in `paramValue` is an `unsigned long` (64-bit on LP64 systems). Values above 2,147,483,647 are silently truncated or sign-wrapped when stored in an `int` return. + +`put_ulong()` correctly uses `unsigned long`, so the asymmetry means every read-back of an `unsigned long` parameter loses the upper 32 bits. + +**Impact**: Any TR-181 parameter that holds a 64-bit counter (interface byte counters, total bytes received/sent) returns an incorrect value whenever the value exceeds 2³¹−1 (approximately 2 GB). CWMP ACS comparisons will fail once counters wrap. + +**Recommended fix**: +```cpp +unsigned long get_ulong(const char* ptr) +{ + const unsigned long *ret = (const unsigned long *)ptr; + return *ret; +} +``` + +--- + +### Gap 8 — Medium: `writeCurlResponse` does not accumulate data into the destination string + +**File**: `src/hostif/src/hostIf_utils.cpp` + +**Observation**: The libcurl write callback: + +```cpp +size_t static writeCurlResponse(void *ptr, size_t size, size_t nmemb, string stream) +{ + size_t realsize = size * nmemb; + string temp(static_cast(ptr), realsize); + stream.append(temp); // appending to a local copy + return realsize; +} +``` + +The `stream` parameter is passed **by value**, not by reference. `stream.append(temp)` modifies a local copy that is destroyed when the function returns. The caller's `response` string in `getJsonRPCData()` is never populated. + +**Impact**: `getJsonRPCData()` always returns an empty string regardless of whether the HTTP JSON-RPC call succeeded. Any profile logic that depends on the WPEFramework JSON-RPC response (device info, security token validation) receives empty data and fails silently. + +**Recommended fix**: +```cpp +size_t static writeCurlResponse(void *ptr, size_t size, size_t nmemb, string &stream) +{ + size_t realsize = size * nmemb; + stream.append(static_cast(ptr), realsize); + return realsize; +} +``` +The matching `CURLOPT_WRITEDATA` must pass `&response` (which is already done correctly by the caller). + +--- + +### Gap 9 — Low: The WEBPA RFC check reads from `/opt/secure/RFC/` but `LEGACY_RFC_ENABLED_PATH` reads from `/opt/RFC/` + +**File**: `src/hostif/src/hostIf_main.cpp` + +**Observation**: Two RFC-related file paths in the same file use different directory roots: + +```cpp +// WEBPA_RFC_ENABLED path: +string fileName = "/opt/secure/RFC/" + string(".RFC_") + feature + ".ini"; + +// Legacy RFC check: +#define LEGACY_RFC_ENABLED_PATH "/opt/RFC/.RFC_LegacyRFCEnabled.ini" +``` + +On some platforms, `/opt/secure/RFC/` is a security-restricted directory while `/opt/RFC/` is accessible to standard processes. If both directories exist but the daemon lacks permission to read `/opt/secure/RFC/`, `GetFeatureEnabled()` will always return `false` and the daemon will shut itself down via `sd_pid_notify(SD_FINALIZING)`. + +**Impact**: Daemon fails to start on systems where `/opt/secure/RFC/` requires elevated privileges, with no meaningful error log distinguishing "WEBPAXG disabled" from "permission denied". + +--- + +### Gap 10 — Low: `mergeDataModel()` uses `sscanf` without bounding the destination buffer + +**File**: `src/hostif/src/hostIf_main.cpp` — `mergeDataModel()` + +**Observation**: + +```cpp +char rdk_profile[256] = {0}; +// ... +int sscanf_result = sscanf(line, "RDK_PROFILE=%s", rdk_profile); +``` + +`sscanf` with `%s` has no field-width limit. If the `RDK_PROFILE=` line in `/etc/device.properties` contains a value longer than 255 characters (e.g., corrupted file), `rdk_profile` is overflowed. + +**Recommended fix**: +```cpp +int sscanf_result = sscanf(line, "RDK_PROFILE=%255s", rdk_profile); +``` + +--- + +### Gap Summary Table + +| # | Severity | File | Problem | Impact | +|---|----------|------|---------|--------| +| 1 | **Critical** | `hostIf_main.cpp` | `GetFeatureEnabled()` uses undeclared `feature` variable instead of `cmd` parameter | Compile error or silent wrong-path check when `WEBPA_RFC_ENABLED` is active | +| 2 | **High** | `hostIf_main.cpp` | `SIGSEGV` / `SIGQUIT` not registered; `SA_ONSTACK` set without `sigaltstack()` | Stack-overflow crashes double-fault; SIGQUIT unhandled | +| 3 | **High** | `hostIf_main.cpp` | `graceful_exit_mutex` never initialized | Undefined behavior on non-Linux POSIX; non-portable | +| 4 | **High** | `hostIf_main.cpp` | Data model failure returns early without IARM/thread cleanup | Zombie IARM connection blocks daemon restart | +| 5 | **Medium** | `IniFile.cpp` | `flush()` truncates and rewrites on every `setValue()` | Excessive flash wear; not suitable for high-frequency updates | +| 6 | **Medium** | `hostIf_main.cpp` | Unknown/empty `RDK_PROFILE` causes `MERGE_FAILURE` and exit | Remote management entirely disabled on unrecognized profile | +| 7 | **Medium** | `hostIf_utils.cpp` | `get_ulong()` returns `int`, truncating 64-bit values to 32 bits | Byte counters > 2 GB return wrong values to ACS and WebPA | +| 8 | **Medium** | `hostIf_utils.cpp` | `writeCurlResponse` takes `string` by value; response data never accumulated | `getJsonRPCData()` always returns empty string; JSON-RPC calls silently fail | +| 9 | **Low** | `hostIf_main.cpp` | RFC paths use `/opt/secure/RFC/` vs `/opt/RFC/` inconsistently | Permission failures look like "feature disabled" | +| 10 | **Low** | `hostIf_main.cpp` | `sscanf(..., "%s", rdk_profile)` has no field-width limit | Corrupted `device.properties` can overflow `rdk_profile[256]` | + +--- + +## Testing + +Unit tests for the core layer are in `src/hostif/src/gtest/`. The test binary is built with `GTEST_ENABLE` defined. + +When modifying the core layer, validate: + +1. Daemon starts cleanly with a valid `mgrlist.conf` and merged data model. +2. `mergeDataModel()` produces a valid `/tmp/data-model.xml` for each supported profile. +3. `loadDataModel()` succeeds and the waldb handle is ready before HTTP/Parodus threads start. +4. Clean shutdown on `SIGTERM` closes all threads and disconnects IARM. +5. `get_ulong` / `put_ulong` round-trip values above 4,294,967,295 correctly after Gap 7 fix. +6. `getJsonRPCData()` actually returns the HTTP response body after Gap 8 fix. + +--- + +## See Also + +- [handlers/docs/README.md](../handlers/docs/README.md) — Request dispatch and transport bridges +- [httpserver/docs/README.md](../httpserver/docs/README.md) — libsoup HTTP server module +- [parodusClient/docs/README.md](../parodusClient/docs/README.md) — WebPA/Parodus integration +- [snmpAdapter/docs/README.md](../snmpAdapter/docs/README.md) — SNMP adapter for DOCSIS and STB OIDs +- [docs/architecture/overview.md](../../../docs/architecture/overview.md) — Daemon-wide architecture +- [docs/api/public-api.md](../../../docs/api/public-api.md) — Public API reference +- [docs/architecture/threading-model.md](../../../docs/architecture/threading-model.md) — Full runtime thread model diff --git a/src/hostif/handlers/docs/README.md b/src/hostif/handlers/docs/README.md new file mode 100644 index 000000000..974e15182 --- /dev/null +++ b/src/hostif/handlers/docs/README.md @@ -0,0 +1,461 @@ +# Handlers Implementation Overview + +## Overview + +The handlers layer in tr69hostif is the request-dispatch boundary between transport-facing entry points and the TR-181 profile implementations. Code in `src/hostif/handlers/src/` accepts requests from IARM, JSON, RBUS, and notification paths, resolves each parameter to the correct manager, and forwards the operation to a concrete handler derived from `msgHandler`. + +This layer does not implement the full device logic for every TR-181 object. Its main responsibilities are routing, request normalization, singleton lifecycle for manager objects, event propagation, and update polling. The actual parameter-specific logic lives mostly under `src/hostif/profiles/`. + +## Source Layout + +| Path | Purpose | +|------|---------| +| `src/hostif/handlers/include/hostIf_msgHandler.h` | Base `msgHandler` interface and dispatcher declarations | +| `src/hostif/handlers/src/hostIf_msgHandler.cpp` | Core GET/SET/attribute dispatch, manager lookup, config loading | +| `src/hostif/handlers/src/hostIf_IARM_ReqHandler.cpp` | IARM bus initialization, RPC registration, IARM request entry points | +| `src/hostif/handlers/src/hostIf_jsonReqHandlerThread.cpp` | JSON request handling thread | +| `src/hostif/handlers/src/hostIf_rbus_Dml_Provider.cpp` | RBUS-facing DML provider integration | +| `src/hostif/handlers/src/hostIf_updateHandler.cpp` | Periodic polling for value-change events | +| `src/hostif/handlers/src/hostIf_NotificationHandler.cpp` | Parodus/WebPA notification enqueue and delivery support | +| `src/hostif/handlers/src/hostIf_*ReqHandler.cpp` | Concrete manager classes for Device, DS, Ethernet, IP, WiFi, DHCPv4, and other profiles | + +## Architecture + +The handlers layer is organized around one abstract interface and a set of singleton manager implementations: + +- `msgHandler` defines the common handler contract: `init()`, `unInit()`, `handleGetMsg()`, `handleSetMsg()`, `handleGetAttributesMsg()`, and `handleSetAttributesMsg()`. +- `HostIf_GetMgr()` performs prefix-based manager lookup using the runtime configuration loaded into `paramMgrhash`. +- Each concrete handler exposes a `getInstance()` singleton accessor and delegates parameter work into one or more profile classes. +- Transport entry points convert external requests into `HOSTIF_MsgData_t`, then call the common dispatcher functions in `hostIf_msgHandler.cpp`. + +### Component Diagram + +```mermaid +graph TB + subgraph Inputs[Request Sources] + IARM[IARM RPC] + JSON[JSON Thread] + RBUS[RBUS Provider] + WEBPA[WebPA / Notification Paths] + end + + subgraph Handlers[Handlers Layer] + IARMH[hostIf_IARM_ReqHandler] + MSG[hostIf_msgHandler] + LOOKUP[HostIf_GetMgr] + UPD[updateHandler] + NOTIF[NotificationHandler] + end + + subgraph Managers[Concrete Managers] + DEV[DeviceClientReqHandler] + DS[DSClientReqHandler] + ETH[EthernetClientReqHandler] + IP[IPClientReqHandler] + WIFI[WiFiReqHandler] + TIME[TimeClientReqHandler] + DHCP[DHCPv4ClientReqHandler] + IFS[InterfaceStackClientReqHandler] + STOR[StorageSrvcReqHandler] + SNMP[SNMPClientReqHandler] + T2[XRdkCentralT2] + XRDK[X_rdk_req_hdlr] + end + + subgraph Profiles[TR-181 Profiles] + PROFILE[Profile classes under src/hostif/profiles] + end + + IARM --> IARMH + JSON --> MSG + RBUS --> MSG + WEBPA --> NOTIF + IARMH --> MSG + MSG --> LOOKUP + LOOKUP --> Managers + Managers --> PROFILE + UPD --> Managers + UPD --> NOTIF +``` + +## Request Routing Model + +At runtime, the dispatcher builds a prefix-to-manager map from the configured hostif manager file. Two code paths exist: + +- `hostIf_initalize_ConfigManger()` parses a whitespace-delimited mapping file. +- `hostIf_ConfigProperties_Init()` parses grouped key/value configuration using GLib `GKeyFile`. + +Both paths populate `paramMgrhash`, which maps parameter prefixes such as `Device.DeviceInfo.` or `Device.WiFi.` to a `HostIf_ParamMgr_t` enum. `HostIf_GetMgr()` then scans the configured prefixes and returns the singleton manager that owns the requested subtree. + +### Request Flow + +```mermaid +sequenceDiagram + participant Caller as External Caller + participant Entry as Transport Entry Point + participant Msg as hostIf_*MsgHandler + participant Lookup as HostIf_GetMgr + participant Handler as Concrete msgHandler + participant Profile as Profile Implementation + + Caller->>Entry: GET/SET/ATTR request + Entry->>Entry: Fill HOSTIF_MsgData_t + Entry->>Msg: hostIf_GetMsgHandler() or hostIf_SetMsgHandler() + Msg->>Lookup: Resolve paramName prefix + Lookup-->>Msg: Singleton manager instance + Msg->>Handler: handleGetMsg() / handleSetMsg() + Handler->>Profile: Read or update parameter + Profile-->>Handler: Value or status + Handler-->>Msg: faultCode / result + Msg-->>Entry: Updated HOSTIF_MsgData_t + Entry-->>Caller: Transport-specific response +``` + +## Key Components + +### `msgHandler` base class + +The `msgHandler` class in `hostIf_msgHandler.h` is the common interface for all manager objects. It enforces a uniform contract for GET, SET, and attribute operations so that transports do not need to know profile-specific types. + +The class is intentionally small. Shared behavior such as routing, request logging, timing telemetry, and configuration lookup stays outside the class in free functions inside `hostIf_msgHandler.cpp`. + +### `hostIf_msgHandler.cpp` + +This file is the core of the handlers subsystem. It provides: + +- `hostIf_GetMsgHandler()` and `hostIf_SetMsgHandler()` for common request dispatch +- `hostIf_GetAttributesMsgHandler()` and `hostIf_SetAttributesMsgHandler()` for attribute operations +- `paramValueToString()` for type-aware logging +- `HostIf_GetMgr()` for runtime manager resolution +- configuration loading helpers for building `paramMgrhash` + +The GET and SET paths also include: + +- request counters for boot-time traffic visibility +- slow-request logging when a request takes more than five seconds +- optional T2 telemetry notifications when thresholds are exceeded +- separate mutexes for GET and SET serialization + +### IARM request bridge + +`hostIf_IARM_ReqHandler.cpp` owns the IARM-facing lifecycle: + +- bus initialization and connection +- registration of TR-069 host interface RPCs +- initial manager startup for Device, DS, and optional SNMP paths +- translation from incoming IARM calls to the common `hostIf_*MsgHandler()` dispatcher APIs +- power-state event handling used to publish deep-sleep notifications when the matching RFC parameter is enabled + +### Update and notification path + +`hostIf_updateHandler.cpp` manages periodic polling for change detection. During initialization it registers callback hooks with the enabled managers, then starts a GLib thread that checks for updates in a 60-second loop. + +When a manager reports a change, `updateHandler::notifyCallback()`: + +1. packages the event into `IARM_Bus_tr69HostIfMgr_EventData_t` +2. broadcasts it on IARM +3. optionally forwards value-change notifications to Parodus when notification support is enabled + +This makes the handlers layer the bridge between passive parameter access and active change distribution. + +## Handler Inventory + +The source tree contains one handler implementation per major TR-181 area or integration domain. Most concrete handlers follow the same broad pattern: + +- singleton allocation with `getInstance()` +- optional `init()` and `unInit()` hooks +- `handleGetMsg()` and `handleSetMsg()` implementations +- optional static `reset()`, `checkForUpdates()`, or `registerUpdateCallback()` helpers for event-driven flows + +### Transport and bridge handlers + +These files do not own a single TR-181 subtree. They connect external transports or background workflows to the common dispatcher. + +| File or class | Operates on | What it does in the module | +|---------------|-------------|-----------------------------| +| `hostIf_IARM_ReqHandler.cpp` | IARM bus RPCs and power events | Registers TR-069 hostif RPC calls on IARM, converts IARM requests into `HOSTIF_MsgData_t`, invokes GET/SET/attribute dispatch, and publishes deep-sleep related notifications when the relevant RFC is enabled | +| `hostIf_msgHandler.cpp` | Common dispatch path | Owns the shared GET/SET/attribute routing logic, request timing logs, boot-time counters, manager lookup, and configuration-driven prefix mapping | +| `hostIf_jsonReqHandlerThread.cpp` | JSON-over-HTTP request path | Starts the HTTP server thread and parses incoming JSON `paramList` payloads with YAJL before those requests are handed into the shared hostif path | +| `hostIf_rbus_Dml_Provider.cpp` | RBUS DML interface | Exposes parameters through RBUS, validates parameters against the loaded data model, converts RBUS value types to hostif types, and forwards RBUS GET requests to `hostIf_GetMsgHandler()` | +| `hostIf_updateHandler.cpp` | Periodic value-change polling | Registers update callbacks with enabled managers, runs the background polling loop, emits add/remove/value-changed IARM events, and forwards change notifications to Parodus when enabled | +| `hostIf_NotificationHandler.cpp` | Parodus/WebPA notification delivery | Builds JSON payloads for value-change and key/value notifications, queues them on a `GAsyncQueue`, and wakes the registered Parodus sender callback | + +### Subtree and feature handlers + +These classes own specific TR-181 areas or integration namespaces and are the objects returned by `HostIf_GetMgr()`. + +| Handler | Operates on | Notes from implementation | +|---------|-------------|---------------------------| +| `DeviceClientReqHandler` | `Device.DeviceInfo.*`, selected bootstrap and firmware paths, and some SNMP-adjacent DeviceInfo parameters | Routes DeviceInfo GET and SET requests into `hostIf_DeviceInfo`, `hostIf_DeviceProcessorInterface`, and `hostIf_DeviceProcessStatusInterface`; handles reset, firmware download, preferred gateway, log upload, reverse SSH, bootstrap updates, and some `Device.DeviceInfo.X_RDK_SNMP.*` paths | +| `DSClientReqHandler` | `Device.Services.STBService.1.Components.*` and related DS-backed capabilities | Initializes `device::Manager`, then dispatches HDMI, VideoDecoder, AudioOutput, SPDIF, VideoOutput, and capability-related requests to the Device Settings service layer | +| `EthernetClientReqHandler` | `Device.Ethernet.Interface.*` and `Device.Ethernet.Interface.{i}.Stats.*` | Handles Ethernet interface state, alias, lower-layer relationships, bitrate, duplex mode, and per-interface statistics; also tracks interface count changes for event reporting | +| `IPClientReqHandler` | `Device.IP.*`, `Device.IP.Interface.*`, `IPv4Address`, optional `IPv6Address`, `ActivePort`, and diagnostics | Dispatches IP stack, interface, address, and active-port reads; when built with optional flags it also covers IPv6 and speed-test related objects; maintains cached entry counts for update detection | +| `TimeClientReqHandler` | `Device.Time.*` | Handles time enablement, Chrony/NTP settings, NTP directive parameters, and bootstrap-sensitive time parameters through `hostIf_Time` | +| `WiFiReqHandler` | `Device.WiFi.*` including Radio, SSID, AccessPoint, EndPoint, WPS, Security, Stats, and optional client roaming | Manages the broad WiFi subtree, supports WiFi global enable and roaming-related SETs, closes all WiFi object instances on shutdown, and tracks object counts for radios, SSIDs, and endpoints | +| `MoCAClientReqHandler` | `Device.MoCA.Interface.*`, QoS, associated devices, stats, and mesh-table related objects | Handles MoCA interface configuration such as enable, alias, privacy, keying, power limits, QoS-related objects, and mesh-entry tracking when the MoCA profile is enabled | +| `DHCPv4ClientReqHandler` | `Device.DHCPv4.Client.*` | Read-only handler in practice for the current code path; returns client interface references, routers, and DNS servers, and reports the client entry count | +| `InterfaceStackClientReqHandler` | `Device.InterfaceStack.*` | Read-only handler that exposes higher-layer and lower-layer relationships between interfaces and reports `InterfaceStackNumberOfEntries` | +| `StorageSrvcReqHandler` | `Device.services.StorageService.*` | Delegates storage-service GET requests to `hostIf_StorageSrvc`; the current implementation exposes reads and leaves SET and attribute support effectively unimplemented | +| `SNMPClientReqHandler` | `Device.X_RDKCENTRAL-COM_DocsIf.*` and `Device.DeviceInfo.X_RDK_SNMP.*` | Bridges hostif requests to the SNMP adapter, supports selected DOCSIS and DeviceInfo-backed SNMP values, initializes the SNMP adapter, and stores notification attributes in a hash table | +| `XREClientReqHandler` | `Device.X_COMCAST-COM_Xcalibur.Client.*`, `...Client.XRE.*`, and related XRE/DevApp control parameters | Handles XRE operational controls such as xconf check-now, session refresh, XRE restart, cache flush, log level changes, and receiver/dev-app restart flows when the XRE profile is enabled | +| `XRdkCentralT2` | `Device.X_RDKCENTRAL-COM_T2.ReportProfiles` and `...ReportProfilesMsgPack` | Pass-through handler that forwards Telemetry 2 profile payloads to RBUS, supports long-string transfer using `paramValueLong`, and cross-checks written report profile data | +| `X_rdk_req_hdlr` | Parameters under the internal `X_RDK_PREFIX_STR` namespace | Thin mutex-protected wrapper around `X_rdk_profile`, used for RDK-specific parameters that are not part of the main standard object handlers | + +### Supporting notes + +- `hostIf_updateHandler.cpp` only polls handlers that register update callbacks; not every concrete manager participates in change detection. +- Some handlers are compiled only when their profile or feature flag is enabled, so their source exists even when the target image excludes them. +- `hostIf_sysScriptHandler.cpp` exists in the directory but is effectively a placeholder in the current tree and is not part of the core `libMsgHandlers.la` source list shown in `Makefile.am`. + +## Build-Time Feature Gating + +The handlers library is assembled in `src/hostif/handlers/Makefile.am` as `libMsgHandlers.la`. The file shows that several managers are compiled conditionally. + +Common feature gates include: + +- `WITH_WIFI_PROFILE` for WiFi handler support +- `WITH_MOCA_PROFILE` for MoCA manager support +- `WITH_DHCP_PROFILE` for DHCPv4 support +- `WITH_INTFSTACK_PROFILE` for InterfaceStack support +- `WITH_STORAGESERVICE_PROFILE` for StorageService support +- `WITH_SNMP_ADAPTER` for SNMP adapter integration +- `WITH_NOTIFICATION_SUPPORT` for value-change notification behavior +- `IS_TELEMETRY2_ENABLED` for T2 metrics and reporting hooks + +Because of these flags, the exact set of managers in a target image can vary by platform build. + +## Threading Model + +The handlers layer is not a single-threaded module. It is entered concurrently from multiple runtime paths. + +| Thread or Context | Entry Point | Role | +|-------------------|-------------|------| +| Main/service startup | `hostIf_IARM_IF_Start()` | Initialize bus-facing managers and register RPCs | +| IARM worker context | `_Gettr69HostIfMgr()`, `_Settr69HostIfMgr()` | Process synchronous bus requests | +| JSON handler thread | `hostIf_jsonReqHandlerThread.cpp` | Process JSON-based requests | +| RBUS context | `hostIf_rbus_Dml_Provider.cpp` | Serve RBUS DML operations | +| Update thread | `updateHandler::run()` | Poll enabled managers for state changes every 60 seconds | +| Detached power-controller thread | `hostIf_getPwrContInterface()` on non-RDKV builds | Connect power controller callbacks used for deep-sleep notifications | + +### Synchronization + +The subsystem uses straightforward locking rather than a global scheduler: + +- `get_handler_mutex` serializes GET dispatch in `hostIf_GetMsgHandler()` +- `set_handler_mutex` serializes SET dispatch in `hostIf_SetMsgHandler()` +- `sendAddRemoveEvents()` uses a static mutex to serialize add/remove event emission +- GLib thread primitives are used for the update worker thread + +The current implementation favors correctness and predictable logging over maximum parallelism. GET and SET operations are serialized separately before the request reaches the concrete handler. + +## Memory and Ownership + +The handlers layer mostly treats `HOSTIF_MsgData_t` as caller-owned request state that is mutated in place. Ownership rules visible in this directory are: + +- transport adapters allocate or receive a `HOSTIF_MsgData_t` and pass it into dispatcher functions +- handlers update the same structure rather than returning a second response object +- `hostIf_Free_stMsgData()` uses `g_free()`, so callers must match the allocation strategy used by their path +- temporary metadata returned by data-model lookup in `hostIf_GetReqHandler()` is explicitly freed after use +- singleton handler instances persist for daemon lifetime and are not recreated per request + +One practical implication is that new handler code should avoid hidden allocations on hot paths unless the matching cleanup is obvious and local. + +## Error Handling + +Error handling in the handlers layer is intentionally transport-neutral: + +- dispatch APIs return integer status codes such as `OK` or `NOK` +- the concrete handler is responsible for setting any TR-069 fault information in `HOSTIF_MsgData_t` +- unsupported or unconfigured parameter prefixes result in a null manager lookup and a failed operation +- invalid data-model parameters are detected early in the IARM GET path before dispatch continues +- exceptions in `hostIf_GetMsgHandler()` are caught and logged to keep the daemon alive + +## Performance Notes + +This layer is not where most hardware interaction happens, but it still affects end-to-end latency. + +Important characteristics from the implementation: + +- manager lookup performs prefix scanning over the configured key set rather than direct trie-style routing +- GET and SET are serialized by dedicated mutexes, so long-running handlers can delay other requests of the same type +- request duration is measured and logged in microseconds +- requests slower than five seconds trigger explicit debug logging and optional telemetry reporting + +If request volume or latency becomes a problem, the first place to inspect is the combination of serialized dispatch and profile-specific blocking operations. + +## Testing and Validation + +When changing code in this directory, validate both routing and behavior: + +1. confirm the target parameter prefix is present in the active manager configuration +2. verify the expected build flag includes the relevant handler source +3. exercise GET, SET, and attribute paths through the transport that owns the issue +4. verify update callbacks still emit IARM and Parodus notifications when applicable +5. run the repo’s unit-test or integration workflow that covers the affected profile + +The most relevant follow-on validation usually lives outside this directory because the underlying parameter logic is in the profile implementation. + +## Known Issues and Gaps + +The following implementation gaps were identified by reviewing the source files in `src/hostif/handlers/src/`. Each entry records the severity, the affected file and approximate line, the problem, and the recommended fix. + +### Gap 1 — High: `_GetAttributestr69HostIfMgr` dispatches SET instead of GET + +**File**: `src/hostif/handlers/src/hostIf_IARM_ReqHandler.cpp` + +**Observation**: `hostIf_GetAttributesReqHandler()` is the function registered as the IARM GET-attributes entry point. Its body calls `hostIf_SetAttributesMsgHandler(stMsgData)` instead of `hostIf_GetAttributesMsgHandler(stMsgData)`. Every IARM GET-attributes RPC call therefore silently executes a SET-attributes operation instead. + +**Impact**: Attribute reads through IARM return SET semantics. Any client expecting to read notification or access attributes will instead trigger an unintended write. This is a copy-paste regression. + +**Recommended fix**: +```cpp +// In hostIf_GetAttributesReqHandler() — change: +ret = hostIf_SetAttributesMsgHandler(stMsgData); // wrong +// to: +ret = hostIf_GetAttributesMsgHandler(stMsgData); // correct +``` + +--- + +### Gap 2 — High: `mgrName` not reset between config file iterations + +**File**: `src/hostif/handlers/src/hostIf_msgHandler.cpp` — `hostIf_initalize_ConfigManger()` and `hostIf_ConfigProperties_Init()` + +**Observation**: In `hostIf_initalize_ConfigManger()`, the local variable `mgrName` is never reassigned to `HOSTIF_INVALID_Mgr` at the start of each `while` iteration. The if-else chain that identifies the manager token has no final `else` branch to reset `mgrName` on an unrecognized token. If a configuration line contains an unknown manager name, `mgrName` retains the value from the previous iteration and the guard `if(mgrName != HOSTIF_INVALID_Mgr)` passes, inserting the wrong manager for that prefix. `hostIf_ConfigProperties_Init()` has the identical problem. + +**Impact**: A typo or unknown manager name in the configuration file silently associates the preceding iteration's manager with a parameter prefix. Requests for that prefix are routed to the wrong handler with no log indication at runtime. + +**Recommended fix** — add a reset at the top of each loop body: +```cpp +while (fscanf(fp, "%99s %15s", param, mgr) != EOF) +{ + mgrName = HOSTIF_INVALID_Mgr; // reset every iteration + if (strcasecmp(mgr, "deviceMgr") == 0) + mgrName = HOSTIF_DeviceMgr; + // ... rest of if-else chain ... +``` + +--- + +### Gap 3 — High: `hostIf_SetMsgHandler()` lacks an exception handler + +**File**: `src/hostif/handlers/src/hostIf_msgHandler.cpp` + +**Observation**: `hostIf_GetMsgHandler()` wraps `pMsgHandler->handleGetMsg()` in a `try/catch(std::exception&)` block. `hostIf_SetMsgHandler()` does not. An uncaught exception thrown by any SET handler propagates through the IARM callback and crashes the daemon. + +**Impact**: Any C++ exception thrown during a SET operation — including those from profile code or vendor-supplied handlers — terminates the daemon rather than returning an error. The asymmetry is particularly visible in that GET is protected while the equally common SET path is not. + +**Recommended fix** — mirror the GET exception guard: +```cpp +try +{ + msgHandler *pMsgHandler = HostIf_GetMgr(stMsgData); + if (pMsgHandler) + ret = pMsgHandler->handleSetMsg(stMsgData); +} +catch (const std::exception& e) +{ + RDK_LOG(RDK_LOG_ERROR, LOG_TR69HOSTIF, + "[%s:%d] Exception caught %s\n", __FUNCTION__, __LINE__, e.what()); +} +``` + +--- + +### Gap 4 — Medium: Hash table created with mismatched hash and equality functions + +**File**: `src/hostif/handlers/src/hostIf_msgHandler.cpp` — lines 413 and 644 + +**Observation**: Both `hostIf_initalize_ConfigManger()` and `hostIf_ConfigProperties_Init()` create `paramMgrhash` with: + +```c +paramMgrhash = g_hash_table_new(g_str_hash, g_int_equal); +``` + +`g_str_hash` computes a hash from the contents of a string, but `g_int_equal` compares keys by pointer identity rather than string content. GLib requires hash and equality functions to be consistent: two keys that compare as equal must produce the same hash. Using string hashing with pointer equality breaks this contract. A direct `g_hash_table_lookup()` with a newly constructed string would hash into the correct bucket but never find the entry because pointer comparison would fail. + +**Impact**: `HostIf_GetMgr()` cannot rely on hash-table lookups at all. The current workaround calls `g_hash_table_get_keys()` and performs an O(n) linear prefix scan for every parameter dispatch, completely defeating the purpose of the hash table and causing performance degradation proportional to the number of configured prefixes. + +**Recommended fix** — use consistent pair: +```c +// String keys, string equality (correct): +paramMgrhash = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, NULL); +``` +With matching `g_str_equal`, `g_hash_table_lookup()` would work correctly and `HostIf_GetMgr()` could be simplified to a direct lookup once prefix matching is also resolved. + +--- + +### Gap 5 — Medium: `g_error_free(NULL)` called in the success path + +**File**: `src/hostif/handlers/src/hostIf_msgHandler.cpp` — `hostIf_ConfigProperties_Init()` + +**Observation**: The function unconditionally calls `g_error_free(error)` at the end of its body. When `g_key_file_load_from_file` succeeds, `error` is not set and remains `NULL`. `g_error_free()` requires a non-NULL pointer; providing `NULL` triggers `g_return_if_fail(error != NULL)`, which logs a critical warning in debug builds and is undefined behavior in strict GLib configurations. + +**Impact**: Every successful startup produces a spurious GLib critical warning in debug builds. On platforms where GLib critical warnings are treated as fatal, this causes a crash on normal daemon startup. + +**Recommended fix**: +```cpp +if (error) + g_error_free(error); +``` + +--- + +### Gap 6 — Low: `updateHandler` polling loop uses non-interruptible `sleep(60)` + +**File**: `src/hostif/handlers/src/hostIf_updateHandler.cpp` + +**Observation**: The polling loop body ends with `sleep(60)` and checks `stopped` only at the top of the loop. Calling `updateHandler::stop()` during daemon shutdown does not wake the sleeping thread; the thread takes up to 60 seconds to observe the flag and exit. + +**Impact**: Daemon shutdown is delayed by up to 60 seconds whenever the update thread is mid-sleep. This can cause systemd to exceed its `TimeoutStopSec` and forcibly terminate the process. + +**Recommended fix** — replace `sleep(60)` with a condition variable timed wait: +```cpp +std::unique_lock lk(stopMutex); +stopCv.wait_for(lk, std::chrono::seconds(60), []{ return stopped; }); +``` +where `stopMutex` and `stopCv` are class-level synchronization primitives and `stop()` signals the condition variable. + +--- + +### Gap 7 — Low: `hostIf_initalize_ConfigManger()` calls `exit()` on file open failure + +**File**: `src/hostif/handlers/src/hostIf_msgHandler.cpp` — `hostIf_initalize_ConfigManger()` + +**Observation**: When `fopen(argList.confFile, "r")` fails, the function sets `bVal = false` and then immediately calls `exit(EXIT_FAILURE)`. Calling `exit()` from a library-level initialization function bypasses any cleanup registered with `atexit()` in `hostIf_main.cpp` and prevents the main thread from logging a controlled shutdown or performing resource teardown. + +**Impact**: On configuration errors the daemon terminates abruptly rather than logging a meaningful message through the main-thread shutdown path. Platform supervisors (systemd) may not receive a clean exit code. + +**Recommended fix** — return the error to the caller: +```cpp +if (fp == NULL) +{ + RDK_LOG(RDK_LOG_ERROR, LOG_TR69HOSTIF, + "[%s:%s] Error opening %s\n", __FILE__, __FUNCTION__, argList.confFile); + return false; // let the caller decide how to handle it +} +``` + +--- + +### Gap Summary Table + +| # | Severity | File | Problem | Impact | +|---|----------|------|---------|--------| +| 1 | High | `hostIf_IARM_ReqHandler.cpp` | `_GetAttributestr69HostIfMgr` calls SET instead of GET attributes handler | All IARM attribute reads silently become writes | +| 2 | High | `hostIf_msgHandler.cpp` | `mgrName` not reset per config iteration; unknown tokens inherit previous manager | Silent wrong-manager routing for misconfigured prefixes | +| 3 | High | `hostIf_msgHandler.cpp` | `hostIf_SetMsgHandler()` has no exception handler | Uncaught exception from any SET handler crashes the daemon | +| 4 | Medium | `hostIf_msgHandler.cpp` | `g_str_hash` + `g_int_equal` mismatch makes hash lookup unreliable | O(n) linear scan used instead of O(1) hash lookup per request | +| 5 | Medium | `hostIf_msgHandler.cpp` | `g_error_free(NULL)` called in success path of `hostIf_ConfigProperties_Init()` | GLib critical warning on every normal startup; potential abort in debug builds | +| 6 | Low | `hostIf_updateHandler.cpp` | Non-interruptible `sleep(60)` in the update loop | Daemon shutdown delayed up to 60 seconds | +| 7 | Low | `hostIf_msgHandler.cpp` | `exit()` called from config-loading function on file open failure | Abrupt termination with no main-thread cleanup | + +--- + +## See Also + +- `src/hostif/profiles/` for the parameter-specific business logic invoked by these handlers +- `src/hostif/include/hostIf_tr69ReqHandler.h` for `HOSTIF_MsgData_t` and shared request types +- `docs/architecture/overview.md` for the daemon-wide component map +- `docs/architecture/threading-model.md` for the broader runtime thread model +- `docs/api/public-api.md` for shared request and dispatcher interfaces \ No newline at end of file diff --git a/src/hostif/httpserver/docs/README.md b/src/hostif/httpserver/docs/README.md new file mode 100644 index 000000000..838ff8143 --- /dev/null +++ b/src/hostif/httpserver/docs/README.md @@ -0,0 +1,504 @@ +# HTTP Server Implementation Overview + +## Overview + +The `src/hostif/httpserver/` module implements the newer local HTTP server used by tr69hostif to process TR-181 GET and SET requests over an HTTP JSON interface. It is separate from the older JSON server in `src/hostif/handlers/` and is started only when the build includes the new server and legacy RFC mode is not enabled. + +At runtime, this module accepts HTTP requests through libsoup, parses WDMP-style JSON payloads, validates parameters against the loaded TR-181 data model, invokes the common hostif dispatcher, and converts the results back into WDMP JSON responses. It also includes a small RFC variable cache used for temporary handling of `RFC_*` keys that are intentionally outside the data model. + +## Source Layout + +| Path | Purpose | +|------|---------| +| `src/hostif/httpserver/src/http_server.cpp` | libsoup server lifecycle, request entry point, readiness signaling | +| `src/hostif/httpserver/src/request_handler.cpp` | request validation, WDMP-to-hostif conversion, hostif invocation, response building | +| `src/hostif/httpserver/src/XrdkCentralComRFCVar.cpp` | RFC variable file discovery and in-memory cache | +| `src/hostif/httpserver/include/http_server.h` | public start/stop APIs for the server thread | +| `src/hostif/httpserver/include/request_handler.h` | request handling API exported to the HTTP server layer | +| `src/hostif/httpserver/src/gtest/gtest_httpserver.cpp` | unit tests for datatype conversion, RFC var store, request validation, and request handling helpers | + +## Architecture + +This module is split into three layers: + +1. server lifecycle and socket binding in `http_server.cpp` +2. request translation and hostif dispatch in `request_handler.cpp` +3. RFC variable cache support in `XrdkCentralComRFCVar.cpp` + +The implementation depends on: + +- libsoup 3 for the embedded HTTP listener +- WDMP request/response helpers for JSON request parsing and response formatting +- the hostif dispatcher in `src/hostif/handlers/` +- the loaded TR-181 data model for validation and wildcard expansion +- `waldb` and related WebPA support libraries already used elsewhere in tr69hostif + +### Component Diagram + +```mermaid +graph TB + subgraph Daemon[tr69hostif daemon] + MAIN[hostIf_main.cpp] + READY[httpServerThreadDone +condition variable] + end + + subgraph HTTPServer[src/hostif/httpserver] + SERVER[http_server.cpp] + HANDLER[HTTPRequestHandler] + REQ[request_handler.cpp] + RFCVAR[XRFCVarStore] + end + + subgraph Core[src/hostif core] + DM[TR-181 Data Model] + MSG[hostIf_GetMsgHandler / +hostIf_SetMsgHandler] + end + + CLIENT[Local HTTP client] --> SERVER + MAIN --> SERVER + SERVER --> HANDLER + HANDLER --> REQ + REQ --> DM + REQ --> MSG + REQ --> RFCVAR + SERVER --> READY + READY --> MAIN +``` + +## Build and Enablement + +The module is built as `libhttpserver.la` from: + +- `src/http_server.cpp` +- `src/request_handler.cpp` +- `src/XrdkCentralComRFCVar.cpp` + +The module links against: + +- `libMsgHandlers.la` +- `libwaldb.la` +- `wdmp-c` +- `libsoup-3.0` +- `cJSON` + +Runtime enablement is controlled in `hostIf_main.cpp`: + +- when `NEW_HTTP_SERVER_DISABLE` is defined, this module is not part of the active startup path +- when `/opt/RFC/.RFC_LegacyRFCEnabled.ini` exists, the daemon treats legacy RFC mode as enabled and does not start the new HTTP server thread +- otherwise `HTTPServerStartThread()` is launched on a dedicated GLib thread named `http_server_thread` + +## How Server Operation Happens + +The server operation in tr69hostif follows a fixed sequence from daemon startup to request completion. + +### Startup sequence + +```mermaid +sequenceDiagram + participant Main as hostIf_main.cpp + participant Thread as http_server_thread + participant Server as libsoup server + participant Ready as readiness signaling + + Main->>Main: Read legacy RFC flag + Main->>Thread: g_thread_try_new(HTTPServerStartThread) + Thread->>Thread: checkDataModelStatus() + Thread->>Server: soup_server_new() + Thread->>Server: soup_server_add_handler("/", HTTPRequestHandler) + Thread->>Server: soup_server_listen_local(httpServerPort) + Thread->>Thread: create /tmp/.tr69hostif_http_server_ready + Thread->>Ready: set httpServerThreadDone=true + Ready-->>Main: notify condition variable + Main->>Main: continue sd_notify READY path +``` + +The important operational points are: + +- the server does not bind until the data model is confirmed ready +- the handler is registered only on `/` +- the thread writes `/tmp/.tr69hostif_http_server_ready` so RFC-related paths can detect readiness externally +- the daemon waits up to 10 seconds on `cv_httpServerThreadDone` before sending its systemd readiness notification + +### Request processing sequence + +```mermaid +sequenceDiagram + participant Client as HTTP client + participant Soup as HTTPRequestHandler + participant WDMP as WDMP parsers + participant Req as handleRequest() + participant DM as Data-model validation + participant HostIf as hostIf dispatcher + + Client->>Soup: GET or POST with JSON body + Soup->>Soup: Read CallerID header + Soup->>WDMP: parse_get_request() / parse_set_request() + Soup->>Req: handleRequest(pcCallerID, reqSt) + Req->>DM: validateAgainstDataModel() + Req->>HostIf: hostIf_GetMsgHandler() / hostIf_SetMsgHandler() + HostIf-->>Req: result + fault code + Req-->>Soup: res_struct + Soup->>WDMP: wdmp_form_get_response() / wdmp_form_set_response() + Soup-->>Client: JSON response with statusCode +``` + +The per-request flow works like this: + +1. `HTTPRequestHandler()` receives the libsoup request. +2. It rejects empty request bodies with `400 Bad Request`. +3. It reads the `CallerID` header. +4. It parses the JSON body using WDMP helpers into a `req_struct`. +5. It calls `handleRequest()` for GET or POST processing. +6. It converts the `res_struct` into WDMP JSON. +7. It rewrites the top-level `statusCode` field so it reflects the first real parameter error instead of the generic WDMP default. +8. It sends the final JSON response with `SOUP_STATUS_OK` when the request was processed. + +### Supported HTTP methods + +The module currently recognizes: + +- `GET` for parameter retrieval +- `POST` for parameter updates + +Operational details: + +- `GET` is allowed even when the `CallerID` header is missing; the caller is logged as `Unknown` +- `POST` is rejected when `CallerID` is missing +- methods other than `GET` and `POST` return `501 Not Implemented` + +## Key Components + +### `http_server.cpp` + +This file owns the embedded server instance and is the only module that directly talks to libsoup. + +Its responsibilities are: + +- create the `SoupServer` +- register `HTTPRequestHandler()` on `/` +- listen on `argList.httpServerPort` +- check data-model readiness before binding +- create the readiness marker file in `/tmp` +- synchronize startup with the main daemon using `mtx_httpServerThreadDone`, `cv_httpServerThreadDone`, and `httpServerThreadDone` +- stop the listener through `HttpServerStop()` by disconnecting the server + +### `HTTPRequestHandler()` + +This is the server’s top-level request callback. It operates as the HTTP boundary adapter for the module. + +It performs: + +- raw request-body validation +- `CallerID` extraction from request headers +- JSON parsing with cJSON +- method-based request parsing using WDMP helpers +- dispatch to `handleRequest()` +- response formatting back into JSON +- per-request timing logs using `getCurrentTime()` and `timeValDiff()` + +### `request_handler.cpp` + +This file contains almost all request semantics. It bridges the HTTP/WDMP shape of the request to the internal `HOSTIF_MsgData_t` model used by the rest of tr69hostif. + +Important helper functions include: + +- `getWdmpDataType()` converts data-model strings such as `string`, `boolean`, and `unsignedInt` into WDMP datatypes +- `getHostIfParamType()` maps WDMP datatypes into `HostIf_ParamType_t` +- `convertAndAssignParamValue()` writes SET values into `HOSTIF_MsgData_t.paramValue` using the internal representation expected by hostif +- `getStringValue()` converts hostif values back into string form for WDMP output +- `validateParamValue()` verifies that incoming SET values match the expected datatype +- `validateAgainstDataModel()` checks existence, access mode, datatype, default value, and bootstrap-update behavior using the merged TR-181 data model +- `invokeHostIfAPI()` calls `hostIf_GetMsgHandler()` or `hostIf_SetMsgHandler()` after building a hostif request envelope +- `handleRFCRequest()` provides the special path for raw `RFC_*` variables that are not represented in the data model + +### `handleRequest()` + +`handleRequest()` is the request engine for the module. + +For GET requests it: + +- allocates a `res_struct` +- handles individual parameter names and wildcard parameter names +- validates normal parameters against the data model +- expands wildcard requests into child parameter names using the data-model API +- invokes hostif for each resolved parameter +- falls back to the parameter’s default value when hostif returns no value but the data model defines one +- supports temporary RFC-variable GET access for names that start with `RFC_` and do not contain `.` + +For SET requests it: + +- rejects unauthorized writes such as `Device.X_CISCO_COM_DeviceControl.RebootDevice` +- rejects wildcard SET operations +- rejects null values +- validates the request against the data model’s access rules and datatype rules +- invokes hostif for normal TR-181 parameters +- routes raw `RFC_*` variables to `handleRFCRequest()` instead of hostif + +## RFC Variable Handling + +`XrdkCentralComRFCVar.cpp` implements `XRFCVarStore`, a small cache for legacy RFC variable access. + +Operational behavior: + +- it reads `/etc/rfc.properties` +- it looks for the `RFC_VAR_FILENAME` property +- it strips quotes from the configured filename +- it loads key/value pairs from that file into an in-memory `unordered_map` +- it serves GET requests for `RFC_*` keys outside the data model +- it supports a cache reload operation through `XRFC_VAR_STORE_RELOADCACHE` + +This path exists because some RFC variables are handled outside the main TR-181 data-model validation path. + +## Threading Model + +This module is simple from a concurrency perspective. + +| Thread or Context | Purpose | +|-------------------|---------| +| `http_server_thread` | Creates and binds the new HTTP server, then serves incoming requests through libsoup callbacks | +| main daemon thread | Starts the HTTP server thread, waits for readiness, and stops the server during graceful shutdown | +| libsoup request callback context | Executes `HTTPRequestHandler()` for each incoming request | + +### Synchronization primitives + +The module uses: + +- `std::mutex mtx_httpServerThreadDone` +- `std::condition_variable cv_httpServerThreadDone` +- `bool httpServerThreadDone` + +These are not used for request serialization. They are used only to coordinate startup readiness between the HTTP server thread and the daemon main thread. + +## Memory Management + +The module uses manual allocation for WDMP request and response structures, so the cleanup path matters. + +Key ownership rules visible in the code are: + +- `HTTPRequestHandler()` allocates `req_struct` and frees it with `wdmp_free_req_struct()` +- `handleRequest()` allocates `res_struct` members and they are later freed with `wdmp_free_res_struct()` +- parameter names and values are duplicated with `strdup()` when building WDMP responses +- wildcard expansion allocates arrays for child parameter names and datatypes, then frees the temporary arrays after response structures are built +- `invokeHostIfAPI()` allocates string output values for WDMP using `getStringValue()` +- `XRFCVarStore` owns its in-memory map for the process lifetime + +The main thing to preserve when modifying this code is symmetry between WDMP allocation helpers, `strdup()` ownership, and the corresponding free calls in the success and error paths. + +## Error Handling + +The module distinguishes between HTTP transport errors and parameter-processing errors. + +Transport-level errors: + +- empty body results in `400 Bad Request` +- malformed JSON results in `400 Bad Request` +- unsupported method results in `501 Not Implemented` +- missing `CallerID` on POST results in an internal-server-style rejection in the current code path + +Parameter-level errors: + +- invalid parameter name becomes `WDMP_ERR_INVALID_PARAMETER_NAME` +- read-only parameter SET becomes `WDMP_ERR_NOT_WRITABLE` +- datatype mismatch becomes `WDMP_ERR_INVALID_PARAMETER_TYPE` +- invalid parameter value becomes `WDMP_ERR_INVALID_PARAMETER_VALUE` +- wildcard SET becomes `WDMP_ERR_WILDCARD_NOT_SUPPORTED` +- empty results may become `WDMP_ERR_VALUE_IS_EMPTY` unless a default value is available + +One implementation detail worth keeping in mind is that the response formatter first creates a generic WDMP response, then `HTTPRequestHandler()` patches the top-level `statusCode` so the returned HTTP JSON better reflects the actual first parameter failure. + +## Performance Notes + +The server is lightweight, but a few behaviors are important operationally: + +- request execution time is measured and logged in `HTTPRequestHandler()` +- wildcard GETs can expand into many child parameters and therefore multiply hostif calls +- each request performs data-model validation before invoking hostif +- GET and SET requests still inherit the serialization behavior of the shared hostif dispatcher once they reach `hostIf_GetMsgHandler()` or `hostIf_SetMsgHandler()` + +This means the module is usually not CPU-heavy on its own; latency is dominated by wildcard expansion, data-model lookups, and downstream profile handlers. + +## Testing + +The module has dedicated unit tests in `src/hostif/httpserver/src/gtest/gtest_httpserver.cpp` covering: + +- RFC variable filename discovery and cache loading +- datatype conversion helpers +- parameter value validation +- RFC request handling +- hostif invocation helpers +- HTTP request handler exposure in test builds + +When modifying this module, validate: + +1. server startup and readiness behavior +2. GET and POST request handling +3. wildcard GET expansion +4. RFC variable GET and reload-cache behavior +5. graceful shutdown through `HttpServerStop()` + +## Platform Notes + +- The implementation uses libsoup 3 and GLib threading primitives. +- The server listens on the port stored in `argList.httpServerPort`. +- The module is intended for local management integration inside tr69hostif, not as a general-purpose external web service. +- Runtime behavior depends on whether legacy RFC mode is active and whether the build disables the new HTTP server entirely. + +## Known Issues and Gaps + +The following implementation gaps were identified by reviewing the source files in `src/hostif/httpserver/src/`. Each entry records the severity, the affected file and line range, the problem, and the recommended fix. + +### Gap 1 — High: False readiness signaling when server fails to bind + +**File**: `src/hostif/httpserver/src/http_server.cpp` — lines 257–275 + +**Observation**: The conditional at line 257 checks the return value of `soup_server_listen_local()` and logs an error if the call fails, but the code falls through without returning or setting an error status. Execution continues to create `/tmp/.tr69hostif_http_server_ready` (line 263), log "Started server successfully" (line 269), set `httpServerThreadDone = true` (line 274), and signal `cv_httpServerThreadDone`. The main thread wakes, sees `httpServerThreadDone == true`, and sends `READY=1` to systemd. + +```cpp +// Current code — no return or error path after listen failure: +if(FALSE == soup_server_listen_local(http_server, httpServerPort, ...)) +{ + RDK_LOG(..., "SERVER: failed in soup_server_listen_local..."); + // falls through — does NOT return +} +// readiness file is created and condition is signalled regardless +``` + +**Impact**: When the port is already in use or the listener fails for any other reason, the daemon signals systemd that it is ready, RFC-facing processes detect the readiness sentinel file, and clients send requests to a socket that is not listening. The failure is invisible from the outside. + +**Recommended fix** — return (and do not signal readiness) on listen failure: +```cpp +if(FALSE == soup_server_listen_local(http_server, httpServerPort, ..., &error)) +{ + RDK_LOG(RDK_LOG_ERROR, ..., "failed: %s", error->message); + g_error_free(error); + // Signal readiness with failure so the main thread can handle it: + std::unique_lock lck(mtx_httpServerThreadDone); + httpServerThreadDone = true; // or use a separate error flag + cv_httpServerThreadDone.notify_all(); + return NULL; +} +``` + +--- + +### Gap 2 — High: `rfcParam` flag not reset between loop iterations + +**File**: `src/hostif/httpserver/src/request_handler.cpp` — `handleRequest()` + +**Observation**: The variable `rfcParam` is declared once before the `switch` statement and is set to `true` inside both the GET and SET loops when a parameter name starts with `RFC_` and contains no `.`. There is no `rfcParam = false` statement at the start of each iteration. Once `rfcParam` is `true`, all subsequent parameters in the same multi-parameter request are incorrectly routed through `handleRFCRequest()` even when they are ordinary TR-181 parameters. + +```cpp +bool rfcParam = false; // declared once, outside the loop +// ... +for (paramIndex = 0; ...) { + if (strncmp(..., "RFC_", 4) == 0 ...) { + rfcParam = true; // set here, never cleared + } + // rfcParam stays true for paramIndex+1, paramIndex+2, ... + if (!rfcParam) + invokeHostIfAPI(...); + else + handleRFCRequest(...); +} +``` + +**Impact**: A multi-parameter GET or SET request that contains even one `RFC_*` key will misroute all TR-181 parameters that follow it in the list. Those parameters are sent to the RFC file-cache path, which will not find them, and `WDMP_ERR_INVALID_PARAMETER_NAME` or an empty value is returned. The bug affects any client that batches RFC keys together with regular TR-181 parameters. + +**Recommended fix** — reset `rfcParam` at the top of each iteration: +```cpp +for (paramIndex = 0; paramIndex < respSt->paramCnt; paramIndex++) +{ + rfcParam = false; // reset per iteration + // rest of the loop body unchanged +``` + +--- + +### Gap 3 — Medium: `WDMP_ULONG` mapped to `hostIf_UnsignedIntType` instead of `hostIf_UnsignedLongType` + +**File**: `src/hostif/httpserver/src/request_handler.cpp` — `getHostIfParamType()` + +**Observation**: The switch in `getHostIfParamType()` groups `WDMP_UINT` and `WDMP_ULONG` in the same case: + +```cpp +case WDMP_UINT: +case WDMP_ULONG: + hostIfDataType = hostIf_UnsignedIntType; // wrong for WDMP_ULONG + break; +``` + +`WDMP_ULONG` should map to `hostIf_UnsignedLongType`, which is the 64-bit-capable type in the hostif layer. + +**Impact**: Parameters declared as `unsignedLong` in the TR-181 data model that carry values above 4,294,967,295 (2³²−1) are silently truncated when read back through the HTTP server. No error is returned. Counters such as interface byte counters and large capacity values are affected. + +**Recommended fix**: +```cpp +case WDMP_UINT: + hostIfDataType = hostIf_UnsignedIntType; + break; +case WDMP_ULONG: + hostIfDataType = hostIf_UnsignedLongType; + break; +``` + +--- + +### Gap 4 — Medium: `validateParamValue()` accesses string characters before checking for empty input + +**File**: `src/hostif/httpserver/src/request_handler.cpp` — `validateParamValue()` + +**Observation**: For `hostIf_IntegerType`, the function accesses `paramValue[0]` and `paramValue[1]` directly: + +```cpp +case hostIf_IntegerType: + if (isdigit(paramValue[0]) || + (paramValue[0] == '-' && isdigit(paramValue[1]))) +``` + +For `hostIf_UnsignedIntType` and `hostIf_UnsignedLongType`, `paramValue[0]` is accessed: +```cpp +case hostIf_UnsignedIntType: +case hostIf_UnsignedLongType: + if (isdigit(paramValue[0])) +``` + +Neither branch checks `paramValue.empty()` first. A caller that sends an empty string value for a numeric parameter triggers undefined behavior through `std::string::operator[]` on an empty string, followed by `isdigit` on an indeterminate character. + +**Impact**: An empty `value` field in a SET request for any numeric parameter can cause a read at `paramValue[0]` that returns an implementation-defined value. On platforms where `std::string::operator[]("")` silently returns the null terminator, the validation returns `false` as expected, but the code path remains exploitable for denial-of-service via malformed requests. + +**Recommended fix** — add an early empty-string guard: +```cpp +case hostIf_IntegerType: + if (paramValue.empty()) { ret = false; break; } + if (isdigit(paramValue[0]) || + (paramValue[0] == '-' && paramValue.length() > 1 && isdigit(paramValue[1]))) + // ... +case hostIf_UnsignedIntType: +case hostIf_UnsignedLongType: + if (paramValue.empty()) { ret = false; break; } + if (isdigit(paramValue[0])) + // ... +``` + +--- + +### Gap Summary Table + +| # | Severity | File | Problem | Impact | +|---|----------|------|---------|--------| +| 1 | High | `http_server.cpp` | Listen failure falls through to false readiness signal | Daemon reports READY=1 to systemd with no active listener | +| 2 | High | `request_handler.cpp` | `rfcParam` flag not reset per loop iteration | Subsequent TR-181 parameters in a batch are misrouted to RFC cache | +| 3 | Medium | `request_handler.cpp` | `WDMP_ULONG` maps to `hostIf_UnsignedIntType` | 64-bit parameter values silently truncated to 32 bits | +| 4 | Medium | `request_handler.cpp` | `validateParamValue()` indexes into string before checking empty | Undefined behavior for empty numeric parameter values in SET requests | + +--- + +## See Also + +- `src/hostif/src/hostIf_main.cpp` for daemon startup, legacy RFC gating, and shutdown integration +- `src/hostif/handlers/include/hostIf_msgHandler.h` for the common dispatcher interface used by this module +- `src/hostif/handlers/docs/README.md` for the handlers-layer overview that this module ultimately calls into +- `docs/architecture/overview.md` for daemon-wide component relationships +- `docs/api/public-api.md` for shared request envelope context \ No newline at end of file diff --git a/src/hostif/parodusClient/docs/README.md b/src/hostif/parodusClient/docs/README.md new file mode 100644 index 000000000..30ce4c1eb --- /dev/null +++ b/src/hostif/parodusClient/docs/README.md @@ -0,0 +1,407 @@ +# Parodus Client Implementation Overview + +## Overview + +The `src/hostif/parodusClient/` module is the WebPA and Parodus integration layer for tr69hostif. It connects the daemon to the local Parodus broker, receives WRP requests from WebPA, translates those requests into the internal hostif parameter model, and sends responses or value-change notifications back through Parodus. + +This module is not a standalone HTTP server. Its runtime role is a long-lived Parodus client with three main responsibilities: + +- establish and maintain the `libparodus` connection +- process incoming GET, SET, GET_ATTRIBUTES, and SET_ATTRIBUTES messages +- publish notifications generated elsewhere in tr69hostif through the Parodus event path + +It also includes: + +- a data-model helper layer under `waldb/` +- notification configuration parsing +- an auxiliary `startParodus/` bootstrap helper used to prepare Parodus launch parameters and runtime configuration + +## Source Layout + +| Path | Purpose | +|------|---------| +| `src/hostif/parodusClient/pal/libpd.cpp` | Parodus connection lifecycle, receive loop, and outbound event sending | +| `src/hostif/parodusClient/pal/webpa_adapter.cpp` | WDMP request orchestration, WebPA request dispatch, and notification callback glue | +| `src/hostif/parodusClient/pal/webpa_parameter.cpp` | GET and SET parameter handling, hostif and RBUS fallback routing | +| `src/hostif/parodusClient/pal/webpa_attribute.cpp` | GET_ATTRIBUTES and SET_ATTRIBUTES translation to hostif | +| `src/hostif/parodusClient/pal/webpa_notification.cpp` | notification source discovery and notify-list parsing | +| `src/hostif/parodusClient/waldb/waldb.cpp` | TR-181 data-model loading, wildcard expansion, and parameter metadata lookup | +| `src/hostif/parodusClient/startParodus/` | startup helper that prepares Parodus runtime configuration and environment | +| `src/hostif/parodusClient/conf/webpa_cfg.json` | Parodus URL and WebPA runtime configuration | +| `src/hostif/parodusClient/conf/notify_webpa_cfg.json` | initial notification list configuration | +| `src/hostif/parodusClient/parodus.service` | systemd service unit for Parodus | +| `src/hostif/parodusClient/parodus.path` | systemd path unit that triggers Parodus startup on route availability | +| `src/hostif/parodusClient/gtest/dm_test.cpp` | unit coverage for data-model, WebPA PAL, notification, and helper functions | + +## Architecture + +The Parodus client path is split into four layers: + +1. daemon integration from `hostIf_main.cpp` +2. Parodus connection management in `libpd.cpp` +3. WebPA request translation in `webpa_adapter.cpp`, `webpa_parameter.cpp`, and `webpa_attribute.cpp` +4. data-model support and notification support in `waldb.cpp` and `webpa_notification.cpp` + +### Component Diagram + +```mermaid +graph TB + subgraph Main[tr69hostif main daemon] + MAIN[hostIf_main.cpp] + UPD[NotificationHandler / updateHandler] + end + + subgraph ParodusClient[src/hostif/parodusClient] + LIBPD[libpd.cpp] + ADAPTER[webpa_adapter.cpp] + PARAM[webpa_parameter.cpp] + ATTR[webpa_attribute.cpp] + NOTIFY[webpa_notification.cpp] + WALDB[waldb.cpp] + end + + subgraph External[External services] + PARODUS[Parodus broker] + WEBPA[WebPA callers] + RBUS[RBUS providers] + DM[TR-181 data model] + end + + MAIN --> LIBPD + WEBPA --> PARODUS + PARODUS --> LIBPD + LIBPD --> ADAPTER + ADAPTER --> PARAM + ADAPTER --> ATTR + PARAM --> WALDB + ATTR --> WALDB + PARAM --> RBUS + WALDB --> DM + UPD --> NOTIFY + NOTIFY --> LIBPD + LIBPD --> PARODUS +``` + +## Build and Runtime Integration + +The module is organized as three subdirectories in [src/hostif/parodusClient/Makefile.am](src/hostif/parodusClient/Makefile.am): + +- `waldb` +- `pal` +- `startParodus` + +The Parodus client library itself is built in [src/hostif/parodusClient/pal/Makefile.am](src/hostif/parodusClient/pal/Makefile.am) as `libparodusclient.la` from: + +- `libpd.cpp` +- `webpa_notification.cpp` +- `webpa_parameter.cpp` +- `webpa_adapter.cpp` +- `webpa_attribute.cpp` + +It links against: + +- `libparodus` +- `libwaldb.la` +- `libMsgHandlers.la` +- `wdmp-c` +- `wrp-c` +- `cJSON` +- `pthread` + +At daemon startup, [src/hostif/src/hostIf_main.cpp](src/hostif/src/hostIf_main.cpp#L475) initializes the notification config file path and starts the Parodus initialization thread by calling `pthread_create(&parodus_init_tid, NULL, libpd_client_mgr, NULL)` when `PARODUS_ENABLE` is compiled in. + +## How Runtime Operation Happens + +The runtime path is best understood as a client loop around Parodus rather than as a socket server owned by tr69hostif. + +### Startup sequence + +```mermaid +sequenceDiagram + participant Main as hostIf_main.cpp + participant Thread as libpd_client_mgr + participant DB as waldb data model + participant Pd as connect_parodus + participant Notify as notification setup + participant Recv as parodus_receive_wait + + Main->>Thread: pthread_create(parodus_init_tid) + Thread->>Thread: create /tmp/webpa directory + Thread->>DB: checkDataModelStatus() + Thread->>Pd: connect_parodus() + Pd->>Pd: read webpa_cfg.json URLs + Pd->>Pd: libparodus_init() with retry backoff + Thread->>Notify: registerNotifyCallback() + Thread->>Notify: setInitialNotify() + Thread->>Recv: enter receive loop +``` + +Operationally this means: + +- tr69hostif starts the Parodus path after the rest of the daemon core is initialized +- the Parodus client depends on the data model already being loaded +- connection is retried with exponential backoff until `libparodus_init()` succeeds +- notification callback registration happens only after the connection attempt path completes +- once initialized, the thread remains in the receive loop until shutdown is requested + +### Request path + +```mermaid +sequenceDiagram + participant WebPA as WebPA caller + participant Parodus as Parodus broker + participant Libpd as parodus_receive_wait + participant Adapter as processRequest + participant Param as webpa_parameter/webpa_attribute + participant HostIf as hostIf dispatcher + participant RBUS as RBUS fallback + + WebPA->>Parodus: WRP request + Parodus->>Libpd: libparodus_receive() + Libpd->>Adapter: processRequest(payload, transaction_uuid) + Adapter->>Adapter: wdmp_parse_request() + Adapter->>Param: getValues/setValues/getAttributes/setAttributes + Param->>HostIf: hostIf_GetMsgHandler()/hostIf_SetMsgHandler() + Param->>RBUS: fallback if not owned by tr69hostif + Param-->>Adapter: WDMP response structures + Adapter-->>Libpd: response payload + Libpd->>Parodus: libparodus_send(response) +``` + +The receive loop in `libpd.cpp` waits for `WRP_MSG_TYPE__REQ` messages. For each request it: + +1. allocates a response WRP structure +2. passes the JSON payload to `processRequest()` +3. swaps source and destination so the reply is sent back to the original requester +4. sets content type to `application/json` +5. sends the reply using `libparodus_send()` + +### Notification path + +```mermaid +sequenceDiagram + participant Source as updateHandler or other source + participant NH as NotificationHandler + participant CB as notificationCallBack + participant Send as sendNotification + participant Pd as Parodus + + Source->>NH: queue notification payload + NH->>CB: registered callback fired + CB->>NH: pop from GAsyncQueue + CB->>Send: sendNotification(payload, source, dest) + Send->>Pd: libparodus_send(WRP event) +``` + +The module does not directly generate most value-change events. Instead, [src/hostif/handlers/src/hostIf_NotificationHandler.cpp](src/hostif/handlers/src/hostIf_NotificationHandler.cpp) queues notification work, and the Parodus client PAL sends that queued work through `notificationCallBack()` and `sendNotification()`. + +## Key Components + +### `libpd.cpp` + +This file owns the Parodus transport lifecycle. + +Its responsibilities are: + +- initialize the notify config file path through `libpd_set_notifyConfigFile()` +- start the client thread through `libpd_client_mgr()` +- load or verify data-model availability before Parodus processing begins +- compute Parodus and client URLs from configuration +- connect to Parodus using `libparodus_init()` with retry backoff +- receive WRP requests with `libparodus_receive()` +- send response and event messages through `libparodus_send()` +- stop the receive loop through `stop_parodus_recv_wait()` +- close the receiver and shut down the `libparodus` instance on exit + +### `webpa_adapter.cpp` + +This file is the request orchestration layer. It converts incoming WDMP JSON requests into the module’s internal request and response structures and selects the proper handling path. + +Important behavior includes: + +- `processRequest()` parses incoming WDMP requests +- GET requests call `getValues()` +- SET requests call `setValues()` +- GET_ATTRIBUTES requests call `getAttributes()` +- SET_ATTRIBUTES requests call `setAttributes()` +- reboot-related SET operations are annotated through `setRebootReason()` before dispatch +- the response is serialized with `wdmp_form_response()` before being returned to `libpd.cpp` + +### `webpa_parameter.cpp` + +This file handles parameter GET and SET requests. + +The operational model is: + +- load parameter metadata from the TR-181 data model when tr69hostif owns the parameter +- expand wildcard requests through `waldb.cpp` +- convert between WebPA datatypes and hostif datatypes +- call `hostIf_GetMsgHandler()` or `hostIf_SetMsgHandler()` for parameters owned by tr69hostif +- fall back to RBUS for parameters not owned by tr69hostif or when the data model is unavailable +- support lengthy payload handling for `Device.X_RDKCENTRAL-COM_T2.ReportProfiles` + +This makes `webpa_parameter.cpp` the main bridge from WebPA semantics to either the hostif dispatcher or the RBUS path. + +### `webpa_attribute.cpp` + +This file handles notification attributes. + +Current behavior: + +- GET_ATTRIBUTES reads hostif notification state through `hostIf_GetAttributesMsgHandler()` +- SET_ATTRIBUTES writes notification state through `hostIf_SetAttributesMsgHandler()` +- attribute operations are intentionally limited to parameters that appear in the configured notify list + +### `webpa_notification.cpp` + +This file manages notification configuration and notification source identity. + +Its responsibilities are: + +- remember the active notification config file path +- parse the `Notify` array from `notify_webpa_cfg.json` +- derive the notification source from `Device.DeviceInfo.X_COMCAST-COM_STB_MAC` +- normalize the MAC address into the `mac:` format used in notifications + +### `waldb.cpp` + +This file is the data-model support layer used by the Parodus client path. + +It provides: + +- `loadDataModel()` to load the merged XML data model from `/tmp/data-model.xml` +- `getParamInfoFromDataModel()` to retrieve parameter metadata +- `getChildParamNamesFromDataModel()` to expand wildcard requests +- `isWildCardParam()` helpers used by the WebPA request logic +- instance-count resolution for object tables using `NumberOfEntries` style parameters + +## Configuration and Service Assets + +### `conf/webpa_cfg.json` + +This file provides runtime defaults for: + +- `ParodusURL` +- `ParodusClientURL` +- server port and retry timing values +- JWT acquisition behavior +- device network interface selection + +### `conf/notify_webpa_cfg.json` + +This file provides the list of parameters that should have initial notification state enabled through the WebPA attribute path. + +### `parodus.service` and `parodus.path` + +These files show that Parodus itself is managed as a separate systemd unit. The path unit watches `/tmp/route_available` and starts the Parodus service when routing becomes available. That service then runs `startParodusMain`, which is implemented under `startParodus/`. + +## Threading Model + +| Thread or Context | Purpose | +|-------------------|---------| +| `parodus_init_tid` | created from `hostIf_main.cpp` to initialize the Parodus client and then enter the receive loop | +| notification callback context | sends queued notifications from `NotificationHandler` through Parodus | +| RBUS client context inside WebPA helpers | handles fallback parameter access for parameters outside tr69hostif ownership | + +### Synchronization + +The module uses only limited explicit synchronization in the PAL layer: + +- `parodus_lock` and `parodus_cond` are used in the receive loop’s timed wait path +- notification delivery relies on the GLib async queue owned by `NotificationHandler` +- most request serialization is delegated to downstream hostif handlers and libparodus behavior rather than enforced directly here + +One practical implication is that the Parodus client path is thin on internal concurrency control and therefore relies on correct ownership and sequencing in surrounding layers. + +## Memory Management + +This module allocates and frees many request, response, and notification objects manually. + +Key ownership patterns are: + +- WRP request and response structures are heap-allocated in `libpd.cpp` and released with `wrp_free_struct()` +- WDMP request and response structures are created in `webpa_adapter.cpp` and released with `wdmp_free_req_struct()` and `wdmp_free_res_struct()` +- wildcard parameter expansion allocates arrays of `param_t` and nested name/value strings in `webpa_parameter.cpp` +- notification payloads and destinations are transferred through `NotificationHandler` and freed after send or error handling +- the data-model XML document is loaded once and held for process lifetime in `waldb.cpp` + +Because the implementation uses multiple ownership conventions across WDMP, WRP, GLib, and local helpers, this module is sensitive to leaks and double-free regressions when code paths are modified. + +## Testing + +The unit test file [src/hostif/parodusClient/gtest/dm_test.cpp](src/hostif/parodusClient/gtest/dm_test.cpp) covers a broad set of helper behavior, including: + +- data-model load and parameter lookup +- datatype conversion helpers +- notification list parsing +- RFC and request helper paths +- Parodus URL handling helpers + +When changing this module, validate: + +1. Parodus connect and reconnect behavior +2. GET and SET request translation +3. wildcard parameter handling +4. notification send behavior +5. RBUS fallback behavior for non-hostif parameters + +## Known Gaps in Current Implementation + +The following issues are visible in the current code and are worth keeping in mind when debugging or extending the module. + +### 1. Initial notification enablement is effectively disabled + +In `setInitialNotify()` inside `webpa_adapter.cpp`, the local variables `notifyparameters` and `notifyListSize` are initialized but the function never calls `getnotifyparamList()` to populate them. The code then checks `if(notifyparameters != NULL)`, which is always false in the current implementation, so the initial notification list is never actually applied. + +Impact: + +- `notify_webpa_cfg.json` can be present and valid, but the initial notify-on behavior is skipped +- the logs report `Initial Notification list is empty` even when configuration exists + +### 2. Parodus URL fallback logic is incorrect + +In `get_parodus_url()` inside `libpd.cpp`, the default-value fallback uses destination-buffer lengths derived from the current contents of `parodus_url` and `client_url`, which are empty at that point. It also copies the client URL using the length of the Parodus URL string in the configured path. + +Impact: + +- fallback URLs may be copied incorrectly or not copied completely +- client URL handling can be truncated or left unterminated +- startup behavior depends more heavily on the config file being well formed than intended + +### 3. Existing `/tmp/webpa` directory is logged as an error + +In `libpd_client_mgr()`, `mkdir("/tmp/webpa", ...)` treats `EEXIST` as a failure and logs an error. `EEXIST` normally means the directory already exists and is usually harmless in this startup path. + +Impact: + +- normal restart scenarios can generate misleading error logs +- operators can be pushed toward false-positive investigation of a healthy state + +### 4. Notification source allocation has ownership and length issues + +In `getNotifySource()` inside `webpa_notification.cpp`, the code allocates `notificationSource`, then overwrites that pointer with `asprintf()`, losing the original allocation. In the failure path it also computes copy lengths using `strlen(notificationSource)` before the fallback string has been assigned. + +Impact: + +- unnecessary heap leakage on the success path +- unsafe string-length handling on the failure path +- notification source generation is more fragile than it needs to be + +### 5. SET/SET_ATTRIBUTES path allocates an unused temporary return array + +In `processRequest()` inside `webpa_adapter.cpp`, the SET and SET_ATTRIBUTES case allocates `retList` using `resObj->paramCnt` before `resObj->paramCnt` is initialized for that branch, and the array is not used for the final response path. + +Impact: + +- no direct functional benefit from the allocation +- unnecessary complexity in an already allocation-heavy path +- increased difficulty when auditing memory behavior in the SET path + +These gaps do not invalidate the overall architecture, but they are real implementation issues and should be considered when diagnosing startup, notification, or WebPA behavior. + +## See Also + +- [src/hostif/src/hostIf_main.cpp](src/hostif/src/hostIf_main.cpp) for daemon startup and Parodus thread creation +- [src/hostif/handlers/docs/README.md](src/hostif/handlers/docs/README.md) for the dispatcher layer that services many Parodus-backed requests +- [src/hostif/httpserver/docs/README.md](src/hostif/httpserver/docs/README.md) for the separate local HTTP server path +- [docs/architecture/data-flow.md](docs/architecture/data-flow.md) for daemon-wide request routing context \ No newline at end of file diff --git a/src/hostif/profiles/DHCPv4/docs/README.md b/src/hostif/profiles/DHCPv4/docs/README.md new file mode 100644 index 000000000..6341e7656 --- /dev/null +++ b/src/hostif/profiles/DHCPv4/docs/README.md @@ -0,0 +1,253 @@ +# DHCPv4 Profile + +## Overview + +The DHCPv4 profile implements the TR-181 `Device.DHCPv4.Client.{i}` object tree. It exposes the current DHCPv4 lease state — the active interface reference, DNS server list, and default gateway (IP Router) list — to TR-069 ACS and WebPA management systems. Data is derived at query time from the live kernel routing table and `/etc/resolv.conf`; no lease database file is parsed directly. + +--- + +## Directory Structure + +``` +src/hostif/profiles/DHCPv4/ +├── Device_DHCPv4_Client.h # Class declaration, enums, struct definitions +├── Device_DHCPv4_Client.cpp # GET handler implementations +├── Makefile.am # Autotools build rules +└── gtest/ + ├── gtest_dhcpv4.cpp # Unit tests + └── Makefile.am +``` + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET Device.DHCPv4.Client.i.*| DISP[hostIf_msgHandler] + DISP --> INST["hostIf_DHCPv4Client::getInstance
dev_id"] + INST --> HASH[("dhcpv4ClientHash
GHashTable")] + INST --> GET[get_Device_DHCPv4_Client_Fields] + GET -->|eDHCPv4Interface| IPIFS["hostIf_IP / hostIf_IPInterface
name match lookup"] + GET -->|eDHCPv4Dnsservers| RESOLV["/etc/resolv.conf
plus ip route get per DNS"] + GET -->|eDHCPv4Iprouters| IPROUTE[ip r grep default grep ifname] + GET -->|Count| DEFROUTE[ip r grep default wc -l] +``` + +--- + +## TR-181 Parameter Coverage + +| TR-181 Parameter | Method | Data Source | +|------------------|--------|-------------| +| `Device.DHCPv4.ClientNumberOfEntries` | GET | `ip r \| grep default \| wc -l` — counts default routes | +| `Device.DHCPv4.Client.{i}.Interface` | GET | Iterates `Device.IP.Interface.*`, matches `nameOfInterface` to OS interface name derived from dev_id | +| `Device.DHCPv4.Client.{i}.DNSServers` | GET | Parses `/etc/resolv.conf` nameserver lines; validates each with `ip route get ` per interface | +| `Device.DHCPv4.Client.{i}.IPRouters` | GET | `ip r \| grep default \| grep ` awk `$3` (gateway field) | + +> **Note**: `Enable`, `Status`, `Alias`, `IPAddress`, `SubnetMask`, `LeaseTimeRemaining`, `DHCPServer`, `RenewedTime`, `SentOptionNumberOfEntries`, and `ReqOptionNumberOfEntries` from the TR-181 specification are not implemented. There is no `handleSetMsg` — all parameters are read-only. + +--- + +## Class Design + +### `hostIf_DHCPv4Client` + +``` +class hostIf_DHCPv4Client +├── static GHashTable* dhcpv4ClientHash // dev_id → instance map +├── static GMutex* m_mutex // guards all class operations +├── static GHashTable* m_notifyHash // change-notification registry +├── static DHCPv4Client dhcpClient // SHARED state (all instances) +│ +├── DHCPv4Client backupDhcpClient // per-instance previous value +├── DHCPv4ClientParamBackUpFlag bBackUpFlags // tracks if backup is valid +│ +├── getInstance(dev_id) → instance +├── getAllInstances() → GList* +├── closeInstance() +├── closeAllInstances() +│ +├── get_Device_DHCPv4_ClientNumberOfEntries() +├── get_Device_DHCPv4_Client_InterfaceReference() +├── get_Device_DHCPv4_Client_DnsServer() +└── get_Device_DHCPv4_Client_IPRouters() +``` + +### Key Structures + +```c +typedef struct DHCPv4Client { + char interface[MAX_IF_LEN]; // 256 bytes: "Device.IP.Interface.N" + char dnsservers[MAX_DNS_SERVER_LEN]; // 256 bytes: comma-separated IPv4 list + char ipRouters[MAX_IP_ROUTER_LEN]; // 256 bytes: comma-separated IPv4 list +} DHCPv4Client; + +typedef struct DHCPv4ClientParamBackUpFlag { + unsigned int interface:1; + unsigned int dnsservers:1; + unsigned int ipRouters:1; +} DHCPv4ClientParamBackUpFlag; +``` + +--- + +## How Operations Work + +### GET Request Flow + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch as hostIf_msgHandler + participant Inst as hostIf_DHCPv4Client + participant Kernel as Kernel / /etc/resolv.conf + + ACS->>Dispatch: GET Device.DHCPv4.Client.1.DNSServers + Dispatch->>Inst: getInstance(1) + Inst->>Inst: getLock() + Inst->>Inst: get_Device_DHCPv4_Client_DnsServer() + Inst->>Inst: get_Device_DHCPv4_Client_Fields(eDHCPv4Dnsservers) + Inst->>Kernel: v_secure_popen("cat /etc/resolv.conf | grep nameserver ...") + Kernel-->>Inst: "8.8.8.8,8.8.4.4," + loop For each DNS IP + Inst->>Kernel: v_secure_popen("ip route get | grep | awk '$5'") + Kernel-->>Inst: interface name + Inst->>Inst: Compare to dev_id interface + end + Inst->>Inst: Populate dhcpClient.dnsservers + Inst->>Inst: Compare to backupDhcpClient (detect change) + Inst->>Inst: Copy to stMsgData->paramValue + Inst->>Inst: releaseLock() + Inst-->>Dispatch: OK + Dispatch-->>ACS: "8.8.8.8,8.8.4.4" +``` + +### Interface Resolution Flow + +When `get_Device_DHCPv4_Client_InterfaceReference()` is called, it: +1. Calls `getInterfaceName(ifname)` to get the OS interface name for `dev_id` +2. Calls `hostIf_IP::get_Device_IP_InterfaceNumberOfEntries()` to enumerate `Device.IP.Interface.*` +3. For each IP interface, calls `pIface->get_Interface_Name()` and compares to `ifname` +4. On match, returns `"Device.IP.Interface.N"` as a TR-181 path reference + +--- + +## Change Detection + +All three GET methods use a backup pattern for notification: + +1. If `bBackUpFlags.` is set (indicating a previous value exists) AND `pChanged != NULL`, the method calls `strncmp()` between the current and backup value +2. If they differ, `*pChanged = true` is set so the `updateHandler` can fire a WebPA notification +3. The backup is always updated to the current value after the comparison + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `v_secure_popen()` fails | Returns `NOK`; stMsgData is not populated | +| No default route found | `get_Device_DHCPv4_ClientNumberOfEntries()` returns 0 | +| No matching IP interface | `Interface` field stays empty; returns `NOK` | +| `getInterfaceName()` fails | Returns `NOK` immediately, no shell commands spawned | +| Invalid DNS IP format | `isValidIPAddr()` rejects; DNS entry skipped | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: `dhcpClient` is a class-level static shared by all instances + +**File**: `Device_DHCPv4_Client.h` / `Device_DHCPv4_Client.cpp` + +**Observation**: The data structure `dhcpClient` (of type `DHCPv4Client`) is declared `static`: + +```cpp +static DHCPv4Client dhcpClient; +``` + +All `hostIf_DHCPv4Client` instances (dev_id 1, 2, 3, …) write to the same `dhcpClient` structure during `get_Device_DHCPv4_Client_Fields()`. When two manager instances call GET concurrently, one will overwrite the other's pending result. + +**Impact**: On a multi-interface device, concurrent GET requests for different DHCPv4 client instances return corrupted or crossed field values. + +**Recommended fix**: Make `dhcpClient` an instance member (not static). + +--- + +### Gap 2 — High: `getLock()` lazy-initializes `m_mutex` without synchronization + +**File**: `Device_DHCPv4_Client.cpp` + +**Observation**: + +```cpp +void hostIf_DHCPv4Client::getLock() +{ + if(!m_mutex) + { + m_mutex = g_mutex_new(); + } + g_mutex_lock(m_mutex); +} +``` + +The `if(!m_mutex)` check and `g_mutex_new()` call are not atomically protected. Two threads calling `getLock()` simultaneously at startup can both observe `m_mutex == NULL` and create two separate mutexes. One mutex is stored, the other is leaked. All future locks use the stored mutex, but the initial caller's lock is on the leaked one — the critical section is left unprotected. + +**Recommended fix**: Initialize `m_mutex` at class construction time or use `g_once`. + +--- + +### Gap 3 — Medium: Only 3 of 14 TR-181 DHCPv4 Client parameters implemented + +**Observation**: TR-181 `Device.DHCPv4.Client.{i}` defines 14 parameters including `Enable`, `Status`, `Alias`, `IPAddress`, `SubnetMask`, `LeaseTimeRemaining`, `DHCPServer`, `RenewedTime`, `SentOption`, and `ReqOption`. The implementation exposes only `Interface`, `DNSServers`, and `IPRouters`, all as read-only GET parameters. Any ACS attempt to GET `IPAddress`, `Enable`, or `Status` returns `NOT_HANDLED`. + +**Impact**: ACS cannot perform full DHCPv4 diagnostics or control. Compliance with BBF TR-181 issue 2 is incomplete. + +--- + +### Gap 4 — Medium: `ClientNumberOfEntries` counts default routes, not distinct DHCP clients + +**File**: `Device_DHCPv4_Client.cpp` — `get_Device_DHCPv4_ClientNumberOfEntries()` + +**Observation**: + +```cpp +cmdOP = v_secure_popen("r", "ip r | grep default|wc -l"); +``` + +This counts the number of default routing entries, not the number of active DHCP leases. On a device with multiple static default routes or policy routing tables, this returns a count that does not correspond to the number of DHCPv4 client instances actually in `dhcpv4ClientHash`. + +**Recommended fix**: Count the keys in `dhcpv4ClientHash` or parse `/var/lib/dhclient/*.leases`. + +--- + +### Gap 5 — Low: Memory leak in constructor + +**File**: `Device_DHCPv4_Client.cpp` + +**Observation**: The constructor allocates a `FILE*` via `cmdOP` but the variable is declared and assigned `NULL` without being used in the constructor body. Reviewing the constructor, `cmdOP` is declared but never assigned a non-NULL value. This is dead code, but there is no cleanup path for any future use. + +--- + +## Testing + +Unit tests are in `gtest/gtest_dhcpv4.cpp`. Run: + +```bash +./run_ut.sh +``` + +When modifying DHCPv4 logic: +1. Verify `Interface` field correctly resolves `Device.IP.Interface.N` references. +2. Verify `DNSServers` parses multi-server entries separated by commas. +3. Verify `IPRouters` returns the gateway for the correct interface. +4. Test change detection: call GET twice with an intermediate route change in between. + +--- + +## See Also + +- [IP Profile README](../../IP/docs/README.md) — `Device.IP.Interface.{i}` used for interface resolution +- [handlers/docs/README.md](../../../handlers/docs/README.md) — Dispatch layer +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview diff --git a/src/hostif/profiles/Device/docs/README.md b/src/hostif/profiles/Device/docs/README.md new file mode 100644 index 000000000..12804af08 --- /dev/null +++ b/src/hostif/profiles/Device/docs/README.md @@ -0,0 +1,232 @@ +# Device Profile (X_RDK_profile) + +## Overview + +The Device profile implements the RDK-specific vendor extension `Device.X_RDK_*` parameter namespace. It provides GET and SET access to WebPA server URLs and WebConfig synchronization URLs that are stored and managed by the Bootstrap (`XBSStore`) subsystem. These parameters allow an ACS or WebPA controller to read and modify the management-plane endpoint configuration of the device. + +--- + +## Directory Structure + +``` +src/hostif/profiles/Device/ +├── x_rdk_profile.h # Singleton class declaration and parameter name constants +├── x_rdk_profile.cpp # GET and SET handler implementations +├── Makefile.am # Autotools build rules +└── gtest/ + ├── gtest_device.cpp # Unit tests + └── Makefile.am +``` + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET/SET Device.X_RDK_*| DISP[hostIf_msgHandler] + DISP --> INST[X_rdk_profile::getInstance] + INST --> GET[handleGetMsg] + INST --> SET[handleSetMsg] + GET --> BSSTORE["XBSStore::getValue
Bootstrap store"] + SET --> BSSTORE2["XBSStore::overrideValue
Bootstrap store"] + BSSTORE --> JSON["/etc/partners_defaults.json
or /opt/partners_defaults.json"] +``` + +--- + +## TR-181 Parameter Coverage + +| TR-181 Parameter | GET | SET | Backend | +|------------------|-----|-----|---------| +| `Device.X_RDK_WebPA_Server.URL` | ✅ | ❌ | XBSStore | +| `Device.X_RDK_WebPA_TokenServer.URL` | ✅ | ❌ | XBSStore | +| `Device.X_RDK_WebPA_DNSText.URL` | ✅ | ✅ | XBSStore | +| `Device.X_RDK_WebConfig.URL` | GET via BSStore routing | — | XBSStore | +| `Device.X_RDK_WebConfig.ForceSync` | GET via BSStore routing | — | XBSStore | + +> **Note**: WebPA Server URL and WebPA TokenServer URL support GET only. The `handleSetMsg` function only handles `X_RDK_WebPA_DNSText.URL`. Setting `X_RDK_WebPA_Server.URL` or `X_RDK_WebPA_TokenServer.URL` returns `NOT_HANDLED`. + +--- + +## Class Design + +### `X_rdk_profile` + +``` +class X_rdk_profile (singleton) +├── static X_rdk_profile* m_instance +├── static std::mutex m +├── static XBSStore* m_bsStore // Bootstrap store reference +│ +├── getInstance() → X_rdk_profile* +├── closeInstance() +│ +├── handleGetMsg(stMsgData) → int // GET dispatcher +└── handleSetMsg(stMsgData) → int // SET dispatcher +``` + +### Parameter Name Constants + +```cpp +#define X_RDK_WebPA_SERVER_URL_STPRING "Device.X_RDK_WebPA_Server.URL" +#define X_RDK_WebPA_TokenServer_URL_STRING "Device.X_RDK_WebPA_TokenServer.URL" +#define X_RDK_WebPA_DNSText_URL_STRING "Device.X_RDK_WebPA_DNSText.URL" +``` + +--- + +## How Operations Work + +### GET Request Flow + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch as hostIf_msgHandler + participant Prof as X_rdk_profile + participant BSStore as XBSStore + + ACS->>Dispatch: GET Device.X_RDK_WebPA_Server.URL + Dispatch->>Prof: handleGetMsg(stMsgData) + Prof->>Prof: strncasecmp(paramName, X_RDK_WebPA_SERVER_URL_STPRING) + Prof->>Prof: get_WebPA_Server_URL(stMsgData) + Prof->>BSStore: getValue(stMsgData) + BSStore->>BSStore: Lookup in in-memory map + BSStore-->>Prof: value string + Prof-->>Dispatch: OK + Dispatch-->>ACS: URL value +``` + +### SET Request Flow + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch as hostIf_msgHandler + participant Prof as X_rdk_profile + participant BSStore as XBSStore + + ACS->>Dispatch: SET Device.X_RDK_WebPA_DNSText.URL = "new_url" + Dispatch->>Prof: handleSetMsg(stMsgData) + Prof->>Prof: strncasecmp(paramName, X_RDK_WebPA_DNSText_URL_STRING) + Prof->>Prof: set_WebPA_DNSText_URL(stMsgData) + Prof->>BSStore: overrideValue(stMsgData) + BSStore->>BSStore: Update in-memory map and persist to disk + BSStore-->>Prof: OK + Prof-->>Dispatch: OK + Dispatch-->>ACS: success +``` + +--- + +## Backend: Bootstrap Store (XBSStore) + +All values are stored in the Bootstrap store (`XBSStore`). This store: +- Loads its initial values from a partner-specific JSON file (`partners_defaults.json`) +- Maintains an in-memory `std::map` of key-value pairs +- On `overrideValue()`, writes an updated value to `tr181store.ini` so it persists across reboots + +For more details see [DeviceInfo/docs/README.md](../../DeviceInfo/docs/README.md#xbsstore--bootstrap-store). + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `paramName == NULL` | Logs error, returns `NOK` without touching `faultCode` | +| Unknown parameter name in GET | Sets `stMsgData->faultCode = fcInvalidParameterName`, returns `NOK` | +| Unknown parameter name in SET | Sets `stMsgData->faultCode = fcInvalidParameterName`, returns `NOK` | +| `XBSStore::getValue()` key not found | Returns `NOK`; paramValue is empty | +| C++ exception thrown | Caught, logs with `e.what()`, sets `fcInternalError`, returns `NOK` | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: `handleSetMsg` only supports one of three writable parameters + +**File**: `x_rdk_profile.cpp` — `handleSetMsg()` + +**Observation**: The SET dispatcher handles only `X_RDK_WebPA_DNSText.URL`. The other two URL parameters (`X_RDK_WebPA_Server.URL` and `X_RDK_WebPA_TokenServer.URL`) are silently returned with `fcInvalidParameterName` on any SET attempt, even though the Bootstrap store can store arbitrary values. The GET for these parameters works fine. + +**Impact**: ACS cannot change the WebPA server URL via TR-069/WebPA without using a different protocol path. This asymmetry between readable and writable parameters is not documented in any TR-181 extension schema. + +--- + +### Gap 2 — Medium: `WebConfig.URL` and `WebConfig.ForceSync` not explicitly routed in this handler + +**Observation**: The `handleGetMsg` in `x_rdk_profile.cpp` does not contain routing logic for `Device.X_RDK_WebConfig.*` parameters. These pass through to `XBSStore::getValue()` indirectly via the BSStore bootstrap routing. However, there is no explicit mapping showing which parameter names are valid, making it impossible to determine supported parameters from the source code alone. + +--- + +### Gap 3 — Low: Stale file comments reference Bluetooth + +**File**: `x_rdk_profile.cpp` + +**Observation**: Both the `@file` Doxygen comment and the `@brief` Doxygen comment describe this file as handling Bluetooth device information: + +```cpp +/** + * @file X_rdk_profile.cpp + * @brief This source file contains the APIs for getting bluetooth device information. + */ +``` + +This file handles WebPA/WebConfig URL configuration. The Bluetooth implementation lives in `XrdkBlueTooth.cpp` in the `DeviceInfo/` directory. The stale comments create misleading cross-references in generated API documentation. + +--- + +### Gap 4 — Low: `getInstance()` is not thread-safe + +**File**: `x_rdk_profile.cpp` + +**Observation**: + +```cpp +X_rdk_profile* X_rdk_profile::getInstance() +{ + if(!m_instance) + { + try { + m_instance = new X_rdk_profile(); + } ... + } + return m_instance; +} +``` + +The `if(!m_instance)` check-and-create is not protected by `m` (the class-level `std::mutex`). Two threads could both observe `m_instance == nullptr` and each create an instance, with one being immediately leaked. + +**Recommended fix**: +```cpp +std::lock_guard lock(m); +if(!m_instance) { + m_instance = new X_rdk_profile(); +} +``` + +--- + +## Testing + +Unit tests are in `gtest/gtest_device.cpp`. Run: + +```bash +./run_ut.sh +``` + +When modifying this profile: +1. Verify GET returns the bootstrap store value for all three URL parameters. +2. Verify SET for `X_RDK_WebPA_DNSText.URL` persists across a simulated restart (check `tr181store.ini`). +3. Verify SET for `X_RDK_WebPA_Server.URL` returns `fcInvalidParameterName`. +4. Test `getInstance()` under concurrent access. + +--- + +## See Also + +- [DeviceInfo/docs/README.md](../../DeviceInfo/docs/README.md) — XBSStore internals, RFC store +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview +- [handlers/docs/README.md](../../../handlers/docs/README.md) — Dispatch layer diff --git a/src/hostif/profiles/DeviceInfo/Device_DeviceInfo.cpp b/src/hostif/profiles/DeviceInfo/Device_DeviceInfo.cpp index 5312d71c8..ec84367f0 100644 --- a/src/hostif/profiles/DeviceInfo/Device_DeviceInfo.cpp +++ b/src/hostif/profiles/DeviceInfo/Device_DeviceInfo.cpp @@ -5498,10 +5498,10 @@ int hostIf_DeviceInfo::get_HotelCheckoutLastResetTime(HOSTIF_MsgData_t* stMsgDat string resp = getJsonRPCData(std::move(postData)); if (resp.empty()) { - RDK_LOG (RDK_LOG_ERROR, LOG_TR69HOSTIF, "[%s] Empty outpu from Thunder call\n", __FUNCTION__); + RDK_LOG (RDK_LOG_ERROR, LOG_TR69HOSTIF, "[%s] Empty output from Thunder call\n", __FUNCTION__); return NOK; } - + RDK_LOG (RDK_LOG_INFO, LOG_TR69HOSTIF, "[%s] curl response string = %s\n", __FUNCTION__, resp.c_str()); cJSON* root = cJSON_Parse(resp.c_str()); @@ -5509,17 +5509,28 @@ int hostIf_DeviceInfo::get_HotelCheckoutLastResetTime(HOSTIF_MsgData_t* stMsgDat if(root) { cJSON* jsonObj = cJSON_GetObjectItem(root, "result"); - if (jsonObj && jsonObj->type == cJSON_Number) + if (jsonObj) { - unsigned long value = (unsigned long)jsonObj->valuedouble; - put_ulong(stMsgData->paramValue, value); - stMsgData->paramtype = hostIf_UnsignedLongType; - stMsgData->paramLen = sizeof(unsigned long); + cJSON *resetTimeObj = cJSON_GetObjectItem(jsonObj, "resetTime"); + + if (resetTimeObj && resetTimeObj->type == cJSON_Number) + { + unsigned long value = (unsigned long)resetTimeObj->valuedouble; + put_ulong(stMsgData->paramValue, value); + stMsgData->paramtype = hostIf_UnsignedLongType; + stMsgData->paramLen = sizeof(unsigned long); + } + else + { + RDK_LOG(RDK_LOG_ERROR, LOG_TR69HOSTIF, "[%s] No resetTime in the output from Thunder plugin\n", __FUNCTION__); + cJSON_Delete(root); + return NOK; + } } else { + RDK_LOG(RDK_LOG_ERROR, LOG_TR69HOSTIF, "[%s] json parse error, no \"result\" in the output from Thunder plugin\n", __FUNCTION__); cJSON_Delete(root); - RDK_LOG (RDK_LOG_ERROR, LOG_TR69HOSTIF, "[%s] JSON-RPC result missing or not a numeric value\n", __FUNCTION__); return NOK; } @@ -5554,21 +5565,35 @@ int hostIf_DeviceInfo::get_HotelCheckoutStatus(HOSTIF_MsgData_t* stMsgData) if(root) { cJSON* jsonObj = cJSON_GetObjectItem(root, "result"); - if (jsonObj && jsonObj->type == cJSON_Number) + if (jsonObj) { - unsigned long value = (unsigned long)jsonObj->valuedouble; - if (value > 0) + cJSON *resetTimeObj = cJSON_GetObjectItem(jsonObj, "resetTime"); + + if (resetTimeObj && resetTimeObj->type == cJSON_Number) { - snprintf(stMsgData->paramValue, TR69HOSTIFMGR_MAX_PARAM_LEN, "%s", "success"); + unsigned long value = (unsigned long)resetTimeObj->valuedouble; + + if (value > 0) + { + snprintf(stMsgData->paramValue, TR69HOSTIFMGR_MAX_PARAM_LEN, "%s", "success"); + } + else + { + snprintf(stMsgData->paramValue, TR69HOSTIFMGR_MAX_PARAM_LEN, "%s", "unknown"); + } } else { - snprintf(stMsgData->paramValue, TR69HOSTIFMGR_MAX_PARAM_LEN, "%s", "unknown"); + RDK_LOG (RDK_LOG_ERROR, LOG_TR69HOSTIF, "[%s] No resetTime in the output from Thunder call\n", __FUNCTION__); + cJSON_Delete(root); + return NOK; } } else { - snprintf(stMsgData->paramValue, TR69HOSTIFMGR_MAX_PARAM_LEN, "%s", "unknown"); + RDK_LOG (RDK_LOG_ERROR, LOG_TR69HOSTIF, "[%s] No result from Thunder call\n", __FUNCTION__); + cJSON_Delete(root); + return NOK; } stMsgData->paramLen = strlen(stMsgData->paramValue); @@ -5581,9 +5606,7 @@ int hostIf_DeviceInfo::get_HotelCheckoutStatus(HOSTIF_MsgData_t* stMsgData) return NOK; } - return OK; - } int hostIf_DeviceInfo::set_X_RDKCENTRAL_COM_LastRebootReason(HOSTIF_MsgData_t *stMsgData) diff --git a/src/hostif/profiles/DeviceInfo/docs/README.md b/src/hostif/profiles/DeviceInfo/docs/README.md new file mode 100644 index 000000000..b58b3ea2e --- /dev/null +++ b/src/hostif/profiles/DeviceInfo/docs/README.md @@ -0,0 +1,296 @@ +# DeviceInfo Profile + +## Overview + +The DeviceInfo profile is the largest and most complex profile in the tr69hostif daemon. It implements the entire `Device.DeviceInfo.*` object tree from TR-181 Issue 2, plus the RDK-specific `Device.DeviceInfo.X_RDKCENTRAL-COM_*` extensions. This includes manufacturer identification, software version management, memory status, process enumeration, reboot control, Bluetooth discovery/pairing, Bootstrap store (BSStore), and RFC configuration store management. + +The profile consists of ten implementation files organized around three distinct functional areas: +1. **Core DeviceInfo** — static and dynamic device attributes +2. **BSStore / RFCStore** — partner configuration and RFC override persistence +3. **Bluetooth** — `btmgr` HAL integration for BLE and classic Bluetooth + +--- + +## Directory Structure + +``` +src/hostif/profiles/DeviceInfo/ +├── Device_DeviceInfo.cpp # Core parameter handler (5,337 lines) +├── Device_DeviceInfo.h # Core class + 200+ parameter enum +├── Device_DeviceInfo_Processor.cpp # Device.DeviceInfo.Processor.{i}.* +├── Device_DeviceInfo_Processor.h +├── Device_DeviceInfo_ProcessStatus.cpp # Device.DeviceInfo.ProcessStatus.* +├── Device_DeviceInfo_ProcessStatus.h +├── Device_DeviceInfo_ProcessStatus_Process.cpp # Per-process stats +├── Device_DeviceInfo_ProcessStatus_Process.h +├── XrdkBlueTooth.cpp # X_RDKCENTRAL-COM_xBlueTooth.* +├── XrdkBlueTooth.h +├── XrdkCentralComBSStore.cpp # Bootstrap store implementation +├── XrdkCentralComBSStore.h +├── XrdkCentralComBSStoreJournal.cpp # BS store change journal +├── XrdkCentralComBSStoreJournal.h +├── XrdkCentralComRFC.cpp # RFC INI file backend +├── XrdkCentralComRFC.h +├── XrdkCentralComRFCStore.cpp # RFC store with 4 dict tiers +├── XrdkCentralComRFCStore.h +├── Makefile.am +└── gtest/ + ├── gtest_main.cpp # Comprehensive tests (4,291 lines) + └── Makefile.am +``` + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA / RBUS] -->|GET/SET Device.DeviceInfo.*| DISP[hostIf_msgHandler] + DISP --> DI["hostIf_DeviceInfo
handleGetMsg / handleSetMsg"] + DISP --> PROC["hostIf_DeviceProcessorInterface
Device.DeviceInfo.Processor.(i)"] + DISP --> PSTAT["hostIf_DeviceProcessStatusInterface
Device.DeviceInfo.ProcessStatus"] + DISP --> PPROC["DeviceProcessStatusProcess
Device.DeviceInfo.ProcessStatus.Process.(i)"] + DISP --> BT["XrdkBluetoothMgr
Device.DeviceInfo.X_RDKCENTRAL-COM_xBlueTooth.*"] + + DI --> BSSTORE["XBSStore
Bootstrap store"] + DI --> RFCSTORE["XRFCStore
RFC store"] + DI --> PROCFS["/proc/meminfo
/proc/uptime
/proc/version"] + DI --> SCRIPTS["triggerResetScript
factory/cold/warm reset"] + DI --> IARMBUS["IARM Bus
Device/MFR services"] + + BSSTORE --> BSJSON["partners_defaults.json
tr181store.ini"] + BSSTORE --> BSJOURNAL["XBSStoreJournal
fwValue tracking"] + RFCSTORE --> RFCINI["/opt/RFC/*.ini
/etc/rfcdefaults/"] + PSTAT --> PROCFS2["/proc/stat"] + PPROC --> PROCFSPID["/proc/PID/status"] + BT --> BTMGR["btmgr HAL"] +``` + +--- + +## Functional Areas + +### 1. Core DeviceInfo Parameters + +`hostIf_DeviceInfo` in `Device_DeviceInfo.cpp` handles the standard TR-181 and RDK extension parameters. Data comes from multiple backends: + +| Category | Source | Examples | +|----------|--------|---------| +| Static identifiers | IARM / MFR services | Manufacturer, ManufacturerOUI, ProductClass, SerialNumber | +| Software versions | `/version.txt`, `/etc/device.properties` | SoftwareVersion, HardwareVersion, AdditionalHardwareVersion | +| Runtime stats | `/proc/meminfo`, `/proc/uptime` | MemoryStatus.Total, MemoryStatus.Free, UpTime | +| Reset control | `triggerResetScript()` | X_RDKCENTRAL-COM_Reset (factory/cold/warehouse/customer) | +| Bootstrap values | `XBSStore` | Partner URL overrides, CMS management endpoint | +| RFC values | `XRFCStore` | Feature enable/disable flags, override parameters | +| Process stats | `/proc/stat`, `/proc/PID/status` | ProcessStatus.CPUUsage, Process.{i}.* | + +### 2. Bootstrap Store (XBSStore) + +The Bootstrap store manages partner-specific default configuration: + +```mermaid +sequenceDiagram + participant Daemon as tr69hostif startup + participant BS as XBSStore::getInstance() + participant JSON as partners_defaults.json + participant INI as tr181store.ini + + Daemon->>BS: getInstance() + BS->>JSON: Load base defaults (JSON array of key-value pairs) + BS->>INI: Load override values (flush on every setValue) + BS->>BS: Merge: INI values override JSON defaults + BS-->>Daemon: ready + + Note over BS,INI: On overrideValue(): write to INI immediately +``` + +**File locations** (in priority order, later overrides earlier): +1. `/etc/partners_defaults.json` — factory installed defaults +2. `/opt/partners_defaults.json` — operator-installed overrides +3. `/opt/tr181store.ini` — RFC/ACS-programmed runtime overrides + +### 3. RFC Store (XRFCStore) + +The RFC store manages feature enablement flags with a four-tier dictionary: + +| Dictionary | Source File | Description | +|-----------|-------------|-------------| +| `rfcdefaults` | `/etc/rfcdefaults/tr69hostif.ini` | Factory RFC defaults | +| `main` | `/opt/RFC/tr69hostif.ini` | Network RFC overrides | +| `localstore` | `/opt/persistent/RFC/` | Locally persisted overrides | +| `non-persistent` | In-memory only | Transient overrides cleared on restart | + +Priority (highest to lowest): `non-persistent` > `main` > `localstore` > `rfcdefaults` + +### 4. Bootstrap Store Journal (XBSStoreJournal) + +The journal records the provenance of every bootstrap store entry: + +```cpp +typedef struct { + std::string fwValue; // Value from the firmware/factory JSON + std::string buildTime; // ISO timestamp when fwValue was set + std::string updatedValue; // Current override value (if any) + HostIf_Source_Type_t source; // RFCUPDATE / ALLUPDATE / BOOTSTRAP +} JournalEntry; +``` + +When `XBSStore::overrideValue()` is called, it records the old firmware value, new value, source type, and timestamp in the journal so audit trails can be retrieved later. + +### 5. Process and CPU Statistics + +`hostIf_DeviceProcessStatusInterface` reads `/proc/stat` to compute `CPUUsage` as a percentage. `DeviceProcessStatusProcess` reads individual `/proc//status` files to populate `Process.{i}.*` table rows. + +### 6. Bluetooth (XrdkBluetoothMgr) + +`XrdkBluetoothMgr` bridges `Device.DeviceInfo.X_RDKCENTRAL-COM_xBlueTooth.*` parameter GET/SET requests to the `btmgr` HAL. Supported sub-objects: +- `DiscoveredDevice.{i}.*` — devices found during scan +- `PairedDevice.{i}.*` — bonded devices +- `ConnectedDevice.{i}.*` — currently connected devices +- `LimitedBeaconDetection.*` — BLE scanning + +--- + +## Key Data Structures + +### DeviceInfo parameter enum (partial) + +`Device_DeviceInfo.h` defines an enum `eDeviceInfoMembers` with over 200 values mapping each TR-181 parameter to an array index for the GET/SET dispatch table. + +### BS Store data flow + +```mermaid +flowchart LR + GET["GET request
Device.X_RDK*"] --> BS[XBSStore::getValue] + BS --> CACHE{In-memory map} + CACHE -->|hit| STR[Return string value] + CACHE -->|miss| NOK[Return NOK] + + SET[SET request] --> OV[XBSStore::overrideValue] + OV --> MAP[Update in-memory map] + MAP --> INI["Flush to tr181store.ini
entire file rewritten"] + OV --> JOURN["XBSStoreJournal::setJournalValue
Record provenance"] +``` + +--- + +## GET Request Flow (DeviceInfo) + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch as hostIf_msgHandler + participant DI as hostIf_DeviceInfo + participant Backend + + ACS->>Dispatch: GET Device.DeviceInfo.MemoryStatus.Total + Dispatch->>DI: getInstance(1) + DI->>DI: handleGetMsg(stMsgData) + DI->>DI: Lookup enum value for paramName + DI->>Backend: get_Device_DeviceInfo_MemStatus_Total(stMsgData) + Backend->>Backend: fopen("/proc/meminfo") + Backend->>Backend: sscanf for "MemTotal:" + Backend-->>DI: fills stMsgData->paramValue (UInt) + DI-->>Dispatch: OK + Dispatch-->>ACS: value in KB +``` + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `/proc/*` file not readable | Returns `NOK`; paramValue empty | +| IARM bus call fails | Returns `NOK`; logs error via RDK_LOG | +| BSStore key not found | Returns `NOK` | +| Unknown parameter name | Sets `fcInvalidParameterName`, returns `NOK` | +| C++ exception in handler | Catches `std::exception`, sets `fcInternalError`, returns `NOK` | +| Bluetooth HAL unavailable | `XrdkBluetoothMgr` returns `NOK`; no crash | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: `XBSStore::flush()` and `IniFile::flush()` rewrite the entire file on every `setValue()` + +**File**: `XrdkCentralComBSStore.cpp`, `IniFile.cpp` + +**Observation**: Every call to `XBSStore::overrideValue()` ultimately calls `IniFile::flush()`, which opens the `.ini` file with `ofstream` (default truncate mode) and rewrites all key-value pairs from scratch: + +```cpp +// IniFile.cpp — FIXME: truncating everytime is bad for flash in general +ofstream outputStream(m_filename.c_str()); +``` + +The FIXME comment is present in the code. On NAND flash storage, truncate+rewrite on every single-value update causes excessive sector erasures, accelerating flash wear. + +**Recommended fix**: Accumulate changes in memory and flush only on explicit sync, on daemon shutdown, or on a timer. + +--- + +### Gap 2 — High: BSStore journal source attribute not enforced + +**File**: `XrdkCentralComBSStoreJournal.cpp` + +**Observation**: `XBSStoreJournal::getJournalSource()` compares the journal source enum to `DEV_DETAIL_BS_UPDATE` but returns a `HostIf_Source_Type_t` enum value. If the journal entry was set by an RFC update, the returned source type may indicate `ALLUPDATE` even for a Bootstrap value, making it impossible to reliably distinguish RFC-overridden versus ACS-overridden bootstrap values. + +--- + +### Gap 3 — Medium: `Device.DeviceInfo.ProcessStatus.Process.{i}` built by reading `/proc//status` for all running PIDs + +**File**: `Device_DeviceInfo_ProcessStatus_Process.cpp` + +**Observation**: `getAllProcesses()` iterates `/proc/*/status` for all numeric PIDs. On a device with hundreds of processes, this can take hundreds of milliseconds on each GET request. There is no caching — every GET rewalks `/proc`. + +**Impact**: A polling ACS that GETs `ProcessNumberOfEntries` frequently causes measurable CPU spikes. + +**Recommended fix**: Cache the process list for a configurable TTL (e.g., 5 seconds). + +--- + +### Gap 4 — Medium: Bluetooth `XrdkBluetoothMgr` is conditionally compiled but the condition is undocumented + +**File**: `XrdkBlueTooth.cpp` + +**Observation**: The Bluetooth implementation is guarded by multiple `#ifdef` blocks without a documented build flag for enabling/disabling the BLE Tile beacon path (`ENABLE_TILE`). The Bluetooth manager calls `BTRMGR_*` HAL functions that may not be present on all RDK platform builds, causing linker failures on non-BT hardware. + +--- + +### Gap 5 — Medium: `X_RDKCENTRAL-COM_Reset` executes scripts without validating input against allowed values + +**File**: `Device_DeviceInfo.cpp` — reset handler + +**Observation**: The Reset parameter accepts values `factory`, `cold`, `warm`, `warehouse`, and `customer`. The handler calls `triggerResetScript()` from `hostIf_utils.cpp` which dispatches to `v_secure_system()` scripts. While `v_secure_system` is used (safe wrapper), the value itself is not validated against the hard-coded set of allowed values before dispatch. An unsupported reset type silently returns `NOK` with no fault code set. + +--- + +### Gap 6 — Low: `Device_DeviceInfo_Processor.cpp` returns hardcoded `Architecture` string + +**File**: `Device_DeviceInfo_Processor.cpp` + +**Observation**: The `Architecture` GET handler reads `/proc/version` or similar but returns a hardcoded fallback string on many build configurations rather than dynamically detecting the CPU architecture via `uname()`. On cross-compiled builds, this may report the build host's architecture instead of the target device's. + +--- + +## Testing + +Unit tests are in `gtest/gtest_main.cpp` (4,291 lines). Run: + +```bash +./run_ut.sh +``` + +Key test areas: +1. BSStore: load from JSON, override with INI, journal entry tracking. +2. RFCStore: four-tier priority resolution, `clearAll()`, `reloadCache()`. +3. DeviceInfo GET: MemoryStatus.Total/Free from mocked `/proc/meminfo`. +4. ProcessStatus: CPUUsage calculation from `/proc/stat`. + +--- + +## See Also + +- [Device/docs/README.md](../../Device/docs/README.md) — X_RDK_profile (WebPA/WebConfig URLs) +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon and IniFile overview +- [handlers/docs/README.md](../../../handlers/docs/README.md) — Dispatch layer diff --git a/src/hostif/profiles/Ethernet/docs/README.md b/src/hostif/profiles/Ethernet/docs/README.md new file mode 100644 index 000000000..c6c1be031 --- /dev/null +++ b/src/hostif/profiles/Ethernet/docs/README.md @@ -0,0 +1,292 @@ +# Ethernet Profile + +## Overview + +The Ethernet profile implements the TR-181 `Device.Ethernet.Interface.{i}.*` and `Device.Ethernet.Interface.{i}.Stats.*` object trees. It provides GET and SET access to physical Ethernet port attributes (link state, MAC address, speed, duplex mode) and comprehensive interface statistics (byte/packet counters). All data is read from the Linux sysfs path `/sys/class/net//` without spawning shell processes. + +--- + +## Directory Structure + +``` +src/hostif/profiles/Ethernet/ +├── Device_Ethernet_Interface.cpp # Interface GET/SET handlers +├── Device_Ethernet_Interface.h # Class, enum, struct definitions +├── Device_Ethernet_Interface_Stats.cpp # Statistics GET handlers +├── Device_Ethernet_Interface_Stats.h # Stats class and enum +├── Makefile.am +└── gtest/ + ├── gtest_ethernet.cpp # Unit tests (364 lines) + └── Makefile.am +``` + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET/SET Device.Ethernet.Interface.*| DISP[hostIf_msgHandler] + DISP --> IFACE["hostIf_EthernetInterface::getInstance
dev_id"] + DISP --> STATS["hostIf_EthernetInterfaceStats::getInstance
dev_id"] + IFACE --> HASH[(ifHash GHashTable)] + IFACE --> SYS1["/sys/class/net/ethN/carrier
enable + status"] + IFACE --> SYS2["/sys/class/net/ethN/address
MAC address"] + IFACE --> SYS3["/sys/class/net/ethN/speed
max bit rate"] + IFACE --> SYS4["/sys/class/net/ethN/duplex
duplex mode"] + STATS --> SYS5["/sys/class/net/ethN/statistics/
bytes_sent, packets_received ..."] + + subgraph NameResolution[Interface Name Resolution] + NAMER["getEthernetInterfaceName
dev_id to ethN"] + NAMER --> IFNAMEIDX["if_nameindex API
enumerate eth* interfaces"] + end + + IFACE --> NameResolution + STATS --> NameResolution +``` + +--- + +## TR-181 Parameter Coverage + +### Interface Parameters (`Device.Ethernet.Interface.{i}.*`) + +| Parameter | GET | SET | sysfs Path | +|-----------|-----|-----|-----------| +| `Enable` | ✅ | ❌ | `carrier` (1=up) | +| `Status` | ✅ | ❌ | `carrier` → "Up"/"Down" | +| `Name` | ✅ | ❌ | `if_nameindex()` | +| `Upstream` | ✅ | ❌ | `carrier` (same as Enable — see Gap 2) | +| `MACAddress` | ✅ | ❌ | `address` | +| `MaxBitRate` | ✅ | ❌ | `speed` (Mbps) | +| `DuplexMode` | ✅ | ❌ | `duplex` → "Full"/"Half"/"Auto" | +| `LastChange` | ❌ | ❌ | Not implemented | +| `LowerLayers` | ❌ | ❌ | Not implemented | +| `Alias` | ❌ | ❌ | Not implemented | +| `CurrentBitRate` | ❌ | ❌ | Not implemented | +| `EEECapability` | ❌ | ❌ | Not implemented | + +### Stats Parameters (`Device.Ethernet.Interface.{i}.Stats.*`) + +All statistics read from `/sys/class/net//statistics/`: + +| Parameter | Counter file | +|-----------|-------------| +| `BytesSent` | `tx_bytes` | +| `BytesReceived` | `rx_bytes` | +| `PacketsSent` | `tx_packets` | +| `PacketsReceived` | `rx_packets` | +| `ErrorsSent` | `tx_errors` | +| `ErrorsReceived` | `rx_errors` | +| `UnicastPacketsSent` | `tx_packets` (approximation) | +| `DiscardPacketsSent` | `tx_dropped` | +| `DiscardPacketsReceived` | `rx_dropped` | +| `MulticastPacketsSent` | `multicast` | +| `BroadcastPacketsSent` | Computed as `tx_packets - tx_unicast - multicast` | +| `UnknownProtoPacketsReceived` | `rx_frame_errors` | + +--- + +## Class Design + +### `hostIf_EthernetInterface` + +``` +class hostIf_EthernetInterface +├── static GHashTable* ifHash // dev_id → instance +├── static GMutex m_mutex // class-wide mutex (see Gap 1) +├── static GHashTable* m_notifyHash // notification hash +├── static EthernetInterface stEthInterface // SHARED state (all instances — see Gap 3) +│ +├── bool backupEnable, backupUpstream // per-instance change detection +├── char backupStatus[], backupName[], ... +├── bool bCalledEnable, bCalledStatus, ... // backup validity flags +│ +└── get_Device_Ethernet_Interface_{Param}() +``` + +### Key Structures + +```c +typedef struct Device_Ethernet_Interface { + bool enable; + char status[_BUF_LEN_16]; // "Up" or "Down" + char name[_BUF_LEN_16]; // "ethN" + bool upStream; + char mACAddress[S_LENGTH]; // "XX:XX:XX:XX:XX:XX" + int maxBitRate; // Mbps, read from /sys/class/net/ethN/speed + char duplexMode[_BUF_LEN_16];// "Full", "Half", "Auto" +} EthernetInterface; +``` + +--- + +## How Operations Work + +### GET Request Flow + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch as hostIf_msgHandler + participant Eth as hostIf_EthernetInterface + participant sysfs + + ACS->>Dispatch: GET Device.Ethernet.Interface.1.MACAddress + Dispatch->>Eth: getInstance(1) + Eth->>Eth: getLock() [g_mutex_init + g_mutex_lock] + Eth->>Eth: get_Device_Ethernet_Interface_MACAddress(stMsgData) + Eth->>Eth: get_Device_Ethernet_Interface_Fields(1, eMACAddress) + Eth->>Eth: getEthernetInterfaceName(1) → "eth0" + Eth->>sysfs: readEthernetInterfaceFile("/sys/class/net/eth0/address") + sysfs-->>Eth: "aa:bb:cc:dd:ee:ff\n" + Eth->>Eth: strncpy to stEthInterface.mACAddress + Eth->>Eth: Copy to stMsgData->paramValue + Eth->>Eth: Check change against backupMACAddress + Eth->>Eth: releaseLock() + Eth-->>Dispatch: OK + Dispatch-->>ACS: "aa:bb:cc:dd:ee:ff" +``` + +### Interface Name Resolution + +`getEthernetInterfaceName(ethInterfaceNum)` enumerates all network interfaces via `if_nameindex()` and returns the Nth interface with a name starting with `"eth"` (1-based). This determines which sysfs directory to read. + +--- + +## Change Detection + +Each parameter has: +1. A `bCalled*` flag indicating whether a backup value has been set +2. A `backup*` field holding the previous value +3. A comparison in each GET function: if `bCalled*` is true and the values differ, `*pChanged = true` + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `if_nameindex()` returns NULL | Logs error, returns `NOK` | +| No Nth `eth*` interface found | Logs error, returns `NOK` | +| `readEthernetInterfaceFile()` file not opened | Returns `NULL`; caller returns `NOK` | +| `malloc` failure in `readEthernetInterfaceFile` | Returns `NULL`; caller returns `NOK` | +| `/sys/class/net/ethN/speed` returns -1 (link down) | Stores -1 as `maxBitRate` (not filtered) | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: `getLock()` calls `g_mutex_init()` on every invocation + +**File**: `Device_Ethernet_Interface.cpp` + +**Observation**: + +```cpp +void hostIf_EthernetInterface::getLock() +{ + g_mutex_init(&hostIf_EthernetInterface::m_mutex); // called every time! + g_mutex_lock(&hostIf_EthernetInterface::m_mutex); +} +``` + +`g_mutex_init()` re-initializes an already-initialized mutex before locking it. According to the GLib documentation, calling `g_mutex_init()` on an already-initialized (and potentially already-locked) mutex is undefined behavior. This same pattern is present in multiple other profile classes. + +**Impact**: Possible data corruption, crash, or lock bypass under concurrent access. + +**Recommended fix**: Initialize the mutex once at class construction or via `G_MUTEX_INIT` static initializer, and remove the `g_mutex_init()` call from `getLock()`. + +--- + +### Gap 2 — High: `Upstream` reads `carrier` (physical link) instead of upstream direction + +**File**: `Device_Ethernet_Interface.cpp` — `eUpstream` case + +**Observation**: The `Upstream` parameter in TR-181 indicates whether the interface connects toward the WAN/upstream network. The implementation reads `/sys/class/net/ethN/carrier`, which only indicates physical link presence: + +```cpp +case eUpstream: + snprintf(cmd, BUFF_LENGTH, "/sys/class/net/%s/carrier", ethernetInterfaceName); + hostIf_EthernetInterface::stEthInterface.upStream = string_to_bool(value); +``` + +`carrier = 1` means a cable is plugged in, not that the interface is the upstream WAN port. + +**Impact**: `Upstream` always returns `true` for any interface with physical link. ACS cannot use this parameter to identify the WAN interface. + +**Recommended fix**: Read `/sys/class/net/ethN/uevent` and check `DEVTYPE=`, or use a device-specific configuration file to map interface names to their upstream/downstream roles. + +--- + +### Gap 3 — High: `stEthInterface` is a class-level static shared by all instances + +**File**: `Device_Ethernet_Interface.h` + +**Observation**: + +```cpp +static EthernetInterface stEthInterface; +``` + +All `hostIf_EthernetInterface` instances (dev_id 1, 2, 3, …) write to the same `stEthInterface` structure during `get_Device_Ethernet_Interface_Fields()`. Concurrent GET requests for `eth0` and `eth1` overwrite each other's in-flight results. + +**Impact**: On a multi-port device, concurrent GET requests return data from whichever interface wrote last. + +**Recommended fix**: Make `stEthInterface` an instance member field. + +--- + +### Gap 4 — Medium: `readEthernetInterfaceFile()` allocates a heap buffer that the caller never frees + +**File**: `Device_Ethernet_Interface.cpp` + +**Observation**: `readEthernetInterfaceFile()` allocates memory with `malloc()` and returns the pointer: + +```cpp +char *buffer = (char *)malloc(sizeof(char) * length); +... +return buffer; +``` + +In `get_Device_Ethernet_Interface_Fields()`, the returned pointer is copied into the target struct and then the pointer goes out of scope without a `free()` call. Each GET call for a field that uses this helper leaks heap memory. + +**Recommended fix**: Add `free(value)` after copying from the returned buffer, or change the helper to write directly into a caller-provided buffer. + +--- + +### Gap 5 — Medium: `MaxBitRate` returns -1 when the interface has no physical link + +**Observation**: `/sys/class/net/ethN/speed` returns `-1` when the Ethernet port has no cable attached. The handler copies this negative value into `stEthInterface.maxBitRate` and returns it to the caller. TR-181 specifies `MaxBitRate` as a non-negative integer in Mbps. Some ACS implementations reject negative values. + +**Recommended fix**: Map -1 to 0 or return `NOK` when speed is unavailable (link down). + +--- + +### Gap 6 — Low: No SET parameter support + +**Observation**: The Ethernet interface profile has no `handleSetMsg` path. TR-181 defines `Enable`, `Alias`, and `MaxBitRate` as writable. Any ACS SET request for these parameters returns `NOT_HANDLED`. + +--- + +## Testing + +Unit tests are in `gtest/gtest_ethernet.cpp`. Run: + +```bash +./run_ut.sh +``` + +When modifying this profile: +1. Verify `Enable` and `Status` both correctly reflect carrier state. +2. Verify `MaxBitRate` returns 0 or `NOK` when no link is present. +3. Verify Stats counters match `/sys/class/net/*/statistics/` values. +4. Test multi-interface scenarios with at least two `eth*` interfaces. + +--- + +## See Also + +- [IP Profile README](../../IP/docs/README.md) — IP interface layer above Ethernet +- [InterfaceStack Profile README](../../InterfaceStack/docs/README.md) — Layer stacking table +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview diff --git a/src/hostif/profiles/IP/docs/README.md b/src/hostif/profiles/IP/docs/README.md new file mode 100644 index 000000000..044fe1944 --- /dev/null +++ b/src/hostif/profiles/IP/docs/README.md @@ -0,0 +1,300 @@ +# IP Profile + +## Overview + +The IP profile implements the TR-181 `Device.IP.*` object tree — the most comprehensive network-layer profile in the daemon. It covers the global IP object, per-interface configuration and address enumeration (IPv4 and IPv6), interface statistics, active TCP/UDP port enumeration, and IP diagnostics (ping, traceroute, speed test, download/upload benchmarks, UDP echo). Data is gathered from the Linux kernel via `getifaddrs()`, `ioctl()`, `/proc/net/tcp`, `/proc/net/tcp6`, and `/sys/class/net/*/statistics/`. + +--- + +## Directory Structure + +``` +src/hostif/profiles/IP/ +├── Device_IP.cpp # Global IP object +├── Device_IP.h +├── Device_IP_Interface.cpp # Per-interface attributes +├── Device_IP_Interface.h +├── Device_IP_Interface_IPv4Address.cpp # IPv4 address table +├── Device_IP_Interface_IPv4Address.h +├── Device_IP_Interface_IPv6Address.cpp # IPv6 address table +├── Device_IP_Interface_IPv6Address.h +├── Device_IP_Interface_Stats.cpp # Per-interface statistics +├── Device_IP_Interface_Stats.h +├── Device_IP_ActivePort.cpp # Active TCP/UDP port table +├── Device_IP_ActivePort.h +├── Device_IP_Diagnostics_IPPing.cpp # ICMP ping diagnostic +├── Device_IP_Diagnostics_IPPing.h +├── Device_IP_Diagnostics_SpeedTest.cpp # Speed test diagnostic +├── Device_IP_Diagnostics_SpeedTest.h +├── Device_IP_Diagnostics_DownloadDiagnostics.h # Header-only C-style API +├── Device_IP_Diagnostics_UploadDiagnostics.h # Header-only C-style API +├── Device_IP_Diagnostics_TraceRoute.h # Header-only C-style API +├── Device_IP_Diagnostics_TraceRoute_RouteHops.h +├── Device_IP_Diagnostics_UDPEchoConfig.h # Header-only C-style API +└── Makefile.am +``` + +> **Note**: There is no `gtest/` subdirectory. The IP profile has no unit tests. + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA / RBUS] -->|GET/SET Device.IP.*| DISP[hostIf_msgHandler] + + DISP --> GIP["hostIf_IP
Device.IP"] + DISP --> IPIF["hostIf_IPInterface
Device.IP.Interface.(i)"] + DISP --> IPV4["hostIf_IPInterfaceIPv4Address
Device.IP.Interface.(i).IPv4Address.(i)"] + DISP --> IPV6["hostIf_IPInterfaceIPv6Address
Device.IP.Interface.(i).IPv6Address.(i)"] + DISP --> STATS["hostIf_IPInterfaceStats
Device.IP.Interface.(i).Stats"] + DISP --> APORT["hostIf_IPActivePort
Device.IP.ActivePort.(i)"] + DISP --> PING[hostIf_IP_Diagnostics_IPPing] + DISP --> SPEED[hostIf_IP_Diagnostics_SpeedTest] + + IPIF --> GETIFADDRS["getifaddrs + ioctl
interface enumeration"] + IPV4 --> GETIFADDRS2["getifaddrs
AF_INET address scan"] + IPV6 --> GETIFADDRS3["getifaddrs
AF_INET6 address scan"] + STATS --> SYSFS["/sys/class/net/N/statistics/*"] + APORT --> PROCNET["/proc/net/tcp
/proc/net/tcp6"] + + IPIF --> SETCMDS["system ifconfig/ifdown/ifup
Enable/Reset/MTU set"] +``` + +--- + +## TR-181 Parameter Coverage + +### `Device.IP` (global) + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `IPv4Capable` | ✅ | ❌ | Hardcoded `true` | +| `IPv4Enable` | ✅ | ✅ | `ioctl SIOCGIFFLAGS` | +| `IPv4Status` | ✅ | ❌ | Derived from enable flag | +| `IPv6Capable` | ✅ | ❌ | Checks for configured IPv6 via `getifaddrs` | +| `IPv6Enable` | ✅ | ✅ | `/proc/sys/net/ipv6/conf/all/disable_ipv6` | +| `IPv6Status` | ✅ | ❌ | Derived | +| `ULAPrefix` | ✅ | ❌ | Linux ULA prefix | +| `InterfaceNumberOfEntries` | ✅ | ❌ | `getifaddrs` count | +| `ActivePortNumberOfEntries` | ✅ | ❌ | `/proc/net/tcp` + `/proc/net/tcp6` line count | + +### `Device.IP.Interface.{i}` (per interface) + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Enable` | ✅ | ✅ | `ioctl SIOCGIFFLAGS` / `ifconfig up/down` | +| `IPv4Enable` | ✅ | ✅ | Interface flags | +| `IPv6Enable` | ✅ | ✅ | Per-interface disable_ipv6 | +| `Status` | ✅ | ❌ | `ioctl` flags | +| `Name` | ✅ | ❌ | `getifaddrs` | +| `Type` | ✅ | ❌ | `Normal` / `Loopback` / `Tunnel` | +| `Reset` | ✅ | ✅ | `ifdown`/`ifup` invocation | +| `MaxMTUSize` | ✅ | ✅ | `ifconfig mtu ` | +| `LastChange` | ❌ | ❌ | Not implemented | +| `LowerLayers` | ❌ | ❌ | Not implemented | +| `Router` | ❌ | ❌ | Not implemented | + +### `Device.IP.Interface.{i}.IPv4Address.{i}` + +| Parameter | GET | Source | +|-----------|-----|--------| +| `Enable` | ✅ | Derived from parent interface state | +| `Status` | ✅ | Derived | +| `IPAddress` | ✅ | `getifaddrs` AF_INET | +| `SubnetMask` | ✅ | `getifaddrs` AF_INET netmask | +| `AddressingType` | ✅ | Heuristic: DHCP if non-static, Static otherwise | + +### `Device.IP.Interface.{i}.IPv6Address.{i}` + +| Parameter | GET | Source | +|-----------|-----|--------| +| `Enable`, `Status` | ✅ | Derived | +| `IPAddress` | ✅ | `getifaddrs` AF_INET6 | +| `Origin` | ✅ | AutoConfigured / DHCPv6 / WellKnown / Static | +| `Prefix`, `PreferredLifetime`, `ValidLifetime` | ✅ | Parsed from kernel addresses | + +### `Device.IP.ActivePort.{i}` + +| Parameter | GET | Source | +|-----------|-----|--------| +| `LocalIPAddress`, `LocalPort` | ✅ | `/proc/net/tcp`, `/proc/net/tcp6` hex decode | +| `RemoteIPAddress`, `RemotePort` | ✅ | `/proc/net/tcp` hex decode | +| `Status` | ✅ | TCP state column → "Listen"/"Established" | + +--- + +## How Operations Work + +### Interface Enumeration + +`hostIf_IP` calls `getifaddrs()` to enumerate all network interfaces. Each interface gets a `dev_id` starting from 1. The mapping is cached in `ifHash`. + +```mermaid +flowchart LR + CALL[GET InterfaceNumberOfEntries] --> GIA[getifaddrs] + GIA --> FILTER[Filter: exclude loopback\nby optional flag] + FILTER --> COUNT[Count → numOfEntries] + COUNT --> HASH[Build ifHash: dev_id → ifname] +``` + +### IPv4 Address GET Flow + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch + participant IPv4 as hostIf_IPInterfaceIPv4Address + participant Kernel + + ACS->>Dispatch: GET Device.IP.Interface.1.IPv4Address.1.IPAddress + Dispatch->>IPv4: getInstance(1, 1) [interface 1, address 1] + IPv4->>Kernel: getifaddrs() + IPv4->>IPv4: Find Nth AF_INET address for interface 1 + IPv4->>IPv4: inet_ntop(AF_INET, addr, ipStr, INET_ADDRSTRLEN) + IPv4->>IPv4: Copy to stMsgData->paramValue + Dispatch-->>ACS: "192.168.1.100" +``` + +### Active Ports Flow + +`hostIf_IPActivePort` reads `/proc/net/tcp` (and `/proc/net/tcp6` for IPv6): +1. Each line has hex-encoded local/remote address+port and TCP state +2. The handler decodes hex IP bytes with byte-swap for endianness +3. TCP state `0A` = "Listen", `01` = "Established"; all others map to "Error" + +--- + +## SET Operations + +| SET Parameter | Implementation | +|---------------|---------------| +| `Device.IP.IPv4Enable` | `ioctl(SIOCSIFFLAGS)` on all interfaces | +| `Device.IP.IPv6Enable` | Writes `0`/`1` to `/proc/sys/net/ipv6/conf/all/disable_ipv6` | +| `Device.IP.Interface.{i}.Enable` | `ifconfig up/down` via `system()` | +| `Device.IP.Interface.{i}.Reset` | `ifdown ; ifup ` via `system()` | +| `Device.IP.Interface.{i}.MaxMTUSize` | `ifconfig mtu ` via `system()` | + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `getifaddrs()` fails | Returns `NOK`; logs `errno` | +| No Nth address for interface | Returns `NOK` | +| `system()` returns non-zero | Returns `NOK` | +| `/proc/net/tcp` not readable | Returns `NOK` | +| `IOCTL` fails | Returns `NOK`; logs `errno` | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: `set_Interface_Enable`, `set_Interface_Reset`, `set_Interface_Mtu` use `system()` instead of `v_secure_system()` + +**File**: `Device_IP_Interface.cpp` + +**Observation**: + +```cpp +int hostIf_IPInterface::set_Interface_Enable(int value) +{ + char cmd[BUFF_LENGTH] = { 0 }; + snprintf(cmd, BUFF_LENGTH, "ifconfig %s down", nameOfInterface); + return (system(cmd) < 0) ? NOK : OK; +} +``` + +`system()` is used instead of the security-hardened `v_secure_system()` wrapper required by the embedded platform. While `nameOfInterface` is derived from kernel interface enumeration (not user data), using bare `system()` bypasses the secure wrapper validation and contradicts the RDK coding standard applied in all other files. + +**Impact**: Any future code path that sets `nameOfInterface` from user input would introduce a command injection vulnerability. + +**Recommended fix**: Replace all `system(cmd)` calls with `v_secure_system(...)` using the format-string variant. + +--- + +### Gap 2 — High: `stIPInterfaceInstance` is a global static struct shared by all instances + +**File**: `Device_IP_Interface.h` + +**Observation**: + +```cpp +static IPInterface stIPInterfaceInstance; +``` + +All `hostIf_IPInterface` instances write to the same shared structure. GET requests for `Device.IP.Interface.1.*` and `Device.IP.Interface.2.*` issued concurrently overwrite each other's in-flight data. + +**Recommended fix**: Make `stIPInterfaceInstance` an instance field. + +--- + +### Gap 3 — High: `set_Interface_Reset` uses `ifdown`/`ifup` which may not exist on all embedded targets + +**File**: `Device_IP_Interface.cpp` + +**Observation**: + +```cpp +snprintf(cmd, BUFF_LENGTH, "ifdown %s", nameOfInterface); +system(cmd); +snprintf(cmd, BUFF_LENGTH, "ifup %s", nameOfInterface); +system(cmd); +``` + +`ifdown`/`ifup` are part of `ifupdown` package and are not available on Yocto-based or Buildroot RDK targets. On such platforms, `Reset` silently fails or partially executes (one command might be found, the other not). + +**Recommended fix**: Use `ip link set down && ip link set up` which is universally available via `iproute2`. + +--- + +### Gap 4 — Medium: `AddressingType` for IPv4 addresses uses a heuristic, not the actual DHCP lease state + +**File**: `Device_IP_Interface_IPv4Address.cpp` + +**Observation**: The `AddressingType` parameter should report `DHCP`, `Static`, `AutoIP`, or `IPCP`. The implementation derives this from whether the address appears in a routing or lease file, using an approximation. On a device where static addresses are configured through DHCP-like tooling (e.g., NetworkManager static leases), this heuristic returns the wrong type. + +--- + +### Gap 5 — Medium: IPv4 Active Ports parser does not handle `/proc/net/udp` + +**File**: `Device_IP_ActivePort.cpp` + +**Observation**: `ActivePort.{i}` in TR-181 covers both TCP and UDP active ports. The implementation reads only `/proc/net/tcp` and `/proc/net/tcp6`. UDP sockets from `/proc/net/udp` and `/proc/net/udp6` are not included. + +**Impact**: `ActivePortNumberOfEntries` undercounts total active ports; any UDP server ports on the device are invisible to ACS. + +--- + +### Gap 6 — Low: No unit tests + +**Observation**: The IP profile directory has no `gtest/` subdirectory. This is the largest network profile (nine `.cpp` files, 5,822 lines) and has zero automated test coverage. + +**Recommended fix**: Add unit tests using mock `getifaddrs()` and mock `/proc/net/tcp` file fixtures. + +--- + +### Gap 7 — Low: Diagnostics (Download/Upload/TraceRoute/UDPEcho) are header-only stubs + +**Observation**: `Device_IP_Diagnostics_DownloadDiagnostics.h`, `Device_IP_Diagnostics_UploadDiagnostics.h`, `Device_IP_Diagnostics_TraceRoute.h`, and `Device_IP_Diagnostics_UDPEchoConfig.h` declare C-style `set/get_Device_IP_Diagnostics_*` functions but none of them have corresponding `.cpp` implementations. These diagnostics are never registered with the manager and any ACS attempt to use them returns `NOT_HANDLED`. + +--- + +## Testing + +There are currently no unit tests for the IP profile. When adding tests: +1. Mock `getifaddrs()` to return deterministic interface lists. +2. Provide fake `/proc/net/tcp` content to test active port parsing. +3. Test IPv6 address origin classification for SLAAC vs. DHCPv6 vs. manual. +4. Test `InterfaceNumberOfEntries` filtering (loopback inclusion/exclusion). + +--- + +## See Also + +- [Ethernet Profile README](../../Ethernet/docs/README.md) — Layer 2 below IP +- [InterfaceStack Profile README](../../InterfaceStack/docs/README.md) — Stacking table +- [DHCPv4 Profile README](../../DHCPv4/docs/README.md) — DHCPv4 client uses IP interface lookup +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview diff --git a/src/hostif/profiles/InterfaceStack/docs/README.md b/src/hostif/profiles/InterfaceStack/docs/README.md new file mode 100644 index 000000000..1efc50a2a --- /dev/null +++ b/src/hostif/profiles/InterfaceStack/docs/README.md @@ -0,0 +1,209 @@ +# InterfaceStack Profile + +## Overview + +The InterfaceStack profile implements the TR-181 `Device.InterfaceStack.{i}.*` object. This object provides a read-only table that describes the adjacency relationships between network interface layers — for example, how an IP interface sits on top of a bridge, which sits on top of a physical Ethernet or MoCA interface. The table is constructed dynamically by walking all Ethernet, MoCA, bridge, and IP interfaces present on the device and inferring their stacking relationships from Linux bridge device memberships. + +This profile is guarded by the `USE_INTFSTACK_PROFILE` build flag. When the flag is not defined, the entire implementation is excluded from the build. + +--- + +## Directory Structure + +``` +src/hostif/profiles/InterfaceStack/ +├── Device_InterfaceStack.cpp # Full implementation (889 lines) +├── Device_InterfaceStack.h # Class declaration +└── Makefile.am +``` + +> **Note**: There is no `gtest/` subdirectory. The InterfaceStack profile has no unit tests. + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET Device.InterfaceStack.*| DISP[hostIf_msgHandler] + DISP --> IFS[hostif_InterfaceStack::getInstance] + DISP --> NUMENT[hostif_InterfaceStack::get_numberOfEntries] + + subgraph BuildPhase[Table Build - populateInterfaceStack] + SYSNET["/sys/class/net/*
enumerate all interfaces"] + BRCTL["bridge fdb / ip link show
bridge membership"] + ETH_IFACE["Device.Ethernet.Interface.*
from hostIf_EthernetInterface"] + MOCA_IFACE["Device.MoCA.Interface.*
from MoCAInterface optional"] + IP_IFACE["Device.IP.Interface.*
from hostIf_IPInterface"] + + SYSNET --> BRCTL + ETH_IFACE --> LAYERMAP["LayerInfo map
higher+lower layer tracking"] + MOCA_IFACE --> LAYERMAP + BRCTL --> BRIDGETABLE[("stBridgeTableHash
bridge to members")] + BRIDGETABLE --> LAYERMAP + IP_IFACE --> LAYERMAP + LAYERMAP --> STKHASH[("stIshash
dev_id to InterfaceStack")] + end + + IFS --> STKHASH + NUMENT --> STKHASH +``` + +--- + +## TR-181 Parameter Coverage + +| Parameter | GET | Description | +|-----------|-----|-------------| +| `Device.InterfaceStack.{i}.HigherLayer` | ✅ | TR-181 path of the upper interface (e.g., `Device.IP.Interface.1`) | +| `Device.InterfaceStack.{i}.LowerLayer` | ✅ | TR-181 path of the lower interface (e.g., `Device.Ethernet.Interface.1`) | +| `Device.InterfaceStackNumberOfEntries` | ✅ | Count of rows in the table | + +--- + +## How the Table is Built + +The implementation builds the `stIshash` table by executing these steps in order: + +### Step 1 — Build the bridge table + +The daemon reads `/sys/class/net/*/brif/` (or executes `ip link show type bridge`) to discover all bridge interfaces and their member ports. The result is stored in `stBridgeTableHash`: + +``` +Bridge "hnbr0" → members: {"bcm0", "eth1"} +``` + +### Step 2 — Build lower-layer entries for physical interfaces + +For every `Device.Ethernet.Interface.{i}`, a layer-info entry `(lower = "Device.Ethernet.Interface.N", higher = "")` is created. If MoCA is enabled (`USE_MoCA_PROFILE`), the same is done for `Device.MoCA.Interface.{i}`. + +### Step 3 — Process bridges + +For each bridge and each bridge member: +- The bridge entry gets `lower = "Device.Bridging.Bridge.N.Port.M"` added +- The member interface's entry gets `higher = "Device.Bridging.Bridge.N.Port.M"` added + +### Step 4 — Fill remaining higher layers from IP interfaces + +For any interface entry that still has an empty `higher` value, the daemon looks for a `Device.IP.Interface.{i}` whose `LowerLayers` parameter references it. + +### Step 5 — Create instances + +For each `(higherLayer, lowerLayer)` pair in the layer map, a new `hostif_InterfaceStack` instance is created and inserted into `stIshash`. + +### Example Output + +Given: +``` +Physical: bcm0 (Device.Ethernet.Interface.1), eth1 (Device.MoCA.Interface.1) +Bridge: hnbr0 bridges {bcm0, eth1} +IP: eth0 (Device.IP.Interface.1) directly on eth1 +``` + +The resulting `InterfaceStack.*` entries are: + +| Instance | HigherLayer | LowerLayer | +|----------|-------------|-----------| +| 1 | `Device.Bridging.Bridge.1.Port.1` | `Device.Ethernet.Interface.1` | +| 2 | `Device.Bridging.Bridge.1.Port.1` | `Device.MoCA.Interface.1` | +| 3 | `Device.IP.Interface.1` | `Device.Bridging.Bridge.1.Port.1` | + +--- + +## Change Detection + +`get_Device_InterfaceStack_HigherLayer()` and `get_Device_InterfaceStack_LowerLayer()` use the standard backup pattern: +- `bCalledHigherLayer` / `bCalledLowerLayer` flags +- `backupHigherLayer` / `backupLowerLayer` arrays +- `*pChanged = true` if value differs from backup + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `USE_INTFSTACK_PROFILE` not defined | Entire implementation compiled out | +| `/sys/class/net` not readable | Logs error, `stIshash` stays empty, `numberOfEntries = 0` | +| Bridge table build fails | Continues without bridge entries | +| No matching IP interface for LowerLayer | `HigherLayer` left empty in that entry | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: `getLock()` calls `g_mutex_init()` on every invocation + +**File**: `Device_InterfaceStack.cpp` + +**Observation**: Several GLib-based profile classes in this codebase share the same pattern: + +```cpp +void hostif_InterfaceStack::getLock() { + g_mutex_init(&stMutex); // BUG: re-initializes on every call + g_mutex_lock(&stMutex); +} +``` + +Calling `g_mutex_init()` on a mutex that is already locked (by another thread calling `getLock()`) is undefined behavior per GLib documentation. + +**Recommended fix**: Initialize `stMutex` once at startup via `G_MUTEX_INIT` or within `populateInterfaceStack()`. + +--- + +### Gap 2 — High: Table rebuild does not invalidate existing GET requests in flight + +**File**: `Device_InterfaceStack.cpp` + +**Observation**: When `populateInterfaceStack()` is called (e.g., due to an interface change event), it calls `closeAllInstances()` to delete all existing `hostif_InterfaceStack` objects and then rebuilds the hash from scratch. Any GET request that obtained a pointer to an existing instance via `getInstance()` before the rebuild will hold a dangling pointer after `closeAllInstances()` returns. + +**Impact**: Crash or memory corruption if an interface-change event coincides with a GET request. + +**Recommended fix**: Use reference counting or a read-write lock to protect the lifetime of all accessed instances. + +--- + +### Gap 3 — Medium: Entire profile disabled when `USE_INTFSTACK_PROFILE` is not set + +**Observation**: The complete `.cpp` file `Device_InterfaceStack.cpp` is wrapped in: + +```cpp +#ifdef USE_INTFSTACK_PROFILE +... +#endif +``` + +This means any `Device.InterfaceStack.*` GET request returns `NOT_HANDLED` without any diagnostic log. ACS receives no indication whether the parameter is unsupported or absent. + +--- + +### Gap 4 — Medium: Bridge membership detection depends on `ip link show` subprocess + +**Observation**: Some code paths use `v_secure_popen("r", "ip link show type bridge ...")` to discover bridges. On embedded targets where `iproute2` is not in PATH or the kernel lacks bridge netlink support, the bridge table remains empty and all bridge-based stacking entries are missing. + +**Recommended fix**: Read bridge membership directly from `/sys/class/net/*/brif/` directory entries, which does not require spawning a subprocess. + +--- + +### Gap 5 — Low: No unit tests + +**Observation**: There is no `gtest/` directory. The table-building algorithm, which involves multiple cross-product joins between Ethernet, MoCA, bridge, and IP interface sets, has no automated verification. Regressions in the stacking logic are difficult to detect. + +--- + +## Testing + +There are no unit tests currently. When adding tests: +1. Mock `/sys/class/net/` with a virtual filesystem with known bridge and interface configurations. +2. Verify `numberOfEntries` matches the expected stacking graph. +3. Test with bridges containing multiple members. +4. Test with MoCA enabled (`USE_MoCA_PROFILE`) and disabled. + +--- + +## See Also + +- [Ethernet Profile README](../../Ethernet/docs/README.md) — Provides lower-layer entries +- [IP Profile README](../../IP/docs/README.md) — Provides higher-layer entries +- [moca Profile README](../../moca/docs/README.md) — Optional MoCA lower layers +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview diff --git a/src/hostif/profiles/STBService/docs/README.md b/src/hostif/profiles/STBService/docs/README.md new file mode 100644 index 000000000..07459a224 --- /dev/null +++ b/src/hostif/profiles/STBService/docs/README.md @@ -0,0 +1,301 @@ +# STBService Profile + +## Overview + +The STBService profile implements the TR-135 (Set-top Box Service) object tree `Device.Services.STBService.1.*`. It exposes the AV capabilities, output port state, and hardware health metrics of an RDK set-top box to TR-069 ACS and WebPA. All hardware access goes through the RDK Device Settings (DS) HAL layer (`libdshal`) using C++ wrapper objects from `device::Host`, `device::VideoOutputPort`, `device::AudioOutputPort`, and related classes. SD card and eMMC health data additionally use the `rdkStorageMgr` HAL. + +--- + +## Directory Structure + +``` +src/hostif/profiles/STBService/ +├── Capabilities.cpp # STBService.1.Capabilities.* +├── Capabilities.h +├── Components_AudioOutput.cpp # STBService.1.Components.AudioOutput.{i}.* +├── Components_AudioOutput.h +├── Components_DisplayDevice.cpp # STBService.1.Components.HDMI.{i}.DisplayDevice.* +├── Components_DisplayDevice.h +├── Components_HDMI.cpp # STBService.1.Components.HDMI.{i}.* +├── Components_HDMI.h +├── Components_SPDIF.cpp # STBService.1.Components.SPDIF.{i}.* +├── Components_SPDIF.h +├── Components_VideoDecoder.cpp # STBService.1.Components.VideoDecoder.{i}.* +├── Components_VideoDecoder.h +├── Components_VideoOutput.cpp # STBService.1.Components.VideoOutput.{i}.* +├── Components_VideoOutput.h +├── Components_XrdkEMMC.cpp # X_RDKCENTRAL-COM_eMMCFlash.* +├── Components_XrdkEMMC.h +├── Components_XrdkSDCard.cpp # X_RDKCENTRAL-COM_SDCard.* +├── Components_XrdkSDCard.h +└── Makefile.am +``` + +> **Note**: There is no `gtest/` subdirectory. The STBService profile has no unit tests. + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET/SET Device.Services.STBService.1.*| DISP[hostIf_msgHandler] + + DISP --> CAP["hostIf_STBServiceCapabilities
Capabilities.*"] + DISP --> AUD["hostIf_STBServiceAudioOutput
Components.AudioOutput.(i).*"] + DISP --> HDMI["hostIf_STBServiceHDMI
Components.HDMI.(i).*"] + DISP --> DISP2["hostIf_STBServiceDisplayDevice
Components.HDMI.(i).DisplayDevice.*"] + DISP --> SPDIF["hostIf_STBServiceSPDIF
Components.SPDIF.(i).*"] + DISP --> VDEC["hostIf_STBServiceVideoDecoder
Components.VideoDecoder.(i).*"] + DISP --> VOUT["hostIf_STBServiceVideoOutput
Components.VideoOutput.(i).*"] + DISP --> EMMC["hostIf_STBServiceXeMMC
Components.X_RDKCENTRAL-COM_eMMCFlash.*"] + DISP --> SDCARD["hostIf_STBServiceXSDCard
Components.X_RDKCENTRAL-COM_SDCard.*"] + + CAP --> DSHAL["DS HAL
device::Host
device::VideoOutputPort
device::AudioOutputPort"] + AUD --> DSHAL + HDMI --> DSHAL + DISP2 --> DSHAL + SPDIF --> DSHAL + VDEC --> DSHAL + VOUT --> DSHAL + EMMC --> STORHAL["rdkStorageMgr HAL
STRM_GetEMMCFlashStatus"] + SDCARD --> STORHAL2["rdkStorageMgr HAL
STRM_GetSDCardStatus"] +``` + +--- + +## TR-181/TR-135 Parameter Coverage + +### `STBService.1.Capabilities` + +| Parameter | GET | Source | +|-----------|-----|--------| +| `VideoDecoder.VideoStandards` | ✅ | DS HAL — HEVC, H264, MPEG2 support flags | +| `VideoDecoder.HEVC.ProfileLevel.{i}.*` | ✅ | Enumerated from DS capability list | +| `AudioStandards` | ✅ | DS HAL audio capability flags | +| `HDMI.SupportedResolutions.{i}.*` | ✅ | DS HAL supported resolution list | + +### `STBService.1.Components.HDMI.{i}` + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Enable` | ✅ | ✅ | DS HAL `videoOutputPort.isEnabled()` | +| `Status` | ✅ | ❌ | DS HAL connection status | +| `Name` | ✅ | ❌ | Port name string | +| `ResolutionMode` | ✅ | ✅ | "Auto" or "Manual" — `dsHDMIResolutionMode` | +| `ResolutionValue` | ✅ | ✅ | DS resolution objects (720p, 1080p, 4K, etc.) | +| `DisplayDevice.*` | ✅ | ❌ | DS HAL connected display device info | + +Supported resolutions (via `dsVideoPixelResolutionMapper`): 720×480, 720×576, 1280×720, 1920×1080, 3840×2160. + +### `STBService.1.Components.AudioOutput.{i}` + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Enable` | ✅ | ✅ | DS HAL `audioOutputPort.setEnable()` | +| `Status` | ✅ | ❌ | DS HAL `audioOutputPort.isEnabled()` | +| `AudioFormat` | ✅ | ❌ | HDMI/SPDIF audio coding type | +| `AudioLevel` | ✅ | ✅ | Gain/level in dB | +| `Alias` | ✅ | ❌ | Port name from DS HAL | +| `CompressionLevel` | ✅ | ✅ | Audio compression setting | +| `AudioDelay` | ✅ | ✅ | Delay in ms | + +### `STBService.1.Components.SPDIF.{i}` + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Enable` | ✅ | ✅ | DS HAL | +| `Status` | ✅ | ❌ | DS HAL | +| `ForceEnable` | ✅ | ✅ | Force stereo PCM override | +| `AudioFormat` | ✅ | ❌ | Auto/PCM/AC3 | +| `AudioDelay` | ✅ | ✅ | Delay in ms | + +### `STBService.1.Components.VideoDecoder.{i}` + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Enable` | ✅ | ❌ | DS HAL video decoder state | +| `Status` | ✅ | ❌ | DS HAL | +| `ContentAR` | ✅ | ❌ | Current display aspect ratio | +| `VideoStandards` | ✅ | ❌ | Supported formats string | + +### `STBService.1.Components.VideoOutput.{i}` + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Enable` | ✅ | ✅ | DS HAL video output port enable | +| `Status` | ✅ | ❌ | DS HAL | +| `VideoFormat` | ✅ | ❌ | Pixel format string | +| `AspectRatio` | ✅ | ✅ | DS HAL aspect ratio | +| `HDCP` | ✅ | ❌ | DS HAL HDCP encryption state | + +### `STBService.1.Components.X_RDKCENTRAL-COM_eMMCFlash.*` + +| Parameter | GET | Source | +|-----------|-----|--------| +| `Capacity` | ✅ | `STRM_GetEMMCFlashStatus()` | +| `LifeElapsedA`, `LifeElapsedB` | ✅ | eMMC health registers via rdkStorageMgr | +| `PreEOLState*` | ✅ | Pre-EOL state for system/EUDA/MLC areas | +| `LotID`, `Manufacturer`, `Model`, `SerialNumber` | ✅ | HAL fields | +| `ReadOnly`, `TSBQualified` | ✅ | Boolean flags | + +### `STBService.1.Components.X_RDKCENTRAL-COM_SDCard.*` + +| Parameter | GET | Source | +|-----------|-----|--------| +| `Capacity`, `LifeElapsed` | ✅ | `STRM_GetSDCardStatus()` | +| `CardFailed`, `ReadOnly`, `Status` | ✅ | rdkStorageMgr flags | +| `LotID`, `Manufacturer`, `Model`, `SerialNumber` | ✅ | HAL fields | + +--- + +## How Operations Work + +### HDMI Resolution Mode SET Flow + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch + participant HDMI as hostIf_STBServiceHDMI + participant DSHAL as DS HAL + + ACS->>Dispatch: SET Components.HDMI.1.ResolutionMode = "Auto" + Dispatch->>HDMI: handleSetMsg(stMsgData) + HDMI->>HDMI: strcmp(paramName, "ResolutionMode") + HDMI->>HDMI: strcpy(dsHDMIResolutionMode, "Auto") + HDMI-->>Dispatch: OK + + ACS->>Dispatch: SET Components.HDMI.1.ResolutionValue = "1920x1080p/60Hz" + Dispatch->>HDMI: handleSetMsg(stMsgData) + HDMI->>HDMI: Parse resolution string + HDMI->>HDMI: Map to dsVideoResolutionSettings_t + HDMI->>DSHAL: videoOutputPort.setResolution(resolution) + DSHAL-->>HDMI: success + HDMI-->>Dispatch: OK +``` + +### Instance Lifecycle + +Each STBService class uses `device::Host::getInstance()` to access the DS HAL device tree: + +```mermaid +flowchart LR + INST[getInstance\ndev_id] --> HASH[(ifHash GHashTable)] + HASH -->|miss| DSHOST[device::Host::getInstance] + DSHOST --> PORT[getVideoOutputPort\nor getAudioOutputPort] + PORT --> NEW[new hostIf_STBService*\nstored in ifHash] + HASH -->|hit| RET[return cached instance] +``` + +If the DS HAL throws `device::IllegalArgumentException` (e.g., port index out of range), the constructor catches it and returns `NULL` from `getInstance()`. + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| DS HAL throws `device::IllegalArgumentException` | Caught in `getInstance()`; no instance created; GET returns `NOT_HANDLED` | +| DS HAL throws `device::Exception` | Caught, logs code and message, returns `NOK` | +| DS HAL throws `dsError_t` | Caught, logs error code, returns `NOK` | +| `rdkStorageMgr` HAL not available | Returns `NOK`; paramValue empty | +| DS HAL not initialized | Typically throws and is caught | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: `getLock()` calls `g_mutex_init()` on every invocation (multiple classes) + +**File**: `Components_HDMI.cpp`, `Components_AudioOutput.cpp`, `Components_VideoOutput.cpp`, and others + +**Observation**: All STBService classes use the same pattern: + +```cpp +void hostIf_STBServiceHDMI::getLock() +{ + g_mutex_init(&hostIf_STBServiceHDMI::m_mutex); // re-initialize on every call + g_mutex_lock(&hostIf_STBServiceHDMI::m_mutex); +} +``` + +This is undefined behavior when the mutex is already locked by another thread. + +**Impact**: Potential deadlock or mutex corruption under concurrent GET/SET access to any STBService component. + +--- + +### Gap 2 — High: No unit tests for any STBService component + +**Observation**: The entire `STBService/` directory has no `gtest/` subdirectory. The profile has 12 source files spanning 5,369 lines of C++ with complex DS HAL interactions and no automated test coverage. DS HAL failures that return silently (logging only) may go undetected for extended periods. + +--- + +### Gap 3 — Medium: `dsHDMIResolutionMode` is a class-level static `char[10]` shared across all HDMI instances + +**File**: `Components_HDMI.h` + +**Observation**: + +```cpp +static char dsHDMIResolutionMode[10]; +``` + +All `hostIf_STBServiceHDMI` instances (multiple HDMI ports) share one `dsHDMIResolutionMode` value. Setting the mode on HDMI port 1 immediately affects the mode reported by HDMI port 2, even if the hardware supports different modes per port. + +--- + +### Gap 4 — Medium: Resolution frame rate mapping is incomplete + +**File**: `Components_HDMI.cpp` + +**Observation**: `dsVideoFrameRateMapper` maps frame rates 24, 25, 30, 50, 60, 23.98, 29.97, and 59.94. Uncommon rates used by some cable standards (e.g., 120 Hz, 144 Hz) are not listed. When the DS HAL returns an unmapped frame rate, `getStringFromEnum()` returns `NULL` and the returned `ResolutionValue` string is malformed. + +--- + +### Gap 5 — Medium: eMMC and SD card health data returned without error if HAL returns zero values + +**File**: `Components_XrdkEMMC.cpp`, `Components_XrdkSDCard.cpp` + +**Observation**: If `STRM_GetEMMCFlashStatus()` or `STRM_GetSDCardStatus()` returns `MSRM_SUCCESS` but populates fields with zero values (device not present or HAL stub), the handlers return the zero values as valid data without distinguishing "device not present" from "device present but all meters at zero". ACS has no way to know whether the eMMC/SD card exists. + +--- + +### Gap 6 — Low: Capabilities VideoStandards string is built by concatenating all supported format names + +**File**: `Capabilities.cpp` + +**Observation**: `VideoDecoder.VideoStandards` is a comma-separated string built by iterating all DS HAL video capability flags. The string length is not bounded. If a future platform adds many new standards, the result could exceed `TR69HOSTIFMGR_MAX_PARAM_LEN` (4 KB) and be silently truncated. + +--- + +## Testing + +There are currently no unit tests. When adding coverage: +1. Mock the DS HAL (`device::Host::getInstance()`) using a stub/fake. +2. Verify HDMI resolution SET correctly maps string values to `dsVideoResolutionSettings_t`. +3. Verify AudioOutput level SET/GET round-trip. +4. Test eMMC/SD card health parameter parsing from mock `rdkStorageMgr` responses. + +--- + +## Platform Notes + +### DS HAL Dependency + +All AV component parameters require the DS HAL dynamic library (`libdshal.so`) at runtime. On RDK devices, this library is provided by the platform vendor. On emulators or headless builds: +- `device::Host::getInstance()` may throw on its first call +- All STBService GETs return `NOT_HANDLED` + +### Build Guard + +The STBService profile is always compiled but the DS HAL headers and library must be available at build time. The `ENABLE_TILE` flag controls Bluetooth LE beacon detection (used by `XrdkBlueTooth` in DeviceInfo, not directly in STBService). + +--- + +## See Also + +- [DeviceInfo/docs/README.md](../../DeviceInfo/docs/README.md) — Bluetooth (XrdkBlueTooth) lives there +- [StorageService/docs/README.md](../../StorageService/docs/README.md) — USB/HDD storage (different from eMMC/SD) +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview diff --git a/src/hostif/profiles/StorageService/docs/README.md b/src/hostif/profiles/StorageService/docs/README.md new file mode 100644 index 000000000..6073d446b --- /dev/null +++ b/src/hostif/profiles/StorageService/docs/README.md @@ -0,0 +1,246 @@ +# StorageService Profile + +## Overview + +The StorageService profile implements the TR-140 (Storage Service) based `Device.StorageService.{i}.*` object tree. It exposes attached physical storage media — external USB drives and SATA hard disks — to TR-069 ACS management, including vendor, model, serial number, capacity, connection type, and health diagnostics obtained via `smartctl`. Storage media enumeration uses `fdisk -l` filtered for non-MTD, non-eMMC devices. + +--- + +## Directory Structure + +``` +src/hostif/profiles/StorageService/ +├── Service_Storage.cpp # StorageService object (instance container) +├── Service_Storage.h +├── Service_Storage_PhyMedium.cpp # Physical medium detail (655 lines) +├── Service_Storage_PhyMedium.h +└── Makefile.am +``` + +> **Note**: There is no `gtest/` subdirectory. The StorageService profile has no unit tests. + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET Device.StorageService.*| DISP[hostIf_msgHandler] + DISP --> SS["hostIf_StorageSrvc
Device.StorageService.(i)"] + DISP --> PM["hostIf_PhysicalMedium
Device.StorageService.(i).PhysicalMedium.(j)"] + + SS --> FDISK1["fdisk -l grep Disk wc -l
PhysicalMediumNumberOfEntries"] + PM --> FDISK2["fdisk -l grep Disk sed Np awk
disk device path /dev/sdX"] + PM --> SMARTCTL["smartctl --scan
smartctl -A /dev/sdX SMART attrs"] + PM --> UDEV["udevadm info
vendor, model, serial number"] + PM --> FDISK3["fdisk -l /dev/sdX
capacity in bytes"] + + subgraph HashKey["Instance Key: storageServiceNum x 100 + phyMedNum"] + PHASH[(phyMedHash GHashTable)] + end + PM --> HashKey +``` + +--- + +## TR-181 Parameter Coverage + +### `Device.StorageService.{i}` (container) + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Alias` | ✅ | ❌ | Constructed from dev_id | +| `Enable` | ✅ | ❌ | Hardcoded `true` | +| `PhysicalMediumNumberOfEntries` | ✅ | ❌ | `fdisk -l \| grep Disk \| egrep -v "mtdblock\|mmcblk" \| wc -l` | + +### `Device.StorageService.{i}.PhysicalMedium.{j}` + +| Parameter | GET | SET | Source | +|-----------|-----|-----|--------| +| `Alias` | ✅ | ❌ | Constructed from dev_id | +| `Name` | ✅ | ❌ | `/dev/sdX` path from `fdisk -l` | +| `Vendor` | ✅ | ❌ | `udevadm info` | +| `Model` | ✅ | ❌ | `udevadm info` | +| `SerialNumber` | ✅ | ❌ | `udevadm info` | +| `FirmwareVersion` | ✅ | ❌ | `udevadm info` | +| `ConnectionType` | ✅ | ❌ | "USB" or "SATA" from `udevadm` bus path | +| `Removable` | ✅ | ❌ | SCSI query or USB indicator | +| `Capacity` | ✅ | ❌ | `fdisk -l /dev/sdX` total bytes | +| `Status` | ✅ | ❌ | SMART overall health assessment | +| `Health` | ✅ | ❌ | SMART raw attribute check (see below) | + +--- + +## How Operations Work + +### Instance Enumeration and Hash Building + +`hostIf_PhysicalMedium::rebuildHash()` orchestrates the full discovery: + +```mermaid +sequenceDiagram + participant Handler + participant FdiskCmd + participant SmartCmd + participant phyMedHash + + Handler->>FdiskCmd: v_secure_popen("fdisk -l | grep Disk | egrep -v mtdblock|mmcblk | wc -l") + FdiskCmd-->>Handler: N (number of disks) + loop For each storageServiceInstance (1..storageMax) + Handler->>Handler: getPhysicalMediumNumberOfEntries(storageServiceInstance) + loop For each phyMedInstance (1..phyMedMax) + Handler->>Handler: new hostIf_PhysicalMedium(storageServiceInstance, phyMedInstance) + Handler->>phyMedHash: insert(key = storageServiceNum×100 + phyMedNum, pRet) + end + end +``` + +The instance key encoding `(storageServiceInstanceNumber * 100) + dev_id` allows up to 99 physical media per storage service instance. + +### Physical Medium Field Retrieval + +Each GET parameter triggers a dedicated subprocess: + +```mermaid +flowchart LR + GET["GET request
for a PhyMed field"] --> SWITCH{switch field} + SWITCH -->|Name| FDISK[fdisk -l grep Disk sed n Xp awk 2] + SWITCH -->|Vendor/Model/Serial| UDEV[udevadm info -q property -n /dev/sdX] + SWITCH -->|Capacity| FDISKCAP[fdisk -l /dev/sdX awk bytes] + SWITCH -->|Status/Health| SMART["smartctl --scan
smartctl -A /dev/sdX grep SMART_PARAMS"] +``` + +### SMART Health Check + +The health check reads these SMART attributes (defined in `STORAGE_PHYMED_SMARTPARAMS`): + +``` +Raw_Read_Error_Rate +Reported_Uncorrect +Airflow_Temperature_Cel +G-Sense_Error_Rate +Reallocated_Sector_Ct +Temperature_Celsius +``` + +If any attribute's raw value (column 9 of `smartctl -A` output) exceeds its threshold, the medium is reported as `PHYMED_HEALTH_FAILING`; otherwise `PHYMED_HEALTH_OK`. + +--- + +## Error/Health Code Mapping + +| Code | Meaning | +|------|---------| +| `PHYMED_HEALTH_OK (101)` | All SMART attributes within threshold | +| `PHYMED_HEALTH_FAILING (102)` | At least one SMART attribute exceeded | +| `PHYMED_HEALTH_ERROR (103)` | `smartctl` command execution failed | +| `PHYMED_HEALTH_INVALID (100)` | Device not found or unknown state | + +--- + +## Known Issues and Gaps + +### Gap 1 — High: Instance hash key encoding limits instances to 99 physical media per storage service + +**File**: `Service_Storage_PhyMedium.cpp` + +**Observation**: The hash key is computed as: + +```cpp +g_hash_table_insert(phyMedHash, + (gpointer)((storageServiceInstance * 100) + phyMedInstance), pRet); +``` + +If `storageServiceInstance > 1` and `phyMedInstance > 99`, the key has the same value as a different `(storageServiceInstance, phyMedInstance)` pair. More critically, `closeInstance()` removes by `pDev->dev_id` alone: + +```cpp +g_hash_table_remove(phyMedHash, (gconstpointer)pDev->dev_id); +``` + +This removes the wrong entry: it looks up by `phyMedInstance` only, not by the full composite key. + +**Impact**: `closeInstance()` removes the wrong hash entry, leaking the actual instance and leaving a dangling or incorrect instance in the hash. + +**Recommended fix**: Store the composite key in the instance and use it in `closeInstance()`. + +--- + +### Gap 2 — High: `getLock()` uses `g_mutex_new()` lazy initialization without synchronization + +**File**: `Service_Storage_PhyMedium.cpp` + +**Observation**: + +```cpp +void hostIf_PhysicalMedium::getLock() +{ + if(!m_mutex) + { + m_mutex = g_mutex_new(); + } + g_mutex_lock(m_mutex); +} +``` + +The check-and-create pattern is not atomic. Two threads can both observe `m_mutex == NULL` and both call `g_mutex_new()`, creating two separate mutexes. One is stored, the other is leaked, and the first caller's critical section is left unprotected. + +--- + +### Gap 3 — Medium: Every GET spawns one or more `udevadm`/`fdisk`/`smartctl` subprocesses + +**Observation**: There is no caching. Each GET call for `Vendor`, `Model`, `SerialNumber`, `Capacity`, or `Health` spawns a fresh subprocess. For a device with multiple attached drives, a simultaneous ACS bulk GET spawns many processes in rapid succession. `smartctl` is particularly slow (≥1 second per disk for full SMART scan). + +**Impact**: An ACS bulk GET poll can block the parameter handler threads for multiple seconds and cause visible CPU spikes. + +**Recommended fix**: Cache enumeration results with a configurable TTL (e.g., 30 seconds for static attributes like Model/SerialNumber, 5 minutes for SMART health). + +--- + +### Gap 4 — Medium: SMART health check uses `egrep` with raw string concatenation + +**File**: `Service_Storage_PhyMedium.cpp` + +**Observation**: + +```cpp +#define CMD_TO_CHECK_SMART_HEALTH "smartctl -A %s | egrep \"%s\" | awk 'BEGIN {ORS=\",\"} {print $9}'" +``` + +The `%s` format for both the device path and the SMART parameter list is used with `v_secure_popen`. The device path (`/dev/sdX`) is derived from `fdisk -l` output. If a disk device name contains special characters (e.g., spaces or shell metacharacters), this would become a shell injection vulnerability even with `v_secure_popen`, unless `v_secure_popen` strictly validates format arguments. + +**Recommended fix**: Always validate that disk names match `/dev/sd[a-z][0-9]?` before using them in format strings. + +--- + +### Gap 5 — Low: No unit tests + +**Observation**: The StorageService profile has no `gtest/` subdirectory. The discovery logic (`rebuildHash`, SMART parsing, udevadm output parsing) has no automated coverage. + +--- + +### Gap 6 — Low: `CMD_TO_GET_MED_NUM` and `CMD_TO_GET_MED_NAME` exclude `mmcblk` devices + +**Observation**: + +```cpp +#define CMD_TO_GET_MED_NUM "fdisk -l | grep Disk | egrep -v \"mtdblock|mmcblk\"| wc -l" +``` + +eMMC and SD cards (which show as `mmcblk*`) are excluded from this count. This is intentional to avoid double-counting devices already exposed by the STBService eMMC/SDCard profiles. However, if a device has an external USB eMMC reader that presents as `sdb`, it will be included and might be misclassified. + +--- + +## Testing + +There are currently no unit tests. When adding coverage: +1. Mock `v_secure_popen()` to return synthetic `fdisk -l` and `smartctl -A` output. +2. Test `rebuildHash()` with 0, 1, and multiple disk scenarios. +3. Test SMART health classification (FAILING vs OK vs ERROR). +4. Test hash key/removal correctness for the composite key. + +--- + +## See Also + +- [STBService/docs/README.md](../../STBService/docs/README.md) — eMMC and SD card health (via rdkStorageMgr) +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview diff --git a/src/hostif/profiles/Time/docs/README.md b/src/hostif/profiles/Time/docs/README.md new file mode 100644 index 000000000..56ef3bb62 --- /dev/null +++ b/src/hostif/profiles/Time/docs/README.md @@ -0,0 +1,290 @@ +# Time Profile + +## Overview + +The Time profile implements the TR-181 `Device.Time.*` object. It provides GET and SET access to the system clock, local/UTC time, timezone, and the Chrony NTP client configuration through a set of RFC-controlled flag files under `/opt/secure/RFC/chrony/`. Standard TR-181 NTP server parameters (`NTPServer1`–`NTPServer5`, `Enable`, `Status`) are declared in the class but return `NOK` — active NTP server management is handled exclusively through the Chrony-specific extension parameters. + +Bootstrap store integration via `XBSStore` allows ACS to read/write NTP configuration values that are partner-specific. + +--- + +## Directory Structure + +``` +src/hostif/profiles/Time/ +├── Device_Time.cpp # Full implementation (640 lines) +├── Device_Time.h # Class declaration with all parameter methods +├── Makefile.am +└── gtest/ + ├── gtest_time.cpp # Unit tests (143 lines) + └── Makefile.am +``` + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET/SET Device.Time.*| DISP[hostIf_msgHandler] + DISP --> TIME[hostIf_Time::getInstance\ndev_id] + TIME --> LOCALTIME[get_Device_Time_CurrentLocalTime\ntime + localtime] + TIME --> UTCTIME[get_Device_Time_CurrentUTCTime\ntime + gmtime] + TIME --> TZ[get_Device_Time_LocalTimeZone\ngettimeofday + strftime %Z] + TIME --> CHRONY[Chrony RFC files\n/opt/secure/RFC/chrony/*] + TIME --> BSSTORE[XBSStore::getValue\nBootstrap store NTP values] + + subgraph ChronyFiles[Chrony RFC flag files] + CHENABLE[chronyd_enabled] + NMINPOLL[ntp_minpoll] + NMAXPOLL[ntp_maxpoll] + NMAXSTEP[ntp_maxstep] + NDIR1[ntp_server1_directive] + NDIR2[ntp_server2_directive] + NDIR3[ntp_server3_directive] + NDIR4[ntp_server4_directive] + NDIR5[ntp_server5_directive] + end + + CHRONY --> ChronyFiles +``` + +--- + +## TR-181 Parameter Coverage + +### Standard `Device.Time.*` + +| Parameter | GET | SET | Notes | +|-----------|-----|-----|-------| +| `Enable` | ❌ (returns NOK) | ❌ (returns NOK) | Not implemented | +| `Status` | ❌ (returns NOK) | — | Not implemented | +| `NTPServer1` | ❌ (returns NOK) | ❌ (returns NOK) | Not implemented (see Gap 1) | +| `NTPServer2` | ❌ (returns NOK) | ❌ (returns NOK) | Not implemented | +| `NTPServer3` | ❌ (returns NOK) | ❌ (returns NOK) | Not implemented | +| `NTPServer4` | ❌ (returns NOK) | ❌ (returns NOK) | Not implemented | +| `NTPServer5` | ❌ (returns NOK) | ❌ (returns NOK) | Not implemented | +| `CurrentLocalTime` | ✅ | — | `time` + `localtime` + `strftime` | +| `CurrentUTCTime` | ✅ | — | `time` + `gmtime` + `strftime` | +| `LocalTimeZone` | ✅ | ❌ (returns NOK) | `%Z` from `strftime` | +| `LocalTimeZoneName` | ❌ | ❌ | Not implemented | + +### RDK-Specific Chrony Extension Parameters + +All values stored in `/opt/secure/RFC/chrony/` flag files: + +| Parameter | GET | SET | Flag File | +|-----------|-----|-----|-----------| +| `X_RDKCENTRAL-COM_ChronyEnable` | ✅ | ✅ | `chronyd_enabled` (existence check) | +| `X_RDKCENTRAL-COM_NTPMinpoll` | ✅ | ✅ | `ntp_minpoll` (integer 4–24) | +| `X_RDKCENTRAL-COM_NTPMaxpoll` | ✅ | ✅ | `ntp_maxpoll` (integer 4–24) | +| `X_RDKCENTRAL-COM_NTPMaxstep` | ✅ | ✅ | `ntp_maxstep` (float,retries e.g. "1.0,3") | +| `X_RDKCENTRAL-COM_NTPServer1Directive` | ✅ | ✅ | `ntp_server1_directive` ("server"/"pool"/"peer") | +| `X_RDKCENTRAL-COM_NTPServer2Directive` | ✅ | ✅ | `ntp_server2_directive` | +| `X_RDKCENTRAL-COM_NTPServer3Directive` | ✅ | ✅ | `ntp_server3_directive` | +| `X_RDKCENTRAL-COM_NTPServer4Directive` | ✅ | ✅ | `ntp_server4_directive` | +| `X_RDKCENTRAL-COM_NTPServer5Directive` | ✅ | ✅ | `ntp_server5_directive` | + +### Bootstrap Store Parameters + +Parameters prefixed with `Device.Time.X_RDKCENTRAL-COM_xBSS.*` or similar (partner-specific) are routed through `XBSStore::getValue` and `XBSStore::overrideValue`. These include NTP server URL defaults from `partners_defaults.json`. + +--- + +## How Operations Work + +### GET CurrentLocalTime + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch + participant Time as hostIf_Time + + ACS->>Dispatch: GET Device.Time.CurrentLocalTime + Dispatch->>Time: get_Device_Time_CurrentLocalTime(stMsgData) + Time->>Time: time(&rawtime) + Time->>Time: timeinfo = localtime(&rawtime) + Time->>Time: strftime(buffer, "%Y-%m-%dT%H:%M:%S", timeinfo) + Time->>Time: strftime(timeZoneTmp, "%z", timeinfo) → "+0530" + Time->>Time: snprintf(buffer + len, ".%06d%s", timeinfo->tm_sec, timeZoneTmp) + Time->>Time: strcpy_s(stMsgData->paramValue, buffer) + Time-->>Dispatch: OK + Dispatch-->>ACS: "2026-03-19T14:30:00.000030+0530" +``` + +### SET Chrony Enable Flow + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch + participant Time as hostIf_Time + participant FS as /opt/secure/RFC/chrony/ + + ACS->>Dispatch: SET X_RDKCENTRAL-COM_ChronyEnable = "true" + Dispatch->>Time: set_Device_Time_Chrony_Enable(stMsgData) + Time->>Time: getStringValue(stMsgData) → "true" + Time->>FS: mkdir("/opt/secure/RFC/chrony", 0755) if not exists + Time->>FS: ofstream(CHRONY_ENABLE_FILE) << "true" + Time-->>Dispatch: OK + Dispatch-->>ACS: success +``` + +For `ChronyEnable = "false"`: The flag file is removed with `std::remove()`. Chrony daemon reads the flag file presence on restart. + +### NTP Poll Interval SET Validation + +`set_Device_Time_NTPMinpoll()` and `set_Device_Time_NTPMaxpoll()` validate that the integer is in the NTP-allowed power-of-2 exponent range [4, 24]: + +```cpp +int minpoll = atoi(minpollStr.c_str()); +if (minpoll < 4 || minpoll > 24) { + return NOK; // Invalid range +} +``` + +--- + +## Change Detection + +`CurrentLocalTime`, `CurrentUTCTime`, and `LocalTimeZone` use the standard backup pattern: +- `bCalledCurrentLocalTime`, `bCalledCurrentUTCTime`, `bCalledLocalTimeZone` flags +- `backupCurrentLocalTime`, `backupCurrentUTCTime`, `backupLocalTimeZone` arrays +- `*pChanged = true` when the formatted time string differs from backup + +Since `CurrentLocalTime` changes every second, the notification system will fire on every poll update cycle. + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `NTPServer1`–`NTPServer5` GET called | Returns `NOK` unconditionally | +| `Enable`, `Status` GET called | Returns `NOK` unconditionally | +| Chrony directory creation fails | Logs error, returns `NOK` | +| Chrony flag file open fails | Logs error, returns `NOK` | +| `std::remove()` fails (not ENOENT) | Logs warning, returns `OK` (best-effort) | +| NTP poll value out of range [4,24] | Logs error, returns `NOK` | +| `ERR_CHK(rc)` on `strcpy_s` failure | Logs internally; does not return `NOK` | + +--- + +## Known Issues and Gaps + +### Gap 1 — Critical: Standard TR-181 `NTPServer1`–`NTPServer5` GET and SET both return `NOK` + +**File**: `Device_Time.cpp` + +**Observation**: + +```cpp +int hostIf_Time::get_Device_Time_NTPServer1(HOSTIF_MsgData_t *, bool *pChanged) { return NOK; } +int hostIf_Time::set_Device_Time_NTPServer1(HOSTIF_MsgData_t* stMsgData) { return NOK; } +// ... same for NTPServer2 through NTPServer5 +``` + +All five standard TR-181 NTP server parameters are declared in the class but never implemented. Any ACS that follows the TR-181 standard and tries to read or configure NTP servers via `Device.Time.NTPServer*` receives an error response. The only supported path is the RDK-specific Chrony extension `NTPServerNDirective` parameters. + +**Impact**: ACS systems that use the standard TR-181 `Device.Time.NTPServer*` parameters cannot manage the device's NTP configuration. Only ACS systems that are specifically aware of the RDK Chrony extension parameters can manage NTP. + +--- + +### Gap 2 — High: `getLock()` calls `g_mutex_init()` on every invocation + +**File**: `Device_Time.cpp` + +**Observation**: + +```cpp +void hostIf_Time::getLock() +{ + g_mutex_init(&hostIf_Time::m_mutex); // re-initializes on every call + g_mutex_lock(&hostIf_Time::m_mutex); +} +``` + +Re-initializing an already-initialized and possibly locked mutex is undefined behavior. See the same gap described in the Ethernet, STBService, and InterfaceStack profiles. + +--- + +### Gap 3 — High: `get_Device_Time_CurrentLocalTime` appends `tm_sec` instead of microseconds + +**File**: `Device_Time.cpp` + +**Observation**: + +```cpp +strftime(buffer, _BUF_LEN_64-1, "%Y-%m-%dT%H:%M:%S", timeinfo); +snprintf(buffer + strlen(buffer), (sizeof(buffer) - strlen(buffer)), + ".%.6d%s", timeinfo->tm_sec, timeZoneTmp); +``` + +The format `".%.6d%s"` with `timeinfo->tm_sec` appends the current seconds (0–59) as a 6-digit zero-padded number after the decimal point. This produces values like `"2026-03-19T14:30:30.000030+0530"` — `30` microseconds when the actual intent was to show sub-second fractional time. The correct value is the microseconds field from `gettimeofday()` (`tv_usec`). + +**Impact**: The fractional second in `CurrentLocalTime` is completely wrong. It ranges from `.000000` to `.000059` based on the current second, not the actual microsecond offset. + +**Recommended fix**: +```cpp +struct timeval tv; +gettimeofday(&tv, NULL); +struct tm *timeinfo = localtime(&tv.tv_sec); +strftime(buffer, sizeof(buffer)-1, "%Y-%m-%dT%H:%M:%S", timeinfo); +snprintf(buffer + strlen(buffer), sizeof(buffer) - strlen(buffer), + ".%06ld%s", (long)tv.tv_usec, timeZoneTmp); +``` + +--- + +### Gap 4 — Medium: Chrony configuration files do not directly reconfigure the running Chrony daemon + +**File**: `Device_Time.cpp` + +**Observation**: The SET handlers write values to flag files under `/opt/secure/RFC/chrony/`. These files are read by a separate script that regenerates the Chrony configuration file (`/etc/chrony/chrony.conf`). The daemon itself is not signaled or restarted after the SET operation. An ACS SET of `NTPMinpoll` is not applied until the next Chrony restart, which might not happen until the next reboot. + +**Impact**: SET operations appear to succeed (return `OK`) but have no immediate effect on the running NTP synchronization behavior. + +--- + +### Gap 5 — Medium: NTP poll validation range [4,24] deviates from the Chrony documentation + +**Observation**: The NTP-recommended poll range per RFC 5905 and Chrony documentation is 4–17 (for a value where the actual poll interval is 2^N seconds). The code comment says `[4, 17]` but the actual validation allows up to 24: + +```cpp +// Validate that minpollStr is a number in a valid range [4, 17] for NTP +int minpoll = atoi(minpollStr.c_str()); +if (minpoll < 4 || minpoll > 24) { // range in code is 4..24, not 4..17 +``` + +The comment and the code disagree. Values 18–24 are accepted but result in poll intervals of 2^18 (3 days) to 2^24 (194 days), which are not practical NTP poll settings. + +--- + +### Gap 6 — Low: `NTPMaxstep` SET does not validate the "float,retries" format + +**Observation**: The `NTPMaxstep` parameter is expected to be in the format `","` (e.g., `"1.0,3"`). The SET handler writes the raw string to the flag file without validating the format. An invalid value like `"abc"` is silently written and would cause Chrony to fail parsing on restart. + +--- + +## Testing + +Unit tests are in `gtest/gtest_time.cpp` (143 lines). Run: + +```bash +./run_ut.sh +``` + +Key test areas: +1. `CurrentLocalTime` format: verify the ISO 8601 format with timezone offset. +2. `LocalTimeZone`: verify abbreviation (e.g., "UTC", "EST") is returned. +3. Chrony enable: verify flag file creation and removal. +4. NTP poll validation: boundary tests at 3 (should fail), 4 (should pass), 24 (should pass), 25 (should fail). + +--- + +## See Also + +- [DeviceInfo/docs/README.md](../../DeviceInfo/docs/README.md) — XBSStore for NTP URL partner defaults +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview +- [handlers/docs/README.md](../../../handlers/docs/README.md) — Dispatch layer diff --git a/src/hostif/profiles/moca/docs/README.md b/src/hostif/profiles/moca/docs/README.md new file mode 100644 index 000000000..5ebd5f5ef --- /dev/null +++ b/src/hostif/profiles/moca/docs/README.md @@ -0,0 +1,301 @@ +# MoCA Profile + +## Overview + +The MoCA (Multimedia over Coax Alliance) profile implements the TR-181 `Device.MoCA.Interface.{i}.*` object tree. It exposes the MoCA network state — node identity, PHY/MAC parameters, associated device table, QoS flow statistics, and the RDK unicast mesh rate table — through the RMH (RDK MoCA HAL) API (`rdk_moca_hal.h`). The profile uses a singleton `MoCADevice` to own the `RMH_Handle` and a singleton `MoCAInterface` for the TR-181 parameter handler, both backed by a `std::mutex`-protected RMH context. + +--- + +## Directory Structure + +``` +src/hostif/profiles/moca/ +├── Device_MoCA_Interface.cpp # Core interface (1,268 lines) +├── Device_MoCA_Interface.h # Classes MoCADevice + MoCAInterface +├── Device_MoCA_Interface_AssociatedDevice.cpp # Associated node table +├── Device_MoCA_Interface_AssociatedDevice.h +├── Device_MoCA_Interface_QoS.cpp # QoS flow counts +├── Device_MoCA_Interface_QoS.h +├── Device_MoCA_Interface_QoS_FlowStats.cpp # Per-flow statistics +├── Device_MoCA_Interface_QoS_FlowStats.h +├── Device_MoCA_Interface_Stats.cpp # Interface throughput stats +├── Device_MoCA_Interface_Stats.h +├── Device_MoCA_Interface_X_RDKCENTRAL_COM_MeshTable.cpp # Unicast PHY rate mesh +├── Device_MoCA_Interface_X_RDKCENTRAL_COM_MeshTable.h +└── Makefile.am +``` + +> **Note**: There is no `gtest/` subdirectory. The MoCA profile has no unit tests. + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA] -->|GET Device.MoCA.Interface.*| DISP[hostIf_msgHandler] + + DISP --> IFACE[MoCAInterface::getInstance\nSingleton for dev_id 0] + DISP --> ASSOC[MoCAInterfaceAssociatedDevice] + DISP --> QOS[MoCAInterfaceQoS] + DISP --> FLOW[MoCAInterfaceQoSFlowStats] + DISP --> STATS[MoCAInterfaceStats] + DISP --> MESH[MoCAInterfaceMeshTable] + + subgraph RMH[RMH Handle Management - MoCADevice singleton] + DEV[MoCADevice::getRmhContext] + DEV --> LOCK[std::lock_guard m_mutex] + LOCK --> DESTROY[RMH_Destroy if alwayRecreate=true] + DESTROY --> INIT[RMH_Initialize loop up to 10s] + INIT --> HANDLE[RMH_Handle] + end + + IFACE --> RMH + ASSOC --> RMH + QOS --> RMH + FLOW --> RMH + STATS --> RMH + MESH --> RMH +``` + +--- + +## TR-181 Parameter Coverage + +### `Device.MoCA.Interface.{i}` + +| Parameter | GET | Notes | +|-----------|-----|-------| +| `Enable` | ✅ | `RMH_Interface_GetEnabled` | +| `Status` | ✅ | `RMH_Network_GetStatus` → "Up"/"Down"/"Error" | +| `Alias` | ✅ | Constructed from dev_id | +| `Name` | ✅ | `RMH_Interface_GetName` | +| `LastChange` | ✅ | `RMH_Interface_GetLastChange` | +| `LowerLayers` | ✅ | Derived from interface name | +| `Upstream` | ✅ | Fixed `false` (MoCA is downstream) | +| `MACAddress` | ✅ | `RMH_Interface_GetMacAddress` | +| `FirmwareVersion` | ✅ | `RMH_Interface_GetFirmwareVersion` | +| `MaxBitRate` | ✅ | `RMH_Interface_GetMaxEgressBW` | +| `MaxIngressBW`, `MaxEgressBW` | ✅ | RMH BW queries | +| `HighestVersion`, `CurrentVersion` | ✅ | `RMH_Network_GetMoCAVersion` | +| `NetworkCoordinator` | ✅ | `RMH_Network_GetNCNodeId` | +| `NodeID` | ✅ | `RMH_Self_GetNodeId` | +| `BackupNC` | ✅ | `RMH_Network_GetBackupNCNodeId` | +| `PrivacyEnabledSetting`, `PrivacyEnabled` | ✅ | `RMH_Privacy_GetEnabled` | +| `CurrentOperFreq`, `LastOperFreq` | ✅ | `RMH_Network_GetRFChannelFreq` | +| `TxPowerLimit` | ✅ | `RMH_Power_GetTxPowerLimit` | +| `TxBcastRate` | ✅ | `RMH_Network_GetTxBroadcastPhyRate` | +| `AssociatedDeviceNumberOfEntries` | ✅ | `RMH_Network_GetAssociatedIds` | +| `X_RDKCENTRAL-COM_MeshTableNumberOfEntries` | ✅ | Computed: N² - N for N nodes | + +### `Device.MoCA.Interface.{i}.AssociatedDevice.{j}` + +| Parameter | GET | +|-----------|-----| +| `MACAddress` | ✅ | +| `NodeID` | ✅ | +| `IsPreferredNC` | ✅ | +| `PHYTxRate`, `PHYRxRate` | ✅ | +| `TxPowerControlReduction` | ✅ | +| `RxPowerLevel` | ✅ | +| `RxBcastPowerLevel`, `RxBcastRate` | ✅ | +| `PacketAggregationCapability` | ✅ | +| `RxSNR` | ✅ | +| `Active` | ✅ | + +### `Device.MoCA.Interface.{i}.Stats` + +All stats use `RMH_Stats_GetTx*` and `RMH_Stats_GetRx*`: BytesSent, BytesReceived, PacketsSent, PacketsReceived, ErrorsSent, ErrorsReceived, UnicastPackets, MulticastPackets, BroadcastPackets, Discards, UnknownProtoPackets. + +### `Device.MoCA.Interface.{i}.X_RDKCENTRAL-COM_MeshTable.{j}` + +| Parameter | GET | +|-----------|-----| +| `MeshTxNodeId` | ✅ | +| `MeshRxNodeId` | ✅ | +| `MeshPHYTxRate` | ✅ | + +--- + +## How Operations Work + +### RMH Handle Acquisition + +Every GET operation calls `MoCADevice::getRmhContext()` to obtain an `RMH_Handle`. The current implementation unconditionally destroys and recreates the handle on every call: + +```mermaid +sequenceDiagram + participant Handler as GET Handler + participant MoCA as MoCADevice + participant RMH as RMH Library + + Handler->>MoCA: getRmhContext() + MoCA->>MoCA: std::lock_guard lock(m_mutex) + MoCA->>RMH: RMH_Destroy(existing handle) [alwayRecreate=true] + MoCA->>RMH: RMH_Initialize(NULL, NULL) + alt MoCA daemon ready + RMH-->>MoCA: new RMH_Handle + else MoCA daemon not yet ready + loop retry up to 10 seconds + MoCA->>MoCA: usleep(1 000 000) + MoCA->>RMH: RMH_Initialize(NULL, NULL) + end + end + MoCA-->>Handler: RMH_Handle + Handler->>RMH: RMH__Get(handle, ...) + RMH-->>Handler: result +``` + +### Associated Device Enumeration + +`get_Associated_Device_NumberOfEntries()` calls `RMH_Network_GetAssociatedIds()` which returns a `RMH_NodeList_Uint32_t` bitmask. The code iterates all 16 possible node IDs and counts those with `nodePresent[nodeId] == true`. + +### Mesh Table Calculation + +`get_MoCA_Mesh_NumberOfEntries()` derives the entry count from the node count N using the formula `N² - N` (number of directed edges in a complete graph, excluding self-edges). This represents all possible unicast PHY rate pairs. + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `RMH_Initialize` fails all retries | Returns `NULL` handle; all subsequent RMH calls skipped; GET returns `NOK` | +| `RMH__Get` returns non-SUCCESS | Logs error with `RMH_ResultToString(ret)`, returns `NOK` | +| `RMH_UNIMPLEMENTED` / `RMH_NOT_SUPPORTED` | Logs warning, continues; handle considered valid | +| `m_mutex == NULL` (lazy init) | `getLock()` calls `g_mutex_new()` — race condition (see Gap 2) | + +--- + +## Known Issues and Gaps + +### Gap 1 — Critical: `alwayRecreate = true` destroys and recreates the RMH handle on every GET request + +**File**: `Device_MoCA_Interface.cpp` — `MoCADevice::getRmhContext()` + +**Observation**: + +```cpp +bool alwayRecreate = true; /* XITHREE-7905 */ + +if (alwayRecreate && rmhContext) { + RMH_Destroy(rmhContext); + rmhContext = NULL; +} +``` + +The workaround for JIRA issue XITHREE-7905 unconditionally destroys and recreates the RMH handle before every use. `RMH_Destroy` tears down the MoCA HAL connection, and `RMH_Initialize` re-establishes it. When MoCA is ready, this adds significant latency (HAL init overhead) to every single GET request. When MoCA is slow to respond, it may block up to 10 seconds with `usleep(1 000 000)` retry loops — holding `m_mutex` the entire time. + +**Impact**: +- Every MoCA parameter GET takes at minimum the HAL init round-trip time. +- During MoCA network join (slow path), a single GET can block for up to 10 seconds. +- The `m_mutex` is held during the entire blocking retry loop, serializing all other MoCA requests. + +--- + +### Gap 2 — High: `getLock()` uses `g_mutex_new()` lazy initialization without synchronization + +**File**: `Device_MoCA_Interface.cpp` + +**Observation**: + +```cpp +void MoCAInterface::getLock() +{ + if(!m_mutex) + { + m_mutex = g_mutex_new(); + } + g_mutex_lock(m_mutex); +} +``` + +This is the same race condition documented in DHCPv4, Ethernet, and StorageService profiles. Two callers can simultaneously observe `m_mutex == NULL` and create two separate mutexes. + +--- + +### Gap 3 — High: `closeRmhContext()` has no return statement despite returning `void*` + +**File**: `Device_MoCA_Interface.cpp` + +**Observation**: + +```cpp +void* MoCADevice::closeRmhContext() { + RMH_Handle rmhContext = (RMH_Handle)getRmhContext(); + if(rmhContext) { + RMH_Destroy(rmhContext); + } + // No return statement! Return type is void* +} +``` + +The function signature returns `void*` but the function body has no `return` statement. This is undefined behavior in C++. The function should return `void` (no return value) or return `NULL`. + +--- + +### Gap 4 — High: `MoCAInterface::getInstance()` ignores `_dev_Id` and always returns instance 0 + +**File**: `Device_MoCA_Interface.cpp` + +**Observation**: + +```cpp +MoCAInterface* MoCAInterface::getInstance(int _dev_Id) +{ + if(NULL == Instance) { + Instance = new MoCAInterface(0); // Always creates with dev_id=0 + } + return Instance; // Always returns the same singleton +} +``` + +Regardless of the `_dev_Id` argument, the same singleton is returned. On a device with multiple MoCA interfaces, all GET requests land on instance 0 and read results for the same hardware interface. + +**Impact**: `Device.MoCA.Interface.2.*` returns exactly the same values as `Device.MoCA.Interface.1.*`. + +--- + +### Gap 5 — Medium: Mesh table entry count calculation uses `N² - N` which overcounts for asymmetric topologies + +**Observation**: The formula `N² - N` counts all directed entries in a complete graph (every node can reach every other node). In a real MoCA network, not all unicast paths have measured PHY rates — some node pairs may not have communicated. The actual `MeshTable` entries returned by `RMH_MeshTable_GetRxEntries()` may be fewer than `N² - N`. + +**Impact**: `MeshTableNumberOfEntries` overestimates the actual number of entries. ACS may request instances beyond what the HAL returns. + +--- + +### Gap 6 — Low: No unit tests + +**Observation**: There is no `gtest/` directory. The profile has 2,418 lines of C++ covering complex RMH HAL interactions with no automated test coverage. + +--- + +## Testing + +There are currently no unit tests. When adding coverage: +1. Create a mock `rdk_moca_hal.h` with stub implementations. +2. Test `getRmhContext()` retry behavior with a mock that fails N times before succeeding. +3. Test `AssociatedDeviceNumberOfEntries` with mock `RMH_NodeList_Uint32_t` values. +4. Test `MeshTableNumberOfEntries` computation for N=2, 3, 4 nodes. + +--- + +## Platform Notes + +### RMH HAL Dependency + +The MoCA profile requires `librdk_moca_hal.so` at runtime. On non-MoCA platforms (devices without coaxial MoCA network), `RMH_Initialize()` will always fail and all MoCA GET parameters return `NOK`. + +### Build Guard + +The MoCA profile is compiled when `USE_MoCA_PROFILE` is defined. When not defined: +- `Device.MoCA.Interface.*` parameters return `NOT_HANDLED` +- InterfaceStack profile skips MoCA lower-layer entries + +--- + +## See Also + +- [InterfaceStack/docs/README.md](../../InterfaceStack/docs/README.md) — MoCA as a lower-layer interface +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview +- [handlers/docs/README.md](../../../handlers/docs/README.md) — Dispatch layer diff --git a/src/hostif/profiles/wifi/docs/README.md b/src/hostif/profiles/wifi/docs/README.md new file mode 100644 index 000000000..9075a8f8a --- /dev/null +++ b/src/hostif/profiles/wifi/docs/README.md @@ -0,0 +1,345 @@ +# WiFi Profile + +## Overview + +The WiFi profile implements the TR-181 `Device.WiFi.*` object tree, covering the complete 802.11 management hierarchy: top-level counts, radio configuration and statistics, SSID interface state, access point management (WPS, security, associated clients), and client endpoint profiles. On RDK-V builds (`RDKV_NM`), all data comes from the `IARM_BUS_NM_SRV_MGR_NAME` WiFi manager via IARM Bus calls. On non-RDKV builds, data is fetched from the WPEFramework Thunder plugin via libcurl JSON-RPC calls (`cJSON`). The entire profile is guarded by `USE_WIFI_PROFILE`. + +--- + +## Directory Structure + +``` +src/hostif/profiles/wifi/ +├── Device_WiFi.cpp # Top-level WiFi container +├── Device_WiFi.h +├── Device_WiFi_Radio.cpp # Radio physical layer config +├── Device_WiFi_Radio.h +├── Device_WiFi_Radio_Stats.cpp # Radio statistics +├── Device_WiFi_Radio_Stats.h +├── Device_WiFi_SSID.cpp # SSID interface state +├── Device_WiFi_SSID.h +├── Device_WiFi_SSID_Stats.cpp # SSID-level statistics +├── Device_WiFi_SSID_Stats.h +├── Device_WiFi_AccessPoint.cpp # AP configuration +├── Device_WiFi_AccessPoint.h +├── Device_WiFi_AccessPoint_AssociatedDevice.cpp # Per-client entries +├── Device_WiFi_AccessPoint_AssociatedDevice.h +├── Device_WiFi_AccessPoint_Security.cpp # AP security settings +├── Device_WiFi_AccessPoint_Security.h +├── Device_WiFi_AccessPoint_WPS.cpp # AP WPS configuration +├── Device_WiFi_AccessPoint_WPS.h +├── Device_WiFi_EndPoint.cpp # Client endpoint +├── Device_WiFi_EndPoint.h +├── Device_WiFi_EndPoint_Profile.cpp # EndPoint connection profile +├── Device_WiFi_EndPoint_Profile.h +├── Device_WiFi_EndPoint_Profile_Security.cpp # EndPoint security +├── Device_WiFi_EndPoint_Profile_Security.h +├── Device_WiFi_EndPoint_Security.cpp # EndPoint security modes +├── Device_WiFi_EndPoint_Security.h +├── Device_WiFi_EndPoint_WPS.cpp # EndPoint WPS +├── Device_WiFi_EndPoint_WPS.h +├── Device_WiFi_X_RDKCENTRAL_COM_ClientRoaming.cpp # Band-steering/roaming +├── Device_WiFi_X_RDKCENTRAL_COM_ClientRoaming.h +└── Makefile.am +``` + +> **Note**: There is no `gtest/` subdirectory. The WiFi profile has no unit tests. + +--- + +## Architecture + +```mermaid +graph TB + ACS[ACS / WebPA / RBUS] -->|GET/SET Device.WiFi.*| DISP[hostIf_msgHandler] + + DISP --> WIFI["hostIf_WiFi
Device.WiFi top-level"] + DISP --> RADIO["hostIf_WiFi_Radio
Device.WiFi.Radio.(i).*"] + DISP --> RADSTA["hostIf_WiFi_Radio_Stats
Device.WiFi.Radio.(i).Stats.*"] + DISP --> SSID["hostIf_WiFi_SSID
Device.WiFi.SSID.(i).*"] + DISP --> SSISTAT["hostIf_WiFi_SSID_Stats
Device.WiFi.SSID.(i).Stats.*"] + DISP --> AP["hostIf_WiFi_AccessPoint
Device.WiFi.AccessPoint.(i).*"] + DISP --> ASSOC["hostIf_WiFi_AccessPoint_AssociatedDevice
Device.WiFi.AccessPoint.(i).AssociatedDevice.(j)"] + DISP --> APSEC["hostIf_WiFi_AccessPoint_Security
Device.WiFi.AccessPoint.(i).Security.*"] + DISP --> APWPS["hostIf_WiFi_AccessPoint_WPS
Device.WiFi.AccessPoint.(i).WPS.*"] + DISP --> EP["hostIf_WiFi_EndPoint
Device.WiFi.EndPoint.(i).*"] + DISP --> ROAM["hostIf_WiFi_X_RDKCENTRAL_COM_ClientRoaming
Device.WiFi.X_RDKCENTRAL-COM_ClientRoaming.*"] + + subgraph RDKVNM[RDKV_NM build path] + IARM["IARM Bus
IARM_BUS_NM_SRV_MGR_NAME
IARM_BUS_WIFI_MGR_API_*"] + end + subgraph NONRDKV[Non-RDKV build path] + CURL["libcurl + cJSON
JSON-RPC to WPEFramework"] + end + + RADIO --> RDKVNM + RADIO --> NONRDKV + SSID --> RDKVNM + AP --> RDKVNM +``` + +--- + +## TR-181 Parameter Coverage + +### `Device.WiFi` + +| Parameter | GET (RDKV_NM) | GET (non-RDKV) | Notes | +|-----------|:---:|:---:|-------| +| `RadioNumberOfEntries` | ✅ | ❌ | IARM `IARM_BUS_WIFI_MGR_RadioEntry` | +| `SSIDNumberOfEntries` | ✅ | ❌ | IARM `IARM_BUS_WIFI_MGR_SSIDEntry` | +| `AccessPointNumberOfEntries` | ✅ | ✅ | Hardcoded `1` (non-RDKV) | +| `EndPointNumberOfEntries` | ✅ | ✅ | Hardcoded `1` | + +### `Device.WiFi.Radio.{i}` + +| Parameter | GET | Notes | +|-----------|-----|-------| +| `Enable` | ✅ | `wifi_getRadioEnable` / IARM | +| `Status` | ✅ | `wifi_getRadioEnable` → "Up"/"Down" | +| `Name` | ✅ | `wifi_getRadioIfName` | +| `SupportedFrequencyBands` | ✅ | "2.4GHz" / "5GHz" | +| `OperatingFrequencyBand` | ✅ | `wifi_getRadioOperatingFrequencyBand` | +| `SupportedStandards` | ✅ | Comma-separated list (a/b/g/n/ac) | +| `OperatingStandards` | ✅ | `wifi_getRadioStandard` | +| `PossibleChannels` | ✅ | `wifi_getRadioPossibleChannels` | +| `AutoChannelEnable` | ✅ | `wifi_getRadioAutoChannelEnable` | +| `Channel` | ✅ | `wifi_getRadioChannel` | +| `TransmitPower` | ✅ | `wifi_getRadioTransmitPower` | +| `MACAddress` | ✅ | `wifi_getRadioBaseBSSID` | +| `MaxBitRate` | ✅ | `wifi_getRadioMaxBitRate` | + +### `Device.WiFi.SSID.{i}` + +| Parameter | GET | Notes | +|-----------|-----|-------| +| `Enable` | ✅ | `wifi_getSSIDEnable` | +| `Status` | ✅ | `wifi_getSSIDStatus` | +| `Name` | ✅ | `wifi_getSSIDIfName` | +| `BSSID` | ✅ | `wifi_getBaseBSSID` | +| `MACAddress` | ✅ | `wifi_getBaseBSSID` | +| `SSID` | ✅ | `wifi_getSSIDName` | + +### `Device.WiFi.AccessPoint.{i}` + +| Parameter | GET | SET | Notes | +|-----------|-----|-----|-------| +| `Enable`, `Status` | ✅ | ✅ | IARM / HAL | +| `SSIDReference` | ✅ | ❌ | Resolved from SSID instance | +| `SSIDAdvertisementEnabled` | ✅ | ✅ | Beacon SSID visibility | +| `WMMEnable` | ✅ | ✅ | WMM QoS | +| `AssociatedDeviceNumberOfEntries` | ✅ | ❌ | Count of connected clients | + +### `Device.WiFi.X_RDKCENTRAL-COM_ClientRoaming` + +| Parameter | GET | SET | Notes | +|-----------|-----|-----|-------| +| `Enable` | ✅ | ✅ | Band-steering global enable | +| `PreAssn5GProbeRetryLimit` | ✅ | ✅ | Pre-association retries before steering | +| `PreAssn5GProbeMinRSSI` | ✅ | ✅ | Min RSSI threshold to steer to 5GHz | +| `PostAssnLevelDeltaConnected` | ✅ | ✅ | Signal delta to trigger roam | +| `PostAssnLevelDeltaDisconnected` | ✅ | ✅ | Signal delta after disconnect | +| And many more 5G/2G roaming parameters | ✅ | ✅ | Full band-steering configuration set | + +--- + +## How Operations Work + +### RDKV_NM Build Path (IARM) + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch + participant Radio as hostIf_WiFi_Radio + participant IARM as IARM Bus (WiFi Mgr) + + ACS->>Dispatch: GET Device.WiFi.Radio.1.Channel + Dispatch->>Radio: get_Device_WiFi_Radio_Channel(stMsgData) + Radio->>IARM: IARM_Bus_Call(IARM_BUS_NM_SRV_MGR_NAME,\n IARM_BUS_WIFI_MGR_API_getSSIDProps, param) + IARM-->>Radio: param.data.radioChannel + Radio->>Radio: put_uint(stMsgData->paramValue, channel) + Radio-->>Dispatch: OK + Dispatch-->>ACS: channel number +``` + +### Non-RDKV Build Path (JSON-RPC via WPEFramework) + +```mermaid +sequenceDiagram + participant ACS + participant Dispatch + participant Radio as hostIf_WiFi_Radio + participant CURL as libcurl + participant Thunder as WPEFramework Thunder + + ACS->>Dispatch: GET Device.WiFi.Radio.1.Channel + Dispatch->>Radio: get_Device_WiFi_Radio_Channel(stMsgData) + Radio->>CURL: getJsonRPCData(JSONRPC_URL, method="getChannel") + CURL->>Thunder: HTTP POST JSON-RPC request + Thunder-->>CURL: JSON response + CURL-->>Radio: parsed channel value + Radio->>Radio: put_uint(stMsgData->paramValue, channel) + Radio-->>Dispatch: OK + Dispatch-->>ACS: channel number +``` + +--- + +## Instance Lifecycle + +```mermaid +flowchart LR + GET["GET request
dev_id"] --> IFHASH[("ifHash
GHashTable")] + IFHASH -->|hit| RET[return cached instance] + IFHASH -->|miss| NEW["new hostIf_WiFi_*
dev_id"] + NEW --> IFHASH + RET --> HAL["Call IARM / JSON-RPC
per parameter"] +``` + +--- + +## Error Handling + +| Condition | Behavior | +|-----------|----------| +| `USE_WIFI_PROFILE` not defined | Entire profile excluded from build | +| IARM Bus call fails | Logs with IARM result code, returns `NOK` | +| JSON-RPC returns empty string | Returns `NOK`; paramValue empty | +| `cJSON_Parse` fails | Returns `NOK` | +| WiFi HAL function not available | Returns `NOK` | +| `WiFiDevice` constructor throws 1 | `getInstance()` catches, logs, returns `NULL` | + +--- + +## Known Issues and Gaps + +### Gap 1 — Critical: `WiFiDevice::ctxt` is uninitialized — constructor always throws + +**File**: `Device_WiFi.cpp` + +**Observation**: The `WiFiDevice` constructor: + +```cpp +WiFiDevice::WiFiDevice(int dev_id):dev_id(dev_id) +{ + // ctxt = WiFiCtl_Open(interface); // COMMENTED OUT + + if(!ctxt) // ctxt is uninitialized — always NULL + { + RDK_LOG(RDK_LOG_ERROR, ..., "Error! Unable to connect to WiFi Device instance %d\n", dev_id); + throw 1; + } +} +``` + +`ctxt` is never assigned (the initialization call is commented out). Since an uninitialized pointer is non-NULL on some platforms, this may or may not throw. But the subsequent `getContext()` returns the garbage pointer, which is then passed to the HAL. On platforms that zero-initialize global/static data, `ctxt == NULL`, and the constructor always throws, making `WiFiDevice` completely unusable. + +**Impact**: `WiFiDevice::getInstance()` catches the exception and inserts `NULL` into `devHash`. Any caller that dereferences the returned `WiFiDevice*` will crash. + +**Note**: The actual WiFi data path in many builds bypasses `WiFiDevice` entirely and goes directly via IARM or JSON-RPC. But `WiFiDevice` is still created during initialization. + +--- + +### Gap 2 — High: `WiFiDevice::init()` returns 1 for success, conflicting with its own comment + +**File**: `Device_WiFi.cpp` + +**Observation**: + +```cpp +//------------------------------------------------------------------------------ +// init: Returns 0 on success, -1 on failure. +//------------------------------------------------------------------------------ +int WiFiDevice::init() +{ + // Initialise the WiFi HAL + // ... (commented out) ... + return 1; // BUG: returns 1, comment says 0 is success +} +``` + +The comment documents `0` as success and `-1` as failure, but the function returns `1`. Callers that check `if (ret != 0) → error` would treat this successful return as an error. + +--- + +### Gap 3 — High: Non-RDKV build path relies on `getJsonRPCData()` which always returns an empty string + +**Observation**: The non-`RDKV_NM` build path uses `getJsonRPCData()` from `hostIf_utils.cpp` for retrieving WiFi parameters from WPEFramework. As documented in [src/hostif/docs/README.md](../../../docs/README.md#gap-8), `getJsonRPCData()` always returns an empty string because `writeCurlResponse()` takes its accumulation buffer by value. All non-RDKV WiFi GET parameters return empty or `NOK`. + +--- + +### Gap 4 — Medium: `AccessPointNumberOfEntries` and `EndPointNumberOfEntries` are hardcoded to 1 + +**File**: `Device_WiFi.cpp` (non-`RDKV_NM` build) + +**Observation**: In the `#ifndef RDKV_NM` path: + +```cpp +int hostIf_WiFi::get_Device_WiFi_AccessPointNumberOfEntries(HOSTIF_MsgData_t *stMsgData) +{ + unsigned int accessPointNumOfEntries = 1; // Always 1 + put_int(stMsgData->paramValue, accessPointNumOfEntries); + return OK; +} +``` + +Dual-band platforms with one 2.4 GHz and one 5 GHz access point (two SSIDs, two APs) return `1` instead of `2`. + +--- + +### Gap 5 — Medium: No unit tests + +**Observation**: There is no `gtest/` directory. The WiFi profile has 28 source files and 4,597+ lines of C++ with no automated test coverage. The dual build path (`RDKV_NM` vs. non-RDKV) makes testing complex. + +--- + +### Gap 6 — Medium: `ClientRoaming` SET parameters are written to HAL but the HAL API is not verified to persist them + +**File**: `Device_WiFi_X_RDKCENTRAL_COM_ClientRoaming.cpp` + +**Observation**: All SET handlers call `wifi_steering_setBandUtilizationThreshold()` or equivalent HAL functions. These functions write to an in-memory HAL state. On some RDK builds the HAL does not persist roaming parameters across reboots, and the values must be re-applied from the RFC store on every startup. If the RFC store is not also updated during the SET call, roaming configuration reverts after reboot. + +--- + +### Gap 7 — Low: `Security.PreSharedKey` and `Security.KeyPassphrase` are both exposed as readable parameters + +**File**: `Device_WiFi_AccessPoint_Security.cpp` + +**Observation**: Both `PreSharedKey` (raw hex PSK) and `KeyPassphrase` (WPA2 passphrase) are exposed via GET. Under the TR-181 specification, PSK and passphrase are write-only credentials that should not be returned to an ACS. Returning these values to any management system that can read TR-181 parameters exposes the network access credentials. + +**Recommended fix**: Return an empty string or a fixed placeholder on GET for all security credential parameters. + +--- + +## Platform Notes + +### Build Guard + +The entire WiFi profile is disabled when `USE_WIFI_PROFILE` is not defined. When disabled, `Device.WiFi.*` returns `NOT_HANDLED` for all parameters. + +### Dual Backend + +| Build Flag | Backend | Data Source | +|-----------|---------|-------------| +| `RDKV_NM` defined | IARM Bus | NM Service Manager WiFi Manager | +| `RDKV_NM` not defined | libcurl + cJSON | WPEFramework Thunder `DeviceInfo`/`WiFiManager` plugin JSON-RPC | + +--- + +## Testing + +There are currently no unit tests. When adding coverage: +1. Create IARM Bus stubs (`IARM_Bus_Call` mock). +2. Test Radio channel/frequency enumeration with multiple radio instances. +3. Test SSID enable/disable sequence. +4. Test AssociatedDevice table population with mock client list. +5. Test ClientRoaming parameter round-trip (SET then GET). + +--- + +## See Also + +- [src/hostif/docs/README.md](../../../docs/README.md) — Core daemon overview and `getJsonRPCData()` bug (Gap 8) +- [Device/docs/README.md](../../Device/docs/README.md) — WebPA server URL management +- [handlers/docs/README.md](../../../handlers/docs/README.md) — Dispatch layer diff --git a/src/hostif/snmpAdapter/docs/README.md b/src/hostif/snmpAdapter/docs/README.md new file mode 100644 index 000000000..f1e37588c --- /dev/null +++ b/src/hostif/snmpAdapter/docs/README.md @@ -0,0 +1,627 @@ +# SNMP Adapter Implementation Overview + +## Overview + +The `src/hostif/snmpAdapter/` module is a thin bridge that translates TR-181 parameter GET and SET requests into SNMP v2c `snmpget` and `snmpset` subprocess calls. It is used exclusively by `SNMPClientReqHandler` to serve the `Device.X_RDKCENTRAL-COM_DocsIf.*` and `Device.DeviceInfo.X_RDK_SNMP.*` subtrees, which map DOCSIS cable modem MIBs and set-top-box SNMP OIDs back into the TR-181 parameter model. + +The adapter maintains an in-memory map loaded at startup from `/etc/tr181_snmpOID.conf` that associates each TR-181 parameter name with an SNMP OID and the target device interface (CM or STB). When a GET or SET arrives, the adapter looks up the OID in this map and invokes the corresponding command-line utility via `v_secure_popen`. + +## Source Layout + +| Path | Purpose | +|------|---------| +| `src/hostif/snmpAdapter/snmpAdapter.h` | Class declaration for `hostIf_snmpAdapter`, public API, static state declarations | +| `src/hostif/snmpAdapter/snmpAdapter.cpp` | Full implementation: config loading, instance management, GET and SET dispatch | +| `src/hostif/snmpAdapter/Makefile.am` | Builds `libSNMPAdapter.la`, links against GLib and libsoup | +| `conf/tr181_snmpOID.conf` | Mapping table: TR-181 parameter name → OID + interface label | +| `src/hostif/handlers/src/hostIf_SNMPClient_ReqHandler.cpp` | Handler wrapper that calls GET/SET/attribute paths and manages locking | + +## Architecture + +The module is shallow: all logic lives in a single class with no sub-components. + +1. On daemon startup, `SNMPClientReqHandler::init()` calls `hostIf_snmpAdapter::init()`, which parses `tr181_snmpOID.conf` into `tr181Map`. +2. For each GET or SET dispatched by the handlers layer, `SNMPClientReqHandler` acquires the module lock, obtains an `hostIf_snmpAdapter` singleton instance for device index 0, and calls `get_ValueFromSNMPAdapter()` or `set_ValueToSNMPAdapter()`. +3. Each operation looks up the parameter name in `tr181Map`, selects the target IP address (STB: 127.0.0.1, CM: 192.168.100.1), and launches a `snmpget` or `snmpset` subprocess via `v_secure_popen`. +4. For `snmpget`, the raw output is parsed by finding the `=` character and copying the right-hand side into `stMsgData->paramValue`. + +### Component Diagram + +```mermaid +graph TB + subgraph Handlers[handlers layer] + SNMPH[SNMPClientReqHandler] + end + + subgraph Adapter[snmpAdapter] + CLASS[hostIf_snmpAdapter] + MAP["tr181Map
key: TR-181 param name
value: OID + interface"] + LOCK["m_mutex
GMutex"] + end + + subgraph OS[OS subprocess] + GET[snmpget -OQ -Ir -v 2c -c community address oid] + SET[snmpset -v 2c -c community address oid type value] + end + + subgraph Targets[SNMP agents] + STB["STB agent
127.0.0.1"] + CM["CM agent
192.168.100.1"] + end + + CONF[/etc/tr181_snmpOID.conf] --> CLASS + SNMPH --> CLASS + CLASS --> MAP + CLASS --> GET + CLASS --> SET + GET --> STB + GET --> CM + SET --> STB + SET --> CM +``` + +### Request Flow Diagram + +```mermaid +sequenceDiagram + participant Handler as SNMPClientReqHandler + participant Adapter as hostIf_snmpAdapter + participant Map as tr181Map + participant Shell as v_secure_popen + + Handler->>Adapter: getLock() + Handler->>Adapter: getInstance(0) + Handler->>Adapter: get_ValueFromSNMPAdapter(stMsgData) + Adapter->>Map: tr181Map.find(paramName) + Map-->>Adapter: OID + interface (STB/CM) + Adapter->>Shell: snmpget -OQ -Ir -v 2c -c
+ Shell-->>Adapter: raw output string + Adapter->>Adapter: parse '=' separator + Adapter-->>Handler: stMsgData->paramValue filled + Handler->>Adapter: releaseLock() +``` + +## How Operation Happens + +### Startup and Configuration Loading + +`hostIf_snmpAdapter::init()` is called once by `SNMPClientReqHandler::init()`, which is invoked during daemon startup from `hostIf_IARM_IF_Start()`. + +The function opens `/etc/tr181_snmpOID.conf` and reads it line by line. Each line has the format: + +``` +TR-181.ParamName = .OID.dotted.notation INTERFACE +``` + +Where `INTERFACE` is either `STB` or `CM`. The parser: + +1. Finds the `=` separator. +2. Searches for the string `STB` in the portion after the key. +3. If found at position `> 0`: sets `interface_value = "STB"`, erases the interface label from the line, then extracts the OID. +4. Otherwise: sets `interface_value = "CM"`, erases `CM` from the line, then extracts the OID. +5. Strips leading and trailing whitespace from both key and OID. +6. Inserts the pair into `tr181Map` as `map[paramName] = [{OID, interface}]`. + +**Example mapping from `conf/tr181_snmpOID.conf`:** + +``` +Device.X_RDKCENTRAL-COM_DocsIf.docsIfCmStatusTxPower = .1.3.6.1.2.1.10.127.1.2.2.1.3.2 CM +Device.DeviceInfo.X_RDK_SNMP.PowerStatus = .1.3.6.1.4.1.4491.2.3.1.1.4.1.1.0 STB +``` + +### GET Operation — `get_ValueFromSNMPAdapter()` + +For each incoming GET request: + +1. Looks up `stMsgData->paramName` in `tr181Map`. +2. If not found: returns `NOK`. +3. If found: selects the SNMP agent IP address based on the interface label. +4. Calls `GetStdoutFromSnmpgetCommand()`: + - Invokes `snmpget -OQ -Ir -v 2c -c
` via `v_secure_popen`. + - Reads all output, up to 1024 bytes at a time, into `consoleString`. +5. Finds the `=` character in the output to split the response. +6. Copies the right-hand-side value (trimmed) into `stMsgData->paramValue`. +7. Sets `stMsgData->paramtype = hostIf_StringType` unconditionally. +8. Returns `OK` on success, `-1` on popen failure, `NOT_HANDLED` on missing parameter. + +### SET Operation — `set_ValueToSNMPAdapter()` + +For each incoming SET request: + +1. Looks up `stMsgData->paramName` in `tr181Map`. +2. If not found: returns `NOT_HANDLED`. +3. If found: selects the target IP address. +4. Matches `stMsgData->paramtype` against `hostIf_StringType`, `hostIf_IntegerType`, or `hostIf_UnsignedIntType` to determine the SNMP type character (`s`, `i`, or `u`). +5. Builds the `snmpset` command string and opens the subprocess via the `CMD` macro. +6. Reads one line of output. +7. Closes the pipe and stores the close status into `ret`. +8. Sets `stMsgData->faultCode` to `fcNoFault` on success or `fcRequestDenied` on failure. + +**Note**: `hostIf_BooleanType`, `hostIf_DateTimeType`, and `hostIf_UnsignedLongType` are not handled for SET operations and return `NOK`. + +### Notification Attribute Handling + +`SNMPClientReqHandler` uses `m_notifyHash` to track which parameters have notification enabled. The `handleSetAttributesMsg()` path allocates an integer `1` and a copy of `paramName`, inserts them, and then immediately frees them — this is a use-after-free (see Gaps section). `handleGetAttributesMsg()` looks up the parameter in `m_notifyHash` and reads the integer value. + +## Key Components + +### `hostIf_snmpAdapter` class + +```cpp +class hostIf_snmpAdapter { + static GHashTable *ifHash; // instance registry, keyed by dev_id + static GMutex *m_mutex; // coarse global lock + static GHashTable *m_notifyHash; // notification attribute storage + static map>> tr181Map; // OID lookup table + + int dev_id; + + // Private: subprocess launcher + int GetStdoutFromSnmpgetCommand(const char *community, + const char *address, + const char *oid, + string &consoleString); +public: + static void init(void); // load tr181_snmpOID.conf → tr181Map + static void unInit(void); // clear tr181Map + + static hostIf_snmpAdapter *getInstance(int dev_id); + static void closeInstance(hostIf_snmpAdapter *); + static GList* getAllInstances(); + static void closeAllInstances(); + + static void getLock(); + static void releaseLock(); + + GHashTable* getNotifyHash(); + + int get_ValueFromSNMPAdapter(HOSTIF_MsgData_t *); + int set_ValueToSNMPAdapter(HOSTIF_MsgData_t *); +}; +``` + +### Configuration File Format + +`/etc/tr181_snmpOID.conf` (installed from `conf/tr181_snmpOID.conf`) contains one entry per line: + +``` + = <.OID> +``` + +Each entry is unique. The file contains two parameter subtrees: + +| Subtree | Interface | Purpose | +|---------|-----------|---------| +| `Device.X_RDKCENTRAL-COM_DocsIf.*` | CM | DOCSIS cable modem MIB values | +| `Device.DeviceInfo.X_RDK_SNMP.*` | STB | Set-top-box SNMP values (power, tuner, firmware) | + +## Threading Model + +The adapter is single-threaded at the operation level. All GET, SET, and attribute requests from `SNMPClientReqHandler` are serialized through the module's own coarse lock. + +| Primitive | Location | Purpose | +|-----------|----------|---------| +| `m_mutex` (GMutex) | Static member of `hostIf_snmpAdapter` | Serializes all `getLock()` / `releaseLock()` callers | + +**All public operations on the adapter must be bracketed by `getLock()` / `releaseLock()`.** The handler does this correctly for GET, SET, and both attribute operations. + +**Important**: `m_mutex` is lazily allocated inside `getLock()` on first call without a prior lock held. This initialization path is not thread-safe (see Gaps section). + +## Memory Management + +| Allocation | Owner | Lifetime | Freed by | +|-----------|-------|----------|---------| +| `hostIf_snmpAdapter` instance (via `new`) | `ifHash` | Daemon lifetime | `closeInstance()` → `delete` | +| `ifHash` GHashTable | Static | Daemon lifetime | Not freed in `unInit()` | +| `m_mutex` GMutex | Static | Created on first lock | Not freed in `unInit()` | +| `m_notifyHash` GHashTable | Static, per-instance destructor | Destroyed in `~hostIf_snmpAdapter()` | `g_hash_table_destroy()` in destructor | +| `tr181Map` entries | `std::map` | Re-populated on each `init()` | `tr181Map.clear()` in `unInit()` | +| `consoleString` in GET | Stack (std::string) | Per-request | Automatic | +| `notifyKey` / `notifyValuePtr` in SET-attributes | `malloc` within `SNMPClientReqHandler` | **Freed before hash insertion — use-after-free** | See Gaps section | + +## API Reference + +### `hostIf_snmpAdapter::init()` + +Loads the TR-181-to-OID mapping table from `/etc/tr181_snmpOID.conf`. + +**Signature:** `static void init(void)` + +**Thread safety:** Must be called before any concurrent access. Typically called once by `SNMPClientReqHandler::init()` during daemon startup. + +**Side effects:** Clears and repopulates the static `tr181Map`. + +--- + +### `hostIf_snmpAdapter::unInit()` + +Clears the OID mapping table. + +**Signature:** `static void unInit(void)` + +**Note:** Does not free `ifHash`, `m_mutex`, or `m_notifyHash`. This leaks resources during daemon shutdown. + +--- + +### `hostIf_snmpAdapter::getInstance(int dev_id)` + +Returns the singleton adapter instance for the given device index. Creates a new instance if one does not exist for that `dev_id`. + +**Signature:** `static hostIf_snmpAdapter *getInstance(int dev_id)` + +**Returns:** Pointer to instance, or `NULL` if allocation fails. + +**Note:** The instance registry `ifHash` is lazily initialized on first call. + +--- + +### `get_ValueFromSNMPAdapter(HOSTIF_MsgData_t *stMsgData)` + +Executes `snmpget` for the TR-181 parameter named in `stMsgData->paramName` and writes the result into `stMsgData->paramValue`. + +**Returns:** +- `OK` (0) — value retrieved and stored +- `NOT_HANDLED` — parameter name not in `tr181Map` +- `-1` — `v_secure_popen` failed + +**Paramtype set:** Always `hostIf_StringType`, regardless of the underlying OID type. + +--- + +### `set_ValueToSNMPAdapter(HOSTIF_MsgData_t *stMsgData)` + +Executes `snmpset` for the TR-181 parameter named in `stMsgData->paramName`. + +**Returns:** +- `OK` or result of `v_secure_pclose` — on success +- `NOK` — pipe open or read failure +- `NOT_HANDLED` — parameter name not in `tr181Map` + +**Supported types:** `hostIf_StringType` (`s`), `hostIf_IntegerType` (`i`), `hostIf_UnsignedIntType` (`u`) + +**Unsupported types:** `hostIf_BooleanType`, `hostIf_DateTimeType`, `hostIf_UnsignedLongType` — these return `NOK` with a log message. + +--- + +### `getLock()` / `releaseLock()` + +Coarse global lock for serializing all adapter operations. + +**Note:** `getLock()` lazily creates `m_mutex` if it is `NULL`. This is not thread-safe for the first call (see Gaps section). + +## Error Handling + +| Condition | Detected in | Return | +|-----------|-------------|--------| +| Parameter not in `tr181Map` | `get_ValueFromSNMPAdapter`, `set_ValueToSNMPAdapter` | `NOK` or `NOT_HANDLED` | +| `v_secure_popen` failure (GET) | `GetStdoutFromSnmpgetCommand` | Returns `-1` | +| `v_secure_popen` failure (SET) | `set_ValueToSNMPAdapter` | `NOK` | +| `snmpget` response missing `=` | `get_ValueFromSNMPAdapter` | Copies empty `resultBuff` (zero bytes) to `paramValue` | +| Config file not found | `init()` | Logs error; `tr181Map` remains empty | +| `getInstance` allocation failure | `getInstance()` | Logs warning; returns `NULL` | + +## Performance Notes + +Every GET and SET operation involves a `fork()` + `exec()` via `v_secure_popen`. This has a latency cost that is orders of magnitude higher than in-process IPC: + +- A single `snmpget` subprocess adds 20-100ms latency depending on SNMP agent responsiveness. +- Wildcard GET expansion that resolves to multiple SNMP parameters will spawn one subprocess per parameter. +- The coarse global mutex (`m_mutex`) serializes all requests, so high-frequency SNMP reads will queue up behind each other. +- There is no caching layer; every request goes directly to the SNMP agent. + +## Platform Notes + +- The adapter is compiled only when `SNMP_ADAPTER_ENABLED` is defined at build time. +- The module depends on the `snmpget` and `snmpset` command-line utilities being installed on the target image (`net-snmp` package). +- `v_secure_popen` from `secure_wrapper` is used as the subprocess launcher and must be available. +- When `IS_YOCTO_ENABLED`, the build links against `-lsecure_wrapper` explicitly (from `Makefile.am`). +- The SNMP community string `hDaFHJG7` is hardcoded at compile time (see Gaps section). + +## Known Issues and Gaps + +The following implementation problems were identified by reviewing `snmpAdapter.cpp`, `snmpAdapter.h`, and `hostIf_SNMPClient_ReqHandler.cpp`. Each entry records severity, location, problem, and recommended fix. + +--- + +### Gap 1 — Critical Security: SNMP community string hardcoded in source + +**File**: `src/hostif/snmpAdapter/snmpAdapter.cpp` — line 58 + +**Observation**: The SNMP v2c community string is defined as a compile-time constant: + +```cpp +#define SNMP_COMMUNITY "hDaFHJG7" +``` + +It appears in every `snmpget` and `snmpset` subprocess invocation and is also logged at `TRACE1` level in the GET path. + +**Impact**: The community string is embedded in the binary and can be extracted with standard tooling. Any process or user on the device that can read logs or the binary has the credential needed to query or set DOCSIS MIB values on both the STB and CM agents. This also means rotating or changing the community string requires a full firmware rebuild and re-flash. + +**Recommended fix** — load the community string from a file or environment variable at runtime: +```cpp +static std::string snmpCommunity; + +void hostIf_snmpAdapter::init(void) { + // Read community from a secured config path + std::ifstream commFile("/etc/snmp_community"); + if (commFile.is_open()) + std::getline(commFile, snmpCommunity); + else + RDK_LOG(RDK_LOG_ERROR, ..., "Cannot read community file\n"); + // ... rest of init ... +} +``` + +--- + +### Gap 2 — Critical: `handleSetAttributesMsg()` uses memory after freeing it + +**File**: `src/hostif/handlers/src/hostIf_SNMPClient_ReqHandler.cpp` — `handleSetAttributesMsg()` + +**Observation**: The function allocates `notifyKey` and `notifyValuePtr`, inserts them into `notifyhash`, and then frees them immediately — twice. Both the success path and the Coverity-appended `free()` at the bottom of the function free the same pointers: + +```cpp +g_hash_table_insert(notifyhash, notifyKey, notifyValuePtr); // hash now holds raw pointers +ret = OK; +free(notifyKey); // freed here — hash holds dangling pointer +free(notifyValuePtr); // freed here +// ... +free(notifyKey); // freed AGAIN — double-free (CID 87911 workaround) +free(notifyValuePtr); // freed AGAIN +``` + +The hash table retains the raw pointers. Any subsequent `handleGetAttributesMsg()` call dereferences the freed `notifyValuePtr` — this is a use-after-free. + +**Impact**: `handleGetAttributesMsg()` reads `*notifyvalue` after the memory has been freed. This is undefined behavior and can produce incorrect notification attribute values or crash the daemon. + +**Recommended fix** — do not free memory that was handed to the hash table; instead use GLib's destructor functions to free on removal: +```cpp +// Create hash with key and value destructor: +GHashTable* notifyhash = g_hash_table_new_full(g_str_hash, g_str_equal, g_free, g_free); +// Then insert — the hash table owns the memory: +g_hash_table_insert(notifyhash, g_strdup(stMsgData->paramName), notifyValuePtr); +// Do NOT call free() on notifyKey or notifyValuePtr after this +``` + +--- + +### Gap 3 — High: `set_ValueToSNMPAdapter()` uses a malformed macro + +**File**: `src/hostif/snmpAdapter/snmpAdapter.cpp` + +**Observation**: The `CMD` macro is defined as: + +```cpp +#define CMD(cmd, length, args...) ({ snprintf(cmd, length, args); fp = (v_secure_popen("r", args); )}) +``` + +The expression `fp = (v_secure_popen("r", args); )` has a semicolon inside parentheses, which is not valid C/C++ syntax. Even under GCC's statement-expression extension, `(expr;)` is not a compound statement — the correct form would be `({ expr; })`. This means the `fp` assignment may not behave as intended depending on compiler version. + +Additionally, the `cmd` buffer (built with `snprintf`) is logged but is never passed to `v_secure_popen`. `v_secure_popen` receives the raw format string and arguments directly. While both paths produce the same substitution from the same `args`, this is fragile and makes the logged command value meaningless for auditing. + +**Impact**: The SET path may not compile cleanly on strict compilers and the command logged to RDK_LOG is built separately from the command actually executed, reducing diagnostic value. + +**Recommended fix** — build the command string first and execute it: +```cpp +snprintf(cmd, BUFF_LENGTH_256, "snmpset -v 2c -c %s %s %s s %s", + SNMP_COMMUNITY, address, oid, stMsgData->paramValue); +RDK_LOG(RDK_LOG_TRACE1, LOG_TR69HOSTIF, "[%s] %s\n", __FUNCTION__, cmd); +fp = v_secure_popen("r", "snmpset -v 2c -c %s %s %s s %s", + SNMP_COMMUNITY, address, oid, stMsgData->paramValue); +``` +Remove the `CMD` macro entirely. + +--- + +### Gap 4 — High: `getLock()` is not thread-safe for first-time initialization + +**File**: `src/hostif/snmpAdapter/snmpAdapter.cpp` + +**Observation**: `getLock()` lazily initializes `m_mutex`: + +```cpp +void hostIf_snmpAdapter::getLock() { + if (!m_mutex) { + m_mutex = g_mutex_new(); // race condition here + } + g_mutex_lock(m_mutex); +} +``` + +If two threads call `getLock()` simultaneously before `m_mutex` is set, both pass the `NULL` check, both call `g_mutex_new()`, and only one assignment wins. The other `GMutex*` is leaked and the winning pointer may not be the one both threads proceed to lock, creating silent non-mutual-exclusion. + +**Impact**: This is a startup race condition. GET and SET requests arriving quickly after daemon initialization (common during boot) can bypass the lock entirely, leading to concurrent map access and potential crashes. + +**Recommended fix** — initialize the mutex once in `init()`: +```cpp +void hostIf_snmpAdapter::init(void) { + if (!m_mutex) + m_mutex = g_mutex_new(); + // ... rest of init ... +} +``` + +--- + +### Gap 5 — High: All GET results typed as `hostIf_StringType` regardless of OID type + +**File**: `src/hostif/snmpAdapter/snmpAdapter.cpp` — `get_ValueFromSNMPAdapter()` + +**Observation**: After retrieving the SNMP response, the result type is unconditionally set to string: + +```cpp +stMsgData->paramtype = hostIf_StringType; +``` + +Integer, unsigned integer, and boolean SNMP OID values are returned as strings. Callers that branch on `paramtype` (for example, `hostIf_GetMsgHandler()` telemetry logging or RBUS type conversion) will misinterpret numeric values. + +**Impact**: Numeric comparisons, range checks, and protocol serialization that depend on `paramtype` correctness will silently treat all SNMP-backed parameters as strings. `getStringValue()` in the httpserver layer has a specific `hostIf_UnsignedLongType` branch that formats values as `%lu`, but will never be used for SNMP parameters. + +**Recommended fix** — infer the type from the OID map or from the `snmpget -OQ` output prefix (e.g., `INTEGER:`, `STRING:`, `Gauge32:`): +```cpp +if (consoleString.find("INTEGER:") != string::npos || + consoleString.find("Gauge32:") != string::npos) { + stMsgData->paramtype = hostIf_IntegerType; +} else { + stMsgData->paramtype = hostIf_StringType; +} +``` + +--- + +### Gap 6 — Medium: `init()` parser misidentifies `STB` at string position 0 + +**File**: `src/hostif/snmpAdapter/snmpAdapter.cpp` — `init()` + +**Observation**: The interface detection uses: + +```cpp +int result = line.find(interface_STB); +if (result > 0) { + interface_value = interface_STB; + ... +} +``` + +`line.find()` returns `string::size_type` (unsigned). After assignment to `int result`, `string::npos` maps to `-1`, which correctly fails `> 0`. However, if `STB` appears at position `0` (start of the line — possible if whitespace trimming changes the line layout), `result == 0` and `0 > 0` is `false`. The entry would be silently treated as a CM parameter and queried against `192.168.100.1` instead of `127.0.0.1`. + +**Impact**: Any configuration entry where the interface label appears at the beginning of the value part would be incorrectly assigned to the CM agent. + +**Recommended fix** — use `string::npos` as the sentinel: +```cpp +size_t result = line.find(interface_STB); +if (result != string::npos) { + interface_value = interface_STB; +``` + +--- + +### Gap 7 — Medium: `~hostIf_snmpAdapter()` destroys a static shared hash table + +**File**: `src/hostif/snmpAdapter/snmpAdapter.cpp` + +**Observation**: The destructor destroys `m_notifyHash`: + +```cpp +hostIf_snmpAdapter::~hostIf_snmpAdapter() { + if (m_notifyHash) { + g_hash_table_destroy(m_notifyHash); + } +} +``` + +`m_notifyHash` is a `static` member shared across all instances. If `closeInstance()` is ever called for any instance other than the last one, the hash table is destroyed. All remaining instances — and any subsequent call to `getNotifyHash()` — will operate on a destroyed table. + +**Impact**: In practice only one instance (device index 0) is ever created, so this is latent. However, if the cleanup path is extended or the adapter is used for multiple devices, this will cause heap corruption. + +**Recommended fix** — move hash table destruction to `unInit()` rather than the destructor: +```cpp +void hostIf_snmpAdapter::unInit(void) { + tr181Map.clear(); + if (m_notifyHash) { + g_hash_table_destroy(m_notifyHash); + m_notifyHash = NULL; + } +} +``` + +--- + +### Gap 8 — Medium: `unInit()` leaks `ifHash` and `m_mutex` + +**File**: `src/hostif/snmpAdapter/snmpAdapter.cpp` + +**Observation**: `unInit()` only calls `tr181Map.clear()`. The instance hash table `ifHash` and the mutex `m_mutex` are never freed. This is typically not a problem for a daemon (resources reclaimed by OS on exit), but it is a problem if `init()` / `unInit()` cycles are used at runtime for configuration reload, as the mutex would be re-created without freeing the old one. + +**Recommended fix** — add cleanup to `unInit()`: +```cpp +void hostIf_snmpAdapter::unInit(void) { + tr181Map.clear(); + if (m_mutex) { + g_mutex_free(m_mutex); + m_mutex = NULL; + } + if (ifHash) { + g_hash_table_destroy(ifHash); + ifHash = NULL; + } +} +``` + +--- + +### Gap 9 — Low: Missing return type on `GetStdoutFromSnmpgetCommand` in header + +**File**: `src/hostif/snmpAdapter/snmpAdapter.h` + +**Observation**: The declaration in the class body is: + +```cpp +GetStdoutFromSnmpgetCommand(const char *community, const char *address, + const char *oid, string &consoleString); +``` + +No return type is declared. The implementation returns `int`. In C++ this is a compile error under `-std=c++11` or later since implicit `int` is not valid. The project presumably compiles with warnings rather than errors for this case, or the method is treated as `int` by older compilers. + +**Recommended fix**: +```cpp +int GetStdoutFromSnmpgetCommand(const char *community, const char *address, + const char *oid, string &consoleString); +``` + +--- + +### Gap 10 — Low: SNMP v2c provides no encryption or authentication + +**File**: All subprocess calls in `snmpAdapter.cpp` + +**Observation**: All SNMP operations use SNMPv2c (`-v 2c`). SNMPv2c community-based security provides no message authentication, no privacy (data is cleartext on the wire), and no per-user access control. The CM agent is accessed at `192.168.100.1`, an IP address that may be reachable from subnets other than the device itself. + +**Impact**: Any device on the same network segment as `192.168.100.1` that knows the community string can read or modify DOCSIS MIB values. The plaintext-on-wire nature means passive network monitoring can capture the community string from any SNMP exchange. + +**Recommended fix** — migrate to SNMPv3 with `authPriv` security level using SHA authentication and AES privacy. The command-line syntax change is: +```bash +# v2c (current): +snmpget -OQ -Ir -v 2c -c
+ +# v3 (recommended): +snmpget -OQ -Ir -v 3 -u -l authPriv \ + -a SHA -A -x AES -X
+``` + +--- + +### Gap Summary Table + +| # | Severity | File | Problem | Impact | +|---|----------|------|---------|--------| +| 1 | **Critical** | `snmpAdapter.cpp` | SNMP community string hardcoded in source | Credential embedded in binary; requires firmware flash to rotate | +| 2 | **Critical** | `hostIf_SNMPClient_ReqHandler.cpp` | `notifyKey`/`notifyValuePtr` freed before hash table uses them + freed twice | Use-after-free in `handleGetAttributesMsg()`; double-free crash | +| 3 | **High** | `snmpAdapter.cpp` | Malformed `CMD` macro with `(expr;)` syntax | SET subprocess may not execute correctly on strict compilers | +| 4 | **High** | `snmpAdapter.cpp` | `getLock()` lazily initializes `m_mutex` without synchronization | Boot-time race condition allows concurrent map access before first lock | +| 5 | **High** | `snmpAdapter.cpp` | All GET responses typed `hostIf_StringType` regardless of OID type | Numeric parameter type information lost; callers misinterpret values | +| 6 | **Medium** | `snmpAdapter.cpp` | `result > 0` check misses `STB` at string position 0 | Config entries with `STB` at position 0 silently route to CM agent | +| 7 | **Medium** | `snmpAdapter.cpp` | Destructor destroys static `m_notifyHash` on any instance close | Latent heap corruption if multiple instances are ever used | +| 8 | **Medium** | `snmpAdapter.cpp` | `unInit()` does not free `ifHash` or `m_mutex` | Memory and mutex leaked during any config-reload cycle | +| 9 | **Low** | `snmpAdapter.h` | Missing return type on `GetStdoutFromSnmpgetCommand` declaration | Compile warning or error on C++11 strict mode | +| 10 | **Low** | `snmpAdapter.cpp` | SNMPv2c used for all operations | Community string transmitted cleartext; no per-user auth or privacy | + +## Testing + +There are no unit tests for the `snmpAdapter` module. The `Makefile.am` builds only `libSNMPAdapter.la` with no test target. Testing is done implicitly through `SNMPClientReqHandler` integration tests when the full daemon is run with a live SNMP agent. + +When modifying this module, manually validate: + +1. `init()` correctly loads all entries from `tr181_snmpOID.conf` and classifies them as STB or CM. +2. `get_ValueFromSNMPAdapter()` returns the expected string value for a known OID against a live or mock SNMP agent. +3. Parameters not in the map return `NOT_HANDLED` without crashing. +4. `getLock()` / `releaseLock()` correctly serializes concurrent callers. +5. `unInit()` followed by `init()` leaves `tr181Map` in a clean state. + +## See Also + +- `src/hostif/handlers/src/hostIf_SNMPClient_ReqHandler.cpp` for the handler wrapper that drives this module +- `src/hostif/handlers/docs/README.md` for the handlers-layer overview +- `conf/tr181_snmpOID.conf` for the mapping table installed at `/etc/tr181_snmpOID.conf` +- `docs/architecture/overview.md` for the daemon-wide component map +- `docs/api/public-api.md` for `HOSTIF_MsgData_t` and shared request types