Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,10 @@ Sub-components within `uploadstblogs/`:
| validation | `validation.h` | Parameter and path validation |
| verification | `verification.h` | Post-upload result verification |

**RDK-C behaviors:**

- **Scheduled log collection** — when `DCM_SCHEDULED_LOG_COLLECT=true` (`/etc/device.properties`, RDK-C only), `dcm_setup` stages the current `/opt/logs` tree into `DCM_LOG_PATH` (via `copy_files_to_dcm_path`) so the daily scheduled DCM upload carries current logs. Default **off** on STB/broadband (external batcher / Maintenance Manager supplies the logs).

---

### backup\_logs — Log Backup
Expand Down Expand Up @@ -525,6 +529,8 @@ make install
| Flag | Effect |
|------|--------|
| `-DRDK_LOGGER_ENABLED` | Use RDK logger instead of stderr |
| `-DRDK_LOGGER_EXT` | Use the **extended** RDK logger init API (`rdk_logger_ext_config_t`, `RDKLOG_OUTPUT_CONSOLE`, `RDKLOG_FORMAT_WITH_TS`). Its **absence** makes `context_manager.c` fall back to the standard `rdk_logger_init(debug.ini)` — required for RDK-C, whose rdk-logger 2.4.0 predates the extended API. |
| `-DRDKC` | RDK-C (camera) platform marker. |
| `-DHAS_MAINTENANCE_MANAGER` | Enable Maintenance Manager integration via IARM |
| `-DGTEST_ENABLE` | Stub out RBUS/IARM for unit testing |
| `-DDCM_DEF_LOG_URL=<url>` | Override default fallback upload URL |
Expand Down Expand Up @@ -608,6 +614,15 @@ if (ret != DCM_SUCCESS) {
- Optional IARM bus integration for Maintenance Manager notifications.
- RDK logger (`librdkloggers`) replaces `fprintf(stderr)` when available.

### RDK-C / Sysvinit (Camera)

RDK-C camera platforms (e.g. XHC1) run **sysvinit**, not systemd, and ship an older rdk-logger. Key differences:

- **No systemd** — the `sys_integration` systemd READY notification is inactive and `dcmd.service` is not installed; `dcmd` is started from the sysvinit `dcm-log-service` init script.
- **DCM–T2 handshake retry** — sysvinit start order is variable (telemetry can start well before `dcmd`), so the reload-config event publish is retried up to `DCM_RELOAD_EVENT_MAX_RETRY` (30 attempts, 1 s apart) so the handshake completes regardless of order.
- **Logger fallback** — built **without** `-DRDK_LOGGER_EXT` (see Conditional Compile Flags), so `context_manager.c` initialises logging via the standard `rdk_logger_init(debug.ini)` path.
- **Scheduled log collection** — `DCM_SCHEDULED_LOG_COLLECT=true` in `/etc/device.properties` stages the current `/opt/logs` tree into `DCM_LOG_PATH` before the scheduled DCM upload (RDK-C cameras have no external batcher). Default **off** on STB/broadband.

### Resource Constraints

| Resource | Typical Budget |
Expand Down
2 changes: 2 additions & 0 deletions Readme.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
See README.md for the full DCM Agent documentation (architecture, modules,
build instructions, RDK-C / sysvinit platform notes, and testing).
33 changes: 31 additions & 2 deletions dcm.c
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,14 @@
#include "dcm_schedjob.h"
#include "uploadstblogs.h"

/*
* Max attempts to publish the reload config event to telemetry before giving up
* and continuing to the scheduling loop. Retrying makes the DCM<->T2 reload
* handshake independent of daemon start order (see the reload publish in main()
* and registerRbusDCMEventListener in telemetry's rbusInterface.c).
*/
#define DCM_RELOAD_EVENT_MAX_RETRY 30

static DCMDHandle *g_pdcmHandle = NULL;

/** @brief Call back function from Scheduler. This function
Expand Down Expand Up @@ -345,7 +353,28 @@ int main(int argc, char* argv[])

DCMInfo("Telemetry Events subscriptions is success\n");

ret = dcmRbusSendEvent(g_pdcmHandle->pRbusHandle);
/*
* Publish the reload event to telemetry. dcmRbusSendEvent only succeeds once
* telemetry has subscribed to the reload event; on RDK-C that subscribe can
* complete just after dcmd is ready, so the first publish may race ahead of
* it. Retry a bounded number of times until the event is delivered. The
* bound preserves the legacy "log and continue" behaviour so dcmd still
* enters its scheduling loop even if telemetry never subscribes.
*/
{
INT32 retryCount = 0;
do {
ret = dcmRbusSendEvent(g_pdcmHandle->pRbusHandle);
if(ret == DCM_SUCCESS) {
break;
}
retryCount++;
DCMInfo("Reload event not delivered yet, retry %d/%d\n",
retryCount, DCM_RELOAD_EVENT_MAX_RETRY);
sleep(1);
} while(retryCount < DCM_RELOAD_EVENT_MAX_RETRY);
}

if(ret) {
DCMError("Reload config event failed!!!\n");
}
Expand Down Expand Up @@ -409,7 +438,7 @@ int main(int argc, char* argv[])
#endif

#ifdef GTEST_ENABLE
void get_dcmRunJobs(const INT8* profileName, VOID *pHandle)
void get_dcmRunJobs(const INT8* profileName, VOID *pHandle)
{
dcmRunJobs(profileName, pHandle);
}
Expand Down
16 changes: 15 additions & 1 deletion docs/Logupload_Behavior/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Log Upload Behavior in the RDKE Stack

> **Explore mode document** — Researched from codebase and GitHub (rdkcentral/dcm-agent,
> rdkcentral/rdkservices, rdkcentral/reboot-manager, rdkcentral/telemetry, rdkcentral/iarmmgrs).
> rdkcentral/rdkservices, rdkcentral/reboot-manager, rdkcentral/telemetry, rdkcentral/iarmmgrs).
> Date: 2026-05-20.

---
Expand All @@ -12,6 +12,20 @@ This folder documents the full behavioral picture of log upload across the RDKE
triggers it, what it does, what events it emits, which other components depend on its
outputs, and how boot-time ordering is (or isn't) enforced.

---

## RDK-C (Camera) Additions

On RDK-C camera platforms (sysvinit; e.g. XHC1), the DCM Agent runs as the native `dcmd`
daemon with an RDK-C-specific behavior layered onto the pipeline below:

- **Scheduled log collection** (`DCM_SCHEDULED_LOG_COLLECT`) — RDK-C cameras have no external
batcher / Maintenance Manager, so with `DCM_SCHEDULED_LOG_COLLECT=true` in
`/etc/device.properties` the DCM strategy's `dcm_setup` stages the current `/opt/logs` tree
into `DCM_LOG_PATH` (via `copy_files_to_dcm_path`) before the scheduled upload, so the daily
archive carries current logs. Default **off** on STB/broadband (sources below supply the logs
externally).

---
## RDKE Log Upload Support

Expand Down
106 changes: 69 additions & 37 deletions test/functional-tests/tests/test_uploadstblogs_upload_strategies.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,26 +48,26 @@ def setup_and_teardown(self):
def test_ondemand_immediate_execution(self):
"""Test: On-demand upload executes immediately"""
create_test_log_files(count=2)

# Trigger on-demand upload (TriggerType=5)
args = "'' 0 0 0 HTTP http://localhost:8080 5 0 ''"

start_time = time.time()
result = run_uploadstblogs(args)
elapsed = time.time() - start_time

# Should start immediately without waiting
assert elapsed < 30, "On-demand upload should execute immediately"

@pytest.mark.order(2)
def test_ondemand_no_schedule_wait(self):
"""Test: On-demand upload doesn't wait for scheduled time"""
create_test_log_files(count=2)

# Execute on-demand
args = "'' 0 0 0 HTTP http://localhost:8080 5 0 ''"
result = run_uploadstblogs(args)

# Check logs for immediate execution
immediate_logs = grep_uploadstb_logs_regex(r"immediate|ondemand|manual")
# Process should complete
Expand All @@ -77,10 +77,10 @@ def test_ondemand_no_schedule_wait(self):
def test_ondemand_telemetry(self):
"""Test: On-demand upload generates appropriate telemetry"""
create_test_log_files(count=1)

args = "'' 0 0 0 HTTP http://localhost:8080 5 0 ''"
result = run_uploadstblogs(args)

# Check for telemetry
telemetry_logs = grep_uploadstb_logs_regex(r"telemetry|marker")
# Should complete
Expand All @@ -105,12 +105,12 @@ def setup_and_teardown(self):
def test_reboot_upload_detection(self):
"""Test: Service detects reboot condition"""
create_test_log_files(count=2)

# Trigger reboot upload (UploadOnReboot=1, TriggerType=2)
args = "'' 0 0 1 HTTP http://localhost:8080 2 0 ''"

result = run_uploadstblogs(args)

# Check for reboot detection (may not be explicitly logged)
reboot_logs = grep_uploadstb_logs_regex(r"reboot|UploadOnReboot|REBOOT|TriggerType.*2")
# Process should complete with reboot parameters
Expand All @@ -122,12 +122,12 @@ def test_reboot_previous_logs_collection(self):
# Create files in PreviousLogs directory
sp.run("mkdir -p /opt/logs/PreviousLogs", shell=True)
sp.run("echo 'previous log content' > /opt/logs/PreviousLogs/prev.log", shell=True)

create_test_log_files(count=1)

args = "'' 0 0 1 HTTP http://localhost:8080 2 0 ''"
result = run_uploadstblogs(args)

# Check for previous log collection
prev_logs = grep_uploadstb_logs_regex(r"previous|PreviousLogs")
# Should process logs
Expand All @@ -137,10 +137,10 @@ def test_reboot_previous_logs_collection(self):
def test_reboot_upload_telemetry(self):
"""Test: Reboot upload generates appropriate telemetry"""
create_test_log_files(count=1)

args = "'' 0 0 1 HTTP http://localhost:8080 2 0 ''"
result = run_uploadstblogs(args)

# Check for telemetry
telemetry_logs = grep_uploadstb_logs_regex(r"telemetry|reboot.*success")
# Should complete
Expand All @@ -165,12 +165,12 @@ def setup_and_teardown(self):
def test_dcm_scheduled_trigger(self):
"""Test: DCM scheduled upload is triggered correctly"""
create_test_log_files(count=2)

# DCM scheduled upload (FLAG=0, DCM_FLAG=0, TriggerType=0)
args = "'' 0 0 0 HTTP http://localhost:8080 0 0 ''"

result = run_uploadstblogs(args)

# Check for DCM processing
dcm_logs = grep_uploadstb_logs_regex(r"DCM|scheduled|FLAG.*0")
assert len(dcm_logs) > 0, "DCM scheduled upload should be processed"
Expand All @@ -179,10 +179,10 @@ def test_dcm_scheduled_trigger(self):
def test_dcm_log_collection(self):
"""Test: DCM scheduled upload collects logs according to configuration"""
create_test_log_files(count=3)

args = "'' 0 0 0 HTTP http://localhost:8080 0 0 ''"
result = run_uploadstblogs(args)

# Check for log collection
collection_logs = grep_uploadstb_logs_regex(r"collect|archive|DCM")
# Should attempt collection
Expand All @@ -192,15 +192,47 @@ def test_dcm_log_collection(self):
def test_dcm_upload_telemetry(self):
"""Test: DCM upload generates telemetry"""
create_test_log_files(count=1)

args = "'' 0 0 0 HTTP http://localhost:8080 0 0 ''"
result = run_uploadstblogs(args)

# Check telemetry
telemetry_logs = grep_uploadstb_logs_regex(r"telemetry|marker|SYST")
# Should complete
assert result.returncode in [0, 1], "Should generate DCM telemetry"

@pytest.mark.order(4)
def test_dcm_scheduled_log_collect_enabled(self):
"""RDK-C: DCM_SCHEDULED_LOG_COLLECT=true stages current logs into DCM_LOG_PATH."""
set_device_property("DCM_SCHEDULED_LOG_COLLECT", "true")
create_test_log_files(count=3)

args = "'' 0 0 0 HTTP http://localhost:8080 0 0 ''"
result = run_uploadstblogs(args)

# The DCM strategy should log the RDK-C current-log collection step
collect_logs = grep_uploadstb_logs_regex(
r"Collecting current logs|Scheduled DCM log collection enabled|Successfully copied")
assert len(collect_logs) > 0, \
"DCM_SCHEDULED_LOG_COLLECT=true should stage current logs into DCM_LOG_PATH"
assert result.returncode in [0, 1], "DCM upload should complete"

@pytest.mark.order(5)
def test_dcm_scheduled_log_collect_disabled(self):
"""RDK-C: without the flag the DCM strategy does NOT stage current logs (STB/broadband parity)."""
set_device_property("DCM_SCHEDULED_LOG_COLLECT", "false")
create_test_log_files(count=3)

args = "'' 0 0 0 HTTP http://localhost:8080 0 0 ''"
result = run_uploadstblogs(args)

# The collection step must NOT run when the flag is false/absent
collect_logs = grep_uploadstb_logs_regex(
r"Collecting current logs|Scheduled DCM log collection enabled")
assert len(collect_logs) == 0, \
"DCM_SCHEDULED_LOG_COLLECT=false must not stage current logs"
assert result.returncode in [0, 1], "DCM upload should complete"


class TestRBUSIntegration:
"""Test suite for RBUS event triggered uploads"""
Expand All @@ -220,9 +252,9 @@ def setup_and_teardown(self):
def test_rbus_parameter_loading(self):
"""Test: Service loads parameters from RBUS/TR-181"""
create_test_log_files(count=1)

result = run_uploadstblogs()

# Check for RBUS initialization
rbus_logs = grep_uploadstb_logs_regex(r"rbus|RBUS|TR-181|Device\.DeviceInfo")
# RBUS parameters may be loaded during context init
Expand All @@ -235,22 +267,22 @@ def test_rbus_triggered_upload_via_cli(self):
rbus_check = sp.run("which rbuscli", shell=True, capture_output=True)
if rbus_check.returncode != 0:
pytest.skip("rbuscli not available")

create_test_log_files(count=1)

# Trigger via RBUS would be done through DCM agent typically
# For direct test, we use command line args
result = run_uploadstblogs()

assert result.returncode in [0, 1], "RBUS-triggered upload should work"

@pytest.mark.order(3)
def test_rbus_configuration_loading(self):
"""Test: Service loads upload configuration from RBUS"""
create_test_log_files(count=1)

result = run_uploadstblogs()

# Check for configuration loading
config_logs = grep_uploadstb_logs_regex(r"load.*TR-181|load.*param|endpoint|RFC")
# Should attempt to load config
Expand All @@ -260,9 +292,9 @@ def test_rbus_configuration_loading(self):
def test_rbus_event_publishing(self):
"""Test: Upload success event is published via RBUS"""
create_test_log_files(count=1)

result = run_uploadstblogs()

# Check for event publishing
event_logs = grep_uploadstb_logs_regex(r"event|publish|success")
# Should complete
Expand All @@ -287,12 +319,12 @@ def setup_and_teardown(self):
def test_strategy_selection_based_on_flags(self):
"""Test: Correct strategy is selected based on flags"""
create_test_log_files(count=1)

# Test different flag combinations
# RRD mode: RRD_FLAG=1
args = "'' 0 0 0 HTTP http://localhost:8080 0 1 /opt/logs/rrd.log"
result = run_uploadstblogs(args)

# Check for strategy selection
strategy_logs = grep_uploadstb_logs_regex(r"strategy|RRD|select")
assert result.returncode in [0, 1], "Should select appropriate strategy"
Expand All @@ -301,22 +333,22 @@ def test_strategy_selection_based_on_flags(self):
def test_multiple_strategy_parameters(self):
"""Test: Service handles multiple strategy parameters"""
create_test_log_files(count=1)

# Test with various parameters
args = "'' 1 1 1 HTTPS https://localhost:8443 1 0 ''"
result = run_uploadstblogs(args)

# Should handle all parameters
assert result.returncode in [0, 1], "Should handle multiple parameters"

@pytest.mark.order(3)
def test_strategy_logging(self):
"""Test: Selected strategy is logged"""
create_test_log_files(count=1)

args = "'' 0 0 1 HTTP http://localhost:8080 2 0 ''"
result = run_uploadstblogs(args)

# Check strategy logging
logs = grep_uploadstb_logs_regex(r"strategy|STRAT_|upload.*type")
# Strategy should be determined
Expand Down
6 changes: 6 additions & 0 deletions uploadstblogs/docs/hld/uploadSTBLogs_HLD.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,4 +246,10 @@ int main(int argc, char** argv) {
## 18. Non-Extended Design Choices
Excluded any unrelated enhancements (alternate compression, multi-protocol expansion, scheduler integration) to preserve diagram fidelity.

## 19. RDK-C Additions (Camera)

One RDK-C-specific behavior extends the base design on camera (sysvinit) platforms; it is inert on STB/broadband:

- **Scheduled log collection** — gated by the `DCM_SCHEDULED_LOG_COLLECT` device property. When enabled, the DCM strategy's `dcm_setup` stages the current `/opt/logs` tree into `DCM_LOG_PATH` (via `copy_files_to_dcm_path`) before archiving, so the scheduled upload carries current logs. Default off ⇒ STB/broadband batch-drain behavior is unchanged.

```
Loading
Loading