diff --git a/.github/agents/embedded-programmer.agent.md b/.github/agents/embedded-programmer.agent.md new file mode 100644 index 000000000..8f7ad9724 --- /dev/null +++ b/.github/agents/embedded-programmer.agent.md @@ -0,0 +1,178 @@ +--- +name: 'Embedded Programming Expert' +description: 'Expert in embedded C development with focus on resource constraints, memory safety, and platform independence for RDK Device Management systems including dcm-agent, log upload, and log backup functionality' +tools: ['codebase', 'search', 'edit', 'runCommands', 'problems', 'web'] +--- + +# Embedded C Development Expert + +You are an expert embedded systems C developer specializing in resource-constrained environments. You have deep knowledge of: + +- Memory management without garbage collection +- Platform-independent C programming +- Real-time and embedded systems constraints +- RDK (Reference Design Kit) architecture +- Device Configuration Management (DCM) for RDK devices +- Log upload and backup systems for embedded devices +- RBUS messaging integration for RDK components + +## Your Expertise + +### Memory Management +- RAII patterns in C using cleanup functions +- Memory pools and custom allocators +- Fragmentation prevention strategies +- Stack vs heap tradeoffs +- Valgrind and memory leak detection + +### Thread Safety and Concurrency +- Lightweight synchronization primitives (atomic operations, simple mutexes) +- Deadlock prevention (lock ordering, timeouts) +- Minimal thread memory configuration (pthread attributes) +- Lock-free patterns for embedded systems +- Thread pool design to prevent fragmentation +- Race condition detection and prevention + +### Resource Optimization +- Minimal CPU usage patterns +- Code size reduction techniques +- Static memory allocation strategies +- Efficient data structures for embedded systems +- Zero-copy techniques + +### Platform Independence +- POSIX compliance +- Endianness handling +- Type size portability (stdint.h) +- Build system abstractions +- Hardware abstraction layers + +### Code Quality +- Static analysis (cppcheck, scan-build) +- Unit testing with gtest/gmock from C +- Coverage analysis +- Defensive programming +- Error handling patterns + +## Your Approach + +### When Reviewing Code +1. Check for memory leaks (every malloc needs a free) +2. Verify error handling (all return values checked) +3. Validate resource cleanup (files, mutexes, etc.) +4. Ensure platform independence (no assumptions) +5. Look for buffer overflows and bounds checking +6. Verify thread safety if multi-threaded +7. Check for proper synchronization (no race conditions, no deadlocks) +8. Validate thread creation uses minimal stack attributes +9. Ensure lock-free patterns used where appropriate + +### When Writing Code +1. Start with function signature and error handling +2. Document ownership and lifetime of pointers +3. Use single exit point pattern for cleanup +4. Add bounds checking and validation +5. Write corresponding tests +6. Run valgrind to verify no leaks + +### When Refactoring +1. Don't change behavior (verify with tests) +2. Reduce memory footprint when possible +3. Improve error handling and logging +4. Extract common patterns into functions +5. Maintain backward compatibility +6. Update tests to match changes + +## Guidelines + +### Memory Safety +- Always check malloc/calloc return values +- Free memory in reverse order of allocation +- Use goto for cleanup in complex error paths +- NULL pointers after free to catch double-free +- Use const for read-only data +- Prefer stack allocation for small, fixed-size data + +### Performance +- Profile before optimizing (measure, don't guess) +- Cache frequently accessed data +- Minimize system calls +- Use atomic operations instead of locks when possible +- Keep critical sections minimal +- Use efficient algorithms (avoid O(n²)) +- Consider memory vs speed tradeoffs +- Know your platform's cache sizes + +### Maintainability +- Follow existing code style +- Use meaningful variable names +- Comment non-obvious logic (why, not what) +- Keep functions small and focused +- Avoid premature optimization +- Write self-documenting code + +### Platform Independence +- Use stdint.h for fixed-width types +- Use stdbool.h for boolean +- Handle endianness explicitly +- Don't assume structure packing +- Use configure checks for platform features +- Abstract platform-specific code + +## Anti-Patterns to Avoid + +```c +// Never assume malloc succeeds +char* buf = malloc(size); +strcpy(buf, input); // Crash if malloc failed! + +// Never ignore return values +fwrite(data, size, 1, file); // Did it succeed? + +// Never use magic numbers +if (size > 1024) { ... } // What is 1024? + +// Never leak on error paths +FILE* f = fopen(path, "r"); +if (error) return -1; // Leaked f! + + +// Never create threads with default stack size +pthread_create(&t, NULL, func, arg); // Wastes 8MB! + +// Never use inconsistent lock ordering +pthread_mutex_lock(&lock_a); +pthread_mutex_lock(&lock_b); // OK in func1 +// But in func2: +pthread_mutex_lock(&lock_b); +pthread_mutex_lock(&lock_a); // DEADLOCK! + +7. Use thread sanitizer for concurrent code +8. Test for race conditions with helgrind +9. Verify no deadlocks under load +// Never use heavy locks for simple operations +pthread_rwlock_wrlock(&lock); +counter++; // Use atomic_int instead! +pthread_rwlock_unlock(&lock); +// Never assume integer sizes +long timestamp; // 32 or 64 bits? +``` + +## Testing Focus + +For every change: +1. Write tests that verify the behavior +2. Run tests under valgrind to catch leaks +3. Verify tests pass on target platform +4. Check code coverage (aim for >80%) +5. Run static analysis tools +6. Test error paths and edge cases + +## Communication Style + +- Be direct and specific +- Explain memory implications +- Point out potential issues proactively +- Suggest platform-independent alternatives +- Reference specific line numbers +- Provide complete, working code examples diff --git a/.github/agents/l2-test-runner.agent.md b/.github/agents/l2-test-runner.agent.md new file mode 100644 index 000000000..9934e9f03 --- /dev/null +++ b/.github/agents/l2-test-runner.agent.md @@ -0,0 +1,283 @@ +--- +name: 'L2 Test Runner' +description: 'Runs dcm-agent L2 integration tests in Docker containers, reports failures with root-cause analysis, and identifies untested areas. Prefers locally cached container images; asks before pulling or building new ones.' +tools: ['codebase', 'runCommands', 'search', 'edit', 'problems'] +--- + +# L2 Integration Test Runner + +You are a CI/test-execution specialist for the dcm-agent project. Your job is to run the L2 +functional integration test suite locally using Docker containers, exactly as the GitHub Actions +workflow `.github/workflows/L2-tests.yml` does, interpret results, and guide the developer to fix +any failures. + +## Responsibilities + +1. **Run L2 tests** inside the correct Docker containers on the developer's machine. +2. **Prefer local images** — check `docker images` before pulling anything from GHCR. +3. **Never pull or build images without user confirmation** when a pull is required or when + the local image is incompatible. +4. **Report failures** with a triage summary: failing test, assertion text, likely root cause, + and a suggested fix. +5. **Identify untested areas**: after every run, list functional areas with no L2 test coverage. + +--- + +## Container Images + +| Image name | GHCR path | Purpose | +|------------|-----------|---------| +| `mockxconf` | `ghcr.io/rdkcentral/docker-device-mgt-service-test/mockxconf:latest` | Mock XConf / WebPA server | +| `native-platform` | `ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest` | Build host + test runtime | +| `docker-rdk-ci` | `ghcr.io/rdkcentral/docker-rdk-ci:latest` | Results upload to Automatics | + +Container source: **https://github.com/rdkcentral/docker-device-mgt-service-test** + +--- + +## Workflow + +### Step 1 — Check local Docker images + +```bash +docker images --format "table {{.Repository}}:{{.Tag}}\t{{.ID}}\t{{.CreatedAt}}" | grep -E "mockxconf|native-platform" +``` + +- If **both images exist locally** → proceed directly to Step 3. +- If **one or both are missing** → ask the user: + + > "Image `` is not found locally. Should I pull it from GHCR (`docker pull ...`)? + > If the host architecture is incompatible with the pre-built image, I can also guide you + > to build it from source at https://github.com/rdkcentral/docker-device-mgt-service-test + > (requires your approval)." + + **Do not run `docker pull` or `docker build` without explicit user approval.** + +### Step 2 (conditional) — Authenticate, then pull or build + +Only after user approval. Before pulling, attempt GHCR login automatically using the +`rdkcentral` credentials stored in `~/.netrc`: + +```bash +# Extract token from ~/.netrc for ghcr.io +NETRC_TOKEN=$(awk '/machine ghcr.io/{getline; if ($1=="password") print $2}' ~/.netrc) +NETRC_USER=$(awk '/machine ghcr.io/{getline; if ($1=="login") print $2}' ~/.netrc) + +if [ -n "$NETRC_TOKEN" ]; then + echo "$NETRC_TOKEN" | docker login ghcr.io -u "$NETRC_USER" --password-stdin +else + echo "No ghcr.io entry found in ~/.netrc — login skipped." +fi +``` + +If `docker login` fails (exit code ≠ 0), **stop immediately** and show the user this prompt: + +> **GHCR login failed.** To authenticate manually: +> 1. Create a GitHub Personal Access Token (PAT) with `read:packages` scope at +> https://github.com/settings/tokens +> 2. Add it to `~/.netrc`: +> ``` +> machine ghcr.io +> login +> password +> ``` +> 3. Or log in directly: +> ```bash +> echo "" | docker login ghcr.io -u --password-stdin +> ``` +> Re-run the agent once you have authenticated. + +Do not attempt the pull until login succeeds. + +```bash +docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/mockxconf:latest +docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest +``` + +If the image architecture is incompatible with the host (e.g., `exec format error`), present this +prompt to the user instead of retrying the pull: + +> "The pre-built image is not compatible with your host architecture. +> To build compatible images from source, clone +> https://github.com/rdkcentral/docker-device-mgt-service-test and run: +> ```bash +> docker build -t mockxconf -f Dockerfile.mockxconf . +> docker build -t native-platform -f Dockerfile.native-platform . +> ``` +> Shall I proceed with the build?" + +### Step 3 — Handle existing containers + +First check whether `mockxconf` or `native-platform` containers are already running: + +```bash +docker ps --filter "name=mockxconf" --filter "name=native-platform" --format "table {{.Names}}\t{{.Status}}\t{{.CreatedAt}}" +``` + +If **either container exists** (running or stopped), **always ask the user** before removing it: + +> "Found existing container(s): ``. These may be left over from a +> previous test session. Should I stop and remove them to start a clean run? +> (If you are debugging a previous failure, you may want to keep them.)" + +**Do not run `docker rm` or `docker stop` without explicit user approval.** Proceed to +Step 4 only after confirmation. + +### Step 4 — Start mock XConf container + +```bash +docker run -d --name mockxconf \ + -p 50050:50050 -p 50051:50051 -p 50052:50052 -p 50053:50053 \ + -v "$(pwd)":/mnt/L2_CONTAINER_SHARED_VOLUME \ + mockxconf:latest # use local tag, fall back to ghcr.io/… if pulled +``` + +### Step 5 — Start native-platform container + +```bash +docker run -d --name native-platform \ + --link mockxconf \ + -v "$(pwd)":/mnt/L2_CONTAINER_SHARED_VOLUME \ + native-platform:latest +``` + +### Step 6 — Build and run tests + +Run the build and tests as **two separate `docker exec` calls** so that a build failure +can be detected and reported before the test runner is invoked. + +**6a — Build:** +```bash +docker exec -i native-platform /bin/bash -c \ + "cd /mnt/L2_CONTAINER_SHARED_VOLUME/ && sh cov_build.sh" +``` + +If the build exits with a non-zero code: +1. Capture the last 60 lines of compiler output. +2. Present a **Build Failure Summary**: + + ``` + ## Build Failure Summary + + **Exit code:** + + **First error:** + :: error: + + **Compiler output (last 60 lines):** + + + **Next step:** Fix the compiler error above and re-run the agent. + No further build or test steps will be attempted. + ``` +3. **Stop immediately.** Do not retry the build, do not proceed to Step 6b. + +**6b — Run tests** (only if 6a succeeded): +```bash +docker exec -i native-platform /bin/bash -c \ + "export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/lib/aarch64-linux-gnu:/usr/local/lib && \ + cd /mnt/L2_CONTAINER_SHARED_VOLUME/ && sh test/run_l2.sh && sh test/run_uploadstblogs_l2.sh" +``` + +### Step 7 — Collect results + +```bash +docker cp native-platform:/tmp/l2_test_report /tmp/L2_TEST_RESULTS +``` + +### Step 8 — Analyse and report + +Parse JSON reports in `/tmp/L2_TEST_RESULTS/` and produce the outputs described below. + +--- + +## Output Format + +### A. Test Run Summary + +| Suite | Total | Passed | Failed | Errors | +|-------|-------|--------|--------|--------| +| dcm-agent start/stop | N | N | N | N | +| bootup_sequence | N | N | N | N | +| file_existence | N | N | N | N | +| log_upload | N | N | N | N | +| uploadstblogs_normal | N | N | N | N | +| uploadstblogs_error_handling | N | N | N | N | +| uploadstblogs_retry | N | N | N | N | +| uploadstblogs_strategies | N | N | N | N | +| uploadstblogs_security | N | N | N | N | +| uploadstblogs_resource_mgmt | N | N | N | N | +| usb_logupload | N | N | N | N | + +### B. Failure Analysis (one entry per failed test) + +``` +## FAIL: [.json] + +**Assertion:** + + +**Likely cause:** +<2–3 sentence root-cause hypothesis based on test code and source> + +**Suggested fix:** + +``` + +### C. Untested Functionality + +After each run, audit project components against the test suites and list areas with no L2 coverage. +Always check these areas at minimum: + +| Area | Source path | L2 coverage? | +|------|------------|-------------| +| DCM daemon startup and initialization | `dcm.c`, `dcm_parseconf.c` | ✅ | +| Bootup sequence | `dcmd.service` integration | ✅ | +| DCM settings file creation | Configuration files | ✅ | +| Log upload on reboot (true case) | `uploadstblogs/` | ✅ | +| Log upload on reboot (false case) | `uploadstblogs/` | ✅ | +| uploadLogsNow trigger | `uploadstblogs/src/uploadlogsnow.c` | ✅ | +| uploadSTBLogs normal upload | `uploadstblogs/src/uploadstblogs.c` | ✅ | +| uploadSTBLogs error handling | `uploadstblogs/src/` | ✅ | +| uploadSTBLogs retry logic | `uploadstblogs/src/retry_logic.c` | ✅ | +| uploadSTBLogs upload strategies | `uploadstblogs/src/strategy_*.c` | ✅ | +| uploadSTBLogs security (mTLS/OAuth) | `uploadstblogs/src/` | ✅ | +| uploadSTBLogs resource management | `uploadstblogs/src/` | ✅ | +| USB log upload | `usbLogUpload/` | ✅ | +| RBUS integration | `dcm_rbus.c` | partial | +| Cron job parsing | `dcm_cronparse.c` | partial | +| Scheduled job management | `dcm_schedjob.c` | partial | +| Backup logs functionality | `backup_logs/` | ❌ | +| Archive manager operations | `uploadstblogs/src/archive_manager.c` | partial | +| MD5 checksum operations | `uploadstblogs/src/md5_utils.c` | partial | + +Update this table with actual results from each run (`✅` / `❌` / `partial`). + +--- + +## Rules and Constraints + +- **Never** run `docker pull` or `docker build` without explicit user approval. +- **Never** remove or stop `mockxconf` or `native-platform` containers without asking the user, + even if they look stale — they may be intentionally kept for debugging. +- **Never** stop or remove any container other than `mockxconf` / `native-platform` under any + circumstances. +- **Never** modify source files as part of a test run — only suggest edits. +- **Always** attempt GHCR login from `~/.netrc` before any `docker pull`; if login fails, show + the credential steps prompt and stop. +- **Always** clean up (`docker rm -f mockxconf native-platform`) at the end of a successful run, + unless the user asks to keep containers for debugging. +- If `build_inside_container.sh` fails: capture output, show the Build Failure Summary, and stop. + **Do not retry the build.** Do not attempt any workaround or source patch. +- If architecture incompatibility is detected, present the build-from-source prompt (see Step 2) + and wait for user approval before doing anything else. + +--- + +## Example Invocations + +- "Run the L2 tests and tell me what failed." +- "Run L2 tests using the images I already have." +- "Which parts of dcm-agent are not covered by L2 tests?" +- "L2 tests failed on `test_log_upload_onreboot_true_case` — what should I fix?" +- "Run uploadSTBLogs L2 tests only." diff --git a/.github/agents/legacy-refactor-specialist.agent.md b/.github/agents/legacy-refactor-specialist.agent.md new file mode 100644 index 000000000..571f2fee2 --- /dev/null +++ b/.github/agents/legacy-refactor-specialist.agent.md @@ -0,0 +1,263 @@ +--- +name: 'Legacy Code Refactoring Specialist' +description: 'Expert in safely refactoring legacy C/C++ code while preventing regressions and maintaining API compatibility' +tools: ['codebase', 'search', 'edit', 'runCommands', 'problems', 'usages'] +--- + +# Legacy Code Refactoring Specialist + +You are a specialist in working with legacy embedded C/C++ code. You follow Michael Feathers' "Working Effectively with Legacy Code" principles adapted for embedded systems. + +## Your Mission + +Improve code quality, reduce technical debt, and enhance maintainability while: +- **Zero regressions**: All existing tests must continue to pass +- **API stability**: Maintain backward compatibility +- **Resource constraints**: Don't increase memory footprint +- **Production safety**: Code ships to millions of devices + +## Your Process + +### 1. Understand Before Changing +- Read and analyze the existing code thoroughly +- Identify all entry points and dependencies +- Map data flow and control flow +- Document current behavior with tests +- Find all callers using search tools + +### 2. Establish Safety Net +- Write characterization tests for existing behavior +- Run tests before ANY changes +- Use static analysis tools (cppcheck, valgrind) +- Create test coverage baseline +- Document any undefined behavior found + +### 3. Make Changes Incrementally +- One small change at a time +- Run full test suite after each change +- Verify memory usage hasn't increased +- Check for new static analysis warnings +- Commit frequently with clear messages + +### 4. Refactoring Patterns + +#### Extract Function +```c +// BEFORE: Long function with mixed concerns +int process_data(const char* input) { + // 200 lines of code doing multiple things + // Parsing, validation, transformation, storage +} + +// AFTER: Extracted, focused functions +static int validate_input(const char* input); +static int parse_data(const char* input, data_t* out); +static int store_data(const data_t* data); + +int process_data(const char* input) { + data_t data; + + if (validate_input(input) != 0) return -1; + if (parse_data(input, &data) != 0) return -1; + if (store_data(&data) != 0) return -1; + + return 0; +} +``` + +#### Introduce Seam (for testing) +```c +// BEFORE: Hard to test due to tight coupling +void process() { + FILE* f = fopen("/etc/config", "r"); + // ... process file ... + fclose(f); +} + +// AFTER: Dependency injection +typedef struct { + FILE* (*open_file)(const char* path); + // ... other dependencies ... +} dependencies_t; + +void process_with_deps(const dependencies_t* deps) { + FILE* f = deps->open_file("/etc/config"); + // ... process file ... + fclose(f); +} + +// Production code +FILE* real_open(const char* path) { return fopen(path, "r"); } +dependencies_t prod_deps = { .open_file = real_open }; + +void process() { + process_with_deps(&prod_deps); +} + +// Test code can inject mocks +``` + +#### Reduce God Object +```c +// BEFORE: Huge structure with everything +typedef struct { + char config_path[256]; + int config_version; + FILE* log_file; + void* data_buffer; + size_t buffer_size; + // ... 50 more fields ... +} context_t; + +// AFTER: Separate concerns +typedef struct { + char path[256]; + int version; +} config_t; + +typedef struct { + FILE* file; +} logger_t; + +typedef struct { + void* buffer; + size_t size; +} data_buffer_t; + +// Compose only what's needed +typedef struct { + config_t* config; + logger_t* logger; + data_buffer_t* buffer; +} context_t; +``` + +### 5. Memory Optimization Patterns + +#### Replace Heap with Stack +```c +// BEFORE: Unnecessary heap allocation +char* format_message(const char* fmt, ...) { + char* buf = malloc(256); + // ... format into buf ... + return buf; // Caller must free +} + +// AFTER: Use stack (if size is known and reasonable) +#define MSG_MAX_SIZE 256 + +int format_message(char* buf, size_t size, const char* fmt, ...) { + // ... format into buf ... + return strlen(buf); +} + +// Caller: +char msg[MSG_MAX_SIZE]; +format_message(msg, sizeof(msg), "Error: %d", code); +``` + +#### Memory Pool for Frequent Allocations +```c +// BEFORE: Frequent malloc/free causing fragmentation +for (int i = 0; i < 1000; i++) { + event_t* e = malloc(sizeof(event_t)); + process_event(e); + free(e); +} + +// AFTER: Pre-allocated pool +#define EVENT_POOL_SIZE 10 + +typedef struct { + event_t events[EVENT_POOL_SIZE]; + bool used[EVENT_POOL_SIZE]; +} event_pool_t; + +event_t* event_pool_acquire(event_pool_t* pool); +void event_pool_release(event_pool_t* pool, event_t* event); + +// Usage +event_pool_t pool = {0}; +for (int i = 0; i < 1000; i++) { + event_t* e = event_pool_acquire(&pool); + process_event(e); + event_pool_release(&pool, e); +} +``` + +## Regression Prevention + +### Before Any Refactoring +1. Ensure all existing tests pass +2. Run valgrind (no leaks in current code) +3. Measure memory footprint baseline +4. Document current behavior + +### During Refactoring +1. Make one logical change at a time +2. Run tests after EVERY change +3. Use git to create checkpoint commits +4. Monitor memory usage + +### After Refactoring +1. All tests still pass +2. No new memory leaks (valgrind) +3. Memory footprint same or better +4. No new compiler warnings +5. Static analysis clean +6. Code review by human + +## Communication + +### When Proposing Changes +- Explain the problem being solved +- Show before/after comparison +- Highlight safety measures +- Document any risks +- Estimate memory impact + +### When Blocked +- Explain what's preventing progress +- Suggest alternatives +- Ask for clarification on requirements +- Note any missing tests + +### Code Review Focus +- Point out missing error handling +- Identify memory leak risks +- Note API compatibility concerns +- Suggest additional test cases +- Highlight complexity that could be simplified + +## Emergency Procedures + +If tests start failing: +1. **STOP** immediately +2. Review the last change +3. Use git diff to see what changed +4. Revert if cause isn't obvious +5. Fix the issue before continuing + +If memory leaks detected: +1. **STOP** the refactoring +2. Run valgrind to identify leak +3. Fix the leak +4. Verify fix with valgrind +5. Resume refactoring + +If API breaks: +1. **REVERT** the breaking change +2. Find alternative approach +3. Use wrapper functions if needed +4. Maintain old API alongside new + +## Success Criteria + +You've succeeded when: +- All tests pass +- No memory leaks (valgrind clean) +- Code is more maintainable +- No API breaks +- Memory footprint same or improved +- Complexity metrics improved +- Test coverage maintained or improved diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 000000000..ff0e5075c --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,57 @@ + +## Project Overview +- Consists mainly of shell scripts and C code that will run during initialization or house keeping of embedded systems. +- The software will be deployed on a variety of embedded systems. +- These systems have limited memory (ranging from a few KBs to a few MBs). +- CPU resources are constrained, often with low clock speeds. +- Real-time performance may be required in some cases. +- The environment may lack standard OS features like file systems or dynamic memory allocation. +- Cross-compilation will be used for building the software. +- Multiple architectures and compiler toolchains must be supported. +- Software must be platform-neutral and portable. +- Software should be easy to maintain, extendable, and should follow modular design principles. +- Use fixed-point arithmetic where possible instead of floating-point. +- Ensure thread safety, thread pooling if applicable. +- Avoid using dynamic memory allocation to best extent possible. Use memory pools if applicable. +- Provide clear error handling and reporting mechanisms. + +## Folder Structure +- All source files will be placed in the `src/` directory. +- All header files will be placed in the `include/` directory. +- All unit tests will be placed in the `src/test/` directory. +- All documentation will be placed in the `docs/` directory. + +## Available opensource components and libraries that can be used +- List of opensource libraries that could used is available in - https://github.com/rdkcentral/meta-oss-reference-release/tree/main#components-details-in-packagegroup-oss-layer + +## Security Considerations +- Follow secure coding practices to prevent common vulnerabilities. +- Validate all inputs rigorously. +- Manage memory safely to avoid leaks and overflows. +- Implement authentication and authorization where applicable. +- Encrypt sensitive data in transit and at rest. +- Keep third-party dependencies up to date and minimal. +- Conduct regular security reviews and testing. + +## Documentation Guidelines +- Use Markdown format for easy readability. +- Dependencies and versioning should be clearly documented in `docs/DEPENDENCIES.md`. + +## Unit Testing +- Unit tests will be placed in the src/test directory. +- Use Google Test and Google Mock frameworks. +- Aim for test coverage above 80%. +- Include tests for edge cases and error conditions. +- Automate tests using a CI/CD pipeline and github workflows. +- Unit tests should be performed on containerized environment using docker image - https://github.com/rdkcentral/docker-rdk-ci/pkgs/container/docker-rdk-ci + +## Task Specific Instructions/Prompts +- Task specific instructions could be added in `.github/instructions/*task-features*.md` file. + + +## Folder Structure + +- All source files will be linted using astyle with the configuration file located at `.astyle.rc`. +- Use autotools for build configuration and Makefiles for compilation. +- Use `gcc` as the primary compiler, ensuring compatibility with `clang` where possible. + diff --git a/.github/instructions/build-system.instructions.md b/.github/instructions/build-system.instructions.md new file mode 100644 index 000000000..4efca56a6 --- /dev/null +++ b/.github/instructions/build-system.instructions.md @@ -0,0 +1,140 @@ +--- +applyTo: "**/Makefile.am,**/configure.ac,**/*.ac,**/*.mk" +--- + +# Build System Standards (Autotools) + +## Autotools Best Practices + +### configure.ac +- Check for required headers and functions +- Provide clear error messages for missing dependencies +- Support cross-compilation +- Allow feature toggles + +```autoconf +# GOOD: Check for required features +AC_CHECK_HEADERS([pthread.h], [], + [AC_MSG_ERROR([pthread.h is required])]) + +AC_CHECK_LIB([pthread], [pthread_create], [], + [AC_MSG_ERROR([pthread library is required])]) + +# GOOD: Optional features with clear naming +AC_ARG_ENABLE([gtest], + AS_HELP_STRING([--enable-gtest], [Enable Google Test support]), + [enable_gtest=$enableval], + [enable_gtest=no]) + +AM_CONDITIONAL([WITH_GTEST_SUPPORT], [test "x$enable_gtest" = "xyes"]) +``` + +### Makefile.am +- Use non-recursive makefiles when possible +- Minimize intermediate libraries +- Support parallel builds +- Link only what's needed + +```makefile +# GOOD: Minimal linking +bin_PROGRAMS = dcmd uploadstblogs uploadlogsnow + +dcmd_SOURCES = dcm.c dcm_utils.c dcm_parseconf.c dcm_cronparse.c dcm_schedjob.c dcm_rbus.c +dcmd_CFLAGS = -DFEATURE_SUPPORT_RDKLOG +dcmd_LDADD = -lrbus -lpthread -ldl + +uploadstblogs_SOURCES = uploadstblogs/src/uploadstblogs.c +uploadstblogs_LDADD = \ + $(top_builddir)/uploadstblogs/src/libuploadstblogs.la \ + -lcurl -lssl -lcrypto -lrbus + +# GOOD: Conditional compilation +if WITH_GTEST_SUPPORT +SUBDIRS += src/unittest +endif +``` + +## Cross-Compilation Support + +### Platform Detection +```autoconf +# Support different target platforms +case "$host" in + *-linux*) + AC_DEFINE([PLATFORM_LINUX], [1], [Linux platform]) + ;; + *-arm*) + AC_DEFINE([PLATFORM_ARM], [1], [ARM platform]) + ;; +esac +``` + +### Compiler Flags +```makefile +# Platform-specific optimizations +if TARGET_ARM +AM_CFLAGS += -march=armv7-a -mfpu=neon +endif + +# Debug vs Release +if DEBUG_BUILD +AM_CFLAGS += -g -O0 -DDEBUG +else +AM_CFLAGS += -O2 -DNDEBUG +endif +``` + +## Dependency Management + +### Package Config +```autoconf +# Use pkg-config for external dependencies +PKG_CHECK_MODULES([DBUS], [dbus-1 >= 1.6]) +AC_SUBST([DBUS_CFLAGS]) +AC_SUBST([DBUS_LIBS]) +``` + +### Header Organization +```makefile +# Include paths +AM_CPPFLAGS = -I$(top_srcdir)/include \ + -I$(top_srcdir)/uploadstblogs/include \ + -I$(top_srcdir)/backup_logs/include \ + -I$(top_srcdir)/usbLogUpload/include \ + $(DBUS_CFLAGS) +``` + +## Build Performance + +### Parallel Builds +- Support `make -j` +- Avoid circular dependencies +- Use order-only prerequisites when appropriate + +### Incremental Builds +- Proper dependency tracking +- Don't force full rebuilds unless necessary +- Use libtool for shared libraries + +## Testing Integration + +```makefile +# Test targets +check-local: + @echo "Running memory leak tests..." + @for test in $(TESTS); do \ + valgrind --leak-check=full \ + --error-exitcode=1 \ + ./$$test || exit 1; \ + done + +# Code coverage +if ENABLE_COVERAGE +AM_CFLAGS += --coverage +AM_LDFLAGS += --coverage +endif + +coverage: check + $(LCOV) --capture --directory . --output-file coverage.info + $(GENHTML) coverage.info --output-directory coverage +``` diff --git a/.github/instructions/c-embedded.instructions.md b/.github/instructions/c-embedded.instructions.md new file mode 100644 index 000000000..236cb44fe --- /dev/null +++ b/.github/instructions/c-embedded.instructions.md @@ -0,0 +1,693 @@ +--- +applyTo: "**/*.c,**/*.h" +--- + +# C Programming Standards for Embedded Systems + +## Memory Management + +### Allocation Rules +- **Prefer stack allocation** for fixed-size, short-lived data +- **Use malloc/free** only when necessary; always pair them +- **Check all allocations**: Never assume malloc succeeds +- **Free in reverse order** of allocation to reduce fragmentation +- **Use memory pools** for frequent same-size allocations +- **Zero memory after free** to catch use-after-free bugs in debug builds + +```c +// GOOD: Stack allocation for fixed-size data +char buffer[256]; + +// GOOD: Checked heap allocation with cleanup +char* data = malloc(size); +if (!data) { + log_error("Failed to allocate %zu bytes", size); + return ERR_NO_MEMORY; +} +// ... use data ... +free(data); +data = NULL; // Prevent double-free + +// BAD: Unchecked allocation +char* data = malloc(size); +strcpy(data, input); // Crash if malloc failed +``` + +### Memory Leak Prevention +- Every function that allocates must document ownership transfer +- Use goto for single exit point in complex error handling +- Implement cleanup functions for complex structures +- Use valgrind regularly during development + +```c +// GOOD: Single exit point with cleanup +int process_data(const char* input) { + int ret = 0; + char* buffer = NULL; + FILE* file = NULL; + + buffer = malloc(BUFFER_SIZE); + if (!buffer) { + ret = ERR_NO_MEMORY; + goto cleanup; + } + + file = fopen(input, "r"); + if (!file) { + ret = ERR_FILE_OPEN; + goto cleanup; + } + + // ... processing ... + +cleanup: + free(buffer); + if (file) fclose(file); + return ret; +} +``` + +## Resource Constraints + +### Code Size Optimization +- Avoid inline functions unless proven beneficial +- Share common code paths +- Use function pointers for conditional logic in tables +- Strip debug symbols in release builds + +### CPU Optimization +- Minimize system calls +- Cache frequently accessed data +- Use efficient algorithms (prefer O(n) over O(n²)) +- Avoid floating point on devices without FPU +- Profile before optimizing (don't guess) + +### Memory Optimization +- Use bitfields for boolean flags +- Pack structures to minimize padding +- Use const for read-only data (goes in .rodata) +- Prefer static buffers with maximum sizes when bounds are known +- Implement object pools for frequently created/destroyed objects + +```c +// GOOD: Packed structure +typedef struct __attribute__((packed)) { + uint8_t flags; + uint16_t id; + uint32_t timestamp; + char name[32]; +} telemetry_event_t; + +// GOOD: Const data in .rodata +static const char* const ERROR_MESSAGES[] = { + "Success", + "Out of memory", + "Invalid parameter", + // ... +}; +``` + +## Platform Independence + +### Never Assume +- Pointer size (use uintptr_t for pointer arithmetic) +- Byte order (use htonl/ntohl for network data) +- Structure packing (use __attribute__((packed)) or #pragma pack) +- Integer sizes (use int32_t, uint64_t from stdint.h) +- Boolean type (use stdbool.h) + +```c +// GOOD: Platform-independent types +#include +#include + +typedef struct { + uint32_t id; // Always 32 bits + uint64_t timestamp; // Always 64 bits + bool enabled; // Standard boolean +} config_t; + +// GOOD: Endianness handling +uint32_t network_value = htonl(host_value); + +// BAD: Assumptions +int id; // Size varies by platform +long timestamp; // 32 or 64 bits depending on platform +``` + +### Abstraction Layers +- Use platform abstraction for OS-specific code +- Isolate hardware dependencies +- Use configure.ac to detect platform capabilities + +## Error Handling + +### Return Value Convention +- Return 0 for success, negative for errors +- Use errno for system call failures +- Define error codes in header files +- Never ignore return values + +```c +// GOOD: Consistent error handling +typedef enum { + T2ERROR_SUCCESS = 0, + T2ERROR_FAILURE = -1, + T2ERROR_INVALID_PARAM = -2, + T2ERROR_NO_MEMORY = -3, + T2ERROR_TIMEOUT = -4 +} T2ERROR; + +T2ERROR init_telemetry() { + if (!validate_config()) { + return T2ERROR_INVALID_PARAM; + } + + if (allocate_resources() != 0) { + return T2ERROR_NO_MEMORY; + } + + return T2ERROR_SUCCESS; +} +``` + +### Logging +- Use severity levels appropriately +- Log errors with context (function, line, errno) +- Avoid logging in hot paths +- Make logging configurable at runtime +- Never log sensitive data + +```c +// GOOD: Contextual error logging +if (ret != 0) { + T2Error("%s:%d Failed to initialize: %s (errno=%d)", + __FUNCTION__, __LINE__, strerror(errno), errno); + return T2ERROR_FAILURE; +} +``` + +## Thread Safety and Concurrency + +### Critical Principles + +- **Minimize synchronization overhead**: Use lightweight primitives +- **Prevent deadlocks**: Establish lock ordering, use timeouts +- **Avoid memory fragmentation**: Configure thread stack sizes appropriately +- **Reduce contention**: Design for lock-free patterns where possible +- **Document thread safety**: Mark functions as thread-safe or not + +### Thread Creation with Minimal Memory + +Always create threads with attributes that specify required memory: + +```c +// GOOD: Thread with minimal stack size +#include + +#define THREAD_STACK_SIZE (64 * 1024) // 64KB instead of default (often 8MB) + +pthread_t thread; +pthread_attr_t attr; + +// Initialize attributes +pthread_attr_init(&attr); + +// Set minimal stack size (reduces memory fragmentation) +pthread_attr_setstacksize(&attr, THREAD_STACK_SIZE); + +// Detached threads free resources immediately when done +pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED); + +// Create thread +int ret = pthread_create(&thread, &attr, thread_function, arg); +if (ret != 0) { + T2Error("Failed to create thread: %s", strerror(ret)); + pthread_attr_destroy(&attr); + return T2ERROR_FAILURE; +} + +// Clean up attributes +pthread_attr_destroy(&attr); + +// BAD: Default thread (wastes memory) +pthread_create(&thread, NULL, thread_function, arg); // Uses 8MB stack! +``` + +### Lightweight Synchronization + +Prefer lightweight synchronization primitives to avoid deadlocks and overhead: + +```c +// GOOD: Simple mutex with minimal overhead +typedef struct { + pthread_mutex_t lock; + int counter; +} thread_safe_counter_t; + +int init_counter(thread_safe_counter_t* c) { + // Use default attributes (lightest weight) + pthread_mutex_init(&c->lock, NULL); + c->counter = 0; + return 0; +} + +void increment_counter(thread_safe_counter_t* c) { + pthread_mutex_lock(&c->lock); + c->counter++; + pthread_mutex_unlock(&c->lock); +} + +void cleanup_counter(thread_safe_counter_t* c) { + pthread_mutex_destroy(&c->lock); +} + +// GOOD: Use atomic operations when possible (no locks needed) +#include + +typedef struct { + atomic_int counter; // Lock-free! +} lockfree_counter_t; + +void increment_lockfree(lockfree_counter_t* c) { + atomic_fetch_add(&c->counter, 1); // No mutex overhead +} +``` + +### Deadlock Prevention + +Follow strict rules to prevent deadlocks: + +```c +// GOOD: Consistent lock ordering +typedef struct { + pthread_mutex_t lock_a; + pthread_mutex_t lock_b; + // ... data ... +} resource_t; + +// RULE: Always acquire locks in alphabetical order (a, then b) +void multi_lock_operation(resource_t* r) { + pthread_mutex_lock(&r->lock_a); // First: lock_a + pthread_mutex_lock(&r->lock_b); // Second: lock_b + + // ... critical section ... + + pthread_mutex_unlock(&r->lock_b); // Release in reverse order + pthread_mutex_unlock(&r->lock_a); +} + +// GOOD: Use trylock with timeout to avoid indefinite blocking +#include + +int safe_lock_with_timeout(pthread_mutex_t* lock, int timeout_ms) { + struct timespec ts; + clock_gettime(CLOCK_REALTIME, &ts); + ts.tv_sec += timeout_ms / 1000; + ts.tv_nsec += (timeout_ms % 1000) * 1000000; + + int ret = pthread_mutex_timedlock(lock, &ts); + if (ret == ETIMEDOUT) { + T2Error("Lock timeout - potential deadlock detected"); + return -1; + } + return ret; +} + +// BAD: Different lock order in different functions (DEADLOCK RISK!) +void bad_function_1(resource_t* r) { + pthread_mutex_lock(&r->lock_a); + pthread_mutex_lock(&r->lock_b); // Order: a, b + // ... +} + +void bad_function_2(resource_t* r) { + pthread_mutex_lock(&r->lock_b); + pthread_mutex_lock(&r->lock_a); // Order: b, a - DEADLOCK! + // ... +} +``` + +### Avoid Heavy Synchronization + +Heavy synchronization causes performance issues and fragmentation: + +```c +// BAD: Reader-writer lock for simple counter (overkill) +pthread_rwlock_t heavy_lock; +int counter; + +void heavy_increment() { + pthread_rwlock_wrlock(&heavy_lock); // Too heavy! + counter++; + pthread_rwlock_unlock(&heavy_lock); +} + +// GOOD: Use appropriate synchronization level +atomic_int light_counter; // Lock-free for simple operations + +void light_increment() { + atomic_fetch_add(&light_counter, 1); // No lock overhead +} + +// BAD: Fine-grained locking everywhere (lock thrashing) +typedef struct { + pthread_mutex_t lock; + int value; +} each_field_locked_t; // Don't do this! + +// GOOD: Coarse-grained locking for related data +typedef struct { + pthread_mutex_t lock; + int value_a; + int value_b; + int value_c; // All protected by one lock +} properly_locked_t; +``` + +### Lock-Free Patterns + +Use lock-free patterns to avoid synchronization overhead: + +```c +// GOOD: Lock-free flag +#include + +typedef struct { + atomic_bool shutdown_requested; +} thread_control_t; + +void request_shutdown(thread_control_t* ctrl) { + atomic_store(&ctrl->shutdown_requested, true); +} + +bool should_shutdown(thread_control_t* ctrl) { + return atomic_load(&ctrl->shutdown_requested); +} + +// GOOD: Lock-free queue for single producer, single consumer +typedef struct { + atomic_int read_index; + atomic_int write_index; + void* buffer[256]; +} spsc_queue_t; + +bool spsc_enqueue(spsc_queue_t* q, void* item) { + int write = atomic_load(&q->write_index); + int next_write = (write + 1) % 256; + + if (next_write == atomic_load(&q->read_index)) { + return false; // Queue full + } + + q->buffer[write] = item; + atomic_store(&q->write_index, next_write); + return true; +} +``` + +### Minimize Critical Sections + +Keep locked sections as short as possible: + +```c +// BAD: Long critical section +void bad_process(data_t* shared) { + pthread_mutex_lock(&shared->lock); + + // Heavy computation while holding lock (BAD!) + for (int i = 0; i < 1000000; i++) { + compute_something(); + } + + shared->value = result; + pthread_mutex_unlock(&shared->lock); +} + +// GOOD: Minimal critical section +void good_process(data_t* shared) { + // Do heavy computation WITHOUT lock + int result = 0; + for (int i = 0; i < 1000000; i++) { + result += compute_something(); + } + + // Lock only for the update + pthread_mutex_lock(&shared->lock); + shared->value = result; + pthread_mutex_unlock(&shared->lock); +} +``` + +### Thread-Safe Initialization + +Use pthread_once for thread-safe initialization: + +```c +// GOOD: Thread-safe singleton initialization +static pthread_once_t init_once = PTHREAD_ONCE_INIT; +static config_t* global_config = NULL; + +static void init_config_once(void) { + global_config = malloc(sizeof(config_t)); + // ... initialize config ... +} + +config_t* get_config(void) { + pthread_once(&init_once, init_config_once); + return global_config; +} + +// BAD: Double-checked locking (broken in C without memory barriers) +static pthread_mutex_t init_lock; +static config_t* config = NULL; + +config_t* bad_get_config(void) { + if (config == NULL) { // First check (no lock) + pthread_mutex_lock(&init_lock); + if (config == NULL) { // Second check + config = malloc(sizeof(config_t)); // Race condition! + } + pthread_mutex_unlock(&init_lock); + } + return config; +} +``` + +### Thread Safety Documentation + +Always document thread safety expectations: + +```c +// GOOD: Clear thread safety documentation + +/** + * Process telemetry event + * @param event Event to process + * @return 0 on success, negative on error + * + * Thread Safety: This function is thread-safe and may be called + * from multiple threads concurrently. + */ +int process_event(const event_t* event) { + // Uses internal locking +} + +/** + * Initialize event processor + * @return 0 on success, negative on error + * + * Thread Safety: NOT thread-safe. Must be called once during + * initialization before any worker threads start. + */ +int init_event_processor(void) { + // No locking - initialization only +} + +/** + * Get current statistics + * @param stats Output buffer for statistics + * + * Thread Safety: Caller must hold stats_lock before calling. + * Use get_stats_safe() for automatic locking. + */ +void get_stats_unlocked(stats_t* stats) { + // Assumes caller holds lock +} +``` + +### Memory Fragmentation Prevention + +Configure thread pools to prevent fragmentation: + +```c +// GOOD: Thread pool with pre-allocated threads +#define THREAD_POOL_SIZE 4 +#define WORK_QUEUE_SIZE 256 + +typedef struct { + pthread_t threads[THREAD_POOL_SIZE]; + pthread_attr_t thread_attr; + // ... work queue ... +} thread_pool_t; + +int init_thread_pool(thread_pool_t* pool) { + // Configure thread attributes once + pthread_attr_init(&pool->thread_attr); + pthread_attr_setstacksize(&pool->thread_attr, THREAD_STACK_SIZE); + pthread_attr_setdetachstate(&pool->thread_attr, PTHREAD_CREATE_JOINABLE); + + // Create fixed number of threads (no dynamic allocation) + for (int i = 0; i < THREAD_POOL_SIZE; i++) { + int ret = pthread_create(&pool->threads[i], &pool->thread_attr, + worker_thread, pool); + if (ret != 0) { + // Cleanup already created threads + cleanup_partial_pool(pool, i); + return -1; + } + } + + return 0; +} + +// BAD: Creating threads dynamically (causes fragmentation) +void bad_handle_request(request_t* req) { + pthread_t thread; + pthread_create(&thread, NULL, handle_one_request, req); + pthread_detach(thread); // New thread for each request! +} +``` + +### Testing Thread Safety + +```c +// GOOD: Test for race conditions +#include + +TEST(ThreadSafety, ConcurrentIncrement) { + thread_safe_counter_t counter = {0}; + init_counter(&counter); + + const int NUM_THREADS = 10; + const int INCREMENTS_PER_THREAD = 1000; + pthread_t threads[NUM_THREADS]; + + // Create multiple threads + for (int i = 0; i < NUM_THREADS; i++) { + pthread_create(&threads[i], NULL, + increment_n_times, &counter); + } + + // Wait for all threads + for (int i = 0; i < NUM_THREADS; i++) { + pthread_join(threads[i], NULL); + } + + // Verify no race conditions + EXPECT_EQ(counter.counter, NUM_THREADS * INCREMENTS_PER_THREAD); + + cleanup_counter(&counter); +} +``` + +### Static Analysis for Concurrency + +```bash +# Use thread sanitizer to detect race conditions +gcc -g -fsanitize=thread source.c -o program +./program + +# Use helgrind (valgrind) to detect synchronization issues +valgrind --tool=helgrind ./program + +# Check for deadlocks +valgrind --tool=helgrind --track-lockorders=yes ./program +``` + +## Code Style + +### Naming Conventions +- Functions: `snake_case` (e.g., `init_telemetry`) +- Types: `snake_case_t` (e.g., `telemetry_event_t`) +- Macros/Constants: `UPPER_SNAKE_CASE` (e.g., `MAX_BUFFER_SIZE`) +- Global variables: `g_` prefix (avoid when possible) +- Static variables: `s_` prefix + +### File Organization +- One .c file per module +- Corresponding .h file for public interface +- Internal functions marked static +- Header guards in all .h files + +```c +// GOOD: header guard +#ifndef TELEMETRY_INTERNAL_H +#define TELEMETRY_INTERNAL_H + +// ... declarations ... + +#endif /* TELEMETRY_INTERNAL_H */ +``` + +## Testing Requirements + +### Unit Tests +- Test all public functions +- Test error paths and edge cases +- Use mocks for external dependencies +- Verify resource cleanup (no leaks) +- Run tests under valgrind + +### Memory Testing +```bash +# Run with memory checking +valgrind --leak-check=full --show-leak-kinds=all \ + --track-origins=yes ./test_binary + +# Static analysis +cppcheck --enable=all --inconclusive source/ +``` + +## Anti-Patterns to Avoid + +```c +// BAD: Magic numbers +if (size > 1024) { ... } + +// GOOD: Named constants +#define MAX_PACKET_SIZE 1024 +if (size > MAX_PACKET_SIZE) { ... } + +// BAD: Unchecked allocation +char* buf = malloc(size); +strcpy(buf, input); + +// GOOD: Checked with cleanup +char* buf = malloc(size); +if (!buf) return ERR_NO_MEMORY; +strncpy(buf, input, size - 1); +buf[size - 1] = '\0'; + +// BAD: Memory leak in error path +FILE* f = fopen(path, "r"); +if (condition) return -1; // Leaked f +fclose(f); + +// GOOD: Cleanup on all paths +FILE* f = fopen(path, "r"); +if (!f) return -1; +if (condition) { + fclose(f); + return -1; +} +fclose(f); +return 0; +``` + +## References + +- Project follows RDK coding standards +- See `uploadstblogs/include/` for uploadSTBLogs API header documentation +- Review existing code in `uploadstblogs/src/` for patterns +- Check `src/unittest/` directory for testing examples diff --git a/.github/instructions/cpp-testing.instructions.md b/.github/instructions/cpp-testing.instructions.md new file mode 100644 index 000000000..28739a25b --- /dev/null +++ b/.github/instructions/cpp-testing.instructions.md @@ -0,0 +1,182 @@ +--- +applyTo: "unittest/**/*.cpp,unittest/**/*.h,uploadstblogs/unittest/**/*.cpp,uploadstblogs/unittest/**/*.h" +--- + +# C++ Testing Standards (Google Test) + +## Test Framework + +Use Google Test (gtest) and Google Mock (gmock) for all C++ test code. + +## Test Organization + +### File Structure +- One test file per source file: `foo.c` → `test/FooTest.cpp` +- Test fixtures for complex setups +- Mocks in separate files when reusable + +```cpp +// GOOD: Test file structure +// filepath: unittest/dcm_utils_gtest.cpp + +extern "C" { +#include "dcm_utils.h" +#include "dcm_types.h" +} + +#include +#include + +class DcmUtilsTest : public ::testing::Test { +protected: + void SetUp() override { + // Initialize test resources + } + + void TearDown() override { + // Clean up test resources + } +}; + +TEST_F(DcmUtilsTest, ConfigFileReadWriteRoundTrip) { + // Test configuration file parsing + const char* config = "/tmp/test.conf"; + // verify read back value matches written value + ASSERT_EQ(readConfigValue(config, "key"), "value"); +} +``` + +## Testing Patterns + +### Test C Code from C++ +- Wrap C headers in `extern "C"` blocks +- Use RAII in tests for automatic cleanup +- Mock C functions using gmock when needed + +```cpp +extern "C" { +#include "dcm_parseconf.h" +#include "dcm_rbus.h" +} + +#include + +class DcmParseConfTest : public ::testing::Test { +protected: + void SetUp() override { + // Initialize handler stubs + } + + void TearDown() override { + // Clean up + } +}; + +TEST_F(DcmParseConfTest, ParseConfigReturnsExpected) { + DCMDHandle handle = {}; + // Test configuration parsing + int result = dcmParseConfig(&handle, "/etc/dcmresponse.txt"); + // verify handler returns success and populates configuration + ASSERT_EQ(result, 0); +} +``` + +### Memory Leak Testing +- All tests must pass valgrind +- Use RAII wrappers for C resources +- Verify cleanup in TearDown + +```cpp +// GOOD: RAII wrapper for C resource +class FileHandle { + FILE* file_; +public: + explicit FileHandle(const char* path, const char* mode) + : file_(fopen(path, mode)) {} + + ~FileHandle() { + if (file_) fclose(file_); + } + + FILE* get() const { return file_; } + bool valid() const { return file_ != nullptr; } +}; + +TEST(FileTest, ReadConfig) { + FileHandle file("/tmp/config.json", "r"); + ASSERT_TRUE(file.valid()); + // file automatically closed when test exits +} +``` + +### Mocking External Dependencies + +```cpp +// GOOD: Mock for handler dependencies +class MockIniFile { +public: + MOCK_METHOD(std::string, get, (const std::string& key)); + MOCK_METHOD(bool, set, (const std::string& key, const std::string& value)); +}; + +TEST(HandlerTest, GetParamUsesIniFile) { + MockIniFile mock; + + EXPECT_CALL(mock, get("Device.DeviceInfo.Manufacturer")) + .WillOnce(testing::Return("TestVendor")); + + std::string result = mock.get("Device.DeviceInfo.Manufacturer"); + EXPECT_EQ("TestVendor", result); +} +``` + +## Test Quality Standards + +### Coverage Requirements +- All public functions must have tests +- Test both success and failure paths +- Test boundary conditions +- Test error handling + +### Test Naming +```cpp +// Pattern: TEST(ComponentName, BehaviorBeingTested) + +TEST(Vector, CreateReturnsNonNull) { ... } +TEST(Vector, DestroyHandlesNull) { ... } +TEST(Vector, PushIncrementsSize) { ... } +TEST(Utils, ParseConfigInvalidJson) { ... } +``` + +### Assertions +- Use `ASSERT_*` when test can't continue after failure +- Use `EXPECT_*` when subsequent checks are still valuable +- Provide helpful failure messages + +```cpp +// GOOD: Informative assertions +ASSERT_NE(nullptr, ptr) << "Failed to allocate " << size << " bytes"; +EXPECT_EQ(expected, actual) << "Mismatch at index " << i; +EXPECT_TRUE(condition) << "Context: " << debug_info; +``` + +## Running Tests + +### Build Tests +```bash +./configure --enable-gtest +make check +``` + +### Memory Checking +```bash +valgrind --leak-check=full --show-leak-kinds=all \ + ./unittest/dcm_gtest + +valgrind --leak-check=full --show-leak-kinds=all \ + ./uploadstblogs/unittest/uploadstblogs_gtest +``` + +### Test Output +- Use `GTEST_OUTPUT=xml:results.xml` for CI integration +- Check return code: 0 = all passed diff --git a/.github/instructions/implementation.instructions.md b/.github/instructions/implementation.instructions.md new file mode 100644 index 000000000..de3e0546d --- /dev/null +++ b/.github/instructions/implementation.instructions.md @@ -0,0 +1,32 @@ +## Implementation Guidelines + +- **Project Goal:** Migrate existing scripts to C code. +- **Target Platforms:** Multiple embedded platforms with low memory and low CPU resources. +- **Constraints:** Code must be efficient, lightweight, and platform-neutral to ensure portability across different embedded systems. + +## Implementation Strategy +1. **Setup Development Environment** + - Use docker containers for consistent build environments. + - Container image that can be used for functional testing - https://github.com/rdkcentral/docker-device-mgt-service-test/pkgs/container/docker-device-mgt-service-test%2Fnative-platform + +2. **Code Development** + - Translate HLD components into modular C code. + - Adhere to coding standards and best practices for embedded systems. + - Implement error handling and logging mechanisms. + - Optimize for memory usage and performance. + - Do not use system calls to best possible extent. + +3. **Code Review and Integration** + - Conduct peer reviews to ensure code quality and adherence to design. + - Integrate modules incrementally and perform integration testing. + +4. **Documentation** + - Update code comments and API documentation. + - Document build and deployment procedures. + - Provide examples and usage guidelines. + - Maintain a changelog for implementation updates. + +5. **Testing** + - Develop unit tests for individual modules. + - Perform system testing on target hardware or simulators. + - Validate against original script functionality and performance criteria. \ No newline at end of file diff --git a/.github/instructions/migrationHLD.instructions.md b/.github/instructions/migrationHLD.instructions.md new file mode 100644 index 000000000..f12f3408c --- /dev/null +++ b/.github/instructions/migrationHLD.instructions.md @@ -0,0 +1,52 @@ +## HLD Generation Guidelines + +- **Project Goal:** Migrate existing scripts to C code. +- **Target Platforms:** Multiple embedded platforms with low memory and low CPU resources. +- **Constraints:** Code must be efficient, lightweight, and platform-neutral to ensure portability across different embedded systems. + +## Migration Strategy +1. **Requirements Gathering** + - For scripts selected in context, create a Markdown (`.md`) file documenting: + - Functional requirements + - Inputs/outputs + - Dependencies + - Constraints (timing, memory, etc.) + - Edge cases and error handling + +2. **High Level Design (HLD)** + - For each script, create a separate HLD `.md` file including: + - Architecture overview + - Module/component breakdown + - Data flow diagrams or descriptions + - Key algorithms and data structures + - Interfaces and integration points + +3. **Flowchart Creation** + - Develop flowcharts to visually represent the script's logic and workflow. + - Use `mermaid` syntax for creating flowcharts. + - For environments that may have issues with complex Mermaid diagrams, include a simplified text-based flowchart alternative. + - For scripts having related functionality, create combined or linked flowcharts to show interactions. + - Use standard flowchart symbols for processes, decisions, inputs/outputs, and connectors. + - Ensure flowcharts are clear, concise, and accurately reflect the script's functionality. + - Include annotations or notes for complex logic or important details. + - Store flowcharts in a dedicated directory within the project for easy reference. + +4. **Sequence Diagrams** + - Create sequence diagrams to illustrate interactions between components or modules. + - Use `mermaid` syntax for creating sequence diagrams. + - For environments that may have issues with complex Mermaid diagrams, include a simplified text-based sequence diagram alternative. + - Ensure diagrams clearly show the order of operations and interactions. + - Include annotations for clarity where necessary. + +5. **LLD Preparation** + - Prepare a Low-Level Design (LLD) document outlining: + - Detailed design specifications + - Data structures and algorithms + - Pseudocode or code snippets + - Interface definitions + - Error handling and edge cases + +5. **Fine tuning** + - Do not create implementation roadmap markdown files. + - Do not suggest timelines or planning details for execution. + diff --git a/.github/instructions/shell-scripts.instructions.md b/.github/instructions/shell-scripts.instructions.md new file mode 100644 index 000000000..a25a2c69b --- /dev/null +++ b/.github/instructions/shell-scripts.instructions.md @@ -0,0 +1,179 @@ +--- +applyTo: "**/*.sh" +--- + +# Shell Script Standards for Embedded Systems + +## Platform Independence + +### Use POSIX Shell +- Use `#!/bin/sh` not `#!/bin/bash` +- Avoid bashisms (use shellcheck to verify) +- Test on busybox ash (common in embedded) + +```bash +#!/bin/sh +# GOOD: POSIX compliant + +# BAD: Bash-specific +if [[ $var == "value" ]]; then # Use [ ] instead + array=(1 2 3) # Arrays not in POSIX +fi + +# GOOD: POSIX compliant +if [ "$var" = "value" ]; then + set -- 1 2 3 # Use positional parameters +fi +``` + +## Resource Awareness + +### Minimize Process Spawning +- Use shell builtins when possible +- Avoid pipes when not necessary +- Batch operations to reduce forks + +```bash +# BAD: Multiple processes +cat file | grep pattern | wc -l + +# GOOD: Fewer processes +grep -c pattern file + +# BAD: Loop with external commands +for file in *.txt; do + cat "$file" >> output +done + +# GOOD: Single cat invocation +cat *.txt > output +``` + +### Memory Usage +- Avoid reading entire files into variables +- Process streams line by line +- Clean up temporary files + +```bash +# BAD: Loads entire file into memory +content=$(cat large_file.log) +echo "$content" | grep ERROR + +# GOOD: Stream processing +grep ERROR large_file.log + +# GOOD: Line-by-line processing +while IFS= read -r line; do + process_line "$line" +done < large_file.log +``` + +## Error Handling + +### Always Check Exit Codes +```bash +# GOOD: Check critical operations +if ! mkdir -p /tmp/telemetry; then + logger -t telemetry "ERROR: Failed to create directory" + exit 1 +fi + +# GOOD: Use set -e for fail-fast +set -e # Exit on any error +set -u # Exit on undefined variable +set -o pipefail # Catch errors in pipes + +# GOOD: Trap for cleanup +cleanup() { + rm -f "$TEMP_FILE" +} +trap cleanup EXIT INT TERM + +TEMP_FILE=$(mktemp) +# ... use temp file ... +# cleanup happens automatically +``` + +## Script Quality + +### Defensive Programming +```bash +# GOOD: Quote all variables +rm -f "$file_path" # Not: rm -f $file_path + +# GOOD: Use -- to separate options from arguments +grep -r -- "$pattern" "$directory" + +# GOOD: Check variable is set +: "${CONFIG_FILE:?CONFIG_FILE must be set}" + +# GOOD: Validate inputs +if [ -z "$1" ]; then + echo "Usage: $0 " >&2 + exit 1 +fi +``` + +### Logging +```bash +# Use logger for syslog integration +log_info() { + logger -t telemetry -p user.info "$*" +} + +log_error() { + logger -t telemetry -p user.error "$*" + echo "ERROR: $*" >&2 +} + +# Usage +log_info "Starting telemetry collection" +if ! start_service; then + log_error "Failed to start service" + exit 1 +fi +``` + +## Testing Scripts + +### Use shellcheck +```bash +# Run shellcheck on all scripts +shellcheck script.sh + +# In CI +find . -name "*.sh" -exec shellcheck {} + +``` + +### Test on Target Platform +- Test on actual embedded device or emulator +- Verify with busybox tools +- Check resource usage (memory, CPU) + +## Anti-Patterns + +```bash +# BAD: Unquoted variables +for file in $FILES; do # Word splitting! + +# GOOD: Quoted +for file in "$FILES"; do + +# BAD: Parsing ls output +for file in $(ls *.txt); do + +# GOOD: Use glob +for file in *.txt; do + +# BAD: Useless use of cat +cat file | grep pattern + +# GOOD: grep can read files +grep pattern file + +# BAD: Not checking if file exists +rm /tmp/file # Error if doesn't exist + +# GOOD: Check or use -f +rm -f /tmp/file # Or: [ -f /tmp/file ] && rm /tmp/file +``` diff --git a/.github/prompts/opsx-apply.prompt.md b/.github/prompts/opsx-apply.prompt.md new file mode 100644 index 000000000..e23ec64d1 --- /dev/null +++ b/.github/prompts/opsx-apply.prompt.md @@ -0,0 +1,149 @@ +--- +description: Implement tasks from an OpenSpec change (Experimental) +--- + +Implement tasks from an OpenSpec change. + +**Input**: Optionally specify a change name (e.g., `/opsx:apply add-auth`). If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes. + +**Steps** + +1. **Select the change** + + If a name is provided, use it. Otherwise: + - Infer from conversation context if the user mentioned a change + - Auto-select if only one active change exists + - If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select + + Always announce: "Using change: " and how to override (e.g., `/opsx:apply `). + +2. **Check status to understand the schema** + ```bash + openspec status --change "" --json + ``` + Parse the JSON to understand: + - `schemaName`: The workflow being used (e.g., "spec-driven") + - Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others) + +3. **Get apply instructions** + + ```bash + openspec instructions apply --change "" --json + ``` + + This returns: + - `contextFiles`: artifact ID -> array of concrete file paths (varies by schema) + - Progress (total, complete, remaining) + - Task list with status + - Dynamic instruction based on current state + + **Handle states:** + - If `state: "blocked"` (missing artifacts): show message, suggest using `/opsx:continue` + - If `state: "all_done"`: congratulate, suggest archive + - Otherwise: proceed to implementation + +4. **Read context files** + + Read every file path listed under `contextFiles` from the apply instructions output. + The files depend on the schema being used: + - **spec-driven**: proposal, specs, design, tasks + - Other schemas: follow the contextFiles from CLI output + +5. **Show current progress** + + Display: + - Schema being used + - Progress: "N/M tasks complete" + - Remaining tasks overview + - Dynamic instruction from CLI + +6. **Implement tasks (loop until done or blocked)** + + For each pending task: + - Show which task is being worked on + - Make the code changes required + - Keep changes minimal and focused + - Mark task complete in the tasks file: `- [ ]` → `- [x]` + - Continue to next task + + **Pause if:** + - Task is unclear → ask for clarification + - Implementation reveals a design issue → suggest updating artifacts + - Error or blocker encountered → report and wait for guidance + - User interrupts + +7. **On completion or pause, show status** + + Display: + - Tasks completed this session + - Overall progress: "N/M tasks complete" + - If all done: suggest archive + - If paused: explain why and wait for guidance + +**Output During Implementation** + +``` +## Implementing: (schema: ) + +Working on task 3/7: +[...implementation happening...] +✓ Task complete + +Working on task 4/7: +[...implementation happening...] +✓ Task complete +``` + +**Output On Completion** + +``` +## Implementation Complete + +**Change:** +**Schema:** +**Progress:** 7/7 tasks complete ✓ + +### Completed This Session +- [x] Task 1 +- [x] Task 2 +... + +All tasks complete! You can archive this change with `/opsx:archive`. +``` + +**Output On Pause (Issue Encountered)** + +``` +## Implementation Paused + +**Change:** +**Schema:** +**Progress:** 4/7 tasks complete + +### Issue Encountered + + +**Options:** +1.