Topic/rdkemw 12922 by jthomp007c · Pull Request #78 · rdkcentral/xr-voice-sdk

jthomp007c · 2026-07-09T19:37:29Z

No description provided.

Status: Code is messy needs major clean-up, but does map FFV HAL functionality and register callbacks for hardware failure and keyword detection, gets privacy mode. I have removed keyword detection from xraudio entirely because the HAL does that. What I'm working on right now is that when the HAL reports keyword detection, xraudio_keyword_detector_session_event() is called which expects to have the detector->callback() set which it is null so there is a segfault.

callback, getting keyword detection, starting to read audio from HAL Am not correctly setting the read fd in xraudio_thread. Am not seeing voice ended cb - did see this previously. Am not stopping reading audio. Audio data is trash

callback works. Keyword detection callback works. It attempts to start a voice session upon keyword detection, but this fails in two ways. One, the session init json is not formatted correctly so VREX rejects the session. Two, the mic data is bad. This is what I'm focusing on right now.

Copilot

Pull request overview

This PR introduces an FFV HAL interface adapter path (non-USE_RDKV_HAL) and adds multiple USE_RDKV_HAL compile-time conditionals across xraudio and xrsr to alter keyword detection / HAL interaction behavior.

Changes:

Add an FFV HAL “interface plugin” adapter (vsdk_ffv_adapter.*) and update plugin loading logic to fall back to it.
Add USE_RDKV_HAL-gated behavior changes in keyword detection, session begin handling, and detector lifecycle logic across xrsr/xraudio.
Update build inputs/options to include the new adapter source.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/xr-speech-router/xrsr.c	Gates keyword detector per-channel fields/logging under `USE_RDKV_HAL`.
src/xr-speech-router/xrsr_xraudio.c	Adds `USE_RDKV_HAL` conditionals around detector lifecycle and event handling.
src/xr-audio/xraudio.h	Makes keyword detector result struct conditional on `USE_RDKV_HAL`.
src/xr-audio/xraudio.c	Adds non-`USE_RDKV_HAL` behavior in device query and sensitivity-limits API.
src/xr-audio/xraudio_thread.c	Adds FFV HAL interface include and `USE_RDKV_HAL` conditional logic in thread processing.
src/xr-audio/xraudio_output.c	Gates EOS/dsp_config init under `USE_RDKV_HAL`.
src/xr-audio/xraudio_input.c	Adds `USE_RDKV_HAL` conditionals around idle/detect dispatch and open logic.
src/xr-audio/xr_ffv_hal_plugin.h	Adds FFV HAL C API wrapper header (copied in-repo).
src/xr_ffv_hal_interface.h	Adds FFV HAL interface function-table definitions and helpers.
src/vsdk.c	Updates plugin loading to support FFV HAL interface adapter path and `USE_RDKV_HAL` split.
src/vsdk_ffv_adapter.h	Declares the adapter API used when not `USE_RDKV_HAL`.
src/vsdk_ffv_adapter.c	Implements adapter bridging FFV HAL interface into xraudio HAL plugin API.
src/CMakeLists.txt	Adds adapter source and enables `-g` globally.

+   char *error = dlerror();
+
+   #ifdef USE_RDKV_HAL
   xraudio_eos_plugin_api_get_t eos_plugin_api_get = (xraudio_eos_plugin_api_get_t)dlsym(handle, "xraudio_eos_plugin_api_get");
   char *error = dlerror();


+   const char *so_path_vd = "/data/jason/usr/lib/libxr-ffv-hal.so";
+   const char *so_path_mw = "/data/jason/usr/usr/lib/libxr-ffv-hal.so";


+   #else
+   return(XRAUDIO_RESULT_OK);
+   #endif


+   #ifdef USE_RDKV_HAL
   if(!obj->detect_active) {
      XLOGD_INFO("ignore keyword detect event");
      return;
   }
+   #endif


         XLOGD_WARN("Rejecting keyword detected from source <%s>, session in progress on source <%s>.  Restarting keyword detector...", xrsr_src_str(src), xrsr_src_str(current_session_src));
         obj->session_rejected = true;
+         #ifdef USE_RDKV_HAL
         xrsr_xraudio_keyword_detect_restart(obj);
+         //TODO inform new HAL to restart detection? The HAL might do that automatically after some time or after reading mic data, check on this
+         #endif


   uint32_t                  chan_selected;
   xraudio_kwd_endpoints_t   endpoints;
+   #ifdef USE_RDKV_HAL
   xraudio_kwd_chan_result_t channels[XRAUDIO_INPUT_MAX_CHANNEL_QTY];
+   #endif
   const char *              detector_name;
   const char *              dsp_name;
   float                     sensitivity;


+   #ifdef USE_RDKV_HAL
+     xraudio_session_record_inst_t *instance = &state->record.instances[XRAUDIO_INPUT_SESSION_GROUP_DEFAULT];
   if(!xraudio_keyword_detector_session_is_armed(&state->record.keyword_detector)) {
      XLOGD_ERROR("detector is not armed");
      return;
   }
+   #endif


               xr-audio/xraudio_atomic.c)

-target_compile_options(xr-voice-sdk PUBLIC -fPIC -rdynamic -Wall -Werror)
+target_compile_options(xr-voice-sdk PUBLIC -fPIC -rdynamic -Wall -Werror -g)


+   static int doit = 1;
+
+   if(eos_event != NULL) {
+      memset(eos_event, 0, sizeof(*eos_event));
+   }
+
+   ssize_t rc = read(obj->fd, data, size);
+   if(rc < 0) {
+      if((errno == EAGAIN) || (errno == EINTR)) {
+         return(0);
+      }
+      return(-1);
+   }
+   if(doit) {
+      for(uint32_t i = 0; i < (uint32_t)rc; i+=8) {
+         XLOGD_INFO("data[%d]: 0x%02x 0x%02x 0x%02x 0x%02x 0x%02x 0x%02x 0x%02x 0x%02x", i, data[i], data[i+1], data[i+2], data[i+3], data[i+4], data[i+5], data[i+6], data[i+7]);
+      }
+      doit = 0;
+   }
+   return((int32_t)rc);


+   #ifndef USE_RDKV_HAL
+   obj->devices_input |= *inputs;
+   XLOGD_INFO("obtained additional input devices from HAL plugin: <%s>", xraudio_devices_input_str(obj->devices_input));
+   #endif



   xraudio_input_sound_focus_set(state->params.obj_input, XRAUDIO_SDF_MODE_KEYWORD_DETECTION);
+   #else
+   XLOGD_INFO("Registering callback %p at state->record.keyword_detector with param 0x%x", detect->callback, detect->param);


Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 10 comments.

+   char *error = dlerror();
+
+   #ifdef USE_RDKV_HAL
   xraudio_eos_plugin_api_get_t eos_plugin_api_get = (xraudio_eos_plugin_api_get_t)dlsym(handle, "xraudio_eos_plugin_api_get");
   char *error = dlerror();


+   #else
+   const char *so_path_vd = "/data/jason/usr/lib/libxr-ffv-hal.so";
+   const char *so_path_mw = "/data/jason/usr/usr/lib/libxr-ffv-hal.so";
+   #endif


   obj->play_bumper          = 0;
-   obj->dsp_config           = *dsp_config;
   obj->hal_plugin           = vsdk_hal_plugin_get();
-   obj->eos_plugin           = vsdk_eos_plugin_get();
   obj->ovc_plugin           = vsdk_ovc_plugin_get();
-
+   #ifdef USE_RDKV_HAL
+   obj->dsp_config           = *dsp_config;



   xraudio_input_sound_focus_set(state->params.obj_input, XRAUDIO_SDF_MODE_KEYWORD_DETECTION);
+   #else
+   XLOGD_INFO("Registering callback %p at state->record.keyword_detector with param 0x%x", detect->callback, detect->param);


 void xraudio_msg_thread_poll(xraudio_thread_state_t *state, void *msg) {
   xraudio_main_queue_msg_thread_poll_t *thread_poll = (xraudio_main_queue_msg_thread_poll_t *)msg;

+   XLOGD_WARN("state->params.kwd_plugin %p, state->params.out_enabled %d, state->params.hal_plugin %p", state->params.kwd_plugin, state->params.out_enabled, state->params.hal_plugin); 


+   //This is not a good test but will do for the moment
+   if(format.sample_size == 2) {
+      XLOGD_INFO("sample_size is %d, not a keyword channel", format.sample_size);
+      return(false);
+   }
+   return(true);


+   FFVhalApiStatus_t status = ffv_api->get_capabilities(handle, &ffv_caps);
+   if(status == EX_NONE) {
+      caps->input_qty      = 1;
+      caps->input_caps[0]  = XRAUDIO_CAPS_INPUT_LOCAL | XRAUDIO_CAPS_INPUT_SELECT;
+      if(ffv_caps.channelTypes[0] != NULL || ffv_caps.channelTypes[1] != NULL) {
+         caps->input_caps[0] |= XRAUDIO_CAPS_INPUT_EOS_DETECTION;
+      }
+      if(ffv_caps.microphoneChannelCount > 1) {
+         caps->input_caps[0] |= XRAUDIO_CAPS_INPUT_LOCAL_32_BIT;
+      }
+      caps->output_qty = 0;
+   }
+}


+   if((obj->hal_obj->ffv_api != NULL) && (obj->hal_obj->ffv_api->close_channel != NULL) && (obj->channel_type != NULL)) {
+      (void)obj->hal_obj->ffv_api->close_channel(obj->hal_obj->ffv_controller, obj->channel_type);
+   }
+   if(obj->fd >= 0) {
+      (void)close(obj->fd);
+   }


+   const char *channel = vsdk_ffv_channel_is_keyword(format) ? "KEYWORD" : "MICROPHONES";
+   FFVhalFileDescriptor fd = -1;
+   XLOGD_INFO("channel %s, device 0x%x", channel, device);
+   if(obj->ffv_api->open_channel(obj->ffv_controller, channel, &fd) != EX_NONE) {


               xr-audio/xraudio_atomic.c)

-target_compile_options(xr-voice-sdk PUBLIC -fPIC -rdynamic -Wall -Werror)
+target_compile_options(xr-voice-sdk PUBLIC -fPIC -rdynamic -Wall -Werror -g)


dwolaver · 2026-07-10T15:24:11Z

+# limitations under the License.
+##########################################################################
+*/
+#ifndef USE_RDKV_HAL


Just don't compile this file at all instead of wrapping it in ifdef

dwolaver · 2026-07-10T15:28:59Z

               xraudio_config.h
               rdkx_logger.h
               vsdk.c
+               vsdk_ffv_adapter.c


conditionally add this file

dwolaver · 2026-07-10T15:29:33Z

               xr-audio/xraudio_atomic.c)

-target_compile_options(xr-voice-sdk PUBLIC -fPIC -rdynamic -Wall -Werror)
+target_compile_options(xr-voice-sdk PUBLIC -fPIC -rdynamic -Wall -Werror -g)


Adding -g shouldn't be necessary. obviously don't commit this...

dwolaver · 2026-07-10T15:32:39Z

 static void  vsdk_parse_options(bool *curtail_xlog, bool *curtail_xraudio, bool *xraudio_allow_input_failure);
 static bool  vsdk_load_plugin_ffv(vsdk_ffv_plugin_handles_t *handles);
 static void *vsdk_load_plugin_ffv_hal(bool *out_enabled);
+#ifdef USE_RDKV_HAL //We can put these back later. Right now it's going to load the default ones along with the new HAL, so just avoiding it for the time being


This should include the entire RDKV FFV HAL (hal, kwd, alg, sdf, ovc) - completely replaced by the new FFV HAL.

dwolaver · 2026-07-10T15:36:29Z

+   #ifdef USE_RDKV_HAL
   const char *so_path_vd = "/vendor/lib/libxraudio-ffv-hal.so";
   const char *so_path_mw = "/usr/lib/libxraudio-ffv-hal.so";
+   #else


I'm not sure there is any value in trying to re-use this function for the new FFV HAL. Seems like it would be better to create a new function instead.

jthomp007c added 4 commits July 3, 2026 11:35

Added vsdk_ffv_adapter.c to use the new adapter functions

8e969f3

Copilot AI review requested due to automatic review settings July 9, 2026 19:37

jthomp007c requested a review from a team as a code owner July 9, 2026 19:37

Copilot started reviewing on behalf of jthomp007c July 9, 2026 19:37 View session

jthomp007c marked this pull request as draft July 9, 2026 19:39

Copilot AI reviewed Jul 9, 2026

View reviewed changes

github-advanced-security AI found potential problems Jul 9, 2026

View reviewed changes

Comment thread src/xr-audio/xraudio_thread.c

xraudio_input_sound_focus_set(state->params.obj_input, XRAUDIO_SDF_MODE_KEYWORD_DETECTION);

#else

XLOGD_INFO("Registering callback %p at state->record.keyword_detector with param 0x%x", detect->callback, detect->param);

jthomp007c requested a review from Copilot July 9, 2026 21:16

Copilot started reviewing on behalf of jthomp007c July 9, 2026 21:16 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

dwolaver reviewed Jul 10, 2026

View reviewed changes

Navigation Menu

Uh oh!

Topic/rdkemw 12922 - #78

Topic/rdkemw 12922#78
jthomp007c wants to merge 4 commits into
developfrom
topic/RDKEMW-12922

jthomp007c commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

dwolaver Jul 10, 2026

Uh oh!

dwolaver Jul 10, 2026

Uh oh!

dwolaver Jul 10, 2026

Uh oh!

dwolaver Jul 10, 2026

Uh oh!

dwolaver Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		const char *so_path_vd = "/data/jason/usr/lib/libxr-ffv-hal.so";
		const char *so_path_mw = "/data/jason/usr/usr/lib/libxr-ffv-hal.so";

Uh oh!

Conversation

jthomp007c commented Jul 9, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

dwolaver Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

dwolaver Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

dwolaver Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

dwolaver Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

dwolaver Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants