Skip to content

Ensure fusion gating is identity when audio disabled#14

Open
Robbyswimmer wants to merge 1 commit into
masterfrom
codex/investigate-audio-processing-and-model-integration-ucfk20
Open

Ensure fusion gating is identity when audio disabled#14
Robbyswimmer wants to merge 1 commit into
masterfrom
codex/investigate-audio-processing-and-model-integration-ucfk20

Conversation

@Robbyswimmer

Copy link
Copy Markdown
Owner

Summary

  • update the cross-attention block to apply residual updates without forcing a layer-norm so that the base hidden state is unchanged when no audio signal is present
  • gate the residual update produced by the fusion adapter, guaranteeing that gate=0 cleanly skips audio fusion while still allowing gradients to flow when audio is active

Testing

  • pytest -q

https://chatgpt.com/codex/tasks/task_e_68d31a3f3598832a955cac98966a179f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant