Skip to content

Optimize and fix GNNLSH#477

Merged
jpata merged 11 commits into
mainfrom
jp_20250421_gnnlsh
Apr 27, 2026
Merged

Optimize and fix GNNLSH#477
jpata merged 11 commits into
mainfrom
jp_20250421_gnnlsh

Conversation

@jpata
Copy link
Copy Markdown
Owner

@jpata jpata commented Apr 23, 2026

  • GNNLSH attention kernels (Gaussian and Attention) and implemented inter-bin
  • Expanded GNNLSHConfig with parameters for kernel_type, use_interbin_attention, and head dimensions.
  • Fixed GNNLSH inter-bin attention export by using symbolic shapes and a manual attention implementation
  • Added new test cases for GNNLSH attention variants in test_mlpf_models.py and introduced test_simple_mha_masking.py to verify attention masking and ONNX compatibility.
  • Removed obsolete CMS data preparation scripts and Tallinn-specific file lists (zee_nopu.txt, zmm.txt, etc.).
  • Updated LitePT tests to gracefully skip when dependencies are unavailable and applied general formatting fixes.

Compare the validation loss of transformer vs. GNNLSH:
Screenshot 2026-04-27 at 10 06 46

Compare the runtime performance on A100 vs loss and ONNX export MAE:
loss_vs_runtime
mae_vs_runtime

@jpata jpata merged commit de861d4 into main Apr 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant