sliding-window-attention

Here are 7 public repositories matching this topic...

FonaTech / Project_Chronos

⚡ Zero-Stall MoE Inference via Lookahead Prediction & Async DMA Prefetching. Optimized for SSD I/O with Hybrid MLA+Sliding Window Attention.

open-source artificial-intelligence lora high-throughput open-models mixture-of-experts llm generative-ai large-language-model streaming-llm predictive-inference sliding-window-attention io-latency-hiding async-dma ssd-offloading lookahead-routing mla-attention dual-layer-moe

Updated Apr 26, 2026
Python

hkproj / mistral-llm-notes

Star

Notes on the Mistral AI model

nlp pytorch mistral llm xformers mistral-7b mixtral mixtral-8x7b sliding-window-attention

Updated Dec 27, 2023
Jupyter Notebook

Fzkuji / swat-attention

Star

🚀 Sliding Window Attention Training for Efficient Large Language Models

efficiency large-language-models sliding-window-attention

Updated Jun 7, 2026
Python

XunhaoLai / ring-sliding-window-attention

Star

Ring sliding window attention implementation with flash attention

parallel-training large-language-models flash-attention sliding-window-attention

Updated Jul 25, 2025
Python

Ashutosh0x / claude-rust

Sponsor

Star

A high-performance terminal-integrated LLM engine in Rust

nlp rust machine-learning deep-learning tokenizer inference tui transformer attention-mechanism rope kv-cache llm long-context sliding-window-attention

Updated Mar 1, 2026
Rust

alhussein-jamil / vit-hilbert-patches

Star

ViT with Hilbert-curve patch ordering for better 2D locality in sliding-window attention. PyTorch.

computer-vision deep-learning pytorch attention vit hilbert-curve vision-transformer sliding-window-attention

Updated Jun 26, 2026
Python

Faithful from-scratch PyTorch reproduction of OpenAI's GPT-OSS architecture (sliding/full attention alternation, learned attention sinks, YaRN 128K, top-2-of-8 MoE), scaled to Chinchilla-optimal 502M total / 247M active training on a single A100 80GB

yarn pytorch from-scratch mixture-of-experts llm sliding-window-attention gpt-oss attention-sinks

Updated Jun 29, 2026
Python

Improve this page

Add a description, image, and links to the sliding-window-attention topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the sliding-window-attention topic, visit your repo's landing page and select "manage topics."

Learn more

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sliding-window-attention

Here are 7 public repositories matching this topic...

FonaTech / Project_Chronos

hkproj / mistral-llm-notes

Fzkuji / swat-attention

XunhaoLai / ring-sliding-window-attention

Ashutosh0x / claude-rust

alhussein-jamil / vit-hilbert-patches

atandra2000 / GPT-OSS-Lite

Improve this page

Add this topic to your repo