Running a 30B MoE LLM on CPU at 30-50+ tok/s with a 256K Context Window via Hybrid Mamba-2 SSM, L3 NTA Slicing, AVX-512 VNNI Tile Repacking & 2MB HugePages
-
Updated
Nov 1, 2026 - Python
#
mamba2
Here are 26 public repositories matching this topic...
Rust-native MoE inference runtime with custom CUDA kernels for Blackwell GPUs. Includes DFlash speculative decoding, multi-tier Engram memory, and entropy-adaptive routing. Targets Qwen3.5-35B-A3B on a single RTX 5060 Ti 16GB.
rust ffi cuda inference moe quantization mamba state-space-models deltanet blackwell engram llm qwen speculative-decoding sm120 mamba2 nemotron-h hybrid-ssm
-
Updated
Jul 28, 2026 - Rust
Hybrid Mamba-2 + Transformer 2.94B LLM (Nemotron-H style) — Korean 3B model pretrained from scratch on 7× NVIDIA B200 GPUs with SFT + DPO alignment
-
Updated
Mar 26, 2026 - Python
Hybrid SSM-Attention language model on Apple Silicon with MLX — interleaving Mamba-2 and Transformer for efficient inference
python machine-learning deep-learning transformer attention language-model ssm mamba hybrid-model mlx state-space-model apple-silicon llm mamba2
-
Updated
Mar 29, 2026 - Python
The fastest Mamba2 SSD kernel for TPU written in Pallas
-
Updated
Jul 31, 2026 - HTML
Mamba-2-130M encrypted inference research with CKKS on DGX Spark. Python reference, native GPU kernel, and reproducible experiments.
cryptography cuda pytorch gpu-acceleration language-models homomorphic-encryption mamba state-space-models fhe privacy-preserving-ml ckks fully-homomorphic-encryption research-prototype llm openfhe mamba2 encrypted-inference fideslib
-
Updated
Oct 2, 2026 - Python
Native AscendC Mamba2 selective scan / SSD forward-backward custom operator for Huawei Ascend 910B3 and 950PR, with CANN, torch_npu, A100 benchmarks and msprof profiling.
pytorch ssd mamba cann performance-optimization state-space-models ascend-npu selective-scan ai-infrastructure ascendc kernel-optimization mamba2 huawei-ascend torch-npu custom-operator ascend-910b triton-ascend ascend-950
-
Updated
Oct 26, 2026 - Python
UAE / Saudi LLM llamafile — Single-file, no-install executables for Falcon, Jais, ALLaM, and more.
executable falcon uae single-file saudi-arabia llm rwkv local-llm gguf llamafile jais allam mamba2 falcon-h1
-
Updated
Jun 20, 2026 - Shell
Helix-v2 1B bilingual hybrid LLM - Nvidia
ai deep-learning german cuda transformers pytorch adapters lora mora dora hybrid-model fine-tuning state-space-models tool-use sentencepiece llm generative-ai large-language-model mamba2 gated-linear-attention
-
Updated
Jul 14, 2026 - Python
VerDee: Vertical Deep Network on Mamba-2 — staged LoRA (shallow/mid/deep), early-exit routing, and domain experts. Pilot on 370M; 16GB GPU friendly. Research/experiment repo.
lora state-space-model peft early-exit huggingface qlora llm-finetuning mamba2 vertical-deep-network
-
Updated
May 25, 2026 - Python
Build Your Own Mamba — From Math to Metal
-
Updated
Jun 13, 2026 - Python
Run NVIDIA's Nemotron-Labs-TwoTower-30B-A3B block-wise diffusion LM natively on Apple Silicon with MLX — AR tower + full two-tower mask-diffusion, 4/6/8-bit + bf16.
macos text-generation nvidia moe quantization mamba mlx diffusion-model mixture-of-experts on-device-ai apple-silicon large-language-models llm nemotron diffusion-language-model mamba2
-
Updated
Jul 2, 2026 - Python
A 390M-parameter Mamba2 + Differential Attention hybrid language model
nlp deep-learning language-model hybrid-architecture mamba2 differential-attention pytorch-state-space-model
-
Updated
May 8, 2026 - Python
GLACIER: Mamba with indexless memory. This project integrates the Mamba SSM with ICE-Lite, a virtual memory engine, to solve context rot. By adding persistent, time-aware memory, GLACIER gives Mamba the long-term recall of a Transformer while retaining its $O(N)$ speed. Apache 2.0 licensed, by Dopove.
python nlp ai deep-learning postgresql persistent-memory glacier ssm mamba state-space-model vector-database llm long-context context-memory memory-augmented transformer-alternative mamba2 context-rot ice-lite dopove
-
Updated
May 22, 2026 - Python
Krait: Strata for NVIDIA Nemotron 3.5 Super. Fast local inference engine for the 120B hybrid Mamba-2/MoE model on one 32 GB GPU (RTX 5090): 1.6x llama.cpp decode, OpenAI & Anthropic API, vision, Docker + OpenWebUI. Based on Strata by Niko1221.
cuda inference strata mamba inference-engine mixture-of-experts openai-api vision-language-model llama-cpp local-llm llm-inference local-ai gguf speculative-decoding open-webui nemotron rtx-5090 nvidia-nemotron mamba2 nemotron-super
-
Updated
Oct 29, 2026 - C++
Unofficial PyTorch reproduction for Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality.
pytorch reproduction state-space-model sequence-modeling efficient-attention mamba2 unofficial-implementation
-
Updated
Jul 2, 2026 - Python
llama.cpp fork with additional SOTA quants and improved performance
cuda inference llama cuda-kernels quantization ssm mamba state-space-models blackwell llama-cpp gguf sm120 mamba2 nemotron-h hybrid-ssm
-
Updated
Apr 26, 2026 - C++
Add this topic to your repo
To associate your repository with the mamba2 topic, visit your repo's landing page and select "manage topics."