Fast and Memory-Efficient Exact Attention for Large Headdim, 1.5x~6x speedup over PyTorch SDPA.
-
Updated
Aug 12, 2026 - Python
#
tensor-cores
Here are 49 public repositories matching this topic...
⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.
-
Updated
May 10, 2025 - Cuda
General Matrix Multiplication using NVIDIA Tensor Cores
matrix-multiplication cuda-kernels gpu-computing nvidia-cuda nvidia-gpu gpu-programming sgemm cuda-programming tensor-cores nvidia-tensor-cores
-
Updated
Jan 25, 2025 - Cuda
Minimal FlashAttention in CUDA C++/CuTe: readable WMMA/CuTe kernels, no NxN workspace, up to 4.5x faster than naive PyTorch
cuda attention cutlass cute gpu-kernels pytorch-extension tensor-cores llm flash-attention flashattention wmma
-
Updated
Jun 2, 2026 - Cuda
Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell
python deep-learning gpu cuda pytorch high-performance-computing triton gpu-acceleration quantization inference-engine mixture-of-experts tensor-cores blackwell ai-inference llm fp8 llm-serving llm-inference speculative-decoding nvfp4
-
Updated
Jun 8, 2026 - Python
Handwritten Flash Attention 2 CUDA kernel for Blackwell (SM120) with TMA, swizzle, double buffering & warp specialization
-
Updated
May 25, 2026 - Cuda
CUDA matrix multiplication benchmarking on Jetson Orin Nano. Four implementations, three power modes, five matrix sizes. 99.5% mathematical validation. C++/CUDA and Python.
machine-learning robotics cuda cublas scientific-computing matrix-multiplication high-performance-computing gpu-computing performance-optimization autonomous-systems edge-computing nvidia-jetson edge-ai embeded-systems tensor-cores ml-deployment jetson-orin-nano gpu-benchmarking power-efficiency-benchmark cuda-optimization
-
Updated
Apr 2, 2026 - Python
Hand-written NVFP4 W4A16 CUDA kernels and chain-MTP speculative serving — Qwen3.6-27B at up to 366 tok/s on four Tesla V100s, hardware with no FP4 support
gpu cuda nvidia cuda-kernels quantization gemm performance-optimization inference-engine volta tensor-cores int4 llm-serving vllm llm-inference qwen speculative-decoding fp4 tesla-v100 nvfp4 sm70
-
Updated
Aug 11, 2026 - Python
Vulkan & GLSL implementation of FlashAttention-2
vulkan glsl artificial-intelligence gpu-acceleration attention gpu-computing deel-learning tensor-cores large-language-models llm flash-attention flash-attention-2
-
Updated
Jan 19, 2025 - C++
CUDA 12-first backend inference for Unsloth on Kaggle — Optimized for small GGUF models (1B-5B) on dual Tesla T4 GPUs (15GB each, SM 7.5)
python machine-learning ai deep-learning jupyter gpu cuda inference pytorch nvidia cuda-kernels google-colab tensor-cores tesla-t4 llm gguf unsloth flashattention
-
Updated
Feb 1, 2026 - Jupyter Notebook
Base-13 FP4 limbs for Ozaki-scheme DGEMM emulation on FP4 tensor cores
hpc emulation cuda triton gemm numerical-linear-algebra dgemm mixed-precision tensor-cores blackwell fp4 ozaki-scheme
-
Updated
Aug 11, 2026 - Python
A from-scratch nano-Triton and nano-Helion: the modern GPU-kernel DSL stack, rebuilt in ~4,000 lines
-
Updated
Jul 25, 2026 - Python
Compiler MVP that detects Transformer fusion patterns, generates optimized CUDA kernels with WMMA Tensor Cores, and executes them on real GPU hardware — 10.5 TFLOPs on RTX 2070, correctness validated against PyTorch.
performance-engineering deep-learning compiler jinja2 gpu cuda transformer fusion systems code-generation turing gemm auto-tuning tensor-cores llm machine-learning-infrastructure kernel-optimization wmma rtx-2070 sm75
-
Updated
Jul 26, 2026 - Python
The only deployed sparse FP4 GEMM on SM120: beats CUTLASS 80b on every shape, wins end-to-end request latency in 81 of 112 serving regimes vs dense NVFP4.
cuda sparse quantization gpu-kernels mixed-precision tensor-cores blackwell vllm llm-inference fp4 sm120 nvfp4 2-4-sparsity
-
Updated
Aug 4, 2026 - Python
Low-bit LLM inference engine in Rust + CUDA. Converts models to a 4-bit .wstone format (3.75x smaller) and runs them on consumer GPUs. Built for memory-bandwidth-bound decode on Turing, where bytes per weight — not TOPS — sets token throughput.
rust machine-learning deep-learning gpu cuda transformers cuda-kernels gpu-computing quantization performance-optimization inference-engine model-compression edge-ai tensor-cores low-bit-quantization int4 llm gptq llm-inference qwen
-
Updated
Jul 29, 2026 - Rust
A benchmarking framework for correlators of FX telescope arrays
-
Updated
Oct 20, 2023 - Cuda
1,100+ FPS YOLOv8/YOLO11 inference on a $400 GPU — every kernel hand-written in one CUDA file. No cuDNN, no TensorRT, no Python at runtime. All scales, all task heads, same detections.
real-time computer-vision deep-learning gpu high-performance cuda yolo object-detection cuda-kernels inference-engine ptx tensor-cores yolov8 yolov11
-
Updated
Jul 30, 2026 - Cuda
FlashAttention v1 forward pass in CUDA for NVIDIA Turing (SM75)
deep-learning cuda pytorch attention gpu-computing turing tensor-cores llm flash-attention machine-learning-infrastructure wmma sm75
-
Updated
Jul 27, 2026 - Python
Production-grade FlashAttention FP8 e4m3 forward kernel for NVIDIA Blackwell consumer GPUs (sm_120a, e.g. RTX PRO 6000). 647–652 TFLOPS at hd=128, sl=8192. Multi-kernel dispatcher, C library with Go and Python bindings
cuda transformer attention gpu-kernels tensor-cores blackwell fp8 flash-attention fp8e4m3 sm120 rtx-pro-6000 sm-120a
-
Updated
Aug 3, 2026 - Cuda
Drop-in exact bf16 flash-attention for CUDA with a deterministic backward, tuned for Blackwell (sm_120 / RTX 5090).
deep-learning gpu cuda inference pytorch transformer triton attention memory-efficient attention-mechanism deterministic memory-optimization bfloat16 tensor-cores blackwell kv-cache llm-inference flash-attention rtx-5090 sm120
-
Updated
Jul 8, 2026 - Python
Improve this page
Add a description, image, and links to the tensor-cores topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the tensor-cores topic, visit your repo's landing page and select "manage topics."