A fast communication-overlapping library for tensor/expert parallelism on GPUs.

Trainable fast and memory-efficient sparse attention

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

Examples of CUDA implementations by Cutlass CuTe

CUTLASS and CuTe Examples

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

This is my 🔥 100 Days of GPU — a wild, hands-on journey through CUDA/CUTLASS kernels, Triton spells, and PTX sorcery.

GEMM and Winograd based convolutions using CUTLASS

study of cutlass

Multiple GEMM operators are constructed with cutlass to support LLM inference.

Kernel Library Wheel for SGLang

A cutlass cute implementation of headdim-64 flashattentionv2 TensorRT plugin for LightGlue. Run on Jetson Orin NX 8GB with TensorRT 8.5.2.

Reading notes on the open source code of AI infrastructure (sglang, llm, cutlass, hpc, etc.)

DeepGEMMPerTensor: clean and efficient FP8 GEMM per tensor kernels without scales

pytorch implements block sparse

Docs

Lightweight and production level C++ Open source Library

This repository showcases common optimization techniques for kernels.