Skip to content

@kvcache-ai kvcache.ai

Change the repository type filter

All

    Repositories list

    20 repositories

    • Mooncake

      Public
      Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
      reinforcement-learninginferencerdma
      reinforcement-learninginferencerdmadisaggregationllmvllmsglangkvcachetrt-llmtokenspeed
      C++
      •
      Apache License 2.0
      •1k•6.1k•186•275•Updated Aug 1, 2026Aug 1, 2026

    • AgentENV

      Public
      AgentENV (AENV) is a distributed platform for running agent environments at scale.
      Rust
      •
      MIT License
      •213•2.7k•27•24•Updated Aug 1, 2026Aug 1, 2026

    • accelerate

      Public
      🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-…
      Python
      •
      Apache License 2.0
      •1.4k•2•0•2•Updated Aug 1, 2026Aug 1, 2026

    • kvcache-blog

      Public
      JavaScript
      •
      MIT License
      •17•22•0•2•Updated Jul 31, 2026Jul 31, 2026

    • firecracker

      Public
      Rust
      •
      Apache License 2.0
      •1•0•0•2•Updated Jul 30, 2026Jul 30, 2026

    • linux

      Public
      Linux kernel source tree for PVM
      C
      •
      Other
      •64k•0•0•0•Updated Jul 30, 2026Jul 30, 2026

    • firecracker-next

      Public
      Secure and fast lightweight microVMs for serverless computing. This is a fork from loopholelabs that adds live migration and optionally PMV support. (No KVM or …
      Rust
      •
      Apache License 2.0
      •2.5k•0•0•0•Updated Jul 30, 2026Jul 30, 2026

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •7.6k•14•0•16•Updated Jul 29, 2026Jul 29, 2026

    • kvcache-blog-private

      Public
      JavaScript
      •
      MIT License
      •17•0•0•0•Updated Jul 25, 2026Jul 25, 2026

    • ktransformers

      Public
      A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
      Python
      •
      Apache License 2.0
      •1.5k•19k•459•20•Updated Jul 24, 2026Jul 24, 2026

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •20k•18•0•0•Updated Jul 1, 2026Jul 1, 2026

    • Model-Optimizer

      Public
      A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models …
      Python
      •
      Apache License 2.0
      •519•1•0•0•Updated Jun 17, 2026Jun 17, 2026

    • transformers

      Public
      🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference an…
      Python
      •
      Apache License 2.0
      •34k•2•0•1•Updated May 9, 2026May 9, 2026

    • sglang_awq

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •7.6k•2•0•0•Updated Apr 22, 2026Apr 22, 2026

    • evalscope

      Public
      A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
      Python
      •
      Apache License 2.0
      •438•0•0•0•Updated Apr 13, 2026Apr 13, 2026

    • DeepEP_fault_tolerance

      Public
      DeepEP: an efficient expert-parallel communication library that supports fault tolerance
      Cuda
      •
      MIT License
      •1.4k•4•0•0•Updated Jan 5, 2026Jan 5, 2026

    • gpustack

      Public
      GPU cluster manager for optimized AI model deployment
      Python
      •
      Apache License 2.0
      •602•1•0•0•Updated Dec 7, 2025Dec 7, 2025

    • TrEnv-X

      Public
      Go
      •
      Apache License 2.0
      •8•97•0•0•Updated Sep 15, 2025Sep 15, 2025

    • sglang-npu

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •7.6k•0•0•0•Updated Aug 12, 2025Aug 12, 2025

    • custom_flashinfer

      Public
      FlashInfer: Kernel Library for LLM Serving
      Cuda
      •
      Apache License 2.0
      •1.2k•8•0•0•Updated Jul 24, 2025Jul 24, 2025
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.