Dấu chân GitHub công khai của vLLM

A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

Cost-efficient and pluggable Infrastructure components for GenAI inference

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Community maintained hardware plugin for vLLM on Ascend

Community maintained hardware plugin for vLLM on Apple Silicon

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Common recipes to run vLLM

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

TPU inference for vLLM, with unified JAX and PyTorch support.

A safetensors extension to efficiently store sparse quantized tensors on disk

A high-performance and light-weight router for vLLM large scale deployment

An LLM post-training framework with vLLM for RL Scaling

Fast and memory-efficient exact attention

Agent skills for vLLM

Không có mô tả nào được cung cấp cho kho lưu trữ này.

vLLM Daily Summarization of Merged PRs

The vLLM XPU kernels for Intel GPU

Không có mô tả nào được cung cấp cho kho lưu trữ này.

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

Community maintained hardware plugin for vLLM on Intel Gaudi

Stateful API logic for agentic applications using vLLM

Community maintained hardware plugin for vLLM on AWS Neuron

vLLM plugin for block-based diffusion language model (dLLM) support

Manages vllm-nccl dependency

Không có mô tả nào được cung cấp cho kho lưu trữ này.

vLLM Model plugin for the encoder-decoder BART model

Không có mô tả nào được cung cấp cho kho lưu trữ này.

vLLM Logo Assets

Không có mô tả nào được cung cấp cho kho lưu trữ này.

vLLM Quantization plugin for GGUF

Performance benchmark & accuracy evaluation for vLLM

Không có mô tả nào được cung cấp cho kho lưu trữ này.

Performance dashboard for vLLM

vLLM Quantization plugin for bitsandbytes

Không có mô tả nào được cung cấp cho kho lưu trữ này.

Không có mô tả nào được cung cấp cho kho lưu trữ này.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Không có mô tả nào được cung cấp cho kho lưu trữ này.

Standalone fork of llm-multimodal from SMG