Pegada pública no GitHub de vLLM

A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

Cost-efficient and pluggable Infrastructure components for GenAI inference

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Community maintained hardware plugin for vLLM on Ascend

Community maintained hardware plugin for vLLM on Apple Silicon

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Common recipes to run vLLM

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

TPU inference for vLLM, with unified JAX and PyTorch support.

A safetensors extension to efficiently store sparse quantized tensors on disk

A high-performance and light-weight router for vLLM large scale deployment

An LLM post-training framework with vLLM for RL Scaling

Fast and memory-efficient exact attention

Agent skills for vLLM

Nenhuma descrição fornecida para este repositório.

vLLM Daily Summarization of Merged PRs

The vLLM XPU kernels for Intel GPU

Nenhuma descrição fornecida para este repositório.

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

Community maintained hardware plugin for vLLM on Intel Gaudi

Stateful API logic for agentic applications using vLLM

Community maintained hardware plugin for vLLM on AWS Neuron

vLLM plugin for block-based diffusion language model (dLLM) support

Manages vllm-nccl dependency

Nenhuma descrição fornecida para este repositório.

vLLM Model plugin for the encoder-decoder BART model

Nenhuma descrição fornecida para este repositório.

vLLM Logo Assets

Nenhuma descrição fornecida para este repositório.

vLLM Quantization plugin for GGUF

Performance benchmark & accuracy evaluation for vLLM

Nenhuma descrição fornecida para este repositório.

Performance dashboard for vLLM

vLLM Quantization plugin for bitsandbytes

Nenhuma descrição fornecida para este repositório.

Nenhuma descrição fornecida para este repositório.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Nenhuma descrição fornecida para este repositório.

Standalone fork of llm-multimodal from SMG