Impronta pubblica su GitHub di vLLM

A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

Cost-efficient and pluggable Infrastructure components for GenAI inference

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Community maintained hardware plugin for vLLM on Ascend

Community maintained hardware plugin for vLLM on Apple Silicon

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Common recipes to run vLLM

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

TPU inference for vLLM, with unified JAX and PyTorch support.

A safetensors extension to efficiently store sparse quantized tensors on disk

A high-performance and light-weight router for vLLM large scale deployment

An LLM post-training framework with vLLM for RL Scaling

Fast and memory-efficient exact attention

Agent skills for vLLM

Nessuna descrizione fornita per questo repository.

vLLM Daily Summarization of Merged PRs

The vLLM XPU kernels for Intel GPU

Nessuna descrizione fornita per questo repository.

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

Community maintained hardware plugin for vLLM on Intel Gaudi

Stateful API logic for agentic applications using vLLM

Community maintained hardware plugin for vLLM on AWS Neuron

vLLM plugin for block-based diffusion language model (dLLM) support

Manages vllm-nccl dependency

Nessuna descrizione fornita per questo repository.

vLLM Model plugin for the encoder-decoder BART model

Nessuna descrizione fornita per questo repository.

vLLM Logo Assets

Nessuna descrizione fornita per questo repository.

vLLM Quantization plugin for GGUF

Performance benchmark & accuracy evaluation for vLLM

Nessuna descrizione fornita per questo repository.

Performance dashboard for vLLM

vLLM Quantization plugin for bitsandbytes

Nessuna descrizione fornita per questo repository.

Nessuna descrizione fornita per questo repository.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Nessuna descrizione fornita per questo repository.

Standalone fork of llm-multimodal from SMG