Back to Resources
⚡

vLLM

LLMs

High-throughput, memory-efficient LLM serving engine.

25kstars3kforksPython
View on GitHub

About

vLLM is designed for fast LLM inference and serving using paged attention. It is widely used for production-scale deployments.

Key Features

  • Paged attention
  • Fast inference
  • OpenAI-compatible API

Tags

InferenceServingPerformanceProduction

Related Resources

🦜

LangChain

Framework for building LLM-powered applications using composable primitives.

🦙

llama.cpp

Run LLaMA-class models efficiently on consumer hardware.

🦙

Ollama

Simple local LLM runtime with a clean developer experience.