Large Language Models

Framework for building LLM-powered applications using composable primitives.

Run LLaMA-class models efficiently on consumer hardware.

Simple local LLM runtime with a clean developer experience.

High-throughput, memory-efficient LLM serving engine.

Unified API interface for 100+ LLM providers.

Programming framework for self-improving LLM pipelines.

The TypeScript toolkit for building AI-powered apps and agents.

Data framework for connecting LLMs to your data via indexing and retrieval.

High-performance serving framework for LLMs and multimodal models.