Large Language Models

Framework for building LLM-powered applications using composable primitives.

Run LLaMA-class models efficiently on consumer hardware.

Simple local LLM runtime with a clean developer experience.

High-throughput, memory-efficient LLM serving engine.

Unified API interface for 100+ LLM providers.

Constrained generation and structured output for LLMs.

Programming framework for self-improving LLM pipelines.