🚀

DeepSpeed

Training & Fine-tuning

Deep learning optimization library for large-scale training.

36kstars4.3kforksPython

About

DeepSpeed from Microsoft enables training of models with billions of parameters using ZeRO optimization, mixed precision, and pipeline parallelism.

Streamlined fine-tuning of LLMs with PEFT and full-param support.

Parameter-efficient fine-tuning methods by Hugging Face.

Transformer Reinforcement Learning for RLHF and alignment.