sgl-project/sglang

▲ 763 stars today★ 38,317⑂ 10,042

SGLang is a high-performance serving framework for large language models and multimodal models.

About sgl-project/sglang

sgl-project/sglang is an open-source project on GitHub, mainly written in Python. SGLang is a high-performance serving framework for large language models and multimodal models. It currently holds 38,317 stars and 10,042 forks with 6,309 open issues, and was last pushed on 2026-10-31 (repository created 2024-01-08).

Project Overview

Git Homed tracks it on the Today's Trending board.

GitHub Repository Details

Repository sgl-project/sglang · default branch main · size 488935 KB · watchers 184 · source: GitHub REST API and repository README

README

SGLang: Fast inference for LLMs and multimodal models

https://github.com/sgl-project/sglang/blob/HEAD/SGLang

https://github.com/sgl-project/sglang/blob/HEAD/PyPI version https://github.com/sgl-project/sglang/blob/HEAD/License: Apache 2.0 https://github.com/sgl-project/sglang/blob/HEAD/PyPI downloads per month

Docs | Cookbook | Website | Blog | Slack

SGLang is an open-source inference framework for large language, vision-language, and diffusion models, optimized for agentic workloads, large-scale serving, and RL rollouts. SGLang Diffusion is its built-in image and video generation engine, included in this repository and the sglang Python package.

👋 Get started below, or meet the community at SGLang Events, including meetups, developer meetings, workshops, and office hours.

SGLang Summit 2026: Meet the builders of open AI
Our flagship annual gathering · November 12–13 · San Francisco
> Go deep on inference, training, and AI hardware through technical talks, live demos, and hands-on workshops. Bring your deployment challenges and meet the engineers and researchers building the stack.

Get Started

Pull the Docker image, which includes SGLang and its dependencies:

docker pull lmsysorg/sglang:latest

Alternatively, install SGLang in an activated Python environment with uv:

uv pip install --prerelease=allow sglang

Next, launch your model:

Supported Hardware

SGLang supports a wide range of GPUs, TPUs, NPUs, CPUs, and Apple Silicon platforms.

| Platform | Representative hardware | | --- | --- | | NVIDIA | B200/B300/GB200/GB300; H100/H200/H800/H20; A100; RTX 30/40/50 series, RTX 6000 Ada / PRO 6000; DGX Spark, Jetson Orin | | AMD | Instinct MI300X, MI325X, MI350X, MI355X | | Google TPU | v6e, v7; SGLang-TorchTPU / SGL-JAX | | Intel (GPU / CPU) | Arc / Arc Pro B-Series GPUs, Xeon CPUs | | Apple Silicon | Macs via Metal / MLX | | Huawei Ascend | A2, A3, 950PR/DT NPUs | | Moore Threads | MTT S5000 GPUs |

Integrations in progress: AWS Trainium, Alibaba T-Head PPU, Cambricon MLU, Qualcomm QAIC, MetaX, Hygon HCU/DCU, Iluvatar CoreX, and more.

SGLang-TorchTPU brings the SGLang serving engine to Google TPUs through PyTorch, torch_tpu, and Pallas kernels. It supports LLM and vision-language serving on TPU7x (Ironwood), including tensor and expert parallelism across hosts. SGL-JAX provides the JAX-based TPU serving path.

See the Cookbook and platform guides for model compatibility and setup.

SGL Ecosystem

| Area | Projects | Purpose | | --- | --- | --- | | Diffusion | SGLang Diffusion | Built into SGLang for image and video generation with diffusion models. | | Audio | SGLang Omni | Audio model serving for text-to-speech (TTS) and automatic speech recognition (ASR). | | Speculative Decoding | SpecForge | Train draft models for speculative decoding and deploy them with SGLang. | | RL and Post-Training | Miles, slime, AReaL, Tunix, verl | Training frameworks that integrate SGLang for rollout generation. | | Deployment and Orchestration | SMG, RBG, llm-d, Ray Serve, NVIDIA Dynamo | Deploy and scale SGLang inference services with routing, load balancing, and cluster orchestration. | | Education | Mini-SGLang, zero-to-sglang, DeepLearning.AI course | Learn inference engine design and efficient text and image generation through code and hands-on courses. |

Development and Contributing

Contributions are welcome, from bug fixes and documentation to model support and performance improvements.

Development setup

Start from the lmsysorg/sglang:dev Docker image, which provides development tools and most dependencies. Clone or mount your SGLang checkout inside the container, then install it in editable mode from the repository root so tests use your local Python changes:

pip install -e "python"

In an activated virtual environment, you can use uv pip install --prerelease=allow -e "python" instead. See the development guide for container setup and testing.

Contribute

1. Fork the repository and create a branch for your changes. For larger changes, discuss your proposal in a GitHub issue or on Slack. 2. Make your changes, run the relevant tests, and add regression coverage for fixes or new behavior. Run pre-commit run --all-files before submitting. 3. Open a pull request describing the change and how you tested it. Include benchmarks or accuracy evaluations when relevant.

See the contributor guide for formatting, testing, and pull request instructions. Documentation contributors can start with the docs guide.

Community and Sponsorship

SGLang is hosted by LMSYS, a non-profit open-source organization.

Trusted by Industry and Research

SGLang serves production workloads across AI labs, cloud platforms, enterprises, and universities.

https://github.com/sgl-project/sglang/blob/HEAD/Organizations adopting SGLang

Acknowledgment

We learned the design and reused code from the following projects: Guidance, vLLM, LightLLM, FlashInfer, Outlines, and LMQL.

GitHub Stars & Activity

38,317Stars
10,042Forks
6,309Open issues
PythonLanguage

GitHub Popularity

GitHub stars38,317
Forks10,042
Open issues6,309
Primary languagePython
LicenseApache-2.0
Stars gained today763
Created2024-01-08
Last pushed2026-10-31

Trending History

Weekly boardrank #66 · ▲ 763 stars

Related GitHub Projects

1

anthropics / skills

Python★ 185,654⑂ 21,919▲ 295 stars
→
2

MemPalace / mempalace

Python★ 60,175⑂ 7,625▲ 258 stars
→
3

vectorize-io / hindsight

Python★ 56,792⑂ 6,826▲ 411 stars
→
4

HKUDS / DeepTutor

Python★ 52,216⑂ 6,396▲ 2,282 stars
→
5

langchain-ai / deepagents

Python★ 30,825⑂ 4,333▲ 46 stars
→
6

ethanplusai / harvey

Python★ 3,563⑂ 731▲ 244 stars
→
7

openclaw / openclaw

TypeScript★ 393,517⑂ 82,710▲ 148 stars
→
8

obra / superpowers

Shell★ 309,680⑂ 27,550▲ 1,259 stars
→

More Trending Repositories