sgl-project / sglang-omni

项目摘要

SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models, extending the SGLang ecosystem to speech and audio inference. It provides production-ready serving infrastructure with streaming support, distributed inference, and OpenAI-compatible APIs for multimodal workloads.

为什么值得关注

It brings SGLang's battle-tested high-performance serving architecture to the underserved audio and multimodal model space, filling a critical gap in open-source inference infrastructure for TTS, ASR, and unified speech-text models.

优势

• Extends SGLang's proven high-performance inference stack to audio and multimodal models
• Unified serving for TTS, ASR, and multimodal models in a single framework
• OpenAI-compatible API and streaming support for easy integration
• Strong community with 81 contributors and active development

局限性

• Very new project (created January 2026) with limited production track record
• No Docker support yet, 249 open issues indicate ongoing stabilization

使用场景

• Production serving of TTS and ASR models at scale
• Unified multimodal inference combining text and audio
• Local voice AI applications via Voxt on Apple Silicon