⬢github Python · 11.6K ★ +10.1K since we first saw it · pushed 9 h ago · Apache-2.0
deepseek-ai/DeepInfer
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
DeepInfer is DeepSeek's open-source, high-performance inference framework for running its DeepSeek models (currently V4.1-Flash) at large scale. It provides offline inference with optimized kernels, parallelism strategies (DP/CP/EP), speculative decoding via confidence-scheduled verification, async scheduling, batch-invariant and deterministic sampling, and structured outputs, running on NVIDIA SM100 GPUs and Huawei Ascend NPUs.
Why now: It's a fresh, fast-growing open-source release (2k+ stars in days) of the framework that powers DeepSeek's own production serving, notable for supporting both NVIDIA and Ascend hardware.
Who it is for: Engineers deploying or benchmarking large-scale LLM inference, especially those serving DeepSeek models on SM100 GPUs or Ascend NPUs.
Stars over our 45 snapshots: 1.4K to 11.6K, since 11 h ago.
Where people talked about it
- ⬢github new repos, most starred 12 min ago
- Yhn DeepSeek just open-sourced its inference engine 12 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/deepseek-ai/DeepInfer