⬢github Rust · 691 ★ +12 since we first saw it · pushed 1 h ago · Apache-2.0
NVIDIA/flare-local-inference
Flare is NVIDIA's open-source local inference server for SM120/SM121 GPUs (like DGX Spark), optimized for long conversations, coding agents, and repeated tool use. Models are prepared with PyTorch/FlashInfer into a compiled runtime package, then served by a small Rust runtime (Helios) without Python, with explicit memory planning and conversation state management.
Why now: A newly released research prototype from NVIDIA that gained traction quickly (~679 stars in a week), likely due to interest in efficient local serving on DGX Spark-class hardware and its unusual Python-build/Rust-serve architecture.
Who it is for: Developers running coding agents and long-context workloads locally on NVIDIA SM120/SM121 workstations who want a specialized alternative to vLLM.
Stars over our 132 snapshots: 679 to 691, since 1 d ago.
Where people talked about it
- ⬢github new repos, most starred 1 d ago
API: https://socialmediatrends-api.osmike.com/v1/repos/NVIDIA/flare-local-inference