⬢github Python · 15.9K ★ +856 since we first saw it · pushed 38 min ago · Apache-2.0
FlashML-org/FreeToken
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
FreeToken is an edge-native inference engine for serving large open-weight Mixture-of-Experts models on consumer hardware like gaming PCs and laptops. It combines GPUs, CPUs, and host memory into one elastic serving platform, with expert caching, semantic-aware KV reuse, runtime VRAM reallocation, quantization support, and OpenAI/Anthropic-compatible APIs plus a desktop app and CLI.
Why now: Buzz around running frontier-scale MoE models (e.g., 290B+ DeepSeek-V4-Flash, Qwen, GLM) fast on consumer RTX GPUs, backed by a recent paper, a Windows/Linux desktop app, and integration with popular coding agents like Codex and Claude Code.
Who it is for: Developers and enthusiasts who want to run large open-weight MoE models locally on gaming PCs or workstations for chat, agents, and coding workflows.
deepseek-v4edgeglminferenceminimaxmoeqwentransformer
Stars over our 129 snapshots: 15.1K to 15.9K, since 1 d ago.
Where people talked about it
- ⬢github FlashML-org/FreeToken 8 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/FlashML-org/FreeToken