⬢github Python · 462 ★ +433 since we first saw it · pushed 11 h ago · Apache-2.0

OmniJev/Flide

🔥🔥 The extremely fast inference engine for decision model

Flide is a Python inference engine for Jev-style decision models. It uses exact state forking to share a model's state across questions, claiming up to 9.5x speedups over vLLM and SGLang, and serves a Jev /v1/systemone API with support for many decision model families on PyPI.

Why now: A recent Show HN post promoting its 9.5x speedup claim and rapid releases adding many decision models, plus an arXiv paper.

Who it is for: Developers serving LLM-based decision models at scale on GPUs.

llminference-enginepytorchcudadecision-makingkv-cache

ai-agentscudadecision-makingdeterministic-inferencegemmainference-enginejevkv-cachelarge-language-modelslinear-attention

Open on GitHub →

Stars over our 303 snapshots: 29 to 462, since 3 d ago.

Where people talked about it

• Yhn Show HN: Flide: Make Jev-like decision models up to 9.5x faster 12 h ago

API: https://socialmediatrends-api.osmike.com/v1/repos/OmniJev/Flide