⬢github Python · 16 ★ +14 since we first saw it · pushed 1 d ago · Apache-2.0
spinfinc/spinf-serve
Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0
spinf-serve is a self-hosted Python inference server for bulk classification and 'decision model' tasks. It reads a piece of content once and scores many questions against it in a single GPU pass, returning calibrated probabilities over fixed options instead of generated text. It exposes an OpenAI-compatible API, runs on a single 24 GB NVIDIA GPU with fp8 weights, and is Apache 2.0 licensed.
Why now: It was just shown on Hacker News as a 'Show HN' post, with the authors publishing the engine behind their hosted spinf API after people asked to self-host it.
Who it is for: Teams doing high-throughput text classification, moderation, or scoring workloads — e.g. labeling millions of documents — on their own NVIDIA hardware.
batch-inferenceclassificationcontent-moderationdecision-modelextractionfp8gemmainference-enginellmopenai-compatible
Stars over our 24 snapshots: 2 to 16, since 10 h ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/spinfinc/spinf-serve