⬢github Python · 0 ★ · pushed 13 h ago
IterateAI/lifeboat-releases
Lifeboat — downloads for macOS, Windows and Linux, plus Docker and Kubernetes install instructions. Run language models on your own hardware behind an OpenAI-compatible API.
Lifeboat lets you run open-weight language models on your own hardware—laptops, GPU servers, edge devices, or clusters—behind an OpenAI- and Anthropic-compatible API, with a built-in load balancer and web console. It supports pip, Docker, Kubernetes, and desktop installs, targets edge hardware like Raspberry Pi and Jetson, and claims significant throughput wins over llama.cpp and ollama. This repo is downloads and install instructions only; the source is not public.
Why now: It was featured in a Show HN post claiming 2–6x more concurrent agent sessions per GPU without quantization, plus published benchmarks showing ~2x throughput vs llama.cpp and ollama on the same hardware.
Who it is for: Developers and teams who want to self-host LLM inference on their own machines or edge devices with an OpenAI-compatible API.
Stars over our 43 snapshots: 0 to 0, since 8 h ago.
Where people talked about it
API: https://socialmediatrends-api.osmike.com/v1/repos/IterateAI/lifeboat-releases