Yhn first seen 1 d ago, last 16 h ago, peak #13
Janas runs LLMs larger than RAM by streaming experts from SSD
Original: Janas: Run LLMs larger than RAM by streaming experts from SSD
A new open-source project called Janas lets people run large language models that exceed their machine's RAM by streaming model experts from an SSD as needed. The approach targets mixture-of-experts architectures, where only a subset of parameters is active per token, making disk-based loading practical. Developers are discussing performance trade-offs and whether consumer SSDs are fast enough to make local inference of huge models viable on modest hardware.
Why now: People are excited about running very large models locally on consumer hardware without expensive RAM upgrades
Evidence
- Janas: Run LLMs larger than RAM by streaming experts from SSD · khalidkhair · 205
API: https://socialmediatrends-api.osmike.com/v1/trends/2042096