local AI models

A project called Strata claims to run Qwen's 125B-parameter Flash Next model on a single consumer RTX 4090 GPU at roughly 100 tokens per second. The claim drew heavy attention on Hacker News, where it ranked near the top of the site. If verified, it would represent a significant step for running large language models on everyday hardware rather than data-centre equipment.

Mac users are discussing how to disable Apple Intelligence in macOS 27 and reclaim the disk space its models and caches occupy. A community script shared online automates the removal, and commenters are weighing whether the storage savings are worth losing the AI features, with some also raising privacy concerns about on-device models.

A developer has launched Jevstiller, a tool that distills Jev into a local AI model, claiming a built-in disagreement bound that limits how far the distilled model can diverge from its source. The project was shared on Hacker News under the Show HN format, drawing moderate attention from the community. Details on the guarantee behind the disagreement bound are published on the project's site.

A developer has shown how to use an iPhone as a second GPU alongside a MacBook, reporting that prefill times for running the Qwen 3.8 27B model are 29 to 44 percent faster. The experiment exploits Apple's unified connectivity to share compute between devices for local AI inference. Commenters are debating how the setup works, its practicality, and what it means for running large models on consumer hardware.