Mmastodon TechnologyAI first seen 6 h ago, last 6 h ago, peak #1

180B-parameter LLM runs locally on a laptop without a GPU

Original: GPU 없이 소비자용 노트북에서 180억 파라미터 LLM을 구동하는 POCKET-Darwin-180B. 4비트 GGUF 양자화로 360GB→111GB 압축, 약 $1,400 하드웨어로 로컬 추론 가능. # ai #

A project called POCKET-Darwin-180B is drawing attention for running a 180-billion-parameter language model on consumer hardware with no discrete GPU. Using 4-bit GGUF quantization, the model is compressed from roughly 360GB down to 111GB, enabling local inference on hardware costing about $1,400. Commenters in AI and open-source circles are highlighting it as a sign that frontier-scale models may soon run off the cloud.

Why now: Interest in running very large language models locally without expensive GPU hardware is growing, and this claims a striking compression result.

POCKET-Darwin-180BGGUFHackaday

Open on mastodon →

Rank over time, top of the chart is #1. 2 snapshots from 6 h ago to 6 h ago.

Evidence

• GPU 없이 소비자용 노트북에서 180억 파라미터 LLM을 구동하는 POCKET-Darwin-180B. 4비트 GGUF 양자화로 360GB→111GB 압축, 약 $1,400 하드웨어로 로컬 추론 가능. # ai # machinelearning # opensource # llm # software # coding # development # engineering # inclusive # community Running a 180B-Parameter LLM on a Laptop Without a… · hackaday@www.urbanmind.net · 3

API: https://socialmediatrends-api.osmike.com/v1/trends/792569