Mmastodon TechnologySoftware first seen 23 h ago, last 22 h ago, peak #4
180B-Parameter AI Model Claims to Run on 8GB Laptop GPU
Original: VIDRAFT의 POCKET-Darwin-180B-GGUF로 180B 파라미터 MoE 모델을 8GB VRAM 노트북에서 구동. 4비트 양자화와 llama.cpp로 250배 하드웨어 요구사항 절감, 로컬 AI 배포 완
VIDRAFT has released POCKET-Darwin-180B-GGUF, a quantized build that reportedly runs a 180-billion-parameter mixture-of-experts model on a laptop with just 8GB of VRAM. Using 4-bit quantization and llama.cpp, the project claims hardware requirements are cut by roughly 250 times, making local deployment of very large AI models feasible on consumer machines.
Why now: Running a 180B-parameter model on modest consumer hardware is a striking claim that excites the local AI and open-source community.
VIDRAFTPOCKET-Darwin-180B-GGUFllama.cpp
Rank over time, top of the chart is #1. 2 snapshots from 23 h ago to 22 h ago.
Evidence
- VIDRAFT의 POCKET-Darwin-180B-GGUF로 180B 파라미터 MoE 모델을 8GB VRAM 노트북에서 구동. 4비트 양자화와 llama.cpp로 250배 하드웨어 요구사항 절감, 로컬 AI 배포 완벽 가이드. # ai # machinelearning # opensource # llm # software # coding # development # engineering # inclusive # community Running a 180B-Parameter MoE Model on… · hackaday@www.urbanmind.net · 3
API: https://socialmediatrends-api.osmike.com/v1/trends/1293580