search
consumer GPUs
Trends
- 1Developers debate hardware for running local LLMs▼Ask HN: What hardware are you using to run local LLMs?
A discussion thread on Hacker News is asking people what hardware they use to run large language models locally. Respondents are comparing consumer GPUs, Apple Silicon Macs, and dedicated workstations, weighing memory requirements, inference speed and cost against cloud API pricing. The conversation reflects growing interest in running capable AI models on personal machines rather than paying for hosted services.
Repos
- physicsdolphin/nv-uefi-unlocker UEFI pre-driver NVIDIA GPU unlocker for Windows Linux Hackintosh and all EFI-bootable system (Rust): SEC2 scrubber ->
- FlashML-org/FreeToken FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- Zagitta/gpu-unlock Your GeForce runs AI at half speed. This fixes it for 20/30/40 series cards. Up to 2× FP16 / BF16 / TF32 tensor throughp
- JustAResearcher/gpu-unlock-ada GPU tensor unlock source for RTX 4090 and experimental RTX 4070 Ti SUPER; firmware generated locally