Yhn HealthFitness first seen 7 d ago, last 11 h ago, peak #8
Qwen3.8-Flash-Next hits 184 tok/s on one RTX 6000 Pro
Original: Running Qwen3.8-Flash-Next NVFP4 at 184 tok/s on a single RTX 6000 Pro
Engineers are reporting a benchmark in which the Qwen3.8-Flash-Next model, quantized in NVFP4 format, generates 184 tokens per second on a single Nvidia RTX 6000 Pro workstation GPU. The result suggests high-end consumer and workstation hardware can now serve mid-size quantized language models at speeds suitable for interactive use without a data center.
Why now: Practical local inference benchmarks combining a new model with 4-bit NVFP4 quantization draw strong interest from developers running AI on their own hardware
Evidence
- Running Qwen3.8-Flash-Next NVFP4 at 184 tok/s on a single RTX 6000 Pro · matteohorvath · 29
API: https://socialmediatrends-api.osmike.com/v1/trends/3837317