search
speculative decoding
Trends
- 1
God of War creator David Jaffe is drawing attention again as gamers speculate about the franchise's future. Discussion intensified after a clue in the series was decoded as confirming the arrival of a new mythology, with fans debating what the next setting could be and revisiting Jaffe's original vision for the series that launched in 2005.
- 2
A new article by Jacob Barrow explains why speculative decoding speeds up large language model inference, walking through how a smaller draft model proposes tokens that a larger model verifies in parallel. The piece is drawing attention among machine learning engineers and researchers discussing practical ways to cut latency and inference costs for LLM applications.
Repos
- deepseek-ai/DeepInfer DeepInfer: High-performance, large-scale inference framework for DeepSeek models
- eagle-hq/EAGLE-4 Official Inference Code of EAGLE-4
- vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs
- prabanta-dev/janas 37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s.