TCP

Engineers are discussing an approach that routes large language model requests across multiple inference providers using TCP-style congestion control. The method treats each provider like a network link, adapting traffic in response to latency and failures so no single provider becomes a bottleneck. Commenters are weighing the tradeoffs of adaptive routing for reliability and cost in production AI systems.