LLM gateway

Consus, a startup from Y Combinator's Fall 2024 batch, has launched an LLM gateway aimed at companies that handle sensitive data. The tool lets businesses route AI model requests while keeping control over confidential information, addressing privacy and compliance concerns that often block enterprise AI adoption. Hacker News users are discussing the launch and its approach to data security.

Check Point Research published findings on security weaknesses in LiteLLM, a widely used open-source gateway for routing requests to large language models. The report details how flaws in the gateway could be exploited, highlighting the risks organisations take on when they centralise AI traffic through middleware. It underscores growing scrutiny of the infrastructure layer of enterprise AI deployments.

Eric, founder of Consus, a Y Combinator F24 startup, has launched a gateway that routes AI requests to infrastructure compliant with ITAR restrictions and controlled unclassified information (CUI) rules. The company says the platform itself meets ITAR requirements, letting companies in regulated industries use large language models without exposing sensitive data. The launch drew attention on Hacker News, where readers debated the compliance claims.

A new warning has been issued over high-risk security vulnerabilities affecting widely used large language model gateways, the infrastructure that routes and manages traffic to AI models. The alert suggests organizations relying on these tools could be exposed to attack. Details on affected products and fixes remain limited.

Security researchers have issued a warning about high-risk vulnerabilities in a widely used LLM gateway tool, the software layer organizations rely on to route and manage access to large language models. Infosecurity Magazine reported the advisory, which is drawing attention because flaws in such gateways could expose AI deployments across many companies at once.

A roundup of the top five AI gateways for multimodal workloads in 2026 is drawing attention among developers. Multimodal AI workloads that pass images, audio, and video create significant payload-size variances, making gateway selection important for performance and cost. The discussion spans devops, open source, and software engineering communities weighing which platforms best handle mixed media traffic.