NVIDIA, AMD, Google, and Meta Unite to Standardize Open-Source AI Inference
AI inference has become one of the most competitive and fragmented spaces in computing. Every major hardware vendor ships a software stack optimized for their own silicon, every major model lab has preferred frameworks, and the gap between "works on NVIDIA" and "works everywhere else" has been a persistent tax on the ecosystem. On November 12, a broad coalition of companies launched the SGLang Consortium to try to close that gap.
LMSYS Org announced the consortium at the opening keynote of the first SGLang Summit in San Francisco, with over 2,000 attendees. The founding members read like a who's-who of AI infrastructure: NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal, RadixArk, and Thinking Machines Lab. Nine companies, spread across chip makers, cloud providers, and inference platforms, committing to a common open-source inference framework.
What SGLang is
SGLang — Structured Generation Language — is an open-source inference runtime originally developed by LMSYS, the organization behind the Chatbot Arena benchmarks and Vicuna. It specializes in efficient structured generation: getting language models to output structured data (JSON schemas, function calls, constrained text) without sacrificing throughput. For production inference workloads where you need reliable structured output at scale, it has become a serious competitor to vLLM and TensorRT-LLM.
As of this month, SGLang powers NVIDIA NIM Certified microservices — NVIDIA's packaged model deployment containers used by enterprise customers. That adoption by NVIDIA's own product line is part of what makes the consortium announcement more than just a press release.
What the consortium actually does
The governance model is designed to prevent the founding members from just taking over the project. Per CASRAI's coverage, LMSYS maintainers retain final authority over technical decisions and code acceptance. Consortium members get roadmap input and can contribute engineering resources, but they don't get veto power over the project direction. The community stays open to contributions from outside the founding group.
The technical priorities the consortium announced focus on: frontier model support (keeping pace as models scale), reasoning and multimodal workloads, distributed inference across multiple nodes, and hardware optimization across different chip architectures. That last one is the key driver — AMD's interest in particular is about not wanting CUDA to be the only inference path that actually works well in production.
Why this matters
The inference layer is where the economics of running AI models actually live. Training happens at a handful of organizations with massive GPU clusters. Inference happens at every company deploying a model — which means every optimization in the inference layer directly affects the cost per query that everyone pays. A fragmented ecosystem where different hardware requires different codebases raises those costs and slows down adoption of new hardware.
The SGLang Consortium is an attempt to coordinate around a shared open-source layer that hardware vendors can optimize for, rather than each one building proprietary stacks and hoping developers target theirs. It's similar in spirit to what the Linux Foundation has done in other infrastructure domains — not eliminating competition between vendors, but moving the competition up the stack.
Whether nine companies with competing commercial interests can sustain coherent collaboration on a shared codebase is the real question. The SGLang Summit recap from LMSYS is optimistic, but it's also written by the people organizing the consortium. The proof will be in what actually ships over the next 12 months.