Developed with SGLang, Atlas Inference surpasses leading AI companies in throughput and cost, running DeepSeek V3 & R1 faster than DeepSeek themselves.
NEW YORK CITY, NEW YORK / ACCESS Newswire / May 28, 2025 / Atlas Cloud, the all-in-one AI competency center for training and deploying AI models, today announced the launch of Atlas Inference, an AI inference platform that dramatically reduces GPU and server requirements, enabling faster, more cost-effective deployment of large language models (LLMs).
Atlas Cloud logo
Atlas Inference, co-developed with SGLang, an AI inference engine, maximizes GPU efficiency by processing more tokens faster and with less hardware. When comparing DeepSeek’s published performance results, Atlas Inference’s 12-node H100 cluster outperformed DeepSeek’s reference implementation of their DeepSeek-V3 model while using two-thirds of the servers. Atlas’ platform reduces infrastructure requirements and operational costs while addressing hardware costs, which represent up to 80% of AI operational expenses.
“We built Atlas Inference to fundamentally break down the economics of AI deployment,” said Jerry Tang, Atlas CEO. “Our platform’s ability to process 54,500 input tokens and 22,500 output tokens per second per node means businesses can finally make high-volume LLM services profitable instead of merely break-even. I believe this will have a significant ripple effect throughout the industry. Simply put, we’re surpassing industry standards set by hyperscalers by delivering superior throughput with fewer resources.”
Atlas Inference’s performance also exceeds major players like Amazon, NVIDIA and Microsoft, delivering up to 2.1 times greater throughput using 12 nodes compared to competitors’ larger setups. It maintains sub-5-second first-token latency and 100-millisecond inter-token latency with more than 10,000 concurrent sessions, ensuring a scaled, superior experience. The platform’s performance is driven by four key innovations:
Prefill/Decode Disaggregation: Separates compute-intensive operations from memory-bound processes to optimize efficiency
DeepExpert (DeepEP) Parallelism with Load Balancers: Ensures over 90% GPU utilization
Two-Batch OverlapTechnology: Increases throughput by enabling larger batches and utilization of both compute and communication phases simultaneously
DisposableTensor Memory Models: Prevents crashes during long sequences for reliable operation
“This platform represents a significant leap forward for AI inference,” said Yineng Zhang, Core Developer at SGLang. “What we built here may become the new standard for GPU utilization and latency management. We believe this will unlock capabilities previously out of reach for the majority of the industry regarding throughput and efficiency.”
Combined with a lower cost per token, linear scaling behavior, and reduced emissions compared to leading vendors, Atlas Inference provides a cost-efficient and scalable AI deployment.
Atlas Inference works with standard hardware and supports custom models, giving customers complete flexibility. Teams can upload fine-tuned models and keep them isolated on dedicated GPUs, making the platform ideal for organizations requiring brand-specific voice or domain expertise.
The platform is available immediately for enterprise customers and early-stage startups.
About Atlas Cloud
Atlas Cloud is your all-in-one AI competency center, powering leading AI teams with safe, simple, and scalable infrastructure for training and deploying models. Atlas Cloud also offers an on-demand GPU platform that delivers fast, serverless compute. Backed by Dell, HPE, and Supermicro, Atlas delivers near instant access to up to 5,000 GPUs across a global SuperCloud fabric with 99% uptime and baked-in compliance. Learn more at atlascloud.ai.
SOURCE: Atlas Cloud
