Gcore Taps NVIDIA Dynamo to Slash AI Inference Costs and Complexity

LUXEMBOURG – February 25, 2026 – Global infrastructure provider Gcore has announced the integration of NVIDIA Dynamo into its AI cloud platform, a move poised to dramatically lower the cost and complexity of deploying large-scale generative AI. The new offering provides NVIDIA's advanced open-source inference framework as a fully managed, one-click service, promising performance gains of up to 6x higher throughput and 2x lower latency.

This integration directly targets one of the most significant hurdles in the AI revolution: the immense operational and financial burden of running generative AI models in production. By simplifying access to sophisticated optimization technology, Gcore aims to democratize high-performance AI, making it more accessible for businesses of all sizes across public, private, hybrid, and on-premises environments.

The High Cost of AI Intelligence

As enterprises race to adopt generative AI, many encounter a steep reality check when moving from pilot projects to full-scale production. The large language models (LLMs) that power these services require vast computational resources, primarily expensive and often scarce Graphics Processing Units (GPUs). A critical challenge is that these GPUs are frequently underutilized, leading to wasted capacity and inflated operational costs.

The problem lies in the nature of AI inference—the process of a trained model making predictions. It involves complex, dynamic workloads with fluctuating demands. "Modern inference isn't just 'run a model'—it's batching, routing, dynamic workloads, longer contexts, and tight SLOs," explained Seva Vayner, Product Director of Edge Cloud and AI at Gcore, in the company's announcement. "In that reality, small scheduling and utilization losses become big performance and cost penalties."

These penalties manifest as memory bottlenecks, inefficient data transfer between nodes, and static resource allocation that can't adapt to real-time demand. For businesses, this translates into higher latency for end-users, lower return on investment for their AI infrastructure, and a significant operational headache requiring specialized MLOps teams to manage the intricate web of GPU scheduling, routing, and memory management.

NVIDIA Dynamo: The Engine for AI Efficiency

At the heart of Gcore's new offering is NVIDIA Dynamo, an open-source framework designed specifically to act as an "operating system for an AI factory." Introduced to tackle the core inefficiencies of distributed inference, Dynamo employs several innovative techniques to maximize GPU performance and throughput.

One of its key architectural features is the disaggregation of the prefill and decode stages of an LLM request. The prefill stage, which processes the initial user prompt, is computationally different from the decode stage, which generates the response token by token. By separating these tasks and assigning them to different, specialized GPU pools, Dynamo prevents one stage from bottlenecking the other, a common source of underutilization, especially with long prompts.

Furthermore, Dynamo incorporates a "KV cache-aware" smart router. The KV cache stores intermediate calculations to speed up token generation. Dynamo's router intelligently directs incoming requests to GPUs that already have relevant data in their cache, drastically reducing redundant computations and minimizing latency. This is paired with an advanced KV Cache Manager that can offload less-used data to cheaper storage tiers like CPU memory or SSDs, freeing up valuable high-bandwidth GPU memory and allowing models to scale beyond the limits of a single GPU. Combined with NIXL, a low-latency communication library for GPU-to-GPU data transfer, these features ensure that more requests are processed faster on the same hardware, directly lowering the cost per token.

Simplifying Complexity with a Single Click

While the technology behind NVIDIA Dynamo is powerful, its implementation can be complex. Gcore's strategic value proposition is to abstract this complexity away entirely. The company is delivering Dynamo as a fully managed solution that can be activated with a single click within the Gcore Customer Portal.

This means customers gain the benefits of advanced GPU optimization—including sophisticated routing, KV cache logic, and dynamic scheduling—without needing to operate or even understand the underlying mechanics. Gcore's platform handles the orchestration, pre-optimizing the framework for popular inference models and ensuring seamless operation.

"By integrating Dynamo as a managed service in Gcore, we bring advanced GPU optimization directly into the runtime path so customers see higher effective throughput and steadier tail latency, without operating the complexity themselves," Vayner stated. This approach effectively lowers the barrier to entry, enabling a wider range of companies, including those without dedicated AI infrastructure teams, to deploy state-of-the-art generative AI services efficiently and cost-effectively. The service is available on Gcore's Everywhere Inference and Everywhere AI platforms.

A Strategic Play in the AI Infrastructure Market

Gcore's integration of NVIDIA Dynamo is more than a technical update; it's a strategic move that positions the company as a nimble and potent competitor in the crowded AI infrastructure landscape. While hyperscale cloud providers like AWS, Azure, and Google Cloud offer a vast suite of AI services, Gcore is carving out a niche focused on performance, efficiency, and deployment flexibility.

A key differentiator is Gcore's sovereign, globally distributed infrastructure. For enterprises in regulated industries or those with strict data residency and compliance requirements, the ability to deploy powerful AI on a sovereign cloud is a significant advantage. By extending Dynamo support across private cloud, hybrid, and on-premises environments, Gcore provides a unified solution that meets organizations where their data resides, avoiding the lock-in of a single public cloud.

This "everywhere" approach, combined with the extreme performance and cost efficiencies unlocked by a managed Dynamo service, creates a compelling value proposition. It allows businesses to optimize their AI workloads not just for speed, but for cost, compliance, and latency, regardless of the deployment environment. As the demand for practical, production-ready AI continues to surge, solutions that deliver both cutting-edge performance and operational simplicity are set to define the next wave of adoption. Gcore will be showcasing live demonstrations of the new integration at the upcoming MWC in Barcelona and GTC in San Jose.