Credo Targets AI Inference Bottlenecks with Open Chiplet Interconnect Standard — Credo Technology Group Holding Ltd

Event summary

Credo plans to standardize its OmniConnect Scale-In Interconnect Solution within the Open Compute Project (OCP) to address AI inference memory wall challenges.
The company established the OCP Open Chiplet Economy (OCE) Lightweight Serial Interconnect (LSI) Workstream and will contribute its OmniConnect lightweight AXI framer specification.
The initiative aims to reduce reliance on High-Bandwidth Memory (HBM) by enabling composable AI architectures with up to 25x greater memory density and 5% higher bandwidth than HBM4.
Credo's solution targets the gap between processor performance and memory access speeds, a critical bottleneck in modern AI inference environments.

The big picture

Credo's move addresses a critical bottleneck in AI infrastructure where memory bandwidth and capacity often limit system performance more than compute power. This initiative aligns with the broader industry shift toward composable architectures that optimize resource utilization for evolving AI workloads. The standardization effort could potentially reshape the economics of AI inference systems by reducing dependence on expensive HBM solutions.

What we're watching

Adoption Pace: The pace at which the OCP community adopts Credo's interconnect standard will determine its market impact.
Competitive Response: Whether competitors will challenge or collaborate on this open standard approach to AI inference architectures.
Technical Validation: How effectively Credo's solution can demonstrate superior performance compared to existing HBM solutions in real-world AI deployments.