Zilliz Aims to Unify the AI Data Pipeline with Vector Lakebase Platform

REDWOOD CITY, CA – June 22, 2026 – Zilliz, the company behind the widely adopted open-source vector database Milvus, has unveiled a significant evolution of its cloud platform that could reshape how enterprises manage the data powering their artificial intelligence systems. The public preview of Zilliz Vector Lakebase marks a strategic push to solve one of the most persistent and costly challenges in AI development: data fragmentation.

The new platform extends the company's core vector database capabilities into a unified data foundation, promising to let the same set of vectors serve production queries, anchor interactive discovery sessions, and fuel massive data-training pipelines—all without creating costly data copies or navigating complex migrations. For businesses struggling to connect the dots between their AI models in production and the data used to train them, this move signals a potential end to the era of siloed AI infrastructure.

Breaking Down the AI Data Silos

Modern AI systems, particularly in the generative AI space, operate in a continuous loop: they serve information, learn from user feedback and new data, mine and prepare better data, and then serve again. In theory, this cycle drives constant improvement. In practice, each stage often requires a separate, specialized system. A high-throughput, low-latency vector database might handle real-time search for a RAG application, while a separate analytics platform like Spark or Snowflake is used for large-scale data processing and model training. An entirely different set of tools might be used for interactive data exploration.

This separation creates immense friction. Moving billions of vectors and their associated metadata between these disparate systems is slow and expensive, often taking days. The complexity is so prohibitive that many organizations abandon the feedback loop altogether, leaving their AI models static and their valuable data retrievable but never fully leveraged for improvement.

Vector Lakebase tackles this problem head-on with what Zilliz calls a “zero-copy semantic data plane” built on shared, lake-native storage. The architecture allows different workloads to run against a single logical copy of the data. “Production vector search is and will remain at the heart of what Zilliz does,” said Charles Xie, Founder and CEO of Zilliz, in the announcement. “Vector Lakebase is what we believe comes next: one data foundation where the same vectors can serve a production query, anchor a discovery session, and power a multi-petabyte training-data pipeline — without copies, migration, or a parallel stack.”

The New Economics of AI Infrastructure

A unified data plane is not just an architectural novelty; it has profound economic implications. A key feature of the new platform is 'On-Demand Search,' a pay-as-you-go compute model designed for workloads that are often idle, such as data discovery, quality analysis, or semantic deduplication. According to Zilliz, these workloads can sit idle more than 97% of the time, yet traditional serverless models often incur significant costs due to “always-on” infrastructure and serverless markups.

Internal benchmarks from the company paint a compelling financial picture. In one test on a billion vectors, an analytics workload that would cost thousands on a comparable serverless path allegedly totaled just $318 using On-Demand Search—a cost reduction of roughly 94%. By billing only for active compute and object storage, this model could democratize advanced AI data processing for businesses that couldn’t justify the expense of dedicated, always-on clusters for intermittent tasks.

This cost-conscious flexibility extends to production workloads through a 'Tiered Real-Time Serving' model. Enterprises can choose from three tiers to balance cost and performance:
* Performance-Optimized: For mission-critical applications demanding over 1,000 queries per second (QPS) at single-digit-millisecond latency.
* Capacity-Optimized: A balanced tier for high-scale applications needing 100-500 QPS with sub-100ms latency.
* Tiered-Storage: A cost-effective option for massive datasets, delivering 10-50 QPS by leveraging a mix of memory, NVMe, and object storage.

This tiered approach acknowledges that not all AI applications are the same, allowing businesses to align infrastructure spending directly with specific performance requirements and ROI calculations.

Under the Hood: Vortex and External Lake Search

The technical foundation of Vector Lakebase rests on two key innovations: a new storage format and the ability to operate directly on external data lakes. The platform’s unified storage is built on Vortex, an open columnar format designed by Zilliz for faster and cheaper random reads than established formats like Parquet. For vector search, which relies on accessing small, specific chunks of data, this is critical. Zilliz claims Vortex can cut data read amplification by over 90%, significantly reducing I/O costs and speeding up queries that access data in object storage.

Perhaps most strategically important for enterprise adoption is the 'External Data Lake Search' capability. This “zero-copy” mode allows Vector Lakebase to build indexes and perform full-spectrum search directly on data already residing in a company's existing data lake, whether in Lance, Iceberg, or Parquet format. Instead of forcing a costly and complex data migration into a separate vector database, this feature brings Zilliz's indexing and search capabilities to the data. This not only simplifies integration but also aligns with modern data governance principles that favor keeping data in a centralized, managed environment.

Reshaping the Competitive Landscape

With Vector Lakebase, Zilliz is carving out a new position in the crowded AI infrastructure market. The platform moves beyond a direct comparison with pure-play managed vector databases like Pinecone or Weaviate. While those services excel at low-latency vector retrieval, Zilliz is now offering a solution for the entire AI data lifecycle, integrating serving, analytics, and discovery into one cohesive offering.

This move also positions the company as a powerful, specialized layer for existing data lakehouse platforms like Databricks and Snowflake. Rather than competing to be the single source of all enterprise data, Zilliz is providing the high-performance engine required for the semantic search and unstructured data workloads that are becoming central to modern AI but are not the native focus of traditional data warehouses.

The launch reflects a broader industry maturation. The initial frenzy around vector databases as standalone retrieval engines is evolving into a more sophisticated understanding of AI as a continuous data-driven process. By building a platform to manage this entire loop, Zilliz is betting that the future of AI infrastructure lies not in a single tool, but in a unified foundation that makes the cycle of serving, learning, and improving both technically feasible and economically viable.

Zilliz Vector Lakebase is now available in public preview on Zilliz Cloud, with deployments across AWS, Google Cloud, and Microsoft Azure, challenging the industry to rethink the foundational layer of the modern AI stack.