AI Coding Safety: Intelligence Beats Model Size, Study Finds

FULTON, MD – March 24, 2026 – A new study is challenging the prevailing "bigger is better" ethos in artificial intelligence, revealing that when it comes to securing software, the size of an AI model matters far less than the quality of its data. Research unveiled today by AI-driven DevSecOps leader Sonatype shows that even the most advanced large language models (LLMs) from tech giants like Google, OpenAI, and Anthropic can leave applications dangerously exposed unless they are grounded in real-time software intelligence.

The findings suggest that the race for larger, more powerful AI could be overlooking a fundamental requirement for enterprise safety: context. Without a live connection to the ever-changing landscape of software dependencies, vulnerabilities, and project policies, AI coding assistants are often just making educated guesses—with potentially disastrous results.

The 'Bigger is Better' Myth Debunked

In a comprehensive study evaluating nearly 37,000 open-source upgrade recommendations, Sonatype found a startling gap between the performance of ungrounded AI models and those augmented with real-time data. The research discovered that even frontier models, lacking this live intelligence, fabricated roughly one in every 16 dependency recommendations. This phenomenon, known as "hallucination," can send developers on wild goose chases, attempting to integrate software packages that don't exist and leading to broken builds and wasted hours.

Furthermore, as the models became larger and more sophisticated, they developed a new, more subtle flaw: excessive caution. Instead of suggesting a flawed upgrade, newer models increasingly defaulted to recommending "no change" at all. While this behavior reduced outright hallucinations, it created a false sense of security. The study found that this inaction left behind a significant risk portfolio, with applications still carrying an average of 800 to 900 Critical and High severity vulnerabilities that could have been fixed.

The quantitative results were stark. Across popular software ecosystems like Maven Central, npm, and PyPI, Sonatype’s grounded "Hybrid" approach delivered a mean security score improvement between 269% and 309%. In stark contrast, the best-performing standalone LLMs only managed improvements between 24% and 68%. The research effectively demonstrated that a small, grounded model could achieve significantly lower risk at up to 71 times lower cost than its much larger, ungrounded counterparts.

A Data Problem, Not Just a Reasoning Problem

The core issue, according to Sonatype's leadership, is a misunderstanding of the problem AI is being asked to solve. While LLMs excel at pattern recognition and reasoning, managing software dependencies is a dynamic data challenge.

"Larger models may be improving at reasoning, but dependency management is not a reasoning problem alone — it is a data problem," said Brian Fox, Co-founder and CTO at Sonatype, in the company's press release. "If a model does not know your actual environment, current vulnerability data, and the policies you operate under, it is just making educated guesses. Grounding AI in that reality is what makes its recommendations useful, credible, and safe for enterprise use."

This "grounding" involves feeding the AI a continuous stream of verified information: what package versions are currently available, which ones have known vulnerabilities, which upgrades are compatible with a specific project, and what corporate governance policies are in effect. Without this context, an AI model is operating in a vacuum, relying on a static, and often outdated, snapshot of the internet it was trained on. This is why it might recommend a package that was secure a year ago but is now known to be compromised.

The Developer's Double-Edged Sword

This disconnect between AI's potential and its practical reliability is acutely felt by software developers on the front lines. With 97% of development and security professionals already using generative AI in their workflows, the push for AI-driven productivity is immense. Many report saving over six hours a week on tasks like code generation and testing.

However, this efficiency comes with a steep price when the tools are unreliable. Developers are increasingly frustrated by AI assistants that suggest non-existent packages or introduce insecure code, creating more rework and security debt. This erodes trust in automation and forces teams to choose between speed and safety. According to one industry analyst leading DevSecOps research, this is a critical juncture. Developers accept a high percentage of AI-generated code without changes, and if that code is based on hallucinations, it can dramatically expand a company's attack surface.

Solutions are emerging that aim to bridge this gap, not by replacing popular tools like GitHub Copilot, but by augmenting them. By intercepting package recommendations in real time and checking them against a live intelligence feed, these systems can steer the AI—and the developer—toward safer, more reliable choices without disrupting the existing workflow.

Securing the Digital Supply Chain

The implications of Sonatype's research extend far beyond individual developer productivity. They strike at the heart of the modern software supply chain, which is overwhelmingly built on open-source components. When ungrounded AI tools pull in dependencies without proper vetting, they create a "shadow AI" ecosystem of untracked and ungoverned code, making security audits nearly impossible.

This creates a fertile ground for new attack vectors, such as "slopsquatting," where malicious actors publish malware under the names of popular hallucinated packages, waiting for an unsuspecting AI to recommend it. For enterprises and government agencies, where software supply chain integrity is a matter of corporate and national security, such risks are untenable.

The study argues for a paradigm shift in how we evaluate and deploy AI in critical functions. The true measure of an AI's value is not its parameter count, but its connection to a verifiable, real-world source of truth. By grounding AI in the dynamic reality of the software ecosystem, organizations can harness its power to automate remediation and guide developers, transforming it from a potential liability into a powerful defensive asset. This approach ensures that as developers work faster with AI's help, they are also working smarter and, most importantly, safer.