H2O.ai's Open-Weights Model Tops JevBench Leaderboard

Event summary

H2O-Lightning-4B ranked first on the JevBench Open-Weights Leaderboard with a score of 72.5 as of October 7, 2026.
The model outperformed the closed API reference system Jev, scoring higher at a lower cost of $0.021 per 1,000 decisions.
H2O-Lightning-4B is a fine-tune of Qwen3.5-4B and is released under the Apache 2.0 license.
The model supports three question types: choice, yes/no, and score, with a median latency of 29 milliseconds per decision.
H2O.ai positions the model as a cost-effective alternative to larger, closed models, emphasizing calibration and speed.

The big picture

H2O.ai's achievement highlights the growing importance of open-weights models in regulated industries, where cost, speed, and calibration are critical. The shift away from closed, proprietary models towards open, customizable solutions reflects broader trends in AI governance and enterprise adoption. With Microsoft entering the space, the competition is set to intensify, potentially driving further innovation in decision-model performance and cost efficiency.

What we're watching

Competitive Dynamics: Whether H2O.ai can sustain its lead as Microsoft and other large players enter the decision model space.
Adoption Pace: The pace at which regulated industries adopt open-weights models like H2O-Lightning-4B for critical decision-making.
Cost Efficiency: How the cost advantages of H2O-Lightning-4B will impact the broader AI model market.