Why AI hardware verification needs formal sign-off
Jagged AI makes formal verification more important, not less
AI is getting harder to judge.
One day, it solves a problem that looks almost impossible.
The next day, it fails at something that feels surprisingly simple.
This is often described as jagged intelligence.
Andrej Karpathy captured this well in his 2025 LLM review. He describes current LLMs as systems that can show “jagged performance characteristics”, at times acting like a “genius polymath” and at other times like a “confused” beginner.
That is an important idea for everyone working in chip design and verification, because hardware design is not a domain where “mostly correct” is good enough.
AI will help create more hardware
AI-assisted development will likely become part of the chip design workflow.
It may help engineers explore architectures, draft RTL, generate properties, documentation, test ideas, or debug hypotheses. This can be useful.
But there is a trap:
Just because AI can generate something that looks convincing does not mean the result is correct.
In software, an AI-generated bug can often be found, patched, and redeployed. In hardware, mistakes are much more expensive.
- A wrong assumption in RTL
- A missing corner case
- A protocol violation under rare backpressure
- An ordering issue that only appears after many cycles.
- A bug that survives until tape-out.
These are not writing mistakes. They are product risks.
AI cannot be the final judge of AI-generated hardware
The problem is that AI is uneven and it will most likely remain so. This makes these systems difficult to trust on their own in domains where mistakes are expensive.
If an AI system generates RTL, another AI system reviewing it is not enough. It may catch issues, explain possible risks, or suggest improvements. But it is still probabilistic. It can miss things, overstate confidence, or produce a plausible explanation for a wrong result.
That is exactly why deterministic and systematic verification becomes more important. The more AI enters hardware design, the more we need methods that do not rely on confidence by wording, but on confidence by evidence. This is where systemized formal verification becomes essential.
Systemized formal verification is the trust layer for AI-assisted hardware design
Formal verification can deliver confidence beyond doubt, but not simply because a formal property was written and a formal tool was run. It creates confidence when it is applied in a structured, repeatable, and reviewable way.
That distinction matters. AI can produce a design candidate. But someone still needs to define what must be true, which scenarios matter, which assumptions are valid, and what evidence is strong enough for sign-off.
This is where systemized formal verification becomes important. We do not see formal as a one-off expert activity. We see it as a systemized sign-off process:
Plan → Prep → Execute → Sign-off
First, the verification goal is made explicit. Then the environment, constraints, and properties are prepared with care. Then the design is challenged across relevant behaviors, protocols, corner cases, and risk areas. Finally, the result is turned into reviewable evidence that engineering teams can use to make decisions.
That is what AI-generated hardware will need. Not just another layer of AI review. Not just more simulation. Not just a set of properties written in isolation. It needs a deterministic, structured process that turns uncertainty into sign-off evidence.
In a world of jagged AI, trust will not come from how convincing the generated RTL looks. Trust will come from systemized, repeatable, and deterministic verification.
Don’t get fooled by the headline version of AI
The headline says: AI can solve complex tasks.
The engineering reality says: AI can still make simple mistakes.
Both can be true at the same time. That is what makes the current generation of AI powerful and dangerous in safety-critical engineering workflows.
For chip teams, the question should not be: “Can AI generate RTL?”
The better question is: “How do we verify what AI generates?”
Because the real bottleneck is not generation. The real bottleneck is confidence.
The era of AI-generated hardware needs AI hardware verification
Why AI hardware verification? As AI becomes more present in semiconductor development, the productivity of design teams may increase significantly, and the verification bottleneck becomes more apparent. AI may help teams generate more RTL, moving from a rough new idea to an actual chip design in a matter of weeks rather than months.
But every new design also creates a new question: Can we trust it?
That question cannot be answered by an informal review process. And it cannot scale if formal verification remains dependent on a few hero experts working through each problem manually.
The pace of AI-generated output will be too high. If design creation becomes faster, verification must become more systemized. Teams will need formal sign-off processes that are repeatable, explainable, and built for scale: clear verification goals, controlled assumptions, reviewable properties, systematic execution, and decision-ready evidence.
That is the real role of formal verification in the AI era. Not as a blocker to AI adoption. But as the system that makes faster development safe enough to trust. AI may help teams move faster. Systemized formal sign-off helps them know when moving faster is safe.