Verifying On-Chain AI: Optimistic Proofs vs zkML Explained
Verifying AI outputs on-chain forces a choice between slow optimistic challenges and insanely expensive zero-knowledge proofs.
Key takeaways
- →Smart contracts cannot natively run neural networks due to gas limits and non-deterministic floating-point math.
- →Optimistic verification is cheap and fast off-chain, but forces a multi-hour delay while waiting for potential fraud challenges.
- →zkML yields instant on-chain finality, but generating the cryptographic proof costs thousands of times more compute than running the AI itself.
- →Fixed-point quantization is mandatory for both approaches to ensure exact output matching across different hardware platforms.
Smart contracts suck at math. Hand the Ethereum Virtual Machine a 10-million parameter neural network and it'll blow straight through its gas limit before finishing its first matrix multiplication. Running even a modest AI inference model directly inside an EVM contract costs tens of thousands of dollars in gas fees for one single prediction.
Yet crypto needs AI. Prediction markets rely on AI agents to settle complex, subjective outcomes. Automated trading vaults use machine learning models to tweak leverage based on real-time order book sentiment. On-chain casinos run trained AI opponents to scale dynamic difficulty. To make any of this happen, the heavy AI calculation must live off-chain. The smart contract only sees the final result.
That creates an immediate trust problem. If an off-chain server tells a smart contract that an AI model called a 'Buy', how does the contract know the server didn't hallucinate—or lie—to trigger a liquidation? You need a verification system. Today, two competing design patterns handle this trade-off: Optimistic Fraud Proofs and Zero-Knowledge Machine Learning (zkML).
The Determinism Trap: Why AI on Blockchains is Hard
Before weighing verification models, you need to know why AI hates blockchains. Blockchains demand 100% consensus across thousands of nodes. Every node must run the exact same state transition and hit the exact same byte-for-byte result.
Standard AI frameworks don't play by those rules. Machine learning runs on floating-point math (decimal numbers governed by the IEEE 754 standard). Run a matrix multiplication on an NVIDIA H100 GPU, an AMD GPU, and an Intel CPU, and you'll easily land on tiny differences in the final decimal places. Hardware-level instruction optimization, parallel thread execution order, and rounding modes cause this split. In standard software, a 0.000000001 variation doesn't matter. In blockchain consensus, a single bit mismatch splits the entire chain.
To put AI on-chain, models first go through quantization. This process converts floating-point weights and inputs into fixed-point integers. Once converted, the model becomes deterministic: every computer on Earth running that exact integer circuit spits out the exact same byte output. After quantization, you can verify the model using either an optimistic design or a zero-knowledge circuit.
Approach 1: Optimistic Verification (Fraud Proofs)
Optimistic verification rests on a simple rule: trust the worker, but give anyone the power to punish a liar. It uses the exact same economic setup that powers Optimistic Rollups like Arbitrum and Optimism.
In an optimistic AI setup, an off-chain node (the Operator) runs the quantized model on local hardware. The Operator submits the answer to the smart contract along with a financial stake (a bond). The contract accepts the answer provisionally right away, but opens a challenge window—usually running anywhere from 30 minutes to seven days depending on what security the app needs.
During this window, independent watchtowers (Challengers) track submitted outputs. They run the identical quantized model locally. If a Challenger spots a fake or incorrect AI output from an Operator, they trigger an on-chain dispute challenge.
The Bisection Game
A smart contract can't re-run the whole model to resolve the fight—it would hit a gas limit wall. Instead, the Operator and Challenger enter an interactive bisection game handled by the contract.
The AI model's execution gets split into discrete steps (like individual layer matrix operations). The protocol forces both parties to pinpoint the exact step where their calculated states split. Using a binary search back and forth over multiple transactions, they isolate millions of operations down to a single instruction step. The smart contract then runs just that single step on-chain to pick the winner. If the Operator lied, their bond gets slashed, part of it goes to the Challenger, and the fake output gets thrown out.
- Pros: Generating the output takes milliseconds. Off-chain compute costs match normal AI scripts almost dollar-for-dollar. You don't need specialized cryptographic hardware.
- Cons: Settlement takes time. You can't instantly finalize a trade or cash out casino winnings while waiting for the challenge window to close.
Approach 2: Zero-Knowledge Machine Learning (zkML)
Zero-Knowledge Machine Learning throws out the challenge window completely. Instead of asking the network to trust an output provisionally, the off-chain prover generates a mathematical proof alongside execution. This proof guarantees a specific set of inputs ran through a specific, public neural network model to yield an exact output—without forcing the contract to re-run the math.
To make this happen, the quantized machine learning model gets mapped into an arithmetic circuit made of addition and multiplication gates over a finite field. Every single activation function (like ReLU or GELU) and layer weight must be proven step-by-step using ZK-SNARK or ZK-STARK proof systems.
Once generated, the proof and result hit the smart contract in a single transaction. The contract runs a light verification script. Verification takes a few milliseconds and burns a fixed, low gas amount, no matter how huge the underlying AI model is.
- Pros: Instant finality. Once the block includes the proof, the state is cryptographically locked in. It unlocks total privacy: you can prove an AI model scored a user's private credit profile without leaking the model weights or personal user data.
- Cons: Massive compute overhead. Generating a ZK proof for matrix operations requires building millions to billions of mathematical constraints. Prover overhead runs 10,000x to 1,000,000x the computational cost of running the raw AI inference by itself.
Optimistic vs zkML: Architectural Comparison
| Metric | Optimistic Verification | zkML Verification |
|---|---|---|
| Off-Chain Compute Cost | Negligible ($0.0001 - $0.001 per run) | Extremely High ($0.50 - $50+ per run) |
| Hardware Requirements | Standard CPU / GPU | Massive RAM + Multi-GPU Clusters |
| On-Chain Gas Cost | Low (posting raw result + hash) | Moderate (fixed ZK verification gas) |
| Time to Finality | Hours to Days (Challenge Window) | Minutes (Prover time) + seconds (Block inclusion) |
| Data Privacy | None (Inputs and weights are public) | High (Inputs or weights can remain hidden) |
| Model Complexity Limit | Very High (Billions of parameters) | Low to Moderate (1M - 50M parameters practical) |
Worked Example: Automated Trading Vault
Let's walk through a real setup: An automated asset vault on Ethereum controls $5,000,000 in user funds. It uses a 20-million parameter AI model to check market volatility and adjust target leverage every hour.
Here is how both approaches play out under the exact same conditions:
Scenario A: Using an Optimistic Oracle
The vault operator runs the 20M parameter model on a single desktop GPU in 30 milliseconds. Compute cost: under $0.0001.
The operator posts the new leverage target (say, 2.5x long ETH) to the vault contract along with a $10,000 ETH bond. Gas to send this transaction costs roughly 45,000 gas ($1.50 under standard network conditions).
The protocol starts a 1-hour challenge window. For 60 minutes, the position sits frozen. Watchtowers review the data. Because the operator reported honestly, no dispute starts. After 60 minutes, the vault opens the trade.
Total latency: 1 hour, 30 milliseconds. Total cost per execution: ~$1.50.
Scenario B: Using zkML
The vault operator converts that same 20M parameter model into a Halo2 arithmetic circuit. Generating the cryptographic proof demands a high-memory server loaded with four enterprise GPUs burning cycles for 8 minutes straight. Compute cost: ~$1.20 in energy and server time.
The operator submits the answer and the ZK proof directly to the contract. The contract checks the proof on the spot. Verification uses about 300,000 gas ($10.00 at standard gas rates).
The trade executes immediately inside the same block housing the proof.
Total latency: 8 minutes (prover time) + 12 seconds (block generation). Total cost per execution: ~$11.20.
The Trade-off Decision
If the trading strategy trades ultra-fast price action where a 1-hour wait kills your edge, the optimistic setup falls apart. The vault has to swallow the heavy zkML prover cost to get 8-minute execution.
On the flip side, if the bot rebalances weekly, paying $11.20 per run for instant finality burns money for nothing. The vault should take the optimistic path, paying $1.50 while the 1-hour challenge window ticks away safely in the background.
How to Deploy an Optimistic AI Pipeline
If you're building an optimistic AI verification pipeline for your app, follow these steps in order to keep computation execution identical across every node:
- Quantize the AI Model: Convert all model weights, bias matrices, and activation functions from floating-point precision (FP32 or FP16) to fixed-point integers (INT8 or INT16). Lock down matrix math rules so rounding acts deterministically across different hardware.
- Generate Execution Step Hashes: Build your execution framework to record state roots at fixed operational checkpoints (like after every neural network layer runs). These intermediate hashes serve as the anchor for bisection checks during disputes.
- Deploy the Core Oracle Contract: Program the smart contract to accept model outputs formatted as plain byte arrays while collecting collateral bonds from the operator address.
- Set Up Independent Watchtower Nodes: Run separate, isolated watcher servers across different cloud providers. Ensure these nodes monitor on-chain output events, re-run quantized scripts locally when alerted, and automatically submit dispute transactions if outputs don't match.
- Implement On-Chain Step Execution: Build a dedicated EVM library that can run a single matrix multiplication step in raw Solidity. This code only fires during the final step of a bisection dispute game to settle disputes and slash bad actors.
Three Critical Pitfalls Developers Face
Plugging AI models into smart contracts breaks easily. Here are three major architecture missteps that wreck live apps:
1. Floating-Point Leakage
Teams frequently quantize 95% of their model but leave activation functions like Sigmoid or GELU wired to native system C math libraries (using standard `math.h` functions). Because those libraries handle floating-point rounding slightly differently across operating systems, your model yields different outputs on Linux than Windows. One shifted bit during an optimistic challenge will slash an honest node.
2. Expecting zkML to Prove Model Quality
Zero-Knowledge proofs only certify that output Y came from running input X through model Z. zkML does not prove model Z is accurate, unbiased, or smart. If you deploy a broken model or feed it terrible training data, zkML will happily stamp a valid cryptographic proof certifying your model made a completely terrible choice.
3. Memory Overflows During zkML Proof Generation
When mapping matrix multiplications into ZK constraints, memory usage climbs exponentially relative to parameter counts. Generating a zkML proof for even a middle-weight model (over 100M parameters) can quickly demand over 256GB of system RAM. Developers routinely design zkML systems on tiny test models, only to watch production servers crash from Out-Of-Memory (OOM) errors at scale.
Frequently Asked Questions
Why can't GPUs run smart contracts directly?
GPUs excel at parallel, non-deterministic execution of floating-point math. Smart contracts require sequential, 100% deterministic execution where every single validator reaches the exact same state bit-for-bit. Letting GPUs run smart contracts directly would trigger non-stop consensus failures across varied hardware architectures.
What is fixed-point quantization and why is it mandatory for on-chain AI?
Fixed-point quantization maps continuous floating-point numbers (decimals) to discrete fixed-point integers. It is mandatory because floating-point hardware instructions vary across different CPUs and GPUs, creating tiny calculation discrepancies. Integer operations execute identically on every chip on Earth, protecting consensus alignment.
Can optimistic and zkML models be combined?
Yes. These are called hybrid or fallback architectures. Protocols run optimistically by default to keep daily execution costs near zero. If a challenger disputes an output, instead of working through a multi-transaction bisection game on-chain, the operator must submit a single zkML proof within a set window to instantly prove their output was correct and clear the challenge.
