AI evolution
AI that evolves AI.
Diablo AI shows companies what is actually happening inside their AI systems: not just that a score moved, but which change moved it, and how sure they can be.
How it works
One loop, from a question to a better AI.
Diablo doesn’t jump to a fix. It works the way a careful researcher would, and every step leaves a record. Turn the dial.
01 / 08
Question
You ask what you want to understand about your AI.
Drag to turn · arrow keys work too
The principle
The AI reasons. The system measures.
The model proposes and explains. It never writes a number: every number you read is computed by code from the recorded results.
The AI reasons
- Proposes hypotheses
- Designs experiments
- Explains what the results mean
The system measures
- Runs the experiments
- Scores every answer
- Computes the statistics
- Grades how far each result can be trusted
Who writes the numbers? Drag the handle.
Shortening the system prompt cut accuracy by {{F1}}−15.0 pp ({{F2}}p = 0.012; 95% CI {{F3}}−26.3 to −3.8 pp). Raising the temperature changed nothing measurable ({{F4}}−1.3 pp, p > 0.99).
Fact table · measured
- F1−15.0 ppE1 · prompt · 69/80 → 57/80 correct
- F2p = 0.012E1 · exact McNemar, paired
- F3−26.3 to −3.8 ppE1 · 95% confidence interval
- F4−1.3 pp, p > 0.99E2 · temperature · 69/80 → 68/80
An illustration of the rule. In the live product the investigator writes placeholders such as {{F3}}, a checker rejects any number it types itself, and code fills in every value.
An example
Watch Diablo find a cause.
Version two of an assistant got worse at arithmetic. Two things changed at once. Which one did it?
Conclusion
Question
Why did Helper v2 get worse at arithmetic?
H1 · competingsupported
The shorter system prompt
H2 · competingruled out
The higher temperature
Experiments · the same 80 problems in both arms
E1 · Only the system prompt changestests H1
Effect · Δ −15.0 pp · p = 0.012
E2 · Only the temperature changestests H2
No effect · Δ −1.3 pp · p > 0.99
Conclusion
The system prompt caused the drop. The temperature did not.
Next: restore the prompt in v2 and verify the fix with a new experiment.
Report ready
An illustrative example, not customer data. Every statistic in it is computed by Diablo’s own code.
Persistent understanding
Every investigation leaves a record.
Today that record is a report you can trace down to the raw outputs. Next, Diablo keeps what it learns across investigations, so your AI system stops being a black box.
What it will keep
- Behaviours
- Capabilities
- Failure modes
- Causes
- Interventions
- Counterexamples
- Successful changes
- Failed experiments
- Open questions
A typical AI agent
“Do this task.”
Changes something right away and hopes it helped.
Diablo
“Understand this AI.”
Asks first, then tests, then concludes.
Before any change, Diablo asks
- 1What do we actually know?
- 2What don’t we know?
- 3What are the possible causes?
- 4Which experiment would tell them apart?
- 5Did the change actually work?
Investigate your AI.
Ask Diablo a question about your AI. Sign in with Google and watch it investigate.