AI models are becoming more reliable when they stop treating every problem as a language task. Instead of answering only from patterns in text, newer systems can write code, execute it, run a simulation, or submit a formal proof to a verification system. This changes the process from asking what sounds plausible to testing what survives an external check.
Code makes reasoning executable
Language models can generate clear explanations, but ordinary text reasoning does not automatically enforce correct arithmetic, loop behavior, or algorithmic logic. Code makes those parts concrete. A model can translate a question into a short program, execute it, and use the result to write a human-readable response.
A 2026 study in Proceedings of Machine Learning Research compared natural-language reasoning with deterministic code execution on a 40-task algorithmic benchmark. The researchers reported that reliable execution outperformed natural-language reasoning by 31.6 percentage points. They also found that writing reasoning in a code-like format was not enough. The main improvement came from actually running the code.
This distinction matters for AI engineering. A model can write a convincing but incorrect program, just as it can write a convincing but incorrect paragraph. Execution can catch errors after the code runs, but it cannot prove that the program correctly represents the original question.
Physics engines test hidden assumptions
Code execution helps with rules and calculations. Physics engines provide a different kind of external check. They calculate how objects should move under defined conditions, including factors such as gravity, friction, mass, contact, and force. The model describes the scene, converts that description into code or simulation settings, and uses the resulting behavior as evidence.
Google Research’s Mind’s Eye demonstrated this approach with MuJoCo, a computational physics engine. The system converted physical questions into rendering code, ran simulations, and supplied the results to a language model. Across 39 physics reasoning tasks, the researchers reported average absolute gains of 27.9% in zero-shot tests and 46.0% in few-shot tests. In one example, simulation helped avoid the mistake that a heavier object must fall faster than a lighter object under the same conditions.
Later work made the feedback loop more flexible. The LLMPhy framework uses a model to generate programs that estimate unknown physical properties, such as friction, damping, and object layout. The simulator returns errors from each trial, and the model revises its estimates. On the researchers’ TraySim benchmark, the combined system improved accuracy by about 3% over several comparison methods.
Formal proofs check software behavior
Code execution and simulation test particular cases. Formal verification seeks a stronger guarantee: a mathematical proof that an implementation satisfies a defined property under stated assumptions. It uses a formal specification, which precisely describes what the software must do, and a proof checker that accepts or rejects the result.
Google DeepMind’s AlphaProof Code shows how AI agents can support this process. The system combines a human-guided specification agent, a formal model of AArch64 machine code, and autonomous prover agents. Applied to the unmodified reference implementation of the Signal protocol, the project generated about 6.5 million lines of machine-checked Lean proof code based on roughly 26,000 lines of trusted specifications and machine-code semantics.
Here, the model does not receive the final authority. Lean’s kernel checks whether the proof follows from the approved specification. This separates the model’s search process from the final acceptance decision and makes the result reproducible.
Simulation cannot fix bad models
These tools reduce uncertainty, but they do not remove it. A physics engine can produce a precise answer for the wrong mass, friction value, geometry, or boundary condition. A program can execute flawlessly while representing the wrong interpretation of a request. A formal proof can be completely valid while proving a specification that omitted an important security requirement.
Reliable AI systems therefore need separate controls for interpretation, execution, and validation. The model should identify assumptions, use tools suited to the question, preserve the inputs and outputs, and distinguish a verified result from an estimate. Teams should also test whether the tool was used correctly rather than treating tool use itself as proof of quality.
The broader design lesson is simple: language models are strongest when they generate hypotheses, while code interpreters, simulators, theorem provers, and other deterministic systems check the parts that can be checked. This combination will not make every answer correct, but it can make errors easier to expose, measure, and audit.
References
- Cherian, A., Corcodel, R., Jain, S., & Romeres, D. (2024). LLMPhy: Complex physical reasoning using large language models and world models. arXiv. https://arxiv.org/abs/2411.08027
- Google DeepMind. (2026, November 10). AlphaProof Code: Verifying Signal’s security at the machine-code level. https://google-deepmind.github.io/code-verification/
- Liu, R., Wei, J., Gu, S. S., Wu, T.-Y., Vosoughi, S., Cui, C., Zhou, D., & Dai, A. M. (2022). Mind’s Eye: Grounded language model reasoning through simulation. arXiv. https://arxiv.org/abs/2210.05359
- Tong, T., Feng, Y., Goel, S., & Roth, D. (2026). Is code better than language for algorithmic reasoning? Proceedings of Machine Learning Research, 306, 121547–121561. https://proceedings.mlr.press/v306/tong26d.html
