NumericalDiff: Numerical Program Differencing via Value-Based Trace Alignment and Intervention

Xiang Chen, Huanhuan Xia

Numerical programs implementing the same computational task can exhibit different numerical behavior because of their execution stacks, program representations, or program logic. Source text and final outputs alone provide limited diagnostic information: numerical differences can propagate through corresponding computations, while distinct executions can produce equal outputs. We present NumericalDiff, an execution-based method that records assignment-level numerical events, aligns traces using a tolerance-aware longest common subsequence independent of variable names and source locations, and optionally reexecutes after replacing one intermediate value with its counterpart. Among the original and all single-intervention executions, it lexicographically maximizes matched events and minimizes unmatched events. NumericalDiff supports code comparison across language and framework front ends, currently including NumPy, PyTorch, JAX, C, C++, Fortran, and Octave. We evaluate NumericalDiff on 36 controlled cases comprising 675 pair/input instances: 36 execution-stack, 588 program-representation, and 51 program-logic instances. All 24 operator-injection instances expose trace…