Most of us here ship code that has to behave the same on someone else's machine, and we lean on containers and lockfiles to get there. Scientific code has the same problem with a nastier failure mode, because when it breaks nothing crashes, the program just returns different numbers.
Three Errors, Not One
Numerical code carries three distinct kinds of error and each one needs a different response. Round-off error comes from finite precision, since IEEE 754 doubles hold about 15 significant decimal digits and subtracting nearly equal values throws most of them away. Truncation error comes from replacing a continuous operation with a discrete one, which is what a finite difference or a time step actually is. Modeling error is the gap between the equations and reality, and no amount of grid refinement touches it.
Refining the discretization fixes the second one, makes the first one slightly worse by adding operations, and does nothing at all for the third. Most arguments about a simulation being wrong turn out to be arguments about which of the three someone is looking at. This guide to scientific computing methods walks through each of them next to the numerical methods they come from.
Why The Same Code Gives Different Answers
Change the core count and the summation order changes. Switch GPU models and the reduction tree changes. Let the compiler vectorize differently and the rounding changes. Different math libraries return different last digits for the same transcendental function. Every one of those is a legal, standards conforming result, and in a chaotic system the difference grows exponentially instead of staying in the noise.
That is why bitwise reproduction is the wrong target for a lot of scientific code. The reproducible quantity is the ensemble average with error bars, not the individual trajectory.
Verification Is Not Validation
Verification asks whether you solved the equations correctly, which is a software question. Is the method implemented without bugs, and does it converge at the order it claims? Validation asks whether you solved the right equations, which is a physics question you answer against experiment.
The method of manufactured solutions is the technique worth stealing here. You choose a solution first, substitute it into the governing equations to generate whatever source terms make it exact, then check that the code converges to it at the expected rate. That turns looks about right into a pass or fail test, which is exactly what unit tests do for ordinary code.
What Actually Fixes It
The habits are ones we already have, applied to the whole environment rather than just the source. Git pins the code version behind each figure. Docker or Singularity pin the libraries and the OS layer. Snakemake or Nextflow record parameters and step order so a run can be replayed instead of remembered. Regression tests catch the day a refactor moves a result by 1e-9, which is the change nobody notices until a reviewer asks about it.
None of that is exotic. It is the same reproducibility discipline we expect from a build, pointed at a program whose output is a number instead of a binary.
