Oberkampf and Roy’s book on “Verification & Validation in Scientific Computing” discusses “five stages of maturity of predictive capability”, drawing on a 1986 NRC report on “Current Capabilities and Future Directions in Computational Fluid Dynamics”. That’s not so current anymore, but this figure from the NRC report is interesting.
Stages I and II (partly) reflect where the geodynamics community was 10 years ago, before the establishment of CIG, while Stages IV and V (partly) reflect where we are now. I see some differences from this engineering approach:
– For scientists, the scientific payoff starts in stage I, as we build knowledge.
– We are ‘subject to surprise’ at every stage.
– Working on natural systems means we usually lack supporting experimental comparisons.
– We’ve progressed from hero codes, written by one or a small number of people, to community-supported codes.
The figure also presents a linear progress towards maturity. In practice, there is a feedback loop: as scientists learn to use a code effectively, we discover the new physics that we want to add to that code, so there should be an arrow looping partially back to stage 1. I suppose that means we’ll never fully mature.