Five stages of maturity


Oberkampf and Roy’s book on “Verification & Validation in Scientific Computing” discusses “five stages of maturity of predictive capability”, drawing on a 1986 NRC report on “Current Capabilities and Future Directions in Computational Fluid Dynamics”.  That’s not so current anymore, but this figure from the NRC report is interesting.

Stages I and II (partly) reflect where the geodynamics community was 10 years ago, before the establishment of CIG, while Stages IV and V (partly) reflect where we are now.   I see some differences from this engineering approach:

– For scientists, the scientific payoff starts in stage I, as we build knowledge.

– We are ‘subject to surprise’ at every stage.

– Working on natural systems means we usually lack supporting experimental comparisons.

– We’ve progressed from hero codes, written by one or a small number of people, to community-supported codes.

The figure also presents a linear progress towards maturity. In practice, there is a feedback loop: as scientists learn to use a code effectively, we discover the new physics that we want to add to that code, so there should be an arrow looping partially back to stage 1. I suppose that means we’ll never fully mature.

Should CIG hold a community-wide meeting every 2 years?

An NSF visiting committee recently suggested that CIG should consider holding a community-wide meeting, in lieu of the current array of domain-specific meetings. Similar community-wide meetings are held by other organizations, such as IRIS, COMPRES, UNAVCO, and EarthScope.   We would like to hear the thoughts of the CIG community on this idea.  Most likely we would hold such a meeting every other year, alternating with years in which we hold smaller, domain-specific meetings. To maximize the impact, the community-wide meeting might consist of 2-3 days of plenary sessions, bracketed by topical tutorials, so that everyone could attend for a few days and then optionally add on a day or two of tutorials.  The NSF visiting committee felt strongly that such a meeting would build community and would seed collaboration and learning between different domains in CIG.

Should CIG hold an all-hands meeting? If so, how should it be structured and how often should it be held?