The potential of an observational data set for calibration of a computationally expensive computer model
We measure the potential of an observational data set to constrain a set of inputs to a complex and computationally expensive computer model. Measures of the quality of constraint could inform strategy for data collection campaigns before any real-world data is collected, as well as acting as an effective sensitivity analysis, and an indication of how well we could constrain projections of future climate change.
We use each member in turn of an ensemble of output from the model, corresponding to some observable part of a modelled system, as a proxy for an observational data set. The ensemble provides a set of known parameter input and model output pairs, which we use to build a cheap statistical proxy for the full computer model, termed an emulator. We use the emulator to constrain the input space for each ensemble member, ruling out “implausible” values for the inputs of held-out ensemble members, given the computer model output. We argue that our ability to constrain uncertain parameter inputs to a model using its own output as data provides a maximum bound for our ability to constrain the model inputs using observations of the real system.
We use an ensemble of the ice sheet model Glimmer to demonstrate our measures of quality of constraint. We investigate the impact of observational and model bias uncertainties on our ability to constrain the ice sheet model.