Information Theory for Model Diagnostics: Structural Error is Indicated by Trade-Off Between Functional and Predictive Performance

Benjamin L. Ruddell, Darren T. Drewry, Grey S. Nearing

Research output: Contribution to journalArticlepeer-review

30 Scopus citations


Because of the possibility of getting the right answers for the wrong reasons, the predictive performance of a complex systems model is not by itself a reliable indicator of hypothesis quality for the purposes of scientific learning about processes. The predictive performance of a structurally adequate model should be an emergent property of its functional performance. In this context, any Pareto trade-off between measures of predictive performance versus functional performance indicates process-level error in the model; this trade-off, if it exists, indicates that the model's predictions are right for the wrong functional reasons. This paper demonstrates a novel concept based on information theory that is capable of attributing observed errors to specific processes. To demonstrate that the concept and method hold true for models and observations of real systems, we employ a minimal single-parameter-variation sensitivity analysis using a sophisticated ecohydrology model, MLCan, for a well-monitored field site (Bondville IL Ameriflux Soybean). We identify both functional and predictive error in MLCan, and also evidence of the hypothesized tradeoffs between the two. This trade-off indicates structural error within MLCan. For example, the sensible heat flux process can be calibrated to achieve good predictive performance at the cost of poor functional performance. In contrast, we find little structural error for processes driven by solar radiation, which appear “right for the right reasons.” This method could be applied broadly to pinpoint process error and structural error in a wide range of system models, beyond the ecohydrological scope demonstrated here.

Original languageEnglish (US)
Pages (from-to)6534-6554
Number of pages21
JournalWater Resources Research
Issue number8
StatePublished - Aug 1 2019
Externally publishedYes


  • Pareto optimality
  • benchmarking
  • complex systems
  • hypothesis testing
  • information theory
  • modeling

ASJC Scopus subject areas

  • Water Science and Technology


Dive into the research topics of 'Information Theory for Model Diagnostics: Structural Error is Indicated by Trade-Off Between Functional and Predictive Performance'. Together they form a unique fingerprint.

Cite this