Daniel Klotz, Martin Gauch, Grey Nearing, Sepp Hochreiter, and Frederik Kratzert

Skillful today, inept tomorrow. Today’s hydrological models have pronounced and complex error dynamics (e.g., small, highly correlated errors for low flows and large, random errors for high flows). Modelers generally accept that simple, variance based evaluation criteria — like the Nash-Sutcliffe Efficiency (NSE) — are not fully able to capture these intricacies. The (implied) consequences of this are however seldom discussed.

This contribution examines how evaluating the model over two data partitions (above and below a chosen threshold) relates to a global model evaluation of both partitions combined (i.e., the usual way of computing the NSE). For our experiments we manipulate dummy simulations with gradient descent to approximate specific NSE values for each partition individually. Specifically, we set the NSE for runoff values that fall below the threshold, and vary the NSE of the simulations above the threshold as well as the threshold itself. This enables us to study how the global NSE relates to the partition NSEs and the threshold. Intuitively, one would wish that the global NSE somehow reflects the performance on the partitions in a comprehensible manner. We do however show that this relation is not trivial.

Our results also show that subdividing the data and evaluating over the resulting partitions yields different information regarding model deficiencies than an overall evaluation. The downside is that we have less data to estimate the NSE. In the future we can use this for model selection and diagnostic purposes.

EGU23-15221, 2023-02-22.

Download
View paper
IARAI Authors
Dr Sepp Hochreiter
Research
Weather and Climate
Keywords
Hydrology, Machine Learning, Rainfall-Runoff, Uncertainty Estimation

©2023 IARAI - INSTITUTE OF ADVANCED RESEARCH IN ARTIFICIAL INTELLIGENCE

Imprint | Privacy Policy

Stay in the know with developments at IARAI

We can let you know if there’s any

updates from the Institute.
You can later also tailor your news feed to specific research areas or keywords (Privacy)
Loading

Log in with your credentials

Forgot your details?

Create Account