Skip to main content
:professors
:professors
all professors

Daniel Klotz

Assistant Professor of Machine Learning in Earth Science

title
Asst. Prof.
email
:back to overview

Publications

TitleYear
Analyzing the generalization capabilities of a hybrid hydrological model for extrapolation to extreme events
2025
Eduardo Acuña Espinoza, Ralf Loritz, Frederik Kratzert, Daniel Klotz, Martin Gauch, Man…
Hydrology and Earth System Sciences 29 (5), 1277-1294
:read more
Authors:Eduardo Acuña Espinoza, Ralf Loritz, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez Chaves, Uwe Ehret
Publication date:05/05/2025
Journal:Hydrology and Earth System Sciences
Volume:29
Issue:5
Pages:1277-1294
Publisher:Copernicus GmbH
Description:Data-driven techniques have shown the potential to outperform process-based models in rainfall–runoff simulation. Recently, hybrid models, which combine data-driven methods with process-based approaches, have been proposed to leverage the strengths of both methodologies, aiming to enhance simulation accuracy while maintaining a certain interpretability. Expanding the set of test cases to evaluate hybrid models under different conditions, we test their generalization capabilities for extreme hydrological events, comparing their performance against long short-term memory (LSTM) networks and process-based models. Our results indicate that hybrid models show performance similar to that of the LSTM network for most cases. However, hybrid models reported slightly lower errors in the most extreme cases and were able to produce higher peak discharges.
Resources:[PDF] from uni-stuttgart.de
:view less
Caravan–A global community dataset for large-sample hydrology, Sci. Data, 10, 61
2025
Frederik Kratzert, Grey Nearing, Nans Addor, Tyler Erickson, Martin Gauch, Oren Gilon, …
:read more
Authors:Frederik Kratzert, Grey Nearing, Nans Addor, Tyler Erickson, Martin Gauch, Oren Gilon, Lukas Gudmundsson, Avinatan Hassidim, Daniel Klotz, Sella Nevo, Guy Shalev, Yossi Matias
Publication date:05/05/2025
Cited by:62
:view less
Rainfall-Runoff modeling with Long Short-Term Memory Networks (LSTM)—an overview
2025
Frederik Kratzert, Martin Gauch, Grey Nearing, Sepp Hochreiter, Daniel Klotz
:read more
Authors:Frederik Kratzert, Martin Gauch, Grey Nearing, Sepp Hochreiter, Daniel Klotz
Publication date:05/05/2025
Description:In recent years, methods of artificial intelligence have become essential components of almost all branches of science and technology. This also holds for the area of hydrology: multi-layered neural networks—also known as Deep Learning models—allow for rainfall-runoff predictions at unprecedented accuracy.This contribution highlights the potenzial of Deep Learning for applications of water management. The first part of this article shows how so-called Long Short-Term Memory networks—a Deep Learning architecture explicitly designed for time series prediction—can be used for rainfall-runoff modeling, and how this approach yields better results than any known hydrologic model on a number of hydrologic problems. The second part of this article outlines important properties of Long Short-Term Memory networks. We show that these models can process any available data. This allows them to draw synergetic information from multiple meteorological datasets, which improves predictions. Further, we demonstrate how these models learn relevant hydrologic processes (e.g., accumulation and melting of snow) without specifically being trained to do so.
:view less
Regionalization of hydrological models using function space optimization
2025
Moritz Feigl, Mathew Herrnegger, Robert Schweppe, Stephan Thober, Daniel Klotz, Luis Sa…
:read more
Authors:Moritz Feigl, Mathew Herrnegger, Robert Schweppe, Stephan Thober, Daniel Klotz, Luis Samaniego, Karsten Schulz
Publication date:05/05/2025
Description:The estimation of distributed hydrological model parameters is a long studied and challenging problem. Parameter transfer function, i.e. mathematical equations that describe the relationship between model parameters and geo-physical catchment properties, make it possible to potentially estimate parameters without the need for calibration. Function Space Optimization (FSO) is a symbolic regression method for automatically estimating parameter transfer functions from data. FSO is based on a text generating neural network that transforms the search for a best fitting transfer function into a continuous optimization problem.In this contribution we discuss FSO and its characteristics and show its applicability with the mesoscale Hydrological Model (mHM). The aim of this application is the estimation of transfer functions for two model parameters: KSat (saturated hydraulic conductivity) and FieldCap (field capacity). We use the data of 7 large German Basins and 5 years of data to estimate the transfer functions and additional numerical parameters of mHM. The resulting functions und parameters are applied without any further calibration to 222 validation basins with 35 different years of data. By applying the model in these “ungauged” basins, it is possible to estimate the transferability in time and space of the estimated transfer functions.The application in ungauged basins result in model performance values which are in similar range compared to the training basins. The median Nash-Sutcliffe Efficiency (NSE) in the training basins in the validation time-period is 0.73, while the median NSE of the validation basins in the same time period is 0.65.Transfer …
:view less
A machine learner’s guide to streamflow prediction
2025
Martin Gauch, Daniel Klotz, Frederik Kratzert, Grey Nearing, Sepp Hochreiter, Jimmy Lin
AI for Earth Sciences Workshop at NeurIPS
:read more
Authors:Martin Gauch, Daniel Klotz, Frederik Kratzert, Grey Nearing, Sepp Hochreiter, Jimmy Lin
Publication date:05/05/2025
Journal:AI for Earth Sciences Workshop at NeurIPS
Description:Although often subconsciously, many people deal with water-related issues on a daily basis. For instance, many regions rely on hydropower plants to produce their electricity, and, at the extreme, floods and droughts pose one of the big environmental threats of climate change. At the same time, many machine learning researchers have started to look beyond their field and wish to contribute to environmental issues of our time. The modeling of streamflow—the amount of water that flows through a river cross-section at a given time—is a natural starting point to such contributions: It encompasses a variety of tasks that will be familiar to machine learning researchers, but it is also a vital component of flood and drought prediction (among other applications). Moreover, researchers can draw upon large open datasets, sensory networks, and remote sensing data to train their models. As a getting-started resource, this guide provides a brief introduction to streamflow modeling for machine learning researchers and highlights a number of possible research directions where machine learning could advance the domain.
Cited by:4
Resources:[PDF] from github.io
:view less
NeuralHydrology–interpreting LSTMs in hydrology
2025
Frederik Kratzert, Mathew Herrnegger, Daniel Klotz, Sepp Hochreiter, Günter Klambauer
Explainable AI: Interpreting, explaining and visualizing deep learning, 347-362
:read more
Authors:Frederik Kratzert, Mathew Herrnegger, Daniel Klotz, Sepp Hochreiter, Günter Klambauer
Publication date:05/05/2025
Journal:Explainable AI: Interpreting, explaining and visualizing deep learning
Pages:347-362
Publisher:Springer International Publishing
Description:Despite the huge success of Long Short-Term Memory networks, their applications in environmental sciences are scarce. We argue that one reason is the difficulty to interpret the internals of trained networks. In this study, we look at the application of LSTMs for rainfall-runoff forecasting, one of the central tasks in the field of hydrology, in which the river discharge has to be predicted from meteorological observations. LSTMs are particularly well-suited for this problem since memory cells can represent dynamic reservoirs and storages, which are essential components in state-space modelling approaches of the hydrological system. On basis of two different catchments, one with snow influence and one without, we demonstrate how the trained model can be analyzed and interpreted. In the process, we show that the network internally learns to represent patterns that are consistent with our qualitative understanding of …
Cited by:135
Resources:[PDF] from arxiv.org
:view less
How to deal w_ missing input data
2025
Martin Gauch, Frederik Kratzert, Daniel Klotz, Grey Nearing, Deborah Cohen, Oren Gilon
EGUsphere 2025, 1-21
:read more
Authors:Martin Gauch, Frederik Kratzert, Daniel Klotz, Grey Nearing, Deborah Cohen, Oren Gilon
Publication date:04/07/2025
Journal:EGUsphere
Volume:2025
Pages:1-21
Publisher:Copernicus Publications
Description:Deep learning hydrologic models have made their way from research to applications. More and more national hydrometeorological agencies, hydro power operators, and engineering consulting companies are building Long Short-Term Memory (LSTM) models for operational use cases. All of these efforts come across similar sets of challenges—challenges that are different from those in controlled scientific studies. In this paper, we tackle one of these issues: how to deal with missing input data? Operational systems depend on the real-time availability of various data products—most notably, meteorological forcings. The more external dependencies a model has, however, the more likely it is to experience an outage in one of them. We introduce and compare three different solutions that can generate predictions even when some of the meteorological input data do not arrive in time, or not arrive at all.
Resources:[PDF] from copernicus.org
:view less
An approach for handling multiple temporal frequencies with different input dimensions using a single LSTM cell
2025
Eduardo Acuña Espinoza, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez C…
Hydrology and Earth System Sciences 29 (6), 1749-1758
:read more
Authors:Eduardo Acuña Espinoza, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez Chaves, Ralf Loritz, Uwe Ehret
Publication date:03/26/2025
Journal:Hydrology and Earth System Sciences
Volume:29
Issue:6
Pages:1749-1758
Publisher:Copernicus Publications
Description:Long short-term memory (LSTM) networks have demonstrated state-of-the-art performance for rainfall-runoff hydrological modelling. However, most studies focus on predictions at a daily scale, limiting the benefits of sub-daily (e.g. hourly) predictions in applications like flood forecasting. Moreover, training an LSTM network exclusively on sub-daily data is computationally expensive and may lead to model learning difficulties due to the extended sequence lengths. In this study, we introduce a new architecture, multi-frequency LSTM (MF-LSTM), designed to use input of various temporal frequencies to produce sub-daily (e.g. hourly) predictions at a moderate computational cost. Building on two existing methods previously proposed by the co-authors of this study, MF-LSTM processes older inputs at coarser temporal resolutions than more recent ones. MF-LSTM gives the possibility of handling different temporal frequencies, with different numbers of input dimensions, in a single LSTM cell, enhancing the generality and simplicity of use. Our experiments, conducted on 516 basins from the CAMELS-US dataset, demonstrate that MF-LSTM retains state-of-the-art performance while offering a simpler design. Moreover, the MF-LSTM architecture reported a 5 times reduction in processing time compared to models trained exclusively on hourly data.
Resources:[HTML] from copernicus.org
:view less
Towards Deep Learning River Network Models
2025
Martin Gauch, Frederik Kratzert, Daniel Klotz, Guy Shalev, Deborah Cohen, Oren Gilon
EGU25 (EGU25-9768)
:read more
Authors:Martin Gauch, Frederik Kratzert, Daniel Klotz, Guy Shalev, Deborah Cohen, Oren Gilon
Publication date:03/15/2025
Journal:EGU25
Issue:EGU25-9768
Publisher:Copernicus Meetings
Description:Deep Learning models for streamflow prediction are now more than five years old (Kratzert et al., 2018, 2019), and lumped LSTMs, trained on as many basins and forcing products as we can get our hands on, continue to pose the state of the art. Or do they? While traditional hydrologic modeling has long moved beyond lumped modeling, Deep Learning methods are only now starting to leverage the inherent graphical topology of rivers through graph neural networks (GNNs). Such models come with their own set of challenges, both from an engineering standpoint (eg, dealing with the sheer amount of data from many small sub-basins) and from a modeling standpoint (eg, ensuring generalization to ungauged basins along the river graph). Yet, GNNs promise more accurate predictions, the ability to assimilate real-time up-and downstream data, make predictions at arbitrary points along a river, or integrate knowledge …
:view less
Analyzing the generalization capabilities of a hybrid hydrological model for extrapolation to extreme events
2025
Eduardo Acuña Espinoza, Ralf Loritz, Frederik Kratzert, Daniel Klotz, Martin Gauch, Man…
Hydrology and Earth System Sciences 29 (5), 1277-1294
:read more
Authors:Eduardo Acuña Espinoza, Ralf Loritz, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez Chaves, Uwe Ehret
Publication date:03/11/2025
Journal:Hydrology and Earth System Sciences
Volume:29
Issue:5
Pages:1277-1294
Publisher:Copernicus Publications
Description:Data-driven techniques have shown the potential to outperform process-based models in rainfall–runoff simulation. Recently, hybrid models, which combine data-driven methods with process-based approaches, have been proposed to leverage the strengths of both methodologies, aiming to enhance simulation accuracy while maintaining a certain interpretability. Expanding the set of test cases to evaluate hybrid models under different conditions, we test their generalization capabilities for extreme hydrological events, comparing their performance against long short-term memory (LSTM) networks and process-based models. Our results indicate that hybrid models show performance similar to that of the LSTM network for most cases. However, hybrid models reported slightly lower errors in the most extreme cases and were able to produce higher peak discharges.
Resources:[HTML] from copernicus.org
:view less
Unveiling the Limits of Deep Learning Models in Hydrological Extrapolation Tasks
2025
Sanika Baste, Daniel Klotz, Eduardo Acuña Espinoza, Andras Bardossy, Ralf Loritz
EGUsphere 2025, 1-24
:read more
Authors:Sanika Baste, Daniel Klotz, Eduardo Acuña Espinoza, Andras Bardossy, Ralf Loritz
Publication date:02/06/2025
Journal:EGUsphere
Volume:2025
Pages:1-24
Publisher:Copernicus Publications
Description:Long Short-Term Memory (LSTM) networks have shown strong performance in rainfall-runoff modelling, often surpassing conventional hydrological models in benchmark studies. However, recent studies raise questions about their ability to extrapolate, particularly under extreme conditions that exceed the range of their training data. This study examines the performance of a stand-alone LSTM trained on 196 catchments in Switzerland when subjected to synthetic design precipitation events of increasing intensity and varying duration. The model’s response is compared to that of a hybrid model and evaluated against hydrological process understanding. Our study reiterates that the stand-alone LSTM is not capable of predicting discharge values above a theoretical limit, and we show that this limit (73 mm d-1) is below the range of the data the model was trained on (183 mm d-1 when trained on CAMELS-CH). Furthermore, the LSTM exhibits a concave runoff response under extreme precipitation, indicating that event runoff coefficients decrease with increasing design precipitation-a phenomenon not observed in the hybrid model used as a benchmark. We show that saturation of the LSTM cell states, alone, does not fully account for this characteristic behavior, as the LSTM does not reach full saturation, particularly for the 1-day events. Instead, its gating structures prevent new information about the current extreme precipitation from being incorporated into the cell states. Adjusting the LSTM architecture, for instance, by increasing the number of hidden states, and/or using a larger, more diverse training dataset can help mitigate the problem. However …
Cited by:1
Resources:[PDF] from copernicus.org
:view less
EARLS: A runoff reconstruction dataset for Europe
2025
Daniel Klotz, Peter Miersch, Thiago VM do Nascimento, Fabrizio Fenicia, Martin Gauch, J…
Earth System Science Data Discussions 2025, 1-33
:read more
Authors:Daniel Klotz, Peter Miersch, Thiago VM do Nascimento, Fabrizio Fenicia, Martin Gauch, Jakob Zscheischler
Publication date:01/03/2025
Journal:Earth System Science Data Discussions
Volume:2025
Pages:1-33
Description:Data drives our understanding of hydrological processes, supports model development, and enables anticipatory water management. This contribution introduces EARLS: European Aggregated Reconstructions for Large-sample Studies. EARLS offers daily streamflow reconstructions for more than 10,000 basins in Europe including uncertainty estimates, covering the period from 1953 to 2023. The reconstruction is derived from a single Long Short-Term Memory (LSTM) based rainfall–runoff model trained on more than 5,000 basins. LSTMs represent the state of the art in rainfall–runoff modeling and are well suited to provide predictions in ungauged basins. We evaluate the quality of the reconstruction through quantitative evaluation on two held-out sets of basins and by conducting a qualitative assessment that compares EARLS-based peak flows and flood timing to previous large-scale hydrological studies. EARLS represents a new generation of datasets that harness the capabilities of Deep Learning to obtain accurate and high-resolution data. EARLS is available at https://doi.org/10.5281/zenodo.13864843 (Klotz et al., 2024b)
Resources:[PDF] from copernicus.org
:view less
Semi-Distributed Hydrological Modeling Based on Deep Learning at Scale
2024
Frederik Kratzert, Martin Gauch, Guy Shalev, Daniel Klotz, Deborah Cohen, Oren Gilon
AGU24
:read more
Authors:Frederik Kratzert, Martin Gauch, Guy Shalev, Daniel Klotz, Deborah Cohen, Oren Gilon
Publication date:12/13/2024
Journal:AGU24
Publisher:AGU
:view less
An approach for handling multiple temporal frequencies with different input dimensions using a single LSTM cell
2024
Eduardo Acuña Espinoza, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez C…
EGUsphere 2024, 1-12
:read more
Authors:Eduardo Acuña Espinoza, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez Chaves, Ralf Loritz, Uwe Ehret
Publication date:12/12/2024
Journal:EGUsphere
Volume:2024
Pages:1-12
Publisher:Copernicus Publications
Description:Long Short-Term Memory (LSTM) networks have demonstrated state-of-the-art performance for rainfall-runoff hydrological modeling. However, most studies focus on daily-scale predictions, limiting the benefits of sub-daily (e.g. hourly) predictions in applications like flood forecasting. Moreover, training an LSTM exclusively on sub-daily data is computationally expensive, and may lead to model-learning difficulties due to the extended sequence lengths. In this study, we introduce a new architecture, multi-frequency LSTM (MF-LSTM), designed to use input of various temporal frequencies to produce sub-daily (e.g. hourly) predictions at a moderate computational cost. Building on two existing methods previously proposed by coauthors of this study, the MF-LSTM processes older inputs at coarser temporal resolutions than more recent ones. The MF-LSTM gives the possibility to handle different temporal frequencies, with different number of input dimensions, in a single LSTM cell, enhancing generality and simplicity of use. Our experiments, conducted on 516 basins from the CAMELS-US dataset, demonstrate that MF-LSTM retains state-of-the-art performance while offering a simpler design. Moreover, the MF-LSTM architecture reported a 5x reduction in processing time, compared to models trained exclusively on hourly data.
Cited by:1
Resources:[PDF] from copernicus.org
:view less
Simplified priors for Object-Centric Learning
2024
Vihang Patil, Andreas Radler, Daniel Klotz, Sepp Hochreiter
arXiv preprint arXiv:2410.00728
:read more
Authors:Vihang Patil, Andreas Radler, Daniel Klotz, Sepp Hochreiter
Publication date:10/01/2024
Journal:arXiv preprint arXiv:2410.00728
Description:Humans excel at abstracting data and constructing emph{reusable} concepts, a capability lacking in current continual learning systems. The field of object-centric learning addresses this by developing abstract representations, or slots, from data without human supervision. Different methods have been proposed to tackle this task for images, whereas most are overly complex, non-differentiable, or poorly scalable. In this paper, we introduce a conceptually simple, fully-differentiable, non-iterative, and scalable method called SAMP Simplified Slot Attention with Max Pool Priors). It is implementable using only Convolution and MaxPool layers and an Attention layer. Our method encodes the input image with a Convolutional Neural Network and then uses a branch of alternating Convolution and MaxPool layers to create specialized sub-networks and extract primitive slots. These primitive slots are then used as queries for a Simplified Slot Attention over the encoded image. Despite its simplicity, our method is competitive or outperforms previous methods on standard benchmarks.
Resources:[PDF] from arxiv.org
:view less
The divide and measure nonconformity–how metrics can mislead when we evaluate on different data partitions
2024
Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, Jakob Zscheischler
Hydrology and Earth System Sciences 28 (15), 3665-3673
:read more
Authors:Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, Jakob Zscheischler
Publication date:08/13/2024
Journal:Hydrology and Earth System Sciences
Volume:28
Issue:15
Pages:3665-3673
Publisher:Copernicus Publications
Cited by:6
:view less
Analyzing the generalization capabilities of hybrid hydrological models for extrapolation to extreme events
2024
Eduardo Acuna Espinoza, Ralf Loritz, Frederik Kratzert, Daniel Klotz, Martin Gauch, Man…
EGUsphere 2024, 1-17
:read more
Authors:Eduardo Acuna Espinoza, Ralf Loritz, Frederik Kratzert, Daniel Klotz, Martin Gauch, Manuel Álvarez Chaves, Nicole Bäuerle, Uwe Ehret
Publication date:07/30/2024
Journal:EGUsphere
Volume:2024
Pages:1-17
Publisher:Copernicus Publications
Description:Data-driven techniques have shown the potential to outperform process-based models for rainfall-runoff simulation. Recently, hybrid models, which combine data-driven methods with process-based approaches, have been proposed to leverage the strengths of both methodologies, aiming to enhance simulation accuracy while maintaining certain interpretability. Expanding the set of test cases to evaluate hybrid models under different conditions, we test their generalization capabilities for extreme hydrological events, comparing their performance against Long Short-Term Memory (LSTM) networks and process based models. Our results indicate that hybrid models show similar performance as LSTM networks for most cases. However, hybrid models reported slightly lower errors in the most extreme cases, and were able to produce higher peak discharges.
Cited by:3
Resources:[PDF] from copernicus.org
:view less
Energy-based hopfield boosting for out-of-distribution detection
2024
Claus Hofmann, Simon Schmid, Bernhard Lehner, Daniel Klotz, Sepp Hochreiter
arXiv preprint arXiv:2405.08766
:read more
Authors:Claus Hofmann, Simon Schmid, Bernhard Lehner, Daniel Klotz, Sepp Hochreiter
Publication date:05/14/2024
Journal:arXiv preprint arXiv:2405.08766
Description:Out-of-distribution (OOD) detection is critical when deploying machine learning models in the real world. Outlier exposure methods, which incorporate auxiliary outlier data in the training process, can drastically improve OOD detection performance compared to approaches without advanced training strategies. We introduce Hopfield Boosting, a boosting approach, which leverages modern Hopfield energy (MHE) to sharpen the decision boundary between the in-distribution and OOD data. Hopfield Boosting encourages the model to concentrate on hard-to-distinguish auxiliary outlier examples that lie close to the decision boundary between in-distribution and auxiliary outlier data. Our method achieves a new state-of-the-art in OOD detection with outlier exposure, improving the FPR95 metric from 2.28 to 0.92 on CIFAR-10 and from 11.76 to 7.94 on CIFAR-100.
Cited by:7
Resources:[PDF] from arxiv.org
:view less
Generalising Tree-Level Sap Flow Across the European Continent using LSTMs
2024
Ralf Loritz, Chen Huan Wu, Daniel Klotz, Martin Gauch, Frederik Kratzert, Maoya Bassiouni
EGU General Assembly Conference Abstracts, 15285
:read more
Authors:Ralf Loritz, Chen Huan Wu, Daniel Klotz, Martin Gauch, Frederik Kratzert, Maoya Bassiouni
Publication date:04/01/2024
Journal:EGU General Assembly Conference Abstracts
Pages:15285
Description:In this presentation, we explore the application of Long Short-Term Memory networks (LSTMs) to predict hourly tree-level sap flow across Europe, utilizing the comprehensive SAPFLUXNET database. This study emphasizes the potential of deep learning in estimating transpiration and understanding forest water use dynamics and plant-climate interactions. By developing LSTM models with varied training sets, we assess their capability to perform in previously unencountered conditions. Our research reveals that these models achieve an average Kling-Gupta Efficiency of 0.77 when trained on 50% of the time series across all forest stands, and 0.52 for models trained on 50% of the forest stands without prior gauging. These continental-scale models not only meet but often exceed the performance of specialized and baseline models across all tree genera and forest types. In this submission, we will discuss the …
:view less
Deep Learning for Spatially Distributed Rainfall-Runoff Modeling
2024
Martin Gauch, Frederik Kratzert, Vusumuzi Dube, Oren Gilon, Daniel Klotz, Asher Metzger…
EGU General Assembly Conference Abstracts, 8899
:read more
Authors:Martin Gauch, Frederik Kratzert, Vusumuzi Dube, Oren Gilon, Daniel Klotz, Asher Metzger, Grey Nearing, Florence Ofori, Guy Shalev, Shlomo Shenzis, Tadele Tekalign, Dana Weitzner, Oleg Zlydenko, Deborah Cohen
Publication date:04/01/2024
Journal:EGU General Assembly Conference Abstracts
Pages:8899
Description:Deep learning approaches have emerged as the state of the art for rainfall-runoff modeling. Yet—until now—the best-performing models have typically been used with inputs that are averaged across possibly large catchment areas, modeling each (sub-) basin independently. This lumped modeling approach is in contrast to reality, where rivers form networks of connected subbasins. This discrepancy limits our ability to accurately and interpretably predict certain types of rivers. Here, we present recent work to build graph-based deep learning models that explicitly account for this network structure. These models promise to unlock improvements in both the quality and interpretability of predictions: Catchment size: Lumped models lack information about spatial heterogeneity, ie, they do not know where inside a basin events (such as precipitation) occur. This makes it hard to model large basins, especially at high …
:view less
Investigating the divide and measure nonconformity
2024
Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, Jakob Zscheischler
EGU General Assembly Conference Abstracts, 8007
:read more
Authors:Daniel Klotz, Martin Gauch, Frederik Kratzert, Grey Nearing, Jakob Zscheischler
Publication date:04/01/2024
Journal:EGU General Assembly Conference Abstracts
Pages:8007
Description:This contribution presents a diagnostic approach to investigate unexpected side effects that can occur during the evaluation of rainfall--runoff models. The diagnostic technique that we use is based on the idea that one can use gradient descent to modify the runoff observations/simulations to obtain warranted observations/simulations. Specifically, we show how to use this concept to manipulate any hydrograph (eg, a copy of the observations) so that it approximates specific NSE values for individual parts of the data. In short, we follow the following recipe to generate the synthetic simulations:(1) copy the observations,(2) add noise,(3) clip the modified discharge to zero, and (4) optimise the obtained simulation values by using gradient descent until a desired NSE value is reached. To show how this diagnostic technique can be used we demonstrate a behaviour of Nash--Sutcliffe Efficiency (NSE) that appears when …
:view less
Analyzing the performance and interpretability of hybrid hydrological models
2024
Eduardo Acuna, Ralf Loritz, Manuel Alvarez, Frederik Kratzert, Daniel Klotz, Martin Gau…
EGU General Assembly Conference Abstracts, 12574
:read more
Authors:Eduardo Acuna, Ralf Loritz, Manuel Alvarez, Frederik Kratzert, Daniel Klotz, Martin Gauch, Nicole Bauerle, Uwe Ehret
Publication date:04/01/2024
Journal:EGU General Assembly Conference Abstracts
Pages:12574
Description:Hydrological hybrid models have been proposed as an option to combine the enhanced performance of deep learning methods with the interpretability of process-based models. Among the various hybrid methods available, the dynamic parameterization of conceptual models using LSTM networks has shown high potential. In this contribution, we extend our previous related work (Acuna Espinoza et al., 2023) by asking the questions: How well can hybrid models predict untrained variables, and how well do they generalize? We address the first question by comparing the internal states of the model against external data, specifically against soil moisture data obtained from ERA5-Land for 60 basins in Great Britain. We show that the process-based layer can reproduce the soil moisture dynamics with a correlation of 0.83, which indicates a good ability of this type of model to predict untrained variables. Moreover, we …
:view less
Generalizing tree–level sap flow across the European continent
2024
Ralf Loritz, Chen Huan Wu, Daniel Klotz, Martin Gauch, Frederik Kratzert, Maoya Bassiouni
Geophysical Research Letters 51 (6), e2023GL107350
:read more
Authors:Ralf Loritz, Chen Huan Wu, Daniel Klotz, Martin Gauch, Frederik Kratzert, Maoya Bassiouni
Publication date:03/28/2024
Journal:Geophysical Research Letters
Volume:51
Issue:6
Pages:e2023GL107350
Description:Sap flow offers key insights about transpiration dynamics and forest‐climate interactions. Accurately simulating sap flow remains challenging due to measurement uncertainties and interactions between global and local environmental controls. Addressing these complexities, this study leveraged Long Short‐Term Memory networks (LSTMs) with SAPFLUXNET to predict hourly tree‐level sap flow across Europe. We built models with diverse training sets to assess performance under previously unseen conditions. The average Kling‐Gupta Efficiency was 0.77 for models trained on 50% of time series across all forest stands, and 0.52 for models trained on 50% of the forest stands. Continental models not only matched but surpassed the performance of specialized and baselines for all genera and forest types, showcasing the capacity of LSTMs to effectively generalize across tree genera, climates, and forest …
Cited by:4
Resources:[PDF] from wiley.com
:view less
Global prediction of extreme floods in ungauged watersheds
2024
Grey Nearing, Deborah Cohen, Vusumuzi Dube, Martin Gauch, Oren Gilon, Shaun Harrigan, A…
Nature 627 (8004), 559-563
:read more
Authors:Grey Nearing, Deborah Cohen, Vusumuzi Dube, Martin Gauch, Oren Gilon, Shaun Harrigan, Avinatan Hassidim, Daniel Klotz, Frederik Kratzert, Asher Metzger, Sella Nevo, Florian Pappenberger, Christel Prudhomme, Guy Shalev, Shlomo Shenzis, Tadele Yednkachw Tekalign, Dana Weitzner, Yossi Matias
Publication date:03/21/2024
Journal:Nature
Volume:627
Issue:8004
Pages:559-563
Publisher:Nature Publishing Group UK
Description:Floods are one of the most common natural disasters, with a disproportionate impact in developing countries that often lack dense streamflow gauge networks. Accurate and timely warnings are critical for mitigating flood risks, but hydrological simulation models typically must be calibrated to long data records in each watershed. Here we show that artificial intelligence-based forecasting achieves reliability in predicting extreme riverine events in ungauged watersheds at up to a five-day lead time that is similar to or better than the reliability of nowcasts (zero-day lead time) from a current state-of-the-art global modelling system (the Copernicus Emergency Management Service Global Flood Awareness System). In addition, we achieve accuracies over five-year return period events that are similar to or better than current accuracies over one-year return period events. This means that artificial intelligence can provide …
Cited by:152
Resources:[HTML] from nature.com
:view less
A data-centric perspective on the information needed for hydrological uncertainty predictions
2024
Andreas Auer, Martin Gauch, Frederik Kratzert, Grey Nearing, Sepp Hochreiter, Daniel Klotz
Hydrology and Earth System Sciences Discussions 2024, 1-37
:read more
Authors:Andreas Auer, Martin Gauch, Frederik Kratzert, Grey Nearing, Sepp Hochreiter, Daniel Klotz
Publication date:03/14/2024
Journal:Hydrology and Earth System Sciences Discussions
Volume:2024
Pages:1-37
Description:Uncertainty estimates are fundamental to assess the reliability of predictive models in hydrology. We use the framework of Conformal Prediction to investigate the impact of temporal and spatial information on uncertainty estimates within hydrological predictions. Integrating recent information significantly enhances overall uncertainty predictions, even with substantial gaps between updates. While local information yields good results on average, it proves insufficient for peak flow predictions. Incorporating global information improves the accuracy of peak flow bounds, corroborating findings from related studies. Overall, the study underscores the importance of continuous data updates and the integration of global information for robust and efficient uncertainty estimation.
Cited by:4
Resources:[HTML] from copernicus.org
:view less
HESS Opinions: Never train an LSTM on a single basin
2024
Frederik Kratzert, Martin Gauch, Daniel Klotz, Grey Nearing
Hydrology and Earth system sciences discussions 2024, 1-19
:read more
Authors:Frederik Kratzert, Martin Gauch, Daniel Klotz, Grey Nearing
Publication date:01/18/2024
Journal:Hydrology and Earth system sciences discussions
Volume:2024
Pages:1-19
Description:Machine learning (ML) has played an increasing role in the hydrological sciences. In particular, certain types of time series modeling strategies are popular for rainfall–runoff modeling. A large majority of studies that use this type of model do not follow best practices, and there is one mistake in particular that is common: training deep learning models on small, homogeneous data sets (i.e., data from one or a small number of watersheds). In this position paper, we show that Long Short Term Memory (LSTM) streamflow models are best when trained with a large amount of hydrologically diverse data.
Cited by:77
Resources:[HTML] from copernicus.org
:view less
Hopfield Boosting for Out-of-Distribution Detection
2023
Claus Hofmann, Simon Lucas Schmid, Bernhard Lehner, Daniel Klotz, Sepp Hochreiter
NeurIPS 2023 Workshop: Associative Memory & Hopfield Networks
:read more
Authors:Claus Hofmann, Simon Lucas Schmid, Bernhard Lehner, Daniel Klotz, Sepp Hochreiter
Publication date:12/15/2023
Journal:NeurIPS 2023 Workshop: Associative Memory & Hopfield Networks
Description:Out-of-distribution (OOD) detection is crucial for real-world machine learning. Outlier exposure methods, which use auxiliary outlier data, can significantly enhance OOD detection. We present Hopfield Boosting, a boosting technique employing modern Hopfield energy (MHE) to refine the boundary between in-distribution (ID) and OOD data. Our method focuses on challenging outlier examples near the decision boundary, achieving a 40% improvement in FPR95 on CIFAR-10, setting a new OOD detection state-of-the-art with outlier exposure.
Resources:[PDF] from elsevierpure.com
:view less
Conformal prediction for time series with modern hopfield networks
2023
Andreas Auer, Martin Gauch, Daniel Klotz, Sepp Hochreiter
Advances in Neural Information Processing Systems 36, 56027-56074
:read more
Authors:Andreas Auer, Martin Gauch, Daniel Klotz, Sepp Hochreiter
Publication date:12/15/2023
Journal:Advances in Neural Information Processing Systems
Volume:36
Pages:56027-56074
Description:To quantify uncertainty, conformal prediction methods are gaining continuously more interest and have already been successfully applied to various domains. However, they are difficult to apply to time series as the autocorrelative structure of time series violates basic assumptions required by conformal prediction. We propose HopCPT, a novel conformal prediction approach for time series that not only copes with temporal structures but leverages them. We show that our approach is theoretically well justified for time series where temporal dependencies are present. In experiments, we demonstrate that our new approach outperforms state-of-the-art conformal prediction methods on multiple real-world time series datasets from four different domains.
Cited by:34
Resources:[PDF] from neurips.cc
:view less
Data Assimilation in Operational Machine Learning Hydrology Models
2023
Grey Stephen Nearing, Frederik Kratzert, Guy Shalev, Daniel Klotz, Martin Gauch, Oren G…
AGU Fall Meeting Abstracts 2023, H23F-02
:read more
Authors:Grey Stephen Nearing, Frederik Kratzert, Guy Shalev, Daniel Klotz, Martin Gauch, Oren Gilon
Publication date:12/01/2023
Journal:AGU Fall Meeting Abstracts
Volume:2023
Pages:H23F-02
Description:Machine learning (ML) models are an increasingly integral component of large-scale hydrology modeling, in both research and operational settings. One example of an operational ML-based hydrology model is Google's Flood Forecasting system (Nevo et al, 2022), which produces short term flood forecasts globally (g. co/floodhub). ML-based hydrology models face many challenges and opportunities that are similar to traditional hydrology models, and one of the most important of those challenges is data assimilation (DA). Like all large-scale hydrology models, ML-based models are most effective when they ingest as much near-real-time data as possible.
:view less
Generalising Tree-Level Sap Flow Across the European Continent
2023
Ralf Loritz, Chen Huan Wu, Daniel Klotz, Martin Gauch, Frederik Kratzert, Maoya Bassiouni
Authorea Preprints
:read more
Authors:Ralf Loritz, Chen Huan Wu, Daniel Klotz, Martin Gauch, Frederik Kratzert, Maoya Bassiouni
Publication date:11/20/2023
Journal:Authorea Preprints
Publisher:Authorea
Description:Sap flow observations provide a basis for estimating transpiration and understanding forest water use dynamics and plant-climate interactions. This study developed a continental modeling approach using Long Short-Term Memory networks (LSTMs) to predict hourly tree-level sap flow across Europe based on the SAPFLUXNET database. We developed models with varying levels of training sets to evaluate performance in unseen conditions. The average Kling-Gupta Efficiency was 0.77 for gauged models trained on 50 % of time series across all forest stands and was 0.52 for ungauged models trained on 50 % of the forest stands. Continental models matched or exceeded performance of specialized and baseline models for all genera and forest stands, demonstrating the potential of LSTMs to generalize hourly sap flow across tree, climate, and forest types. This work highlights hence the potential of deep learning models to generalize sap flow for enhancing tree to continental ecohydrological investigations.
Cited by:1
Resources:[PDF] from essopenarchive.org
:view less
In defense of metrics: Metrics sufficiently encode typical human preferences regarding hydrological model performance
2023
Martin Gauch, Frederik Kratzert, Oren Gilon, Hoshin Gupta, Juliane Mai, Grey Nearing, B…
Water Resources Research 59 (6), e2022WR033918
:read more
Authors:Martin Gauch, Frederik Kratzert, Oren Gilon, Hoshin Gupta, Juliane Mai, Grey Nearing, Bryan Tolson, Sepp Hochreiter, Daniel Klotz
Publication date:06/01/2023
Journal:Water Resources Research
Volume:59
Issue:6
Pages:e2022WR033918
Description:Building accurate rainfall–runoff models is an integral part of hydrological science and practice. The variety of modeling goals and applications have led to a large suite of evaluation metrics for these models. Yet, hydrologists still put considerable trust into visual judgment, although it is unclear whether such judgment agrees or disagrees with existing quantitative metrics. In this study, we tasked 622 experts to compare and judge more than 14,000 pairs of hydrographs from 13 different models. Our results show that expert opinion broadly agrees with quantitative metrics and results in a clear preference for a Machine Learning model over traditional hydrological models. The expert opinions are, however, subject to significant amounts of inconsistency. Nevertheless, where experts agree, we can predict their opinion purely from quantitative metrics, which indicates that the metrics sufficiently encode human …
Cited by:29
Resources:[PDF] from wiley.com
:view less
From Hindcast to Forecast with Deep Learning Streamflow Models
2023
Grey Nearing, Martin Gauch, Daniel Klotz, Frederik Kratzert, Asher Metzger, Guy Shalev,…
EGU General Assembly Conference Abstracts, EGU-16974
:read more
Authors:Grey Nearing, Martin Gauch, Daniel Klotz, Frederik Kratzert, Asher Metzger, Guy Shalev, Shlomo Shenzis, Tadele Tekalign, Dana Weitzner, Oren Gilon
Publication date:05/01/2023
Journal:EGU General Assembly Conference Abstracts
Pages:EGU-16974
Description:Deep learning has become the de facto standard for streamflow simulation. While there are examples of deep learning based streamflow forecast models (eg, 1-5), the majority of the development and research has been done with hindcast models. The primary challenge in using deep learning models for forecasting (eg, flood forecasting) is that the meteorological input data are drawn from different distributions in hindcast vs. forecast. The (relatively small) amount of research that has been done on deep learning streamflow forecasting has largely used an encoder-decoder approach to account for forecast distribution shifts. This is, for example, what Google's operational flood forecasting model uses [4]. In this work we show that the encoder-decoder approach results in artifacts in forecast trajectories that are not detectable with standard hydrological metrics, but which can cause forecasts to have incorrect trends (eg …
Cited by:1
:view less
The persistence of errors: How evaluating models over data partitions relates to a global evaluation
2023
Daniel Klotz, Martin Gauch, Grey Nearing, Sepp Hochreiter, Frederik Kratzert
EGU General Assembly Conference Abstracts, EGU-15221
:read more
Authors:Daniel Klotz, Martin Gauch, Grey Nearing, Sepp Hochreiter, Frederik Kratzert
Publication date:05/01/2023
Journal:EGU General Assembly Conference Abstracts
Pages:EGU-15221
Description:Skillful today, inept tomorrow. Today's hydrological models have pronounced and complex error dynamics (eg, small, highly correlated errors for low flows and large, random errors for high flows). Modellers generally accept that simple, variance based evaluation criteria—like the Nash-Sutcliffe Efficiency (NSE)—are not fully able to capture these intricacies. The (implied) consequences of this are however seldom discussed. This contribution examines how evaluating the model over two data partitions (above and below a chosen threshold) relates to a global model evaluation of both partitions combined (ie, the usual way of computing the NSE). For our experiments we manipulate dummy simulations with gradient descent to approximate specific NSE values for each partition individually. Specifically, we set the NSE for runoff values that fall below the threshold, and vary the NSE of the simulations above the …
:view less
Peeking Inside Hydrologists’ Minds: Comparing Human Judgment and Quantitative Metrics of Hydrographs
2023
Martin Gauch, Frederik Kratzert, Oren Gilon, Hoshin Gupta, Juliane Mai, Grey Nearing, B…
EGU General Assembly Conference Abstracts, EGU-12261
:read more
Authors:Martin Gauch, Frederik Kratzert, Oren Gilon, Hoshin Gupta, Juliane Mai, Grey Nearing, Bryan Tolson, Sepp Hochreiter, Daniel Klotz
Publication date:05/01/2023
Journal:EGU General Assembly Conference Abstracts
Pages:EGU-12261
Description:Everyone wants their hydrologic models to be as good as possible. But how do we know if a model is accurate or not? In the spirit of rigorous and reproducible science, the answer should be: we calculate metrics. Yet, as humans, we sometimes follow a scheme of" I know a good model when I see it" and manually inspect hydrographs to assess their quality. This is certainly a valid method for sanity checks, but it is unclear whether these subjective visual ratings agree with metric-based rankings. Moreover, the consistency of such inspections is unclear, as different observers might come to different conclusions about the same hydrographs. In this presentation, we report a large-scale study where we collected responses from 622 experts, who compared and judged more than 14,000 pairs of hydrographs from 13 different models. Our results show that overall, human ratings broadly agree with quantitative metrics in a …
:view less
The Great Lakes Runoff Intercomparison Project (GRIP-GL)
2023
Juliane Mai, Hongren Shen, Bryan Tolson, Étienne Gaborit, Richard Arsenault, James Crai…
EGU General Assembly Conference Abstracts, EGU-968
:read more
Authors:Juliane Mai, Hongren Shen, Bryan Tolson, Étienne Gaborit, Richard Arsenault, James Craig, Vincent Fortin, Lauren Fry, Martin Gauch, Daniel Klotz, Frederik Kratzert, Nicole O'Brien, Daniel Princz, Sinan Rasiya Koya, Tirthankar Roy, Frank Seglenieks, Narayan Shretha, Andre Guy Temgoua, Vincent Vionnet, Jonathan Waddell
Publication date:05/01/2023
Journal:EGU General Assembly Conference Abstracts
Pages:EGU-968
Description:Model intercomparison studies are carried out to test and compare the simulated outputs of various model setups over the same study domain. The Great Lakes region is such a domain of high public interest as it not only resembles a challenging region to model with its trans-boundary location, strong lake effects, and regions of strong human impact but is also one of the most densely populated areas in the United States and Canada. This study brought together a wide range of researchers setting up their models of choice in a highly standardized experimental setup using the same geophysical datasets, forcings, common routing product, and locations of performance evaluation across the 1x106 km2 study domain. The study comprises 13 models covering a wide range of model types from Machine Learning based, basin-wise, subbasin-based, and gridded models that are either locally or globally calibrated or …
:view less
Towards flood warnings everywhere-data-driven rainfall-runoff modeling at global scale
2023
Frederik Kratzert, Martin Gauch, Daniel Klotz, Asher Metzger, Grey Nearing, Guy Shalev,…
EGU General Assembly Conference Abstracts, EGU-5326
:read more
Authors:Frederik Kratzert, Martin Gauch, Daniel Klotz, Asher Metzger, Grey Nearing, Guy Shalev, Shlomo Shenzis, Tadele Tekalign, Dana Weitzner, Oren Gilon
Publication date:05/01/2023
Journal:EGU General Assembly Conference Abstracts
Pages:EGU-5326
Description:The goal of Google's Flood Forecasting Initiative is to provide timely and actionable flood warnings to everyone, globally. Until recently, Google provided operational flood warnings only for specific partner countries, namely India, Bangladesh, Sri Lanka, Colombia, and Brazil. In 2021 our flood alerting system sent out around 115 million flood notifications, reaching over 23 million people in the affected local areas. In all of the regions mentioned above, our operational model relies on partnerships with local governments to provide real-time measurements of observed discharge or water level (Nevo et al. 2021). However, relying on real-time measurement data makes it harder to scale to new regions as a) this data does not exist everywhere, and b) even if it exists, it requires significant per-country time and resource investment. Building on research results from the last few years (eg, Kratzert et al. 2019a, Kratzert et al …
Cited by:3
:view less
Caravan-A global community dataset for large-sample hydrology
2023
Frederik Kratzert, Grey Nearing, Nans Addor, Tyler Erickson, Martin Gauch, Oren Gilon, …
Scientific Data 10 (1), 61
:read more
Authors:Frederik Kratzert, Grey Nearing, Nans Addor, Tyler Erickson, Martin Gauch, Oren Gilon, Lukas Gudmundsson, Avinatan Hassidim, Daniel Klotz, Sella Nevo, Guy Shalev, Yossi Matias
Publication date:01/31/2023
Journal:Scientific Data
Volume:10
Issue:1
Pages:61
Publisher:Nature Publishing Group UK
Description:High-quality datasets are essential to support hydrological science and modeling. Several CAMELS (Catchment Attributes and Meteorology for Large-sample Studies) datasets exist for specific countries or regions, however these datasets lack standardization, which makes global studies difficult. This paper introduces a dataset called Caravan (a series of CAMELS) that standardizes and aggregates seven existing large-sample hydrology datasets. Caravan includes meteorological forcing data, streamflow data, and static catchment attributes (e.g., geophysical, sociological, climatological) for 6830 catchments. Most importantly, Caravan is both a dataset and open-source software that allows members of the hydrology community to extend the dataset to new locations by extracting forcing data and catchment attributes in the cloud. Our vision is for Caravan to democratize the creation and use of globally-standardized …
Cited by:131
Resources:[HTML] from nature.com
Full View
:view less
Few-shot learning by dimensionality reduction in gradient space
2022
Martin Gauch, Maximilian Beck, Thomas Adler, Dmytro Kotsur, Stefan Fiel, Hamid Eghbal-z…
, 1043-1064
:read more
Authors:Martin Gauch, Maximilian Beck, Thomas Adler, Dmytro Kotsur, Stefan Fiel, Hamid Eghbal-zadeh, Johannes Brandstetter, Johannes Kofler, Markus Holzleitner, Werner Zellinger, Daniel Klotz, Sepp Hochreiter, Sebastian Lehner
Publication date:11/28/2022
Pages:1043-1064
Publisher:PMLR
Description:We introduce SubGD, a novel few-shot learning method which is based on the recent finding that stochastic gradient descent updates tend to live in a low-dimensional parameter subspace. In experimental and theoretical analyses, we show that models confined to a suitable predefined subspace generalize well for few-shot learning. A suitable subspace fulfills three criteria across the given tasks: it (a) allows to reduce the training error by gradient flow,(b) leads to models that generalize well, and (c) can be identified by stochastic gradient descent. SubGD identifies these subspaces from an eigendecomposition of the auto-correlation matrix of update directions across different tasks. Demonstrably, we can identify low-dimensional suitable subspaces for few-shot learning of dynamical systems, which have varying properties described by one or few parameters of the analytical system description. Such systems are ubiquitous among real-world applications in science and engineering. We experimentally corroborate the advantages of SubGD on three distinct dynamical systems problem settings, significantly outperforming popular few-shot learning methods both in terms of sample efficiency and performance.
Cited by:8
Resources:[PDF] from mlr.press
:view less
The great lakes runoff intercomparison project phase 4: the great lakes (GRIP-GL)
2022
Juliane Mai, Hongren Shen, Bryan A Tolson, Étienne Gaborit, Richard Arsenault, James R …
Hydrology and Earth System Sciences 26 (13), 3537-3572
:read more
Authors:Juliane Mai, Hongren Shen, Bryan A Tolson, Étienne Gaborit, Richard Arsenault, James R Craig, Vincent Fortin, Lauren M Fry, Martin Gauch, Daniel Klotz, Frederik Kratzert, Nicole O'Brien, Daniel G Princz, Sinan Rasiya Koya, Tirthankar Roy, Frank Seglenieks, Narayan K Shrestha, André GT Temgoua, Vincent Vionnet, Jonathan W Waddell
Publication date:07/08/2022
Journal:Hydrology and Earth System Sciences
Volume:26
Issue:13
Pages:3537-3572
Publisher:Copernicus Publications
Description:Model intercomparison studies are carried out to test and compare the simulated outputs of various model setups over the same study domain. The Great Lakes region is such a domain of high public interest as it not only resembles a challenging region to model with its transboundary location, strong lake effects, and regions of strong human impact but is also one of the most densely populated areas in the USA and Canada. This study brought together a wide range of researchers setting up their models of choice in a highly standardized experimental setup using the same geophysical datasets, forcings, common routing product, and locations of performance evaluation across the 1 km study domain. The study comprises 13 models covering a wide range of model types from machine-learning-based, basin-wise, subbasin-based, and gridded models that are either locally or globally calibrated or calibrated for one of each of the six predefined regions of the watershed. Unlike most hydrologically focused model intercomparisons, this study not only compares models regarding their capability to simulate streamflow () but also evaluates the quality of simulated actual evapotranspiration (AET), surface soil moisture (SSM), and snow water equivalent (SWE). The latter three outputs are compared against gridded reference datasets. The comparisons are performed in two ways – either by aggregating model outputs and the reference to basin level or by regridding all model outputs to the reference grid and comparing the model simulations at each grid-cell. The main results of this study are as follows: The comparison of models … The comparison of models …
Cited by:80
Resources:[HTML] from copernicus.org
:view less
Deep learning rainfall–runoff predictions of extreme events
2022
Jonathan M Frame, Frederik Kratzert, Daniel Klotz, Martin Gauch, Guy Shalev, Oren Gilon…
Hydrology and Earth System Sciences 26 (13), 3377-3392
:read more
Authors:Jonathan M Frame, Frederik Kratzert, Daniel Klotz, Martin Gauch, Guy Shalev, Oren Gilon, Logan M Qualls, Hoshin V Gupta, Grey S Nearing
Publication date:07/05/2022
Journal:Hydrology and Earth System Sciences
Volume:26
Issue:13
Pages:3377-3392
Publisher:Copernicus Publications
Description:The most accurate rainfall–runoff predictions are currently based on deep learning. There is a concern among hydrologists that the predictive accuracy of data-driven models based on deep learning may not be reliable in extrapolation or for predicting extreme events. This study tests that hypothesis using long short-term memory (LSTM) networks and an LSTM variant that is architecturally constrained to conserve mass. The LSTM network (and the mass-conserving LSTM variant) remained relatively accurate in predicting extreme (high-return-period) events compared with both a conceptual model (the Sacramento Model) and a process-based model (the US National Water Model), even when extreme events were not included in the training period. Adding mass balance constraints to the data-driven model (LSTM) reduced model skill during extreme events.
Cited by:204
Resources:[HTML] from copernicus.org
:view less
Deficiencies in Hydrological Modelling Practices
2022
Daniel Klotz, Martin Gauch, Grey Nearing, Sepp Hochreiter, Frederik Kratzert
EGU General Assembly Conference Abstracts, EGU22-12403
:read more
Authors:Daniel Klotz, Martin Gauch, Grey Nearing, Sepp Hochreiter, Frederik Kratzert
Publication date:05/01/2022
Journal:EGU General Assembly Conference Abstracts
Pages:EGU22-12403
Description:The goal of this contribution is to demonstrate deficiencies that we observe in hydrological modelling studies. Our hope is that awareness of potential mistakes, errors, and habits will support accurate communication and analysis—and consequently lead to better modelling practises in our community. By deficiencies, we broadly mean wrong assumptions, false conclusions, and artificial limitations that impair our modelling efforts. To give some explicit examples: Model calibration: Often, only two data splits are used: one for model calibration and one for model validation. To provide a robust estimate of model quality on unseen data, one should, however, involve a three-way split: a calibration set used for parameter adaptation, a validation set used for hyperparameter tuning and intermediate model evaluations, and a test set used only once to derive the final, independent model accuracy. Artificial restrictions: Studies …
:view less
Rate my hydrograph: Evaluating the conformity of expert judgment and quantitative metrics
2022
Martin Gauch, Frederik Kratzert, Juliane Mai, Bryan Tolson, Grey Nearing, Hoshin Gupta,…
EGU General Assembly Conference Abstracts, EGU22-8396
:read more
Authors:Martin Gauch, Frederik Kratzert, Juliane Mai, Bryan Tolson, Grey Nearing, Hoshin Gupta, Sepp Hochreiter, Daniel Klotz
Publication date:05/01/2022
Journal:EGU General Assembly Conference Abstracts
Pages:EGU22-8396
Cited by:3
:view less
Uncertainty estimation with deep learning for rainfall–runoff modeling
2022
Daniel Klotz, Frederik Kratzert, Martin Gauch, Alden Keefe Sampson, Johannes Brandstett…
Hydrology and Earth System Sciences 26 (6), 1673-1693
:read more
Authors:Daniel Klotz, Frederik Kratzert, Martin Gauch, Alden Keefe Sampson, Johannes Brandstetter, Günter Klambauer, Sepp Hochreiter, Grey Nearing
Publication date:03/31/2022
Journal:Hydrology and Earth System Sciences
Volume:26
Issue:6
Pages:1673-1693
Publisher:Copernicus Publications
Description:Deep learning is becoming an increasingly important way to produce accurate hydrological predictions across a wide range of spatial and temporal scales. Uncertainty estimations are critical for actionable hydrological prediction, and while standardized community benchmarks are becoming an increasingly important part of hydrological model development and research, similar tools for benchmarking uncertainty estimation are lacking. This contribution demonstrates that accurate uncertainty predictions can be obtained with deep learning. We establish an uncertainty estimation benchmarking procedure and present four deep learning baselines. Three baselines are based on mixture density networks, and one is based on Monte Carlo dropout. The results indicate that these approaches constitute strong baselines, especially the former ones. Additionally, we provide a post hoc model analysis to put forward some qualitative understanding of the resulting models. The analysis extends the notion of performance and shows that the model learns nuanced behaviors to account for different situations.
Cited by:142
Resources:[HTML] from copernicus.org
:view less
NeuralHydrology—A Python library for Deep Learning research in hydrology
2022
Frederik Kratzert, Martin Gauch, Grey Nearing, Daniel Klotz
Journal of Open Source Software 7 (71), 4050
:read more
Authors:Frederik Kratzert, Martin Gauch, Grey Nearing, Daniel Klotz
Publication date:03/04/2022
Journal:Journal of Open Source Software
Volume:7
Issue:71
Pages:4050
Description:Since ancient times humans have strived to describe environmental processes related to water (Angelakis et al., 2012; Biswas, 1970). Throughout this history, hydrologists built various process-based prediction models that simulate processes from soil moisture to streamflow generation (a collection of historical references can be found in Loague (2010)). More recently, Deep Learning models have emerged as extremely powerful and more accurate alternatives to these traditional modeling approaches (Gauch et al., 2021; Klotz et al., 2021; Kratzert, Klotz, Shalev, et al., 2019; Kratzert, Klotz, Herrnegger, et al., 2019; Lees et al., 2021). For hydrologists, embracing this new data-driven paradigm is challenging (Beven, 2020; Nearing et al., 2021): not only are the models conceptually different, but they are also built, optimized, and evaluated with different strategies and toolsets. NeuralHydrology is a Python library based on PyTorch (Paszke et al., 2019) that is designed to build, apply, and experiment with Deep Learning models with a strong focus on hydrological applications. Originally designed for our internal research needs, the library was generalized and open-sourced to allow anyone to experiment with Deep Learning models as easily as possible: pre-built models and data loaders allow for quick experiments, yet the framework is also easily extensible to new models, data sets, loss functions, or metrics to suit more advanced use-cases.
Cited by:76
Resources:[PDF] from theoj.org
:view less
Forward vs. Inverse Methods for Using Near-Real-Time Streamflow Observation Data in Long Short-Term Memory Networks
2021
Grey Nearing, Frederik Kratzert, Daniel Klotz, Martin Gauch, Jonathan Frame, Sella Nevo
AGU Fall Meeting Abstracts 2021, H31H-06
:read more
Authors:Grey Nearing, Frederik Kratzert, Daniel Klotz, Martin Gauch, Jonathan Frame, Sella Nevo
Publication date:12/01/2021
Journal:AGU Fall Meeting Abstracts
Volume:2021
Pages:H31H-06
Description:Ingesting near-real-time observation data is a critical component of many operational hydrological forecasting systems. There are generally two classes of strategies for ingesting streamflow observations into rainfall-runoff models: autoregression (AR) is a core component of many statistical hydrology models, and data assimilation (DA) is used in conceptual and process-based models. Long Short Term Memory networks (LSTMs) are currently the most accurate and extrapolatable streamflow models available from the hydrological science community. LSTMs are frequently used for autoregression, and like dynamical systems models, LSTMs have an explicit state space, which means that they can be used with DA. AR is more straightforward to implement than DA (which necessarily requires some type of inverse method), however DA has the advantage of being less sensitive to missing data. Missing data is a …
:view less
Hydrological concept formation inside long short-term memory (LSTM) networks
2021
Thomas Lees, Steven Reece, Frederik Kratzert, Daniel Klotz, Martin Gauch, Jens De Bruij…
Hydrology and Earth System Sciences Discussions 2021, 1-37
:read more
Authors:Thomas Lees, Steven Reece, Frederik Kratzert, Daniel Klotz, Martin Gauch, Jens De Bruijn, Reetik Kumar Sahu, Peter Greve, Louise Slater, Simon Dadson
Publication date:11/23/2021
Journal:Hydrology and Earth System Sciences Discussions
Volume:2021
Pages:1-37
Description:Neural networks have been shown to be extremely effective rainfall-runoff models, where the river discharge is predicted from meteorological inputs. However, the question remains, what have these models learned? Is it possible to extract information about the learned relationships that map inputs to outputs? And do these mappings represent known hydrological concepts? Small-scale experiments have demonstrated that the internal states of Long Short-Term Memory Networks (LSTMs), a particular neural network architecture predisposed to hydrological modelling, can be interpreted. By extracting the tensors which represent the learned translation from inputs (precipitation, temperature) to outputs (discharge), this research seeks to understand what information the LSTM captures about the hydrological system. We assess the hypothesis that the LSTM replicates real-world processes and that we can extract information about these processes from the internal states of the LSTM. We examine the cell-state vector, which represents the memory of the LSTM, and explore the ways in which the LSTM learns to reproduce stores of water, such as soil moisture and snow cover. We use a simple regression approach to map the LSTM state-vector to our target stores (soil moisture and snow). Good correlations (R2 > 0.8) between the probe outputs and the target variables of interest provide evidence that the LSTM contains information that reflects known hydrological processes comparable with the concept of variable-capacity soil moisture stores. The implications of this study are threefold: 1) LSTMs reproduce known hydrological processes. 2) While … Neural networks have been shown to be extremely effective rainfall-runoff models, where the river discharge is predicted from meteorological inputs. However, the question remains, what have these models learned? Is it possible to extract information about the learned relationships that map inputs to outputs? And do these mappings represent known hydrological concepts? Small-scale experiments have demonstrated that the internal states of Long Short-Term Memory Networks (LSTMs), a particular neural network architecture predisposed to hydrological modelling, can be interpreted. By extracting the tensors which represent the learned translation from inputs (precipitation, temperature) to outputs (discharge), this research seeks to understand what information the LSTM captures about the hydrological system. We assess the hypothesis that the LSTM replicates real-world processes and that we can extract information about these processes from the internal states of the LSTM. We examine the cell-state vector, which represents the memory of the LSTM, and explore the ways in which the LSTM learns to reproduce stores of water, such as soil moisture and snow cover. We use a simple regression approach to map the LSTM state-vector to our target stores (soil moisture and snow). Good correlations (R2 > 0.8) between the probe outputs and the target variables of interest provide evidence that the LSTM contains information that reflects known hydrological processes comparable with the concept of variable-capacity soil moisture stores. The implications of this study are threefold: 1) LSTMs reproduce known hydrological processes. 2) While …
Cited by:114
Resources:[HTML] from copernicus.org
:view less
Data assimilation and autoregression for using near-real-time streamflow observations in long short-term memory networks
2021
Grey S Nearing, Daniel Klotz, Alden Keefe Sampson, Frederik Kratzert, Martin Gauch, Jon…
Hydrology and earth system sciences discussions 2021, 1-25
:read more
Authors:Grey S Nearing, Daniel Klotz, Alden Keefe Sampson, Frederik Kratzert, Martin Gauch, Jonathan M Frame, Guy Shalev, Sella Nevo
Publication date:10/25/2021
Journal:Hydrology and earth system sciences discussions
Volume:2021
Pages:1-25
Description:Ingesting near-real-time observation data is a critical component of many operational hydrological forecasting systems. In this paper we compare two strategies for ingesting near-real-time streamflow observations into Long Short-Term Memory (LSTM) rainfall-runoff models: autoregression (a forward method) and variational data assimilation. Autoregression is both more accurate and more computationally efficient than data assimilation. Autoregression is sensitive to missing data, however an appropriate (and simple) training strategy mitigates this problem.
Cited by:43
Resources:[HTML] from copernicus.org
:view less
Erratum zu: Regionalisierung hydrologischer Modelle mit Function Space Optimization
2021
Moritz Feigl, Mathew Herrnegger, Robert Schweppe, Stephan Thober, Daniel Klotz, Luis Sa…
Österreichische Wasser-und Abfallwirtschaft 73 (7-8), 351-351
:read more
Authors:Moritz Feigl, Mathew Herrnegger, Robert Schweppe, Stephan Thober, Daniel Klotz, Luis Samaniego, Karsten Schulz
Publication date:08/01/2021
Journal:Österreichische Wasser-und Abfallwirtschaft
Volume:73
Issue:7-8
Pages:351-351
Publisher:Springer Vienna
Description:Erratum zu: Regionalisierung hydrologischer Modelle mit Function Space Optimization Page 1 Erratum Österr Wasser- und Abfallw https://doi.org/10.1007/s00506-021-00782-0 Erratum zu: Regionalisierung hydrologischer Modelle mit Function Space Optimization Moritz Feigl · Mathew Herrnegger · Robert Schweppe · Stephan Thober · Daniel Klotz · Luis Samaniego · Karsten Schulz © Österreichischer Wasser- und Abfallwirtschaftsverband (ÖWAV) and Springer-Verlag GmbH Austria, ein Teil von Springer Nature 2021 Erratum zu: Österr Wasser- und Abfallw 2021 https://doi.org/10.1007/s00506-02100766-0 Im Originalbeitrag wurde ein Co-Autor vergessen: Dr. Luis Samaniego, Department Hydrosystemmodellierung, UFZ-Helmholtz Zentrum für Umweltforschung, Permoserstraße 15, 04318, Leipzig, Deutschland. Der Originalbeitrag wurde korrigiert. Die Online-Version des Originalartikels ist unter https://doi. org/…
:view less
Regionalisierung hydrologischer Modelle mit Function Space Optimization
2021
Moritz Feigl, Mathew Herrnegger, Robert Schweppe, Stephan Thober, Daniel Klotz, Luis Sa…
Österreichische Wasser-und Abfallwirtschaft 73 (7), 281-294
:read more
Authors:Moritz Feigl, Mathew Herrnegger, Robert Schweppe, Stephan Thober, Daniel Klotz, Luis Samaniego, Karsten Schulz
Publication date:08/01/2021
Journal:Österreichische Wasser-und Abfallwirtschaft
Volume:73
Issue:7
Pages:281-294
Publisher:Springer Vienna
Description:Das Schätzen von räumlich verteilten Parametern hydrologischer Modelle ist ein bereits lang erforschtes und anspruchsvolles Problem. Parameter-Transferfunktionen, die einen funktionellen Zusammenhang zwischen Modellparametern und geophysikalischen Gebietseigenschaften herstellen, sind eine potenzielle Möglichkeit, Parameter ohne Kalibrierung zu schätzen. Function Space Optimization (FSO) ist eine symbolische Regressionsmethode, die automatisiert Transferfunktionen aus Daten schätzen kann. Sie basiert auf einem textgenerierenden neuronalen Netzwerk, das die Suche nach einer optimalen Funktion in ein kontinuierliches Optimierungsproblem umwandelt. In diesem Beitrag beschreiben wir die Funktionsweise von FSO und geben ein Beispiel der Anwendung mit dem mesoscale Hydrological Model (mHM). Ziel der Anwendung ist die Schätzung zweier Transferfunktionen für die Parameter KSat …
Cited by:1
Resources:[HTML] from springer.com
:view less
Niederschlags-Abfluss-Modellierung mit Long Short-Term Memory (LSTM)
2021
Frederik Kratzert, Martin Gauch, Grey Nearing, Sepp Hochreiter, Daniel Klotz
Österreichische Wasser-und Abfallwirtschaft 73 (7), 270-280
:read more
Authors:Frederik Kratzert, Martin Gauch, Grey Nearing, Sepp Hochreiter, Daniel Klotz
Publication date:08/01/2021
Journal:Österreichische Wasser-und Abfallwirtschaft
Volume:73
Issue:7
Pages:270-280
Publisher:Springer Vienna
Description:Methoden der künstlichen Intelligenz haben sich in den letzten Jahren zu essenziellen Bestandteilen fast aller Bereiche von Wissenschaft und Technik entwickelt. Dies gilt auch für die Hydrologie: Vielschichtige neuronale Netzwerke – auch bekannt als Modelle des Deep Learning – ermöglichen hier Vorhersagen von Niederschlagsabflussmengen in zuvor unerreichter Präzision. Dieser Beitrag beleuchtet das Potenzial von Deep Learning für wasserwirtschaftliche Anwendungen. Der erste Teil des Artikels zeigt, wie sogenannte Long Short-Term Memory-Netzwerke – eine spezifisch für Zeitreihen entwickelte Methode des Deep Learnings – für die Niederschlags-Abfluss-Modellierung verwendet werden, und wie diese für eine Reihe hydrologischer Probleme bessere Ergebnisse als jedes andere bekannte hydrologische Modell erzielen. Der zweite Teil demonstriert wesentliche Eigenschaften der Long Short-Term …
Cited by:10
Resources:[HTML] from springer.com
:view less
Mc-lstm: Mass-conserving lstm
2021
Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz, Christina Halmich, Markus Holzleitne…
, 4275-4286
:read more
Authors:Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz, Christina Halmich, Markus Holzleitner, Grey S Nearing, Sepp Hochreiter, Günter Klambauer
Publication date:07/01/2021
Pages:4275-4286
Publisher:PMLR
Description:The success of Convolutional Neural Networks (CNNs) in computer vision is mainly driven by their strong inductive bias, which is strong enough to allow CNNs to solve vision-related tasks with random weights, meaning without learning. Similarly, Long Short-Term Memory (LSTM) has a strong inductive bias towards storing information over time. However, many real-world systems are governed by conservation laws, which lead to the redistribution of particular quantities {—} eg in physical and economical systems. Our novel Mass-Conserving LSTM (MC-LSTM) adheres to these conservation laws by extending the inductive bias of LSTM to model the redistribution of those stored quantities. MC-LSTMs set a new state-of-the-art for neural arithmetic units at learning arithmetic operations, such as addition tasks, which have a strong conservation law, as the sum is constant over time. Further, MC-LSTM is applied to traffic forecasting, modeling a pendulum, and a large benchmark dataset in hydrology, where it sets a new state-of-the-art for predicting peak flows. In the hydrology example, we show that MC-LSTM states correlate with real world processes and are therefore interpretable.
Cited by:103
Resources:[PDF] from mlr.press
:view less
A note on leveraging synergy in multiple meteorological data sets with deep learning for rainfall–runoff modeling
2021
Frederik Kratzert, Daniel Klotz, Sepp Hochreiter, Grey S Nearing
Hydrology and Earth System Sciences 25 (5), 2685-2703
:read more
Authors:Frederik Kratzert, Daniel Klotz, Sepp Hochreiter, Grey S Nearing
Publication date:05/20/2021
Journal:Hydrology and Earth System Sciences
Volume:25
Issue:5
Pages:2685-2703
Publisher:Copernicus GmbH
Description: A deep learning rainfall–runoff model can take multiple meteorological forcing products as input and learn to combine them in spatially and temporally dynamic ways. This is demonstrated with Long Short-Term Memory networks (LSTMs) trained over basins in the continental US, using the Catchment Attributes and Meteorological data set for Large Sample Studies (CAMELS). Using meteorological input from different data products (North American Land Data Assimilation System, NLDAS, Maurer, and Daymet) in a single LSTM significantly improved simulation accuracy relative to using only individual meteorological products. A sensitivity analysis showed that the LSTM combines precipitation products in different ways, depending on location, and also in different ways for the simulation of different parts of the hydrograph.
Cited by:102
Resources:[HTML] from copernicus.org
:view less
Rainfall–runoff prediction at multiple timescales with a single Long Short-Term Memory network
2021
Martin Gauch, Frederik Kratzert, Daniel Klotz, Grey Nearing, Jimmy Lin, Sepp Hochreiter
Hydrology and Earth System Sciences 25 (4), 2045-2062
:read more
Authors:Martin Gauch, Frederik Kratzert, Daniel Klotz, Grey Nearing, Jimmy Lin, Sepp Hochreiter
Publication date:04/19/2021
Journal:Hydrology and Earth System Sciences
Volume:25
Issue:4
Pages:2045-2062
Publisher:Copernicus Publications
Description:Long Short-Term Memory (LSTM) networks have been applied to daily discharge prediction with remarkable success. Many practical applications, however, require predictions at more granular timescales. For instance, accurate prediction of short but extreme flood peaks can make a lifesaving difference, yet such peaks may escape the coarse temporal resolution of daily predictions. Naively training an LSTM on hourly data, however, entails very long input sequences that make learning difficult and computationally expensive. In this study, we propose two multi-timescale LSTM (MTS-LSTM) architectures that jointly predict multiple timescales within one model, as they process long-past inputs at a different temporal resolution than more recent inputs. In a benchmark on 516 basins across the continental United States, these models achieved significantly higher Nash–Sutcliffe efficiency (NSE) values than the US National Water Model. Compared to naive prediction with distinct LSTMs per timescale, the multi-timescale architectures are computationally more efficient with no loss in accuracy. Beyond prediction quality, the multi-timescale LSTM can process different input variables at different timescales, which is especially relevant to operational applications where the lead time of meteorological forcings depends on their temporal resolution.
Cited by:226
Resources:[HTML] from copernicus.org
:view less
Knowledge-guided machine learning (kgml) platform to predict integrated water cycle and associated extremes
2021
Dipankar Dwivedi, Grey Nearing, Hoshin Gupta, Alden Keefe Sampson, Laura Condon, Benjam…
(AI4ESP-1037)
:read more
Authors:Dipankar Dwivedi, Grey Nearing, Hoshin Gupta, Alden Keefe Sampson, Laura Condon, Benjamin Ruddell, Daniel Klotz, Uwe Ehret, Laura Read, Praveen Kumar, Ty Ferre, Carl Steefel
Publication date:04/15/2021
Issue:AI4ESP-1037
Publisher:Artificial Intelligence for Earth System Predictability (AI4ESP) Collaboration (United States)
Description:Focal Area(s): Predictive modeling through the use of AI techniques and insight gleaned from complex data (both observed and simulated). Science Challenge: Although advanced predictive capabilities of the water cycle are critical to address environmental needs and develop sustainable solutions for energy demands, there is no robust framework, to say the least, that seamlessly integrates local to intermediate to global scales and a gamut of biogeophysical information to enhance understanding of the integrated water cycle and its associated extremes.
Cited by:2
Resources:[PDF] from osti.gov
:view less
Large-scale river network modeling using graph neural networks
2021
Frederik Kratzert, Daniel Klotz, Martin Gauch, Christoph Klingler, Grey Nearing, Sepp H…
EGU General Assembly Conference Abstracts, EGU21-13375
:read more
Authors:Frederik Kratzert, Daniel Klotz, Martin Gauch, Christoph Klingler, Grey Nearing, Sepp Hochreiter
Publication date:04/01/2021
Journal:EGU General Assembly Conference Abstracts
Pages:EGU21-13375
Description:In the recent past, several studies have demonstrated the ability of deep learning (DL) models, especially based on Long Short-Term Memory (LSTM) networks, for rainfall-runoff modeling. However, almost all of these studies were limited to (multiple) individual catchments or small river networks, consisting of only a few connected catchments. In this study, we investigate large-scale, spatially distributed rainfall-runoff modeling using DL models. Our setup consists of two independent model components: One model for the runoff-generation process and one for the routing. The former is an LSTM-based model that predicts the discharge contribution of each sub-catchment in a river network. The latter is a Graph Neural Network (GNN) that routes the water along the river network network in hierarchical order. The first part is set up to simulate unimpaired runoff for every sub-catchment. Then, the GNN routes the water …
Cited by:8
:view less
Multi-Timescale LSTM for Rainfall-Runoff Forecasting
2021
Martin Gauch, Frederik Kratzert, Grey Nearing, Jimmy Lin, Sepp Hochreiter, Johannes Bra…
EGU General Assembly Conference Abstracts, EGU21-9714
:read more
Authors:Martin Gauch, Frederik Kratzert, Grey Nearing, Jimmy Lin, Sepp Hochreiter, Johannes Brandstetter, Daniel Klotz
Publication date:04/01/2021
Journal:EGU General Assembly Conference Abstracts
Pages:EGU21-9714
Description:Rainfall-runoff predictions are generally evaluated on reanalysis datasets such as the DayMet, Maurer, or NLDAS forcings in the CAMELS dataset. While useful for benchmarking, this does not fully reflect real-world applications. There, meteorological information is much coarser, and fine-grained predictions are at best available until the present. For any prediction of future discharge, we must rely on forecasts, which introduce an additional layer of uncertainty. Thus, the model inputs need to switch from past data to forecast data at some point, which raises several questions: How can we design models that support this transition? How can we design tests that evaluate the performance of the model? Aggravating the challenge, the past and future data products may include different variables or have different temporal resolutions. We demonstrate how to seamlessly integrate past and future meteorological data in one …
:view less
Uncertainty estimation with LSTM based rainfall-runoff models
2021
Daniel Klotz, Frederik Kratzert, Martin Gauch, Alden K Sampson, Günter Klambauer, Johan…
EGU General Assembly Conference Abstracts, EGU21-13308
:read more
Authors:Daniel Klotz, Frederik Kratzert, Martin Gauch, Alden K Sampson, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter, Grey Nearing
Publication date:04/01/2021
Journal:EGU General Assembly Conference Abstracts
Pages:EGU21-13308
Description:Uncertainty is a central part of hydrological inquiry. Deep Learning provides us with new tools for estimating these inherent uncertainties. The currently best performing rainfall-runoff models are based on Long Short-Term Memory (LSTM) networks. However, most LSTM-based modelling studies focus on point estimates. Building on the success of LSTMs for estimating point predictions, this contribution explores different extensions to directly provide uncertainty estimations. We find that the resulting models provide excellent estimates in our benchmark for daily rainfall-runoff across hundreds basins. We provide an intuitive overview of these strong results, the benchmarking procedure, and the approaches used for obtaining them. In short, we extend the LSTMs in two ways to obtain uncertainty estimations. First, we parametrize LSTMs so that they directly provide uncertainty estimates in the form of mixture densities …
Cited by:1
:view less
What role does hydrological science play in the age of machine learning?
2021
Grey S Nearing, Frederik Kratzert, Alden Keefe Sampson, Craig S Pelissier, Daniel Klotz…
Water Resources Research 57 (3), e2020WR028091
:read more
Authors:Grey S Nearing, Frederik Kratzert, Alden Keefe Sampson, Craig S Pelissier, Daniel Klotz, Jonathan M Frame, Cristina Prieto, Hoshin V Gupta
Publication date:03/01/2021
Journal:Water Resources Research
Volume:57
Issue:3
Pages:e2020WR028091
Description:This paper is derived from a keynote talk given at the Google's 2020 Flood Forecasting Meets Machine Learning Workshop. Recent experiments applying deep learning to rainfall‐runoff simulation indicate that there is significantly more information in large‐scale hydrological data sets than hydrologists have been able to translate into theory or models. While there is a growing interest in machine learning in the hydrological sciences community, in many ways, our community still holds deeply subjective and nonevidence‐based preferences for models based on a certain type of “process understanding” that has historically not translated into accurate theory, models, or predictions. This commentary is a call to action for the hydrology community to focus on developing a quantitative understanding of where and when hydrological process understanding is valuable in a modeling discipline increasingly dominated by …
Cited by:512
Resources:[PDF] from wiley.com
Full View
:view less
Hydrological Modeling with Deep Learning (Invited)(Invited Presentation)
2021
Grey Stephen Nearing, Frederik Kratzert, Daniel Klotz, Alden Keefe Sampson, Craig Pelis…
101st American Meteorological Society Annual Meeting
:read more
Authors:Grey Stephen Nearing, Frederik Kratzert, Daniel Klotz, Alden Keefe Sampson, Craig Pelissier, Jonathan Frame
Publication date:01/15/2021
Journal:101st American Meteorological Society Annual Meeting
Publisher:AMS
:view less
What is the role of hydrological science in the age of machine learning?
2020
Grey Stephen Nearing, Frederik Kratzert, Alden Keefe Sampson, Craig Pelissier, Daniel K…
AGU Fall Meeting Abstracts 2020, H188-01
:read more
Authors:Grey Stephen Nearing, Frederik Kratzert, Alden Keefe Sampson, Craig Pelissier, Daniel Klotz, Cristina Prieto, Jonathan M Frame, Hoshin Gupta
Publication date:12/01/2020
Journal:AGU Fall Meeting Abstracts
Volume:2020
Pages:H188-01
Description:Recent experiments applying deep learning to rainfall-runoff simulation indicate that there is significantly more information in large-scale hydrological data sets than hydrologists have been able to translate into theory or models. We argue that these results challenge certainsacred cows' in the surface hydrology community, and may be a bellwether for the discipline as a whole. While there is growing interest in machine learning in the hydrological sciences community, in many ways our community still holds deeply subjective and non-evidence-based preferences for process understanding that has historically not translated into accurate theory, models, or predictions. We suggest that, due to the perennial failure in the surface hydrology community to develop scale-relevant theories, one possible future is a discipline based primarily in machine learning and other AI methods, with a more limited role for what we …
:view less
Examining the uncertainty estimation properties of LSTM based rainfall-runoff models
2020
Daniel Klotz, Frederik Kratzert, Martin Gauch, Alden Keefe Sampson, Sepp Hochreiter, Gr…
AGU Fall Meeting Abstracts 2020, H219-05
:read more
Authors:Daniel Klotz, Frederik Kratzert, Martin Gauch, Alden Keefe Sampson, Sepp Hochreiter, Grey Stephen Nearing
Publication date:12/01/2020
Journal:AGU Fall Meeting Abstracts
Volume:2020
Pages:H219-05
Description:Existing studies on Long-Short Term Memory (LSTM) based rainfall-runoff modelling predominantly focus on the performance of point estimates (eg Kratzert et al. 2019). These estimates are by their very nature imprecise, given the uncertainty in the available information, the forcing inputs and the streamflow observations used to train the model.
:view less
LSTM-Based Rainfall-Runoff Modeling at Arbitrary Time Scales
2020
Martin Gauch, Frederik Kratzert, Daniel Klotz, Grey Stephen Nearing, Jimmy Lin
AGU Fall Meeting Abstracts 2020, H188-03
:read more
Authors:Martin Gauch, Frederik Kratzert, Daniel Klotz, Grey Stephen Nearing, Jimmy Lin
Publication date:12/01/2020
Journal:AGU Fall Meeting Abstracts
Volume:2020
Pages:H188-03
Description:In recent years, rainfall-runoff models based on machine learning techniques, particularly Long Short-Term Memory (LSTM) networks, have proven highly successful. They outperform conceptual hydrologic models, predict multiple basins in a single model, and allow for predictions in ungauged basins (Kratzert et al. 2018, Kratzert et al. 2019, Kratzert et al. 2019a). Yet, there remain open challenges toward operational use of such models.
:view less
A deep learning architecture for conservative dynamical systems: Application to rainfall-runoff modeling
2020
Grey Nearing, Frederik Kratzert, Daniel Klotz, Pieter-Jan Hoedt, Günter Klambauer, Sepp…
AI for Earth Sciences Workshop at NeurIPS
:read more
Authors:Grey Nearing, Frederik Kratzert, Daniel Klotz, Pieter-Jan Hoedt, Günter Klambauer, Sepp Hochreiter, Hoshin Gupta, Sella Nevo, Yossi Matias
Publication date:12/01/2020
Journal:AI for Earth Sciences Workshop at NeurIPS
Description:The most accurate and generalizable rainfall-runoff models produced by the hydrological sciences community to-date are based on deep learning, and in particular, on Long Short Term Memory networks (LSTMs). Although LSTMs have an explicit state space and gates that mimic input-state-output relationships, these models are not based on physical principles. We propose a deep learning architecture that is based on the LSTM and obeys conservation principles. The model is benchmarked on the mass-conservation problem of simulating streamflow.
Cited by:15
Resources:[PDF] from github.io
:view less
Function space optimization: A symbolic regression method for estimating parameter transfer functions for hydrological models
2020
Moritz Feigl, Mathew Herrnegger, Daniel Klotz, Karsten Schulz
Water resources research 56 (10), e2020WR027385
:read more
Authors:Moritz Feigl, Mathew Herrnegger, Daniel Klotz, Karsten Schulz
Publication date:10/01/2020
Journal:Water resources research
Volume:56
Issue:10
Pages:e2020WR027385
Cited by:30
:view less
The runoff model-intercomparison project over lake erie and the great lakes
2020
Bryan A Tolson, Juliane Mai
EGU General Assembly Conference Abstracts, 10319
:read more
Authors:Bryan A Tolson, Juliane Mai
Publication date:05/01/2020
Journal:EGU General Assembly Conference Abstracts
Pages:10319
Description:The Great Lakes Runoff Inter-comparison Project (GRIP) includes a wide range of lumped and distributed models that are used operationally and/or for research purposes across Canada and the United States. Participating models are GEM-Hydro, WRF-Hydro, MESH, VIC, WATFLOOD, SWAT, mHM, Noah-MP, HYPE, LBRM, GR4J, HMETS, and purely statistical models. The latter are added to assess the information content of the forcing and geophysical datasets. As part of the Integrated Modelling Program for Canada (IMPC) under the Global Water Futures (GWF) program, the project is aiming to run all these models over several regions in Canada. We started with the Lake Erie watershed and then extended the study to the whole Great Lakes domain. One of the main contributions of the project is that we identified a standard dataset for model building that all participants in the inter-comparison project can access …
Cited by:1
:view less
Towards deep learning based flood forecasting for ungauged basins
2020
Frederik Kratzert, Daniel Klotz, Guy Shalev, Sella Nevo, Günter Klambauer, Grey Nearing…
EGU General Assembly Conference Abstracts, 8932
:read more
Authors:Frederik Kratzert, Daniel Klotz, Guy Shalev, Sella Nevo, Günter Klambauer, Grey Nearing, Sepp Hochreiter
Publication date:05/01/2020
Journal:EGU General Assembly Conference Abstracts
Pages:8932
Description:Floods are among the most destructive natural hazards in the world. To reduce flood induced damages and casualties, streamflow forecasts should be as accurate as possible. As of today, streamflow forecasts are usually made with either conceptual or process-based hydrological models. The problem these models usually have is that they perform best when calibrated for a specific basin, and performance degrades drastically if the models are used in places without historic streamflow measurements. To make things worse, some of the most devastating floods occur in developing and low-income countries, where historic records of streamflow measurements are scarce. Therefore, a central task for enhancing flood forecasts and helping local authorities to manage these areas is to provide high-quality streamflow forecasts in ungauged rivers. Although the IAHS dedicated an entire decade (2003-2012) to advance …
Cited by:1
:view less
The performance of LSTM models from basin to continental scales
2020
Frederik Kratzert, Daniel Klotz, Günter Klambauer, Grey Nearing, Sepp Hochreiter
EGU General Assembly Conference Abstracts, 8855
:read more
Authors:Frederik Kratzert, Daniel Klotz, Günter Klambauer, Grey Nearing, Sepp Hochreiter
Publication date:05/01/2020
Journal:EGU General Assembly Conference Abstracts
Pages:8855
Description:Simulation accuracy among traditional hydrological models usually degrades significantly when going from single basin to regional scale. Hydrological models perform best when calibrated for specific basins, and do worse when a regional calibration scheme is used. One reason for this is that these models do not (have to) learn hydrological processes from data. Rather, they have a predefined model structure and only a handful of parameters adapt to specific basins. This often yields less-than-optimal parameter values when the loss is not determined by a single basin, but by many through regional calibration. The opposite is true for data driven approaches where models tend to get better with more and diverse training data. We examine whether this holds true when modeling rainfall-runoff processes with deep learning, or if, like their process-based counterparts, data-driven hydrological models degrade when …
:view less
Machine learning is central to the future of hydrological modeling
2020
Grey Nearing, Frederik Kratzert, Craig Pelissier, Daniel Klotz, Jonathan Frame, Hoshin …
EGU General Assembly Conference Abstracts, 6111
:read more
Authors:Grey Nearing, Frederik Kratzert, Craig Pelissier, Daniel Klotz, Jonathan Frame, Hoshin Gupta
Publication date:05/01/2020
Journal:EGU General Assembly Conference Abstracts
Pages:6111
Description:This talk addresses aspects of three of the seven UPH themes:(i) time variability and change,(ii) space variability and scaling, and (iii) modeling methods. During the community contribution phase of the 23 Unsolved Problems effort, one of the suggested questions was" Does Machine Learning have a real role in hydrological modeling?" The final UPH paper claimed that" Most hydrologists would probably agree that [extrapolating to changing conditions] will require a more process-based rather than calibration-based approach as calibrated conceptual models do not usually extrapolate well." In this talk we will present a collection of recent experiments that demonstrate how catchment models based on deep learning can account for both temporal nonstationarity and spatial information transfer (eg, from gauged to ungauged catchments), often achieving significantly superior predictive performance compared to other …
Cited by:3
:view less
Learning from mistakes: Online updating for deep learning models.
2020
Daniel Klotz, Frederik Kratzert, Alden K Sampson, Günter Klambauer, Sepp Hochreiter, Gr…
EGU General Assembly Conference Abstracts, 8853
:read more
Authors:Daniel Klotz, Frederik Kratzert, Alden K Sampson, Günter Klambauer, Sepp Hochreiter, Grey Nearing
Publication date:05/01/2020
Journal:EGU General Assembly Conference Abstracts
Pages:8853
Description:Accurate streamflow forecasts are important for many operational purposes, like hydropower operation or flood risk management. It is obvious that for data-driven models best prediction performance would be obtained if recent streamflow observations were used as an additional model input. Therefore, there exists a certain imperative which demands to use forecasting models that use discharge signals whenever available. Forecasting models are, however, not well suited when continuous measurement of discharge can not be guaranteed or for applications in ungauged settings. Regarding the former, missing data can have long lasting repercussions on data-driven models if large data-windows are used for the input. Regarding the latter, data-driven forecast models are not applicable at all. Additionally, we would like to point out that data-driven simulation models need to represent the underlying hydrological …
:view less
Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets
2019
Frederik Kratzert, Daniel Klotz, Guy Shalev, Günter Klambauer, Sepp Hochreiter, Grey Ne…
Hydrology and Earth System Sciences 23 (12), 5089-5110
:read more
Authors:Frederik Kratzert, Daniel Klotz, Guy Shalev, Günter Klambauer, Sepp Hochreiter, Grey Nearing
Publication date:12/17/2019
Journal:Hydrology and Earth System Sciences
Volume:23
Issue:12
Pages:5089-5110
Publisher:Copernicus Publications
Description:Regional rainfall–runoff modeling is an old but still mostly outstanding problem in the hydrological sciences. The problem currently is that traditional hydrological models degrade significantly in performance when calibrated for multiple basins together instead of for a single basin alone. In this paper, we propose a novel, data-driven approach using Long Short-Term Memory networks (LSTMs) and demonstrate that under a “big data” paradigm, this is not necessarily the case. By training a single LSTM model on 531 basins from the CAMELS dataset using meteorological time series data and static catchment attributes, we were able to significantly improve performance compared to a set of several different hydrological benchmark models. Our proposed approach not only significantly outperforms hydrological models that were calibrated regionally, but also achieves better performance than hydrological models that were calibrated for each basin individually. Furthermore, we propose an adaption to the standard LSTM architecture, which we call an Entity-Aware-LSTM (EA-LSTM), that allows for learning catchment similarities as a feature layer in a deep learning model. We show that these learned catchment similarities correspond well to what we would expect from prior hydrological understanding.
Cited by:743
Resources:[HTML] from copernicus.org
:view less
Large-Scale Rainfall-Runoff Modeling using the Long Short-Term Memory Network
2019
Frederik Kratzert, Daniel Klotz, Günter Klambauer, Sepp Hochreiter, Grey Stephen Nearing
AGU Fall Meeting 2019
:read more
Authors:Frederik Kratzert, Daniel Klotz, Günter Klambauer, Sepp Hochreiter, Grey Stephen Nearing
Publication date:12/13/2019
Journal:AGU Fall Meeting 2019
Publisher:AGU
:view less
Physically-Based Machine Learning for Hydrological Modeling
2019
Grey Stephen Nearing, Hoshin Vijai Gupta, Frederik Kratzert, Daniel Klotz, Alden Keefe …
AGU Fall Meeting Abstracts 2019, H33L-2106
:read more
Authors:Grey Stephen Nearing, Hoshin Vijai Gupta, Frederik Kratzert, Daniel Klotz, Alden Keefe Sampson
Publication date:12/01/2019
Journal:AGU Fall Meeting Abstracts
Volume:2019
Pages:H33L-2106
Description:A current'Grand Challenge'across the Natural Sciences is to combine the power and versatility of machine learning for extracting patterns and information from large data sets with the physically-based or process-based understanding of dynamical systems that has resulted from centuries of scientific inquiry across different scientific disciplines.
:view less
Function Space Optimization (FSO): A novel method for estimating parameter transfer functions for hydrological models
2019
Moritz Feigl, Mathew Herrnegger, Daniel Klotz, Karsten Schulz
AGU Fall Meeting Abstracts 2019, H31E-04
:read more
Authors:Moritz Feigl, Mathew Herrnegger, Daniel Klotz, Karsten Schulz
Publication date:12/01/2019
Journal:AGU Fall Meeting Abstracts
Volume:2019
Pages:H31E-04
Description:Parameter estimation for spatially distributed rainfall-runoff models is a long studied and difficult problem. Linkages between catchment properties (eg elevation, soil texture,…) and model parameters have the potential to overcome some of the major difficulties associated with parameter estimation. One way to establish these linkages is to use explicit equations that act as" parameter transfer functions". These transfer functions map spatially distributed catchment properties to the model parameters. They would allow the estimation of consistent spatially distributed model parameter fields and potentially enable parameter extrapolation to other basins (ie regionalization).
:view less
Toward improved predictions in ungauged basins: Exploiting the power of machine learning
2019
Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Alden K Sampson, Sepp Hochreiter, G…
Water Resources Research 55 (12), 11344-11354
:read more
Authors:Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Alden K Sampson, Sepp Hochreiter, Grey S Nearing
Publication date:12/01/2019
Journal:Water Resources Research
Volume:55
Issue:12
Pages:11344-11354
Description:Long short‐term memory (LSTM) networks offer unprecedented accuracy for prediction in ungauged basins. We trained and tested several LSTMs on 531 basins from the CAMELS data set using k‐fold validation, so that predictions were made in basins that supplied no training data. The training and test data set included ∼30 years of daily rainfall‐runoff data from catchments in the United States ranging in size from 4 to 2,000 km2 with aridity index from 0.22 to 5.20, and including 12 of the 13 IGPB vegetated land cover classifications. This effectively “ungauged” model was benchmarked over a 15‐year validation period against the Sacramento Soil Moisture Accounting (SAC‐SMA) model and also against the NOAA National Water Model reanalysis. SAC‐SMA was calibrated separately for each basin using 15 years of daily data. The out‐of‐sample LSTM had higher median Nash‐Sutcliffe Efficiencies across the …
Cited by:604
Resources:[PDF] from wiley.com
:view less
Large-Scale Rainfall-Runoff Modeling using the Long Short-Term Memory Network
2019
GS Nearing, F Kratzert, D Klotz, G Klambauer, S Hochreiter
AGU Fall Meeting Abstracts 2019, H53B-05
:read more
Authors:GS Nearing, F Kratzert, D Klotz, G Klambauer, S Hochreiter
Publication date:12/01/2019
Journal:AGU Fall Meeting Abstracts
Volume:2019
Pages:H53B-05
Description:An outstanding problem in the hydrological modeling community is how to effectively use large-scale data sets for enhancing rainfall-runoff models. Currently, hydrological models perform best when calibrated for a specific basin, ignoring additional information that could be extracted from other (similar behaving) basins.
:view less
Accurate hydrologic modeling using less information
2019
Guy Shalev, Ran El-Yaniv, Daniel Klotz, Frederik Kratzert, Asher Metzger, Sella Nevo
arXiv preprint arXiv:1911.09427
:read more
Authors:Guy Shalev, Ran El-Yaniv, Daniel Klotz, Frederik Kratzert, Asher Metzger, Sella Nevo
Publication date:11/21/2019
Journal:arXiv preprint arXiv:1911.09427
Cited by:8
:view less
Using LSTMs for climate change assessment studies on droughts and floods
2019
Frederik Kratzert, Daniel Klotz, Johannes Brandstetter, Pieter-Jan Hoedt, Grey Nearing,…
arXiv preprint arXiv:1911.03941
:read more
Authors:Frederik Kratzert, Daniel Klotz, Johannes Brandstetter, Pieter-Jan Hoedt, Grey Nearing, Sepp Hochreiter
Publication date:11/10/2019
Journal:arXiv preprint arXiv:1911.03941
Description:Climate change affects occurrences of floods and droughts worldwide. However, predicting climate impacts over individual watersheds is difficult, primarily because accurate hydrological forecasts require models that are calibrated to past data. In this work we present a large-scale LSTM-based modeling approach that -- by training on large data sets -- learns a diversity of hydrological behaviors. Previous work shows that this model is more accurate than current state-of-the-art models, even when the LSTM-based approach operates out-of-sample and the latter in-sample. In this work, we show how this model can assess the sensitivity of the underlying systems with regard to extreme (high and low) flows in individual watersheds over the continental US.
Cited by:5
Resources:[PDF] from arxiv.org
:view less
Physically informed machine learning for hydrological modeling under climate nonstationarity
2019
Grey S Nearing, Craig S Pelissier, Frederik Kratzert, Daniel Klotz, Hoshin V Gupta, Jon…
:read more
Authors:Grey S Nearing, Craig S Pelissier, Frederik Kratzert, Daniel Klotz, Hoshin V Gupta, Jonathan M Frame, Alden K Sampson
Publication date:10/24/2019
Description:There is an understanding in the hydrological sciences community that physical realism is necessary for providing hydrological forecasts under changing conditions (Blöschl et al., 2019; Clark et al., 2016; Milly et al., 2008). At present, however, machine learning (ML) generally provides the best estimates of most hydrological states and fluxes, even in extrapolation (eg, Best et al., 2015; Kratzert et al., 2019a, b; Nearing et al., 2018). A notable example of this was provided by Kratzert et al.(2019a), who showed that Long Short Term Memory networks (LSTMs) produce, on average, better predictions in basins that did not supply training data (effectively ungaged basins) than a conceptual model well-calibrated to gauge data in individual basins (gaged basins). This is significant in that the 2003-2012 decadal problem of the International Association of Hydrological Sciences (IAHS) wasPrediction in Ungauged Basins' …
Cited by:11
:view less
Towards the quantification of uncertainty for deep learning based rainfallrunoff models
2019
Daniel Klotz, Frederik Kratzert, Mathew Herrnegger, Sepp Hochreiter, Günter Klambauer
Geophys. Res. Abstr 21
:read more
Authors:Daniel Klotz, Frederik Kratzert, Mathew Herrnegger, Sepp Hochreiter, Günter Klambauer
Publication date:01/01/2019
Journal:Geophys. Res. Abstr
Volume:21
Description:All forms of hydrological endeavors are pervaded by uncertainty. Therefore it is of little surprise that obtaining realistic estimates of prediction uncertainties has become a persistent ambition. m This contribution analyzes the capability of using a specific regularization-mechanism, called dropout, for uncertainty estimation of a neural network based rainfall-runoff models. The particular form of network under examination is based on the Long Short-Term Memory (LSTM) architecture, which was explicitly designated for time series applications (Hochreiter, citation). Recently, Kratzert el al.(2018a) have shown that LSTM based modeling approaches can be used as data-driven models for describing the rainfall-runoff relationship. The authors envisioned future applications for task like online prediction, where the model would be used in conjunction with a stack of other models. We believe that this envisioned application will only be possible if the networks will earn the trustworthiness from the wider hydrological community by exhibiting properties that go beyond mere performance. One potential avenue for cashing-out on this point is to open the black-box and analyze if the neural network reproduced internally (known) hydrological patterns (such as snow-accumulation and depletion processes, see Kratzert 2018b). A complementary approach, which is extremely important for any environmental model, is to characterize the uncertainty of a given prediction/simulation.
Cited by:2
Resources:[PDF] from copernicus.org
:view less
Using large data sets towards generating a catchment aware hydrological model for global applications
2019
Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Sepp Hochreiter, Günter Klambauer
EGU General Assembly Conference Abstracts, 13795
:read more
Authors:Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Sepp Hochreiter, Günter Klambauer
Publication date:01/01/2019
Journal:EGU General Assembly Conference Abstracts
Pages:13795
Description:One reason for the success of deep learning across many applications (eg image classification or machine translation) are large and publicly available data sets. In Hydrology, more specifically in the area of rainfall-runoff modelling, such publicly available data sets are still rare but are increasingly emerging from the community (eg CAMELS [1, 2] and CAMELS-CL [3]). Having such data sets allows to test new data-driven approaches for old but still outstanding hydrological problems. In this study we build upon the idea of Kratzert et al.[4] using LSTM (Long Short-Term Memory) networks as a regional hydrological model. However, substantial changes are made:(a) We train a single model on hundreds of catchments across the USA, not only for a single HUC (hydrological unit) as in [4](b) We do not only use meteorological time series as inputs but also static catchment characteristics through an adaption of the original LSTM architecture and therefore allow the LSTM to learn a catchment aware hydrological model. Here, we only use catchment characteristics of [2] that do not rely on discharge observation. With this setting we can therefore also investigate, if LSTMs can be used for prediction in ungauged basins. Concretely, we perform 6-fold cross-validation on 531 basins of the CAMELS data set. Out of the 6 splits, 5 are used for training and validation and a single split (approximately 89 basins) as test set for the final evaluation. This mimics an ungauged basin setting by not using any data from the basins in the test set during the entire training period.
Resources:[PDF] from copernicus.org
:view less
New symbolic regression methods for estimating parameter transfer functions for hydrological models.
2019
Moritz Feigl, Daniel Klotz, Mathew Herrnegger, Karsten Schulz
Geophysical Research Abstracts 21
:read more
Authors:Moritz Feigl, Daniel Klotz, Mathew Herrnegger, Karsten Schulz
Publication date:01/01/2019
Journal:Geophysical Research Abstracts
Volume:21
Description:Parameter estimation for spatially distributed rainfall-runoff models is a long studied and difficult problem. Linkages between catchment properties (such as elevation, slope or soil texture) and model parameters have the potential to cope with some of the major difficulties associated with parameter estimation. One way to establish these linkages is to use explicit equations that act as" parameter transfer functions", which map spatially distributed catchment properties to the model parameters. These transfer functions would allow the estimation of consistent spatially distributed model parameter fields and potentially allow to extrapolate parameter to other basins (ie regionalization, see: Samaniego et al., 2010; Samaniego et al., 2017). However, the form and structure of transfer functions is often only implicitly assumed or needs to be derived by a laborious literature guided trial and error process (see: Samaniego et al …
:view less
A glimpse into the Unobserved: Runoff simulation for ungauged catchments with LSTMs
2018
Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Sepp Hochreiter
:read more
Authors:Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Sepp Hochreiter
Publication date:12/01/2018
Description:Runoff predictions of a river from meteorological inputs is a key task in the field of hydrology. However, current hydrological models require a substantial amount of parameter tuning on basis of historical records. If no historical runoff observations are available it is very challenging to produce good predictions. In this study we explore the capability of LSTMs for simulating the runoff for these ungauged cases. A single LSTM is trained to learn a general hydrological model from hundreds of catchments throughout the contiguous United States of America and evaluated against catchments not used during training. Our results suggest that LSTMs a) are able to learn a general hydrological model and b) in the majority of catchments outperform an established hydrological model, which was especially trained for these catchments.
Cited by:9
Resources:[PDF] from openreview.net
:view less
Do internals of neural networks make sense in the context of hydrology?
2018
Frederik Kratzert, Mathew Herrnegger, Daniel Klotz, Sepp Hochreiter, Günter Klambauer
AGU fall meeting abstracts 2018, H13B-06
:read more
Authors:Frederik Kratzert, Mathew Herrnegger, Daniel Klotz, Sepp Hochreiter, Günter Klambauer
Publication date:12/01/2018
Journal:AGU fall meeting abstracts
Volume:2018
Pages:H13B-06
Description:In recent years, neural networks gained a new wave of popularity in many application domains, such as computer vision or natural language processing. However in applied environmental sciences, like rainfall-runoff modelling in hydrology, neural networks tend to have a rather bad reputation. This can be attributed to their black-box-ness and the difficulty or impossibility to understand network internals leading to a prediction.
Cited by:8
:view less
Rainfall–runoff modelling using long short-term memory (LSTM) networks
2018
Frederik Kratzert, Daniel Klotz, Claire Brenner, Karsten Schulz, Mathew Herrnegger
Hydrology and Earth System Sciences 22 (11), 6005-6022
:read more
Authors:Frederik Kratzert, Daniel Klotz, Claire Brenner, Karsten Schulz, Mathew Herrnegger
Publication date:11/22/2018
Journal:Hydrology and Earth System Sciences
Volume:22
Issue:11
Pages:6005-6022
Publisher:Copernicus Publications
Description:Rainfall–runoff modelling is one of the key challenges in the field of hydrology. Various approaches exist, ranging from physically based over conceptual to fully data-driven models. In this paper, we propose a novel data-driven approach, using the Long Short-Term Memory (LSTM) network, a special type of recurrent neural network. The advantage of the LSTM is its ability to learn long-term dependencies between the provided input and output of the network, which are essential for modelling storage effects in e.g. catchments with snow influence. We use 241 catchments of the freely available CAMELS data set to test our approach and also compare the results to the well-known Sacramento Soil Moisture Accounting Model (SAC-SMA) coupled with the Snow-17 snow routine. We also show the potential of the LSTM as a regional hydrological model in which one model predicts the discharge for a variety of catchments. In our last experiment, we show the possibility to transfer process understanding, learned at regional scale, to individual catchments and thereby increasing model performance when compared to a LSTM trained only on the data of single catchments. Using this approach, we were able to achieve better model performance as the SAC-SMA  Snow-17, which underlines the potential of the LSTM for hydrological modelling applications.
Cited by:1455
Resources:[HTML] from copernicus.org
:view less
Nonlinear control with a Raspberry PI. Automation of the experiment setup, ball on wheel “with a Raspberry PI and open source hard-and software: Automatisierung des …
2018
Sven-Olaf Lindert, Christian Höfler, Kurt Schlacher
e & i Elektrotechnik und Informationstechnik 135, 286-293
:read more
Authors:Sven-Olaf Lindert, Christian Höfler, Kurt Schlacher
Publication date:06/01/2018
Journal:e & i Elektrotechnik und Informationstechnik
Volume:135
Pages:286-293
Publisher:Springer Vienna
:view less
Demonstrating the “unit hydrograph” and flow routing processes involving active student participation–a university lecture experiment
2018
Karsten Schulz, Reinhard Burgholzer, Daniel Klotz, Johannes Wesemann, Mathew Herrnegger
Hydrology and Earth System Sciences 22 (4), 2607-2613
:read more
Authors:Karsten Schulz, Reinhard Burgholzer, Daniel Klotz, Johannes Wesemann, Mathew Herrnegger
Publication date:05/02/2018
Journal:Hydrology and Earth System Sciences
Volume:22
Issue:4
Pages:2607-2613
Publisher:Copernicus GmbH
Description: The unit hydrograph (UH) has been one of the most widely employed hydrological modelling techniques to predict rainfall–runoff behaviour of hydrological catchments, and is still used to this day. Its concept is based on the idea that a unit of effective precipitation per time unit (e.g. mm h−1) will always lead to a specific catchment response in runoff. Given its relevance, the UH is an important topic that is addressed in most (engineering) hydrology courses at all academic levels. While the principles of the UH seem to be simple and easy to understand, teaching experiences in the past suggest strong difficulties in students' perception of the UH theory and application. In order to facilitate a deeper understanding of the theory and application of the UH for students, we developed a simple and cheap lecture theatre experiment which involved active student participation. The seating of the students in the lecture theatre represented the “hydrological catchment” in its size and form. A set of plastic balls, prepared with a piece of magnetic strip to be tacked to any white/black board, each represented a unit amount of effective precipitation. The balls are evenly distributed over the lecture theatre and routed by some given rules down the catchment to the “catchment outlet”, where the resulting hydrograph is monitored and illustrated at the black/white board. The experiment allowed an illustration of the underlying principles of the UH, including stationarity, linearity, and superposition of the generated runoff and subsequent routing. In addition, some variations of the experimental setup extended the UH concept to demonstrate the impact of elevation, different …
Cited by:4
Resources:[HTML] from copernicus.org
:view less
Symbolic regression for the estimation of transfer functions of hydrological models
2017
D Klotz, M Herrnegger, K Schulz
Water Resources Research 53 (11), 9402-9423
:read more
Authors:D Klotz, M Herrnegger, K Schulz
Publication date:11/01/2017
Journal:Water Resources Research
Volume:53
Issue:11
Pages:9402-9423
Description:Current concepts for parameter regionalization of spatially distributed rainfall‐runoff models rely on the a priori definition of transfer functions that globally map land surface characteristics (such as soil texture, land use, and digital elevation) into the model parameter space. However, these transfer functions are often chosen ad hoc or derived from small‐scale experiments. This study proposes and tests an approach for inferring the structure and parametrization of possible transfer functions from runoff data to potentially circumvent these difficulties. The concept uses context‐free grammars to generate possible proposition for transfer functions. The resulting structure can then be parametrized with classical optimization techniques. Several virtual experiments are performed to examine the potential for an appropriate estimation of transfer function, all of them using a very simple conceptual rainfall‐runoff model with …
Cited by:34
Resources:[PDF] from wiley.com
Full View
:view less
Consistent Parameter and Transfer Function Estimation using Context Free Grammars
2017
Daniel Klotz, Mathew Herrnegger, Karsten Schulz
EGU General Assembly Conference Abstracts, 13532
:read more
Authors:Daniel Klotz, Mathew Herrnegger, Karsten Schulz
Publication date:04/01/2017
Journal:EGU General Assembly Conference Abstracts
Pages:13532
Description:This contribution presents a method for the inference of transfer functions for rainfall-runoff models. Here, transfer functions are defined as parametrized (functional) relationships between a set of spatial predictors (eg elevation, slope or soil texture) and model parameters. They are ultimately used for estimation of consistent, spatially distributed model parameters from a limited amount of lumped global parameters. Additionally, they provide a straightforward method for parameter extrapolation from one set of basins to another and can even be used to derive parameterizations for multi-scale models [see: Samaniego et al., 2010]. Yet, currently an actual knowledge of the transfer functions is often implicitly assumed. As a matter of fact, for most cases these hypothesized transfer functions can rarely be measured and often remain unknown. Therefore, this contribution presents a general method for the concurrent estimation of the structure of transfer functions and their respective (global) parameters. Note, that by consequence an estimation of the distributed parameters of the rainfall-runoff model is also undertaken. The method combines two steps to achieve this. The first generates different possible transfer functions. The second then estimates the respective global transfer function parameters. The structural estimation of the transfer functions is based on the context free grammar concept. Chomsky first introduced context free grammars in linguistics [Chomsky, 1956]. Since then, they have been widely applied in computer science. But, to the knowledge of the authors, they have so far not been used in hydrology. Therefore, the contribution gives an …
Resources:[PDF] from copernicus.org
:view less
visCOS: An R-package to evaluate model performance of hydrological models
2016
Daniel Klotz, Mathew Herrnegger, Johannes Wesemann, Karsten Schulz
EGU General Assembly Conference Abstracts, EPSC2016-7137
:read more
Authors:Daniel Klotz, Mathew Herrnegger, Johannes Wesemann, Karsten Schulz
Publication date:04/01/2016
Journal:EGU General Assembly Conference Abstracts
Pages:EPSC2016-7137
Description:The evaluation of model performance is a central part of (hydrological) modelling. Much attention has been given to the development of evaluation criteria and diagnostic frameworks.(Klemeš, 1986; Gupta et al., 2008; among many others). Nevertheless, many applications exist for which objective functions do not yet provide satisfying summaries. Thus, the necessity to visualize results arises in order to explore a wider range of model capacities, be it strengths or deficiencies. Visualizations are usually devised for specific projects and these efforts are often not distributed to a broader community (eg via open source software packages). Hence, the opportunity to explicitly discuss a state-of-the-art presentation technique is often missed. We therefore present a comprehensive R-package for evaluating model performance by visualizing and exploring different aspects of hydrological time-series. The presented package comprises a set of useful plots and visualization methods, which complement existing packages, such as hydroGOF (Zambrano-Bigiarini et al., 2012). It is derived from practical applications of the hydrological models COSERO and COSEROreg (Kling et al., 2014). visCOS, providing an interface in R, represents an easy-to-use software package for visualizing and assessing model performance and can be implemented in the process of model calibration or model development.
Resources:[PDF] from copernicus.org
:view less
Incorporation of an evolutionary algorithm to estimate transfer-functions for a parameter regionalization scheme of a rainfall-runoff model
2016
Daniel Klotz, Mathew Herrnegger, Karsten Schulz
EGU General Assembly Conference Abstracts, EPSC2016-7181
:read more
Authors:Daniel Klotz, Mathew Herrnegger, Karsten Schulz
Publication date:04/01/2016
Journal:EGU General Assembly Conference Abstracts
Pages:EPSC2016-7181
Description:This contribution presents a framework, which enables the use of an Evolutionary Algorithm (EA) for the calibration and regionalization of the hydrological model COSEROreg. COSEROreg uses an updated version of the HBV-type model COSERO (Kling et al. 2014) for the modelling of hydrological processes and is embedded in a parameter regionalization scheme based on Samaniego et al.(2010). The latter uses subscale-information to estimate model via a-priori chosen transfer functions (often derived from pedotransfer functions). However, the transferability of the regionalization scheme to different model-concepts and the integration of new forms of subscale information is not straightforward.(i) The usefulness of (new) single sub-scale information layers is unknown beforehand.(ii) Additionally, the establishment of functional relationships between these (possibly meaningless) sub-scale information layers and …
:view less
Extending amulti-scale parameter regionalization (MPR) method by introducing parameter constrained optimization and flexible transfer functions
2015
Daniel Klotz, Mathew Herrnegger, Karsten Schulz
EGU General Assembly Conference Abstracts, 7099
:read more
Authors:Daniel Klotz, Mathew Herrnegger, Karsten Schulz
Publication date:04/01/2015
Journal:EGU General Assembly Conference Abstracts
Pages:7099
Description:A multi-scale parameter-estimation method, as presented by Samaniego et al.(2010), is implemented and extended for the conceptual hydrological model COSERO. COSERO is a HBV-type model that is specialized for alpine-environments, but has been applied over a wide range of basins all over the world (see: Kling et al., 2014 for an overview). Within the methodology available small-scale information (DEM, soil texture, land cover, etc.) is used to estimate the coarse-scale model parameters by applying a set of transfer-functions (TFs) and subsequent averaging methods, whereby only TF hyper-parameters are optimized against available observations (eg runoff data). The parameter regionalisation approach was extended in order to allow for a more meta-heuristical handling of the transfer-functions. The two main novelties are: 1. An explicit introduction of constrains into parameter estimation scheme: The …
:view less
Towards high resolution soil property maps for Austria
2015
Christoph Schürz, Daniel Klotz, Mathew Herrnegger, Karsten Schulz
EGU General Assembly Conference Abstracts, 6289
:read more
Authors:Christoph Schürz, Daniel Klotz, Mathew Herrnegger, Karsten Schulz
Publication date:04/01/2015
Journal:EGU General Assembly Conference Abstracts
Pages:6289
Description:Soil hydraulic properties, such as soil texture, soil water retention characteristics, hydraulic conductivity, or soil depth are important inputs for hydrologic catchment modelling. However, the availability of such data in Austria is often insufficient to fulfill requirements of well-established hydrological models. Either, soil data is available in sufficient spatial resolution but only covers a small extent of the considered area, or the data is comprehensive but rather coarse in its spatial resolution. Furthermore, the level of detail and quality of the data differs between the available data sets. In order to generate a comprehensive soil data set for whole Austria that includes main soil physical properties, as well as soil depth and organic carbon content in a high spatial resolution (10x10 to 100x100m²) several available soil data bases are merged and harmonized. Starting point is a high resolution soil texture map that only covers …
:view less
Improvement of hydrological flood forecasting through an event based output correction method
2014
Daniel Klotz, Hans Peter Nachtnebel
EGU General Assembly Conference Abstracts, 10520
:read more
Authors:Daniel Klotz, Hans Peter Nachtnebel
Publication date:05/01/2014
Journal:EGU General Assembly Conference Abstracts
Pages:10520
Description:This contribution presents an output correction method for hydrological models. A conceptualisation of the method is presented and tested in an alpine basin in Salzburg, Austria. The aim is to develop a method which is not prone to the drawbacks of autoregressive models. Output correction methods are an attractive option for improving hydrological predictions. They are complementary to the main modelling process and do not interfere with the modelling process itself. In general, output correction models estimate the future error of a prediction and use the estimation to improve the given prediction. Different estimation techniques are available dependent on the utilized information and the estimation procedure itself. Autoregressive error models are widely used for such corrections. Autoregressive models with exogenous inputs (ARX) allow the use of additional information for the error modelling, eg measurements …
Cited by:1
:view less
Exploring Soil Moisture Dynamics and Variability Across Scales and Geological Settings Using Gaussian Mixture-Long Short-Term Memory Networks
Balazs Bischof, Daniel Klotz, Hoshin Gupta, Erwin Zehe, Ralf Loritz
Available at SSRN 5221598
:read more
Authors:Balazs Bischof, Daniel Klotz, Hoshin Gupta, Erwin Zehe, Ralf Loritz
Journal:Available at SSRN 5221598
Description:Soil moisture is a key variable for a range of hydrological and ecological processes, yet capturing its small-scale variability and preferential flow phenomena remains challenging. Recent advancements in deep learning have demonstrated potential in predicting hydrological variables, but conventional data-driven models often struggle to represent small-scale variability effectively. In this study, we integrate Long-Short Term Memory (LSTMs) networks and Gaussian Mixture Models (GMMs) to simulate soil moisture dynamics while explicitly quantifying its associated variability. Unlike deterministic approaches, our probabilistic framework accounts for nonlinear relationships between inputs and outputs while modeling the inherent small-scale variability in soil moisture. We apply this methodology to an in-situ soil moisture dataset from the Attert experimental basin, where the experimental design clusters three soil moisture profiles for each location and depth, providing an ideal framework for examining small-scale variability of soil moisture across distinct geological and pedological settings. Our results demonstrate that the proposed model effectively reproduces soil moisture dynamics across multiple depths and scales, achieving an average Kling-Gupta Efficiency (KGE) of 0.52, Rank Correlation of 0.72, and Root Mean Squared Error of 0.036 m3m-3, while also capturing the key aspects of small-scale variability and sensor uncertainty. Furthermore, the modeled distributions offer new insights into the spatiotemporal structure of soil moisture and underscore the value of probabilistic modeling in hydrological approaches. By explicitly incorporating …
Resources:[PDF] from ssrn.com
:view less
Competition Priors for Object-Centric Learning
Vihang Prakash Patil, Andreas Radler, Daniel Klotz, Sepp Hochreiter
:read more
Authors:Vihang Prakash Patil, Andreas Radler, Daniel Klotz, Sepp Hochreiter
Description:Humans are very good at abstracting from data and constructing concepts that are then reused. This is missing in current learning systems. The field of object-centric learning tries to bridge this gap by learning abstract representations, often called slots, from data without human supervision. Different methods have been proposed to tackle this task for images, whereas most are overly complex, non-differentiable, or poorly scalable. In this paper, we introduce a conceptually simple, fully-differentiable, non-iterative, and scalable method called **COP** (**C**ompetition **O**ver **P**ixel features). It is implementable using only Convolution and MaxPool layers and an Attention layer. Our method encodes the input image with a convolutional neural network and then uses a branch of alternating convolution and MaxPool layers to create competition and extract primitive slots. These primitive slots are then used as queries for a variant of Cross-Attention over the encoded image. Despite its simplicity, our method is competitive or outperforms previous methods on standard benchmarks. The code is publicly available.
Resources:[PDF] from openreview.net
:view less
The Great Lakes Runoff Intercomparison Project Phase 4: The Great Lakes (GRIP-GL)–Supplementary Material–
Juliane Mai, Hongren Shen, Bryan A Tolson, Étienne Gaborit, Richard Arsenault, James R …
:read more
Authors:Juliane Mai, Hongren Shen, Bryan A Tolson, Étienne Gaborit, Richard Arsenault, James R Craig, Vincent Fortin, Lauren M Fry, Martin Gauch, Daniel Klotz, Frederik Kratzert, Nicole O’Brien, Daniel G Princz, Sinan Rasiya Koya, Tirthankar Roy, Frank Seglenieks, Narayan K Shrestha, André GT Temgoua, Vincent Vionnet, Jonathan W Waddell10
Description:The ERA5-Land SWE estimates (Muñoz Sabater, 2019) were compared with the SWE observations available in the version 2 of the CanSWE dataset (Vionnet et al., 2021). CanSWE combines historical manual (snow surveys) and automatic SWE measurements collected across Canada by different provincial and territorial agencies as well as hydro-power companies. 5 CanSWE is an evolving dataset. Please refer to the following repository to check for version updates of the dataset: https://10.5281/zenodo. 4734371. In this study, only manual SWE measurements were used to make sure that the snow data would be available over the period from 2000 to 2017. In Ontario and Quebec, manual SWE data are collected on a biweekly to monthly basis. The times series of ERA5-Land SWE estimates were extracted at the grid cells corresponding to the snow observation locations available in 10 CanSWE. Only stations with at least 34 observations (ie, on average at least two observations per year during the study period from 2001 to 2017) were kept in the analysis. A total of 272 stations was considered and the results are shown in Fig. S1.
Resources:[PDF] from copernicus.org
:view less
Prediction in Ungauged Basins with Long Short-Term
Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Alden K Sampson, Sepp Hochreiter, G…
:read more
Authors:Frederik Kratzert, Daniel Klotz, Mathew Herrnegger, Alden K Sampson, Sepp Hochreiter, Grey S Nearing
Description:Abstract 15 Long Short-Term Memory (LSTM) networks offer unprecedented accuracy for predic-16 tion in ungauged basins. We trained and tested an LSTM on the CAMELS basins (ap-17 proximately 30 years of daily rainfall/runoff data from 531 catchments in the US of sizes 18 ranging from 4 km2 to 2,000 km2) using k-fold validation, so that predictions were made 19 in basins that supplied no training data. This effectively ‘ungauged’model was bench-20 marked over a 15-year validation period against the Sacramento Soil Moisture Account-21 ing (SAC-SMA) model and also against the NOAA National Water Model reanalysis. 22 SAC-SMA was calibrated separately for each basin using 15 years of daily data (ie, this 23 is a ‘gauged’model). The out-of-sample LSTM had higher median Nash-Sutcliffe Effi-24 ciencies across the 531 basins (0.69) than either the calibrated SAC-SMA (0.64) or the 25 National Water Model (0.58). We outline several future research directions that would 26 help develop this technology into a comprehensive regional hydrology model. 27
Resources:[PDF] from eartharxiv.org
:view less
:back to overview