🌎 TEDA - A Computational Toolbox for Teaching Ensemble Based Data Assimilation

🌎 TEDA - A Computational Toolbox for Teaching Ensemble Based Data Assimilation

Introducing TEDA: The Premier Python Toolbox for Teaching Ensemble-based Data Assimilation

Class Diagram - TEDA

We are thrilled to present TEDA, a cutting-edge Python toolbox designed to facilitate the teaching and learning of ensemble-based data assimilation (DA). TEDA caters to the needs of educators and learners alike, offering a comprehensive platform to explore the captivating world of meteorological anomalies, climate change, and DA methodologies.

TEDA stands out as an exceptional resource in the landscape of DA software, as it prioritizes the educational experience. While operational software tends to focus on practical applications, TEDA empowers undergraduate and graduate students by providing an intuitive platform to grasp the intricacies of ensemble-based DA concepts and foster enthusiasm for scientific exploration.

Equipped with a diverse range of features and functionalities, TEDA enriches the learning process by offering powerful visualization tools. Students can delve into error statistics, model errors, observational errors, error distributions, and the dynamic evolution of errors. These visualizations provide multiple perspectives on numerical results, enabling students to develop a deep understanding of ensemble-based DA principles.

TEDA offers numerous ensemble-based DA methods, allowing students to explore an extensive range of techniques. From the stochastic ensemble Kalman filter (EnKF) to dual EnKF formulations, EnKF via Cholesky decomposition, EnKF based on modified Cholesky decomposition, EnKF based on B-localization, and beyond, TEDA equips learners with a comprehensive toolkit to study and compare various DA methodologies.

To enhance the learning experience further, TEDA incorporates a selection of captivating toy models. These models, including the Duffing equation (2 variables), the Lorenz 63 model (3 variables), and the Lorenz 96 model (40 variables), exhibit chaotic behavior under specific parameter configurations. Such models provide an excellent opportunity for students to experiment, validate, and gain hands-on experience with diverse DA methods.

TEDA’s foundation is rooted in the Object-Oriented Programming (OOP) paradigm, enabling effortless integration of new techniques and models. This ensures that TEDA remains at the forefront of the rapidly evolving field of ensemble-based DA, allowing educators and researchers to incorporate the latest advancements into their teaching and experimentation.

Embark on a captivating journey of discovery and knowledge with TEDA as it unveils the secrets of data assimilation. Simulate various DA scenarios, experiment with different model configurations, and witness firsthand the transformative impact of ensemble-based DA methods on forecast accuracy and data analysis.

Unlock your true potential in the realm of data assimilation with TEDA. Whether you are an enthusiastic student or a dedicated educator, TEDA stands as your ultimate companion in unraveling the complexities of ensemble-based DA.

Keywords: Data Assimilation, Ensemble Kalman Filter, Education, Python.

đź’» GitHub Repo - https://github.com/enino84/TEDA

🚀 How easy is to use TEDA?

Choose the desired method from the pool of available methods:

from analysis.analysis_enkf_modified_cholesky import AnalysisEnKFModifiedCholesky

Select a model to perform the simulations:

model = Lorenz96()

Create a background object with the desired parameters:

background = Background(model, ensemble_size=20)

Create an analysis object with the desired method and its parameters + the chosen model:

analysis = AnalysisEnKFModifiedCholesky(model, r=2)

Define the observation parameters:

observation = Observation(m=32, std_obs=0.01)

Set up the parameters for the simulation:

params = {'obs_freq': 0.1, 'obs_times': 10, 'inf_fact': 1.04}
simulation = Simulation(model, background, analysis, observation, params=params)

and then, run the simulation!

simulation.run()

You can then request the backgound and analysis errors per assimilation step for plotting purposes or statistical computations:

import matplotlib.pyplot as plt

errb, erra = simulation.get_errors()

plt.figure(figsize=(12, 10))
plt.plot(np.log10(errb),'-ob')
plt.plot(np.log10(erra),'-or')
plt.show()

✔️ Supported methods

Class Description Reference
AnalysisEnKFShrinkagePrecision Compute the precision matrix for EnKF Shrinkage Precision. Nino-Ruiz, Elias D., and Adrian Sandu. “Ensemble Kalman filter implementations based on shrinkage covariance matrix estimation.” Ocean Dynamics 65 (2015): 1423-1439.
AnalysisEnKFModifiedCholesky Compute the precision matrix for EnKF Modified Cholesky. Nino-Ruiz, Elias D., Adrian Sandu, and Xinwei Deng. “An ensemble Kalman filter implementation based on modified Cholesky decomposition for inverse covariance matrix estimation.” SIAM Journal on Scientific Computing 40.2 (2018): A867-A886.
AnalysisEnKFNaive Perform assimilation using Naive EnKF method. Nino Ruiz, Elias D., Adrian Sandu, and Jeffrey Anderson. “An efficient implementation of the ensemble Kalman filter based on an iterative Sherman–Morrison formula.” Statistics and Computing 25 (2015): 561-577.
AnalysisEnSRF Perform assimilation using EnSRF method. Tippett, Michael K., et al. “Ensemble square root filters.” Monthly weather review 131.7 (2003): 1485-1490.
AnalysisETKF Perform assimilation using ETKF method. Bishop, Craig H., Brian J. Etherton, and Sharanya J. Majumdar. “Adaptive sampling with the ensemble transform Kalman filter. Part I: Theoretical aspects.” Monthly weather review 129.3 (2001): 420-436.
AnalysisLETKF Perform assimilation using LETKF method. Hunt, Brian R., Eric J. Kostelich, and Istvan Szunyogh. “Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter.” Physica D: Nonlinear Phenomena 230.1-2 (2007): 112-126.
AnalysisLEnKF Perform assimilation using LEnKF method. Ott, Edward, et al. “A local ensemble Kalman filter for atmospheric data assimilation.” Tellus A: Dynamic Meteorology and Oceanography 56.5 (2004): 415-428.
AnalysisEnKFBLoc Perform assimilation using EnKF B-Loc method. Greybush, Steven J., et al. “Balance and ensemble Kalman filter localization techniques.” Monthly Weather Review 139.2 (2011): 511-522.
AnalysisEnKFCholesky Perform assimilation using EnKF Cholesky decomposition. Mandel, Jan. Efficient implementation of the ensemble Kalman filter. University of Colorado at Denver and Health Sciences Center, Center for Computational Mathematics, 2006.
AnalysisEnKF Perform assimilation using EnKF full covariance matrix. Evensen, Geir. Data assimilation: the ensemble Kalman filter. Vol. 2. Berlin: springer, 2009.
Analysis Perform assimilation given background and observations. Abstract class to define general methods for all assimilation steps.

Analysis (Abstract class for analysis methods)

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisEnKF (Analysis EnKF full covariance matrix)

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisEnKFBLoc (Analysis EnKF B-Loc)

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisEnKFCholesky (EnKF implementation Cholesky)

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisEnKFModifiedCholesky (Analysis EnKF Modified Cholesky decomposition)

Method Description
get_precision_matrix Returns the computed precision matrix. It takes the following parameters:
- DX: Deviation matrix.
- regularization_factor: Value used as alpha in the ridge model.
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisEnKFNaive (Analysis EnKF Naive)

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.

AnalysisEnKFShrinkagePrecision (Analysis EnKF Shrinkage Precision)

Method Description
get_precision_matrix Returns the computed precision matrix. It takes the following parameters:
- DX: Deviation matrix.
- regularization_factor: Value used as alpha in the ridge model.
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisEnSRF (Analysis EnSRF)

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisETKF (Analysis Ensemble Transform Kalman Filter (ETKF))

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.

AnalysisLEnKF (Analysis LEnKF)

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.

AnalysisLETKF (Analysis Local Ensemble Transform Kalman Filter (LETKF))

Method Description
perform_assimilation Perform the assimilation step given the background and observations. It takes the following parameters:
- background: Background object representing the background ensemble or state.
- observation: Observation object representing the observation used for assimilation.
get_analysis_state Return the computed column mean of the ensemble Xa. It doesn’t take any parameters.
inflate_ensemble Compute the new ensemble Xa given the inflation factor. It takes the following parameter:
- inflation_factor: Double number indicating the inflation factor to be applied to the ensemble.
get_ensemble Return the ensemble Xa. It doesn’t take any parameters.
get_error_covariance Return the computed covariance matrix of the ensemble Xa. It doesn’t take any parameters.