Reference Evapotranspiration (ETo) Methods Comparison

A reproducible analysis framework for comparing up to 18 empirical and semi-empirical ET0 estimation methods against Penman-Monteith (FAO-56).

This repository is organized as an open, citable research compendium. The current stage includes the executable ETo pipeline, method rankings, uncertainty/sensitivity diagnostics, data-quality reports, optional PCA of meteorological drivers, tests, citation metadata, licensing, contribution guidance, and documentation for reproducing the analysis.

Key Findings

This repository now configures 18 alternative ET0 methods plus Penman-Monteith FAO-56 as reference. The current demonstration workflow runs end to end for two contrasting Brazilian climates:

Piracicaba, SP (Cwa — humid subtropical with dry winter)
Manaus, AM (Af — tropical rainforest)

Manaus and Piracicaba are demonstration sites, not a fixed multicity study design. Additional sites can be added through configs/sites.yml without changing the code structure.

Main results:

Temperature-based methods (Thornthwaite, Camargo) systematically underestimate ETo in both climates, with errors exceeding 30% RMSE
Priestley-Taylor shows excellent performance in humid Manaus (RMSE < 15%), but moderate performance in Piracicaba
Hargreaves-Samani (calibrated) achieves best balance across both sites when radiation data is unavailable
All simplified methods degrade more in Manaus than Piracicaba, highlighting the challenge of tropical humid climates

→ See outputs/tables/summary_rankings.csv for a ranked comparison across sites and scales (also available as outputs/reports/summary_rankings.md).

Per-method metrics remain in outputs/tables/{site}_{daily|monthly}_metrics.csv. The reproducibility workflow now also writes data-quality reports, uncertainty/sensitivity tables, bias-by-ETo-bin figures, summary rankings, and optional PCA outputs. After running compute-eto, metrics use the pipeline-calculated ET0 series in outputs/results/{site}_daily_eto.csv; if those files are absent, the legacy precomputed columns in data/cleaned/ remain the fallback.

What This Repository Provides

Ready-to-Use Results (for Demonstration Sites)

Clone and run the pipeline in under 5 minutes to generate:

📊 Metrics Tables (outputs/tables/)

summary_rankings.csv — All methods ranked by site (Manaus, Piracicaba), scale (daily, monthly), and selected ranking rule
{site}_daily_metrics.csv — RMSE, MAE, MBE, Pearson r, R², Willmott d, confidence c, and performance classification for daily estimates
{site}_monthly_metrics.csv — Same metrics aggregated monthly
{site}_bootstrap_metric_intervals.csv — Bootstrap confidence intervals for method metrics
{site}_seasonal_error_metrics.csv — Seasonal error diagnostics
{site}_bias_by_eto_bin.csv — Bias summarized by Penman-Monteith ETo range
→ Start with summary_rankings.csv to see which methods perform best in each climate

📈 Figures (outputs/figures/{site}/)

Taylor diagrams (daily and monthly) — Visual summary of method agreement
Scatter plots — Method vs Penman-Monteith comparisons
Time series — Temporal patterns over the year
Monthly totals — Seasonal ETo accumulation by method
Bias-by-ETo-bin plots — Bias behavior across low-to-high reference ETo ranges
Optional PCA biplots — Meteorological driver structure when pca is run

📁 Intermediate Data (data/cleaned/, outputs/results/)

Cleaned daily time series: data/cleaned/{site}_daily.csv
7-day rolling means: outputs/results/{site}_rolling_7d.csv
Monthly aggregations: outputs/results/{site}_monthly_totals.csv
Computed daily ETo series: outputs/results/{site}_daily_eto.csv

📄 Reports (outputs/reports/)

{site}_data_quality.csv and data_quality_summary.csv — Missing-value, date-range, and interpolation audits
{site}_uncertainty_sensitivity.md — Site-level uncertainty and sensitivity narrative
summary.csv and summary.md — Best method by site, temporal scale, rank, and ranking rule
summary_rankings.md — Readable version of the ranked method table

Extensible Framework

The pipeline works for any location where you have meteorological data:

Add site metadata to configs/sites.yml
Add compatible data to data/raw/ or adapt the reader for your source format
Keep method metadata in configs/methods.yml
Run the same pipeline and get the same output structure for any number of configured sites

Quick Start

Prerequisites

Python 3.10 or higher (CI uses Python 3.12)
- Check your version: python3 --version
pip for package management
Git for cloning the repository

Step-by-Step Installation

1. Clone the repository

git clone https://github.com/brunomartinsmv/eto-methods-comparison.git
cd eto-methods-comparison

2. Verify data is present

ls data/raw/

You should see Evapo.xlsx among the files. It contains the raw meteorological data for Piracicaba and Manaus (2024).

3. Create a virtual environment (recommended to avoid dependency conflicts)

python3 -m venv .venv

4. Activate the virtual environment

# On macOS/Linux:
source .venv/bin/activate

# On Windows (Command Prompt):
.venv\Scripts\activate.bat

# On Windows (PowerShell):
.venv\Scripts\Activate.ps1

Your prompt should now show (.venv) prefix.

5. Install dependencies

pip install --upgrade pip
pip install -r requirements.txt

This installs: pandas, numpy, matplotlib, openpyxl, and scientific libraries.

For development and test workflows, install the package with development dependencies:

pip install -e ".[dev]"

Recommended workflow

For a first run (and for full regeneration), use:

MPLCONFIGDIR=/tmp/matplotlib-cache python -m scripts.cli quickstart --year 2024

This command:

Checks that data/raw/Evapo.xlsx is present (fails early with a clear message if not)
Cleans data, computes ET₀, metrics, figures, validation, and rankings
Writes feasibility + site HTML reports
Exports the supplement package
Builds outputs/index.html for browsing results

Then open outputs/index.html in a browser.

For a single configured site:

python -m scripts.cli quickstart --year 2024 --site manaus

Step-by-step alternative (when you want more control):

python -m scripts.cli inspect --site manaus          # Check which methods are feasible
python -m scripts.cli run-site --site manaus --compute-eto  # Full pipeline for one site
python -m scripts.cli report-site --site manaus      # Consolidated Markdown/HTML report
python -m scripts.cli build-index                    # Global index (outputs/index.html)

To compute only one method plus the Penman-Monteith reference:

python -m scripts.cli run-method --site manaus --method hs

6. Check your results

ls outputs/tables/
# Should show: piracicaba_daily_metrics.csv, piracicaba_monthly_metrics.csv
#              manaus_daily_metrics.csv, manaus_monthly_metrics.csv,
#              summary_rankings.csv, and uncertainty/sensitivity tables

ls outputs/figures/piracicaba/
# Should show: multiple .png files (Taylor diagrams, scatter plots, time series,
# monthly totals, and bias-by-ETo-bin figures)

ls outputs/reports/
# Should show: data-quality reports, summary reports, and uncertainty narratives

open outputs/index.html

Output Naming Standard

The current CLI uses snake-case suffixes for generated files:

Daily cleaned data: data/cleaned/{site}_daily.csv
7-day rolling results: outputs/results/{site}_rolling_7d.csv
Monthly totals: outputs/results/{site}_monthly_totals.csv
Computed daily ET0 methods: outputs/results/{site}_daily_eto.csv
Daily metrics: outputs/tables/{site}_daily_metrics.csv
Monthly metrics: outputs/tables/{site}_monthly_metrics.csv
Daily scatter figures: outputs/figures/{site}/{site}_daily_scatter_{method}_vs_pm.png
Daily time-series figures: outputs/figures/{site}/{site}_daily_series_{method}_vs_pm.png
Monthly totals figure: outputs/figures/{site}/{site}_monthly_totals.png
Taylor figures: outputs/figures/{site}/{site}_daily_taylor.png and {site}_monthly_taylor.png
Bias-by-ETo-bin figure: outputs/figures/{site}/{site}_bias_by_eto_bin.png
Bootstrap intervals: outputs/tables/{site}_bootstrap_metric_intervals.csv
Seasonal errors: outputs/tables/{site}_seasonal_error_metrics.csv
Bias-by-ETo-bin table: outputs/tables/{site}_bias_by_eto_bin.csv
Data-quality reports: outputs/reports/{site}_data_quality.csv
Method rankings: outputs/tables/summary_rankings.csv and outputs/reports/summary_rankings.md
Summary reports: outputs/reports/summary.csv and outputs/reports/summary.md
Uncertainty/sensitivity report: outputs/reports/{site}_uncertainty_sensitivity.md
Optional PCA outputs: outputs/tables/{site}_pca_loadings.csv, outputs/tables/{site}_pca_explained_variance.csv, and outputs/figures/{site}/{site}_pca_biplot.png

Older rolling7d files are legacy names. Use rolling_7d for current pipeline outputs.

Running Individual Steps

If you only need specific outputs:

python -m scripts.cli clean      # Clean raw data → data/cleaned/
python -m scripts.cli compute-eto --year 2024  # Compute ET0 from cleaned weather variables → outputs/results/
python -m scripts.cli calibrate --site manaus --method hargreaves_samani  # Optional local calibration
python -m scripts.cli aggregate  # Create aggregations → outputs/results/
python -m scripts.cli metrics    # Compute RMSE, r, R², Willmott d, c, etc. → outputs/tables/ (prefers computed ET0 if available)
python -m scripts.cli plots      # Generate all figures → outputs/figures/
python -m scripts.cli pca        # Optional PCA on meteorological drivers
python -m scripts.cli analyze-uncertainty  # Bootstrap, seasonal, and bias-bin diagnostics
python -m scripts.cli validate-data  # Audit dates, missing values, interpolation traces
python -m scripts.cli summarize      # Rank methods with the default composite rule
python -m scripts.cli summarize --ranking rmse       # Rank by lowest RMSE
python -m scripts.cli summarize --ranking composite  # Rank by the documented multi-metric rule
python -m scripts.cli quickstart --year 2024       # Full pipeline + reports + index + supplement
python -m scripts.cli inspect --site manaus        # Pre-flight method feasibility report
python -m scripts.cli run-site --site manaus       # Full single-site pipeline wrapper
python -m scripts.cli run-method --site manaus --method hs  # Compute one method + PM reference
python -m scripts.cli report-site --site manaus    # Consolidated Markdown/HTML site report
python -m scripts.cli build-index                  # Navigation index (outputs/index.html)
python -m scripts.cli clean-outputs --dry-run      # List regenerable output files
python -m scripts.cli export-supplement             # Collect supplemental outputs

summarize accepts --ranking rmse, --ranking mae, --ranking c, --ranking willmott_d, or --ranking composite. The default is composite, which ranks methods within each site and temporal scale by: highest confidence coefficient c, then lowest rmse, lowest mae, highest willmott_d, and lowest absolute mbe. The selected rule is written to selection_rule, and the chosen order is written to rank in both outputs/reports/summary.csv and outputs/tables/summary_rankings.csv.

metrics remains backward compatible: if outputs/results/{site}_daily_eto.csv does not exist, it falls back to data/cleaned/{site}_daily.csv and uses the precomputed et_* columns from the cleaned data.

calibrate writes separate local-calibration artifacts instead of replacing original methods. Current calibrable methods are hargreaves_samani, turc, and radiation_temperature. It uses cleaned meteorological drivers, and when the default cleaned input is used it also uses outputs/results/{site}_daily_eto.csv for the pipeline-computed et_penman_monteith reference when that file exists, matching metrics. For custom cleaned inputs, pass the matching computed-result directory with --results-input; when supplied, the matching {site}_daily_eto.csv file is required. For explicit validation periods:

python -m scripts.cli calibrate --site manaus --method hargreaves_samani \
  --results-input outputs/results \
  --train-start 2024-01-01 --train-end 2024-06-30 \
  --test-start 2024-07-01 --test-end 2024-12-31

The outputs are outputs/tables/{site}_{method}_calibration_coefficients.csv and outputs/tables/{site}_{method}_calibration_metrics.csv, including train/test periods, the minimize_train_rmse objective, fitted coefficients, and before/after metrics.

Expected runtime: ~30 seconds for full pipeline on both sites (2024 data) [Tested in a MacBook Air M4, so results may vary].

Troubleshooting

Problem: ModuleNotFoundError: No module named 'scripts'

Solution: Make sure you're running from the repository root directory and using -m flag:

cd /path/to/eto-methods-comparison
python -m scripts.cli all --year 2024

Problem: FileNotFoundError: data/raw/Evapo.xlsx not found

Solution:

Verify the file exists: ls data/raw/Evapo.xlsx
If missing, the data file may not have been committed to git. Contact the repository maintainer or see "Using Your Own Data" section below.

Problem: ImportError: openpyxl or similar import errors

Solution: Reinstall dependencies:

pip install --upgrade pip
pip install -r requirements.txt --force-reinstall

Problem: Virtual environment won't activate on Windows PowerShell

Solution: Enable script execution:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
.venv\Scripts\Activate.ps1

Problem: Plots look wrong or metrics are NaN

Solution:

Check for missing data: python -c "import pandas as pd; df = pd.read_csv('data/cleaned/piracicaba_daily.csv'); print(df.isnull().sum())"
Ensure you're using --year 2024 (the year with complete data)
If using your own data, verify all required meteorological variables are present

Problem: Permission denied when creating directories

Solution: Check write permissions:

chmod -R u+w outputs/ data/cleaned/

Still having issues?

Check Python version: python3 --version (must be ≥3.8)
Try running in a fresh virtual environment
Open an issue on GitHub with the full error message

Using Your Own Data

The pipeline is designed for any location with meteorological data. Here's how to add a new city:

Required Meteorological Variables

To compute the full configured set of 18 alternative ET0 methods, you generally need daily data for:

Minimum requirements (for basic methods):

Date
Temperature (mean, min, max)
Latitude of the site

For complete analysis (including Penman-Monteith):

Net radiation (Rn) or solar radiation
Wind speed at 2m height (u2)
Relative humidity or vapor pressure
Soil heat flux (G) — often assumed as 0 for daily calculations

Data Sources

Option 1: INMET (Brazilian stations)

Portal: https://portal.inmet.gov.br/
Follow the instructions provided
Download automatic station data (CSV format)
Best for Brazilian locations with high-quality automated measurements
For Brazil it's the fastest option, but our station data sometimes has gaps due to the conditions of the station, so you'll need to fill that with (interpolation and knn are the easiest ways and works fine).
Btw, INMET stands for Metereological National Institute (Instituto Nacional de Meterologia in Brazilian Portuguese)

Option 2: ERA5 Reanalysis (global coverage)

Portal: https://cds.climate.copernicus.eu/
Product: ERA5 hourly/daily data on single levels
Variables needed: 2m temperature, surface solar radiation, 10m wind, dewpoint
Automated download tool: Use this repository to download and process ERA5 data automatically:
```
git clone https://github.com/brunomartinsmv/era5-daily-statistics-data-download.git
```
Follow the instructions there to get daily statistics for any location globally.
Best for locations without ground stations or historical gap-filling (but maybe takes a lot to download, due to your order being queued).

Option 3: Your local station (equivalent to INMET)

Most countries has some station to collect data, you can use that as well.

Adding a New Site to the Pipeline

1. Prepare your data file

Format as Excel (.xlsx) with one sheet per site
Columns must include: date, temperature variables, radiation, wind, humidity
See data/raw/Evapo.xlsx as reference

2. Edit configuration (configs/sites.yml)

Add your site to the sites mapping:

sites:
  cuiaba:
    sheet: Cuiaba
    lat: -15.6
    lon: -56.1
    alt_m: 165.0
    biome: Cerrado
    climate_class: Aw
    region: Centro-Oeste
    country: Brazil
    state: MT

biome, climate_class, region, country, and state are optional interpretive metadata. They are useful for summaries and optional grouping, but the project does not require them and does not become a fixed regional or multicity study when they are present.

Method column mappings and short labels live in configs/methods.yml.

3. Run the pipeline

python -m scripts.cli all --year 2024

To run one configured site:

python -m scripts.cli all --year 2024 --site manaus
python -m scripts.cli all --year 2024 --site piracicaba

The pipeline will automatically:

Process your new site
Generate metrics comparing all methods
Create figures in outputs/figures/cuiaba/
Compute performance tables in outputs/tables/cuiaba_*.csv
Write uncertainty/sensitivity diagnostics for the site

Data Format Example

Your Excel sheet should look like:

date       | temp_mean | temp_min | temp_max | radiation | wind_2m | rh_mean
2024-01-01 | 25.3      | 19.2     | 32.1     | 22.5      | 1.8     | 68.2
2024-01-02 | 26.1      | 20.0     | 33.5     | 23.1      | 2.1     | 65.5
...

Note: Column names should match those expected by scripts/io.py, or configure a custom reader block in configs/sites.yml for CSV/Excel files with different headers.

Flexible input readers

Sites can declare a custom reader in configs/sites.yml:

sites:
  my_station:
    lat: -15.6
    lon: -56.1
    alt_m: 165.0
    reader:
      format: generic
      path: data/raw/my_station.csv
      skiprows: 0
      column_map:
        "Temp Média": tmed_c
        "Data": date

The default `evapo_legacy` format keeps compatibility with `Evapo.xlsx` (sheet name + 4 header rows).

Repository Structure

.
├── data/
│   ├── raw/            # Original meteorological data (Evapo.xlsx)
│   └── cleaned/        # Processed daily time series
├── outputs/
│   ├── results/        # Intermediate aggregations (rolling_7d, monthly totals)
│   ├── figures/        # All generated plots (Taylor diagrams, scatter plots, time series)
│   ├── reports/        # Data quality reports
│   └── tables/         # **Metrics tables (RMSE, r, R², Willmott d, c) ← Start here**
├── scripts/            # Analysis pipeline (CLI, cleaning, metrics, plotting)
├── notebooks/          # Educational Jupyter notebooks with step-by-step explanations
├── docs/               # Detailed methodology, equations, and interpretation guides
└── requirements.txt    # Python dependencies

Files under outputs/**/legacy/ are historical notebook-era outputs, not the official preprint result set. See docs/legacy.md.

For a documentation map, start with docs/README.md. For version history, see CHANGELOG.md. For exact reproduction commands, see docs/reproducibility.md. For contribution expectations, see CONTRIBUTING.md. Participants are also covered by the project CODE_OF_CONDUCT.md.

How to Cite

If you use this repository or its outputs in academic work, please cite:

Vieira, B. M. M. (2026). Reference Evapotranspiration (ETo) Methods Comparison (Version 2.1.0) [Software]. Universidade Federal do Mato Grosso. https://doi.org/10.5281/zenodo.21341983

DOI: https://doi.org/10.5281/zenodo.21341983

The repository also includes CITATION.cff, which GitHub can use to generate citation formats automatically. Add the arXiv DOI to CITATION.cff and this section after the preprint is available.

BibTeX:

@misc{vieira2026eto,
  author = {Vieira, Bruno Martins M.},
  title = {Reference Evapotranspiration (ETo) Methods Comparison},
  year = {2026},
  version = {2.1.0},
  howpublished = {Software and analysis outputs},
  institution = {Universidade Federal do Mato Grosso},
  doi = {10.5281/zenodo.21341983},
  url = {https://github.com/brunomartinsmv/eto-methods-comparison}
}

Methods Overview

The repository configuration targets 18 alternative ET0 estimation methods plus Penman-Monteith FAO-56 as the reference. The compute-eto command calculates the 15 methods with status: computed from standardized meteorological variables and writes daily calculated series to outputs/results/{site}_daily_eto.csv.

Reference standard:

Penman-Monteith (FAO-56) — Energy balance + aerodynamic approach, requires full met data

Computed 15-method comparison scope: Camargo, Hargreaves-Samani, Makkink, McCloud, Priestley-Taylor, Turc, Global Radiation, Ivanov, Jensen-Heise, Garcia-Lopez, Net Radiation, Radiation-Temperature, Lungeon, Stephens-Stewart, and Hicks-Hess.

The configuration also preserves existing computed legacy/auxiliary method columns: Thornthwaite, Thornthwaite-Camargo, and locally corrected Hargreaves-Samani.

For detailed equations, assumptions, limitations, and climate suitability of each method, see docs/methodology.md.

For Students & Learners

New to ETo estimation? Start here:

Read the methodology first: docs/methodology.md explains each method's physics (will be more explained in the future), equations, and when to use them.
Explore the notebooks: notebooks/ contains step-by-step Jupyter notebooks that explain:
- Why we estimate ETo and what it represents
- How each method works conceptually
- How to interpret Taylor diagrams and performance metrics
- Common pitfalls and climate-specific considerations
Run the pipeline: Follow the Quick Start above to generate results for Piracicaba and Manaus.
Interpret the results:
- Start with outputs/tables/*_daily_metrics.csv — these show which methods perform best
- Check Taylor diagrams in outputs/figures/{site}/{site}_daily_taylor.png for visual summary
- Compare daily vs monthly performance to understand temporal aggregation effects
Experiment: Try adding a new city using your own data or ERA5 downloads.

References

Allen, R. G., Pereira, L. S., Raes, D., & Smith, M. (1998). Crop Evapotranspiration — Guidelines for computing crop water requirements (FAO Irrigation and Drainage Paper No. 56). FAO.
Thornthwaite, C. W. (1948). An approach toward a rational classification of climate. Geographical Review.
Hargreaves, G. H., & Samani, Z. A. (1985). Reference crop evapotranspiration from temperature. Applied Engineering in Agriculture.
Priestley, C. H. B., & Taylor, R. J. (1972). On the assessment of surface heat flux and evaporation using large-scale parameters. Monthly Weather Review.

For complete references and method-specific citations, see docs/methodology.md.

License

This project is distributed under the MIT License. See LICENSE.

Contact

For questions, suggestions, or collaboration:

Author: Bruno Martins M. Vieira
Institution: Universidade Federal do Mato Grosso
GitHub Issues: Report issues or ask questions

Contributing

Contributions that improve reproducibility, documentation, tests, method auditability, or support for additional well-documented sites are welcome. Please read CONTRIBUTING.md before opening a pull request.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Reference Evapotranspiration (ETo) Methods Comparison

Key Findings

What This Repository Provides

Ready-to-Use Results (for Demonstration Sites)

Extensible Framework

Quick Start

Prerequisites

Step-by-Step Installation

Recommended workflow

Output Naming Standard

Running Individual Steps

Troubleshooting

Using Your Own Data

Required Meteorological Variables

Data Sources

Adding a New Site to the Pipeline

Data Format Example

Flexible input readers

The default `evapo_legacy` format keeps compatibility with `Evapo.xlsx` (sheet name + 4 header rows).

Repository Structure

How to Cite

Methods Overview

For Students & Learners

References

License

Contact

Contributing

About

Releases

Packages

Used by

Contributors

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 132 Commits
.github/workflows		.github/workflows
configs		configs
data		data
docs		docs
notebooks		notebooks
outputs		outputs
scripts		scripts
tasks		tasks
tests		tests
.gitignore		.gitignore
.zenodo.json		.zenodo.json
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
CITATION.cff		CITATION.cff
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
pyproject.toml		pyproject.toml
requirements-lock.txt		requirements-lock.txt
requirements.txt		requirements.txt

Folders and files

Latest commit

History

Repository files navigation

Reference Evapotranspiration (ETo) Methods Comparison

Key Findings

What This Repository Provides

Ready-to-Use Results (for Demonstration Sites)

Extensible Framework

Quick Start

Prerequisites

Step-by-Step Installation

Recommended workflow

Output Naming Standard

Running Individual Steps

Troubleshooting

Using Your Own Data

Required Meteorological Variables

Data Sources

Adding a New Site to the Pipeline

Data Format Example

Flexible input readers

The default evapo_legacy format keeps compatibility with Evapo.xlsx (sheet name + 4 header rows).

Repository Structure

How to Cite

Methods Overview

For Students & Learners

References

License

Contact

Contributing

About

Topics

Resources

Code of conduct

Contributing

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

The default `evapo_legacy` format keeps compatibility with `Evapo.xlsx` (sheet name + 4 header rows).