diff --git a/README.md b/README.md new file mode 100644 index 0000000..7b72e9e --- /dev/null +++ b/README.md @@ -0,0 +1,165 @@ +# HAB2: Harmful Algal Bloom Analysis using Empirical Dynamic Modeling + +## Overview + +This repository contains a comprehensive analysis of harmful algal blooms (HABs) in the Charles River using Empirical Dynamic Modeling (EDM) techniques. The project focuses on understanding the dynamics of cyanobacteria (phycocyanin) and chlorophyll concentrations through time series analysis, cross-mapping, and multivariate modeling approaches. + +## Project Structure + +``` +HAB2/ +├── 1_LIB/ # Data library and processed datasets +│ ├── 1_0_data_wrangling/ # Data preprocessing and cleaning +│ ├── 1_2_univariate/ # Univariate analysis results +│ ├── 1_3_CCM/ # Convergent Cross Mapping results +│ └── 1_4_multivariate/ # Multivariate analysis results +├── 2_FIGURES/ # R Markdown analysis notebooks +│ ├── 00_data_wrangling.Rmd # Data preprocessing pipeline +│ ├── 01_univariate_Draft2_EDM.Rmd # Univariate EDM analysis +│ ├── 02_crossmap_EDM.Rmd # Cross-mapping analysis +│ ├── 03_multivariate_EDM.Rmd # Multivariate EDM analysis +│ └── figures_and_tables.Rmd # Summary figures and tables +├── 3_OUTPUT/ # Generated figures and results +│ ├── 4_2_univariate/ # Univariate analysis outputs +│ ├── 4_3_CCM/ # Cross-mapping outputs +│ └── 4_5_figures_and_tables/ # Final figures and tables +└── 4_VAULT/ # Archive of previous analyses +``` + +## Data Sources + +The analysis incorporates multiple environmental datasets: + +- **EPA Buoy Data**: High-frequency (15-minute) water quality measurements from Charles River buoy +- **MWRA Data**: Water quality parameters including nutrients, bacteria, and physical properties +- **Logan Airport Weather**: Meteorological data for atmospheric forcing +- **CSO Data**: Combined sewer overflow discharge information +- **CAM017**: Specific CSO discharge data + +### Key Variables + +- **Phycocyanin (RFU)**: Primary indicator of cyanobacteria abundance +- **Chlorophyll (RFU)**: General phytoplankton biomass indicator +- **Physical Parameters**: Temperature, pH, dissolved oxygen, turbidity, salinity +- **Nutrients**: Ammonium, nitrate+nitrite, phosphate, total phosphorus +- **Meteorological**: Temperature, precipitation, wind speed/direction, pressure + +## Analysis Methods + +### 1. Univariate EDM Analysis +- **Simplex Projection**: Finding optimal embedding dimensions +- **S-Map**: Nonlinear state-space modeling with varying localization +- **AR(1) Benchmark**: Linear autoregressive model comparison +- **Time Scales**: Analysis across multiple temporal resolutions (15-min, hourly, daily, morning) + +### 2. Cross-Mapping (CCM) +- **Convergent Cross Mapping**: Detecting causal relationships between variables +- **Surrogate Testing**: Statistical significance testing using seasonal surrogates +- **Library Size Analysis**: Convergence testing across different data lengths + +### 3. Multivariate EDM +- **Kitchen Sink Models**: Comprehensive multivariate forecasting +- **Variable Selection**: Identifying key drivers of algal bloom dynamics +- **Prediction Skill**: Out-of-sample forecasting performance evaluation + +## Key Findings + +### Temporal Dynamics +- **Phycocyanin**: 6-11 day autocorrelation timescales +- **Chlorophyll**: 3-6 day autocorrelation timescales +- **Seasonal Patterns**: Strong annual cycles in both variables + +### Causal Relationships +- **Temperature**: Strong bidirectional causality with phycocyanin +- **pH**: Significant cross-mapping relationships +- **Nutrients**: Complex interactions with bloom dynamics +- **Meteorological Forcing**: Weather patterns influence bloom development + +### Prediction Performance +- **Nonlinear Models**: Outperform linear benchmarks at multiple timescales +- **Optimal Embedding**: E=3-5 dimensions for most variables +- **Forecast Horizon**: Skillful predictions up to 7-10 days + +## Usage + +### Prerequisites +```r +# Required R packages +library(tidyverse) +library(rEDM) +library(googlesheets4) +library(oce) +library(padr) +library(ggplot2) +library(zoo) +library(quantreg) +library(forecast) +``` + +### Running Analyses + +1. **Data Wrangling**: Start with `2_FIGURES/00_data_wrangling.Rmd` +2. **Univariate Analysis**: Run `01_univariate_Draft2_EDM.Rmd` +3. **Cross-Mapping**: Execute `02_crossmap_EDM.Rmd` +4. **Multivariate Analysis**: Process `03_multivariate_EDM.Rmd` + +### Data Loading +The project includes automated data loading functions that: +- Check for local cached data files +- Download from GitHub if needed +- Fall back to Google Sheets for raw data +- Cache processed datasets for faster subsequent runs + +## Output Files + +### Figures +- **ACF/PACF Plots**: Autocorrelation analysis across time scales +- **EDM Results**: Simplex and S-Map performance comparisons +- **Cross-Mapping**: CCM convergence plots and significance tests +- **Multivariate**: Kitchen sink model performance + +### Tables +- **Summary Statistics**: Key findings and model performance metrics +- **Surrogate Tests**: Statistical significance of causal relationships +- **Parameter Estimates**: Optimal embedding dimensions and localization parameters + +## Research Applications + +This analysis framework can be applied to: +- **Water Quality Management**: Early warning systems for harmful algal blooms +- **Ecosystem Monitoring**: Understanding phytoplankton community dynamics +- **Climate Impact Assessment**: Evaluating environmental change effects +- **Predictive Modeling**: Forecasting bloom events for public health protection + +## Technical Notes + +### Data Quality +- Missing data handled through interpolation and gap-filling +- Outlier detection and removal procedures +- Seasonal decomposition for trend analysis + +### Model Validation +- Out-of-sample prediction testing +- Cross-validation procedures +- Surrogate data testing for significance + +### Computational Considerations +- Parallel processing for large surrogate ensembles +- Memory management for high-frequency data +- Efficient data storage using RDS format + +## Citation + +If you use this code or data in your research, please cite appropriately and acknowledge the data sources (EPA, MWRA, NOAA). + +## License + +This project is licensed under the terms specified in the LICENSE file. + +## Contact + +For questions about this analysis or collaboration opportunities, please contact the research team. + +--- + +*Last updated: 2024*