From f850b19645493bc6b0cfbf8e53bb3de2ea901d16 Mon Sep 17 00:00:00 2001 From: arnavsharma990 <2006arnavsharma@gmail.com> Date: Fri, 20 Feb 2026 12:30:52 +0530 Subject: [PATCH] docs: fix Phase 1 documentation issues - Fix typos in README.md: 'it easiest' -> 'it is easiest', 'handelled' -> 'handled', 'distrubted' -> 'distributed', 'varable' -> 'variable', 'gaurantee' -> 'guarantee' - Fix typo in CHANGELOG.md: 'ouput' -> 'output' - Fix CHANGELOG link text: '\25' -> '\#25' for PR #25 - Fix TODO comment typos in mapping.py: 'involved' -> 'involve', 'xrarrays' -> 'xarray's', 'lets' -> 'let's' - Fix docstring formatting: normalize Returns section underline in subsetting.py - Add missing Parameters/Returns sections to check_point_in_dataset and check_step - Fix incorrect Returns docstring in check_chunk_size (doesn't return ds) - Fix docstring parameter mismatch in map_dims_and_variables (remove incorrect arch_dim parameter) - Add Returns section to validate_config docstring Co-authored-by: Cursor --- CHANGELOG.md | 4 ++-- README.md | 10 +++++----- mllam_data_prep/config.py | 3 +++ mllam_data_prep/ops/chunking.py | 4 ++-- mllam_data_prep/ops/mapping.py | 4 +--- mllam_data_prep/ops/selection.py | 32 +++++++++++++++++++++++++++++-- mllam_data_prep/ops/subsetting.py | 2 +- 7 files changed, 44 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 846a62f..ce87a83 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [unreleased](https://github.com/mllam/mllam-data-prep/compare/v0.7.0...HEAD) ### Fixes -- fix bug where coordinate selection of an unshared dimension isn't applied to subsequent ouput variables when an output variable without this dimension is processed before the others [\#90](https://github.com/mllam/mllam-data-prep/pull/90) @zweihuehner & @leifdenby +- fix bug where coordinate selection of an unshared dimension isn't applied to subsequent output variables when an output variable without this dimension is processed before the others [\#90](https://github.com/mllam/mllam-data-prep/pull/90) @zweihuehner & @leifdenby ## [v0.7.0](https://github.com/mllam/mllam-data-prep/release/tag/v0.7.0) @@ -90,7 +90,7 @@ interface and addresses bugs around optional dependencies for ### Added -- add access to CLI via `mllam_data_prep` and add tests for CLI with/without `dask.distributed` ![\25](https://github.com/mllam/mllam-data-prep/pull/25). +- add access to CLI via `mllam_data_prep` and add tests for CLI with/without `dask.distributed` ![\#25](https://github.com/mllam/mllam-data-prep/pull/25). - add optional output path argument to parser. ![\#26](https://github.com/mllam/mllam-data-prep/pull/26) ### Changed diff --git a/README.md b/README.md index 9203101..b82a83c 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ python -m pip install mllam-data-prep[dask-distributed] ## Developing `mllam-data-prep` -To work on developing `mllam-data-prep` it easiest to install and manage the dependencies with [pdm](https://pdm.fming.dev/). To get started clone your fork of [the main repo](https://github.com/mllam/mllam-data-prep) locally: +To work on developing `mllam-data-prep` it is easiest to install and manage the dependencies with [pdm](https://pdm.fming.dev/). To get started clone your fork of [the main repo](https://github.com/mllam/mllam-data-prep) locally: ```bash git clone https://github.com//mllam-data-prep @@ -41,7 +41,7 @@ pdm use --venv in-project pdm install ``` -All the linting is handelled by `pre-commit` which can be setup to automatically be run on each `git commit` by installing the git commit hook: +All the linting is handled by `pre-commit` which can be setup to automatically be run on each `git commit` by installing the git commit hook: ```bash pdm run pre-commit install @@ -93,7 +93,7 @@ When you run the above command the console will print a URL to the dask dashboard, which you can open in a browser to monitor the progress of the dataset creation (and see the memory usage of the workers). -![example of using mllam-data-prep with dask.distrubted for parallel processing](docs/using_dask_distributed.png) +![example of using mllam-data-prep with dask.distributed for parallel processing](docs/using_dask_distributed.png) ### Usage as a python module @@ -395,7 +395,7 @@ The `inputs` section defines the source datasets to extract data from. Each sour - `function`: the function used to derive a variable. This should be a string with the full namespace of the function, e.g. `mllam_data_prep.ops.derived_variables.physical_field.calculate_toa_radiation`. - `kwargs`: arguments to `function`. This is a dictionary where each key is the named argument to `function` and each value is the input to the function. Here we distinguish between values to be extracted/selected from the input dataset and values supplied by the users themselves. Arguments with values to be extracted from the input dataset need to be prefixed with "ds_input." to distinguish them from other arguments. See the 'Derived Variables' section for more details. - `coord_ranges` - - `varable` : the variable to extract the coordinate from + - `variable` : the variable to extract the coordinate from `start`: start value of the variable `end`: end value of the variable @@ -439,4 +439,4 @@ Currently, the following derived variables are included as part of `mllam-data-p The schema version of the configuration file is defined by the `schema_version` attribute at the top of the configuration file. This is used to keep track of changes to the configuration file format. The schema version is used to check that the configuration file is compatible with the version of `mllam-data-prep` that you are using. If the schema version of the configuration file is not compatible with the version of `mllam-data-prep` that you are using you will get an error message telling you that the schema version is not compatible. -The schema version is updated whenever the configuration format changes, with the new schema version matching the minimum version of `mllam-data-prep` that is required to use the new configuration format. As `mllam-data-prep` is still in rapid development (and hasn't reached version `v1.0.0` yet) we unfortunately make no gaurantee about backward compatibility. However, the [CHANGELOG.md](CHANGELOG.md) will always contain migration instructions when the config format changes. +The schema version is updated whenever the configuration format changes, with the new schema version matching the minimum version of `mllam-data-prep` that is required to use the new configuration format. As `mllam-data-prep` is still in rapid development (and hasn't reached version `v1.0.0` yet) we unfortunately make no guarantee about backward compatibility. However, the [CHANGELOG.md](CHANGELOG.md) will always contain migration instructions when the config format changes. diff --git a/mllam_data_prep/config.py b/mllam_data_prep/config.py index b7a0ab4..f79ae6c 100644 --- a/mllam_data_prep/config.py +++ b/mllam_data_prep/config.py @@ -22,9 +22,12 @@ def validate_config(config_inputs): Parameters ---------- config_inputs: Dict[str, InputDataset] + Dictionary mapping input dataset names to their configuration Returns ------- + None + Raises InvalidConfigException if validation fails """ for input_dataset_name, input_dataset in config_inputs.items(): diff --git a/mllam_data_prep/ops/chunking.py b/mllam_data_prep/ops/chunking.py index 9df27e9..75d1444 100644 --- a/mllam_data_prep/ops/chunking.py +++ b/mllam_data_prep/ops/chunking.py @@ -19,8 +19,8 @@ def check_chunk_size(ds, chunks): Returns ------- - ds: xr.Dataset - Dataset with chunking applied + None + Logs a warning if chunk size exceeds CHUNK_MAX_SIZE_WARNING """ for var_name, var_data in ds.data_vars.items(): diff --git a/mllam_data_prep/ops/mapping.py b/mllam_data_prep/ops/mapping.py index 9482ff8..7cedc2e 100644 --- a/mllam_data_prep/ops/mapping.py +++ b/mllam_data_prep/ops/mapping.py @@ -45,8 +45,6 @@ def map_dims_and_variables(ds, dim_mapping, expected_input_var_dims): dim_mapping : dict The mapping of the input dimensions to the architecture dimensions. - arch_dim : str - The name of the architecture dimension to map to expected_input_var_dims : list The list of dimensions that each variable in the input dataset should have @@ -126,7 +124,7 @@ def map_dims_and_variables(ds, dim_mapping, expected_input_var_dims): combined_dim_name=arch_dim, ) else: - # TODO: this will have to involved xrarrays MultiIndex, but lets leave + # TODO: this will have to involve xarray's MultiIndex, but let's leave # this until we need it raise NotImplementedError(len(dims)) # set a flag we can use later to identify which coordinate the variables diff --git a/mllam_data_prep/ops/selection.py b/mllam_data_prep/ops/selection.py index 37b91c1..41c754d 100644 --- a/mllam_data_prep/ops/selection.py +++ b/mllam_data_prep/ops/selection.py @@ -94,7 +94,21 @@ def select_by_kwargs(ds, **coord_ranges): def check_point_in_dataset(coord, point, ds): """ - check that the requested point is in the data. + Check that the requested point is in the data. + + Parameters + ---------- + coord : str + Name of the coordinate to check + point : Any + The point value to check for in the coordinate + ds : xr.Dataset + Dataset containing the coordinate to check + + Returns + ------- + None + Raises ValueError if point is not found in the coordinate """ if point is not None and point not in ds[coord].values: raise ValueError( @@ -104,7 +118,21 @@ def check_point_in_dataset(coord, point, ds): def check_step(sel_step, coord, ds): """ - check that the step requested is exactly what the data has + Check that the step requested is exactly what the data has. + + Parameters + ---------- + sel_step : pd.Timedelta or datetime.timedelta + The requested step size + coord : str + Name of the coordinate to check + ds : xr.Dataset + Dataset containing the coordinate to check + + Returns + ------- + None + Raises ValueError if step size is not constant or doesn't match requested step """ all_steps = ds[coord].diff(dim=coord).values first_step = all_steps[0].astype("timedelta64[s]").astype(datetime.timedelta) diff --git a/mllam_data_prep/ops/subsetting.py b/mllam_data_prep/ops/subsetting.py index 80f2ce1..a3fd629 100644 --- a/mllam_data_prep/ops/subsetting.py +++ b/mllam_data_prep/ops/subsetting.py @@ -17,7 +17,7 @@ def extract_variable(ds, var_name, coords_to_sample=dict()): coordinates Returns - ---------- + ------- da: xr.DataArray Extracted variable (subsetted along the specified coordinates) """