docs(dataset): migrate WeatherDataset to Google-style and add type hints - #505
docs(dataset): migrate WeatherDataset to Google-style and add type hints#505yukthagangadhari5 wants to merge 8 commits into
Conversation
|
@yukthagangadhari5 please follow PR template |
kshirajahere
left a comment
There was a problem hiding this comment.
Thanks for the docstring/type-hint cleanup effort here. I think there is useful work in this PR, but I don’t think it is mergeable in the current form because it is carrying some non-doc behavior changes as well.
I reproduced this locally with:
pytest tests/test_datasets.py -k "ensemble or out_of_bounds or forecast_len or no_forcing or shorter_forcing" -q
which fails multiple current ensemble tests, including:
test_ensemble_len_scales_with_default_all_memberstest_ensemble_index_mapping_is_time_majortest_ensemble_forcing_uses_same_member_when_available
My suggestion would be:
- rebase onto current
main - keep only pure docstring/type-hint edits in this PR
- move the
interrogatehook to a separate PR if that is still wanted
I’ve left a few inline comments on the concrete spots where behavior changed.
|
|
||
| # check that there are enough forecast steps available to create | ||
| # samples given the number of autoregressive steps requested | ||
| if self.datastore.is_ensemble: |
There was a problem hiding this comment.
This seems to reintroduce pre-#332 behavior. WeatherDataset on main now iterates all ensemble members by default, whereas this branch warns that only the first member is used and later hardcodes ensemble_member=0 again below. I can reproduce current test failures from this revert (test_ensemble_len_scales_with_default_all_members, test_ensemble_index_mapping_is_time_major, test_ensemble_forcing_uses_same_member_when_available).
| else: | ||
| da_forcing = None | ||
| da_forcing = self.da_forcing | ||
| if da_forcing is not None and "ensemble_member" in da_forcing.dims: |
There was a problem hiding this comment.
This also looks like a revert of merged #332 behavior. Main now supports forcing data with its own ensemble_member dimension by selecting the matching member; raising NotImplementedError here breaks the current contract again.
| Args: | ||
| stage (str, optional): "fit", "test", or None. | ||
| """ | ||
| common_kwargs = { |
There was a problem hiding this comment.
WeatherDataModule still stores load_single_member, but setup() no longer forwards it into WeatherDataset(...). That makes the flag a no-op through the main DataModule path, which seems like an accidental functional change rather than part of the docstring migration.
| - id: flake8 | ||
| description: Check Python code for correctness, consistency and adherence to best practices | ||
| additional_dependencies: [Flake8-pyproject] | ||
| - repo: https://github.com/econchick/interrogate |
There was a problem hiding this comment.
This feels out of scope for this PR. Adding a repo-wide interrogate --fail-under=100 hook changes contributor workflow for the entire package, not just weather_dataset.py or the files touched here. If we want this policy, I think it should be proposed and landed separately.
| @@ -1,7 +1,8 @@ | |||
| # Standard library | |||
| import dataclasses | |||
| import argparse | |||
There was a problem hiding this comment.
This import looks unused, and the branch currently fails flake8 on touched files. I think the type-hint/docstring cleanup should probably be rebased and kept lint-clean before review, especially since the PR already has larger scope drift elsewhere.
|
Hi @kshirajahere, I have updated the PR description to follow the repository's official template. Thanks for pointing that out! |
|
@yukthagangadhari5 i reckon its still not in the "correct" PR template |
|
Follow this mate |
|
@yukthagangadhari5 I looked through your changes here and there seem to be changes to the type hints, the docstrings and the code. For everything related to docstrings please directly contribute to #196. I you still think there is a need to improve the type hints specifically, you can reduce this PR in scope and rebase it on the main branch. |
Description
This PR refactors
WeatherDatasetandWeatherDataModuleinneural_lam/weather_dataset.pyto use Google-style docstrings. This ensures compatibility withmkdocstringsand improves clarity for researchers.Key Changes
Args:,Returns:,Raises:).(B, T, L, C)to docstrings._get_dataloaderhelper inWeatherDataModuleto reduce code duplication.Type of change
How Has This Been Tested?
from neural_lam.weather_dataset import WeatherDataset.Checklist: