Skip to content

UHI use case: Krakow dataset implementation#113

Open
wkravchyk wants to merge 5 commits into
WUR-AI:developfrom
wkravchyk:feature/uhi-krakow
Open

UHI use case: Krakow dataset implementation#113
wkravchyk wants to merge 5 commits into
WUR-AI:developfrom
wkravchyk:feature/uhi-krakow

Conversation

@wkravchyk

Copy link
Copy Markdown

What does this PR do?

Implementation of the Kraków UHI dataset (90 m GSD grid with no time-series data, just one acquisition date with most intensive UHI effect: 2024-08-29):

  • added the script that generates model-ready Kraków dataset: src/data_preprocessing/make_model_ready_heat_krakow.py
  • added the script that does spatial stratification of the Kraków dataset into train/val/test splits: src/data_preprocessing/heat_krakow_spatial_splits.py
  • added HeatKrakowDataset class: src/data/heat_krakow_dataset.py
  • added config files for Kraków dataset data, metrics and experiments
  • added model-ready Kraków dataset .CSV file and .pth file: data/heat_krakow

All files were checked with pre-commit and passed all of the checks.

Before submitting

  • Did you make sure title is self-explanatory and the description concisely explains the PR?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you test your PR locally with pytest command?

Comment thread configs/experiment/heat_krakow_tabular.yaml

@vdplasthijs vdplasthijs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay @wkravchyk . All looks good, just wanted to double-check re the data. Either remove the csv from here (add it to your gitignore) and share it via Teams. Or if it's just a small file it's fine. What do you think is best?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We normally don't upload the data into the repo to keep the size down, except for the butterfly case so we have 1 working example. How large is the file?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(But all the data is on teams instead)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed the .CSV file with the tabular data, it is not that large (~22MB), but for now it is probably better to store it on Teams.

@wkravchyk

Copy link
Copy Markdown
Author

In the commit 41be611 I added experiment and model config files for Kraków UHI dataset that use Tessera embeddings as one of the modalities.

One more thing that I have changed is an override for setup_tessera function in HeatKrakowDataset - calling the function inherited from BaseDataset where self.records.pop(i) was mutating the list mid-iteration, caused it to skip every second missing file and eventually trigger a PyTorch DataLoader IndexError: list index out of range.
I am not sure if this is an issue connected with my approach to use Tessera in Krakow dataset implementation, but I changed only HeatKrakowDataset, so the fix is isolated to the Krakow implementation and won't disrupt other use cases relying on the base class.

Since this is still related to Kraków UHI dataset implementation I'm not creating a new PR, just committing here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants