Avoid rate-limiting when a dataset is already downloaded

[Kostis-S-Z]

Problem

download_dataset and load_dataset always called _get_download_plan, which hits the rate-limited POST /datasets/{id}/download endpoint, in order to obtain the archive filename and checksum. This happened even when no download was necessary (e.g. the archive was already present on disk). As a result, users re-running loads/downloads on cached datasets were getting rate-limited despite no data transfer taking place.

See #102

Solution

Source the filename, id, and checksum from GET /datasets/{id} (get_dataset_details) instead of from the download-session endpoint, and only create a download session when a download will actually happen. The existence check now runs before any call to the rate-limited endpoint.

Core changes

• New _download_dataset orchestrator (replaces old _resolve_and_execute_download_plan function) that owns the full flow:
  1. resolve dir
  2. check for cached archive
  3. (only if needed) create download session
  4. resume/overwrite handling
  5. execute
  6. finalize.
• _get_download_plan is now only called when there is no cached downloaded dataset available. Its signature takes a resolved target_filepath instead of a download_directory, and it no longer returns filename (dropped from DownloadPlan), since the filename now comes from dataset details.
• get_dataset_details is the single source of dataset metadata: it validates that filename/id are present
• resolve_dataset_id was removed.

How the flow changed

Before: resolve ID (GET) → always create download session (POST, rate-limited) → decide whether to download.

After: get details (GET, provides id + filename + checksum) → check if archive already exists → skip entirely if cached, otherwise create the download session (POST) and download.

Implications for users

• No more rate-limiting on cached datasets, meaning repeated load_dataset/download_dataset calls on already-downloaded archives no longer touch the rate-limited endpoint.
• No public / breaking API changes. get_dataset_details, download_dataset, load_dataset, and save_dataset_to_disk keep identical signatures

[moz-johanndiedrick]

This looks good to me! Nice improvements :)