Fix type issue in worker.py#512
Conversation
There was a problem hiding this comment.
Pull request overview
This PR focuses on eliminating Pyright type errors by tightening/clarifying typings around Luigi task status constants, config path typing, and flatten/mapping utilities used across worker/task execution and GCS metadata serialization.
Changes:
- Updated
gokart/worker.pyto import task status constants fromluigi.task_status, adjustedconfig_pathto TypedDict-friendly literals, and added type narrowing/casts around Luigiflatten()/dependency iteration. - Refined
FlattenableItemstyping ingokart/utils.py(explicitlist | tuple | dictrecursion) and aligned mapping logic with the narrowed union. - Added casts/narrowing in
gokart/task.pyandgokart/gcs_obj_metadata_client.pyto satisfy static typing for nested flattenable structures.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
gokart/worker.py |
Adjusts Luigi imports and adds type narrowing/casts around dependencies/outputs; updates Config config_path typing. |
gokart/utils.py |
Narrows FlattenableItems TypeAlias and updates map_flattenable_items branching accordingly. |
gokart/task.py |
Adds explicit narrowing/casts for requires() and required_task_outputs construction to satisfy static typing. |
gokart/gcs_obj_metadata_client.py |
Updates flattenable traversal typing for required task outputs serialization and adjusts accepted container types. |
Comments suppressed due to low confidence (1)
gokart/gcs_obj_metadata_client.py:136
- The TypeError message says "dict, or iterable" but this branch only accepts
list | tuple. Either broaden the implementation back tocollections.abc.Iterable(while still excludingstr) or update the error message to match the actually supported types so callers aren’t misled.
return {k: GCSObjectMetadataClient._get_serialized_string(v) for k, v in required_task_outputs.items()}
elif isinstance(required_task_outputs, list | tuple):
return [GCSObjectMetadataClient._get_serialized_string(ro) for ro in required_task_outputs]
else:
raise TypeError(
f'Unsupported type for required_task_outputs: {type(required_task_outputs)}. '
'It should be RequiredTaskOutput, dict, or iterable of RequiredTaskOutput.'
)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if not requires.complete(self.check_complete): | ||
| # not all requirements are complete, return them which adds them to the tree | ||
| new_deps = [(t.task_module, t.task_family, t.to_str_params()) for t in requires.flat_requirements] | ||
| new_deps = [(t.task_module, t.task_family, t.to_str_params()) for t in requires.flat_requirements if isinstance(t, Task)] |
There was a problem hiding this comment.
I also agree copilot changes, because this change will drop some tasks unexpectedly
There was a problem hiding this comment.
Thanks for the feedback! I agree that silently dropping non-Task elements is risky. I've changed this to use cast(list[Task], requires.flat_requirements) instead of the isinstance filter.
This keeps the runtime behavior identical to the original code (a non-Task element would still raise an AttributeError immediately) while satisfying the type checker.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| if not requires.complete(self.check_complete): | ||
| # not all requirements are complete, return them which adds them to the tree | ||
| new_deps = [(t.task_module, t.task_family, t.to_str_params()) for t in requires.flat_requirements] | ||
| new_deps = [(t.task_module, t.task_family, t.to_str_params()) for t in requires.flat_requirements if isinstance(t, Task)] |
There was a problem hiding this comment.
I also agree copilot changes, because this change will drop some tasks unexpectedly
hiro-o918
left a comment
There was a problem hiding this comment.
LGTM!
thank you for your PRs!
What I did
Resolve all Pyright type errors in
gokart/worker.py(23 → 0):DISABLED,DONE,FAILED,PENDING,UNKNOWNfromluigi.schedulertoluigi.task_statusconfig_path— Pyright treatsdict()calls asdict[str, str], which is not assignable toConfigPath(TypedDict). Dict literals{...}are correctly inferred as TypedDict-compatibleflatten()results — Luigi'sflatten()returnslist[Any], so elements can bestr. Addedcast(Task, ...)andisinstanceguards before accessing Task-specific attributes