Validate Dag and task IDs in the Go SDK#69965
Conversation
|
PR description says "Adds the same validation to the Java SDK". |
You’re right, already fixed it. Thanks! |
1ce3197 to
d2b2962
Compare
There was a problem hiding this comment.
LGTM overall, thanks.
UPDATE:
As mentioned in #69937 (review), we will need to wait for the coordinator side first.
|
@Andrushika — There are 2 unresolved review thread(s) on this PR from @jason810496. Could you either push a fix or reply in each thread explaining why the feedback doesn't apply? Once you believe the feedback is addressed, mark the thread as resolved so the reviewer isn't re-pinged needlessly. Thanks! Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this two-stage triage process so that our maintainers' limited time is spent where it matters most: the conversation with you. |
There was a problem hiding this comment.
Hi @Andrushika,
After revisiting this, I'd like to make this best-effort build time validation instead of task runtime validation or dag processing side validation for both Go and Java SDK (You can raise PR for TS SDK as well if you like! or I can clean them up as well).
The reason is that the following snippet will validate the task runtime in server side and be served as source of truth for validation.
airflow/airflow-core/src/airflow/serialization/definitions/dag.py
Lines 637 to 648 in ffde962
For the a..b case, we can just raise the warning (since we can't trust the env / config at client side).
Please let me know if there's other direction that makes more sense to you.
Thanks.
|
Got it, thanks for revisiting! I will move the validation into build time as you described. Would be happy to also clean up ts-sdk after current work is done. |
|
@jason810496 A small question to clarify: since we are going to validate the IDs in build time, I am planning to implement it in Since you mentioned we should keep the validation best-effort, I want to confirm how strict the pack step should be. The I have two options and would like to ask your opinion:
Which one do you prefer? Thanks! |
There was a problem hiding this comment.
I would prefer to keep all of them as warnings and still allow building, because it can prevent drifting that TP mentioned.
Sure, let's go with returning warning for all the cases. Thanks! I don't have strong opinion on keeping it strict regarding the validation since it's best-effort anyway.
d2b2962 to
d177572
Compare
|
All tests passed, ready for review!
A little fact I found: On the current I guess we should also handle this in the core? If it is, I would be happy to open a follow-up PR! |
Validate Dag and task IDs in the Go SDK
Why
Go SDK currently accepts any string as Dag/task ID. Python SDK already validates this with validate_key (max 250 chars, only letters, digits, -, ., _), and ts-sdk added the same check in #69400.
What
Adds the same validation to the Go SDK, so invalid IDs fail early at registration.
related: #69400, #69937
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code Opus 4.8 following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.