Skip to content

Add time-range filtering to the runs endpoint#2

Open
Aryan95614 wants to merge 2 commits into
w2-status-filterfrom
w3-time-range
Open

Add time-range filtering to the runs endpoint#2
Aryan95614 wants to merge 2 commits into
w2-status-filterfrom
w3-time-range

Conversation

@Aryan95614

Copy link
Copy Markdown
Owner

Stacked on Netflix#481 (base is w2-status-filter). Retarget to Netflix master once Netflix#481 merges; the diff here is just the single time-range commit.

What

Adds optional ts_from / ts_to query params to GET /flows/{flow_id}/runs to scope runs by start time (ts_epoch), with inclusive epoch-millisecond bounds. Composes with the existing ?status= filter, so ?status=failed&ts_from=...&ts_to=... returns failed runs inside the window.

How

  • Refactors get_all_runs into a condition-builder: each supplied filter is one ANDed predicate in the same WHERE, applied before LIMIT/OFFSET so filters compose and ride on top of pagination. The zero-filter case keeps the original join-free path, leaving the unfiltered listing byte-identical.
  • ts_epoch is an indexed base column (runs_v3_idx_ts_epoch), so a time-range query runs join-free. Only status needs the lateral joins (enable_joins=bool(statuses)).
  • Non-integer bounds return 400 (fail loud for programmatic consumers); a window matching no rows returns 200 with an empty list. Values are bound as placeholders, never interpolated.

Tests

Integration tests for inclusive bounds, deterministic partitioning, non-integer rejection, and status plus time-range composition. Full integration suite green; the one unrelated failure is the pre-existing tasks_test flake.

Extend GET /flows/{flow_id}/runs with optional ts_from/ts_to query params
(inclusive epoch-millisecond bounds) to scope runs by start time.

Refactor get_all_runs into a condition-builder so each filter is one ANDed
predicate in a single WHERE, applied before LIMIT/OFFSET; filters compose and
stay correct under pagination. The zero-filter case keeps the original join-free
path, leaving the unfiltered listing byte-identical. ts_epoch is an indexed base
column, so a time-range query runs join-free; only status needs the lateral joins.

Non-integer bounds return 400; a window matching no rows returns 200 with an empty
list. Values are bound as placeholders, never interpolated.

Add integration tests for inclusive bounds, deterministic partitioning, non-integer
rejection, and status+time-range composition.
Add an optional 'user' query param to GET /flows/{flow_id}/runs, filtering by the
verified run owner. A run counts as a user's only when its system_tags carry
'user:<name>', reusing the ui_backend's exact user definition so the filter agrees
with what the UI shows (Step Functions runs, which lack the tag, read as no-user
and never match). That is also the expression runs_v3_idx_user_asc_ts_epoch_desc
is built on.

Drops in as one more ANDed predicate via the get_all_runs condition-builder, so it
composes with the existing status and time-range filters into compound filtering
(e.g. ?status=failed&user=alice&ts_from=...). Repeatable for OR. No joins needed;
system_tags and user_name are base columns. Values are bound as placeholders.

Add integration tests for the verified-user match, OR over users, exclusion of
unverified runs, and status+user+time composition.
Comment on lines +100 to +107
- name: "ts_from"
in: "query"
description: "only runs started at/after this epoch-millisecond timestamp (inclusive)."
required: false
type: "integer"
- name: "ts_to"
in: "query"
description: "only runs started at/before this epoch-millisecond timestamp (inclusive)."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be simpler to reuse the builtin_conditions_query_dict approach that is present in the ui_backend already. This would allow multiple operators for the field being queried, for example
ts_epoch:gt=123&ts_epoch:lt=567

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants