Skip to content

fix(pagination): stop *_all iterators repeating the first page (SCR-52)#7

Merged
0xGhostCasper merged 1 commit into
mainfrom
fix/scr-52-pagination-repeat-page
Jun 30, 2026
Merged

fix(pagination): stop *_all iterators repeating the first page (SCR-52)#7
0xGhostCasper merged 1 commit into
mainfrom
fix/scr-52-pagination-repeat-page

Conversation

@0xGhostCasper

Copy link
Copy Markdown
Member

What

The shared paginate() helper (used by every *_all iterator — search_all, get_followers_all, get_user_tweets_all, etc.) advanced even when the backend returned the same cursor it was just given. That re-fetches the page it already yielded — the customer-reported "search_all() repeats the first page" symptom.

Fix

Break the loop when next_cursor is falsy or equals the cursor we just sent:

next_cursor = response.get("next_cursor")
if not next_cursor or next_cursor == cursor:
    break
cursor = next_cursor

One-line guard at the shared chokepoint — fixes the repeat for all *_all methods regardless of which endpoint's backend returns a non-advancing cursor.

Test

  • New test_echoed_cursor_stops_iteration — mocks only two pages where page 2 echoes its cursor; a third fetch would raise StopIteration, so the test fails without the guard. All 14 pagination tests pass; ruff clean.

Version

Bumps 0.15.30.15.4 + CHANGELOG. (Publish only fires on GitHub release, not on merge.)

Closes SCR-52

The shared paginate() helper advanced even when the backend returned the
same cursor it was given, re-fetching the page just yielded (the
'repeats first page' symptom on search_all and the other *_all iterators).
Break when next_cursor is falsy OR equals the cursor we just sent.

Bumps 0.15.3 -> 0.15.4.
@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@0xGhostCasper
0xGhostCasper merged commit 3e89d40 into main Jun 30, 2026
7 checks passed
@0xGhostCasper
0xGhostCasper deleted the fix/scr-52-pagination-repeat-page branch June 30, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants