Skip to content

fix: paginate mobile delta sync - #5

Open
djbclark wants to merge 1 commit into
sidinsearch:mainfrom
djbclark:dpr/mobile-delta-sync-pagination
Open

fix: paginate mobile delta sync#5
djbclark wants to merge 1 commit into
sidinsearch:mainfrom
djbclark:dpr/mobile-delta-sync-pagination

Conversation

@djbclark

@djbclark djbclark commented Jul 31, 2026

Copy link
Copy Markdown

Why this is useful

The mobile app should eventually reflect every server-side change, even after it has been offline or a large import or recategorization updates many records. Previously, delta sync requested only one bounded result set and then advanced its sync cursor. If more rows had changed than fit in that response, the remaining changes could be skipped permanently, leaving the mobile library incomplete or stale without an obvious error.

This change makes the client fetch every page before advancing its cursor. Stable ordering prevents rows with identical timestamps from being skipped or duplicated across page boundaries, and a failed page now fails the sync instead of being mistaken for successful completion.

Summary

  • Add offset pagination and exact has_more metadata to the backend delta-sync endpoint.
  • Use deterministic updated_at, shortcode ordering so rows with the same timestamp remain stable across pages.
  • Make the mobile client consume every delta page before advancing its sync cursor.
  • Propagate page-fetch failures so a partial sync cannot be mistaken for successful completion.

Verification

  • PYTHONPATH=backend <project-python> -m unittest backend.tests.test_sync_pagination -v — passed.
  • npx tsc --noEmit in superbrain-app — passed.
  • Final branch is one commit ahead of current upstream main; the regression test covers deterministic paging across equal timestamps.

Roadmap FYI

The broader plan for small SuperBrain contributions that do not depend on YouTube approving enhanced YouTube Data API access, including increased quota, is tracked in djbclark/superbrain#3. These contributions may support YouTube-related workflows, but OAuth subscription discovery, YouTube playlist management and backfill, and other approval-dependent API work are explicitly excluded from this PR and held back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant