Offline-First Learning - PWA Caching, Background Sync & Optimistic Updates

[hoangsonww]

🔎 Summary

Make the LMS truly usable with poor/no internet by implementing offline-first behavior: cache course/lesson content, queue user actions (enrollments, notes, quiz attempts) while offline, and sync in the background once the connection returns—without data loss or duplicates.

---

🎯 Goals

• Cache read-heavy routes (home, courses, lessons, quizzes) for offline browsing.
• Queue writes while offline (enroll, save progress, create notes, submit quiz attempts) with optimistic UI.
• Background sync + retry with backoff; idempotent server processing.
• Conflict handling & user feedback (synced / failed / needs review).
• Minimal server changes; reuse Redis for idempotency + ETag support.

🧱 Non-Goals (v1)

• Full media/offline video downloads.
• Realtime collaboration.
• Full OCR/attachments offline.

---

🖥️ UX

• Show an Offline badge when network is down.
• Toasts: “Saved locally • will sync when online.”
• “Sync Center” panel listing pending/failed actions with retry button.
• Lesson pages indicate cached vs. fresh (small icon).
• Quiz attempts: allow local submit; sync later; if answers changed on server (edge), mark “conflict”.

---

🧩 Frontend (Angular) Design

• Ensure PWA is enabled; extend ngsw-config.json:

  {
    "assetGroups": [{ "name": "app", "installMode": "prefetch", "resources": { "files": ["/**/*"] } }],
    "dataGroups": [
      {
        "name": "api-read",
        "urls": ["/api/courses/**", "/api/lessons/**", "/api/quizzes/**", "/api/categories/**"],
        "cacheConfig": { "maxSize": 200, "maxAge": "7d", "strategy": "freshness", "timeout": "5s" }
      }
    ]
  }

• IndexedDB stores via idb:

  • courses, lessons, quizzes, progress, notes
  • mutationsQueue (pending writes)

• Queue item shape:

  type Mutation = {
    id: string;                // uuid
    url: string; method: 'POST'|'PUT'|'PATCH'|'DELETE';
    headers?: Record<string,string>;
    body?: any;
    createdAt: number;
    attempts: number;
    idempotencyKey: string;    // same as id
  }

• Background Sync: register a Sync event ('lms-sync') in the service worker; drain mutationsQueue with exponential backoff.

• Optimistic UI: apply local state immediately; rollback if server rejects.

• Citations/ETag: read endpoints honor ETag; cache revalidation reduces payloads.

---

🗄️ Backend (Django DRF) Changes

• Idempotency: accept Idempotency-Key header on write routes; store a short-lived key in Redis to dedupe duplicates (TTL e.g. 24h).

  # pseudo
  key = request.headers.get('Idempotency-Key')
  if key and redis.setnx(f"idemp:{key}", "1"):
      redis.expire(f"idemp:{key}", 86400)
      response = process()
      # optionally store response snapshot
  else:
      return cached_response_or_409()

• ETag / If-None-Match: add ETag to read responses (hash of payload/version); 304 when unchanged.

• Versioning for conflict resolution: add version (int) on mutable models (notes, progress, quiz attempts). On update: require If-Match: <version>, else 412 Precondition Failed.

• Bulk submit endpoint for queued quiz attempts:

  • POST /api/quiz-attempts/bulk → [ { idempotencyKey, attempt } ]
  • Process atomically per item; return per-item status.

---

🔐 Security & Limits

• Validate payload sizes; cap queue length on client.
• Sanitize notes content server-side.
• Do not log sensitive content; redact IDs in error logs.
• Respect auth: offline queue should include token snapshot; refresh token on sync if needed.

---

📈 Observability

• Counters: offline_mutations_enqueued, synced_ok, synced_failed.
• Logs with idempotencyKey correlation.
• Simple /health/cache endpoint to verify Redis connectivity for idempotency.

---

✅ Acceptance Criteria

• App loads and navigates without network after first visit; courses/lessons render from cache.
• Performing actions offline shows success toasts and adds items to Sync Center.
• On reconnect, items auto-sync; UI reflects success/failure per item.
• Duplicate taps do not create duplicate server records (idempotency verified).
• Read endpoints return 304 with If-None-Match when unchanged.
• Tests cover offline queue, retries, idempotency, and conflicts.

---

⚙️ ENV / Config

• IDEMPOTENCY_TTL_SECONDS=86400
• ENABLE_ETAG=true
• SYNC_MAX_RETRIES=5
• SYNC_BACKOFF_BASE_MS=500

---

🧪 Testing

• Frontend (Cypress): simulate offline via cy.viewport/route2 + Cypress.automation('remote:debugger:protocol', ...).
• Unit: queue utils, backoff, SW registration, IndexedDB adapters.
• Backend (pytest): idempotency Redis tests; ETag/If-None-Match; version conflict (412) path.
• Integration: bulk quiz attempts with mixed success.

---

📝 Tasks

• FE: Extend ngsw-config.json for API caching (read endpoints).
• FE: IndexedDB layer (idb) and mutationsQueue.
• FE: Service Worker sync ('lms-sync') + exponential backoff.
• FE: Sync Center UI + toasts + offline banner.
• FE: Optimistic updates for notes/progress/enroll/quiz submit.
• BE: Redis idempotency middleware/util for DRF write routes.
• BE: Add ETag & If-None-Match to read serializers/views.
• BE: Add version field + If-Match handling on mutable models.
• BE: POST /api/quiz-attempts/bulk endpoint.
• Tests: FE unit/e2e; BE pytest for idempotency/etag/versioning.
• Docs: README “Offline-First” section with screenshots & limits.
• CI: run FE e2e in a job that toggles offline/online.

---

🚀 Rollout Plan

1. Ship read-only caching behind PWA_OFFLINE_READ=1.
2. Enable queue + idempotency for notes only; monitor.
3. Extend to enrollments/progress; finally quizzes.
4. Collect metrics & tune cache TTL/backoff.

---

This upgrade makes the LMS resilient on unstable networks and improves perceived speed, while keeping server changes minimal and safe via Redis-backed idempotency and ETags.