Skip to content

Add composite database indexes for gallery queries#8

Open
cassioeskelsen wants to merge 2 commits into
gosku:mainfrom
cassioeskelsen:perf/add-gallery-indexes
Open

Add composite database indexes for gallery queries#8
cassioeskelsen wants to merge 2 commits into
gosku:mainfrom
cassioeskelsen:perf/add-gallery-indexes

Conversation

@cassioeskelsen

Copy link
Copy Markdown

Summary

  • Add 5 composite indexes to Image and FujifilmRecipe models targeting the most frequent query patterns in gallery views
  • No behavior changes — indexes only improve query performance

Why

The gallery views use ORDER BY and WHERE patterns that currently require full table scans + in-memory sorts. With a catalog of ~10k images, these queries take 100-200ms. Composite indexes bring them down to ~5-10ms via index scans.

Indexes added

Index Model Covers
idx_image_taken_id Image Default gallery sort (-taken_at, id)
idx_image_rating_taken Image "Rating first" toggle (-rating, -taken_at, id)
idx_image_recipe_rating Image Per-recipe image pages (recipe_id, -rating, -taken_at)
idx_image_favorite_taken Image Favorites filter (is_favorite, -taken_at)
idx_recipe_film_sim FujifilmRecipe Graph/explorer film simulation lookups

Query patterns matched

# filter_queries.py / queries.py — default gallery
qs.order_by("-taken_at", "id")

# filter_queries.py / queries.py — "Rating first"
qs.order_by("-rating", "-taken_at", "id")

# queries.py — images for a specific recipe
qs.filter(fujifilm_recipe_id=X).order_by("-rating", "-taken_at", "id")

# recipes/queries.py — recipes by film simulation
FujifilmRecipe.objects.filter(film_simulation=X)

Test plan

  • All 1084 tests pass (indexes don't change behavior)
  • Verify migration applies cleanly on a fresh database
  • Verify gallery load time improves with a large catalog

🤖 Generated with Claude Code

cassioeskelsen and others added 2 commits April 10, 2026 20:29
The gallery views rely on several query patterns that currently
perform full table scans and in-memory sorts:

- Default gallery: ORDER BY taken_at DESC, id
- "Rating first" toggle: ORDER BY rating DESC, taken_at DESC, id
- Recipe image list: WHERE fujifilm_recipe_id = ? ORDER BY rating, taken_at
- Favorites filter: WHERE is_favorite = true ORDER BY taken_at DESC
- Recipe lookups by film simulation type

With ~10k images these queries take 100-200ms (seq scan + sort).
Composite indexes bring them down to 5-10ms (index scan).

Indexes added:
- idx_image_taken_id: covers default gallery sort
- idx_image_rating_taken: covers "Rating first" sort
- idx_image_recipe_rating: covers per-recipe image pages
- idx_image_favorite_taken: covers favorites filter
- idx_recipe_film_sim: covers recipe graph film simulation lookups

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@gosku gosku left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR! Good catch — I hadn't noticed the queries were getting that slow. In the past I'd implemented this kind of multi-filter gallery with Elasticsearch, where everything is indexed by default, so I never had to think about it explicitly.

The PR looks good overall. I've left some comments on the code and one about the structure of the PR in general:


(convention): The PR contains a no-op sync commit that introduces no changes:

14c408f — Merge branch 'main' into perf/add-gallery-indexes

This commit exists solely to sync the branch with main, which should be done with git rebase main instead. Please rebase and force-push to remove it before merging.

See: Do only one thing with each commit / Clean up mistakes by rewriting history

Comment thread src/data/models.py

class Meta:
indexes = [
models.Index(fields=["film_simulation"], name="idx_recipe_film_sim"),

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(suggestion): The FujifilmRecipe table tends to stay small relative to the image catalog — recipes are unique combinations of camera settings, so the cardinality is naturally limited. For a small table, PostgreSQL's query planner may prefer a sequential scan over an index lookup, making this index add write overhead without a measurable query benefit. Worth considering whether it's needed.



class Migration(migrations.Migration):

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(question)[non-blocking]: This migration is safe to skip for non-developer users since a missing index will not break the app. However, it is a good reminder that we need a way for users who just want to run the project to apply migrations easily — e.g. a make update command that runs git pull origin main followed by python manage.py migrate. Not required for this PR, but important to have before we ship any migration that adds or modifies fields or tables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants