Delete some solr updater dead code + small local solr-updater perf improvements

[cdrini]

We're going to be extending solr updater with a few different types of data soon; preparatory refactor to remove some dead weight before we extend it.

Technical

• Deleted old LegacyDataProvider which we only used for the local reindex, but can switch to BetterDataProvider there too!
• Tried to make it less likely to get stuck waiting for solr to come up, but not sure I helped there

Testing

✅ Tested locally that solr updater/make reindex works on volume-less run, and that edits are indexed as they happen.

Screenshot

Stakeholders

[Copilot]

Pull request overview

Prepares the Solr updater codepath for upcoming extensions by removing deprecated/unused updater pathways (legacy data provider + helper wrapper) and making small local-dev performance/reliability adjustments.

Changes:

• Removes the legacy Solr data provider path and the do_updates wrapper, renaming the default provider to DatabaseDataProvider.
• Avoids using the IA metadata__unlimited service flag in local dev (both Solr metadata fetch and lending browse URLs).
• Tweaks local reindex tooling/perf (batching in solr_updater, Makefile parallelization, Solr healthcheck timing).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
scripts/solr_updater/solr_updater.py	Adds typing + replaces web.group with itertools.batched; switches to update.update_keys.
openlibrary/tests/solr/test_data_provider.py	Updates tests to use DatabaseDataProvider.
openlibrary/solr/update.py	Removes dead do_updates and drops the “legacy” CLI/provider option.
openlibrary/solr/data_provider.py	Removes LegacyDataProvider, renames default provider to DatabaseDataProvider, and gates IA “service” param in local dev.
openlibrary/core/lending.py	Gates IA “service” param in local dev; removes rate_limit_exempt arg.
openlibrary/core/env.py	Adds LOCAL_DEV to the shared environment helper (get_ol_env()).
Makefile	Simplifies reindex-solr and parallelizes subject indexing.
compose.override.yaml	Increases Solr healthcheck start period/timeout for local reliability.

[cdrini]

This name is a bit misleading, since it doesn't only access the db, it basically accesses "normal" open library data sources available during prod and local dev, eg db and solr. Not sure what to call that!

[RayBB]

Naming seems hard. What you put seems fine. Only other thing I can think is maybe StandardDataProvider

[RayBB]

There's a lot of change going on here but I looked at it fairly closely and nothing jumped out at me as a problem. Also had the AIs give it a glance and they thought it all looked great too.

Unless there's anything in particular you're concerned about I think we should ship it!

[RayBB]

Naming seems hard. What you put seems fine. Only other thing I can think is maybe StandardDataProvider

[RayBB]

Nice! Did it help?

[cdrini]

I'm not if it helped but I did discover why it sometimes hangs. We add a script file to reset solr if the solr config had been modified? That broke in the solr upgrade to solr10 apparently, and in such a way that it hangs indefinitely. So if there's been a change to one of our solr configs, everyone's dev environments will stop because solr will hang and not start up.