Lazy service definitions

[Horacio Duran (perrito666)]

Summary

Adds lazy, per-service configuration loading to PyProvider.

Instead of parsing a single large YAML file during provider initialization, applications can split service definitions into individual YAML files backed by a small JSON manifest. Each service definition is parsed and cached only when that service is first requested.

Changes

• Added service_provider_from_directory(...) for lazy service configuration loading.
• Added LazyServiceConfig, a mapping-compatible configuration source.
• Added split_service_definitions(...) to generate:
  • manifest.json
  • One YAML file per service
• Uses readable service-name filenames when safe.
• Falls back to a deterministic SHA-256 filename for unsafe or excessively long service names.
• Removes stale generated service files listed in the previous manifest.
• Exposes loaded_service_names for diagnostics and verification.
• Preserves existing PyProvider behavior for:
  • Service references
  • Namespaces
  • Parent providers
  • Lazy service construction
  • get, set, reset, and attribute access
• Keeps service_provider_from_yaml(...) unchanged for backward compatibility.
• Bumps PyProvider to 2.1.0.

Generated Directory Format

app_services.d/
├── manifest.json
├── auth-helper.yaml
├── orders.creator.yaml
└── <sha256>.yaml

Example manifest:

{
  "format": 1,
  "provider_name": "core-provider",
  "services": {
    "auth-helper": "auth-helper.yaml",
    "orders.creator": "orders.creator.yaml"
  }
}

Provider initialization reads only manifest.json:

from pyrovider import service_provider_from_directory

provider = service_provider_from_directory("app_services.d")

A service definition is loaded and parsed on first use:

service = provider.get("auth-helper")

Motivation

Large service configuration files were parsed eagerly during every application worker startup, even though most services were not needed immediately.

The directory-backed provider moves that work to first use while preserving the existing service construction semantics.

Performance

Measured against ShipHero's configuration containing approximately 1,503 services:

Initialization mode	Median
Eager YAML	235 ms
Lazy manifest	1.5 ms

In containerized application startup measurements:

Service	Eager config load	Lazy manifest load
app-endpoints	460 ms	3.9 ms
public-api	417 ms	2.7 ms

The broader worker-startup measurements improved by approximately 1.4 to 1.9 seconds in single local runs, although some of that difference may be attributable to normal startup and filesystem variance.

Compatibility

Existing consumers using service_provider_from_yaml(...) are unaffected.

Adoption is explicit:

provider = service_provider_from_directory("app_services.d")

Applications can migrate independently without changing decorators or service lookup behavior.

Testing

• Added tests for lazy first-use parsing and caching.
• Added reference and namespace behavior coverage.
• Added deterministic hashed filename coverage.
• Added stale generated-file cleanup coverage.
• Full test suite: 46 passed
• Ruff: passed
• Formatting: passed
• Mypy: passed
• Source distribution and wheel builds: passed

Follow-Up

After publishing pyrovider==2.1.0, consuming applications should:

1. Update their PyProvider dependency.
2. Generate and commit their service definition directory.
3. Switch from service_provider_from_yaml(...) to service_provider_from_directory(...).
4. Remove any temporary eager-loading fallback after rollout.