Skip to content

Slice B+C — Short-name resolution via derivation + schema $id index #1783

Description

@KirylKurnosenka

Slice B+C — Short-name resolution via derivation + schema $id index

Part of #1781.

Goal

Make every materialized platform-bucket config entity short-name addressed,
inbound and outbound, via deterministic derivation (no per-entity flag, no stored
alias index). Add the $id index that lets blob-stored schemas resolve by their
JSON-Schema $id.

  • Inbound: gpt-4 resolves to the blob entity models/platform/gpt-4; canonical
    ids keep resolving as a harmless superset.
  • Outbound: listings/responses emit gpt-4, never models/platform/gpt-4.

Config.selectDeployment is the single deployment chokepoint, so inbound is a
one-place change; the rest is outbound naming, listing de-duplication, and a
call-site sweep.

Part B — short-name resolution (derivation)

config/.../Config.java

  • Add a resolve(entities, typeSegment, id) helper: map.get(id) (verbatim), else
    map.get(typeSegment + "/platform/" + id) (derived). Use string-literal type
    segments ("models", "applications", …) — the config module has no dependency
    on storage/ResourceTypes.
  • Rewrite selectDeployment to use resolve for applications, models,
    toolsets, interceptors (preserve precedence).
  • Add getRole(String), getInterceptor(String), getModel(String) accessors via
    resolve. (getModel is needed because ModelController must resolve specifically
    a Model; selectDeployment could return a same-named app.)

server/.../config/ConfigPostProcessor.java

  • Set entity.name = lastSegment(mapKey) for all materialized name-addressed types
    (uniform: a file key has no slash → identity; a canonical key → short name),
    instead of = mapKey. Add a lastSegment helper.

server/.../config/MergedConfigStore.java

  • Blob shadows file (unconditional): when adding a blob entity at rebuild, remove
    the same-short-named file entry (map.remove(lastSegment(canonicalId))) so
    .values() holds one entry per logical entity and short-name lookups resolve to
    blob. Mirror in putEntityInPlace / removeEntityInPlace for the partial-update
    path.

Call-site sweep (direct point-lookups that bypass selectDeployment)

Route these through the new accessors:

  • ModelgetModel / selectDeployment instead of getModels().get(id):
    ModelController.getModel, DeploymentController.
  • Roleconfig.getRole(userRole) instead of getRoles().get(...):
    RateLimiter, ShareService.
  • Interceptor existence/point lookups → config.getInterceptor(name) (i.e.
    getInterceptor(ref) != null for containsKey sites): DeploymentPostController,
    CollectResponseAttachmentsFn, ResourceController, BlobEntityValidator.

CRITICALConfigPostProcessor.validateCrossReferences checks each
model.interceptors ref with config.getInterceptors().containsKey(ref) inside the
rebuild
(via processModels, and on partial update via validateSingleModel). Once
an interceptor is a migrated blob entity (canonical key, file entry shadowed), a
containsKey(shortName) returns false, so a model referencing it by short name is
judged to have a dangling reference and skipped from the merged Config (or
aborts the rebuild in strict mode) — a self-inflicted outage on the migration path.
Change this to resolve-aware existence (config.getInterceptor(ref) != null). Safe
because the blob scan completes before the semantic pass runs
validateCrossReferences.

Internal split

  • B (models/interceptors/roles) — independent; these are already materialized.
  • B (applications/toolsets) — depends on Slice A (apps/toolsets must be
    materialized in platform first). Land this before migrating any app/toolset so
    they surface under short names.

Part C — schema $id index

Schemas (app-type + catalog) are referenced by the body-internal $id /
catalogSchemaId URI, not a derivable path segment, so derivation does not apply.

  • Add @JsonIgnore schemaAliasesById / catalogSchemaAliasesById to Config
    (default Map.of()), built at rebuild from blob bodies ($id → canonical id).
  • Extend Config.getCustomApplicationSchema(URI) / getCatalogSchema(URI) to fall
    back through these indexes (file entries are already keyed by $id via
    JsonArrayToSchemaMapDeserializer; the index covers blob entries).
  • This is always-on and independent of the bucket signal; outbound is a non-issue (the
    emitted identifier is the body-internal $id).

Design notes on maintaining the index:

  • Eviction on $id change. When a schema's $id is updated in place, the alias
    index evicts whatever it previously held for that canonical id before recording the
    new $id, so the old $id stops resolving as soon as the body no longer carries it
    — no dangling alias.
  • Uniqueness at write time. Before persisting a schema, the incoming $id is
    checked against the alias index; if it's already claimed by a different canonical
    id, the write is rejected (409 Conflict). This is wired independently into the
    three places a schema can be created or updated — ConfigResourceController.handlePut
    (single-entity PUT), AdminApplyController.applySchema (batch real-apply), and
    AdminApplyController.validateOnly (batch precheck) — including keeping
    AdminApplyController's in-flight scratch index in sync so two colliding Schema
    entries in the same /v1/admin/apply batch are caught at precheck time.

Compatibility note

Any entity already blob-stored in platform today flips its outbound name from
canonical to short; inbound canonical still works via derivation. Intended.

Testing

  • Unit — ConfigTest: resolve / selectDeployment / getRole /
    getInterceptor / getModel — verbatim (canonical) hit, derived (short-name) hit,
    miss. getCustomApplicationSchema / getCatalogSchema fall back through the $id
    index.
  • Unit — MergedConfigStoreTest: rebuild union (file-only, migrated blob with
    file shadowed) — one map entry per logical entity, blob wins, name = short name;
    the $id → canonical index is built for both schema types; short-name uniqueness is
    structural (blob path).
  • Unit — MergedConfigStorePartialUpdateTest / MergedConfigStoreReplicaUpdateTest:
    putEntityInPlace / removeEntityInPlace keep the file shadow consistent.
  • Unit — ConfigPostProcessorTest: name = lastSegment, and the critical
    validateCrossReferences fix (a model referencing a migrated interceptor by short
    name is not dropped).
  • Unit — RateLimiterTest, ShareServiceTest, ModelControllerTest,
    ToolSetControllerTest, BlobEntityValidatorTest, ApplicationSchemaServiceTest,
    CatalogSchemaServiceTest
    : resolve/emit by short name via the new accessors.
  • Integration — CanonicalIdListingTest (+ ConfigModelListTest / ListingTest
    / ApplicationDeploymentApiTest / CustomApplicationApiTest): end-to-end listings
    emit short names, no duplicates mid-migration. CanonicalIdListingTest is the
    existing canonical-id listing coverage and needs updating (migrated entities now
    surface as short names).
  • Backward-compat baseline: for not-yet-migrated (file-only) config, resolution is
    a verbatim hit and blob-shadows-file does not trigger, so the existing suite should
    stay green.

Depends on

Slice A for the applications/toolsets portion (models/interceptors/roles and the
schema $id index are independent of A).

Metadata

Metadata

Labels

enhancementNew feature or request

Projects

Status
Code Review

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions