Import VectorSearchIndex from its canonical module

[jennsun]

Summary

context: https://databricks.slack.com/archives/C065NC65Q9F/p1780691337648119

Fix ImportError: cannot import name 'VectorSearchIndex' from 'databricks.vector_search.client' that breaks every consumer of databricks_openai.vector_search_retriever_tool (transitively: databricks_langchain, app-templates LangGraph template, etc.) once databricks-vectorsearch>=0.74 removes the client.py re-export.

Root cause

VectorSearchIndex has lived canonically in databricks.vector_search.index since databricks-vectorsearch==0.50 (verified against 0.50/0.55/0.68/0.73 wheels). It was importable from databricks.vector_search.client only as a side effect of client.py doing from databricks.vector_search.index import VectorSearchIndex near the top of its own module.

The new databricks-vectorsearch>=0.74 release (which now also pulls databricks-ai-search>=0.73 as a transitive dep) removed that side-effect import, so consumers that wrote from databricks.vector_search.client import VectorSearchIndex blow up — the bridge being one of them. (Aside: databricks-ai-search==0.73 exposes VectorSearchIndex as a back-compat alias for AISearchIndex via databricks.ai_search.index, but the rename doesn't matter for this fix — the canonical databricks.vector_search.index path keeps working on every released version of databricks-vectorsearch.)

Fix

Import VectorSearchIndex from its canonical location:

from databricks.vector_search.index import VectorSearchIndex

Applied in 5 files — 1 production module (integrations/openai/src/databricks_openai/vector_search_retriever_tool.py) and 4 test/test-util files that had the same broken import. Net diff: +6/−5.

Companion change in integrations/langchain/pyproject.toml: add databricks-openai = { path = "../openai", editable = true } to [tool.uv.sources] so the langchain test job uses the in-flight databricks-openai source (which has the fix) instead of the broken PyPI release, mirroring the existing databricks-ai-bridge source pin. This is a dev/CI-only directive — [tool.uv.sources] is stripped from published wheels, so end users still resolve databricks-openai from PyPI per the unchanged databricks-openai>=0.14.0 dep.

Reproduction (mimics the failure path the user hit on the LangGraph template)

python -m venv .venv && .venv/bin/pip install databricks-vectorsearch==0.74 databricks-openai

# Before this PR (legacy import path — user's traceback):
python -c "from databricks.vector_search.client import VectorSearchIndex"
# ImportError: cannot import name 'VectorSearchIndex' from 'databricks.vector_search.client'

# After this PR (the fixed module imports cleanly):
python -c "from databricks_openai.vector_search_retriever_tool import VectorSearchRetrieverTool"
# (no error)

For local reproduction without access to a 0.74 build, simulate the breakage on an older vectorsearch by importing the module and deleting the attribute: import databricks.vector_search.client as c; del c.VectorSearchIndex — same effect.

Validation

• All 53 unit tests in integrations/openai/tests/unit_tests/test_vector_search_retriever_tool.py pass on the fixed source under simulated 0.74 breakage (del databricks.vector_search.client.VectorSearchIndex injected via conftest).
• Static check: grep -rn "from databricks.vector_search.client import VectorSearchIndex" returns zero hits across the fixed codebase.
• Canonical path verified against databricks-vectorsearch==0.50/0.55/0.68/0.73 wheels — all have databricks.vector_search.index.VectorSearchIndex defined as a class.

Release implications

Once merged, this needs a databricks-ai-bridge patch release so downstream consumers (app-templates LangGraph template, databricks-langchain, databricks-openai) pick up the fix from PyPI without a git pin.

The companion app-templates change at databricks/app-templates#235 temporarily pins the langgraph templates at this branch's git URL so customers aren't blocked while the release goes out; that pin should be removed once the bridge release ships.

CI status

3 non-Required cross-version matrix jobs fail (openai_test (3.10, v0.3.0 / v0.4.0 / v0.5.0)); all Required checks are green.

These 3 failures pre-exist on main. Verified via a no-op baseline PR (#436): an empty commit off upstream/main triggers CI and reproduces 14 failures, all 3 of which are also failing here — plus 11 more that this PR fixes. So PR #435 is a strict improvement over current main state.

The matrix runs frozen historical sources from old tags against current PyPI and is hit by two unrelated upstream API drifts:

1. databricks-vectorsearch>=0.74 removed the VectorSearchIndex re-export from databricks.vector_search.client. Historical sources still import from that path. (Same bug this PR fixes for HEAD; the historical sources are frozen by tag and can't pick up the fix.)
2. mlflow 2.x removed mlflow.get_last_active_trace() in favor of get_last_active_trace_id(). Historical test code still calls the old name; the matrix's existing mlflow<3 pin is too loose.

Both predate this PR and are surfaced only because main hadn't re-run CI since 2026-05-13. Being handed off to the integration-tests owner to triage separately (options: tighter per-entry mlflow: pin, install each historical tag's uv.lock for exact dep parity, or retire the cross-version matrix).

Test plan

• CI: existing unit tests pass on this branch
• Local: uv sync in app-templates/agent-langgraph with the companion override and start the agent server — no ImportError on agent boot
• After merge + patch release: drop the app-templates override

cc @vadim-antonov (Databricks Vector Search) — heard you've been working on a related change on the VS side; please flag if there's something here that should be coordinated.