feat: add namespaces support#2033
Conversation
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Secrets | View in Orca |
|
@copilot review |
Here's a summary of the review findings and fixes applied: CI Failure (already fixed in Additional fix ( All 12 unit tests pass, and both Code Review and CodeQL Security scans are clean. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev/1.38 #2033 +/- ##
===========================================
Coverage ? 86.66%
===========================================
Files ? 307
Lines ? 23416
Branches ? 0
===========================================
Hits ? 20294
Misses ? 3122
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@copilot review |
CI is green at HEAD (
Here's a summary of all the changes in the PR: New code
rbac/models.py
users
util.py
Tests
|
4d9912b to
6a2b620
Compare
Adds full CRUD support for the Weaviate namespaces feature (requires Weaviate 1.38.0+): create/get/list/delete namespace endpoints, RBAC manage_namespaces permission, namespace-scoped DB user creation, and UserDB.namespace field. Includes unit and integration test suites with a dedicated docker-compose fixture. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add namespaces_permissions=[] to all Role(...) constructions in integration/test_rbac.py to match the new required dataclass field - Wait for namespaces port (8094) in ci/compose.sh - Use _decode_json_response_list helper in namespaces.list_all - Round-trip via roles.get(...) in namespace permission integration tests - Use pass instead of return None in delete callback Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Agent-Logs-Url: https://github.com/weaviate/weaviate-python-client/sessions/35eb3c0e-1e3a-4ae7-bde7-6c6c481dbe5a Co-authored-by: jfrancoa <23482278+jfrancoa@users.noreply.github.com>
When a global user (operator) passes a namespaced collection name of the form "namespace:CollectionName" (required on namespace-enabled clusters), only the collection portion after the colon should be capitalized. The namespace prefix must stay lowercase as it follows [a-z][a-z0-9]*. This single-point fix covers all 30+ call sites that use _capitalize_first_letter, including collections.delete/get/exists, batch operations, RBAC permissions, backup/export, and filters. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds 15 mock-based tests in mock_tests/test_namespaces.py covering the
HTTP contract for the new module without requiring a Weaviate cluster.
Brings weaviate/namespaces/base.py from 33% to 100% coverage and exercises
the new code paths in weaviate/users/base.py (namespace request body and
UserDB.namespace response field).
Notable cases that catch real regressions:
- 404 on get -> None (guards ok_in=[200, 404])
- list_all on null/empty body -> [] (guards the "or []" fallback)
- Every public method asserts the 1.38.0 version guard (catches future
contributors lowering the minimum or adding a method without the guard)
- POST /v1/users/db/{id} body shape with and without namespace
- UserDB.namespace populated from server response and defaulting to None
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Capture and assert the create() request body is empty JSON, locking the contract that the namespace name lives in the URL path only - Annotate method_call parametrize argument as Callable - Use Dict[str, Any] instead of bare dict for captured request bodies Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The server returns 202 Accepted for DELETE /namespaces/{name} because
cleanup of classes, aliases, and namespaced users completes
asynchronously. The client was declaring ok_in=[204], so successful
deletions surfaced as errors even though the server-side delete went
through. Align the status code with the spec and document the async
behaviour, and update the mock test accordingly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The server gained an optional home_node on namespace create plus a new
PUT /namespaces/{name} to modify it, and a read-only state field on the
namespace object. Bring the client to parity:
- Namespace model gains home_node and state (Literal active/deleting)
- create() accepts optional home_node; new update() backed by PUT
- create/get/list/update share a _ns_from_dict parser
- sync/async .pyi stubs updated; NamespaceState exported from outputs
- mock, unit, and integration tests for the new fields and update()
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Order the ports following alphabetical order of docker-compose files. - Add verification of version in user creation if passing namespace.
Codecov flagged 71 missing patch lines, all in integration/test_namespaces.py. The namespaces integration tests are version-gated to Weaviate 1.38+, but the CI integration matrix only runs 1.36, so they are skipped and their bodies count as uncovered. The bare `--cov` in the pytest invocation pulls test files into the report, and .coveragerc only omitted *tests*/* (mock_tests/), missing integration/. Add *integration/* to the omit lists. Test files are not coverage subjects, and the coverage they contribute to weaviate/ (via --cov=weaviate) is unaffected. Production code (weaviate/namespaces/*) remains 100% covered by mock+unit tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
And add missing REPLICATION_MAXIMUM_FACTOR required for namespaces.
test_delete_namespace asserted that get() returns None immediately after delete(), but namespace deletion is asynchronous: delete() returns 202, the server marks the namespace state="deleting", and a background sweep removes it later (NAMESPACE_CLEANUP_INTERVAL, default 30s). Against a real 1.38 cluster get() therefore returned Namespace(state="deleting"), failing the assertion. Poll until the namespace is gone (asserting it stays in "deleting" while present) with a 60s timeout that tolerates the default sweep interval. Also set NAMESPACE_CLEANUP_INTERVAL=2s in the namespaces CI compose so the sweep runs quickly and the test finishes in ~1s instead of ~30s. Verified against a live 1.38.0-rc.0 cluster: all 9 namespace integration tests pass (delete completes in ~0.6s with the shortened interval). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The server no longer accepts a separate `namespace` field when creating a DB user; the namespace is now derived from a namespace-qualified id of the form "<namespace>:<user>" passed in the URL path (weaviate PR #11501, resolveUserKeyForCreate). Update the client to match: - users.db.create: remove the `namespace` parameter; always POST an empty body. The (optionally qualified) id is already carried in the path. Drop the now-unused 1.38 version guard tied to the namespace field. - sync/async .pyi stubs: update the create signature accordingly. - UserDB.namespace and get/list parsing are unchanged: the server still returns `namespace` in the user response for global operators (handlers_db_users.go). Tests: - mock_tests: replace the namespace-in-body tests with test_users_db_create_qualified_user_id_goes_in_path (asserts the qualified id lands in the URL path and the body stays empty) and test_users_db_create_posts_empty_body. These pin the wire contract and are the version-independent regression guard. - integration: create the namespaced user with the qualified id directly. - Bump WEAVIATE_138 to 1.38.0-rc.0-b9ea106, the first build containing PR #11501. Verified: mock + unit suites pass (34); 8/9 namespace integration tests pass against a live cluster. The qualified-create integration test requires the b9ea106 image, which was still publishing to Docker Hub at commit time; the server source at b9ea106 is git-verified to implement this contract and CI runs the live check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First published build containing the namespace-qualified user-create change (PR #11501). Verified end-to-end: all 9 namespace integration tests pass against this image, including test_create_namespaced_user with a "<namespace>:<user>" id. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
client.namespaceswith full CRUD:create,update,get,list_all,delete(requires Weaviate 1.38.0+)create(name=..., home_node=...)accepts an optionalhome_nodeto pin shard placement; when omitted the cluster auto-selectsupdate(name=..., home_node=...)(PUT /namespaces/{name}) to change a namespace's home node — affects future placement only, existing shards are not movedNamespacemodel now exposeshome_nodeand the read-onlystate(active/deleting), parsed from create/update/get/list responses;NamespaceStateexported fromweaviate.outputs.namespacesPermissions.namespaces(namespace=..., manage=True)RBAC factory andNamespacesAction.MANAGEenum, wired intoRoleparsingclient.users.db.create(user_id=..., namespace=...)to bind a DB user to a namespace;UserDB.namespacefield populated from server responsesci/docker-compose-namespaces.ymlfor integration test fixtures (ports 8094/50064, RBAC + dynamic DB users +NAMESPACES_ENABLED=true)test/test_namespaces.py), 16 mock (mock_tests/test_namespaces.py), 9 integration (integration/test_namespaces.py), all passingTest plan
pytest test/test_namespaces.py mock_tests/test_namespaces.py— unit + mock tests (no server required); mock tests pin the exact wire format for create (with/withouthome_node), update (PUT body + 200), get (404→None), list, and delete (202)WEAVIATE_VERSION=1.38.x docker compose -f ci/docker-compose-namespaces.yml up -dpytest integration/test_namespaces.py— integration tests covering create/update/get/list/delete namespace,home_nodecreate + update + read-back, namespaced user creation, and RBAC permission management🤖 Generated with Claude Code