UNOMI-139 UNOMI-880 UNOMI-878 UNOMI-884 UNOMI-904: multi-tenant core, unified cache, scheduler, migrations, V2 compatibility#757
Open
sergehuber wants to merge 12 commits into
Conversation
… unified cache, scheduler, migrations, V2 compat Why one PR These changes share the same execution model (tenant context, unified cache, scheduler, and persistence) and are wired through the same services, blueprints, and integration tests. Splitting them into separate commits would produce artificial intermediate states that neither compile nor pass ITs, and would force heavy merge or rebase work across the same files repeatedly. They are therefore delivered together as one coherent “3.1 platform slice” for review, even though the work maps to several JIRA tickets. This change set backports and wires the Unomi 3.1 multi-tenant platform work into the integration branch: tenant-aware execution, centralized caching, the new scheduler service, V3 migration assets, and optional V2 REST compatibility. Tracing and full type-validation services from unomi-3-dev are not included; call sites use ParserHelper / defensive handling where those stacks were removed for a stacked follow-up branch. UNOMI-139 Multi-tenancy - Introduce tenant model and APIs (Tenant, TenantService, API keys, security/audit interfaces, quotas, lifecycle hooks) and propagate tenantId on Item and related types. - Add ExecutionContextManager and thread-local tenant context so services, persistence, and REST layers resolve the active tenant consistently. - Adjust persistence (Elasticsearch/OpenSearch) and services to scope reads/writes and inheritance (e.g. system tenant vs tenant overrides). UNOMI-880 Unified multi-tenant caching - Add MultiTypeCacheService / CacheableTypeConfig and AbstractMultiTypeCachingService for shared cache lifecycle, bundle predefined items, periodic refresh, and statistics. - Migrate DefinitionsServiceImpl and SegmentServiceImpl (and related paths) onto the unified cache; remove duplicated ad-hoc cache listeners where superseded. - Add Karaf shell cache commands (unomi:cache) and align tests with empty stats / CSV output edge cases. UNOMI-878 Cluster-aware task scheduling - Replace legacy scheduler usage with the new SchedulerService API (scheduled tasks, task executors, persistence-backed vs in-memory tasks). - Add shell commands for task inspection and management; integrate periodic work (e.g. cache refresh, rules refresh, purges) with the new scheduler where backported from unomi-3-dev. - Update dependents (e.g. Groovy actions service) to schedule work through the new API instead of removed executor accessors. UNOMI-884 Migration to V3 - Add migration Groovy scripts and request bodies for 3.1.0 (tenant document IDs, system item id fixes, tenant initialization, legacy queryBuilder updates, profile visit fixes as applicable). - Extend migration configuration and tests (e.g. MigrationUtilsTest) to cover new steps aligned with multi-tenancy and index updates. UNOMI-904 V2 API compatibility mode - Add REST authentication configuration (Metatype) for toggling V2 compatibility and choosing the default tenant when the mode is on. - Implement AuthenticationFilter handling so legacy V2-style access patterns can be enabled during migration without forcing every client to API keys immediately. Reliability and tests - All integration tests (AllITs) pass in a green run; they remain flaky locally—roughly one in four full-suite runs still fails with unrelated timing or environment symptoms (shared Elasticsearch, indexing lag, suite order). A follow-up should harden CI or add targeted waits where needed; this commit does not claim zero flake rate. - Align condition/action resolution with HEAD + selective unomi-3-dev fixes (PropertyHelper for past-event numeric parameters, null conditionType in evaluation, optional action type resolution in the dispatcher, segment recalculation paths). - Stabilize integration tests: wait for segment visibility after GraphQL create, relax cache stats assertions when global stats are empty, adjust invalid-segment test when numeric strings are coerced without validation service. Docs: https://issues.apache.org/jira/browse/UNOMI-139 https://issues.apache.org/jira/browse/UNOMI-880 https://issues.apache.org/jira/browse/UNOMI-878 https://issues.apache.org/jira/browse/UNOMI-884 https://issues.apache.org/jira/browse/UNOMI-904
… on CI - PastEventConditionOSQueryBuilder: accept java.util.Date or String for fromDate/toDate (parity with PastEventConditionESQueryBuilder) to fix EventServiceIT past-event tests under OpenSearch. - itests: chmod -R ugo+rwx snapshots_repository after unzip on Unix so Elasticsearch in Docker (UID 1000) can verify FS snapshot repos on Linux runners (e.g. GitHub Actions).
Increase retries and delay for unomi:crud list after create/read so rule and goal list assertions tolerate persistence/cache lag on slow runners.
Run mikepenz/action-junit-report after every integration matrix leg (if: always), with update_check and per-matrix check_name, so a green re-run replaces the stale red JUnit check. Use fail_on_failure=false and require_tests=false; continue-on-error so missing XML cannot fail the job.
…' into UNOMI-139-UNOMI-880-multitenancy
…80-multitenancy # Conflicts: # api/src/main/java/org/apache/unomi/api/actions/ActionType.java # api/src/main/java/org/apache/unomi/api/conditions/ConditionType.java
…80-multitenancy # Conflicts: # .vscode/settings.json
…80-multitenancy # Conflicts: # tools/shell-dev-commands/pom.xml
…80-multitenancy # Conflicts: # extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/RouterCamelContext.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked PR (merge order)
UNOMI-888-import-export-javadocUNOMI-139-UNOMI-880-multitenancyThis PR is stacked: it should merge into
UNOMI-888-import-export-javadocfirst; after that PR is onmaster(or the next stack parent), this slice reduces to the multi-tenant / cache / scheduler / migration / V2-compat delta relative to that line.Merge-base with
origin/master(when last checked):c85fb8856— same tip asorigin/masterat fork time for the underlying stack; the branch contains prior stacked commits (UNOMI-920 → UNOMI-879 → UNOMI-888) before this work.Why a single PR
These changes share the same execution model (tenant context, unified cache, scheduler, and persistence) and are wired through the same services, blueprints, and integration tests. Splitting them into separate commits would produce artificial intermediate states that neither compile nor pass ITs, and would force heavy merge or rebase work across the same files repeatedly. They are therefore delivered together as one coherent “3.1 platform slice” for review, even though the work maps to several JIRA tickets.
This change set backports and wires the Unomi 3.1 multi-tenant platform work into the integration branch: tenant-aware execution, centralized caching, the new scheduler service, V3 migration assets, and optional V2 REST compatibility. Tracing and full type-validation services from unomi-3-dev are not included; call sites use ParserHelper / defensive handling where those stacks were removed for a stacked follow-up branch.
UNOMI-139 Multi-tenancy
UNOMI-880 Unified multi-tenant caching
UNOMI-878 Cluster-aware task scheduling
UNOMI-884 Migration to V3
UNOMI-904 V2 API compatibility mode
Reliability and tests
Docs: https://issues.apache.org/jira/browse/UNOMI-139
https://issues.apache.org/jira/browse/UNOMI-880
https://issues.apache.org/jira/browse/UNOMI-878
https://issues.apache.org/jira/browse/UNOMI-884
https://issues.apache.org/jira/browse/UNOMI-904