Skip to content

UNOMI-139 UNOMI-880 UNOMI-878 UNOMI-884 UNOMI-904: multi-tenant core, unified cache, scheduler, migrations, V2 compatibility#757

Open
sergehuber wants to merge 12 commits into
UNOMI-888-import-export-javadocfrom
UNOMI-139-UNOMI-880-multitenancy
Open

UNOMI-139 UNOMI-880 UNOMI-878 UNOMI-884 UNOMI-904: multi-tenant core, unified cache, scheduler, migrations, V2 compatibility#757
sergehuber wants to merge 12 commits into
UNOMI-888-import-export-javadocfrom
UNOMI-139-UNOMI-880-multitenancy

Conversation

@sergehuber
Copy link
Copy Markdown
Contributor

@sergehuber sergehuber commented May 14, 2026

Stacked PR (merge order)

Role Branch
Base (merge into) UNOMI-888-import-export-javadoc
Head (this PR) UNOMI-139-UNOMI-880-multitenancy

This PR is stacked: it should merge into UNOMI-888-import-export-javadoc first; after that PR is on master (or the next stack parent), this slice reduces to the multi-tenant / cache / scheduler / migration / V2-compat delta relative to that line.

Merge-base with origin/master (when last checked): c85fb8856 — same tip as origin/master at fork time for the underlying stack; the branch contains prior stacked commits (UNOMI-920 → UNOMI-879 → UNOMI-888) before this work.


Why a single PR

These changes share the same execution model (tenant context, unified cache, scheduler, and persistence) and are wired through the same services, blueprints, and integration tests. Splitting them into separate commits would produce artificial intermediate states that neither compile nor pass ITs, and would force heavy merge or rebase work across the same files repeatedly. They are therefore delivered together as one coherent “3.1 platform slice” for review, even though the work maps to several JIRA tickets.

This change set backports and wires the Unomi 3.1 multi-tenant platform work into the integration branch: tenant-aware execution, centralized caching, the new scheduler service, V3 migration assets, and optional V2 REST compatibility. Tracing and full type-validation services from unomi-3-dev are not included; call sites use ParserHelper / defensive handling where those stacks were removed for a stacked follow-up branch.

UNOMI-139 Multi-tenancy

  • Introduce tenant model and APIs (Tenant, TenantService, API keys, security/audit interfaces, quotas, lifecycle hooks) and propagate tenantId on Item and related types.
  • Add ExecutionContextManager and thread-local tenant context so services, persistence, and REST layers resolve the active tenant consistently.
  • Adjust persistence (Elasticsearch/OpenSearch) and services to scope reads/writes and inheritance (e.g. system tenant vs tenant overrides).

UNOMI-880 Unified multi-tenant caching

  • Add MultiTypeCacheService / CacheableTypeConfig and AbstractMultiTypeCachingService for shared cache lifecycle, bundle predefined items, periodic refresh, and statistics.
  • Migrate DefinitionsServiceImpl and SegmentServiceImpl (and related paths) onto the unified cache; remove duplicated ad-hoc cache listeners where superseded.
  • Add Karaf shell cache commands (unomi:cache) and align tests with empty stats / CSV output edge cases.

UNOMI-878 Cluster-aware task scheduling

  • Replace legacy scheduler usage with the new SchedulerService API (scheduled tasks, task executors, persistence-backed vs in-memory tasks).
  • Add shell commands for task inspection and management; integrate periodic work (e.g. cache refresh, rules refresh, purges) with the new scheduler where backported from unomi-3-dev.
  • Update dependents (e.g. Groovy actions service) to schedule work through the new API instead of removed executor accessors.

UNOMI-884 Migration to V3

  • Add migration Groovy scripts and request bodies for 3.1.0 (tenant document IDs, system item id fixes, tenant initialization, legacy queryBuilder updates, profile visit fixes as applicable).
  • Extend migration configuration and tests (e.g. MigrationUtilsTest) to cover new steps aligned with multi-tenancy and index updates.

UNOMI-904 V2 API compatibility mode

  • Add REST authentication configuration (Metatype) for toggling V2 compatibility and choosing the default tenant when the mode is on.
  • Implement AuthenticationFilter handling so legacy V2-style access patterns can be enabled during migration without forcing every client to API keys immediately.

Reliability and tests

  • All integration tests (AllITs) pass in a green run locally; they remain flaky —roughly one in four full-suite runs still fails with unrelated timing or environment symptoms (shared Elasticsearch, indexing lag, suite order). A follow-up should harden CI or add targeted waits where needed; this commit does not claim zero flake rate, but this is not specific to this PR, others without these changes have the same issue
  • Align condition/action resolution with HEAD + selective unomi-3-dev fixes (PropertyHelper for past-event numeric parameters, null conditionType in evaluation, optional action type resolution in the dispatcher, segment recalculation paths).
  • Stabilize integration tests: wait for segment visibility after GraphQL create, relax cache stats assertions when global stats are empty, adjust invalid-segment test when numeric strings are coerced without validation service.

Docs: https://issues.apache.org/jira/browse/UNOMI-139
https://issues.apache.org/jira/browse/UNOMI-880
https://issues.apache.org/jira/browse/UNOMI-878
https://issues.apache.org/jira/browse/UNOMI-884
https://issues.apache.org/jira/browse/UNOMI-904

… unified cache, scheduler, migrations, V2 compat

Why one PR
These changes share the same execution model (tenant context, unified
cache, scheduler, and persistence) and are wired through the same
services, blueprints, and integration tests. Splitting them into separate
commits would produce artificial intermediate states that neither compile
nor pass ITs, and would force heavy merge or rebase work across the same
files repeatedly. They are therefore delivered together as one coherent
“3.1 platform slice” for review, even though the work maps to several JIRA
tickets.

This change set backports and wires the Unomi 3.1 multi-tenant platform work
into the integration branch: tenant-aware execution, centralized caching,
the new scheduler service, V3 migration assets, and optional V2 REST
compatibility. Tracing and full type-validation services from unomi-3-dev
are not included; call sites use ParserHelper / defensive handling where
those stacks were removed for a stacked follow-up branch.

UNOMI-139 Multi-tenancy
- Introduce tenant model and APIs (Tenant, TenantService, API keys,
  security/audit interfaces, quotas, lifecycle hooks) and propagate
  tenantId on Item and related types.
- Add ExecutionContextManager and thread-local tenant context so services,
  persistence, and REST layers resolve the active tenant consistently.
- Adjust persistence (Elasticsearch/OpenSearch) and services to scope
  reads/writes and inheritance (e.g. system tenant vs tenant overrides).

UNOMI-880 Unified multi-tenant caching
- Add MultiTypeCacheService / CacheableTypeConfig and
  AbstractMultiTypeCachingService for shared cache lifecycle, bundle
  predefined items, periodic refresh, and statistics.
- Migrate DefinitionsServiceImpl and SegmentServiceImpl (and related
  paths) onto the unified cache; remove duplicated ad-hoc cache listeners
  where superseded.
- Add Karaf shell cache commands (unomi:cache) and align tests with empty
  stats / CSV output edge cases.

UNOMI-878 Cluster-aware task scheduling
- Replace legacy scheduler usage with the new SchedulerService API
  (scheduled tasks, task executors, persistence-backed vs in-memory tasks).
- Add shell commands for task inspection and management; integrate periodic
  work (e.g. cache refresh, rules refresh, purges) with the new scheduler
  where backported from unomi-3-dev.
- Update dependents (e.g. Groovy actions service) to schedule work through
  the new API instead of removed executor accessors.

UNOMI-884 Migration to V3
- Add migration Groovy scripts and request bodies for 3.1.0 (tenant
  document IDs, system item id fixes, tenant initialization, legacy
  queryBuilder updates, profile visit fixes as applicable).
- Extend migration configuration and tests (e.g. MigrationUtilsTest) to
  cover new steps aligned with multi-tenancy and index updates.

UNOMI-904 V2 API compatibility mode
- Add REST authentication configuration (Metatype) for toggling V2
  compatibility and choosing the default tenant when the mode is on.
- Implement AuthenticationFilter handling so legacy V2-style access
  patterns can be enabled during migration without forcing every client
  to API keys immediately.

Reliability and tests
- All integration tests (AllITs) pass in a green run; they remain flaky
  locally—roughly one in four full-suite runs still fails with unrelated
  timing or environment symptoms (shared Elasticsearch, indexing lag,
  suite order). A follow-up should harden CI or add targeted waits where
  needed; this commit does not claim zero flake rate.
- Align condition/action resolution with HEAD + selective unomi-3-dev
  fixes (PropertyHelper for past-event numeric parameters, null
  conditionType in evaluation, optional action type resolution in the
  dispatcher, segment recalculation paths).
- Stabilize integration tests: wait for segment visibility after GraphQL
  create, relax cache stats assertions when global stats are empty,
  adjust invalid-segment test when numeric strings are coerced without
  validation service.

Docs: https://issues.apache.org/jira/browse/UNOMI-139
      https://issues.apache.org/jira/browse/UNOMI-880
      https://issues.apache.org/jira/browse/UNOMI-878
      https://issues.apache.org/jira/browse/UNOMI-884
      https://issues.apache.org/jira/browse/UNOMI-904
@sergehuber sergehuber requested a review from Copilot May 14, 2026 06:08

This comment was marked as low quality.

sergehuber and others added 11 commits May 14, 2026 11:47
… on CI

- PastEventConditionOSQueryBuilder: accept java.util.Date or String for
  fromDate/toDate (parity with PastEventConditionESQueryBuilder) to fix
  EventServiceIT past-event tests under OpenSearch.
- itests: chmod -R ugo+rwx snapshots_repository after unzip on Unix so
  Elasticsearch in Docker (UID 1000) can verify FS snapshot repos on
  Linux runners (e.g. GitHub Actions).
Increase retries and delay for unomi:crud list after create/read so rule
and goal list assertions tolerate persistence/cache lag on slow runners.
Run mikepenz/action-junit-report after every integration matrix leg
(if: always), with update_check and per-matrix check_name, so a green
re-run replaces the stale red JUnit check. Use fail_on_failure=false and
require_tests=false; continue-on-error so missing XML cannot fail the job.
…80-multitenancy

# Conflicts:
#	api/src/main/java/org/apache/unomi/api/actions/ActionType.java
#	api/src/main/java/org/apache/unomi/api/conditions/ConditionType.java
…80-multitenancy

# Conflicts:
#	.vscode/settings.json
…80-multitenancy

# Conflicts:
#	tools/shell-dev-commands/pom.xml
…80-multitenancy

# Conflicts:
#	extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/RouterCamelContext.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants