diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6a434c79a..90a5649b5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,147 +6,179 @@ -# Contributing to openframe-oss-lib +# Contributing to OpenFrame OSS Lib -Thank you for contributing to **openframe-oss-lib**! This guide covers everything you need to know: code style, branching strategy, commit conventions, pull request process, and security requirements. +Thank you for contributing to `openframe-oss-lib`! This document covers everything you need to get started: code style conventions, branching strategy, commit message format, pull request process, and security guidelines. --- -## Community First +## πŸ“£ Community First -All contribution discussions happen on the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We do **not** use GitHub Issues or GitHub Discussions. +> **All discussions, questions, and feature requests** are managed on [OpenMSP Slack](https://www.openmsp.ai/). +> We do **not** use GitHub Issues or GitHub Discussions on this repository. -Before starting a large contribution: +[![Join OpenMSP Slack](https://img.shields.io/badge/Slack-OpenMSP-blue?logo=slack)](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) -1. Join the [OpenMSP Slack](https://www.openmsp.ai/) -2. Describe what you're planning in the `#openframe-dev` channel -3. Get feedback from maintainers before investing significant effort +| Channel | Purpose | +|---------|---------| +| `#dev-questions` | Technical questions and help | +| `#roadmap` | Feature requests and ideas | +| `#bugs` | Bug reports and triage | --- -## Development Environment Setup +## πŸš€ Getting Started + +Before contributing: + +1. Set up your [development environment](./docs/development/setup/environment.md) +2. Follow the [local development guide](./docs/development/setup/local-development.md) to build and test locally +3. Join the [OpenMSP Slack community](https://www.openmsp.ai/) for discussion and guidance + +--- + +## πŸ› οΈ Development Environment ### Prerequisites -| Tool | Minimum Version | -|------|----------------| -| Java (JDK) | 21 | -| Apache Maven | 3.9+ | -| Git | 2.x | -| Docker | 24.x | +| Tool | Version | +|------|---------| +| **Java (JDK)** | 21+ | +| **Apache Maven** | 3.8+ | +| **Git** | 2.x+ | +| **Docker** | 24.x+ (for integration tests) | +| **Node.js** | 18+ (for frontend-core only) | -### Initial Setup +### Setup ```bash # Clone the repository git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib -# Configure GitHub Packages credentials -# Add to ~/.m2/settings.xml (see Prerequisites guide) - # Build all modules (skip tests for speed) mvn install -DskipTests + +# Run unit tests for a specific module +mvn test -pl openframe-core + +# Run integration tests (requires Docker) +mvn verify -pl openframe-data-mongo-sync ``` -### Recommended IDE +### GitHub Packages Access -**IntelliJ IDEA** (Community or Ultimate) is the recommended IDE. After importing the project: +Add your credentials to `~/.m2/settings.xml`: -1. Set **Project SDK** to Java 21 -2. Enable **Annotation Processors** for Lombok (`Settings β†’ Compiler β†’ Annotation Processors`) -3. Enable Maven delegate: `Settings β†’ Build Tools β†’ Maven β†’ Runner β†’ Delegate IDE build/run actions to Maven` -4. Increase heap: `Help β†’ Change Memory Settings β†’ Xmx: 4096 MB` +```xml + + + + github + YOUR_GITHUB_USERNAME + YOUR_GITHUB_TOKEN + + + +``` --- -## Code Style and Conventions +## 🎨 Code Style and Conventions ### Java Style | Convention | Rule | |-----------|------| -| Indentation | 4 spaces (no tabs) | -| Line length | Max 120 characters | -| Imports | No wildcard imports; organize by static β†’ java β†’ jakarta β†’ spring β†’ other | -| Braces | Opening brace on same line | -| Naming | `camelCase` methods/fields, `PascalCase` classes, `UPPER_SNAKE` constants | - -### Lombok Usage +| **Lombok** | Use `@Data`, `@Builder`, `@Slf4j`, `@RequiredArgsConstructor` freely | +| **Immutability** | Prefer `final` fields; use Lombok `@Value` for pure DTOs | +| **Package structure** | Follow existing `com.openframe.*` package hierarchy | +| **Naming** | `*Service` for business logic, `*Repository` for data access, `*Controller` for REST, `*DataFetcher` for GraphQL | +| **Exception handling** | Throw from the `openframe-exception` hierarchy (`NotFoundException`, `BadRequestException`, etc.) | +| **Logging** | Use `@Slf4j` (Lombok); log at `DEBUG` for operations, `WARN`/`ERROR` for unexpected states | -Use Lombok to reduce boilerplate: +### Service Class Pattern ```java -// Prefer these annotations -@Data // getters, setters, equals, hashCode, toString -@Value // immutable value objects -@Builder // fluent builders -@RequiredArgsConstructor // constructor injection -@Slf4j // logging +@Slf4j +@Service +@RequiredArgsConstructor +public class DeviceService { + private final MachineRepository machineRepository; + private final TenantIdProvider tenantIdProvider; + + public Optional findDevice(String machineId) { + log.debug("Looking up device: {}", machineId); + return machineRepository.findByMachineId( + machineId, tenantIdProvider.getTenantId() + ); + } +} ``` -### Spring Boot Conventions - -Use constructor injection (via `@RequiredArgsConstructor`) over field injection: +### GraphQL DataFetcher Pattern ```java -// Correct: Constructor injection -@Service -@RequiredArgsConstructor -public class MyService { - private final MyRepository repository; - private final AnotherService anotherService; +@DgsComponent +public class DeviceDataFetcher { + + @DgsQuery + public GenericConnection devices( + @InputArgument DeviceFilterInput filter, + @InputArgument ConnectionArgs connectionArgs) { + // delegate to service layer + } } ``` -Prefer `@ConfigurationProperties` over `@Value` for configuration classes. - ### Multi-Tenancy Requirements -Every service and repository method that accesses tenant-scoped data **must** include `tenantId` in queries: +**Every new domain document must:** ```java -// Correct: Always scope to tenant -Optional findByIdAndTenantId(String id, String tenantId); +// 1. Implement TenantScoped +@Document(collection = "my_collection") +public class MyDocument implements TenantScoped { + @Id + private String id; -// Wrong: Missing tenant scope β€” never do this for tenant-scoped data -Optional findById(String id); + private String tenantId; // Required for all domain documents +} ``` -### Exception Handling +Also ensure a compound MongoDB index that includes `tenantId` is defined in the appropriate `MongoIndexConfig`. -Use the standard exception hierarchy from `openframe-exception`: +--- -```java -throw new NotFoundException("Organization not found: " + id); -throw new BadRequestException("Invalid email format"); -throw new ForbiddenException("Access denied for tenant: " + tenantId); -throw new ConflictException("Email already exists"); -throw new ValidationException("Required field missing: name"); -``` +## 🌿 Branch Naming -Never throw raw `RuntimeException` or `Exception` directly. +Use descriptive, hyphen-separated branch names with a type prefix: ---- +| Prefix | Use Case | Example | +|--------|---------|---------| +| `feat/` | New features | `feat/add-webhook-support` | +| `fix/` | Bug fixes | `fix/notification-pagination-cursor` | +| `chore/` | Maintenance tasks | `chore/upgrade-spring-boot-3.4` | +| `refactor/` | Code refactoring | `refactor/device-service-split` | +| `docs/` | Documentation changes | `docs/update-auth-flow-diagram` | +| `test/` | Test additions/fixes | `test/add-ticket-repository-it` | -## Branch Naming +```bash +# Create a feature branch +git checkout -b feat/add-device-tagging-bulk-update -| Type | Pattern | Example | -|------|---------|---------| -| Feature | `feature/` | `feature/add-nats-retry-logic` | -| Bug fix | `fix/` | `fix/tenant-context-not-cleared` | -| Refactor | `refactor/` | `refactor/notification-repository` | -| Documentation | `docs/` | `docs/update-kafka-readme` | -| Dependency updates | `deps/` | `deps/upgrade-spring-boot-3.4` | +# Create a fix branch +git checkout -b fix/cassandra-tenant-scope-query +``` --- -## Commit Message Format +## πŸ’¬ Commit Message Format -Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification: +Follow the **Conventional Commits** specification: ```text -(): +(): [optional body] @@ -159,202 +191,220 @@ Follow the [Conventional Commits](https://www.conventionalcommits.org/) specific |------|------------| | `feat` | New feature or capability | | `fix` | Bug fix | -| `refactor` | Code change that neither fixes a bug nor adds a feature | +| `chore` | Build, dependency, or tooling changes | +| `refactor` | Code restructure without behavior change | | `test` | Adding or updating tests | | `docs` | Documentation only changes | -| `chore` | Build system, dependency updates, CI changes | | `perf` | Performance improvements | +| `ci` | CI/CD pipeline changes | -### Scope +### Scope Examples -Use the module name (without the `openframe-` prefix) as scope: +| Scope | Module | +|-------|--------| +| `core` | `openframe-core` | +| `gateway` | `openframe-gateway-service-core` | +| `auth` | `openframe-authorization-service-core` | +| `api` | `openframe-api-service-core` | +| `data` | `openframe-data-mongo-*` | +| `stream` | `openframe-stream-service-core` | +| `management` | `openframe-management-service-core` | +| `frontend` | `openframe-frontend-core` | +| `security` | `openframe-security-core` | + +### Commit Examples ```text -feat(security-core): add PKCE utility for code challenge generation -fix(data-mongo-sync): resolve tenant context leak in batch operations -refactor(gateway-service-core): extract rate limit logic into service -test(data-nats): add integration test for notification broadcast -chore(deps): upgrade spring-boot to 3.3.2 -``` +feat(api): add bulk device tag assignment endpoint -### Example Commit +Adds GraphQL mutation for bulk assignment of tags to multiple +devices in a single operation. Includes DataLoader optimization +to prevent N+1 on tag resolution. +``` ```text -feat(authorization-service-core): add Microsoft SSO provider strategy +fix(data): correct cursor encoding for notification pagination -Implements MicrosoftClientRegistrationStrategy to support Microsoft -Entra ID (Azure AD) authentication flows alongside existing Google SSO. +The cursor codec was using unstable field order causing +inconsistent pagination results. Now uses deterministic +JSON serialization. +``` -Closes: #discussion in #openframe-dev slack +```text +chore(deps): upgrade Spring Boot to 3.3.1 + +Addresses CVE-2024-XXXX in spring-web. ``` --- -## Pull Request Process +## πŸ” Pull Request Process ### Before Opening a PR -- [ ] Build passes: `mvn install -DskipTests` -- [ ] Tests pass: `mvn test -pl ` -- [ ] No secrets committed β€” review all changed files -- [ ] Follows code style: Lombok, constructor injection, tenant scoping -- [ ] Edge cases covered: unit tests added for new logic +- [ ] Branch is up to date with `main` +- [ ] All unit tests pass: `mvn test` +- [ ] Relevant integration tests pass: `mvn verify -pl ` +- [ ] New code follows project conventions (Lombok, tenant scoping, etc.) +- [ ] New domain documents implement `TenantScoped` +- [ ] No secrets or hardcoded credentials in code +- [ ] Javadoc added for new public APIs -### PR Title Format +### PR Description Template ```text -feat(security-core): add PKCE utility for authorization flows -fix(data-mongo-sync): resolve tenant context leak -``` +## Summary +Brief description of what this PR does and why. -### PR Description Template +## Changes +- List key changes +- Affected modules: openframe-xxx, openframe-yyy -```markdown -## What does this PR do? +## Testing +- [ ] Unit tests added/updated +- [ ] Integration tests added/updated +- [ ] Manually tested against local stack -Brief description of the change. +## Related +Link to Slack discussion or related PR (if any). +``` -## Why? +### Review Checklist -Motivation for the change. +Reviewers verify: -## How was it tested? +- [ ] Code follows OpenFrame conventions (naming, Lombok, tenant scoping) +- [ ] New endpoints have appropriate `@PreAuthorize` or security config +- [ ] No N+1 query problems (DataLoaders used for related entity loading) +- [ ] Integration tests cover the happy path +- [ ] Error cases return appropriate exceptions from `openframe-exception` +- [ ] Multi-tenant isolation maintained (`tenantId` filters present) -- [ ] Unit tests added/updated -- [ ] Integration tests added/updated -- [ ] Manual testing performed +--- -## Checklist +## πŸ“¦ Adding a New Module -- [ ] No secrets in code or tests -- [ ] All new endpoints have authorization rules -- [ ] New DB queries are tenant-scoped -- [ ] Input validation on all new DTOs -- [ ] No breaking changes (or breaking changes are documented) -``` +When adding a new module to `openframe-oss-lib`: ---- +1. Add to parent `pom.xml` in the `` section +2. Inherit from parent: `openframe-oss-lib` +3. Add to `` in parent POM with `${revision}` version +4. Follow naming convention: `openframe--` +5. Include a `README.md` describing the module's purpose +6. Write at least one unit test before the first PR -## Testing Guidelines +--- -### Test Structure +## πŸ”’ Security Guidelines -```text -openframe-/ -└── src/ - └── test/java/com/openframe/... - β”œβ”€β”€ FooTest.java # Unit test (runs in mvn test) - └── FooIT.java # Integration test (runs in mvn verify) -``` +### Secrets Management -### Running Tests +Never hardcode secrets in source code or configuration files. Always use environment variables: ```bash -# Unit tests for a specific module -mvn test -pl openframe-core - -# Integration tests (requires Docker) -mvn verify -pl openframe-data-mongo-sync - -# Skip all tests during build -mvn install -DskipTests +# Correct: use environment variables +export SPRING_DATA_MONGODB_URI=mongodb://user:secret@host/db ``` -### Unit Test Template +Never commit `.env` files or any file containing credentials to version control. -```java -@ExtendWith(MockitoExtension.class) -class MyServiceTest { +### Multi-Tenant Safety Checklist - @Mock - private MyRepository repository; +For any new data-access code: - @InjectMocks - private MyService service; +- [ ] Document implements `TenantScoped` +- [ ] Repository queries include `tenantId` filter +- [ ] Custom repository implementations use `TenantIdProvider` +- [ ] Compound index includes `{ tenantId: 1, ... }` - @Test - void shouldReturnExpectedResult() { - // Given - when(repository.findById("id-1")).thenReturn(Optional.of(new MyEntity("id-1"))); +### Input Validation - // When - var result = service.getById("id-1"); +All API inputs must use Jakarta Bean Validation constraints: - // Then - assertThat(result).isPresent(); - assertThat(result.get().getId()).isEqualTo("id-1"); - } -} +```java +@NotNull +private String machineId; + +@Positive +private Integer timeoutSeconds; ``` -### Test Conventions +### API Key Best Practices -- Use **Given / When / Then** structure in test methods -- Test method names should describe behavior: `shouldReturnNotFoundWhenEntityDoesNotExist` -- Always clean up test data in `@AfterEach` for integration tests -- Integration tests use Testcontainers β€” Docker must be running +- Rotate API keys regularly using `ApiKeyController` +- Store keys encrypted using `EncryptionService` from `openframe-core-crypto` +- Never log API key values β€” only log key IDs +- Apply appropriate `APIKeyType` scope --- -## Security Requirements +## πŸ§ͺ Testing Guidelines -Before opening a PR, verify: +### Test Structure -- [ ] No secrets, tokens, or keys in code or test fixtures -- [ ] All new endpoints have explicit authorization rules -- [ ] New database queries include `tenantId` scope -- [ ] Input validation annotations on all request DTOs -- [ ] Sensitive fields are encrypted at rest using `EncryptionService` -- [ ] No raw SQL/NoSQL string interpolation +| Pattern | Type | Requirement | +|---------|------|-------------| +| `*Test.java` | Unit test | No Docker; runs in `mvn test` | +| `*IT.java` | Integration test | Requires Docker; runs in `mvn verify` | -**For security vulnerabilities**, do **not** open a public GitHub issue. Contact the team via the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in a direct message to the maintainers, or visit [flamingo.run](https://flamingo.run) for the security contact. +### Unit Test Example ---- +```java +class SlugUtilTest { -## Adding a New Module + @Test + void shouldConvertNameToSlug() { + String name = "My MSP Organization"; + String slug = SlugUtil.toSlug(name); + assertThat(slug).isEqualTo("my-msp-organization"); + } +} +``` -When adding a new module to the library: +### Integration Test Example -1. Create the module directory following the naming convention: `openframe-/` -2. Add a `pom.xml` that inherits from the parent POM -3. Add the module to the parent `pom.xml` `` section -4. Add the module to `` in the parent with `${revision}` -5. Write unit tests before submitting -6. Update the README and architecture documentation +```java +@SpringBootTest +@Testcontainers +class OrganizationRepositoryIT extends BaseMongoIntegrationTest { -```xml - -openframe-my-new-module - - - - com.openframe.oss - openframe-my-new-module - ${revision} - -``` + @Autowired + private OrganizationRepository organizationRepository; ---- + @Test + void shouldSaveAndRetrieveOrganization() { + Organization org = new Organization(); + org.setTenantId("oss"); + org.setName("Test MSP"); + + Organization saved = organizationRepository.save(org); + + assertThat(saved.getId()).isNotNull(); + } +} +``` -## Versioning +### Speed Up Integration Tests -All modules are versioned together using `${revision}` in the parent POM. Version bumps are managed by the maintainers. Contributors do **not** need to update version numbers in PRs. +```bash +# Enable Testcontainers container reuse +echo "testcontainers.reuse.enable=true" >> ~/.testcontainers.properties +``` -Versioning follows [Semantic Versioning](https://semver.org/): +### Coverage Guidelines -- **Major:** Breaking API changes -- **Minor:** New backward-compatible features -- **Patch:** Backward-compatible bug fixes +| Code Type | Coverage Target | +|-----------|----------------| +| Domain services | 80%+ unit test coverage | +| Utility classes | 90%+ unit test coverage | +| Repository custom implementations | Integration test coverage | +| Controllers / DataFetchers | Integration test coverage | --- -## Getting Help - -Stuck? Reach out on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in `#openframe-dev`. +## πŸ“– Documentation -- 🌐 **OpenFrame Platform:** [https://openframe.ai](https://openframe.ai) -- πŸ’¬ **OpenMSP Community:** [https://www.openmsp.ai/](https://www.openmsp.ai/) -- 🦩 **Flamingo:** [https://flamingo.run](https://flamingo.run) +For deeper reference on development workflows, see the [Development Documentation](./docs/development/README.md). --- diff --git a/README.md b/README.md index 3ccf01ff2..487417f48 100644 --- a/README.md +++ b/README.md @@ -10,82 +10,105 @@ License

-# openframe-oss-lib +# OpenFrame OSS Lib -**openframe-oss-lib** is the foundational backend library powering the [OpenFrame](https://openframe.ai) platform β€” the AI-driven, open-source MSP infrastructure stack built by [Flamingo](https://flamingo.run). +**`openframe-oss-lib`** is the modular backbone of [OpenFrame](https://openframe.ai) β€” Flamingo's AI-powered MSP platform that replaces expensive proprietary software with open-source alternatives enhanced by intelligent automation. -This repository provides the **shared Spring Boot libraries** that every OpenFrame service is built upon: multi-tenant authentication, API layers (REST + GraphQL), reactive gateway routing, event streaming, real-time messaging, polyglot persistence, and distributed management workflows. +This library provides the shared, composable building blocks that power the entire OpenFrame stack: from multi-tenant identity and OAuth2 authorization, to reactive APIs, event-driven stream processing, and an embeddable AI chat engine. -[![OpenFrame: 5-Minute MSP Platform Walkthrough - Cut Vendor Costs & Automate Ops](https://img.youtube.com/vi/er-z6IUnAps/maxresdefault.jpg)](https://www.youtube.com/watch?v=er-z6IUnAps) +> This repository does **not** ship a standalone application β€” it provides libraries and service-core modules that are embedded into deployable OpenFrame-compatible services. --- -## Features - -- **Multi-Tenant by Default** β€” Every module implements strict tenant isolation: per-tenant RSA key pairs, tenant-scoped cache keys, tenant-scoped scheduler locks, and tenant ID embedded in every JWT -- **Multi-Tenant OAuth2 Authorization Server** β€” Per-tenant RSA key pairs, JWT issuance, PKCE support, dynamic client registration, SSO integration (Google, Microsoft), and invitation-based onboarding -- **Relay-Compliant GraphQL + REST APIs** β€” Netflix DGS GraphQL with DataLoaders for N+1 prevention alongside versioned REST controllers; cursor-based pagination throughout -- **Reactive Edge Gateway** β€” Spring Cloud Gateway + WebFlux + Netty; multi-issuer JWT validation, API key rate limiting, WebSocket proxying, and tool upstream resolution -- **Event Ingestion & Streaming** β€” Kafka / Debezium CDC processing pipeline with tool-specific deserialization, event enrichment, unified event type normalization, and Kafka Streams enrichment -- **Real-Time Messaging** β€” NATS JetStream publish/subscribe for device and tool notifications; persist-first notification strategy with read-state tracking -- **Polyglot Persistence** β€” MongoDB (sync + reactive), Redis (tenant-aware caching), Apache Pinot (analytics), Apache Cassandra (log storage) -- **Distributed Schedulers** β€” ShedLock + Redis for cluster-safe distributed job scheduling; offline device detection, API key stats sync, MDM fleet setup -- **External REST API** β€” API key–authenticated integration surface for third-party tools with OpenAPI documentation -- **Tool SDKs** β€” First-class Java clients for Tactical RMM, Fleet MDM, and MeshCentral -- **Modular Maven Structure** β€” 30+ independent modules; include only what your service needs -- **Spring Boot 3.3 / Java 21** β€” Modern LTS Java with Spring Security OAuth2 and virtual thread readiness +[![Watch What's New in OpenFrame 0.7.8](https://img.youtube.com/vi/BQAjDB4ED2Y/maxresdefault.jpg)](https://www.youtube.com/watch?v=BQAjDB4ED2Y) --- -## Architecture +[![OpenFrame: 5-Minute MSP Platform Walkthrough - Cut Vendor Costs & Automate Ops](https://img.youtube.com/vi/er-z6IUnAps/hqdefault.jpg)](https://www.youtube.com/watch?v=er-z6IUnAps) + +--- + +## ✨ Key Features + +| Feature | Description | +|---------|-------------| +| **Multi-Tenant Identity** | OAuth2 Authorization Server with per-tenant RSA key pairs and OIDC support | +| **Reactive API Layer** | REST + GraphQL (Netflix DGS) with Relay-style pagination and DataLoader batching | +| **Gateway & WebSocket Routing** | Spring Cloud Gateway with JWT validation, rate limiting, and tool proxying | +| **MongoDB Data Layer** | Reactive + sync repositories, cursor pagination, multi-tenant scoping | +| **Kafka Stream Processing** | Debezium CDC ingestion, event normalization, and Kafka Streams enrichment | +| **Management & Operations** | Distributed schedulers, migration support (Mongock), NATS initialization | +| **Frontend UI & AI Chat** | Embeddable React AI assistant (Guide + Mingo modes), Kanban, Notifications | +| **Integrated Tool SDKs** | Native SDKs for MeshCentral, Tactical RMM, and Fleet MDM | +| **Shared API Contracts** | Centralized DTOs, filters, and pagination primitives for all services | + +--- + +## πŸ—οΈ Architecture + +The platform is structured as an event-driven, reactive microservice stack with strict multi-tenant isolation: ```mermaid flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service Core"] - Gateway --> ApiCore["API Service Core (GraphQL + REST)"] - - ApiCore --> Authz["Authorization Service Core (OAuth2 / JWT)"] - ApiCore --> Stream["Stream Service Core (Kafka / Debezium)"] - ApiCore --> Management["Management Service Core (Schedulers)"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - ApiCore --> Redis["Redis"] - ApiCore --> Pinot["Apache Pinot"] - Stream --> Kafka["Apache Kafka"] - Stream --> Cassandra["Apache Cassandra"] - Management --> NATS["NATS JetStream"] + Frontend["Frontend Core UI and Chat"] + Gateway["Gateway Service Core"] + Auth["Authorization Service Core"] + API["API Service Core (HTTP + GraphQL)"] + Management["Management Service Core"] + Data["Data Models and Repositories (MongoDB)"] + Stream["Stream Processing (Kafka)"] + Cassandra["Cassandra (Unified Event Log)"] + Tools["Integrated Tools (MeshCentral / Tactical RMM / Fleet MDM)"] + Redis["Redis (Cache + Distributed Locks)"] + NATS["NATS (Agent Messaging)"] + + Frontend --> Gateway + Gateway --> Auth + Gateway --> API + Gateway --> Tools + API --> Data + API --> Stream + Stream --> Cassandra + Stream --> Data + Management --> Data + Management --> Redis + Management --> NATS + Tools --> Stream ``` ---- - -## Technology Stack - -| Layer | Technology | -|-------|-----------| -| Language | Java 21 | -| Framework | Spring Boot 3.3 | -| Build Tool | Apache Maven 3.9+ | -| Auth | Spring Authorization Server, Spring Security OAuth2 | -| API | Netflix DGS (GraphQL), Spring MVC (REST) | -| Gateway | Spring Cloud Gateway + WebFlux + Netty | -| Persistence | MongoDB (sync + reactive), Redis, Cassandra, Apache Pinot | -| Messaging | Apache Kafka / Debezium CDC, NATS JetStream | -| Testing | JUnit 5, Testcontainers, RestAssured | -| Distributed Locking | ShedLock + Redis | -| Multi-tenancy | ThreadLocal tenant context, per-tenant RSA keys | +### Service Modules + +| Module | Role | +|--------|------| +| `openframe-gateway-service-core` | Reactive edge β€” JWT validation, rate limiting, WebSocket proxying, tool routing | +| `openframe-authorization-service-core` | Multi-tenant OAuth2/OIDC β€” JWT issuance, SSO (Google/Microsoft) | +| `openframe-api-service-core` | REST + GraphQL (Netflix DGS) β€” DataLoader batching, Relay pagination | +| `openframe-api-lib` | Shared DTOs, filter criteria, pagination primitives | +| `openframe-data-mongo-common` | MongoDB document models and base repositories | +| `openframe-data-mongo-sync` | Synchronous MongoDB repositories with custom cursor pagination | +| `openframe-data-mongo-reactive` | Reactive MongoDB repositories for auth flows | +| `openframe-data-redis` | API key stats, rate limiting, ShedLock distributed locking | +| `openframe-data-kafka` | Kafka producers with retry and recovery | +| `openframe-data-nats` | NATS pub/sub, agent notifications, command dispatch | +| `openframe-stream-service-core` | Debezium CDC ingestion, event normalization, Kafka Streams | +| `openframe-management-service-core` | Cluster coordination, tool lifecycle, schedulers, migrations | +| `openframe-client-core` | Device registration, authentication, tool installation | +| `openframe-security-core` | JWT primitives, `AuthPrincipal`, cookie service | +| `openframe-frontend-core` | React component library, AI chat engine (Guide + Mingo modes) | --- -## Quick Start +## πŸš€ Quick Start ### Prerequisites -- Java 21 JDK ([Eclipse Temurin 21](https://adoptium.net/) recommended) -- Apache Maven 3.9+ -- Docker 24.x (for integration tests) -- GitHub Personal Access Token (PAT) with `read:packages` scope +| Tool | Version | +|------|---------| +| **Java (JDK)** | 21+ | +| **Apache Maven** | 3.8+ | +| **Git** | 2.x+ | +| **Docker** | 24.x+ | +| **Node.js** | 18+ (for frontend-core only) | ### 1. Clone the Repository @@ -94,7 +117,7 @@ git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib ``` -### 2. Configure GitHub Packages +### 2. Configure GitHub Packages Access Add your GitHub credentials to `~/.m2/settings.xml`: @@ -104,45 +127,45 @@ Add your GitHub credentials to `~/.m2/settings.xml`: github YOUR_GITHUB_USERNAME - YOUR_GITHUB_PAT + YOUR_GITHUB_TOKEN ``` -### 3. Build the Library +Generate a GitHub Personal Access Token (PAT) with `read:packages` scope from [GitHub Settings β†’ Developer Settings](https://github.com/settings/tokens). + +### 3. Build All Modules ```bash +# Build all modules (skip tests for speed) mvn install -DskipTests ``` -### 4. Add a Module as a Dependency +### 4. Use a Module as a Dependency ```xml - - - - com.openframe.oss - openframe-oss-lib - 5.79.3 - pom - import - - - - - - + com.openframe.oss - openframe-api-service-core - - + openframe-core + 6.0.10 + +``` + +Or inherit the parent POM for aligned dependency management: + +```xml + + com.openframe.oss + openframe-oss-lib + 6.0.10 + ``` ### 5. Run Tests ```bash -# Unit tests for a specific module +# Unit tests (no Docker required) mvn test -pl openframe-core # Integration tests (requires Docker) @@ -151,50 +174,71 @@ mvn verify -pl openframe-data-mongo-sync --- -## Module Overview - -| Module | Description | -|--------|-------------| -| `openframe-core` | Core utilities, pagination, validation | -| `openframe-exception` | Standard exception hierarchy | -| `openframe-core-crypto` | Encryption utilities | -| `openframe-security-core` | JWT, PKCE, cookie service | -| `openframe-security-oauth` | OAuth2 BFF layer | -| `openframe-authorization-service-core` | Multi-tenant OAuth2 authorization server | -| `openframe-api-lib` | API contracts, filter DTOs, cursor pagination | -| `openframe-api-service-core` | REST + GraphQL API service layer | -| `openframe-gateway-service-core` | Reactive gateway, routing, security | -| `openframe-data-mongo-common` | MongoDB domain documents | -| `openframe-data-mongo-sync` | Synchronous MongoDB repositories | -| `openframe-data-mongo-reactive` | Reactive MongoDB repositories | -| `openframe-data-redis` | Redis tenant-aware cache | -| `openframe-data-kafka` | Kafka multi-tenant configuration | -| `openframe-data-nats` | NATS real-time messaging | -| `openframe-data-cassandra` | Cassandra log storage | -| `openframe-data-pinot` | Apache Pinot analytics queries | -| `openframe-management-service-core` | Distributed schedulers, startup initializers | -| `openframe-stream-service-core` | Kafka streams, Debezium event enrichment | -| `openframe-external-api-service-core` | External REST API for integrations | -| `sdk/fleetmdm` | Fleet MDM Java SDK | -| `sdk/tacticalrmm` | Tactical RMM Java SDK | +## 🧰 Technology Stack + +### Backend + +| Technology | Version | Role | +|-----------|---------|------| +| **Java** | 21 | Primary language | +| **Spring Boot** | 3.3.0 | Application framework | +| **Spring Cloud** | 2023.0.3 | Cloud-native patterns | +| **Netflix DGS** | 9.0.3 | GraphQL framework | +| **MongoDB** | Reactive + Sync | Primary data store | +| **Apache Kafka** | 3.x | Event streaming | +| **NATS** | 0.6.2+3.5 | Lightweight messaging | +| **Redis** | Spring Data Redis | Caching + distributed locks | +| **Apache Cassandra** | – | Event log storage | +| **Apache Pinot** | 1.2.0 | Analytics | +| **Lombok** | 1.18.30 | Boilerplate reduction | +| **Testcontainers** | 1.21.4 | Integration testing | + +### Frontend (`openframe-frontend-core`) + +| Technology | Role | +|-----------|------| +| **React** | UI component library | +| **TypeScript** | Type-safe UI development | +| **Tailwind CSS** | Utility-first styling | +| **NATS WebSocket** | Real-time chat transport | +| **SSE** | Guide mode AI chat transport | +| **Storybook** | Component development and documentation | +| **Vitest** | Unit testing | --- -## Documentation +## πŸ“š Documentation + +πŸ“– See the [Documentation](./docs/README.md) for comprehensive guides including: -πŸ“š See the [Documentation](./docs/README.md) for comprehensive guides covering architecture, setup, development workflows, and reference documentation. +- [Getting Started](./docs/getting-started/introduction.md) β€” Introduction and platform overview +- [Prerequisites](./docs/getting-started/prerequisites.md) β€” Environment setup +- [Quick Start](./docs/getting-started/quick-start.md) β€” Up and running in 5 minutes +- [First Steps](./docs/getting-started/first-steps.md) β€” Key things to explore +- [Architecture Overview](./docs/development/architecture/README.md) β€” System design and data flows +- [Contributing Guidelines](./docs/development/contributing/guidelines.md) β€” How to contribute --- -## Community +## 🀝 Community -All discussions, questions, and support happen on the **[OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA)**. We do not use GitHub Issues or GitHub Discussions. +All discussions, questions, and feature requests are managed on the **OpenMSP Slack** community. + +[![Join OpenMSP Slack](https://img.shields.io/badge/Slack-OpenMSP-blue?logo=slack)](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) + +- **Questions**: `#dev-questions` channel +- **Feature Requests**: `#roadmap` channel +- **Bug Reports**: `#bugs` channel + +> We do **not** use GitHub Issues or GitHub Discussions. Everything is managed on [OpenMSP Slack](https://www.openmsp.ai/). + +--- -- 🌐 **OpenFrame Platform:** [https://openframe.ai](https://openframe.ai) -- πŸ’¬ **OpenMSP Slack:** [https://www.openmsp.ai/](https://www.openmsp.ai/) -- 🦩 **Flamingo:** [https://flamingo.run](https://flamingo.run) +## πŸ”— Links -[![OpenFrame v0.5.2: Autonomous AI Agent Architecture for MSPs](https://img.youtube.com/vi/PexpoNdZtUk/maxresdefault.jpg)](https://www.youtube.com/watch?v=PexpoNdZtUk) +- [OpenFrame Platform](https://openframe.ai) +- [Flamingo](https://flamingo.run) +- [OpenMSP Community](https://www.openmsp.ai/) --- diff --git a/docs/README.md b/docs/README.md index f5d99817c..548152565 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,108 +1,95 @@ -# openframe-oss-lib Documentation +# OpenFrame OSS Lib β€” Documentation -Welcome to the documentation for **openframe-oss-lib** β€” the foundational backend library powering the [OpenFrame](https://openframe.ai) platform. +Welcome to the `openframe-oss-lib` documentation. This is the modular backbone of [OpenFrame](https://openframe.ai) β€” Flamingo's AI-powered MSP platform. -This section contains comprehensive guides covering getting started, development workflows, security practices, testing strategies, and full reference architecture documentation for every module. +> πŸ“£ All discussions, questions, and feature requests are managed on [OpenMSP Slack](https://www.openmsp.ai/). We do **not** use GitHub Issues or GitHub Discussions. --- ## πŸ“š Table of Contents -### Getting Started - -- [Introduction](./getting-started/introduction.md) β€” What is openframe-oss-lib, key features, and high-level architecture -- [Prerequisites](./getting-started/prerequisites.md) β€” Required software, Java setup, GitHub Packages access, IDE configuration -- [Quick Start](./getting-started/quick-start.md) β€” Clone, build, and add modules as dependencies in 5 minutes -- [First Steps](./getting-started/first-steps.md) β€” Module structure, domain model exploration, security modules, and gateway overview +- [Getting Started](#-getting-started) +- [Development](#-development) +- [Reference Architecture](#-reference-architecture) +- [Architecture Diagrams](#-architecture-diagrams) +- [Quick Links](#-quick-links) --- -### Development +## πŸš€ Getting Started + +New to `openframe-oss-lib`? Start here: -- [Development Overview](./development/README.md) β€” Index of all development documentation and quick navigation -- [Environment Setup](./development/setup/environment.md) β€” IDE configuration (IntelliJ / VS Code), Maven setup, Docker, environment variables -- [Local Development](./development/setup/local-development.md) β€” Clone, build, iterate, debug, and manage dependencies locally -- [Architecture Overview](./development/architecture/README.md) β€” System design, component relationships, data flows, and key design decisions -- [Security Best Practices](./development/security/README.md) β€” Auth patterns, multi-tenant key isolation, secrets management, input validation -- [Testing Overview](./development/testing/README.md) β€” Test structure, running tests, integration test infrastructure, writing new tests -- [Contributing Guidelines](./development/contributing/guidelines.md) β€” Code style, branching, commit conventions, and pull request process +| Document | Description | +|----------|-------------| +| [Introduction](./getting-started/introduction.md) | What OpenFrame OSS Lib is, key features, and the platform overview | +| [Prerequisites](./getting-started/prerequisites.md) | Required tools, system requirements, and infrastructure dependencies | +| [Quick Start](./getting-started/quick-start.md) | Clone, build, and use a module as a dependency in 5 minutes | +| [First Steps](./getting-started/first-steps.md) | Explore module structure, domain models, tests, and the community | --- -### Reference Architecture - -- [Repository Overview](./reference/architecture/README.md) β€” Full module index and end-to-end system view - -#### API Foundation -- [API Contracts and Pagination](./reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md) β€” Relay-style pagination, cursor codec, mutation inputs -- [API Domain Filters & DTOs](./reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md) β€” Strongly-typed filter DTOs for devices, events, logs, and organizations -- [API Lib Core Services](./reference/architecture/api-lib-core-services/api-lib-core-services.md) β€” Reusable domain services: tool connections, ticket queries, device status -- [API Organization Mapping](./reference/architecture/api-organization-mapping/api-organization-mapping.md) β€” Organization-level API mapping layer - -#### API Service Core -- [API Service Core Overview](./reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md) β€” User SSO services and processors -- [API Service Core Services](./reference/architecture/api-service-core-user-sso-services-and-processors/services.md) β€” Core service implementations -- [API Service Core Processors](./reference/architecture/api-service-core-user-sso-services-and-processors/processors.md) β€” Processor implementations -- [REST Controllers](./reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md) β€” Internal REST endpoints: organizations, devices, users, invitations, API keys -- [GraphQL Data Fetchers](./reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md) β€” Relay-compliant GraphQL execution layer -- [GraphQL DTOs](./reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md) β€” GraphQL input/output types -- [DataLoaders](./reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md) β€” Batched DataLoader implementations for N+1 prevention -- [Relay Type Resolution](./reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md) β€” Polymorphic Relay node type resolution -- [Config and Security](./reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md) β€” JWT resource server, multi-issuer support, OAuth client initialization - -#### Authorization Service Core -- [Server and Tenant](./reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md) β€” Multi-tenant OAuth2 authorization server, tenant discovery and registration -- [Auth Controllers and DTOs](./reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md) β€” Authentication controllers and data transfer objects -- [Keys and Authorization Persistence](./reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md) β€” Per-tenant RSA key pairs, JWT issuance, persistence -- [SSO Flow and Utils](./reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md) β€” Google and Microsoft SSO flows, PKCE support - -#### Gateway Service Core -- [Gateway Security and Routing](./reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md) β€” Reactive edge gateway, JWT validation, API key rate limiting, WebSocket proxying - -#### Stream Service Core -- [Kafka and Handlers](./reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md) β€” Debezium CDC ingestion, event enrichment, unified event type mapping - -#### External API Service Core -- [External API Service](./reference/architecture/external-api-service-core/external-api-service-core.md) β€” Public REST interface for third-party integrations - -#### Management Service Core -- [Initializers and Schedulers](./reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md) β€” Startup initializers, ShedLock distributed schedulers - -#### Security Core -- [Security Core and OAuth BFF](./reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md) β€” PKCE utilities, JWT encoder/decoder, OAuth BFF login flow - -#### Data Layer -- [MongoDB Domain Model](./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md) β€” Canonical MongoDB documents: User, Organization, Device, Ticket, Tool -- [MongoDB Base Repositories](./reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md) β€” Base repository interfaces and patterns -- [MongoDB Query Filters](./reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md) β€” MongoDB query filter construction -- [MongoDB Sync Config and Custom Repositories](./reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md) β€” Synchronous repositories, index configuration, custom queries -- [MongoDB Reactive Repositories](./reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md) β€” Reactive MongoDB repository layer -- [Redis Cache](./reference/architecture/data-redis-cache/data-redis-cache.md) β€” Tenant-aware cache key prefixing, Spring Cache integration -- [Kafka Configuration and Retry](./reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md) β€” Multi-tenant Kafka config, topic provisioning, retry handling -- [NATS Notifications](./reference/architecture/data-nats-notifications/data-nats-notifications.md) β€” Persist-first notification strategy, read-state tracking, NATS publishing -- [Pinot Repositories](./reference/architecture/data-pinot-repositories/data-pinot-repositories.md) β€” Apache Pinot analytics queries for logs and device facets +## πŸ› οΈ Development + +Guides for working with and contributing to the codebase: + +| Document | Description | +|----------|-------------| +| [Development Overview](./development/README.md) | Technology stack, module organization, and community links | +| [Environment Setup](./development/setup/environment.md) | IDE setup, Java 21, Maven, Docker, and Node.js installation | +| [Local Development](./development/setup/local-development.md) | Build commands, running tests, debugging, and Maven workflows | +| [Architecture Overview](./development/architecture/README.md) | High-level system design, request flows, and design decisions | +| [Security Guidelines](./development/security/README.md) | Auth architecture, secrets management, and secure coding practices | +| [Testing Guide](./development/testing/README.md) | Test structure, Testcontainers, writing unit and integration tests | +| [Contributing Guidelines](./development/contributing/guidelines.md) | Code style, branch naming, commit messages, and PR process | --- -### Architecture Diagrams +## πŸ“ Reference Architecture + +Deep-dive documentation for each major module, generated from source code analysis: + +| Module | Description | +|--------|-------------| +| [API Lib Contracts](./reference/architecture/api-lib-contracts/api-lib-contracts.md) | Shared DTOs, filter criteria, Relay pagination primitives, and mappers | +| [API Service Core (HTTP + GraphQL)](./reference/architecture/api-service-core-http-and-graphql/api-service-core-http-and-graphql.md) | REST controllers, GraphQL DGS DataFetchers, DataLoaders, and security | +| [Authorization Service Core](./reference/architecture/authorization-service-core/authorization-service-core.md) | Multi-tenant OAuth2/OIDC, JWT issuance, SSO (Google/Microsoft), Mongo persistence | +| [Gateway Service Core](./reference/architecture/gateway-service-core/gateway-service-core.md) | Reactive edge layer β€” JWT validation, API key auth, rate limiting, WebSocket proxying | +| [Data Models and Repositories (Mongo)](./reference/architecture/data-models-and-repositories-mongo/data-models-and-repositories-mongo.md) | MongoDB documents, reactive + sync repositories, cursor pagination, tenant isolation | +| [Management Service Core](./reference/architecture/management-service-core/management-service-core.md) | Operational control plane β€” schedulers, migrations (Mongock), NATS initialization | +| [Stream Processing (Kafka)](./reference/architecture/stream-processing-kafka/stream-processing-kafka.md) | Debezium CDC ingestion, event normalization, Kafka Streams enrichment, Cassandra persistence | +| [Frontend Core UI and Chat](./reference/architecture/frontend-core-ui-and-chat/frontend-core-ui-and-chat.md) | Embeddable AI assistant, Kanban board, Ticket Center, Notifications, DataTable system | + +--- -Visual Mermaid diagrams are available for every module under: +## πŸ—ΊοΈ Architecture Diagrams -```text -docs/diagrams/architecture/ -``` +Visual Mermaid diagrams are available in the `docs/diagrams/architecture/` directory. Each module has multiple diagrams covering data flows, component relationships, and sequence interactions: -Diagrams cover request flows, data flows, class relationships, and sequence diagrams for all major components. +| Diagram Set | Module | +|------------|--------| +| `docs/diagrams/architecture/README*.mmd` | Overall platform architecture | +| `docs/diagrams/architecture/api-service-core-http-and-graphql*.mmd` | API layer flows | +| `docs/diagrams/architecture/authorization-service-core*.mmd` | Auth and token flows | +| `docs/diagrams/architecture/gateway-service-core*.mmd` | Gateway routing and security | +| `docs/diagrams/architecture/data-models-and-repositories-mongo*.mmd` | Data layer architecture | +| `docs/diagrams/architecture/management-service-core*.mmd` | Management and scheduling flows | +| `docs/diagrams/architecture/stream-processing-kafka*.mmd` | Event ingestion pipeline | +| `docs/diagrams/architecture/frontend-core-ui-and-chat*.mmd` | UI and chat architecture | +| `docs/diagrams/architecture/api-lib-contracts*.mmd` | Contract layer relationships | --- -## πŸ“– Quick Links +## πŸ”— Quick Links -- [Project README](../README.md) β€” Main project overview and quick start -- [Contributing Guide](../CONTRIBUTING.md) β€” How to contribute to openframe-oss-lib -- [OpenFrame Platform](https://openframe.ai) β€” The OpenFrame MSP platform -- [OpenMSP Community (Slack)](https://www.openmsp.ai/) β€” Community discussions and support -- [Flamingo](https://flamingo.run) β€” The team behind OpenFrame +| Resource | Link | +|----------|------| +| **Project README** | [../README.md](../README.md) | +| **Contributing Guide** | [../CONTRIBUTING.md](../CONTRIBUTING.md) | +| **OpenFrame Platform** | [https://openframe.ai](https://openframe.ai) | +| **Flamingo** | [https://flamingo.run](https://flamingo.run) | +| **OpenMSP Community** | [https://www.openmsp.ai/](https://www.openmsp.ai/) | +| **Join Slack** | [OpenMSP Slack Invite](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) | --- diff --git a/docs/development/README.md b/docs/development/README.md index eb274e770..dcc95c0b7 100644 --- a/docs/development/README.md +++ b/docs/development/README.md @@ -1,104 +1,112 @@ # Development Documentation -Welcome to the **openframe-oss-lib** development documentation. This section covers everything you need to contribute to, extend, and maintain the OpenFrame OSS library. +Welcome to the `openframe-oss-lib` development documentation. This section covers everything you need to contribute to and work with the OpenFrame OSS library modules. ---- - -## Overview - -**openframe-oss-lib** is a Java 21 / Spring Boot 3.3 multi-module Maven library. It is the shared backend infrastructure stack for the OpenFrame MSP platform. The development section explains how to set up your environment, understand the architecture, write tests, secure your code, and contribute effectively. +[![OpenFrame v0.3.7 - Enhanced Developer Experience](https://img.youtube.com/vi/O8hbBO5Mym8/maxresdefault.jpg)](https://www.youtube.com/watch?v=O8hbBO5Mym8) --- -## Documentation Index +## Quick Navigation | Document | Description | |----------|-------------| -| [Environment Setup](./setup/environment.md) | IDE configuration, extensions, and dev tools | +| [Environment Setup](./setup/environment.md) | IDE, tools, and editor configuration | | [Local Development](./setup/local-development.md) | Clone, build, run, and debug locally | -| [Architecture Overview](./architecture/README.md) | System design, module relationships, data flows | -| [Security Best Practices](./security/README.md) | Auth patterns, secrets management, input validation | -| [Testing Overview](./testing/README.md) | Test structure, running tests, writing new tests | -| [Contributing Guidelines](./contributing/guidelines.md) | Code style, PR process, commit conventions | +| [Architecture Overview](./architecture/README.md) | System design, module relationships, and data flows | +| [Security Guidelines](./security/README.md) | Authentication, authorization, and secrets management | +| [Testing Guide](./testing/README.md) | Test structure, running tests, and writing new tests | +| [Contributing Guidelines](./contributing/guidelines.md) | Code style, branching, commit messages, and PR process | --- -## Quick Navigation - -### I want to... - -**Set up my local environment** -β†’ Start with [Environment Setup](./setup/environment.md), then [Local Development](./setup/local-development.md) - -**Understand how the system works** -β†’ Read the [Architecture Overview](./architecture/README.md) - -**Add a new feature or fix a bug** -β†’ Follow the [Contributing Guidelines](./contributing/guidelines.md) and review [Local Development](./setup/local-development.md) - -**Write tests for my changes** -β†’ See [Testing Overview](./testing/README.md) - -**Understand security requirements** -β†’ Read [Security Best Practices](./security/README.md) +## Technology Stack + +`openframe-oss-lib` is built on the following technology stack: + +### Backend +| Technology | Version | Role | +|-----------|---------|------| +| **Java** | 21 | Primary language | +| **Spring Boot** | 3.3.0 | Application framework | +| **Spring Cloud** | 2023.0.3 | Cloud-native patterns | +| **Netflix DGS** | 9.0.3 | GraphQL framework | +| **MongoDB** | Reactive + Sync | Primary data store | +| **Apache Kafka** | 3.x | Event streaming | +| **NATS** | 0.6.2+3.5 | Lightweight messaging | +| **Redis** | Spring Data Redis | Caching + distributed locks | +| **Apache Cassandra** | – | Event log storage | +| **Apache Pinot** | 1.2.0 | Analytics | +| **gRPC** | 1.58.0 | Internal service communication | +| **Lombok** | 1.18.30 | Boilerplate reduction | +| **JWT (jjwt)** | 0.11.5 | Token handling | +| **Testcontainers** | 1.21.4 | Integration testing | + +### Frontend (openframe-frontend-core) +| Technology | Role | +|-----------|------| +| **React** | UI component library | +| **TypeScript** | Type-safe UI development | +| **Tailwind CSS** | Utility-first styling | +| **NATS WebSocket** | Real-time chat transport | +| **SSE** | Guide mode AI chat transport | +| **Storybook** | Component development and documentation | +| **Vitest** | Unit testing | --- -## Tech Stack at a Glance - -| Layer | Technology | -|-------|-----------| -| Language | Java 21 | -| Framework | Spring Boot 3.3 | -| Build Tool | Apache Maven 3.9+ | -| Multi-tenancy | Thread-local tenant context, per-tenant RSA keys | -| Auth | Spring Authorization Server, Spring Security OAuth2 | -| Persistence | MongoDB (sync + reactive), Redis, Cassandra, Apache Pinot | -| Messaging | Kafka / Debezium CDC, NATS JetStream | -| Gateway | Spring Cloud Gateway + WebFlux + Netty | -| API | Relay-compliant GraphQL (Netflix DGS), REST | -| Testing | JUnit 5, Testcontainers, RestAssured | -| Distributed Locking | ShedLock + Redis | - ---- - -## Repository Structure - -```text -openframe-oss-lib/ -β”œβ”€β”€ pom.xml # Parent POM (unified versioning) -β”œβ”€β”€ openframe-core/ # Core utilities -β”œβ”€β”€ openframe-exception/ # Exception hierarchy -β”œβ”€β”€ openframe-core-crypto/ # Encryption -β”œβ”€β”€ openframe-security-core/ # JWT, PKCE, cookies -β”œβ”€β”€ openframe-security-oauth/ # OAuth2 BFF -β”œβ”€β”€ openframe-authorization-service-core/# Multi-tenant auth server -β”œβ”€β”€ openframe-api-lib/ # API contracts, DTOs -β”œβ”€β”€ openframe-api-service-core/ # REST + GraphQL API -β”œβ”€β”€ openframe-gateway-service-core/ # Reactive gateway -β”œβ”€β”€ openframe-client-core/ # Agent/client endpoints -β”œβ”€β”€ openframe-data-mongo-common/ # MongoDB documents -β”œβ”€β”€ openframe-data-mongo-sync/ # Sync repositories -β”œβ”€β”€ openframe-data-mongo-reactive/ # Reactive repositories -β”œβ”€β”€ openframe-data-redis/ # Redis cache -β”œβ”€β”€ openframe-data-kafka/ # Kafka configuration -β”œβ”€β”€ openframe-data-nats/ # NATS messaging -β”œβ”€β”€ openframe-data-cassandra/ # Cassandra storage -β”œβ”€β”€ openframe-data-pinot/ # Pinot analytics -β”œβ”€β”€ openframe-management-service-core/ # Schedulers, initializers -β”œβ”€β”€ openframe-stream-service-core/ # Kafka streams -β”œβ”€β”€ openframe-external-api-service-core/ # External REST API -β”œβ”€β”€ openframe-test-service-core/ # Integration test utilities -β”œβ”€β”€ sdk/ -β”‚ β”œβ”€β”€ fleetmdm/ # Fleet MDM SDK -β”‚ └── tacticalrmm/ # Tactical RMM SDK -└── ... +## Module Organization + +The repository follows a layered module structure: + +```mermaid +graph TD + subgraph foundation["Foundation"] + exception["openframe-exception"] + core["openframe-core"] + crypto["openframe-core-crypto"] + end + + subgraph data["Data Layer"] + common["openframe-data-mongo-common"] + sync["openframe-data-mongo-sync"] + reactive["openframe-data-mongo-reactive"] + redis["openframe-data-redis"] + kafka["openframe-data-kafka"] + nats["openframe-data-nats"] + cassandra["openframe-data-cassandra"] + pinot["openframe-data-pinot"] + end + + subgraph services["Service Cores"] + api["openframe-api-service-core"] + apilib["openframe-api-lib"] + auth["openframe-authorization-service-core"] + gateway["openframe-gateway-service-core"] + management["openframe-management-service-core"] + stream["openframe-stream-service-core"] + client["openframe-client-core"] + security["openframe-security-core"] + end + + subgraph tools["Tool SDKs"] + fleet["sdk/fleetmdm"] + tactical["sdk/tacticalrmm"] + tacticalSdk["openframe-tactical-sdk"] + end + + foundation --> data + data --> services + services --> tools ``` --- -## Community +## Development Community + +All development discussions happen on [OpenMSP Slack](https://www.openmsp.ai/). -All development discussions happen in the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We do not use GitHub Issues or GitHub Discussions. +- **Questions**: `#dev-questions` channel +- **Feature Requests**: `#roadmap` channel +- **Bug Reports**: `#bugs` channel -[![OpenFrame v0.5.2: Autonomous AI Agent Architecture for MSPs](https://img.youtube.com/vi/PexpoNdZtUk/maxresdefault.jpg)](https://www.youtube.com/watch?v=PexpoNdZtUk) +Join at: [OpenMSP Slack Invite](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) diff --git a/docs/development/architecture/README.md b/docs/development/architecture/README.md index e03e09358..3b9f4a713 100644 --- a/docs/development/architecture/README.md +++ b/docs/development/architecture/README.md @@ -1,199 +1,179 @@ # Architecture Overview -**openframe-oss-lib** implements a layered, multi-tenant service-oriented architecture. This document provides a high-level overview of the system design, component relationships, and key data flows. +`openframe-oss-lib` is the modular backbone of the OpenFrame platform. This document provides a high-level architectural overview, core component relationships, data flow patterns, and key design decisions. + +For detailed per-module documentation, see the [Reference Architecture docs](./reference/architecture/README.md). --- ## High-Level Architecture +The system is organized as a layered, event-driven microservice platform: + ```mermaid flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core\n(Spring Cloud Gateway + WebFlux)"] - Gateway --> ExternalAPI["External API Service Core\n(REST + API Keys)"] - Gateway --> ApiCore["API Service Core\n(REST + GraphQL)"] - - ApiCore --> Authz["Authorization Service Core\n(OAuth2 / JWT)"] - ApiCore --> Stream["Stream Service Core\n(Kafka / Debezium)"] - ApiCore --> Management["Management Service Core\n(Schedulers / Initializers)"] - ApiCore --> ClientSvc["Client Core\n(Agent Registration)"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - ApiCore --> Redis["Redis"] - Stream --> Kafka["Apache Kafka"] - Stream --> Pinot["Apache Pinot"] - Stream --> Cassandra["Apache Cassandra"] - Management --> NATS["NATS JetStream"] - Management --> Mongo + Frontend["Frontend Core UI and Chat"] + Gateway["Gateway Service Core"] + Auth["Authorization Service Core"] + API["API Service Core (HTTP + GraphQL)"] + Management["Management Service Core"] + Data["Data Models and Repositories (MongoDB)"] + Stream["Stream Processing (Kafka)"] + Cassandra["Cassandra (Unified Event Log)"] + Tools["Integrated Tools\n(MeshCentral / Tactical RMM / Fleet MDM)"] + Redis["Redis\n(Cache + Distributed Locks)"] + NATS["NATS\n(Agent Messaging)"] + + Frontend --> Gateway + Gateway --> Auth + Gateway --> API + Gateway --> Tools + API --> Data + API --> Stream + Stream --> Cassandra + Stream --> Data + Management --> Data + Management --> Redis + Management --> NATS + Tools --> Stream ``` --- -## Core Modules - -| Module | Responsibility | Key Technologies | -|--------|---------------|-----------------| -| `openframe-gateway-service-core` | Edge layer: JWT auth, API key rate limiting, WebSocket proxying, tool routing | Spring Cloud Gateway, WebFlux, Netty | -| `openframe-authorization-service-core` | Multi-tenant OAuth2 auth server: JWT issuance, SSO, per-tenant keys | Spring Authorization Server, Spring Security | -| `openframe-security-core` | JWT encoding/decoding, PKCE, cookie management | Nimbus JOSE, Spring Security | -| `openframe-security-oauth` | OAuth2 BFF: browser-facing login/callback/refresh/logout endpoints | Spring Security OAuth2 | -| `openframe-api-service-core` | Internal API: GraphQL (Relay) + REST controllers | Netflix DGS, Spring MVC | -| `openframe-api-lib` | API contracts: filter DTOs, cursor pagination, mutation types | Spring, Jackson | -| `openframe-external-api-service-core` | External API: API key–authenticated REST endpoints for integrations | Spring MVC, OpenAPI | -| `openframe-client-core` | Agent/device registration, tool agent endpoints | Spring MVC, NATS | -| `openframe-stream-service-core` | Kafka/Debezium CDC: event ingestion, normalization, enrichment | Kafka Streams, Spring Kafka | -| `openframe-management-service-core` | Startup initializers, distributed schedulers, tool orchestration | ShedLock, Spring Retry | -| `openframe-data-mongo-common` | MongoDB domain documents: canonical persistence model | Spring Data MongoDB | -| `openframe-data-mongo-sync` | Synchronous MongoDB repositories + index configuration | Spring Data MongoDB | -| `openframe-data-mongo-reactive` | Reactive MongoDB repositories | Spring Data MongoDB Reactive | -| `openframe-data-redis` | Tenant-aware Redis cache, reactive repositories | Spring Data Redis | -| `openframe-data-kafka` | Multi-tenant Kafka configuration, topic provisioning, retry | Spring Kafka | -| `openframe-data-nats` | NATS JetStream publishers, notification broadcasting | NATS Spring Cloud Stream | -| `openframe-data-cassandra` | Tenant-scoped Cassandra log storage | Spring Data Cassandra | -| `openframe-data-pinot` | Apache Pinot analytics queries | Pinot Java Client | -| `sdk/fleetmdm` | Fleet MDM Java client | Spring WebClient | -| `sdk/tacticalrmm` | Tactical RMM Java client | Spring WebClient | +## Core Components Table + +| Module | Type | Key Responsibility | +|--------|------|--------------------| +| `openframe-gateway-service-core` | Edge | JWT validation, rate limiting, WebSocket proxying, tool routing | +| `openframe-authorization-service-core` | Auth | Multi-tenant OAuth2/OIDC, JWT issuance, SSO (Google/Microsoft) | +| `openframe-api-service-core` | API | REST + GraphQL (Netflix DGS), DataLoader batching, Relay pagination | +| `openframe-api-lib` | Contracts | Shared DTOs, filter criteria, pagination primitives | +| `openframe-data-mongo-common` | Data Models | Document definitions, base repositories | +| `openframe-data-mongo-sync` | Persistence | Synchronous MongoDB repositories with custom queries | +| `openframe-data-mongo-reactive` | Persistence | Reactive MongoDB repositories for auth flows | +| `openframe-data-redis` | Cache | API key stats, rate limiting, ShedLock distributed locking | +| `openframe-data-kafka` | Messaging | Kafka producers with retry and recovery | +| `openframe-data-nats` | Messaging | NATS pub/sub, agent notifications, command dispatch | +| `openframe-stream-service-core` | Events | Debezium CDC ingestion, event normalization, Kafka Streams | +| `openframe-management-service-core` | Ops | Cluster coordination, tool lifecycle, schedulers, migrations | +| `openframe-client-core` | Agents | Device registration, authentication, tool installation | +| `openframe-security-core` | Security | JWT primitives, `AuthPrincipal`, cookie service | +| `openframe-frontend-core` | UI | React component library, AI chat engine (Guide + Mingo modes) | --- -## Data Flow: Request Processing +## Request Flow: Frontend to Data ```mermaid sequenceDiagram - participant C as Client - participant GW as Gateway - participant Auth as Auth Service - participant API as API Service - participant DB as MongoDB - - C->>GW: HTTP Request (with Bearer token or API key) - GW->>GW: Validate JWT (multi-issuer) / API key - GW->>API: Forwarded request + X-User-Id header - API->>DB: Query data (via Spring Data repositories) - DB-->>API: Domain documents - API-->>GW: Response - GW-->>C: HTTP Response + participant Browser as "Browser / Client" + participant Gateway as "Gateway Service" + participant Auth as "Auth Service" + participant API as "API Service" + participant Mongo as "MongoDB" + + Browser->>Gateway: HTTP Request (with JWT cookie) + Gateway->>Gateway: Extract JWT from cookie + Gateway->>Auth: Validate JWT (issuer check) + Auth-->>Gateway: Token valid + Gateway->>API: Forward request + Authorization header + API->>API: Resolve AuthPrincipal + API->>Mongo: Query with tenantId filter + Mongo-->>API: Result documents + API-->>Browser: JSON / GraphQL response ``` --- -## Data Flow: Authentication (OAuth2) +## Event Ingestion Flow (Kafka / Debezium) ```mermaid -sequenceDiagram - participant B as Browser - participant BFF as OAuth BFF - participant AuthSrv as Authorization Server - participant KS as TenantKeyService - participant DB as MongoDB - - B->>BFF: GET /oauth/login - BFF->>AuthSrv: Redirect (PKCE + state) - AuthSrv->>B: Login UI - B->>AuthSrv: Credentials - AuthSrv->>KS: Get tenant RSA key pair - KS->>DB: Load/create TenantKey - AuthSrv->>BFF: callback?code=... - BFF->>AuthSrv: Exchange code β†’ tokens - AuthSrv-->>BFF: JWT (access + refresh) - BFF->>B: Set HttpOnly cookies +flowchart LR + ToolA["MeshCentral / Tactical / Fleet"] --> Debezium["Debezium CDC"] + Debezium --> KafkaIn["Kafka Inbound Topics"] + KafkaIn --> Listener["JsonKafkaListener"] + Listener --> Processor["GenericJsonMessageProcessor"] + Processor --> Deserializer["Tool-Specific Deserializer"] + Deserializer --> Enrichment["IntegratedToolDataEnrichmentService"] + Enrichment --> Mapper["EventTypeMapper (UnifiedEventType)"] + Mapper --> Handler["DebeziumMessageHandler"] + Handler --> Cassandra["Cassandra (UnifiedLogEvent)"] + Handler --> KafkaOut["Outbound Kafka Topics"] ``` --- -## Data Flow: Event Streaming (Kafka / Debezium) +## Multi-Tenancy Model + +Multi-tenancy is enforced at the data layer through `TenantScoped`: ```mermaid -flowchart LR - subgraph Tools["Integrated Tools"] - T1["Tactical RMM"] - T2["Fleet MDM"] - T3["MeshCentral"] - end - subgraph Processing["Stream Service Core"] - L["Kafka Listener"] - D["Tool Deserializer"] - E["Enrichment Service"] - M["EventTypeMapper"] - H["Message Handler"] - end - subgraph Storage["Storage"] - C["Cassandra\n(UnifiedLogEvent)"] - K["Kafka\n(Enriched Topics)"] - end - - T1 --> L - T2 --> L - T3 --> L - L --> D - D --> E - E --> M - M --> H - H --> C - H --> K +flowchart TD + Request["Incoming Request"] --> TenantCtx["TenantContextFilter"] + TenantCtx --> TenantId["TenantContext (ThreadLocal)"] + TenantId --> Repo["Tenant-Scoped Repository"] + Repo --> Query["MongoDB Query with tenantId filter"] + Query --> MongoDB[("MongoDB")] ``` ---- +**Key principles:** +- Every domain document implements `TenantScoped` (has `tenantId` field) +- OSS deployments default `TENANT_ID` to `oss` +- SaaS deployments use per-request tenant resolution +- Each tenant has independent RSA key pairs for JWT signing -## Multi-Tenancy Design +--- -Every module implements strict tenant isolation: +## Security Architecture ```mermaid flowchart TD - Request["HTTP Request"] --> TenantFilter["TenantContextFilter\n(Extracts tenant ID)"] - TenantFilter --> ThreadLocal["TenantContext\n(ThreadLocal)"] - ThreadLocal --> KeyService["TenantKeyService\n(Per-tenant RSA keys)"] - ThreadLocal --> Repo["Tenant-Scoped Repositories"] - ThreadLocal --> LockKey["ShedLock Key\nof:{tenantId}:job-lock:..."] - ThreadLocal --> CacheKey["Redis Key\nof:{tenantId}:..."] + User["User"] -->|"Login"| AuthServer["Authorization Service Core"] + AuthServer -->|"Per-tenant RSA signing"| JWT["Tenant-Scoped JWT"] + JWT --> Gateway["Gateway Service Core"] + Gateway -->|"JWT validation + role check"| API["API Service Core"] + API -->|"AuthPrincipal injection"| Handler["Controller / DataFetcher"] ``` -Key multi-tenancy patterns: - -| Pattern | Implementation | -|---------|---------------| -| Tenant context propagation | `TenantContext` ThreadLocal, set by `TenantContextFilter` | -| Per-tenant JWT signing keys | `TenantKeyService` with RSA key pairs stored in MongoDB | -| Tenant-scoped cache keys | `OpenframeRedisKeyBuilder` prefixes every key with tenant ID | -| Tenant-scoped scheduler locks | ShedLock keys include `tenantId` and `environment` | -| JWT claim injection | `tenant_id`, `userId`, `roles` are embedded in every access token | +**Authentication layers:** +1. **Gateway**: validates JWT, extracts from cookie/header/query param +2. **Authorization Service**: issues tenant-scoped JWTs with custom claims (`tenant_id`, `userId`, `roles`) +3. **API Service**: OAuth2 Resource Server β€” verifies signatures, resolves `AuthPrincipal` +4. **External API**: API key authentication with per-minute/hour/day rate limiting --- ## Key Design Decisions -### 1. Reactive Gateway, Blocking Services +### 1. Module Composability +Each module is independently buildable and testable. Services consume only the modules they need rather than a monolithic dependency. -The gateway (`openframe-gateway-service-core`) is fully reactive (WebFlux + Netty). Internal services use Spring MVC (blocking), keeping service-level logic simple while the gateway handles high concurrency. +### 2. Relay-Style Pagination +The API layer uses cursor-based pagination (`ConnectionArgs`, `CursorCodec`, `GenericEdge`) following the Relay GraphQL specification for consistent, stable pagination across all list endpoints. -### 2. GraphQL + REST Coexistence +### 3. DataLoader Pattern (N+1 Prevention) +All GraphQL resolvers that load related entities use DataLoaders (`OrganizationDataLoader`, `MachineDataLoader`, etc.) to batch database calls and prevent N+1 query problems. -- **Internal API:** Relay-compliant GraphQL (Netflix DGS) with DataLoaders for N+1 prevention -- **External API:** REST with OpenAPI documentation and API key authentication -- Both APIs share the same domain services and repositories +### 4. Repository Abstraction +Base repository interfaces (`BaseUserRepository`, `BaseTenantRepository`) provide technology-agnostic contracts, with separate reactive (`openframe-data-mongo-reactive`) and synchronous (`openframe-data-mongo-sync`) implementations. -### 3. Event-Driven Normalization +### 5. Tool-Agnostic Event Model +All tool events (MeshCentral, Tactical RMM, Fleet MDM) are normalized to `UnifiedEventType` through the stream processing pipeline, giving a consistent event vocabulary across all integrations. -Integrated tools (Tactical RMM, Fleet MDM, MeshCentral) emit raw Debezium CDC events into Kafka. The Stream Service normalizes these into a unified `UnifiedEventType` before persisting to Cassandra or re-publishing. - -### 4. Startup Orchestration - -All bootstrapping logic is centralized in `openframe-management-service-core` using Spring `ApplicationRunner`. This ensures consistent initialization order across deployments. +### 6. Extensibility via Processor Hooks +Key lifecycle operations (agent registration, SSO configuration, user processing) expose processor interfaces (`AgentRegistrationProcessor`, `UserProcessor`) that can be replaced without modifying core logic. --- ## Reference Documentation -Detailed architecture documentation is available for each module: - -- [Gateway Service Core](./reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md) -- [Authorization Service Core](./reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md) -- [Stream Service Core](./reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md) -- [Management Service Core](./reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md) -- [External API Service Core](./reference/architecture/external-api-service-core/external-api-service-core.md) -- [Data Mongo Domain Model](./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md) -- [Data Kafka Configuration](./reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md) -- [Security Core and OAuth BFF](./reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md) +For deeper dives into each module, see the generated reference documentation: -[![Watch What's New in OpenFrame 0.7.8](https://img.youtube.com/vi/BQAjDB4ED2Y/maxresdefault.jpg)](https://www.youtube.com/watch?v=BQAjDB4ED2Y) +- [API Service Core (HTTP + GraphQL)](./reference/architecture/api-service-core-http-and-graphql/api-service-core-http-and-graphql.md) +- [Authorization Service Core](./reference/architecture/authorization-service-core/authorization-service-core.md) +- [Gateway Service Core](./reference/architecture/gateway-service-core/gateway-service-core.md) +- [Data Models and Repositories (Mongo)](./reference/architecture/data-models-and-repositories-mongo/data-models-and-repositories-mongo.md) +- [Stream Processing (Kafka)](./reference/architecture/stream-processing-kafka/stream-processing-kafka.md) +- [Management Service Core](./reference/architecture/management-service-core/management-service-core.md) +- [Frontend Core (UI and Chat)](./reference/architecture/frontend-core-ui-and-chat/frontend-core-ui-and-chat.md) +- [API Lib Contracts](./reference/architecture/api-lib-contracts/api-lib-contracts.md) diff --git a/docs/development/contributing/guidelines.md b/docs/development/contributing/guidelines.md index d12792a64..fc9086305 100644 --- a/docs/development/contributing/guidelines.md +++ b/docs/development/contributing/guidelines.md @@ -1,17 +1,18 @@ # Contributing Guidelines -Thank you for contributing to **openframe-oss-lib**! This guide covers code style, branching strategy, commit conventions, and the pull request process. +Thank you for contributing to `openframe-oss-lib`! This document covers code style, branching strategy, commit conventions, and the pull request process. --- -## Community First +## Getting Started -All contribution discussions happen on the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We do **not** use GitHub Issues or GitHub Discussions. +Before contributing: -Before starting a large contribution: -1. Join the [OpenMSP Slack](https://www.openmsp.ai/) -2. Describe what you're planning in the `#openframe-dev` channel -3. Get feedback from maintainers before investing significant effort +1. Set up your [development environment](../setup/environment.md) +2. Follow the [local development guide](../setup/local-development.md) to build and test locally +3. Join the [OpenMSP Slack community](https://www.openmsp.ai/) for discussion and guidance + +> **All discussions, questions, and feature requests** are managed on [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). There are no GitHub Issues or Discussions on this repository. --- @@ -19,104 +20,112 @@ Before starting a large contribution: ### Java Style -The project follows standard Java conventions with the following specifics: +The project follows standard Java conventions with some OpenFrame-specific patterns: | Convention | Rule | |-----------|------| -| Indentation | 4 spaces (no tabs) | -| Line length | Max 120 characters | -| Imports | No wildcard imports; organize by static β†’ java β†’ jakarta β†’ spring β†’ other | -| Braces | Allman-adjacent style (opening brace on same line) | -| Naming | `camelCase` methods/fields, `PascalCase` classes, `UPPER_SNAKE` constants | +| **Lombok** | Use `@Data`, `@Builder`, `@Slf4j`, `@RequiredArgsConstructor` freely | +| **Immutability** | Prefer `final` fields; use Lombok `@Value` for pure DTOs | +| **Package structure** | Follow existing `com.openframe.*` package hierarchy | +| **Naming** | `*Service` for business logic, `*Repository` for data access, `*Controller` for REST, `*DataFetcher` for GraphQL | +| **Exception handling** | Throw from the `openframe-exception` hierarchy (`NotFoundException`, `BadRequestException`, etc.) | +| **Logging** | Use `@Slf4j` (Lombok); log at `DEBUG` for operations, `WARN`/`ERROR` for unexpected states | ### Lombok Usage -Use Lombok to reduce boilerplate. Preferred annotations: - -```java -// Prefer these -@Data // getters, setters, equals, hashCode, toString -@Value // immutable value objects -@Builder // fluent builders -@RequiredArgsConstructor // constructor injection -@Slf4j // logging - -// Avoid manual getters/setters when @Data or @Value apply -``` - -### Spring Boot Conventions - -- Use constructor injection (via `@RequiredArgsConstructor`) over field injection (`@Autowired`) -- Prefer `@ConfigurationProperties` over `@Value` for configuration -- Use `@Service`, `@Repository`, `@Component`, `@Controller` consistently -- Configuration classes should be annotated with `@Configuration` - ```java -// Correct: Constructor injection +// Preferred patterns +@Data // Getters, setters, equals, hashCode, toString +@Builder // Builder pattern for DTOs +@Value // Immutable value objects +@Slf4j // Logger injection +@RequiredArgsConstructor // Constructor injection +@NoArgsConstructor // For JPA/MongoDB entities + +// Example service class +@Slf4j @Service @RequiredArgsConstructor -public class MyService { - private final MyRepository repository; - private final AnotherService anotherService; -} - -// Avoid: Field injection -@Service -public class MyService { - @Autowired - private MyRepository repository; +public class DeviceService { + private final MachineRepository machineRepository; + private final TenantIdProvider tenantIdProvider; + + public Optional findDevice(String machineId) { + log.debug("Looking up device: {}", machineId); + return machineRepository.findByMachineId( + machineId, tenantIdProvider.getTenantId() + ); + } } ``` -### Multi-Tenancy Requirements - -Every service and repository method that accesses tenant-scoped data **must** include `tenantId` in queries: +### GraphQL DataFetcher Conventions ```java -// Correct: Always scope to tenant -Optional findByIdAndTenantId(String id, String tenantId); - -// Wrong: Missing tenant scope -Optional findById(String id); // Never use for tenant-scoped data +// Use @DgsComponent, @DgsQuery, @DgsMutation annotations +// Follow existing DataFetcher naming patterns +@DgsComponent +public class DeviceDataFetcher { + + @DgsQuery + public GenericConnection devices( + @InputArgument DeviceFilterInput filter, + @InputArgument ConnectionArgs connectionArgs) { + // ... + } +} ``` -### Exception Handling +### Multi-Tenancy Requirements -Use the standard exception hierarchy from `openframe-exception`: +**Every new domain document must:** ```java -// Use specific exception types -throw new NotFoundException("Organization not found: " + id); -throw new BadRequestException("Invalid email format"); -throw new ForbiddenException("Access denied for tenant: " + tenantId); -throw new ConflictException("Email already exists"); -throw new ValidationException("Required field missing: name"); -``` +// 1. Implement TenantScoped +@Document(collection = "my_collection") +public class MyDocument implements TenantScoped { + @Id + private String id; + + private String tenantId; // ← Required + // ... +} -Never throw `RuntimeException` or `Exception` directly. +// 2. Have a compound index including tenantId +// (defined in MongoIndexConfig) +``` --- ## Branch Naming -Use descriptive branch names with a type prefix: +Use descriptive, hyphen-separated branch names with a type prefix: -| Type | Pattern | Example | -|------|---------|---------| -| Feature | `feature/` | `feature/add-nats-retry-logic` | -| Bug fix | `fix/` | `fix/tenant-context-not-cleared` | -| Refactor | `refactor/` | `refactor/notification-repository` | -| Documentation | `docs/` | `docs/update-kafka-readme` | -| Dependency updates | `deps/` | `deps/upgrade-spring-boot-3.4` | +| Prefix | Use Case | Example | +|--------|---------|---------| +| `feat/` | New features | `feat/add-webhook-support` | +| `fix/` | Bug fixes | `fix/notification-pagination-cursor` | +| `chore/` | Maintenance tasks | `chore/upgrade-spring-boot-3.4` | +| `refactor/` | Code refactoring | `refactor/device-service-split` | +| `docs/` | Documentation changes | `docs/update-auth-flow-diagram` | +| `test/` | Test additions/fixes | `test/add-ticket-repository-it` | + +```bash +# Example: Create a feature branch +git checkout -b feat/add-device-tagging-bulk-update + +# Example: Create a fix branch +git checkout -b fix/cassandra-tenant-scope-query +``` --- ## Commit Message Format -Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification: +Follow the **Conventional Commits** specification: ```text -(): +(): [optional body] @@ -129,41 +138,45 @@ Follow the [Conventional Commits](https://www.conventionalcommits.org/) specific |------|------------| | `feat` | New feature or capability | | `fix` | Bug fix | -| `refactor` | Code change that neither fixes a bug nor adds a feature | +| `chore` | Build, dependency, or tooling changes | +| `refactor` | Code restructure without behavior change | | `test` | Adding or updating tests | | `docs` | Documentation only changes | -| `chore` | Build system, dependency updates, CI changes | | `perf` | Performance improvements | - -### Scope - -Use the module name (without `openframe-` prefix) as scope: - -```text -feat(security-core): add PKCE utility for code challenge generation -fix(data-mongo-sync): resolve tenant context leak in batch operations -refactor(gateway-service-core): extract rate limit logic into service -test(data-nats): add integration test for notification broadcast -docs(api-service-core): update GraphQL data fetcher documentation -chore(deps): upgrade spring-boot to 3.3.2 -``` +| `ci` | CI/CD pipeline changes | + +### Scope Examples + +| Scope | Module | +|-------|--------| +| `core` | `openframe-core` | +| `gateway` | `openframe-gateway-service-core` | +| `auth` | `openframe-authorization-service-core` | +| `api` | `openframe-api-service-core` | +| `data` | `openframe-data-mongo-*` | +| `stream` | `openframe-stream-service-core` | +| `management` | `openframe-management-service-core` | +| `frontend` | `openframe-frontend-core` | +| `security` | `openframe-security-core` | ### Examples ```text -feat(authorization-service-core): add Microsoft SSO provider strategy +feat(api): add bulk device tag assignment endpoint -Implements MicrosoftClientRegistrationStrategy to support Microsoft -Entra ID (Azure AD) authentication flows alongside existing Google SSO. +Adds GraphQL mutation for bulk assignment of tags to multiple +devices in a single operation. Includes DataLoader optimization +to prevent N+1 on tag resolution. -Closes: #discussion in #openframe-dev slack -``` +fix(data): correct cursor encoding for notification pagination -```text -fix(stream-service-core): handle null tenant_id in debezium enrichment +The cursor codec was using unstable field order causing +inconsistent pagination results. Now uses deterministic +JSON serialization. + +chore(deps): upgrade Spring Boot to 3.3.1 -When a Debezium event is missing the tenant header, the enrichment service -now falls back to domain-based tenant resolution instead of throwing NPE. +Addresses CVE-2024-XXXX in spring-web. ``` --- @@ -172,93 +185,63 @@ now falls back to domain-based tenant resolution instead of throwing NPE. ### Before Opening a PR -1. **Build passes:** `mvn install -DskipTests` -2. **Tests pass:** `mvn test -pl ` -3. **No secrets committed:** Review all changed files -4. **Follows code style:** Check Lombok, constructor injection, tenant scoping -5. **Covers edge cases:** Add unit tests for new logic - -### PR Title - -Use the same format as commit messages: - -```text -feat(security-core): add PKCE utility for authorization flows -fix(data-mongo-sync): resolve tenant context leak -``` +- [ ] Branch is up to date with `main` +- [ ] All unit tests pass: `mvn test` +- [ ] Relevant integration tests pass: `mvn verify -pl ` +- [ ] New code follows project conventions (Lombok, tenant scoping, etc.) +- [ ] New domain documents implement `TenantScoped` +- [ ] No secrets or hardcoded credentials in code +- [ ] Javadoc added for new public APIs ### PR Description Template -```markdown -## What does this PR do? - -Brief description of the change. - -## Why? - -Motivation for the change. +```text +## Summary +Brief description of what this PR does and why. -## How was it tested? +## Changes +- List key changes +- Affected modules: openframe-xxx, openframe-yyy +## Testing - [ ] Unit tests added/updated - [ ] Integration tests added/updated -- [ ] Manual testing performed - -## Checklist +- [ ] Manually tested against local stack -- [ ] No secrets in code or tests -- [ ] All new endpoints have authorization rules -- [ ] New DB queries are tenant-scoped -- [ ] Input validation on all new DTOs -- [ ] No breaking changes (or breaking changes are documented) +## Related +Link to Slack discussion or related PR (if any). ``` -### Review Checklist (for Reviewers) - -- [ ] Code follows established patterns (constructor injection, Lombok, multi-tenancy) -- [ ] New functionality is tested -- [ ] Security considerations are addressed (tenant scope, input validation, no secrets) -- [ ] Error handling uses the standard exception hierarchy -- [ ] No performance regressions (N+1 queries, missing indexes) - ---- +### Review Checklist -## Versioning +Reviewers should verify: -All modules are versioned together using `${revision}` in the parent POM. Version bumps are managed by the maintainers. Contributors do not need to update the version number in PRs. - -The version follows [Semantic Versioning](https://semver.org/): -- **Major:** Breaking API changes -- **Minor:** New backward-compatible features -- **Patch:** Backward-compatible bug fixes +- [ ] Code follows OpenFrame conventions (naming, Lombok, tenant scoping) +- [ ] New endpoints have appropriate `@PreAuthorize` or security config +- [ ] No N+1 query problems (DataLoaders used for related entity loading) +- [ ] Integration tests cover the happy path +- [ ] Error cases return appropriate exceptions from `openframe-exception` +- [ ] Multi-tenant isolation maintained (tenantId filters present) --- -## Adding a New Module - -When adding a new module to the library: +## Module Addition Guidelines -1. Create the module directory following the existing naming convention (`openframe-/`) -2. Add a `pom.xml` that inherits from the parent -3. Add the module to the parent `pom.xml` `` section -4. Add the module to `` in the parent with `${revision}` -5. Write unit tests before submitting -6. Update the README and architecture documentation +When adding a new module to `openframe-oss-lib`: -```xml - -openframe-my-new-module - - - - com.openframe.oss - openframe-my-new-module - ${revision} - -``` +1. **Add to parent `pom.xml`** in the `` section +2. **Inherit from parent**: `openframe-oss-lib` +3. **Add to ``** in parent POM with `${revision}` version +4. **Follow naming convention**: `openframe--` +5. **Include a `README.md`** describing the module's purpose +6. **Write at least one unit test** before the first PR --- ## Getting Help -Stuck on a contribution? Reach out on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in `#openframe-dev`. +- **Questions**: Ask in [OpenMSP Slack](https://www.openmsp.ai/) `#dev-questions` +- **Feature Ideas**: Share in `#roadmap` channel +- **Bug Reports**: Report in `#bugs` channel + +We review contributions regularly. Thank you for helping build the open-source MSP platform of the future! diff --git a/docs/development/security/README.md b/docs/development/security/README.md index 36431181f..7a04b1ede 100644 --- a/docs/development/security/README.md +++ b/docs/development/security/README.md @@ -1,292 +1,283 @@ # Security Best Practices -This guide covers the security patterns, conventions, and requirements for developing with and contributing to **openframe-oss-lib**. +This guide covers security architecture, authentication and authorization patterns, secrets management, and secure development guidelines for `openframe-oss-lib`. ---- - -## Authentication and Authorization Patterns - -### JWT-Based Authentication - -Every service in OpenFrame uses JWT bearer tokens for authentication. The library provides: +[![OpenFrame v0.3.0 - Remote File Manager & Unified Authentication Architecture](https://img.youtube.com/vi/mibUHvcVIHs/maxresdefault.jpg)](https://www.youtube.com/watch?v=mibUHvcVIHs) -- **`openframe-security-core`** β€” JWT encoder/decoder beans, RSA key loading -- **`openframe-authorization-service-core`** β€” Multi-tenant OAuth2 Authorization Server -- **`openframe-gateway-service-core`** β€” Multi-issuer JWT validation at the edge - -**Key principle:** JWTs are issued per-tenant with tenant-scoped RSA key pairs. Never use a shared signing key across tenants. +--- -#### JWT Claims Structure +## Authentication and Authorization Architecture -Every access token must contain: +### Overview -| Claim | Description | -|-------|-------------| -| `tenant_id` | Tenant identifier for multi-tenancy | -| `userId` | Authenticated user's ID | -| `roles` | User roles (`ADMIN`, `OWNER`, `AGENT`) | -| `iss` | Issuer URL (tenant-specific) | -| `exp` | Expiration timestamp | +OpenFrame implements a **multi-layered security model** across three service tiers: -```java -// Correct: Extract tenant from JWT principal -@GetMapping("/me") -public ResponseEntity getCurrentUser(@AuthenticationPrincipal Jwt jwt) { - String tenantId = jwt.getClaimAsString("tenant_id"); - String userId = jwt.getClaimAsString("userId"); - // ... -} +```mermaid +flowchart TD + Client["Client (Browser / Agent)"] + Gateway["Gateway Service Core\n(JWT Validation + Role Enforcement)"] + Auth["Authorization Service Core\n(OAuth2 / OIDC Token Issuance)"] + API["API Service Core\n(OAuth2 Resource Server)"] + ExternalAPI["External API\n(API Key + Rate Limiting)"] + + Client -->|"JWT"| Gateway + Client -->|"API Key"| ExternalAPI + Gateway -->|"Validated Request"| API + Gateway --> Auth + ExternalAPI -->|"Rate-limited Request"| API + Auth -->|"Tenant-scoped JWT"| Client ``` -### Multi-Tenant Key Isolation +### Layer 1: Authorization Service (Token Issuance) -Each tenant has its own RSA key pair managed by `TenantKeyService`: +The `openframe-authorization-service-core` module handles: +- **OAuth2/OIDC Authorization Server** (Spring Authorization Server 1.3.1) +- **Per-tenant RSA key pair generation** for JWT signing +- **SSO integration** (Google, Microsoft) via dynamic client registration +- **Password reset** and invitation flows +- **Tenant isolation** via `TenantContextFilter` (ThreadLocal) -```mermaid -flowchart LR - Token["JWT Signing Request"] --> CTX["TenantContext.getTenantId()"] - CTX --> KS["TenantKeyService.getOrCreateActiveKey()"] - KS --> DB["MongoDB: TenantKey collection"] - KS --> RSA["RSA Key Pair"] - RSA --> JWT["Signed JWT"] +**Custom JWT Claims:** +```text +tenant_id – Tenant identifier +userId – Platform user ID +roles – User roles (OWNER implies ADMIN) ``` -> **Never** share RSA key material between tenants or inject keys via environment variables in production. Always use the `TenantKeyService` for key lifecycle management. +### Layer 2: Gateway (Token Validation) -### Role-Based Access Control +The `openframe-gateway-service-core` module: +- Validates JWTs using **per-tenant RSA public keys** (Caffeine cached) +- Enforces **role-based access** (`ROLE_ADMIN`, `ROLE_AGENT`, `SCOPE_*`) +- Supports **multi-source token extraction**: cookie β†’ header β†’ query param +- Applies **CORS policies** +- Enforces **rate limits** on external API access -Gateway-level role enforcement is configured in `GatewaySecurityConfig`: +### Layer 3: API Service (Resource Protection) -| Path Pattern | Required Role | -|-------------|--------------| -| `/api/**` | `ADMIN` | -| `/tools/agent/**` | `AGENT` | -| `/ws/tools/agent/**` | `AGENT` | -| `/ws/nats` | `ADMIN` or `AGENT` | -| `/external-api/**` | API key (no JWT required) | - -When adding new routes, always explicitly define authorization rules. Never rely on implicit `permitAll()` for sensitive endpoints. +The `openframe-api-service-core` module: +- Acts as an **OAuth2 Resource Server** +- Injects `AuthPrincipal` via `@AuthenticationPrincipal` +- Caches JWT decoders by issuer (Caffeine) +- Does **not** re-validate tokens β€” relies on Gateway enforcement --- -## OAuth2 Best Practices - -### PKCE (Proof Key for Code Exchange) +## API Key Security -All browser-initiated authorization flows **must** use PKCE. The `PKCEUtils` class provides: +External API access uses API keys with rate limiting: -```java -// Always use PKCE for browser flows -String codeVerifier = PKCEUtils.generateCodeVerifier(); -String codeChallenge = PKCEUtils.generateCodeChallenge(codeVerifier); -String state = PKCEUtils.generateState(); +```mermaid +flowchart TD + Req["External Request to /external-api/**"] + Check{"X-API-Key header present?"} + Validate["Validate API Key in DB"] + Rate{"Rate limit check"} + Forward["Forward to API service"] + + Req --> Check + Check -->|"No"| Reject401["401 Unauthorized"] + Check -->|"Yes"| Validate + Validate -->|"Invalid"| Reject401 + Validate -->|"Valid"| Rate + Rate -->|"Exceeded"| Reject429["429 Too Many Requests"] + Rate -->|"OK"| Forward ``` -Never implement custom PKCE logic β€” use the provided utility class. +**Rate limit headers returned:** +- `X-RateLimit-Limit-Minute` +- `X-RateLimit-Remaining-Minute` +- `X-RateLimit-Limit-Hour` +- `X-RateLimit-Remaining-Hour` -### Token Storage +**Best Practices for API Keys:** +- Rotate API keys regularly using the `ApiKeyController` +- Store API keys encrypted β€” the `openframe-core-crypto` module provides `EncryptionService` for AES encryption +- Never log API key values β€” only log key IDs +- Use `APIKeyType` to scope keys appropriately -Tokens are stored as **HTTP-only cookies** by the OAuth BFF controller. This pattern prevents XSS-based token theft: +--- -```text -Set-Cookie: access_token=...; HttpOnly; Secure; SameSite=Strict -Set-Cookie: refresh_token=...; HttpOnly; Secure; SameSite=Strict -``` +## Secrets Management -> **Never** expose tokens in JavaScript-accessible storage (localStorage, sessionStorage) or URL parameters. +### Environment Variables vs. Configuration -### API Keys +Never hardcode secrets in source code or configuration files. Always use environment variables: -External API consumers authenticate via `X-API-Key` header. API keys follow a two-part format: +```bash +# βœ… CORRECT: Use environment variables +export SPRING_DATA_MONGODB_URI=mongodb://user:secret@host/db -```text -X-API-Key: . +# ❌ WRONG: Hardcoded in application.yml +# spring.data.mongodb.uri: mongodb://user:hardcoded@host/db ``` -Key security requirements: - -- Store only the **hashed** secret in MongoDB (`ApiKey.secretHash`) -- Never log raw API key values -- Apply rate limiting via `RateLimitService` for all API key–authenticated routes -- Rotate API keys on suspected compromise - ---- - -## Data Encryption and Secure Storage +### Encryption at Rest -### Encryption Service - -The `openframe-core-crypto` module provides symmetric encryption for sensitive fields stored in MongoDB: +The `openframe-core-crypto` module provides `EncryptionService` for encrypting sensitive data before storing in MongoDB: ```java +// Use EncryptionService to encrypt sensitive values before storage +// Example pattern used for tool credentials and RSA private keys @Autowired private EncryptionService encryptionService; -// Encrypt before storing -String encrypted = encryptionService.encrypt(plainText); - -// Decrypt on read -String plain = encryptionService.decrypt(encrypted); +String encrypted = encryptionService.encrypt(sensitiveValue); +String decrypted = encryptionService.decrypt(encrypted); ``` -Fields that should always be encrypted in MongoDB: +**Fields encrypted in MongoDB:** +- RSA private keys in `TenantKey` documents +- Tool credentials in `ToolCredentials` +- OAuth client secrets -- Tool credentials (`ToolCredentials`) -- API key secrets (`ApiKey.secretHash` β€” hashed, not encrypted) -- SSO provider client secrets (`SSOConfig`) +### Key Management -### Password Hashing - -Passwords are hashed using BCrypt via the `PasswordEncoder` bean provided by `ManagementConfiguration`: - -```java -// Correct: Always hash passwords before storing -String hashed = passwordEncoder.encode(rawPassword); - -// Correct: Verify passwords using the encoder -boolean matches = passwordEncoder.matches(rawPassword, hashed); -``` - -Never store raw passwords in any form β€” not in logs, databases, or environment variables. +Per-tenant RSA keys are managed by `TenantKeyService`: +- Private keys are **encrypted before storage** using `EncryptionService` +- Public keys are served via the JWKS endpoint for JWT validation +- Keys can be independently rotated per tenant without affecting other tenants --- ## Input Validation and Sanitization -### Bean Validation +### Jakarta Validation -All request DTOs must use Jakarta Bean Validation annotations: +All API inputs use Jakarta Bean Validation constraints: ```java +// Example from RunCommandInput @NotNull -@ValidEmail // Custom OpenFrame validator -private String email; - -@NotBlank -@TenantDomain // Custom OpenFrame validator -private String tenantDomain; -``` +private String machineId; -Custom validators in `openframe-core`: +@NotNull +private String command; -| Annotation | Validates | -|-----------|----------| -| `@ValidEmail` | Email format and domain | -| `@TenantDomain` | Tenant domain slug format | +@Positive +private Integer timeoutSeconds; +``` -### SQL / NoSQL Injection Prevention +### Tenant Domain Validation -MongoDB queries are always executed via Spring Data repositories or `MongoTemplate` with typed objects β€” never with raw string interpolation: +The `openframe-core` module provides custom validators: ```java -// Correct: Use typed Spring Data query method -List users = userRepository.findByTenantIdAndEmail(tenantId, email); - -// Wrong: Never build raw query strings -// mongoTemplate.find(Query.query(Criteria.where("email").is("' OR '1'='1")), User.class); +// @TenantDomain validates slug format for tenant identifiers +// @ValidEmail validates email addresses consistently ``` ---- - -## Common Security Vulnerabilities and Mitigations +### GraphQL Input Validation -| Vulnerability | Risk | Mitigation in openframe-oss-lib | -|-------------|------|-------------------------------| -| Cross-Tenant Data Access | CRITICAL | TenantContext + tenant-scoped repositories; always include `tenantId` in queries | -| JWT Token Forgery | HIGH | Per-tenant RSA keys; multi-issuer validation; short expiry | -| CSRF | MEDIUM | OAuth2 state parameter + PKCE; HTTP-only cookies with SameSite | -| API Key Exposure | HIGH | Keys hashed at rest; rate limiting; `X-API-Key` header only | -| XSS Token Theft | HIGH | HTTP-only cookies; CSP headers enforced at gateway | -| Mass Assignment | MEDIUM | Explicit DTO mapping; never expose domain documents directly | -| Insecure Direct Object Reference | HIGH | Always scope queries with `tenantId` + authorization checks | +GraphQL mutations validate inputs using the same Jakarta constraints through Spring's validation integration. Invalid inputs return structured `MutationError` responses rather than exceptions. --- -## Secrets Management +## Multi-Tenant Isolation -### Local Development +**Critical principle:** All data access must be scoped to the current tenant. -For local development, use `application-local.yml` (never committed) to override sensitive properties: +```java +// TenantScoped documents always include tenantId +@Document(collection = "devices") +public class Machine implements TenantScoped { + private String tenantId; + // ... +} -```yaml -# application-local.yml (gitignored) -jwt: - private-key: classpath:keys/local-private.pem - public-key: classpath:keys/local-public.pem -spring: - data: - mongodb: - uri: mongodb://localhost:27017/openframe-local +// Repositories automatically filter by tenantId +// DefaultTenantIdProvider reads from TENANT_ID env var +// or falls back to "oss" for OSS single-tenant mode ``` -### Environment Variable Conventions +**Security checklist for new repositories:** +- βœ… Document implements `TenantScoped` +- βœ… Repository queries include `tenantId` filter +- βœ… Custom repository implementations use `TenantIdProvider` +- βœ… Compound index includes `{ tenantId: 1, ... }` -In production deployments, secrets must be injected as environment variables, never hardcoded: +--- -```bash -# JWT keys -JWT_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----..." -JWT_PUBLIC_KEY="-----BEGIN PUBLIC KEY-----..." +## SSO Security -# MongoDB -SPRING_DATA_MONGODB_URI="mongodb+srv://user:pass@cluster..." +SSO flows (Google, Microsoft) use secure cookie-based state passing: -# Redis -SPRING_DATA_REDIS_HOST="redis.internal" -``` +- `SsoCookieCodec` encrypts SSO state cookies +- `SsoAuthorizationRequestResolver` validates OIDC state parameters +- Dynamic client registration uses `DynamicClientRegistrationRepository` to prevent cross-tenant SSO contamination -> Use your platform's secret manager (AWS Secrets Manager, HashiCorp Vault, Kubernetes Secrets) to inject these values. Never commit secrets to version control. +**SSO Security Checklist:** +- βœ… Validate `state` parameter on all OAuth2 callbacks +- βœ… Use HTTP-only, Secure cookies for session state +- βœ… Scope SSO client registrations to specific tenants +- βœ… Never expose client secrets in frontend code -### CI/CD Secret Handling - -GitHub Actions secrets are referenced via: +--- -```yaml -env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} -``` +## Common Vulnerability Mitigations -Never echo, log, or print secrets in CI steps. +| Vulnerability | Mitigation in OpenFrame | +|--------------|------------------------| +| **JWT Forgery** | Per-tenant RSA signing; issuer validated; strict decoder caching | +| **Cross-Tenant Data Leakage** | `tenantId` filter on all queries; `TenantContextFilter` on all requests | +| **API Key Brute Force** | Rate limiting per minute/hour/day at gateway level | +| **CSRF** | Stateless JWT-based auth; `OriginSanitizerFilter` in gateway | +| **Injection** | MongoDB query uses Spring Data criteria, not raw queries; strict input validation | +| **Secret Exposure** | Encryption at rest for private keys and credentials; env var management | +| **Unauthorized Tool Access** | Agent auth (`ROLE_AGENT`) separate from admin auth (`ROLE_ADMIN`) | --- -## Security Testing and Code Review Guidelines +## Security Testing Guidelines -### Pre-Commit Checklist +### Unit Testing Security Logic -Before opening a Pull Request, verify: +```java +// Use TestAuthenticationManager for mocking auth in tests +// Located in: openframe-client-core/src/test/... +@Autowired +private TestAuthenticationManager testAuthenticationManager; +``` -- [ ] No secrets, tokens, or keys in code or test fixtures -- [ ] All new endpoints have explicit authorization rules -- [ ] New database queries include `tenantId` scope -- [ ] Input validation annotations on all request DTOs -- [ ] Sensitive fields are encrypted at rest -- [ ] No raw SQL/NoSQL string interpolation +### Integration Testing Auth Flows -### Integration Test Security +The `openframe-test-service-core` module provides full auth flow helpers: -Integration tests should: +```java +// AuthFlow, AuthFlowOSS, AuthFlowSAAS provide test auth flows +// Use AuthHelper for test authentication setup +AuthParts auth = authFlow.loginAsAdmin(); +``` -- Use randomly generated test data (not hardcoded UUIDs matching production patterns) -- Clean up test data after each test -- Never use production URLs or credentials +### Security Code Review Checklist ---- +Before merging any security-related code: -## Origin Sanitization +- [ ] No secrets or credentials in source code +- [ ] All API inputs validated with Jakarta constraints +- [ ] New endpoints have appropriate role restrictions +- [ ] New MongoDB documents implement `TenantScoped` +- [ ] Sensitive fields encrypted before storage +- [ ] New API keys scoped to minimum required permissions +- [ ] SSO state parameters validated in callbacks +- [ ] Rate limiting applied to public-facing endpoints -The `OriginSanitizerFilter` in the gateway sanitizes the `Origin` header to prevent header injection attacks. Never bypass this filter for external-facing routes. +--- -```mermaid -flowchart LR - Request["Incoming Request"] --> OSF["OriginSanitizerFilter"] - OSF --> AHF["AddAuthorizationHeaderFilter"] - AHF --> JWT["JWT Validation"] - JWT --> Route["Route Handler"] -``` +## Environment Variables and Secrets Management ---- +> **Never commit secrets to version control.** Use environment variables, secret management systems (Vault, AWS Secrets Manager, Kubernetes Secrets), or `.env` files that are excluded from Git. -## Reporting Security Issues +```bash +# Add to .gitignore +.env +.env.local +*.secrets + +# Use descriptive names for secret environment variables +JWT_SIGNING_KEY_SECRET=... +MONGODB_PASSWORD=... +REDIS_AUTH_PASSWORD=... +``` -For security vulnerabilities, do **not** open a public GitHub issue. Contact the team via the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) in a direct message to the maintainers, or email the security contact listed on [flamingo.run](https://flamingo.run). +For questions or to report security issues, contact the team via [OpenMSP Slack](https://www.openmsp.ai/). diff --git a/docs/development/setup/environment.md b/docs/development/setup/environment.md index c2c95cf53..e104430ff 100644 --- a/docs/development/setup/environment.md +++ b/docs/development/setup/environment.md @@ -1,209 +1,224 @@ # Development Environment Setup -This guide walks you through setting up a complete development environment for **openframe-oss-lib**. +This guide covers setting up a complete development environment for working with `openframe-oss-lib`. --- -## IDE Recommendations +## Required Tools -### IntelliJ IDEA (Recommended) +Install the following tools before beginning: + +| Tool | Version | Installation | +|------|---------|-------------| +| **JDK 21** | 21 (LTS) | [Adoptium](https://adoptium.net/) / [SDKMAN](https://sdkman.io/) | +| **Apache Maven** | 3.8+ | [maven.apache.org](https://maven.apache.org/download.cgi) | +| **Git** | 2.x+ | [git-scm.com](https://git-scm.com/) | +| **Docker Desktop** | 24+ | [docker.com](https://www.docker.com/products/docker-desktop/) | +| **Node.js** | 18+ | [nodejs.org](https://nodejs.org/) (for frontend-core) | -IntelliJ IDEA Community or Ultimate is the recommended IDE for this project. It provides the best support for: +--- -- Maven multi-module projects -- Spring Boot auto-configuration detection -- Lombok annotation processing -- Java 21 features (records, virtual threads, sealed classes) +## Java Environment Setup -**Download:** [https://www.jetbrains.com/idea/](https://www.jetbrains.com/idea/) +### Using SDKMAN (Recommended) -### VS Code (Alternative) +[SDKMAN](https://sdkman.io/) makes managing multiple JDK versions easy: -VS Code with the **Extension Pack for Java** is a lighter-weight alternative. +```bash +# Install SDKMAN +curl -s "https://get.sdkman.io" | bash +source "$HOME/.sdkman/bin/sdkman-init.sh" -Required extensions: +# Install Java 21 (Temurin) +sdk install java 21.0.4-tem -- Extension Pack for Java (Microsoft) -- Spring Boot Extension Pack (VMware) -- Lombok Annotations Support +# Set as default +sdk default java 21.0.4-tem ---- +# Verify +java -version +``` + +### Using System Package Manager + +```bash +# macOS with Homebrew +brew install openjdk@21 + +# Ubuntu / Debian +sudo apt-get install openjdk-21-jdk -## IntelliJ IDEA Setup +# Fedora / RHEL +sudo dnf install java-21-openjdk-devel +``` -### Step 1 β€” Import the Project +--- -1. Open IntelliJ IDEA -2. Select **File β†’ Open** -3. Navigate to the cloned `openframe-oss-lib` directory -4. Select the root `pom.xml` β†’ click **Open as Project** -5. Wait for Maven to import all 30+ modules +## Maven Setup -### Step 2 β€” Configure Project SDK +```bash +# macOS +brew install maven -1. Open **File β†’ Project Structure β†’ Project** -2. Set **SDK** to Java 21 -3. Set **Language Level** to `21` +# Ubuntu / Debian +sudo apt-get install maven -### Step 3 β€” Enable Annotation Processors (Lombok) +# Verify +mvn -version +``` -1. Open **Settings β†’ Build, Execution, Deployment β†’ Compiler β†’ Annotation Processors** -2. Check **Enable annotation processing** -3. Select **Obtain processors from project classpath** +### Configure GitHub Packages Access + +Edit `~/.m2/settings.xml` (create it if it doesn't exist): + +```xml + + + + github + YOUR_GITHUB_USERNAME + YOUR_GITHUB_PERSONAL_ACCESS_TOKEN + + + +``` -Without this step, Lombok-generated code will show errors. +> Create a GitHub PAT with `read:packages` and `write:packages` scopes at [github.com/settings/tokens](https://github.com/settings/tokens). -### Step 4 β€” Maven Delegate (Recommended) +--- + +## IDE Setup -1. Open **Settings β†’ Build, Execution, Deployment β†’ Build Tools β†’ Maven β†’ Runner** -2. Check **Delegate IDE build/run actions to Maven** +### IntelliJ IDEA (Recommended) -This ensures builds use Maven directly rather than IntelliJ's internal compiler, avoiding configuration drift. +1. **Open the project**: File β†’ Open β†’ select the root `pom.xml` +2. **Enable annotation processing**: Settings β†’ Build β†’ Compiler β†’ Annotation Processors β†’ βœ… Enable annotation processing +3. **Install recommended plugins**: -### Step 5 β€” Increase Memory (Optional but Recommended) +| Plugin | Purpose | +|--------|---------| +| **Lombok** | Required for `@Data`, `@Builder`, `@Slf4j` annotations | +| **Spring Boot** | Run configurations and property completion | +| **GraphQL** | Schema syntax highlighting | +| **Docker** | Container management | -Edit `Help β†’ Change Memory Settings` and increase to at least: +**Recommended Settings for IntelliJ:** ```text -Xmx: 4096 MB -``` +Settings β†’ Build, Execution, Deployment β†’ Build Tools β†’ Maven + βœ“ Use plugin registry + Maven home path: [your Maven installation] + User settings file: ~/.m2/settings.xml ---- +Settings β†’ Editor β†’ Code Style β†’ Java + Use project code style (if .editorconfig is present) +``` -## VS Code Setup +### VS Code -Install the Extension Pack for Java: +Install the **Extension Pack for Java**: ```bash code --install-extension vscjava.vscode-java-pack -code --install-extension vmware.vscode-spring-boot +code --install-extension pivotal.vscode-spring-boot ``` -Add to `.vscode/settings.json` in your workspace: +Add to your workspace `settings.json`: ```json { - "java.configuration.runtimes": [ - { - "name": "JavaSE-21", - "path": "/path/to/jdk-21" - } - ], - "java.compile.nullAnalysis.mode": "disabled", - "maven.executable.path": "/path/to/mvn" + "java.configuration.updateBuildConfiguration": "automatic", + "java.compile.nullAnalysis.mode": "automatic", + "java.jdt.ls.java.home": "/path/to/java21" } ``` --- -## Required Development Tools - -| Tool | Installation | -|------|-------------| -| Java 21 JDK | [Adoptium Temurin 21](https://adoptium.net/) | -| Maven 3.9+ | [https://maven.apache.org/download.cgi](https://maven.apache.org/download.cgi) | -| Docker Desktop | [https://www.docker.com/products/docker-desktop/](https://www.docker.com/products/docker-desktop/) | -| Git | System package manager or [https://git-scm.com/](https://git-scm.com/) | - ---- - -## Environment Variables for Development +## Docker Setup -Set these in your shell profile (`~/.bashrc`, `~/.zshrc`, or equivalent): +Docker is required for integration tests (Testcontainers) and local infrastructure: ```bash -# Java 21 home (adjust path for your OS and distribution) -export JAVA_HOME=/path/to/jdk-21 -export PATH="$JAVA_HOME/bin:$PATH" +# Verify Docker is running +docker info -# Increase Maven heap for large multi-module builds -export MAVEN_OPTS="-Xmx4g -XX:MaxMetaspaceSize=512m" +# Verify Docker Compose +docker compose version -# GitHub Packages credentials (required for dependency resolution) -export GITHUB_ACTOR="your-github-username" -export GITHUB_TOKEN="your-github-pat" +# Test Testcontainers connectivity +docker pull mongo:6 +docker pull redis:7 ``` -> Note: `$GITHUB_TOKEN` must have `read:packages` permission to resolve OSS library dependencies. +> On **Linux**, add your user to the `docker` group to avoid `sudo`: +> ```bash +> sudo usermod -aG docker $USER +> newgrp docker +> ``` --- -## Git Configuration +## Node.js Setup (Frontend Development) -Configure your Git identity: +If working on the `openframe-frontend-core` module: ```bash -git config --global user.name "Your Name" -git config --global user.email "your-email@example.com" -``` - -Configure line endings (important for cross-platform teams): - -```bash -# macOS / Linux -git config --global core.autocrlf input - -# Windows -git config --global core.autocrlf true +# Install Node.js via nvm (recommended) +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash +nvm install 18 +nvm use 18 + +# Verify +node --version +npm --version ``` --- -## Useful Maven Commands +## Environment Variables Reference -| Command | Purpose | -|---------|---------| -| `mvn install -DskipTests` | Build all modules, skip tests | -| `mvn test -pl openframe-core` | Run unit tests for a specific module | -| `mvn verify -pl openframe-data-mongo-sync` | Run integration tests for a module | -| `mvn clean install -DskipTests` | Clean build all modules | -| `mvn dependency:tree -pl openframe-api-service-core` | View dependency tree | -| `mvn versions:display-dependency-updates` | Check for dependency updates | -| `mvn flatten:flatten` | Apply CI-friendly version flattening | +For local development, set these shell environment variables. Add them to your `~/.zshrc`, `~/.bashrc`, or equivalent: ---- - -## Checkstyle and Code Quality (Optional) - -The project uses standard Spring Boot conventions. For consistent code style: - -- Java files follow standard Java conventions -- Lombok reduces boilerplate (avoid raw getters/setters when Lombok `@Data`, `@Value`, etc. apply) -- Import ordering follows IntelliJ defaults - ---- +```bash +# OpenFrame OSS - Single tenant mode +export TENANT_ID=oss -## Docker Configuration +# MongoDB +export SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/openframe -Docker is required for integration tests via Testcontainers. Ensure: +# Redis +export SPRING_REDIS_HOST=localhost +export SPRING_REDIS_PORT=6379 -```bash -# Docker daemon is running -docker info +# Kafka +export SPRING_KAFKA_BOOTSTRAP_SERVERS=localhost:9092 -# Pull commonly used images in advance for faster test runs -docker pull mongo:7 -docker pull nats:2 +# NATS +export NATS_SERVER_URL=nats://localhost:4222 ``` -Testcontainers will automatically manage container lifecycle during tests. +> These are only needed when running full service instances locally. Unit tests and most module tests use Testcontainers and do **not** require these. --- -## Verification +## Verification Checklist -After completing setup, verify everything works: +Run these commands to confirm your environment is ready: ```bash -# Full build with tests skipped -mvn install -DskipTests +# Java 21 +java -version 2>&1 | grep "21" -# Run unit tests for core module -mvn test -pl openframe-core +# Maven 3.8+ +mvn -version | grep "Apache Maven 3" -# Check Java version is 21 -java -version +# Docker running +docker ps -# Check Maven version is 3.9+ -mvn -version +# GitHub Packages accessible +mvn dependency:resolve -pl openframe-core -q && echo "Maven packages OK" ``` diff --git a/docs/development/setup/local-development.md b/docs/development/setup/local-development.md index 6fa2f1815..df5f1d27e 100644 --- a/docs/development/setup/local-development.md +++ b/docs/development/setup/local-development.md @@ -1,48 +1,52 @@ # Local Development Guide -This guide covers everything you need to work with **openframe-oss-lib** locally: cloning, building, iterating, and debugging. +This guide walks you through setting up the `openframe-oss-lib` project for local development, including building modules, running tests, and debugging. --- ## Clone and Initial Setup ```bash -# 1. Clone the repository +# Clone the repository git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib -# 2. Verify Java 21 is active +# Verify Java version java -version - -# 3. Configure GitHub Packages (if not already done) -# See prerequisites.md for settings.xml configuration - -# 4. Build all modules (skip tests for speed) -mvn install -DskipTests +# Should show: openjdk version "21.x.x" ``` --- -## Understanding the Multi-Module Build +## Building the Project -This is a Maven multi-module project. Key concepts: +### Full Build (All Modules, Skip Tests) + +```bash +mvn install -DskipTests +``` -- The **root `pom.xml`** is the **parent POM** β€” it defines shared dependencies, plugin versions, and the module list -- Each module has its own `pom.xml` that inherits from the parent -- **Unified versioning** β€” all modules share the same version via `${revision}` (currently `5.79.3`) -- The `flatten-maven-plugin` resolves `${revision}` at build time +This installs all modules to your local Maven cache (`~/.m2/repository`), making them available as dependencies for other local projects. -### Building a Single Module +### Build a Specific Module ```bash -# Build only openframe-core and its dependencies -mvn install -pl openframe-core -am -DskipTests +# Build only the core module +mvn install -pl openframe-core -DskipTests + +# Build a module and its dependencies +mvn install -pl openframe-api-lib -am -DskipTests -# Build only the security modules -mvn install -pl openframe-security-core,openframe-security-oauth -am -DskipTests +# Build multiple modules at once +mvn install -pl openframe-core,openframe-exception,openframe-data-mongo-common -DskipTests ``` -The `-am` flag (`--also-make`) ensures upstream dependencies are built first. +### Clean Build + +```bash +# Clean all build outputs and rebuild +mvn clean install -DskipTests +``` --- @@ -50,169 +54,189 @@ The `-am` flag (`--also-make`) ensures upstream dependencies are built first. ### Unit Tests -Unit tests follow the naming conventions `*Test.java` and `*Tests.java`: +Unit tests run without Docker and execute quickly: ```bash -# Run unit tests for a specific module +# Run tests for a specific module mvn test -pl openframe-core +mvn test -pl openframe-exception +mvn test -pl openframe-data-pinot -# Run all unit tests across the project +# Run all unit tests across all modules mvn test ``` ### Integration Tests -Integration tests are named `*IT.java` and require Docker (Testcontainers): +Integration tests use **Testcontainers** and require Docker: ```bash -# Ensure Docker is running first -docker info - # Run integration tests for MongoDB sync module mvn verify -pl openframe-data-mongo-sync # Run integration tests for NATS module mvn verify -pl openframe-data-nats + +# Run integration tests for API service core +mvn verify -pl openframe-api-service-core + +# Run all integration tests (slow - runs all module verifications) +mvn verify ``` -> Integration tests spin up real service containers (MongoDB, NATS) via Testcontainers. They run during the `verify` phase. +> **Testcontainers** automatically pulls and starts Docker containers for MongoDB, Redis, NATS, and other dependencies. No manual Docker Compose setup is needed for tests. -### Running a Specific Test +### Run a Specific Test Class ```bash # Run a specific test class -mvn test -pl openframe-data-mongo-sync -Dtest=NotificationReadStateServiceIT +mvn test -pl openframe-data-pinot -Dtest=PinotQueryBuilderTest # Run a specific test method -mvn test -pl openframe-data-mongo-sync -Dtest="NotificationReadStateServiceIT#shouldMarkNotificationAsRead" +mvn test -pl openframe-api-service-core -Dtest=CommandDispatchServiceTest#testDispatch ``` --- -## Development Workflow +## Module Development Workflow -### Typical Feature Development Flow +### Working on a Data Module -```mermaid -graph TD - A["Create feature branch"] --> B["Implement changes"] - B --> C["Run unit tests: mvn test -pl "] - C --> D["Run integration tests: mvn verify -pl "] - D --> E["Build full project: mvn install -DskipTests"] - E --> F["Open PR to main branch"] -``` +```bash +cd openframe-data-mongo-sync -### Watch Mode / Hot Reload +# Run all tests for this module +mvn test -openframe-oss-lib is a **library**, not a standalone application. There is no hot-reload in the traditional sense. Instead, iterate by: +# Run only integration tests +mvn verify -Dsurefire.skip=true -1. Making code changes in the module -2. Running `mvn install -pl -DskipTests` to install to local `.m2` -3. Your downstream service (which depends on this library) picks up the new version +# Run only unit tests (skip integration) +mvn test -Dfailsafe.skip=true +``` -For faster iteration in the downstream service: +### Working on the API Service Core ```bash -# Install specific module to local repo quickly -mvn install -pl openframe-security-core -DskipTests -q -``` +cd openframe-api-service-core ---- +# Build dependencies first +mvn install -pl openframe-core,openframe-exception,openframe-data-mongo-common,\ +openframe-data-mongo-sync,openframe-api-lib,openframe-security-core \ +-am -DskipTests + +# Then run API service core tests +mvn test -pl openframe-api-service-core +``` -## Debugging +### Working on Frontend Core -### IntelliJ Remote Debug (for downstream services) +```bash +cd openframe-frontend-core -When running a downstream Spring Boot service that uses these library modules, attach the IntelliJ debugger: +# Install Node.js dependencies +npm install -1. Run the service with: `mvn spring-boot:run -Dspring-boot.run.jvmArguments="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"` -2. In IntelliJ: **Run β†’ Edit Configurations β†’ Remote JVM Debug** -3. Set host to `localhost`, port to `5005` -4. Click **Debug** +# Start Storybook for component development +npm run storybook -### IntelliJ Test Debugging +# Run unit tests +npm run test -Right-click any test class or method β†’ **Debug 'TestName'**. IntelliJ uses Maven's test infrastructure with Lombok annotation processing enabled. +# Build the library +npm run build +``` --- -## Local Docker-Compose for Integration Tests +## Debug Configuration -A `docker-compose.yml` exists for the MongoDB sync integration test environment: +### IntelliJ IDEA Debug -```bash -# Start MongoDB for manual integration testing -cd openframe-data-mongo-sync/src/test/docker -docker-compose up -d +1. Open **Run/Debug Configurations** +2. Add a new **JUnit** configuration +3. Set: + - **Test kind**: Class + - **Class**: `com.openframe.data.mongo.sync.SomeTest` + - **Working directory**: `$MODULE_WORKING_DIR$` +4. Add **JVM options** for Testcontainers if needed: + ```text + -Dtestcontainers.reuse.enable=true + ``` -# Run integration tests against the running container -mvn verify -pl openframe-data-mongo-sync -``` +### Enable Testcontainers Container Reuse -> For most use cases, Testcontainers handles container lifecycle automatically. The docker-compose file is useful for persistent debugging sessions. +To speed up repeated integration test runs by reusing Docker containers: + +```bash +# Create or edit ~/.testcontainers.properties +echo "testcontainers.reuse.enable=true" >> ~/.testcontainers.properties +``` --- -## Dependency Management +## CI-Friendly Versioning -### Adding a New Dependency +The project uses Maven's CI-friendly `${revision}` property. The version is defined in the parent POM: -1. Add the version property to the root `pom.xml` `` section (if it's a new dependency) -2. Add the `` entry to `` in the root POM -3. Reference the dependency in the module's `pom.xml` **without a version** +```xml + + 6.0.10 + +``` -Example β€” adding a new library: +The `flatten-maven-plugin` resolves this during the `process-resources` phase. When building: -```xml - -1.2.3 - - - - com.example - my-library - ${my.library.version} - - - - - com.example - my-library - +```bash +# Override version for a specific build +mvn install -Drevision=6.0.11-SNAPSHOT -DskipTests ``` --- -## Common Issues +## Useful Maven Commands Reference + +```bash +# Show effective POM for a module +mvn help:effective-pom -pl openframe-core + +# Show dependency tree for a module +mvn dependency:tree -pl openframe-api-service-core + +# Check for dependency updates +mvn versions:display-dependency-updates -pl openframe-core -| Issue | Solution | -|-------|---------| -| `Cannot resolve symbol` (Lombok) | Enable annotation processing in IDE settings | -| Tests fail with `Connection refused` | Start Docker before running integration tests | -| `${revision}` not resolved | Run `mvn flatten:flatten` or upgrade Maven to 3.9+ | -| Slow builds | Use `-DskipTests`, `-T4` (parallel builds), or `-pl module -am` | -| `dependency:resolve` fails | Check `~/.m2/settings.xml` for GitHub Packages credentials | +# Validate the POM files +mvn validate + +# Show all modules in reactor +mvn help:evaluate -Dexpression=project.modules +``` --- -## Useful Development Aliases +## Local Publishing -Add to your shell profile for convenience: +To publish artifacts to the GitHub Maven Package Registry (for maintainers with write access): ```bash -# Build specific module quickly -alias mbi='mvn install -DskipTests' -alias mbt='mvn test' -alias mbv='mvn verify' - -# Build and install a specific module -function mbm() { - mvn install -pl "$1" -am -DskipTests -} +# Publish all modules +mvn deploy -DskipTests + +# Publish a specific module +mvn deploy -pl openframe-core -DskipTests ``` -Usage: +> This requires GitHub token configuration in `~/.m2/settings.xml` with `write:packages` scope. -```bash -mbm openframe-security-core -``` +--- + +## Common Issues + +| Problem | Solution | +|---------|---------| +| `Compilation failure: cannot find symbol` | Ensure annotation processing is enabled in your IDE; run `mvn generate-sources` | +| `Docker not available` in tests | Ensure Docker daemon is running: `docker info` | +| `Unable to connect to GitHub Packages` | Check `~/.m2/settings.xml` for correct `id=github` server entry | +| `revision` placeholder in deployed POM | The `flatten-maven-plugin` must run β€” use `mvn install` not `mvn package` | +| Out of memory during build | Increase Maven heap: `export MAVEN_OPTS="-Xmx2g"` | diff --git a/docs/development/testing/README.md b/docs/development/testing/README.md index 6bd092b81..a51b5f72e 100644 --- a/docs/development/testing/README.md +++ b/docs/development/testing/README.md @@ -1,295 +1,303 @@ # Testing Overview -This guide describes the testing strategy, structure, and conventions used across **openframe-oss-lib**. +`openframe-oss-lib` has a comprehensive testing strategy covering unit tests, integration tests with Testcontainers, and end-to-end test infrastructure. This guide explains the test structure, how to run tests, and how to write new tests. --- -## Test Structure and Organization +## Test Structure -Tests are co-located with their source code in the standard Maven layout: +Tests are organized into two main categories across all modules: -```text -openframe-/ -β”œβ”€β”€ src/ -β”‚ β”œβ”€β”€ main/java/com/openframe/... # Production code -β”‚ └── test/java/com/openframe/... # Test code -β”‚ β”œβ”€β”€ com/openframe/.../FooTest.java # Unit test -β”‚ └── com/openframe/.../FooIT.java # Integration test +```mermaid +graph LR + Tests["Tests in openframe-oss-lib"] + Unit["Unit Tests\n(src/test/java)"] + Integration["Integration Tests\n(src/test/java - *IT.java)"] + Infra["Test Infrastructure\nopenframe-test-service-core"] + + Tests --> Unit + Tests --> Integration + Tests --> Infra ``` -### Naming Conventions +### Unit Tests -| Convention | Type | Lifecycle Phase | -|-----------|------|----------------| -| `*Test.java` | Unit test | `mvn test` (Surefire) | -| `*Tests.java` | Unit test | `mvn test` (Surefire) | -| `*TestCase.java` | Unit test | `mvn test` (Surefire) | -| `*IT.java` | Integration test | `mvn verify` (Failsafe) | +- Located in `src/test/java/` following the pattern `*Test.java` +- No infrastructure dependencies (no Docker, no database) +- Fast execution (milliseconds) +- Configured via Maven Surefire plugin -The Maven Surefire plugin in the parent POM is configured to include all four patterns: +### Integration Tests -```xml - - **/Test*.java - **/*Test.java - **/*Tests.java - **/*TestCase.java - **/*IT.java - -``` +- Located in `src/test/java/` following the pattern `*IT.java` +- Use **Testcontainers** for real infrastructure (MongoDB, Redis, NATS) +- Slower execution (seconds to minutes) +- Configured via Maven Failsafe plugin +- Run with `mvn verify` + +### Test Infrastructure Module + +The `openframe-test-service-core` module provides a shared test harness for end-to-end API testing: + +| Component | Purpose | +|-----------|---------| +| `AuthFlow`, `AuthFlowOSS`, `AuthFlowSAAS` | Complete auth flow helpers | +| `AuthHelper`, `RequestSpecHelper` | Authentication and REST test setup | +| `*Api` classes (`DeviceApi`, `TicketApi`, etc.) | Typed API client helpers | +| `*Generator` classes | Test data factories | +| `*Page` classes | Page Object Model for UI tests | +| `MongoDB`, `Redis` | Test infrastructure containers | --- ## Running Tests -### All Unit Tests +### Run Unit Tests Only ```bash -# Run all unit tests across the entire project +# Run all unit tests across all modules mvn test # Run unit tests for a specific module mvn test -pl openframe-core -mvn test -pl openframe-data-nats +mvn test -pl openframe-exception mvn test -pl openframe-data-pinot +mvn test -pl openframe-api-service-core ``` -### Integration Tests - -Integration tests require a running Docker daemon (Testcontainers): +### Run Integration Tests ```bash -# Run integration tests for a specific module +# Run integration tests for a specific module (requires Docker) mvn verify -pl openframe-data-mongo-sync - -# Run integration + unit tests for a module +mvn verify -pl openframe-data-nats mvn verify -pl openframe-api-service-core -# Run all integration tests (may be slow β€” requires Docker) +# Run all integration tests mvn verify ``` -### Skipping Tests +### Run a Specific Test ```bash -# Skip all tests during build -mvn install -DskipTests +# Run a single test class +mvn test -pl openframe-data-pinot -Dtest=PinotQueryBuilderTest -# Skip integration tests only -mvn install -DskipITs +# Run a specific test method +mvn test -pl openframe-api-service-core \ + -Dtest=CommandDispatchServiceTest#testDispatch -# Run only unit tests (skip integration) -mvn test -DskipITs +# Run tests matching a pattern +mvn test -pl openframe-data-mongo-sync -Dtest="Notification*" ``` ---- - -## Integration Test Infrastructure +### Skip Tests -### Testcontainers - -Integration tests use [Testcontainers](https://testcontainers.com/) for infrastructure dependencies. Containers are automatically started and stopped per test class or suite. +```bash +# Skip all tests during build +mvn install -DskipTests -**MongoDB Integration Tests:** +# Skip only integration tests +mvn install -Dfailsafe.skip=true -```java -// Base class pattern used in openframe-data-mongo-sync -@SpringBootTest(classes = IntegrationTestApplication.class) -@ActiveProfiles("integration") -public abstract class BaseMongoIntegrationTest { - // Testcontainers manages MongoDB lifecycle -} +# Skip only unit tests +mvn install -Dsurefire.skip=true ``` -**NATS Integration Tests:** +--- -```java -// Base class pattern in openframe-data-nats -@SpringBootTest(classes = PublisherIntegrationTestApplication.class) -public abstract class BaseIntegrationTest { - // Testcontainers manages NATS lifecycle -} -``` +## Test Configuration -### Docker Compose (Alternative) +### Maven Surefire (Unit Tests) -For manual testing sessions, a Docker Compose file is available for MongoDB: +The parent POM configures Surefire to include: -```bash -cd openframe-data-mongo-sync/src/test/docker -docker-compose up -d +```xml + + **/Test*.java + **/*Test.java + **/*Tests.java + **/*TestCase.java + **/*IT.java + ``` ---- - -## Test Utilities - -### `openframe-test-service-core` - -This dedicated module provides reusable test infrastructure for **end-to-end and integration testing** of OpenFrame services: - -| Class | Purpose | -|-------|---------| -| `AuthHelper` | Authentication flow helpers | -| `RequestSpecHelper` | REST-Assured request specification builders | -| `AuthGenerator` | Generate test authentication tokens | -| `OrganizationGenerator` | Generate test organization data | -| `DeviceGenerator` | Generate test device data | -| `TicketGenerator` | Generate test ticket data | -| `NotificationFixtures` | Pre-built notification test data | +### Testcontainers -Example usage: +Integration tests use Testcontainers to start real infrastructure automatically. Base classes handle container lifecycle: ```java -@Autowired -private OrganizationGenerator organizationGenerator; - -@Test -void shouldCreateOrganization() { - var org = organizationGenerator.createOrganization("Test Org"); - assertThat(org.getId()).isNotNull(); +// Example: BaseMongoIntegrationTest (openframe-data-mongo-sync) +// Provides: @Testcontainers, MongoDB container, Spring context +public abstract class BaseMongoIntegrationTest { + // MongoDB container started once per test class + // Spring Boot test context shared for speed } ``` -### GraphQL Test Helpers - -GraphQL integration tests use pre-built query helpers: +**Speed Up Integration Tests:** -```java -// Available in openframe-test-service-core -DeviceQueries.listDevices(filterInput) -OrganizationQueries.listOrganizations(filterInput) -TicketQueries.listTickets(filterInput) +```bash +# Enable container reuse across test runs +echo "testcontainers.reuse.enable=true" >> ~/.testcontainers.properties ``` --- ## Writing New Tests -### Unit Test Template +### Unit Test Example ```java import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.ExtendWith; -import org.mockito.InjectMocks; -import org.mockito.Mock; -import org.mockito.junit.jupiter.MockitoExtension; - import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.when; - -@ExtendWith(MockitoExtension.class) -class MyServiceTest { - - @Mock - private MyRepository repository; - @InjectMocks - private MyService service; +class SlugUtilTest { @Test - void shouldReturnExpectedResult() { + void shouldConvertNameToSlug() { // Given - when(repository.findById("id-1")).thenReturn(Optional.of(new MyEntity("id-1"))); + String name = "My MSP Organization"; // When - var result = service.getById("id-1"); + String slug = SlugUtil.toSlug(name); // Then - assertThat(result).isPresent(); - assertThat(result.get().getId()).isEqualTo("id-1"); + assertThat(slug).isEqualTo("my-msp-organization"); } } ``` -### Integration Test Template (MongoDB) +### Integration Test Example (MongoDB) ```java -import org.junit.jupiter.api.AfterEach; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.testcontainers.junit.jupiter.Testcontainers; -class MyRepositoryIT extends BaseMongoIntegrationTest { +@SpringBootTest +@Testcontainers +class OrganizationRepositoryIT extends BaseMongoIntegrationTest { @Autowired - private MyRepository repository; - - @AfterEach - void cleanup() { - repository.deleteAll(); - } + private OrganizationRepository organizationRepository; @Test - void shouldPersistAndRetrieveEntity() { + void shouldSaveAndRetrieveOrganization() { // Given - var entity = new MyEntity("test-id", "test-name"); - repository.save(entity); + Organization org = new Organization(); + org.setTenantId("oss"); + org.setName("Test MSP"); // When - var found = repository.findById("test-id"); + Organization saved = organizationRepository.save(org); // Then - assertThat(found).isPresent(); - assertThat(found.get().getName()).isEqualTo("test-name"); + assertThat(saved.getId()).isNotNull(); + assertThat(organizationRepository.findById(saved.getId())) + .isPresent() + .hasValueSatisfying(o -> assertThat(o.getName()).isEqualTo("Test MSP")); } } ``` -### Naming and Structure Conventions +### GraphQL Integration Test Example -- Use **Given / When / Then** structure in test methods -- Test method names should describe behavior: `shouldReturnNotFoundWhenEntityDoesNotExist` -- One assertion concept per test where possible -- Always clean up test data in `@AfterEach` for integration tests -- Use `@DisplayName` for complex test scenarios +```java +// From openframe-api-service-core integration tests +// Uses NotificationDataFetcherIT pattern: +@SpringBootTest +class MyDataFetcherIT extends BaseMongoIntegrationTest { ---- + @Autowired + private DgsQueryExecutor dgsQueryExecutor; -## Test Coverage + @Test + void shouldFetchDevices() { + String query = """ + query { + devices(first: 10) { + edges { + node { id hostname } + } + } + } + """; + + // Execute with mock authentication context + var result = dgsQueryExecutor.executeAndExtractJsonPath( + query, "$.data.devices.edges[*].node.hostname" + ); + + assertThat(result).isNotNull(); + } +} +``` + +--- -The project does not enforce a specific line coverage percentage, but the following guidelines apply: +## Test Data Generators -| Component Type | Expected Coverage | -|---------------|------------------| -| Core utilities (`openframe-core`, `openframe-exception`) | High (>80%) | -| Domain services | Medium-High (>70%) | -| Configuration classes | Low (Spring-managed beans) | -| Repository implementations | Covered by integration tests | +The `openframe-test-service-core` module provides ready-to-use data generators: -Focus on **behavioral coverage** (testing what the code does) over line coverage metrics. +```java +// Generate test data for API tests +OrganizationGenerator.createRequest() // β†’ CreateOrganizationRequest +TicketGenerator.createInput() // β†’ CreateTicketInput +DeviceGenerator.machine() // β†’ Machine document +AuthGenerator.credentials() // β†’ test credentials +``` --- ## Notification Integration Tests -The notification subsystem has particularly thorough integration test coverage in `openframe-data-mongo-sync`: +The notification system has dedicated performance and integration tests: + +| Test | Purpose | +|------|---------| +| `NotificationLoadTestIT` | Load testing notification reads | +| `NotificationReadStateIndexUsageIT` | Validates index usage | +| `CustomNotificationRepositoryPaginationIT` | Cursor pagination correctness | +| `NotificationReadStateServiceIT` | Read/unread state management | -| Test Class | What It Tests | -|-----------|--------------| -| `NotificationContextDispatchIT` | Notification dispatch with custom context | -| `NotificationReadStateIndexesIT` | MongoDB index usage for read state | -| `NotificationLoadTestIT` | Load and performance testing | -| `NotificationReadStateIndexUsageIT` | Query plan analysis | -| `CustomNotificationRepositoryPaginationIT` | Cursor-based notification pagination | +These tests also serve as **performance benchmarks** using `PerfResultRecorder` and `MeasurementStats`. + +--- -Run them with: +## Frontend Tests (openframe-frontend-core) ```bash -mvn verify -pl openframe-data-mongo-sync -Dtest=Notification*IT +cd openframe-frontend-core + +# Run all unit tests +npm run test + +# Run tests in watch mode +npm run test -- --watch + +# Run with coverage +npm run test -- --coverage ``` +Frontend tests use **Vitest** with the configuration in `vitest.config.ts`. + --- -## Pinot Repository Tests +## Coverage Requirements -The `openframe-data-pinot` module contains unit tests for query building: +While formal coverage thresholds are not enforced by default, follow these guidelines: -```bash -mvn test -pl openframe-data-pinot -``` +| Type | Target Coverage | +|------|----------------| +| Domain services | 80%+ unit test coverage | +| Utility classes | 90%+ unit test coverage | +| Repository custom implementations | Integration test coverage | +| Controllers / DataFetchers | Integration test coverage | + +--- + +## CI Test Execution -Key test classes: +Tests run automatically in CI. The Surefire configuration includes `*IT.java` in unit test scanning, but integration tests that require Testcontainers run in the `verify` phase via Failsafe. -- `PinotQueryBuilderTest` β€” Validates SQL query generation -- `PinotClientDeviceRepositoryTest` β€” Device query logic -- `PinotClientLogRepositoryTest` β€” Log query logic +For questions about test failures or patterns, join the [OpenMSP Slack community](https://www.openmsp.ai/). diff --git a/docs/diagrams/architecture/README-10.mmd b/docs/diagrams/architecture/README-10.mmd new file mode 100644 index 000000000..936f0059c --- /dev/null +++ b/docs/diagrams/architecture/README-10.mmd @@ -0,0 +1,15 @@ +sequenceDiagram + participant User + participant Gateway + participant Auth + participant API + participant Stream + participant Data + + User->>Gateway: Login Request + Gateway->>Auth: OAuth2 Flow + Auth-->>Gateway: JWT + Gateway->>API: Authenticated Request + API->>Data: CRUD Operation + API->>Stream: Emit Event + Stream->>Data: Persist Unified Event diff --git a/docs/diagrams/architecture/README-11.mmd b/docs/diagrams/architecture/README-11.mmd new file mode 100644 index 000000000..1d85c2f60 --- /dev/null +++ b/docs/diagrams/architecture/README-11.mmd @@ -0,0 +1,17 @@ +flowchart LR + Frontend + Gateway + Auth + API + Management + Data + Stream + + Frontend --> Gateway + Gateway --> Auth + Gateway --> API + API --> Data + API --> Stream + Stream --> Data + Management --> Data + Management --> Stream diff --git a/docs/diagrams/architecture/README-2.mmd b/docs/diagrams/architecture/README-2.mmd index f0028d3df..70c43b485 100644 --- a/docs/diagrams/architecture/README-2.mmd +++ b/docs/diagrams/architecture/README-2.mmd @@ -1,7 +1,4 @@ -flowchart TD - Browser --> AuthController["Auth Controllers"] - AuthController --> TenantContext["TenantContextFilter"] - TenantContext --> AuthServer["AuthorizationServerConfig"] - AuthServer --> TenantKeyService["TenantKeyService"] - TenantKeyService --> Mongo - AuthServer --> JWT["Signed JWT"] +flowchart LR + Frontend --> API + API --> Contracts["Api Lib Contracts"] + Contracts --> Data diff --git a/docs/diagrams/architecture/README-3.mmd b/docs/diagrams/architecture/README-3.mmd index 8873c03c9..5d31642f1 100644 --- a/docs/diagrams/architecture/README-3.mmd +++ b/docs/diagrams/architecture/README-3.mmd @@ -1,7 +1,8 @@ flowchart TD - Request --> OriginSanitizer - OriginSanitizer --> JwtAuth - JwtAuth --> RoleCheck - RoleCheck --> RouteDecision - RouteDecision --> ToolResolver - ToolResolver --> UpstreamTool + Client["Frontend / API Client"] --> Gateway + Gateway --> API + API --> Controllers + API --> GraphQL + Controllers --> Services + GraphQL --> Services + Services --> Data diff --git a/docs/diagrams/architecture/README-4.mmd b/docs/diagrams/architecture/README-4.mmd index 36a3b850e..c952c366c 100644 --- a/docs/diagrams/architecture/README-4.mmd +++ b/docs/diagrams/architecture/README-4.mmd @@ -1,6 +1,5 @@ flowchart TD - DebeziumEvent --> Deserializer - Deserializer --> Enrichment - Enrichment --> EventTypeMapper - EventTypeMapper --> Cassandra - EventTypeMapper --> KafkaOut + User --> AuthServer["Authorization Service Core"] + AuthServer --> JWT["Tenant-Scoped JWT"] + JWT --> Gateway + Gateway --> API diff --git a/docs/diagrams/architecture/README-5.mmd b/docs/diagrams/architecture/README-5.mmd index 259b45977..058bc5856 100644 --- a/docs/diagrams/architecture/README-5.mmd +++ b/docs/diagrams/architecture/README-5.mmd @@ -1,4 +1,8 @@ flowchart LR - StreamService --> PinotCluster - PinotCluster --> PinotRepositories - PinotRepositories --> ApiService + Browser --> Gateway + Agent --> Gateway + ExternalAPI --> Gateway + + Gateway --> API + Gateway --> Auth + Gateway --> Tools diff --git a/docs/diagrams/architecture/README-6.mmd b/docs/diagrams/architecture/README-6.mmd index ef6842c36..cb2493c0c 100644 --- a/docs/diagrams/architecture/README-6.mmd +++ b/docs/diagrams/architecture/README-6.mmd @@ -1,5 +1,6 @@ flowchart TD Startup --> Initializers - Initializers --> StreamsReady - StreamsReady --> Schedulers - Schedulers --> ExternalSystems + Initializers --> Data + Scheduler --> Redis["ShedLock (Redis)"] + Scheduler --> Data + Scheduler --> Stream diff --git a/docs/diagrams/architecture/README-7.mmd b/docs/diagrams/architecture/README-7.mmd index 2f8a49559..f008ad9ae 100644 --- a/docs/diagrams/architecture/README-7.mmd +++ b/docs/diagrams/architecture/README-7.mmd @@ -1,10 +1,5 @@ -sequenceDiagram - participant Browser - participant BFF - participant AuthServer - - Browser->>BFF: GET /oauth/login - BFF->>AuthServer: Redirect with PKCE - AuthServer->>BFF: Callback with code - BFF->>AuthServer: Exchange code - BFF->>Browser: Set cookies +flowchart TD + Services --> Repositories + Repositories --> CustomImpl["Custom Repository"] + CustomImpl --> MongoTemplate + MongoTemplate --> MongoDB diff --git a/docs/diagrams/architecture/README-8.mmd b/docs/diagrams/architecture/README-8.mmd index 05e53be6c..06d0df681 100644 --- a/docs/diagrams/architecture/README-8.mmd +++ b/docs/diagrams/architecture/README-8.mmd @@ -1,19 +1,10 @@ flowchart TD - Client --> Gateway - Gateway --> ExternalAPI - Gateway --> ApiCore - - ApiCore --> Authz - ApiCore --> Mongo - ApiCore --> Redis - ApiCore --> Pinot - - Authz --> Mongo - Authz --> JWT - - Stream --> Kafka - Stream --> Cassandra - Stream --> Pinot - - Management --> Redis - Management --> NATS + Tools --> Debezium + Debezium --> KafkaTopic + KafkaTopic --> Listener + Listener --> Deserializer + Deserializer --> Enrichment + Enrichment --> Mapper + Mapper --> Handler + Handler --> Cassandra + Handler --> OutboundKafka diff --git a/docs/diagrams/architecture/README-9.mmd b/docs/diagrams/architecture/README-9.mmd new file mode 100644 index 000000000..14d825ff4 --- /dev/null +++ b/docs/diagrams/architecture/README-9.mmd @@ -0,0 +1,6 @@ +flowchart TD + EmbeddableChat --> UnifiedChat + UnifiedChat --> GuideMode["SSE Mode"] + UnifiedChat --> MingoMode["NATS Mode"] + GuideMode --> API + MingoMode --> Stream diff --git a/docs/diagrams/architecture/README.mmd b/docs/diagrams/architecture/README.mmd index 7f3f41a75..28ee9ef52 100644 --- a/docs/diagrams/architecture/README.mmd +++ b/docs/diagrams/architecture/README.mmd @@ -1,15 +1,21 @@ flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service"] - Gateway --> ApiCore["API Service Core"] + Frontend["Frontend Core UI and Chat"] + Gateway["Gateway Service Core"] + Auth["Authorization Service Core"] + API["API Service Core (HTTP + GraphQL)"] + Management["Management Service Core"] + Data["Data Models and Repositories Mongo"] + Stream["Stream Processing Kafka"] + Cassandra["Cassandra (Unified Events)"] + Tools["Integrated Tools (MeshCentral, Tactical, Fleet)"] - ApiCore --> Authz["Authorization Service Core"] - ApiCore --> Stream["Stream Service Core"] - ApiCore --> Management["Management Service Core"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - Stream --> Kafka["Kafka"] - Stream --> Pinot["Pinot"] - ApiCore --> Redis["Redis"] - Management --> NATS["NATS"] + Frontend --> Gateway + Gateway --> Auth + Gateway --> API + API --> Data + API --> Stream + Stream --> Cassandra + Stream --> Data + Management --> Data + Management --> Stream + Tools --> Stream diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-2.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-2.mmd deleted file mode 100644 index 171348e9c..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-2.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Request["Query Request with Filters"] --> Service["Query Service"] - Service --> Repo["Custom Repository"] - Repo --> Result["CountedGenericQueryResult"] - Result --> API["Serialized API Response"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-3.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-3.mmd deleted file mode 100644 index 76d425bb5..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["Client"] --> Args["ConnectionArgs"] - Args --> Decode["CursorCodec.decode()"] - Decode --> Query["Repository Query with Limit"] - Query --> Encode["CursorCodec.encode()"] - Encode --> Response["Connection Response with Edges"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-4.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-4.mmd deleted file mode 100644 index 0bba98fe8..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-4.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - Raw["Raw Internal Cursor"] --> Encode["Base64 Encode"] - Encode --> Opaque["Opaque Cursor String"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-5.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-5.mmd deleted file mode 100644 index 293cd83af..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-5.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - Opaque["Opaque Cursor"] --> Decode["Base64 Decode"] - Decode --> Raw["Raw Internal Cursor"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-6.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-6.mmd deleted file mode 100644 index 91f5ede92..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-6.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["Client Mutation"] --> Input["MutationDeleteInput"] - Input --> Validate["Validation Layer"] - Validate --> Service["Domain Service"] - Service --> Repo["Repository Delete by ID"] - Repo --> Response["Mutation Result"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination-7.mmd b/docs/diagrams/architecture/api-contracts-and-pagination-7.mmd deleted file mode 100644 index 62c5f782c..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination-7.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Step1["Client Requests first 20 after Cursor"] --> Step2["ConnectionArgs Validated"] - Step2 --> Step3["CursorCodec.decode()"] - Step3 --> Step4["Repository Query with Limit 21"] - Step4 --> Step5["Determine hasNextPage"] - Step5 --> Step6["CursorCodec.encode() for Edges"] - Step6 --> Step7["Return Connection Response"] diff --git a/docs/diagrams/architecture/api-contracts-and-pagination.mmd b/docs/diagrams/architecture/api-contracts-and-pagination.mmd deleted file mode 100644 index 2e19bb2b8..000000000 --- a/docs/diagrams/architecture/api-contracts-and-pagination.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["Client Application"] --> API["REST Controllers / GraphQL DataFetchers"] - API --> Contracts["Api Contracts And Pagination"] - Contracts --> Services["Domain Query Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("MongoDB")] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-2.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-2.mmd deleted file mode 100644 index 238250c63..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - LogFilterCriteria["LogFilterCriteria"] --> DateRange["Date Range"] - LogFilterCriteria --> EventTypes["Event Types"] - LogFilterCriteria --> ToolTypes["Tool Types"] - LogFilterCriteria --> Severities["Severities"] - LogFilterCriteria --> Organizations["Organization IDs"] - LogFilterCriteria --> Device["Device ID"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-3.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-3.mmd deleted file mode 100644 index feae0ce16..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - DeviceFilterCriteria["DeviceFilterCriteria"] --> Statuses["Device Statuses"] - DeviceFilterCriteria --> Types["Device Types"] - DeviceFilterCriteria --> OsTypes["OS Types"] - DeviceFilterCriteria --> Orgs["Organization IDs"] - DeviceFilterCriteria --> Tags["Tag Keys / Values"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-4.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-4.mmd deleted file mode 100644 index 278d589a8..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Organization["Organization (Mongo Document)"] --> OrganizationResponse["OrganizationResponse DTO"] - OrganizationResponse --> RestAPI["REST API"] - OrganizationResponse --> GraphQL["GraphQL API"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-5.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-5.mmd deleted file mode 100644 index 779b2cc46..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - GraphQLInput["GraphQL Filter Input"] --> Criteria["FilterCriteria DTO"] - Criteria --> Service["Query Service"] - Service --> MongoFilter["Mongo Query Filter"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos-6.mmd b/docs/diagrams/architecture/api-domain-filters-dtos-6.mmd deleted file mode 100644 index ced28036e..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - DeviceFilterCriteria --> DeviceQueryFilter["DeviceQueryFilter"] - EventFilterCriteria --> EventQueryFilter["EventQueryFilter"] - ToolFilterCriteria --> ToolQueryFilter["ToolQueryFilter"] diff --git a/docs/diagrams/architecture/api-domain-filters-dtos.mmd b/docs/diagrams/architecture/api-domain-filters-dtos.mmd deleted file mode 100644 index 5a45aaf4d..000000000 --- a/docs/diagrams/architecture/api-domain-filters-dtos.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Client["Client (REST or GraphQL)"] --> ApiInput["GraphQL Input / REST Request"] - ApiInput --> DomainFilter["Api Domain Filters Dtos"] - DomainFilter --> QueryLayer["Mongo Query Filters"] - QueryLayer --> Repository["Mongo Repositories"] - Repository --> Database[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-contracts-2.mmd b/docs/diagrams/architecture/api-lib-contracts-2.mmd new file mode 100644 index 000000000..a80679195 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-contracts-2.mmd @@ -0,0 +1,6 @@ +flowchart TD + Client["Client Filters"] --> Criteria["DeviceFilterCriteria"] + Criteria --> Service["Service Layer"] + Service --> Repo["Mongo Repository"] + Repo --> Results["Filtered Devices"] + Results --> Filters["DeviceFilters Response"] diff --git a/docs/diagrams/architecture/api-lib-contracts-3.mmd b/docs/diagrams/architecture/api-lib-contracts-3.mmd new file mode 100644 index 000000000..c527b4e78 --- /dev/null +++ b/docs/diagrams/architecture/api-lib-contracts-3.mmd @@ -0,0 +1,9 @@ +sequenceDiagram + participant Client + participant ApiService as "API Service" + participant Agent + + Client->>ApiService: RunCommandInput + ApiService->>Agent: Dispatch command + Agent-->>ApiService: executionId + ApiService-->>Client: CommandDispatchResponse diff --git a/docs/diagrams/architecture/api-lib-contracts-4.mmd b/docs/diagrams/architecture/api-lib-contracts-4.mmd new file mode 100644 index 000000000..f607e202e --- /dev/null +++ b/docs/diagrams/architecture/api-lib-contracts-4.mmd @@ -0,0 +1,5 @@ +flowchart TD + Request["Create / Update Request"] --> Mapper["OrganizationMapper"] + Mapper --> Entity["Organization Entity"] + Entity --> Mapper + Mapper --> Response["OrganizationResponse"] diff --git a/docs/diagrams/architecture/api-lib-contracts-5.mmd b/docs/diagrams/architecture/api-lib-contracts-5.mmd new file mode 100644 index 000000000..0f0bfefab --- /dev/null +++ b/docs/diagrams/architecture/api-lib-contracts-5.mmd @@ -0,0 +1,13 @@ +flowchart TD + Contracts["Api Lib Contracts"] + Api["API Service Core"] + Auth["Authorization Service"] + Data["Mongo Data Layer"] + Gateway["Gateway Service"] + Frontend["Frontend UI"] + + Frontend --> Gateway + Gateway --> Auth + Gateway --> Api + Api --> Contracts + Api --> Data diff --git a/docs/diagrams/architecture/api-lib-contracts.mmd b/docs/diagrams/architecture/api-lib-contracts.mmd new file mode 100644 index 000000000..cf600b21c --- /dev/null +++ b/docs/diagrams/architecture/api-lib-contracts.mmd @@ -0,0 +1,8 @@ +flowchart LR + Frontend["Frontend UI and Chat"] -->|"GraphQL / REST"| ApiService["API Service Core"] + ApiService -->|"Uses DTOs"| Contracts["Api Lib Contracts"] + Contracts -->|"Maps to"| DataModels["Mongo Data Models"] + ApiService -->|"Queries"| DataModels + Stream["Stream Processing Kafka"] -->|"Publishes Events"| DataModels + Gateway["Gateway Service Core"] -->|"Secured Requests"| ApiService + Auth["Authorization Service Core"] -->|"JWT / Tenant Context"| ApiService diff --git a/docs/diagrams/architecture/api-lib-core-services-2.mmd b/docs/diagrams/architecture/api-lib-core-services-2.mmd deleted file mode 100644 index 8f71addba..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - DataLoader["InstalledAgentDataLoader"] --> Service["InstalledAgentService"] - Service --> Repo["InstalledAgentRepository"] - Repo --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-core-services-3.mmd b/docs/diagrams/architecture/api-lib-core-services-3.mmd deleted file mode 100644 index 5a2e40ac8..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - ToolDataLoader["ToolConnectionDataLoader"] --> ToolService["ToolConnectionService"] - ToolService --> ToolRepo["ToolConnectionRepository"] - ToolRepo --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-core-services-4.mmd b/docs/diagrams/architecture/api-lib-core-services-4.mmd deleted file mode 100644 index 45ae4bcf6..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - API["API Layer"] --> TicketService["TicketQueryService"] - TicketService --> QueryFilter["TicketQueryFilter"] - TicketService --> Repo["TicketRepository"] - Repo --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-core-services-5.mmd b/docs/diagrams/architecture/api-lib-core-services-5.mmd deleted file mode 100644 index d07ad4a9b..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Save["Save KnowledgeBaseItem"] --> MongoEvent["BeforeConvertEvent"] - MongoEvent --> Listener["KnowledgeBasePublishLifecycleListener"] - Listener --> Stamp["Set publishedAt if null"] - Stamp --> Persist[("MongoDB")] diff --git a/docs/diagrams/architecture/api-lib-core-services-6.mmd b/docs/diagrams/architecture/api-lib-core-services-6.mmd deleted file mode 100644 index 7181e803f..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-6.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - DeviceStatusUpdate["Machine Status Updated"] --> Processor["DeviceStatusProcessor"] - Processor --> DefaultImpl["DefaultDeviceStatusProcessor"] diff --git a/docs/diagrams/architecture/api-lib-core-services-7.mmd b/docs/diagrams/architecture/api-lib-core-services-7.mmd deleted file mode 100644 index 91e5a1f40..000000000 --- a/docs/diagrams/architecture/api-lib-core-services-7.mmd +++ /dev/null @@ -1,13 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> DataFetcher["MachineDataFetcher"] - DataFetcher --> AgentLoader["InstalledAgentDataLoader"] - DataFetcher --> ToolLoader["ToolConnectionDataLoader"] - - AgentLoader --> AgentService["InstalledAgentService"] - ToolLoader --> ToolService["ToolConnectionService"] - - AgentService --> AgentRepo["InstalledAgentRepository"] - ToolService --> ToolRepo["ToolConnectionRepository"] - - AgentRepo --> Mongo[("MongoDB")] - ToolRepo --> Mongo diff --git a/docs/diagrams/architecture/api-lib-core-services.mmd b/docs/diagrams/architecture/api-lib-core-services.mmd deleted file mode 100644 index 86fb571e2..000000000 --- a/docs/diagrams/architecture/api-lib-core-services.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Controllers["REST Controllers"] --> Services["Api Lib Core Services"] - DataFetchers["GraphQL Data Fetchers"] --> Services - Services --> Repositories["Mongo Repositories"] - Repositories --> MongoDB[("MongoDB")] - - Services --> Processors["Domain Processors"] diff --git a/docs/diagrams/architecture/api-organization-mapping-2.mmd b/docs/diagrams/architecture/api-organization-mapping-2.mmd deleted file mode 100644 index 7964c3000..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Request["CreateOrganizationRequest"] --> MapperCreate["toEntity()"] - MapperCreate --> GenId["Generate UUID"] - MapperCreate --> MapFields["Map Scalar Fields"] - MapperCreate --> MapContact["Map Contact Information"] - MapContact --> MapAddress["Map Address"] - MapContact --> MapContacts["Map Contact Persons"] - GenId --> OrgEntity["Organization Entity"] - MapFields --> OrgEntity - MapContact --> OrgEntity diff --git a/docs/diagrams/architecture/api-organization-mapping-3.mmd b/docs/diagrams/architecture/api-organization-mapping-3.mmd deleted file mode 100644 index 4b4b0d04a..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - UpdateRequest["UpdateOrganizationRequest"] --> CheckField["Field != null?"] - CheckField -->|"Yes"| ApplyUpdate["Set Field on Entity"] - CheckField -->|"No"| Skip["Keep Existing Value"] - ApplyUpdate --> OrgEntity["Updated Organization"] - Skip --> OrgEntity diff --git a/docs/diagrams/architecture/api-organization-mapping-4.mmd b/docs/diagrams/architecture/api-organization-mapping-4.mmd deleted file mode 100644 index 24ccc880a..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart LR - OrgEntity["Organization Entity"] --> MapperResponse["toResponse()"] - MapperResponse --> MapScalars["Map Scalar Fields"] - MapperResponse --> MapStatus["Enum to String"] - MapperResponse --> MapNested["Map Contact Information DTO"] - MapScalars --> Response["OrganizationResponse"] - MapStatus --> Response - MapNested --> Response diff --git a/docs/diagrams/architecture/api-organization-mapping-5.mmd b/docs/diagrams/architecture/api-organization-mapping-5.mmd deleted file mode 100644 index 1a2d95abc..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Dto["ContactInformationDto"] --> CheckFlag{{"Same As Physical?"}} - CheckFlag -->|"Yes"| CopyAddr["Copy Physical Address"] - CheckFlag -->|"No"| MapMail["Map Mailing Address"] - CopyAddr --> Entity["ContactInformation Entity"] - MapMail --> Entity diff --git a/docs/diagrams/architecture/api-organization-mapping-6.mmd b/docs/diagrams/architecture/api-organization-mapping-6.mmd deleted file mode 100644 index 46c956ee5..000000000 --- a/docs/diagrams/architecture/api-organization-mapping-6.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart LR - Client["Client App"] --> RestAPI["REST or GraphQL API"] - RestAPI --> MapperCreate["OrganizationMapper"] - MapperCreate --> Service["Organization Service"] - Service --> Repo["Mongo Repository"] - Repo --> DB["MongoDB"] - DB --> Repo - Repo --> Service - Service --> MapperResponse["OrganizationMapper"] - MapperResponse --> RestAPI - RestAPI --> Client diff --git a/docs/diagrams/architecture/api-organization-mapping.mmd b/docs/diagrams/architecture/api-organization-mapping.mmd deleted file mode 100644 index 38fda5d45..000000000 --- a/docs/diagrams/architecture/api-organization-mapping.mmd +++ /dev/null @@ -1,23 +0,0 @@ -flowchart LR - subgraph ApiLayer["API Layer"] - RestController["OrganizationController"] - GraphQLFetcher["OrganizationDataFetcher"] - end - - Mapper["OrganizationMapper"] - - subgraph DomainLayer["Domain & Persistence"] - OrgEntity["Organization Entity"] - MongoRepo["Mongo Repository"] - end - - RestController -->|"Create/Update Request"| Mapper - GraphQLFetcher -->|"Mutation Input"| Mapper - - Mapper -->|"Entity"| OrgEntity - OrgEntity --> MongoRepo - - MongoRepo --> OrgEntity - OrgEntity -->|"Response DTO"| Mapper - Mapper -->|"OrganizationResponse"| RestController - Mapper -->|"GraphQL DTO"| GraphQLFetcher diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-2.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-2.mmd deleted file mode 100644 index 1d4c2908b..000000000 --- a/docs/diagrams/architecture/api-service-core-config-and-security-2.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Request["HTTP Request"] --> Security["OAuth2 Resource Server"] - Security --> Controller["Controller Method"] - Controller --> Resolver["AuthPrincipalArgumentResolver"] - Resolver --> Principal["AuthPrincipal Injected"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-3.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-3.mmd deleted file mode 100644 index 5c7317a24..000000000 --- a/docs/diagrams/architecture/api-service-core-config-and-security-3.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Incoming["Incoming Request with JWT"] --> ExtractIssuer["Extract iss Claim"] - ExtractIssuer --> CacheCheck["Check JwtAuthenticationProvider Cache"] - CacheCheck -->|"Miss"| CreateDecoder["Create JwtDecoder from Issuer"] - CreateDecoder --> StoreCache["Store in Caffeine Cache"] - CacheCheck -->|"Hit"| UseProvider["Reuse Cached Provider"] - StoreCache --> Authenticate["Authenticate JWT"] - UseProvider --> Authenticate - Authenticate --> Principal["SecurityContext Updated"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-4.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-4.mmd deleted file mode 100644 index da7c9a7a1..000000000 --- a/docs/diagrams/architecture/api-service-core-config-and-security-4.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Startup["Application Startup"] --> LoadProps["Read OAuth Properties"] - LoadProps --> QueryRepo["Find Client by ID"] - QueryRepo -->|"Exists"| CompareSecret["Compare Secrets"] - CompareSecret -->|"Different"| UpdateSecret["Update and Save"] - CompareSecret -->|"Same"| Skip["No Action"] - QueryRepo -->|"Missing"| CreateClient["Create OAuthClient"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-5.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-5.mmd deleted file mode 100644 index e5f850975..000000000 --- a/docs/diagrams/architecture/api-service-core-config-and-security-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - GraphQLInput["GraphQL Query Input"] --> Scalar["Custom Scalar Coercing"] - Scalar --> Validation["Format Validation"] - Validation --> Parsed["Java Type (LocalDate / Instant / Long)"] - Parsed --> DataFetcher["Data Fetcher Execution"] diff --git a/docs/diagrams/architecture/api-service-core-config-and-security-6.mmd b/docs/diagrams/architecture/api-service-core-config-and-security-6.mmd deleted file mode 100644 index 265582add..000000000 --- a/docs/diagrams/architecture/api-service-core-config-and-security-6.mmd +++ /dev/null @@ -1,14 +0,0 @@ -sequenceDiagram - participant Client - participant Gateway - participant Api as "API Service" - participant Security as "SecurityConfig" - participant Controller - - Client->>Gateway: Request with Cookie or Token - Gateway->>Gateway: Validate JWT - Gateway->>Api: Forward request with Authorization header - Api->>Security: Resolve issuer and authenticate - Security->>Api: Populate SecurityContext - Api->>Controller: Invoke handler - Controller->>Controller: Inject AuthPrincipal diff --git a/docs/diagrams/architecture/api-service-core-config-and-security.mmd b/docs/diagrams/architecture/api-service-core-config-and-security.mmd deleted file mode 100644 index da769acb9..000000000 --- a/docs/diagrams/architecture/api-service-core-config-and-security.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - Gateway["Gateway Service"] -->|"JWT Forwarded"| ApiService["API Service Runtime"] - ApiService -->|"Uses"| ConfigModule["Api Service Core Config And Security"] - ApiService --> Controllers["REST Controllers"] - ApiService --> GraphQL["GraphQL Data Fetchers"] - ApiService --> Services["Domain Services"] - Services --> DataLayer["Mongo / Kafka / Redis"] diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-2.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-2.mmd deleted file mode 100644 index 0d3879f3c..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-2.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Resolver as "GraphQL Resolver" - participant Loader as "DataLoader" - participant Service as "Service/Repository" - - Resolver->>Loader: load([id1, id2, id3]) - Loader->>Service: Batch fetch entities - Service-->>Loader: List of entities - Loader-->>Resolver: Ordered results diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-3.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-3.mmd deleted file mode 100644 index 8533426d6..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Resolver --> InstalledAgentDataLoader - InstalledAgentDataLoader --> InstalledAgentService - InstalledAgentService --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-4.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-4.mmd deleted file mode 100644 index f66c56913..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Resolver --> KnowledgeBaseItemDataLoader - KnowledgeBaseItemDataLoader --> KnowledgeBaseItemRepository - KnowledgeBaseItemRepository --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-5.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-5.mmd deleted file mode 100644 index 85557302f..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Resolver --> OrganizationDataLoader - OrganizationDataLoader --> OrganizationRepository - OrganizationRepository --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-dataloaders-6.mmd b/docs/diagrams/architecture/api-service-core-dataloaders-6.mmd deleted file mode 100644 index a624e2aa6..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders-6.mmd +++ /dev/null @@ -1,20 +0,0 @@ -flowchart TD - subgraph graphql_layer["GraphQL Layer"] - DF["Data Fetchers"] - DL["DataLoaders"] - end - - subgraph service_layer["Service Layer"] - SVC["Domain Services"] - end - - subgraph data_layer["Data Layer"] - REPO["Mongo Repositories"] - DB[("MongoDB")] - end - - DF --> DL - DL --> SVC - DL --> REPO - SVC --> DB - REPO --> DB diff --git a/docs/diagrams/architecture/api-service-core-dataloaders.mmd b/docs/diagrams/architecture/api-service-core-dataloaders.mmd deleted file mode 100644 index bc6df6f13..000000000 --- a/docs/diagrams/architecture/api-service-core-dataloaders.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> GQL["GraphQL Data Fetchers"] - GQL --> DL["Api Service Core Dataloaders"] - DL --> Service["Domain Services"] - DL --> Repo["Mongo Repositories"] - Service --> Mongo[("MongoDB")] - Repo --> Mongo diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-2.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-2.mmd deleted file mode 100644 index 75dcb15cb..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Query["GraphQL Query"] --> Args["ConnectionArgs"] - Args --> Pagination["CursorPaginationCriteria"] - Pagination --> ServiceQuery["Service.query(...)"] - ServiceQuery --> Result["QueryResult"] - Result --> Mapper["GraphQL Mapper"] - Mapper --> Connection["Relay Connection"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-3.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-3.mmd deleted file mode 100644 index 39727fc93..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-3.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart LR - Query["devices query"] --> DeviceFetcher - DeviceFetcher --> DeviceService - DeviceFetcher --> MachineNode["Machine"] - MachineNode -->|"tags"| TagLoader - MachineNode -->|"installedAgents"| AgentLoader - TagLoader --> TagService - AgentLoader --> InstalledAgentService diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-4.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-4.mmd deleted file mode 100644 index 679e3471f..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Mutation["createArticle"] --> GetUser["Extract JWT User"] - GetUser --> Mapper - Mapper --> Service["KnowledgeBaseService"] - Service --> DB["Mongo Repository"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-5.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-5.mmd deleted file mode 100644 index 7b60b20bd..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-5.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - NodeQuery["node(id)"] --> Decode["Relay.fromGlobalId"] - Decode --> TypeSwitch["NodeType enum"] - TypeSwitch --> DeviceService - TypeSwitch --> OrganizationService - TypeSwitch --> EventService - TypeSwitch --> ToolService - TypeSwitch --> TagService diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-6.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-6.mmd deleted file mode 100644 index cf844476a..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-6.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Query["notifications"] --> CurrentRecipient - CurrentRecipient --> Principal["AuthPrincipal"] - Principal --> NotificationService - NotificationService --> ReadStateService diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-7.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-7.mmd deleted file mode 100644 index 24a82acbb..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - NotificationContext --> Resolver - Resolver -->|"type discriminator"| TypeMap - TypeMap --> GraphQLType diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-8.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers-8.mmd deleted file mode 100644 index cf75e6de7..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers-8.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - GraphQL["GraphQL DataFetchers"] --> Services - Services --> Domain["Domain Models"] - Services --> Repositories - Services --> Messaging["NATS / Kafka"] - Services --> Cache["Redis"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-datafetchers.mmd b/docs/diagrams/architecture/api-service-core-graphql-datafetchers.mmd deleted file mode 100644 index fe7cec603..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-datafetchers.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart LR - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetchers["DGS DataFetchers"] - DataFetchers --> Mappers["GraphQL Mappers"] - DataFetchers --> Services["Application Services"] - Services --> Repositories["Mongo / Pinot / Cassandra Repositories"] - - DataFetchers --> DataLoaders["DataLoaders (Batching)"] - DataLoaders --> Services - - DataFetchers --> Security["SecurityContext + JWT"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-2.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-2.mmd deleted file mode 100644 index 149360786..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - Query["GraphQL Query"] --> Args["Connection Arguments"] - Args --> Fetcher["Data Fetcher"] - Fetcher --> Service["Service Layer"] - Service --> Edge["GenericEdge"] - Edge --> Connection["CountedGenericConnection"] - Connection --> Client["GraphQL Client"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-3.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-3.mmd deleted file mode 100644 index 402106ef8..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - GraphQLInput["GraphQL Filter Input"] --> DataFetcher["Data Fetcher"] - DataFetcher --> Criteria["Domain Filter Criteria"] - Criteria --> Repository["Custom Repository"] - Repository --> Results["Domain Documents"] - Results --> Connection["Connection DTO"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-4.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-4.mmd deleted file mode 100644 index c3ebeb57e..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-4.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["GraphQL Mutation"] --> Input["Mutation Input DTO"] - Input --> Validation["Bean Validation"] - Validation --> Processor["Service or Processor"] - Processor --> Repository["Mongo Repository"] - Repository --> Response["Response DTO"] diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos-5.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos-5.mmd deleted file mode 100644 index fb0a10865..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos-5.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - UI["Frontend UI"] --> Query["GraphQL Query or Mutation"] - Query --> DTOIn["Input DTO"] - DTOIn --> Fetcher["Data Fetcher"] - Fetcher --> Service["Core Service"] - Service --> Repo["Mongo Repository"] - Repo --> Domain["Domain Document"] - Domain --> DTOOut["Edge or Response DTO"] - DTOOut --> Connection["CountedGenericConnection"] - Connection --> UI diff --git a/docs/diagrams/architecture/api-service-core-graphql-dtos.mmd b/docs/diagrams/architecture/api-service-core-graphql-dtos.mmd deleted file mode 100644 index 0a767217f..000000000 --- a/docs/diagrams/architecture/api-service-core-graphql-dtos.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetcher["GraphQL Data Fetchers"] - DataFetcher --> Dtos["Api Service Core Graphql Dtos"] - Dtos --> Services["Core Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("Database")] diff --git a/docs/diagrams/architecture/api-service-core-http-and-graphql-2.mmd b/docs/diagrams/architecture/api-service-core-http-and-graphql-2.mmd new file mode 100644 index 000000000..2e2bbe913 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-http-and-graphql-2.mmd @@ -0,0 +1,12 @@ +sequenceDiagram + participant Client + participant Gateway + participant ApiService + participant AuthServer + + Client->>Gateway: HTTP Request + Gateway->>AuthServer: Validate JWT + AuthServer-->>Gateway: Token valid + Gateway->>ApiService: Forward request with Authorization header + ApiService->>ApiService: Resolve AuthPrincipal + ApiService-->>Client: JSON / GraphQL response diff --git a/docs/diagrams/architecture/api-service-core-http-and-graphql-3.mmd b/docs/diagrams/architecture/api-service-core-http-and-graphql-3.mmd new file mode 100644 index 000000000..80ee1fa69 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-http-and-graphql-3.mmd @@ -0,0 +1,7 @@ +flowchart LR + Request["HTTP Request"] --> Controller["@RestController"] + Controller --> Service["Application Service"] + Service --> Repository["Mongo Repository"] + Repository --> Service + Service --> Controller + Controller --> Response["HTTP Response"] diff --git a/docs/diagrams/architecture/api-service-core-http-and-graphql-4.mmd b/docs/diagrams/architecture/api-service-core-http-and-graphql-4.mmd new file mode 100644 index 000000000..045bb0fd2 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-http-and-graphql-4.mmd @@ -0,0 +1,8 @@ +flowchart TD + Query["GraphQL Query"] --> DataFetcher["@DgsQuery / @DgsMutation"] + DataFetcher --> Mapper["GraphQL Mapper"] + Mapper --> Service["Domain Service"] + Service --> Repository["Mongo Repository"] + Repository --> Service + Service --> Mapper + Mapper --> Connection["Connection / Edge DTO"] diff --git a/docs/diagrams/architecture/api-service-core-http-and-graphql-5.mmd b/docs/diagrams/architecture/api-service-core-http-and-graphql-5.mmd new file mode 100644 index 000000000..89a994dbb --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-http-and-graphql-5.mmd @@ -0,0 +1,8 @@ +flowchart TD + NodeQuery["node(id)"] --> NodeFetcher["NodeDataFetcher"] + NodeFetcher --> TypeSwitch["NodeTypeResolver"] + TypeSwitch --> Machine["Machine"] + TypeSwitch --> Organization["Organization"] + TypeSwitch --> Event["Event"] + TypeSwitch --> Ticket["Ticket"] + TypeSwitch --> KnowledgeBaseItem["KnowledgeBaseItem"] diff --git a/docs/diagrams/architecture/api-service-core-http-and-graphql-6.mmd b/docs/diagrams/architecture/api-service-core-http-and-graphql-6.mmd new file mode 100644 index 000000000..0e10121f9 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-http-and-graphql-6.mmd @@ -0,0 +1,6 @@ +flowchart LR + ClientArgs["first / after"] --> ConnectionArgs + ConnectionArgs --> CursorCriteria + CursorCriteria --> ServiceQuery + ServiceQuery --> QueryResult + QueryResult --> ConnectionDTO diff --git a/docs/diagrams/architecture/api-service-core-http-and-graphql-7.mmd b/docs/diagrams/architecture/api-service-core-http-and-graphql-7.mmd new file mode 100644 index 000000000..00ff28fb2 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-http-and-graphql-7.mmd @@ -0,0 +1,5 @@ +flowchart TD + GraphQLField["Resolve field"] --> DataLoader + DataLoader --> BatchLoad["Batch Repository Query"] + BatchLoad --> ResultMap + ResultMap --> Response diff --git a/docs/diagrams/architecture/api-service-core-http-and-graphql.mmd b/docs/diagrams/architecture/api-service-core-http-and-graphql.mmd new file mode 100644 index 000000000..338c84359 --- /dev/null +++ b/docs/diagrams/architecture/api-service-core-http-and-graphql.mmd @@ -0,0 +1,14 @@ +flowchart TD + Client["Frontend / API Client"] --> Gateway["Gateway Service Core"] + Gateway --> ApiService["Api Service Core Http And Graphql"] + + ApiService --> Controllers["REST Controllers"] + ApiService --> GraphQL["GraphQL DataFetchers"] + + Controllers --> Services["Application Services"] + GraphQL --> Services + + Services --> Mongo["Mongo Repositories"] + Services --> Stream["Kafka / Stream Processing"] + + ApiService --> Authz["Authorization Service Core"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-2.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-2.mmd deleted file mode 100644 index 4e1ad0da6..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Input["AssignableTarget Object"] --> CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| Org["Return Organization"] - CheckOrg -->|"No"| CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| Machine["Return Machine"] - CheckMachine -->|"No"| CheckTicket{"Ticket?"} - CheckTicket -->|"Yes"| Ticket["Return Ticket"] - CheckTicket -->|"No"| CheckKB{"KnowledgeBaseItem?"} - CheckKB -->|"Yes"| KB["Return KnowledgeBaseItem"] - CheckKB -->|"No"| Error["Throw IllegalArgumentException"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-3.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-3.mmd deleted file mode 100644 index 5341a0d37..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-3.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - NodeInput["Node Object"] --> CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| ReturnMachine["Return Machine"] - CheckMachine -->|"No"| CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| ReturnOrg["Return Organization"] - CheckOrg -->|"No"| CheckEvent{"Event?"} - CheckEvent -->|"Yes"| ReturnEvent["Return Event"] - CheckEvent -->|"No"| Continue["Other instanceof checks"] - Continue --> ReturnType["Return Matching Type"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-4.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-4.mmd deleted file mode 100644 index d122b76ab..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - DataFetcher["DataFetcher"] --> Domain["Domain Object"] - Domain --> Resolver["Node or AssignableTarget Resolver"] - Resolver --> SchemaType["GraphQL Schema Type"] - SchemaType --> Response["GraphQL Response"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution-5.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution-5.mmd deleted file mode 100644 index 60885d35a..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution-5.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart TB - DomainModel["Mongo Domain Model"] --> RelayResolver["Relay Type Resolver"] - RelayResolver --> GraphQLSchema["GraphQL Schema Types"] diff --git a/docs/diagrams/architecture/api-service-core-relay-type-resolution.mmd b/docs/diagrams/architecture/api-service-core-relay-type-resolution.mmd deleted file mode 100644 index df97025e3..000000000 --- a/docs/diagrams/architecture/api-service-core-relay-type-resolution.mmd +++ /dev/null @@ -1,14 +0,0 @@ -flowchart TD - Client["GraphQL Client"] --> Query["GraphQL Query"] - Query --> DGS["DGS Runtime"] - DGS --> Resolver["Type Resolver"] - Resolver --> DomainObj["Domain Object"] - DomainObj --> GraphQLType["GraphQL Type Name"] - - subgraph relay_layer["Relay Type Resolution Layer"] - Resolver - end - - subgraph domain_layer["Mongo Domain Model"] - DomainObj - end diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-10.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-10.mmd deleted file mode 100644 index 93e3f4c68..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-10.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Admin["Admin"] --> Controller["UserController"] - Controller --> Service["UserService"] - Service --> Repo["User Repository"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-11.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-11.mmd deleted file mode 100644 index d45884088..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-11.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Request["Incoming HTTP Request"] --> Filter["JWT Filter"] - Filter --> Principal["AuthPrincipal"] - Principal --> Controller["REST Controller"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-2.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-2.mmd deleted file mode 100644 index b7227563f..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-2.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Admin - participant Controller as AgentRegistrationSecretController - participant Service as AgentRegistrationSecretService - - Admin->>Controller: POST /agent/registration-secret/generate - Controller->>Service: generateNewSecret() - Service-->>Controller: AgentRegistrationSecretResponse - Controller-->>Admin: 201 Created diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-3.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-3.mmd deleted file mode 100644 index 38721fa21..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - User["Authenticated User"] --> Controller["ApiKeyController"] - Controller --> Service["ApiKeyService"] - Service --> Repo["BaseApiKeyRepository"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-4.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-4.mmd deleted file mode 100644 index 8f96fb140..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Agent["Agent"] --> Controller["DeviceController"] - Controller --> Service["DeviceService"] - Service --> DeviceDoc["Device Document"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-5.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-5.mmd deleted file mode 100644 index 41904c5b9..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-5.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Admin["Admin Action"] --> Controller["ForceAgentController"] - Controller --> InstallSvc["ForceToolInstallationService"] - Controller --> UpdateSvc["ForceToolAgentUpdateService"] - Controller --> ClientSvc["ForceClientUpdateService"] - - InstallSvc --> Kafka["Kafka Event"] - UpdateSvc --> Kafka - ClientSvc --> Kafka diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-6.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-6.mmd deleted file mode 100644 index c5f8a4036..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-6.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Admin - participant Controller as InvitationController - participant Service as InvitationService - - Admin->>Controller: POST /invitations - Controller->>Service: createInvitation(request) - Service-->>Controller: InvitationResponse - Controller-->>Admin: 201 Created diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-7.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-7.mmd deleted file mode 100644 index bb0eb550e..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Request["GET /me"] --> AuthCheck["AuthPrincipal Present?"] - AuthCheck -->|"Yes"| Response["Return User Info"] - AuthCheck -->|"No"| Unauthorized["401 Unauthorized"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-8.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-8.mmd deleted file mode 100644 index 5270d4876..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-8.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Admin["Admin"] --> Controller["OrganizationController"] - Controller --> CommandSvc["OrganizationCommandService"] - Controller --> DomainSvc["OrganizationService"] - CommandSvc --> Repo["Organization Repository"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers-9.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers-9.mmd deleted file mode 100644 index 845810ee8..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers-9.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Admin["Admin"] --> Controller["SSOConfigController"] - Controller --> Service["SSOConfigService"] - Service --> Strategy["Provider Strategy"] - Strategy --> Provider["Google / Microsoft"] diff --git a/docs/diagrams/architecture/api-service-core-rest-controllers.mmd b/docs/diagrams/architecture/api-service-core-rest-controllers.mmd deleted file mode 100644 index 6feac1f96..000000000 --- a/docs/diagrams/architecture/api-service-core-rest-controllers.mmd +++ /dev/null @@ -1,29 +0,0 @@ -flowchart TD - Client["Client / UI / Agent"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] - - subgraph rest_layer["REST Controller Layer"] - Controllers["Api Service Core Rest Controllers"] - end - - subgraph service_layer["Service Layer"] - CommandServices["Command Services"] - QueryServices["Query Services"] - DomainProcessors["Domain Processors"] - end - - subgraph data_layer["Data Layer"] - Mongo["Mongo Repositories"] - Redis["Redis Cache"] - Kafka["Kafka / Events"] - end - - ApiCore --> Controllers - Controllers --> CommandServices - Controllers --> QueryServices - Controllers --> DomainProcessors - - CommandServices --> Mongo - QueryServices --> Mongo - DomainProcessors --> Kafka - DomainProcessors --> Redis diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-2.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-2.mmd deleted file mode 100644 index 9eb324363..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-2.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - Request["SSOConfigRequest"] --> Normalize["Normalize Domains"] - Normalize --> GenericCheck["validateGenericPublicDomain()"] - GenericCheck --> ExistsCheck["validateExists()"] - ExistsCheck --> Decision{"Auto Provision?"} - Decision -->|"No"| Save["Save Config"] - Decision -->|"Yes"| RequireDomain["Require At Least One Domain"] - RequireDomain --> MicrosoftCheck{"Microsoft?"} - MicrosoftCheck -->|"Yes"| RequireTenant["Require msTenantId"] - MicrosoftCheck -->|"No"| Save - RequireTenant --> Save diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-3.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-3.mmd deleted file mode 100644 index 025f8d4a5..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-3.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - ListUsers["listUsers()"] --> FetchPage["UserRepository.findAll()"] - FetchPage --> MapResponse["UserMapper.toResponse()"] - MapResponse --> PostProcess["UserProcessor.postProcessUserGet()"] - - DeleteUser["softDeleteUser()"] --> ValidateSelf["Prevent Self Delete"] - ValidateSelf --> ValidateOwner["Prevent OWNER Delete"] - ValidateOwner --> MarkDeleted["Set Status DELETED"] - MarkDeleted --> SaveUser["UserRepository.save()"] - SaveUser --> PostDelete["UserProcessor.postProcessUserDeleted()"] diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-4.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-4.mmd deleted file mode 100644 index 8c3ba9978..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Upsert["upsertConfig()"] --> Exists{"Config Exists?"} - Exists -->|"Yes"| Update["Update Existing"] - Exists -->|"No"| Create["Create New"] - Update --> Encrypt["Encrypt Client Secret"] - Create --> Encrypt - Encrypt --> Save["SSOConfigRepository.save()"] - Save --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] diff --git a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors.mmd b/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors.mmd deleted file mode 100644 index 5d8c79678..000000000 --- a/docs/diagrams/architecture/api-service-core-user-sso-services-and-processors.mmd +++ /dev/null @@ -1,18 +0,0 @@ -flowchart TD - Controller["REST / GraphQL Controllers"] --> UserService["UserService"] - Controller --> SSOService["SSOConfigService"] - - UserService --> UserRepo["UserRepository"] - UserService --> UserProcessor["UserProcessor"] - - SSOService --> SSORepo["SSOConfigRepository"] - SSOService --> Encryption["EncryptionService"] - SSOService --> DomainValidation["DomainValidationService"] - SSOService --> SSOProcessor["SSOConfigProcessor"] - - subgraph processors_layer["Post-Processing Layer"] - UserProcessor --> DefaultUserProcessor["DefaultUserProcessor"] - SSOProcessor --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] - InvitationProcessor --> DefaultInvitationProcessor["DefaultInvitationProcessor"] - AgentSecretProcessor --> DefaultAgentProcessor["DefaultAgentRegistrationSecretProcessor"] - end diff --git a/docs/diagrams/architecture/authorization-service-core-10.mmd b/docs/diagrams/architecture/authorization-service-core-10.mmd new file mode 100644 index 000000000..be2730515 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-10.mmd @@ -0,0 +1,8 @@ +flowchart TD + User --> Request["POST /password-reset/request"] + Request --> Service["PasswordResetService"] + Service --> Token["Generate Reset Token"] + + User --> Confirm["POST /password-reset/confirm"] + Confirm --> Validate["Validate Token"] + Validate --> Update["Update Password"] diff --git a/docs/diagrams/architecture/authorization-service-core-11.mmd b/docs/diagrams/architecture/authorization-service-core-11.mmd new file mode 100644 index 000000000..088a3aff3 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-11.mmd @@ -0,0 +1,5 @@ +flowchart TD + AuthServer["OAuth2Authorization"] --> Mapper["MongoAuthorizationMapper"] + Mapper --> Entity["MongoOAuth2Authorization"] + Entity --> Repo["MongoOAuth2AuthorizationRepository"] + Repo --> Mongo[("MongoDB")] diff --git a/docs/diagrams/architecture/authorization-service-core-12.mmd b/docs/diagrams/architecture/authorization-service-core-12.mmd new file mode 100644 index 000000000..cefd0c8f1 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-12.mmd @@ -0,0 +1,14 @@ +flowchart LR + Frontend["Frontend"] --> Gateway + Gateway -->|"Bearer Token"| Api + Gateway -->|"JWT Validation"| Authorization + Authorization -->|"Token Issuance"| Gateway + + Authorization --> Data + + subgraph Services["OpenFrame Services"] + Authorization["Authorization Service Core"] + Api["API Service Core"] + Gateway["Gateway Service Core"] + Data["Mongo Data Layer"] + end diff --git a/docs/diagrams/architecture/authorization-service-core-2.mmd b/docs/diagrams/architecture/authorization-service-core-2.mmd new file mode 100644 index 000000000..f14a05a67 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-2.mmd @@ -0,0 +1,5 @@ +flowchart TD + Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] + Filter --> Context["TenantContext (ThreadLocal)"] + Context --> AuthFlow["Authentication / Token Flow"] + AuthFlow --> Clear["TenantContext.clear()"] diff --git a/docs/diagrams/architecture/authorization-service-core-3.mmd b/docs/diagrams/architecture/authorization-service-core-3.mmd new file mode 100644 index 000000000..82be9be87 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-3.mmd @@ -0,0 +1,7 @@ +flowchart TD + Config["AuthorizationServerConfig"] --> SecurityChain["SecurityFilterChain (Order 1)"] + SecurityChain --> OAuth2["OAuth2AuthorizationServerConfigurer"] + OAuth2 --> OIDC["OIDC Enabled"] + OAuth2 --> JWT["JWT Resource Server"] + Config --> JWKSource["Tenant-aware JWKSource"] + Config --> TokenCustomizer["JWT Token Customizer"] diff --git a/docs/diagrams/architecture/authorization-service-core-4.mmd b/docs/diagrams/architecture/authorization-service-core-4.mmd new file mode 100644 index 000000000..4a77a4089 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-4.mmd @@ -0,0 +1,7 @@ +flowchart TD + TokenRequest["Access Token Request"] --> KeyService["TenantKeyService"] + KeyService --> Repo["TenantKeyRepository"] + KeyService --> Generator["RsaAuthenticationKeyPairGenerator"] + KeyService --> Encrypt["EncryptionService"] + KeyService --> JWT["RSAKey (JWK)"] + JWT --> Encoder["NimbusJwtEncoder"] diff --git a/docs/diagrams/architecture/authorization-service-core-5.mmd b/docs/diagrams/architecture/authorization-service-core-5.mmd new file mode 100644 index 000000000..779361d14 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-5.mmd @@ -0,0 +1,4 @@ +flowchart TD + Principal["Authenticated User"] --> Customizer["OAuth2TokenCustomizer"] + Customizer --> Claims["Add Claims"] + Claims --> JWT["Signed JWT"] diff --git a/docs/diagrams/architecture/authorization-service-core-6.mmd b/docs/diagrams/architecture/authorization-service-core-6.mmd new file mode 100644 index 000000000..debed29b2 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-6.mmd @@ -0,0 +1,8 @@ +flowchart TD + User["User"] -->|"/login"| FormLogin["Form Login"] + User -->|"/oauth2/authorization/{provider}"| OAuthLogin["OIDC Login"] + + OAuthLogin --> DecoderFactory["Microsoft-Aware JWT Decoder"] + OAuthLogin --> OidcUserService["Custom OIDC User Service"] + OidcUserService --> AutoProvision["Auto Provisioning"] + AutoProvision --> UserService["UserService"] diff --git a/docs/diagrams/architecture/authorization-service-core-7.mmd b/docs/diagrams/architecture/authorization-service-core-7.mmd new file mode 100644 index 000000000..77d8d4cd0 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-7.mmd @@ -0,0 +1,5 @@ +flowchart TD + OAuthFlow["OAuth2 Login Flow"] --> Repo["DynamicClientRegistrationRepository"] + Repo --> Context["TenantContext"] + Repo --> DynamicService["DynamicClientRegistrationService"] + DynamicService --> Client["ClientRegistration"] diff --git a/docs/diagrams/architecture/authorization-service-core-8.mmd b/docs/diagrams/architecture/authorization-service-core-8.mmd new file mode 100644 index 000000000..ebc11c745 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-8.mmd @@ -0,0 +1,8 @@ +flowchart TD + User --> Init["/oauth/register/sso"] + Init --> Cookie["Set SSO Registration Cookie"] + Cookie --> Provider["Redirect to Google/Microsoft"] + Provider --> Callback["OIDC Callback"] + Callback --> Handler["TenantRegSsoHandler"] + Handler --> TenantService["TenantRegistrationService"] + TenantService --> Redirect["Redirect to Tenant OAuth Flow"] diff --git a/docs/diagrams/architecture/authorization-service-core-9.mmd b/docs/diagrams/architecture/authorization-service-core-9.mmd new file mode 100644 index 000000000..7e526636a --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core-9.mmd @@ -0,0 +1,7 @@ +flowchart TD + User --> Accept["/invitations/accept/sso"] + Accept --> Cookie["Set Invite Cookie"] + Cookie --> Provider["Redirect to Provider"] + Provider --> Callback["OIDC Callback"] + Callback --> InviteHandler["InviteSsoHandler"] + InviteHandler --> InviteService["InvitationRegistrationService"] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-2.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-2.mmd deleted file mode 100644 index 61f25309b..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-2.mmd +++ /dev/null @@ -1,11 +0,0 @@ -sequenceDiagram - participant Browser - participant Controller as InvitationRegistrationController - participant SSOService as SsoInvitationService - participant IdP as External IdP - - Browser->>Controller: GET /invitations/accept/sso - Controller->>SSOService: startAccept(request) - SSOService-->>Controller: SsoAuthorizeData - Controller->>Browser: Set-Cookie + 303 Redirect - Browser->>IdP: OAuth2 Authorization Request diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-3.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-3.mmd deleted file mode 100644 index b6038e501..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Start["User Clicks Register with SSO"] --> Clear["Clear Auth State"] - Clear --> StartReg["SsoTenantRegistrationService.startRegistration()"] - StartReg --> Cookie["Set COOKIE_SSO_REG"] - Cookie --> Redirect["Redirect to IdP"] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-4.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-4.mmd deleted file mode 100644 index 5e5fca6db..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - User["User enters email"] --> Controller["TenantDiscoveryController"] - Controller --> Service["TenantDiscoveryService"] - Service --> Response["TenantDiscoveryResponse"] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-5.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-5.mmd deleted file mode 100644 index 2b315124b..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Controllers["Auth Controllers"] --> TenantLayer["Tenant Context Layer"] - Controllers --> SSOFlow["SSO Flow & Handlers"] - Controllers --> AuthServer["OAuth2 Authorization Server"] - AuthServer --> Persistence["Mongo Authorization Service"] diff --git a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos.mmd b/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos.mmd deleted file mode 100644 index 9ab0a2ede..000000000 --- a/docs/diagrams/architecture/authorization-service-core-auth-controllers-and-dtos.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Client["Browser / Client App"] --> Controllers["Auth Controllers"] - Controllers --> Services["Domain Services"] - Services --> Persistence["Authorization & Tenant Persistence"] - Services --> Security["Spring Security / OAuth2"] - Security --> Client diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-2.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-2.mmd deleted file mode 100644 index f417b8ed2..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Start["Generate Key Pair"] --> InitRandom["Initialize SecureRandom"] - InitRandom --> InitGenerator["Initialize KeyPairGenerator"] - InitGenerator --> Generate["Generate RSA KeyPair"] - Generate --> ConvertPem["Convert to PEM"] - ConvertPem --> CreateKid["Create kid UUID"] - CreateKid --> Result["Return AuthenticationKeyPair"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-3.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-3.mmd deleted file mode 100644 index e99c37703..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-3.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - Request["Request RSAKey for Tenant"] --> CheckActive["Check Active Key Count"] - CheckActive --> Found{"Active Key Exists?"} - Found -->|"Yes"| LoadDoc["Load TenantKey Document"] - Found -->|"No"| Create["Generate New Key"] - Create --> Encrypt["Encrypt Private PEM"] - Encrypt --> Save["Save TenantKey"] - Save --> LoadDoc - LoadDoc --> Parse["Parse PEM to RSA Keys"] - Parse --> BuildJwk["Build Nimbus RSAKey"] - BuildJwk --> Return["Return RSAKey"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-4.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-4.mmd deleted file mode 100644 index 485dad1ca..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - SaveCall["Save Authorization"] --> ExtractPkce["Extract PKCE Parameters"] - ExtractPkce --> MapEntity["Map to Mongo Entity"] - MapEntity --> Persist["Save to Mongo"] - Persist --> Verify["Debug Verify PKCE"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-5.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-5.mmd deleted file mode 100644 index a88a11966..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-5.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - Lookup["Find By Token"] --> TypeCheck{"Token Type?"} - TypeCheck -->|"Access"| AccessQuery["Find By Access Token"] - TypeCheck -->|"Refresh"| RefreshQuery["Find By Refresh Token"] - TypeCheck -->|"Code"| CodeQuery["Find By Authorization Code"] - TypeCheck -->|"Null"| MultiQuery["Try All Token Fields"] - AccessQuery --> MapBack["Map to Domain"] - RefreshQuery --> MapBack - CodeQuery --> MapBack - MultiQuery --> MapBack - MapBack --> ReturnAuth["Return OAuth2Authorization"] diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-6.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-6.mmd deleted file mode 100644 index 5d14b5009..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence-6.mmd +++ /dev/null @@ -1,18 +0,0 @@ -sequenceDiagram - participant User - participant AuthServer - participant KeyService - participant AuthService - participant MongoDB - - User->>AuthServer: Authorization Request with PKCE - AuthServer->>AuthService: Save OAuth2Authorization - AuthService->>MongoDB: Persist Authorization Code - AuthServer->>KeyService: Get Tenant Signing Key - KeyService->>MongoDB: Load or Create TenantKey - KeyService->>AuthServer: Return RSAKey - AuthServer->>User: Issue JWT Access Token - User->>AuthServer: Token Exchange - AuthServer->>AuthService: Find Authorization by Code - AuthService->>MongoDB: Retrieve Authorization - AuthServer->>User: Return Access and Refresh Tokens diff --git a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence.mmd b/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence.mmd deleted file mode 100644 index cdd77c54f..000000000 --- a/docs/diagrams/architecture/authorization-service-core-keys-and-authorization-persistence.mmd +++ /dev/null @@ -1,23 +0,0 @@ -flowchart TD - ClientApp["OAuth2 Client Application"] --> AuthServer["Authorization Server"] - - subgraph key_layer["Tenant Key Layer"] - TenantKeyService["TenantKeyService"] --> KeyGenerator["RsaAuthenticationKeyPairGenerator"] - TenantKeyService --> PemUtil["PemUtil"] - TenantKeyService --> TenantKeyRepo["TenantKeyRepository"] - TenantKeyService --> EncryptionService["EncryptionService"] - end - - subgraph client_layer["Client Registration Layer"] - RegisteredRepo["MongoRegisteredClientRepository"] --> MongoClientRepo["RegisteredClientMongoRepository"] - end - - subgraph auth_layer["Authorization Persistence Layer"] - AuthService["MongoAuthorizationService"] --> AuthMapper["MongoAuthorizationMapper"] - AuthService --> MongoAuthRepo["MongoOAuth2AuthorizationRepository"] - AuthMapper --> MongoEntity["MongoOAuth2Authorization"] - end - - AuthServer --> TenantKeyService - AuthServer --> RegisteredRepo - AuthServer --> AuthService diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-2.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-2.mmd deleted file mode 100644 index eabc1c302..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] - Filter --> ThreadLocal["TenantContext ThreadLocal"] - ThreadLocal --> Services["UserService and Key Services"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-3.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-3.mmd deleted file mode 100644 index 668f63b02..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Request["OAuth2 Endpoint Request"] --> Match["Authorization Endpoints Matcher"] - Match --> Authenticated["Require Authentication"] - Authenticated --> JWTValidation["JWT Resource Server Enabled"] - JWTValidation --> Response["OAuth2 Response"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-4.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-4.mmd deleted file mode 100644 index 0fc1b3d3e..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - JWTRequest["JWKS or Token Signing"] --> TenantId["TenantContext.getTenantId()"] - TenantId --> KeyService["TenantKeyService.getOrCreateActiveKey"] - KeyService --> RSAKey["Tenant RSAKey"] - RSAKey --> JWKSet["JWKSet Returned"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-5.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-5.mmd deleted file mode 100644 index 01f61db5e..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Auth["Authenticated Principal"] --> Lookup["UserService.findActiveByEmailAndTenant"] - Lookup --> Claims["Add Claims to JWT"] - Claims --> Token["Signed Access Token"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-6.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-6.mmd deleted file mode 100644 index ae1c09e93..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-6.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - User["User Clicks SSO"] --> OAuthLogin["oauth2Login()"] - OAuthLogin --> Resolver["SsoAuthorizationRequestResolver"] - Resolver --> Provider["External IdP"] - Provider --> Callback["OIDC Callback"] - Callback --> OidcService["Custom OidcUserService"] - OidcService --> AutoProvision["Auto Provision If Needed"] - AutoProvision --> Success["AuthSuccessHandler"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-7.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-7.mmd deleted file mode 100644 index cf74b4686..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-7.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - OAuthFlow["OAuth2 Login Request"] --> Repo["DynamicClientRegistrationRepository"] - Repo --> TenantCtx["TenantContext"] - TenantCtx --> DynamicSvc["DynamicClientRegistrationService"] - DynamicSvc --> ClientReg["ClientRegistration"] diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-8.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant-8.mmd deleted file mode 100644 index 9532e7d9d..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant-8.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Step1["Incoming Request"] --> Step2["TenantContextFilter Resolves Tenant"] - Step2 --> Step3{{"OAuth2 Endpoint?"}} - Step3 -->|"Yes"| Step4["AuthorizationServerConfig"] - Step3 -->|"No"| Step5["SecurityConfig"] - Step4 --> Step6["JWT Issued with Tenant Claims"] - Step5 --> Step7["Authenticated Session or SSO"] - Step6 --> EndNode["Response"] - Step7 --> EndNode diff --git a/docs/diagrams/architecture/authorization-service-core-server-and-tenant.mmd b/docs/diagrams/architecture/authorization-service-core-server-and-tenant.mmd deleted file mode 100644 index 93afcb216..000000000 --- a/docs/diagrams/architecture/authorization-service-core-server-and-tenant.mmd +++ /dev/null @@ -1,20 +0,0 @@ -flowchart TD - Client["Browser or API Client"] --> Gateway["Gateway Service"] - Gateway --> Authz["Authorization Service Core Server And Tenant"] - - subgraph authz_core["Authorization Layer"] - TenantFilter["TenantContextFilter"] - AuthServerConfig["AuthorizationServerConfig"] - SecurityConfigBean["SecurityConfig"] - DynamicClientRepo["DynamicClientRegistrationRepository"] - TenantKeySvc["TenantKeyService"] - end - - Authz --> TenantFilter - TenantFilter --> AuthServerConfig - TenantFilter --> SecurityConfigBean - AuthServerConfig --> TenantKeySvc - SecurityConfigBean --> DynamicClientRepo - - AuthServerConfig --> JWKS["Per-Tenant JWKS"] - AuthServerConfig --> JWT["Signed JWT Tokens"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-2.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-2.mmd deleted file mode 100644 index c1938fa0b..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - OidcUser["OIDC User Claims"] --> Email["email"] - OidcUser --> Preferred["preferred_username"] - OidcUser --> Upn["upn"] - OidcUser --> Unique["unique_name"] - - Email --> Resolved["Resolved Email"] - Preferred --> Resolved - Upn --> Resolved - Unique --> Resolved diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-3.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-3.mmd deleted file mode 100644 index 0151dca81..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-3.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Cookie["Invite Cookie"] --> Decode["Decode Invite Payload"] - Oidc["OIDC User"] --> Extract["Extract Names and Email"] - Decode --> BuildReq["Build InvitationRegistrationRequest"] - Extract --> BuildReq - BuildReq --> Register["InvitationRegistrationService.registerByInvitation"] - Register --> Redirect["Clear Cookie and Redirect"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-4.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-4.mmd deleted file mode 100644 index 579d6dffa..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Cookie["Registration Cookie"] --> Decode["Decode Tenant Payload"] - Oidc["OIDC User"] --> Email["Resolve Email"] - Decode --> Validate["Validate Tenant Name and Domain"] - Email --> BuildReq["Build TenantRegistrationRequest"] - Validate --> BuildReq - BuildReq --> Register["TenantRegistrationService.registerTenant"] - Register --> Redirect["Clear Cookie and Redirect"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-5.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-5.mmd deleted file mode 100644 index cc926f590..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - SSOConfigService["SSO Config Service"] --> GoogleStrategy["Google Client Registration Strategy"] - SSOConfigService --> MicrosoftStrategy["Microsoft Client Registration Strategy"] - - GoogleStrategy --> GoogleProps["Google SSO Properties"] - MicrosoftStrategy --> MicrosoftProps["Microsoft SSO Properties"] diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-6.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-6.mmd deleted file mode 100644 index 12322027f..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils-6.mmd +++ /dev/null @@ -1,14 +0,0 @@ -sequenceDiagram - participant Browser - participant AuthServer as "Authorization Server" - participant Success as "Auth Success Handler" - participant Flow as "SSO Flow Handler" - participant Service as "Registration Service" - - Browser->>AuthServer: OAuth2 Login (Google or Microsoft) - AuthServer->>Success: onAuthenticationSuccess - Success->>Success: Update lastLogin - Success->>Flow: Delegate to flow handler - Flow->>Service: Register user or tenant - Service->>Flow: Return tenant context - Flow->>Browser: Redirect to target tenant diff --git a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils.mmd b/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils.mmd deleted file mode 100644 index ce65039e2..000000000 --- a/docs/diagrams/architecture/authorization-service-core-sso-flow-and-utils.mmd +++ /dev/null @@ -1,15 +0,0 @@ -flowchart TD - Browser["Browser"] -->|"OAuth2 Login"| AuthServer["Authorization Server"] - AuthServer -->|"Authentication Success"| AuthSuccessHandler["Auth Success Handler"] - - AuthSuccessHandler -->|"Delegate"| SsoFlowHandlers["SSO Flow Handlers"] - AuthSuccessHandler -->|"Update User"| UserService["User Service"] - - SsoFlowHandlers --> InviteHandler["Invite SSO Handler"] - SsoFlowHandlers --> TenantRegHandler["Tenant Registration SSO Handler"] - - InviteHandler --> InvitationService["Invitation Registration Service"] - TenantRegHandler --> TenantRegService["Tenant Registration Service"] - - AuthSuccessHandler --> RedirectsUtil["Redirect Utilities"] - AuthSuccessHandler --> AuthStateUtils["Auth State Utilities"] diff --git a/docs/diagrams/architecture/authorization-service-core.mmd b/docs/diagrams/architecture/authorization-service-core.mmd new file mode 100644 index 000000000..7312b2799 --- /dev/null +++ b/docs/diagrams/architecture/authorization-service-core.mmd @@ -0,0 +1,7 @@ +flowchart LR + Browser["User Browser"] -->|"Login / OAuth2"| AuthServer["Authorization Service Core"] + AuthServer -->|"JWT Access Token"| Gateway["Gateway Service Core"] + Gateway -->|"Validated Request"| ApiService["API Service Core"] + + AuthServer -->|"Persist Authorizations"| Mongo[("MongoDB")] + AuthServer -->|"Load Users / Tenants"| DataLayer["Data Models & Repositories"] diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-2.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-2.mmd deleted file mode 100644 index 9ba2cc60f..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-2.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - SpringBoot[Spring Boot] --> DefaultKafkaAutoConfig[KafkaAutoConfiguration] - DefaultKafkaAutoConfig -->|Excluded| OssKafkaConfig - OssKafkaConfig --> OssTenantKafkaAutoConfiguration - OssTenantKafkaAutoConfiguration --> ProducerFactory - OssTenantKafkaAutoConfiguration --> ConsumerFactory - OssTenantKafkaAutoConfiguration --> KafkaTemplate - OssTenantKafkaAutoConfiguration --> KafkaAdmin diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-3.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-3.mmd deleted file mode 100644 index 28fb95c63..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - ConfigProps[KafkaTopicProperties] --> AdminBean[KafkaAdmin] - AdminBean --> TopicBuilder - TopicBuilder --> NewTopic - NewTopic --> KafkaCluster diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-4.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-4.mmd deleted file mode 100644 index 24014625a..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - KafkaCluster --> ListenerContainer - ListenerContainer --> ConsumerFactory - ConsumerFactory --> JsonDeserializer - ListenerContainer --> AckMode[RECORD Ack Mode] diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-5.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-5.mmd deleted file mode 100644 index 7647e6808..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-5.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - DebeziumMessage --> Payload - Payload --> Before[Before State] - Payload --> After[After State] - Payload --> Source - Payload --> Operation - Payload --> Timestamp - Source --> Database - Source --> Table - Source --> Connector diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry-6.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry-6.mmd deleted file mode 100644 index f36f73356..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Producer --> KafkaCluster - KafkaCluster -->|Failure| RecoveryHandler - RecoveryHandler --> StructuredLog diff --git a/docs/diagrams/architecture/data-kafka-configuration-and-retry.mmd b/docs/diagrams/architecture/data-kafka-configuration-and-retry.mmd deleted file mode 100644 index b54be1cd0..000000000 --- a/docs/diagrams/architecture/data-kafka-configuration-and-retry.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - AppService[Application Service] --> KafkaInfra[Data Kafka Configuration And Retry] - KafkaInfra --> KafkaCluster[Kafka Cluster] - KafkaCluster --> StreamService[Stream Service Core] - KafkaCluster --> OtherConsumers[Other Consumers] diff --git a/docs/diagrams/architecture/data-models-and-repositories-mongo-2.mmd b/docs/diagrams/architecture/data-models-and-repositories-mongo-2.mmd new file mode 100644 index 000000000..1a0d74b3a --- /dev/null +++ b/docs/diagrams/architecture/data-models-and-repositories-mongo-2.mmd @@ -0,0 +1,5 @@ +flowchart LR + Request["Incoming Request"] --> TenantContext["TenantIdProvider"] + TenantContext --> Repository["Tenant-Scoped Repository"] + Repository --> Query["Mongo Query with tenantId"] + Query --> MongoDB[("MongoDB")] diff --git a/docs/diagrams/architecture/data-models-and-repositories-mongo-3.mmd b/docs/diagrams/architecture/data-models-and-repositories-mongo-3.mmd new file mode 100644 index 000000000..2b9d5764e --- /dev/null +++ b/docs/diagrams/architecture/data-models-and-repositories-mongo-3.mmd @@ -0,0 +1,5 @@ +flowchart TD + GraphQL["GraphQL Input Filter"] --> FilterObject["QueryFilter DTO"] + FilterObject --> CustomRepo["Custom Repository"] + CustomRepo --> MongoQuery["Mongo Query + Criteria"] + MongoQuery --> MongoDB[("MongoDB")] diff --git a/docs/diagrams/architecture/data-models-and-repositories-mongo-4.mmd b/docs/diagrams/architecture/data-models-and-repositories-mongo-4.mmd new file mode 100644 index 000000000..bc69eb6fb --- /dev/null +++ b/docs/diagrams/architecture/data-models-and-repositories-mongo-4.mmd @@ -0,0 +1,6 @@ +flowchart TD + Client["Client"] --> QueryBuilder["Build Query"] + QueryBuilder --> ApplyCursor["Apply _id comparison"] + ApplyCursor --> ApplySort["Apply Sort + Limit"] + ApplySort --> Execute["mongoTemplate.find"] + Execute --> Page["Result Page"] diff --git a/docs/diagrams/architecture/data-models-and-repositories-mongo-5.mmd b/docs/diagrams/architecture/data-models-and-repositories-mongo-5.mmd new file mode 100644 index 000000000..b68e44ca4 --- /dev/null +++ b/docs/diagrams/architecture/data-models-and-repositories-mongo-5.mmd @@ -0,0 +1,4 @@ +flowchart LR + Tickets[("tickets collection")] --> Group["$group"] + Group --> Project["$project"] + Project --> Metrics["Aggregated Metrics"] diff --git a/docs/diagrams/architecture/data-models-and-repositories-mongo.mmd b/docs/diagrams/architecture/data-models-and-repositories-mongo.mmd new file mode 100644 index 000000000..1d29cfd72 --- /dev/null +++ b/docs/diagrams/architecture/data-models-and-repositories-mongo.mmd @@ -0,0 +1,9 @@ +flowchart TD + Controllers["API / GraphQL / Auth Controllers"] --> Services["Application Services"] + Services --> Repositories["Mongo Repositories"] + Repositories --> CustomImpl["Custom Repository Implementations"] + Repositories --> ReactiveImpl["Reactive Repositories"] + CustomImpl --> MongoTemplate["MongoTemplate"] + ReactiveImpl --> ReactiveMongo["ReactiveMongoRepository"] + MongoTemplate --> MongoDB[("MongoDB")] + ReactiveMongo --> MongoDB diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-2.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-2.mmd deleted file mode 100644 index aece0a12e..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - ApiKey["API Key Document"] --> ApiKeyRepo["BaseApiKeyRepository"] - ApiKeyRepo --> UserScope["User-Scoped Queries"] - ApiKeyRepo --> Expiry["Expiration Queries"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-3.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-3.mmd deleted file mode 100644 index 42408866f..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Tenant["Tenant Document"] --> TenantRepo["BaseTenantRepository"] - TenantRepo --> DomainLookup["Domain-Based Resolution"] - DomainLookup --> AuthLayer["Authorization & SSO"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-4.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-4.mmd deleted file mode 100644 index 85b9e185f..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Tool["Integrated Tool Document"] --> ToolRepo["BaseIntegratedToolRepository"] - ToolRepo --> TypeLookup["Type-Based Resolution"] - TypeLookup --> Gateway["Gateway & Integration Layer"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-5.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-5.mmd deleted file mode 100644 index de3fabd01..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - UserDoc["User Document"] --> UserRepo["BaseUserRepository"] - UserRepo --> EmailLookup["Email-Based Lookup"] - UserRepo --> StatusCheck["Status Validation"] - StatusCheck --> Security["Authentication & Access Control"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-6.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-6.mmd deleted file mode 100644 index 19d21ead5..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-6.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Request["Incoming Request"] --> DomainExtract["Extract Domain"] - DomainExtract --> TenantRepo["BaseTenantRepository"] - TenantRepo --> TenantResolved["Tenant Context Established"] - TenantResolved --> UserRepo["BaseUserRepository"] - UserRepo --> AuthResult["User Validated"] diff --git a/docs/diagrams/architecture/data-mongo-base-repositories-7.mmd b/docs/diagrams/architecture/data-mongo-base-repositories-7.mmd deleted file mode 100644 index 8d3041b86..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories-7.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - Base["Data Mongo Base Repositories"] --> SyncImpl["Mongo Sync Implementations"] - Base --> ReactiveImpl["Mongo Reactive Implementations"] - SyncImpl --> ServiceLayer["Service Layer"] - ReactiveImpl --> ServiceLayer diff --git a/docs/diagrams/architecture/data-mongo-base-repositories.mmd b/docs/diagrams/architecture/data-mongo-base-repositories.mmd deleted file mode 100644 index da1da35d2..000000000 --- a/docs/diagrams/architecture/data-mongo-base-repositories.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Domain["Mongo Domain Model"] --> BaseRepo["Data Mongo Base Repositories"] - BaseRepo --> Blocking["Blocking Repositories"] - BaseRepo --> Reactive["Reactive Repositories"] - - Blocking --> Services["Application Services"] - Reactive --> Services diff --git a/docs/diagrams/architecture/data-mongo-domain-model-2.mmd b/docs/diagrams/architecture/data-mongo-domain-model-2.mmd deleted file mode 100644 index a298cd876..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-2.mmd +++ /dev/null @@ -1,22 +0,0 @@ -classDiagram - class User { - id - email - firstName - lastName - roles - status - createdAt - updatedAt - } - - class AuthUser { - tenantId - passwordHash - loginProvider - externalUserId - lastLogin - imageUrl - } - - User <|-- AuthUser diff --git a/docs/diagrams/architecture/data-mongo-domain-model-3.mmd b/docs/diagrams/architecture/data-mongo-domain-model-3.mmd deleted file mode 100644 index 1f61e011e..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -stateDiagram-v2 - [*] --> ACTIVE - ACTIVE --> OFFLINE - OFFLINE --> ACTIVE - ACTIVE --> MAINTENANCE - MAINTENANCE --> ACTIVE diff --git a/docs/diagrams/architecture/data-mongo-domain-model-4.mmd b/docs/diagrams/architecture/data-mongo-domain-model-4.mmd deleted file mode 100644 index 9fcd8cbc2..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-4.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Ticket["Ticket"] --> Note["TicketNote"] - Ticket --> Attachment["TicketAttachment"] - Ticket --> User["User (Assigned)"] - Ticket --> Device["Device"] - Ticket --> Org["Organization"] diff --git a/docs/diagrams/architecture/data-mongo-domain-model-5.mmd b/docs/diagrams/architecture/data-mongo-domain-model-5.mmd deleted file mode 100644 index dfbf696fa..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-5.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart TD - Notification["Notification"] --> ReadState["NotificationReadState"] - ReadState --> Recipient["User or Organization"] diff --git a/docs/diagrams/architecture/data-mongo-domain-model-6.mmd b/docs/diagrams/architecture/data-mongo-domain-model-6.mmd deleted file mode 100644 index 86de08803..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - TagAssignment["TagAssignment"] --> Device - TagAssignment --> Ticket - TagAssignment --> Organization diff --git a/docs/diagrams/architecture/data-mongo-domain-model.mmd b/docs/diagrams/architecture/data-mongo-domain-model.mmd deleted file mode 100644 index 6171a289c..000000000 --- a/docs/diagrams/architecture/data-mongo-domain-model.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Gateway["Gateway Service"] --> ApiService["API Service Core"] - ApiService --> DomainModel["Data Mongo Domain Model"] - AuthService["Authorization Service"] --> DomainModel - StreamService["Stream Service"] --> DomainModel - ManagementService["Management Service"] --> DomainModel - ExternalApi["External API Service"] --> DomainModel - - DomainModel --> MongoDB[("MongoDB")] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-2.mmd b/docs/diagrams/architecture/data-mongo-query-filters-2.mmd deleted file mode 100644 index 631c49dee..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-2.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Filter["EventQueryFilter"] --> ByUser["userIds"] - Filter --> ByType["eventTypes"] - Filter --> ByStart["startDate"] - Filter --> ByEnd["endDate"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-3.mmd b/docs/diagrams/architecture/data-mongo-query-filters-3.mmd deleted file mode 100644 index af95d5ac1..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - OrgFilter["OrganizationQueryFilter"] --> SizeRange["Employee Range"] - OrgFilter --> Contract["Active Contract"] - OrgFilter --> Status["Organization Status"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-4.mmd b/docs/diagrams/architecture/data-mongo-query-filters-4.mmd deleted file mode 100644 index 2e3aebcb4..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-4.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - TicketFilter["TicketQueryFilter"] --> Statuses["TicketStatus List"] - TicketFilter --> OrgScope["Organization IDs"] - TicketFilter --> Assignees["Assignee IDs"] - TicketFilter --> Labels["Label IDs"] - TicketFilter --> Devices["Device IDs"] - TicketFilter --> Source["Creation Sources"] - TicketFilter --> TimeFrom["createdAtFrom"] - TicketFilter --> TimeTo["createdAtTo"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-5.mmd b/docs/diagrams/architecture/data-mongo-query-filters-5.mmd deleted file mode 100644 index 3e677a756..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - UserFilter["UserQueryFilter"] --> Email["emailRegex"] - UserFilter --> Name["nameRegex"] - UserFilter --> Status["UserStatus"] diff --git a/docs/diagrams/architecture/data-mongo-query-filters-6.mmd b/docs/diagrams/architecture/data-mongo-query-filters-6.mmd deleted file mode 100644 index 17d98509a..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters-6.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - ServiceLayer["Service Layer"] --> QueryFilter["Query Filter"] - QueryFilter --> CustomRepository["Custom Repository Impl"] - CustomRepository --> CriteriaBuilder["Mongo Criteria Builder"] - CriteriaBuilder --> Mongo[("MongoDB Collection")] diff --git a/docs/diagrams/architecture/data-mongo-query-filters.mmd b/docs/diagrams/architecture/data-mongo-query-filters.mmd deleted file mode 100644 index 8dc0be82a..000000000 --- a/docs/diagrams/architecture/data-mongo-query-filters.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - ApiLayer["API Layer\nREST + GraphQL"] --> FilterDTOs["Filter Input DTOs"] - FilterDTOs --> QueryFilters["Data Mongo Query Filters"] - QueryFilters --> CustomRepos["Custom Mongo Repositories"] - CustomRepos --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> CustomRepos diff --git a/docs/diagrams/architecture/data-mongo-reactive-repositories-2.mmd b/docs/diagrams/architecture/data-mongo-reactive-repositories-2.mmd deleted file mode 100644 index 14dd06f78..000000000 --- a/docs/diagrams/architecture/data-mongo-reactive-repositories-2.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Service as "Auth Service" - participant Repo as "ReactiveOAuthClientRepository" - participant DB as "MongoDB" - - Service->>Repo: findByClientId(clientId) - Repo->>DB: Reactive query - DB-->>Repo: OAuthClient document - Repo-->>Service: Mono diff --git a/docs/diagrams/architecture/data-mongo-reactive-repositories-3.mmd b/docs/diagrams/architecture/data-mongo-reactive-repositories-3.mmd deleted file mode 100644 index 6c8b5f7c2..000000000 --- a/docs/diagrams/architecture/data-mongo-reactive-repositories-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - BaseRepo["BaseUserRepository"] --> ReactiveRepo["ReactiveUserRepository"] - ReactiveMongo["ReactiveMongoRepository"] --> ReactiveRepo - ReactiveRepo --> UserDoc["User Document"] diff --git a/docs/diagrams/architecture/data-mongo-reactive-repositories.mmd b/docs/diagrams/architecture/data-mongo-reactive-repositories.mmd deleted file mode 100644 index 86ebbcd15..000000000 --- a/docs/diagrams/architecture/data-mongo-reactive-repositories.mmd +++ /dev/null @@ -1,12 +0,0 @@ -flowchart TD - App["Application Context"] --> Config["MongoReactiveConfig"] - Config -->|"@EnableReactiveMongoRepositories"| Scan["Reactive Repository Scan"] - - Scan --> OAuthRepo["ReactiveOAuthClientRepository"] - Scan --> UserRepo["ReactiveUserRepository"] - - OAuthRepo --> OAuthClient["OAuthClient Document"] - UserRepo --> UserDoc["User Document"] - - OAuthClient --> MongoDB[("MongoDB")] - UserDoc --> MongoDB diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-2.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-2.mmd deleted file mode 100644 index 82524c587..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Factory["MongoDatabaseFactory"] --> Resolver["DefaultDbRefResolver"] - Resolver --> Converter["MappingMongoConverter"] - Conversions["MongoCustomConversions"] --> Converter diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-3.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-3.mmd deleted file mode 100644 index 8150f2900..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Startup["Application Startup"] --> EnsureIndexes["Ensure Required Indexes"] - EnsureIndexes --> DropLegacy["Drop Stale Indexes"] - DropLegacy --> LogResult["Log Success Or Skip"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-4.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-4.mmd deleted file mode 100644 index d94e29831..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-4.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - ClientRequest["Client Request With Cursor"] --> ParseCursor["Parse ObjectId"] - ParseCursor --> LoadCursorDoc["Load Cursor Document"] - LoadCursorDoc --> CompareSortField["Compare Sort Field"] - CompareSortField --> AddCriteria["Add $or Criteria"] - AddCriteria --> ApplySort["Sort By Field And _id"] - ApplySort --> Limit["Apply Limit"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-5.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-5.mmd deleted file mode 100644 index c303aee69..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Match["Match itemId"] --> Group["Group By targetType"] - Group --> Count["Count"] - Count --> MapResult["Map To EnumMap"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-6.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-6.mmd deleted file mode 100644 index 093df5ca8..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - SearchCriteria["Search OR Criteria"] --> Combine - CursorCriteria["Cursor OR Criteria"] --> Combine - Combine["Combine Using $and"] --> FinalQuery["Final Query"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-7.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-7.mmd deleted file mode 100644 index a321e8c2d..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - FetchReadStates["Fetch NotificationReadState"] --> ExtractIds - ExtractIds["Extract Notification IDs"] --> FetchNotifications - FetchNotifications["Fetch Notification Documents"] --> Merge["Merge With Status"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-8.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-8.mmd deleted file mode 100644 index fb0f8a57d..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-8.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - MatchResolved["Match Resolved Tickets"] --> ProjectDiff["Compute resolvedAt - createdAt"] - ProjectDiff --> GroupAvg["Average Resolution Time"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-9.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-9.mmd deleted file mode 100644 index 306ed7466..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories-9.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - RetryStart["Retry Attempt"] --> OnError["onError()"] - OnError --> LogWarn["Log Warning"] - RetryEnd["Retry Complete"] --> Close - Close --> SuccessOrFail["Log Success Or Error"] diff --git a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories.mmd b/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories.mmd deleted file mode 100644 index b4114df5b..000000000 --- a/docs/diagrams/architecture/data-mongo-sync-config-and-custom-repositories.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - ApiLayer["API Service Layer"] --> RepositoryLayer["Custom Repositories"] - RepositoryLayer --> MongoTemplate["MongoTemplate"] - MongoTemplate --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> RepositoryLayer - QueryFilters["Query Filter DTOs"] --> RepositoryLayer - - Config["Mongo Sync Config"] --> RepositoryLayer - IndexConfig["Mongo Index Config"] --> MongoDB diff --git a/docs/diagrams/architecture/data-nats-notifications-2.mmd b/docs/diagrams/architecture/data-nats-notifications-2.mmd deleted file mode 100644 index 1f9f77542..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-2.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Start["Broadcast Request"] --> Validate["Validate NotificationCommand"] - Validate --> Persist["Persist Notification"] - Persist --> CreateReadStates["Create Read States"] - CreateReadStates --> Publish["Publish to NATS"] - Publish --> End["Complete"] - - CreateReadStates -->|"failure"| Rollback["Delete Notification"] diff --git a/docs/diagrams/architecture/data-nats-notifications-3.mmd b/docs/diagrams/architecture/data-nats-notifications-3.mmd deleted file mode 100644 index bd2f7d461..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Cmd["NotificationCommand"] --> Build["Build Notification"] - Build --> Save["Save to Repository"] - Save --> Category["Resolve Category"] - Category --> RS["Create Read States"] - RS --> Publish["Publish Per Recipient"] diff --git a/docs/diagrams/architecture/data-nats-notifications-4.mmd b/docs/diagrams/architecture/data-nats-notifications-4.mmd deleted file mode 100644 index 07c44667a..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-4.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Doc["Notification Document"] --> Msg["NotificationMessage"] - - Msg --> Id["id"] - Msg --> Severity["severity"] - Msg --> Title["title"] - Msg --> Desc["description"] - Msg --> Created["createdAt"] - Msg --> Context["context"] diff --git a/docs/diagrams/architecture/data-nats-notifications-5.mmd b/docs/diagrams/architecture/data-nats-notifications-5.mmd deleted file mode 100644 index 4dbe3a60f..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Persist["Persist Notification"] --> Durable["Durable Storage"] - Durable --> RealTime["Real-Time Publish"] - - RealTime -->|"failure"| CatchUp["Client Catch-Up via API"] diff --git a/docs/diagrams/architecture/data-nats-notifications-6.mmd b/docs/diagrams/architecture/data-nats-notifications-6.mmd deleted file mode 100644 index c34ee6d92..000000000 --- a/docs/diagrams/architecture/data-nats-notifications-6.mmd +++ /dev/null @@ -1,13 +0,0 @@ -sequenceDiagram - participant Service - participant Broadcaster - participant Mongo - participant NATS - participant Client - - Service->>Broadcaster: broadcast(command) - Broadcaster->>Mongo: save(notification) - Broadcaster->>Mongo: createReadStates() - Broadcaster->>NATS: publish(subject, message) - NATS->>Client: deliver real-time event - Client->>Mongo: query for reconciliation diff --git a/docs/diagrams/architecture/data-nats-notifications.mmd b/docs/diagrams/architecture/data-nats-notifications.mmd deleted file mode 100644 index 15eddc886..000000000 --- a/docs/diagrams/architecture/data-nats-notifications.mmd +++ /dev/null @@ -1,13 +0,0 @@ -flowchart TD - Caller["Application Service"] --> Command["NotificationCommand"] - Command --> Broadcaster["NotificationBroadcaster"] - - Broadcaster --> Repo["NotificationRepository"] - Broadcaster --> ReadState["NotificationReadStateService"] - Broadcaster --> Registry["NotificationContextDescriptorRegistry"] - - Broadcaster -->|"optional"| Publisher["NotificationNatsPublisher"] - Publisher --> Nats["NATS Server"] - - Repo --> Mongo[("MongoDB")] - ReadState --> Mongo diff --git a/docs/diagrams/architecture/data-pinot-repositories-2.mmd b/docs/diagrams/architecture/data-pinot-repositories-2.mmd deleted file mode 100644 index d1b088f2f..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-2.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - AppConfig["Spring Context"] --> PinotConfig["PinotConfig"] - PinotConfig --> BrokerConn["pinotBrokerConnection()"] - PinotConfig --> ControllerConn["pinotControllerConnection()"] - BrokerConn --> PinotCluster["Pinot Broker"] - ControllerConn --> PinotController["Pinot Controller"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-3.mmd b/docs/diagrams/architecture/data-pinot-repositories-3.mmd deleted file mode 100644 index f1a0d62fd..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-3.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - Start["Facet Request"] --> BuildQuery["PinotQueryBuilder"] - BuildQuery --> ApplyFilters["applyDeviceFilters()"] - ApplyFilters --> ExcludeField{"Exclude Facet Field?"} - ExcludeField -->|"Yes"| SkipFilter["Skip That Filter"] - ExcludeField -->|"No"| ApplyFilter["Apply Filter"] - SkipFilter --> GroupBy["GROUP BY facetField"] - ApplyFilter --> GroupBy - GroupBy --> Execute["executeKeyCountQuery()"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-4.mmd b/docs/diagrams/architecture/data-pinot-repositories-4.mmd deleted file mode 100644 index 45bd4e272..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-4.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart LR - Request["Log Query Request"] --> Builder["PinotQueryBuilder"] - Builder --> DateRange["whereDateRange()"] - DateRange --> Filters["whereIn() / whereEquals()"] - Filters --> Search["whereRelevanceLogSearch()"] - Search --> Cursor["whereCursor()"] - Cursor --> Sorting["orderBySortInput()"] - Sorting --> Limit["limit()"] - Limit --> Execute["executeLogQuery()"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-5.mmd b/docs/diagrams/architecture/data-pinot-repositories-5.mmd deleted file mode 100644 index 2868297f8..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Query["Incoming Query"] --> TenantFilter["Apply tenantId"] - TenantFilter --> PinotExec["Execute on Shared Table"] - PinotExec --> TenantScoped["Tenant Scoped Result"] diff --git a/docs/diagrams/architecture/data-pinot-repositories-6.mmd b/docs/diagrams/architecture/data-pinot-repositories-6.mmd deleted file mode 100644 index 263641427..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories-6.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart LR - Client["UI / API Request"] --> ApiService["API Service"] - ApiService --> LogRepo["PinotClientLogRepository"] - LogRepo --> PinotBroker["Pinot Broker"] - PinotBroker --> PinotServer["Pinot Servers"] - PinotServer --> PinotBroker - PinotBroker --> LogRepo - LogRepo --> ApiService - ApiService --> Client diff --git a/docs/diagrams/architecture/data-pinot-repositories.mmd b/docs/diagrams/architecture/data-pinot-repositories.mmd deleted file mode 100644 index 5bedb0d88..000000000 --- a/docs/diagrams/architecture/data-pinot-repositories.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - StreamService["Stream Service Core Kafka And Handlers"] -->|"Ingest Events"| PinotCluster["Apache Pinot Cluster"] - MongoDB["Data Mongo Domain Model"] -->|"Transactional Data"| Services["API & Management Services"] - PinotCluster -->|"Analytical Queries"| PinotRepos["Data Pinot Repositories"] - PinotRepos -->|"Facet Counts & Logs"| Services diff --git a/docs/diagrams/architecture/data-redis-cache-2.mmd b/docs/diagrams/architecture/data-redis-cache-2.mmd deleted file mode 100644 index 03a831121..000000000 --- a/docs/diagrams/architecture/data-redis-cache-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Start["Application Startup"] --> RedisEnabled{"Redis Enabled?"} - RedisEnabled -->|Yes| CreateManager["Create RedisCacheManager"] - RedisEnabled -->|No| Skip["Skip Redis Cache Config"] - CreateManager --> DefaultConfig["Apply Default TTL 6h"] - DefaultConfig --> FleetOverride["Override Fleet TTL 1h"] - FleetOverride --> Ready["CacheManager Ready"] diff --git a/docs/diagrams/architecture/data-redis-cache-3.mmd b/docs/diagrams/architecture/data-redis-cache-3.mmd deleted file mode 100644 index 4ce667c3d..000000000 --- a/docs/diagrams/architecture/data-redis-cache-3.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart LR - TenantA["Tenant A"] --> KeyBuilder - TenantB["Tenant B"] --> KeyBuilder - KeyBuilder --> RedisKeyA["tenantA:deviceCache::123"] - KeyBuilder --> RedisKeyB["tenantB:deviceCache::123"] - RedisKeyA --> RedisServer[("Redis")] - RedisKeyB --> RedisServer diff --git a/docs/diagrams/architecture/data-redis-cache.mmd b/docs/diagrams/architecture/data-redis-cache.mmd deleted file mode 100644 index e90265500..000000000 --- a/docs/diagrams/architecture/data-redis-cache.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - AppServices["Application Services"] --> CacheAbstraction["Spring Cache Abstraction"] - CacheAbstraction --> CacheManager["RedisCacheManager"] - CacheManager --> RedisConnection["RedisConnectionFactory"] - CacheManager --> KeyBuilder["OpenframeRedisKeyBuilder"] - RedisConnection --> RedisServer[("Redis Server")] - - ReactiveServices["Reactive Services"] --> ReactiveTemplate["ReactiveRedisTemplate"] - ReactiveTemplate --> ReactiveConnection["ReactiveRedisConnectionFactory"] - ReactiveConnection --> RedisServer diff --git a/docs/diagrams/architecture/external-api-service-core-2.mmd b/docs/diagrams/architecture/external-api-service-core-2.mmd deleted file mode 100644 index f12ed4f87..000000000 --- a/docs/diagrams/architecture/external-api-service-core-2.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Request["GET /api/v1/devices"] --> Criteria["DeviceFilterCriteria"] - Criteria --> Pagination["CursorPaginationCriteria"] - Pagination --> Query["DeviceService.queryDevices()"] - Query --> Result["Query Result"] - Result --> Mapper["DeviceMapper"] - Mapper --> Response["DevicesResponse"] diff --git a/docs/diagrams/architecture/external-api-service-core-3.mmd b/docs/diagrams/architecture/external-api-service-core-3.mmd deleted file mode 100644 index ab70d3f39..000000000 --- a/docs/diagrams/architecture/external-api-service-core-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - EventRequest["GET /api/v1/events"] --> Filter["EventFilterCriteria"] - Filter --> Service["EventService.queryEvents()"] - Service --> Mapper["EventMapper"] - Mapper --> EventsResponse["EventsResponse"] diff --git a/docs/diagrams/architecture/external-api-service-core-4.mmd b/docs/diagrams/architecture/external-api-service-core-4.mmd deleted file mode 100644 index 67917cb93..000000000 --- a/docs/diagrams/architecture/external-api-service-core-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - OrgRequest["Organization Request"] --> QueryService["OrganizationQueryService"] - OrgRequest --> CommandService["OrganizationCommandService"] - CommandService --> Validation["Archive Rules"] diff --git a/docs/diagrams/architecture/external-api-service-core-5.mmd b/docs/diagrams/architecture/external-api-service-core-5.mmd deleted file mode 100644 index 821c7120d..000000000 --- a/docs/diagrams/architecture/external-api-service-core-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["External Client"] --> ProxyController["IntegrationController"] - ProxyController --> RestProxyService["RestProxyService"] - RestProxyService --> ToolRepo["IntegratedToolRepository"] - RestProxyService --> Resolver["ProxyUrlResolver"] - Resolver --> TargetTool["Integrated Tool API"] diff --git a/docs/diagrams/architecture/external-api-service-core.mmd b/docs/diagrams/architecture/external-api-service-core.mmd deleted file mode 100644 index 1e702571b..000000000 --- a/docs/diagrams/architecture/external-api-service-core.mmd +++ /dev/null @@ -1,16 +0,0 @@ -flowchart LR - Client["External Client"] -->|"X-API-Key"| ExternalAPI["External Api Service Core"] - - ExternalAPI --> DeviceService["Device Service"] - ExternalAPI --> EventService["Event Service"] - ExternalAPI --> LogService["Log Service"] - ExternalAPI --> OrganizationService["Organization Services"] - ExternalAPI --> ToolService["Tool Service"] - - ExternalAPI --> ProxyService["Rest Proxy Service"] - ProxyService --> IntegratedTool["Integrated Tool"] - - DeviceService --> MongoDB[("MongoDB")] - EventService --> MongoDB - LogService --> Pinot[("Apache Pinot")] - ToolService --> MongoDB diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-10.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-10.mmd new file mode 100644 index 000000000..48421720f --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-10.mmd @@ -0,0 +1,9 @@ +flowchart LR + Backend --> NATS["NATS / JetStream"] + Backend --> SSE["SSE"] + + NATS --> Subscription["useNatsDialogSubscription"] + SSE --> Adapter["useSseChatAdapter"] + + Subscription --> Processor + Adapter --> Processor diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-2.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-2.mmd new file mode 100644 index 000000000..7233b0d37 --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-2.mmd @@ -0,0 +1,11 @@ +flowchart TD + EmbeddableChat --> UnifiedChat["useUnifiedChat"] + + UnifiedChat --> GuideMode["Guide Mode (SSE)"] + UnifiedChat --> MingoMode["Mingo Mode (NATS)"] + + GuideMode --> SSEAdapter["useSseChatAdapter"] + MingoMode --> NatsAdapter["useNatsChatAdapter"] + + SSEAdapter --> SSEBackend["API Service (SSE)"] + NatsAdapter --> NatsBackend["Stream Processing (NATS)"] diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-3.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-3.mmd new file mode 100644 index 000000000..600120e7a --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-3.mmd @@ -0,0 +1,5 @@ +flowchart LR + Chunk["ChunkData"] --> Processor["useRealtimeChunkProcessor"] + Processor --> Segments["MessageSegment[]"] + Segments --> Message["Message"] + Message --> UI["ChatMessageRow"] diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-4.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-4.mmd new file mode 100644 index 000000000..d8cbbad61 --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-4.mmd @@ -0,0 +1,6 @@ +flowchart TD + DialogList --> History["MingoChatHistory"] + History --> Row["History Row"] + Row --> Rename["Rename Modal"] + Row --> Archive["Archive Modal"] + Archive --> ArchivePage["ChatArchivePage"] diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-5.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-5.mmd new file mode 100644 index 000000000..ccf35792c --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-5.mmd @@ -0,0 +1,4 @@ +flowchart TD + Identity["useChatIdentity"] --> Check{Auth Tier?} + Check -->|anon| Empty["Sign-in EmptyState"] + Check -->|authenticated| TicketCenterAuthed diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-6.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-6.mmd new file mode 100644 index 000000000..60a1eac56 --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-6.mmd @@ -0,0 +1,6 @@ +flowchart LR + Ticket["TicketCard"] --> DndContext + DndContext --> DragStart + DragStart --> DragOver + DragOver --> DragEnd + DragEnd --> OnChange["BoardChange"] diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-7.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-7.mmd new file mode 100644 index 000000000..c24face0b --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-7.mmd @@ -0,0 +1,5 @@ +flowchart TD + NotificationContext --> Drawer + Drawer --> UnreadList + UnreadList --> Tile["NotificationTile"] + Tile --> MarkRead diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-8.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-8.mmd new file mode 100644 index 000000000..c46153050 --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-8.mmd @@ -0,0 +1,5 @@ +flowchart LR + Table --> Header + Table --> Row + Header --> SortChange + Row --> RowClick diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat-9.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat-9.mmd new file mode 100644 index 000000000..42b843ba9 --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat-9.mmd @@ -0,0 +1,5 @@ +flowchart TD + Viewport --> Breakpoint{md / lg?} + Breakpoint -->|Desktop| PersistentSidebar + Breakpoint -->|Tablet| OverlaySidebar + OverlaySidebar --> Backdrop diff --git a/docs/diagrams/architecture/frontend-core-ui-and-chat.mmd b/docs/diagrams/architecture/frontend-core-ui-and-chat.mmd new file mode 100644 index 000000000..8f742ff26 --- /dev/null +++ b/docs/diagrams/architecture/frontend-core-ui-and-chat.mmd @@ -0,0 +1,24 @@ +flowchart LR + User["User"] --> UI["Frontend Core Ui And Chat"] + + subgraph ChatLayer["Chat System"] + EmbeddableChat["EmbeddableChat"] + Composer["ChatComposer"] + MessageRow["ChatMessageRow"] + DialogHistory["MingoChatHistory"] + end + + subgraph FeatureLayer["Feature Surfaces"] + TicketCenter["TicketCenter"] + Board["Board"] + Notifications["NotificationDrawer"] + DataTable["DataTable Components"] + end + + UI --> ChatLayer + UI --> FeatureLayer + + ChatLayer --> Runtime["Chat Runtime Context"] + ChatLayer --> Transport["SSE / NATS / WebSocket"] + + Transport --> Backend["API Service + Stream Processing"] diff --git a/docs/diagrams/architecture/gateway-service-core-2.mmd b/docs/diagrams/architecture/gateway-service-core-2.mmd new file mode 100644 index 000000000..3df57ecea --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-2.mmd @@ -0,0 +1,12 @@ +flowchart TD + Netty["Netty & WebFlux Runtime"] --> Filters["Security & Global Filters"] + Filters --> Controllers["REST Controllers"] + Filters --> WsRoutes["WebSocket Routes"] + + Controllers --> RestProxy["REST Proxy Layer"] + WsRoutes --> WsProxy["WebSocket Proxy Layer"] + + RestProxy --> UpstreamResolvers["Tool Upstream Resolvers"] + WsProxy --> UpstreamResolvers + + UpstreamResolvers --> Downstream["Integrated Tools / Services"] diff --git a/docs/diagrams/architecture/gateway-service-core-3.mmd b/docs/diagrams/architecture/gateway-service-core-3.mmd new file mode 100644 index 000000000..39cba02f7 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-3.mmd @@ -0,0 +1,6 @@ +flowchart TD + Request["Incoming JWT"] --> IssuerResolver["Issuer Resolver"] + IssuerResolver --> Cache["Caffeine Cache"] + Cache --> JwtDecoder["JWT Decoder"] + JwtDecoder --> Validator["Issuer + Default Validators"] + Validator --> Authenticated["Authenticated Principal"] diff --git a/docs/diagrams/architecture/gateway-service-core-4.mmd b/docs/diagrams/architecture/gateway-service-core-4.mmd new file mode 100644 index 000000000..478726b86 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-4.mmd @@ -0,0 +1,8 @@ +flowchart TD + Req["Request /external-api/**"] --> CheckHeader["X-API-Key Present?"] + CheckHeader -->|"No"| Reject["401 Unauthorized"] + CheckHeader -->|"Yes"| Validate["Validate API Key"] + Validate -->|"Invalid"| Reject + Validate -->|"Valid"| RateCheck["Rate Limit Check"] + RateCheck -->|"Exceeded"| TooMany["429 Too Many Requests"] + RateCheck -->|"Allowed"| Forward["Forward Request"] diff --git a/docs/diagrams/architecture/gateway-service-core-5.mmd b/docs/diagrams/architecture/gateway-service-core-5.mmd new file mode 100644 index 000000000..2570ddb42 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-5.mmd @@ -0,0 +1,6 @@ +flowchart LR + ClientWS["WebSocket Client"] --> GatewayWS["WebSocket Gateway"] + GatewayWS --> ToolResolver["Tool Upstream Resolver"] + ToolResolver --> Mesh["MeshCentral"] + ToolResolver --> Tactical["Tactical RMM"] + GatewayWS --> Nats["NATS"] diff --git a/docs/diagrams/architecture/gateway-service-core-6.mmd b/docs/diagrams/architecture/gateway-service-core-6.mmd new file mode 100644 index 000000000..f429dbe05 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core-6.mmd @@ -0,0 +1,5 @@ +flowchart TD + Request["Tool Request"] --> ResolverRegistry["Resolver Registry"] + ResolverRegistry -->|"meshcentral-server"| MeshResolver["MeshCentral Resolver"] + ResolverRegistry -->|"tactical-rmm"| TacticalResolver["Tactical RMM Resolver"] + ResolverRegistry -->|"other"| DefaultResolver["Default Resolver"] diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-2.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-2.mmd deleted file mode 100644 index 1fb4bf101..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Request["Incoming Request"] --> Origin["OriginSanitizerFilter"] - Origin --> AuthHeader["AddAuthorizationHeaderFilter"] - AuthHeader --> JwtValidation["JWT Authentication"] - JwtValidation --> RoleCheck["Role Authorization Rules"] - RoleCheck --> ApiKeyCheck{"External API Path?"} - ApiKeyCheck -->|Yes| ApiKeyFilter["ApiKeyAuthenticationFilter"] - ApiKeyCheck -->|No| Continue["Continue Routing"] - ApiKeyFilter --> Continue - Continue --> Controller["Controller / Route"] diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-3.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-3.mmd deleted file mode 100644 index 8a4cf108f..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-3.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - Req["/external-api/**"] --> CheckKey["Check X-API-Key Header"] - CheckKey --> Valid{"Valid Key?"} - Valid -->|No| Reject401["Return 401"] - Valid -->|Yes| RateCheck["RateLimitService.isAllowed"] - RateCheck --> Allowed{"Allowed?"} - Allowed -->|No| Reject429["Return 429 + Retry-After"] - Allowed -->|Yes| AddHeaders["Add RateLimit Headers"] - AddHeaders --> AddContext["Inject X-User-Id + X-Api-Key-Id"] - AddContext --> Forward["Forward to External API"] diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-4.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-4.mmd deleted file mode 100644 index dda93d4dd..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart LR - Request["Tool Request"] --> Registry["ToolUpstreamResolverRegistry"] - Registry -->|Specific Tool| Mesh["MeshCentralUpstreamResolver"] - Registry -->|Specific Tool| Tactical["TacticalRmmUpstreamResolver"] - Registry -->|Fallback| Default["DefaultToolUpstreamResolver"] - Mesh --> Proxy - Tactical --> Proxy - Default --> Proxy diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing-5.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing-5.mmd deleted file mode 100644 index 44f992bcd..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing-5.mmd +++ /dev/null @@ -1,12 +0,0 @@ -flowchart TD - Client --> Gateway - Gateway --> OriginFilter - OriginFilter --> AuthHeader - AuthHeader --> JwtResolver - JwtResolver --> RoleAuth - RoleAuth --> RouteDecision - RouteDecision -->|REST| IntegrationController - RouteDecision -->|WebSocket| WebSocketGatewayConfig - IntegrationController --> UpstreamResolver - WebSocketGatewayConfig --> UpstreamResolver - UpstreamResolver --> IntegratedTool diff --git a/docs/diagrams/architecture/gateway-service-core-security-and-routing.mmd b/docs/diagrams/architecture/gateway-service-core-security-and-routing.mmd deleted file mode 100644 index 7b597a83e..000000000 --- a/docs/diagrams/architecture/gateway-service-core-security-and-routing.mmd +++ /dev/null @@ -1,20 +0,0 @@ -flowchart TD - Client["Client / Agent / UI"] --> Gateway["Gateway Service Core Security And Routing"] - - subgraph security_layer["Security Layer"] - AuthHeader["AddAuthorizationHeaderFilter"] - JwtConfig["JwtAuthConfig"] - SecurityConfig["GatewaySecurityConfig"] - ApiKeyFilter["ApiKeyAuthenticationFilter"] - OriginFilter["OriginSanitizerFilter"] - end - - subgraph routing_layer["Routing & Proxy Layer"] - IntegrationCtrl["IntegrationController"] - WsConfig["WebSocketGatewayConfig"] - UpstreamResolvers["ToolUpstreamResolvers"] - end - - Gateway --> security_layer - security_layer --> routing_layer - routing_layer --> Upstream["Integrated Tools / NATS / Services"] diff --git a/docs/diagrams/architecture/gateway-service-core.mmd b/docs/diagrams/architecture/gateway-service-core.mmd new file mode 100644 index 000000000..a54c6bc60 --- /dev/null +++ b/docs/diagrams/architecture/gateway-service-core.mmd @@ -0,0 +1,11 @@ +flowchart LR + Browser["Frontend UI"] --> Gateway["Gateway Service Core"] + Agent["OpenFrame Agent"] --> Gateway + ExternalAPI["External API Client"] --> Gateway + + Gateway --> ApiService["API Service Core"] + Gateway --> AuthService["Authorization Service Core"] + Gateway --> Management["Management Service Core"] + Gateway --> ToolMesh["MeshCentral"] + Gateway --> ToolTactical["Tactical RMM"] + Gateway --> Nats["NATS WebSocket"] diff --git a/docs/diagrams/architecture/management-service-core-2.mmd b/docs/diagrams/architecture/management-service-core-2.mmd new file mode 100644 index 000000000..4922e6f3a --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-2.mmd @@ -0,0 +1,3 @@ +flowchart LR + Scheduler["Scheduled Task"] --> ShedLock["ShedLock"] + ShedLock --> Redis["Redis Lock Provider"] diff --git a/docs/diagrams/architecture/management-service-core-3.mmd b/docs/diagrams/architecture/management-service-core-3.mmd new file mode 100644 index 000000000..ad36b5703 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-3.mmd @@ -0,0 +1,5 @@ +flowchart TD + Controller["DevicePinotResyncController"] --> Repo["MachineRepository"] + Controller --> EventService["MachineTagEventService"] + Repo --> Mongo["MongoDB"] + EventService --> Stream["Event Processing"] diff --git a/docs/diagrams/architecture/management-service-core-4.mmd b/docs/diagrams/architecture/management-service-core-4.mmd new file mode 100644 index 000000000..946391f1e --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-4.mmd @@ -0,0 +1,5 @@ +flowchart TD + Request["Save Tool Request"] --> Service["IntegratedToolService"] + Service --> Mongo["MongoDB"] + Service --> Debezium["DebeziumService"] + Service --> Hooks["Post Save Hooks"] diff --git a/docs/diagrams/architecture/management-service-core-5.mmd b/docs/diagrams/architecture/management-service-core-5.mmd new file mode 100644 index 000000000..dd403dd31 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-5.mmd @@ -0,0 +1,6 @@ +flowchart TD + Startup["Application Startup"] --> SecretInit["Agent Registration Secret"] + Startup --> ToolAgentInit["Integrated Tool Agent Config"] + Startup --> NatsInit["NATS Stream Config"] + Startup --> ClientInit["Client Configuration"] + Startup --> TacticalInit["Tactical RMM Scripts"] diff --git a/docs/diagrams/architecture/management-service-core-6.mmd b/docs/diagrams/architecture/management-service-core-6.mmd new file mode 100644 index 000000000..176a7cdfd --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-6.mmd @@ -0,0 +1,6 @@ +flowchart LR + Init["NATS Stream Initializer"] --> Stream1["TOOL_INSTALLATION"] + Init --> Stream2["CLIENT_UPDATE"] + Init --> Stream3["TOOL_UPDATE"] + Init --> Stream4["TOOL_CONNECTIONS"] + Init --> Stream5["INSTALLED_AGENTS"] diff --git a/docs/diagrams/architecture/management-service-core-7.mmd b/docs/diagrams/architecture/management-service-core-7.mmd new file mode 100644 index 000000000..1517f80b8 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-7.mmd @@ -0,0 +1,4 @@ +flowchart TD + Migration["Mongock Migration"] --> BackfillVersion["Backfill Document Version"] + Migration --> BackfillOrder["Backfill Ticket Order"] + Migration --> StatusMigration["Migrate Ticket Status Model"] diff --git a/docs/diagrams/architecture/management-service-core-8.mmd b/docs/diagrams/architecture/management-service-core-8.mmd new file mode 100644 index 000000000..2bf43f931 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core-8.mmd @@ -0,0 +1,5 @@ +flowchart TD + Scheduler["Schedulers"] --> AgentVersion["Agent Version Fallback"] + Scheduler --> ApiKeySync["API Key Stats Sync"] + Scheduler --> Heartbeat["Device Offline Detection"] + Scheduler --> FleetMdm["Fleet MDM Setup"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-2.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-2.mmd deleted file mode 100644 index 1c674ad9f..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Scheduler["Scheduled Job"] --> LockProvider["RedisLockProvider"] - LockProvider --> Redis[("Redis")] - Redis --> LockKey["Tenant Scoped Lock Key"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-3.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-3.mmd deleted file mode 100644 index 2762a6a25..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-3.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Start["Application Startup"] --> SecretInit["AgentRegistrationSecretInitializer"] - SecretInit --> Service["AgentRegistrationSecretManagementService"] - Service --> Create["createInitialSecret()"] - Create --> Processor["DefaultAgentRegistrationSecretManagementProcessor"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-4.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-4.mmd deleted file mode 100644 index b31c2e02b..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Init["NatsStreamConfigurationInitializer"] --> DefaultStreams["Default Stream Configurations"] - Init --> Additional["AdditionalStreamConfigurationProvider"] - DefaultStreams --> NatsService["NatsStreamManagementService"] - Additional --> NatsService diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-5.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-5.mmd deleted file mode 100644 index beb2eac54..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-5.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Startup["Application Startup"] --> LoadTool["Load Tactical RMM Tool"] - LoadTool --> FetchScripts["Fetch Existing Scripts"] - FetchScripts --> Compare["Compare by Name"] - Compare -->|"Missing"| Create["Create Script"] - Compare -->|"Exists"| Update["Update Script"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-6.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-6.mmd deleted file mode 100644 index 6b701a8a0..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-6.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Tick["Scheduled Tick"] --> ClientCheck["Check Client PublishState"] - ClientCheck --> RetryClient["Publish If Needed"] - Tick --> AgentCheck["Check Tool Agents"] - AgentCheck --> RetryAgent["Publish If Needed"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-7.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-7.mmd deleted file mode 100644 index d13f80b57..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-7.mmd +++ /dev/null @@ -1,3 +0,0 @@ -flowchart LR - Scheduler["Heartbeat Scheduler"] --> Service["DeviceHeartbeatOfflineDetectionService"] - Service --> Mongo[("MongoDB Device Records")] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-8.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-8.mmd deleted file mode 100644 index 434e76c29..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-8.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Client["API Client"] --> Controller["IntegratedToolController"] - Controller --> Save["IntegratedToolService.saveTool()"] - Save --> TenantCheck["TenantIdProvider"] - TenantCheck -->|"Registered"| Debezium["DebeziumService"] - Save --> Hooks["IntegratedToolPostSaveHook"] diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-9.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-9.mmd deleted file mode 100644 index f545d4e47..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers-9.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Boot["Application Boot"] --> Initializers - Initializers --> Messaging["NATS Streams Ready"] - Messaging --> Schedulers - Schedulers --> External["External Systems"] - Controllers --> External diff --git a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers.mmd b/docs/diagrams/architecture/management-service-core-initializers-and-schedulers.mmd deleted file mode 100644 index 7da107187..000000000 --- a/docs/diagrams/architecture/management-service-core-initializers-and-schedulers.mmd +++ /dev/null @@ -1,33 +0,0 @@ -flowchart TD - subgraph ConfigLayer["Configuration Layer"] - MgmtConfig["ManagementConfiguration"] - RetryConfig["RetryConfiguration"] - ShedLock["ShedLockConfig"] - end - - subgraph Initializers["Startup Initializers"] - SecretInit["AgentRegistrationSecretInitializer"] - AgentInit["IntegratedToolAgentInitializer"] - NatsInit["NatsStreamConfigurationInitializer"] - ClientInit["OpenFrameClientConfigurationInitializer"] - TacticalInit["TacticalRmmScriptsInitializer"] - end - - subgraph Schedulers["Background Schedulers"] - VersionFallback["AgentVersionUpdatePublishFallbackScheduler"] - ApiKeySync["ApiKeyStatsSyncScheduler"] - Heartbeat["DeviceHeartbeatOfflineDetectionScheduler"] - FleetMdm["FleetMdmSetupScheduler"] - end - - subgraph Controllers["Operational Controllers"] - ToolCtrl["IntegratedToolController"] - PinotCtrl["DevicePinotResyncController"] - ReleaseCtrl["ReleaseVersionController"] - end - - MgmtConfig --> Initializers - ShedLock --> Schedulers - RetryConfig --> Schedulers - Controllers --> Schedulers - Initializers --> Schedulers diff --git a/docs/diagrams/architecture/management-service-core.mmd b/docs/diagrams/architecture/management-service-core.mmd new file mode 100644 index 000000000..941b3f5f5 --- /dev/null +++ b/docs/diagrams/architecture/management-service-core.mmd @@ -0,0 +1,11 @@ +flowchart TD + Gateway["Gateway Service Core"] --> API["API Service Core HTTP and GraphQL"] + API --> Data["Data Models and Repositories Mongo"] + API --> Stream["Stream Processing Kafka"] + + Management["Management Service Core"] --> Data + Management --> Redis["Redis"] + Management --> NATS["NATS Streams"] + Management --> KafkaConnect["Kafka Connect / Debezium"] + + Stream --> Data diff --git a/docs/diagrams/architecture/processors-2.mmd b/docs/diagrams/architecture/processors-2.mmd deleted file mode 100644 index f5990ef98..000000000 --- a/docs/diagrams/architecture/processors-2.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - Generator["Secret Generation Logic"] --> Persist["Persist AgentRegistrationSecret"] - Persist --> ProcessorCall["postProcessSecretGenerated()"] - ProcessorCall --> DefaultProcessor["DefaultAgentRegistrationSecretProcessor"] diff --git a/docs/diagrams/architecture/processors-3.mmd b/docs/diagrams/architecture/processors-3.mmd deleted file mode 100644 index 6a9ee343e..000000000 --- a/docs/diagrams/architecture/processors-3.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - InvitationService["Invitation Service"] --> SaveInvitation["Persist Invitation"] - SaveInvitation --> ProcessorHook["postProcessInvitationCreated()"] - ProcessorHook --> DefaultInvitationProcessor["DefaultInvitationProcessor"] diff --git a/docs/diagrams/architecture/processors-4.mmd b/docs/diagrams/architecture/processors-4.mmd deleted file mode 100644 index 695fd9f01..000000000 --- a/docs/diagrams/architecture/processors-4.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - SSOService["SSOConfigService"] --> SaveConfig["Persist SSOConfig"] - SaveConfig --> ProcessorHook["postProcessConfigSaved()"] - ProcessorHook --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] diff --git a/docs/diagrams/architecture/processors-5.mmd b/docs/diagrams/architecture/processors-5.mmd deleted file mode 100644 index e24006f05..000000000 --- a/docs/diagrams/architecture/processors-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - UserController["UserController"] --> UserService["UserService"] - UserService --> PersistUser["Persist User"] - PersistUser --> ProcessorHook["postProcessUserUpdated()"] - ProcessorHook --> DefaultUserProcessor["DefaultUserProcessor"] diff --git a/docs/diagrams/architecture/processors-6.mmd b/docs/diagrams/architecture/processors-6.mmd deleted file mode 100644 index e738e3e2e..000000000 --- a/docs/diagrams/architecture/processors-6.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart TD - SpringContext["Spring Application Context"] --> CheckCustom["Custom Bean Present?"] - CheckCustom -->|"Yes"| UseCustom["Use Custom Implementation"] - CheckCustom -->|"No"| UseDefault["Use Default Implementation"] diff --git a/docs/diagrams/architecture/processors-7.mmd b/docs/diagrams/architecture/processors-7.mmd deleted file mode 100644 index 13c39d0d0..000000000 --- a/docs/diagrams/architecture/processors-7.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - Services["Services Module"] -->|"Business Logic"| Domain["Domain Model"] - Services -->|"Calls"| Processors["Processors Module"] - Processors -->|"Side Effects"| ExternalSystems["External Systems"] diff --git a/docs/diagrams/architecture/processors-8.mmd b/docs/diagrams/architecture/processors-8.mmd deleted file mode 100644 index 6394e195e..000000000 --- a/docs/diagrams/architecture/processors-8.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - API["API Layer"] --> ServiceLayer["Service Layer"] - ServiceLayer --> RepositoryLayer["Repository Layer"] - ServiceLayer --> ProcessorLayer["Processors"] - ProcessorLayer --> Integration["Integrations / Events / Audit"] diff --git a/docs/diagrams/architecture/processors.mmd b/docs/diagrams/architecture/processors.mmd deleted file mode 100644 index 8b6c815e4..000000000 --- a/docs/diagrams/architecture/processors.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - Controller["REST or GraphQL Controller"] --> Service["Core Service"] - Service --> DomainModel["Domain Model"] - Service --> Processor["Processor Interface"] - Processor --> DefaultImpl["Default Implementation"] - Processor --> CustomImpl["Custom Implementation (Optional)"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-2.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-2.mmd deleted file mode 100644 index 7ee915858..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-2.mmd +++ /dev/null @@ -1,11 +0,0 @@ -flowchart TD - JwtConfig["JwtConfig"] -->|"loadPublicKey()"| PublicKey["RSAPublicKey"] - JwtConfig -->|"loadPrivateKey()"| PrivateKey["RSAPrivateKey"] - - PublicKey --> RsaKey["RSAKey Builder"] - PrivateKey --> RsaKey - - RsaKey --> JwkSet["JWKSet"] - JwkSet --> JwtEncoder["NimbusJwtEncoder"] - - PublicKey --> JwtDecoder["NimbusJwtDecoder"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-3.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-3.mmd deleted file mode 100644 index 682c32589..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - PemString["PEM Private Key"] --> StripHeaders["Remove BEGIN/END markers"] - StripHeaders --> Base64Decode["Base64 Decode"] - Base64Decode --> KeySpec["PKCS8EncodedKeySpec"] - KeySpec --> KeyFactory["KeyFactory RSA"] - KeyFactory --> RsaPrivate["RSAPrivateKey"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-4.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-4.mmd deleted file mode 100644 index 57de5ec02..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-4.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart TD - Verifier["Code Verifier (random 32 bytes)"] --> Hash["SHA-256"] - Hash --> Challenge["Base64URL Encode"] - - Challenge -->|"Sent to Authorization Server"| AuthServer["Authorization Server"] - Verifier -->|"Stored by BFF"| BffStore["State JWT / Cookie"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-5.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-5.mmd deleted file mode 100644 index 6508a734f..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-5.mmd +++ /dev/null @@ -1,12 +0,0 @@ -sequenceDiagram - participant Browser - participant BFF as OAuth BFF Controller - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/login - BFF->>Auth: Redirect with state and PKCE challenge - Auth->>Browser: Login UI - Auth->>BFF: Callback with code and state - BFF->>Auth: Exchange code for tokens - Auth->>BFF: Access and Refresh tokens - BFF->>Browser: Set HttpOnly cookies and redirect diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-6.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-6.mmd deleted file mode 100644 index 32886c510..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-6.mmd +++ /dev/null @@ -1,9 +0,0 @@ -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: POST /oauth/refresh - BFF->>Auth: Refresh token request - Auth->>BFF: New tokens - BFF->>Browser: Set new cookies diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-7.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-7.mmd deleted file mode 100644 index 76d03d47e..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-7.mmd +++ /dev/null @@ -1,8 +0,0 @@ -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/logout - BFF->>Auth: Revoke refresh token - BFF->>Browser: Clear auth cookies diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff-8.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff-8.mmd deleted file mode 100644 index add9b4272..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff-8.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Requested["Requested redirectTo"] -->|"Has value"| UseRequested["Use requested value"] - Requested -->|"Empty"| RefererCheck["Check Referer header"] - RefererCheck -->|"Present"| UseReferer["Use Referer"] - RefererCheck -->|"Missing"| DefaultRoot["Use /"] diff --git a/docs/diagrams/architecture/security-core-and-oauth-bff.mmd b/docs/diagrams/architecture/security-core-and-oauth-bff.mmd deleted file mode 100644 index 6b9d48043..000000000 --- a/docs/diagrams/architecture/security-core-and-oauth-bff.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart LR - Browser["Browser Client"] -->|"GET /oauth/login"| Bff["OAuth BFF Controller"] - Bff -->|"Redirect to authorize"| AuthServer["Authorization Server"] - AuthServer -->|"Callback with code"| Bff - Bff -->|"Exchange code for tokens"| AuthServer - Bff -->|"Set HttpOnly Cookies"| Browser - - Browser -->|"API calls with cookies"| Gateway["Gateway Service"] - Gateway -->|"Validate JWT"| JwtDecoder["JwtDecoder Bean"] diff --git a/docs/diagrams/architecture/services-2.mmd b/docs/diagrams/architecture/services-2.mmd deleted file mode 100644 index cf3d02dd0..000000000 --- a/docs/diagrams/architecture/services-2.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - Request["User Update Request"] --> Load["Load User from Repository"] - Load --> Check{"User Exists?"} - Check -->|"No"| Error["Throw IllegalArgumentException"] - Check -->|"Yes"| Modify["Apply Field Updates"] - Modify --> Save["Save via UserRepository"] - Save --> PostProcess["UserProcessor.postProcessUserUpdated()"] - PostProcess --> Response["Return UserResponse"] diff --git a/docs/diagrams/architecture/services-3.mmd b/docs/diagrams/architecture/services-3.mmd deleted file mode 100644 index 0dcbea462..000000000 --- a/docs/diagrams/architecture/services-3.mmd +++ /dev/null @@ -1,9 +0,0 @@ -flowchart TD - DeleteRequest["Soft Delete Request"] --> LoadUser["Load User"] - LoadUser --> SelfCheck{"Requester == Target?"} - SelfCheck -->|"Yes"| SelfError["Throw Self Delete Exception"] - SelfCheck -->|"No"| OwnerCheck{"Has OWNER Role?"} - OwnerCheck -->|"Yes"| OwnerError["Throw OperationNotAllowedException"] - OwnerCheck -->|"No"| MarkDeleted["Set Status = DELETED"] - MarkDeleted --> SaveDeleted["Save User"] - SaveDeleted --> PostDelete["UserProcessor.postProcessUserDeleted()"] diff --git a/docs/diagrams/architecture/services-4.mmd b/docs/diagrams/architecture/services-4.mmd deleted file mode 100644 index 38db55be0..000000000 --- a/docs/diagrams/architecture/services-4.mmd +++ /dev/null @@ -1,8 +0,0 @@ -flowchart TD - AdminRequest["Upsert SSOConfigRequest"] --> Validate["validateAutoProvision()"] - Validate --> Normalize["Normalize Domains"] - Normalize --> DomainValidation["DomainValidationService.validate*"] - DomainValidation --> Encrypt["Encrypt Client Secret"] - Encrypt --> SaveConfig["Save via SSOConfigRepository"] - SaveConfig --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] - PostProcess --> ReturnResponse["Return SSOConfigResponse"] diff --git a/docs/diagrams/architecture/services-5.mmd b/docs/diagrams/architecture/services-5.mmd deleted file mode 100644 index d5f4c90fe..000000000 --- a/docs/diagrams/architecture/services-5.mmd +++ /dev/null @@ -1,4 +0,0 @@ -flowchart LR - DomainService["DomainValidationService"] --> Interface["DomainExistenceValidator"] - Interface --> DefaultImpl["DefaultDomainExistenceValidator"] - Interface --> CustomImpl["SaaS Override Implementation"] diff --git a/docs/diagrams/architecture/services-6.mmd b/docs/diagrams/architecture/services-6.mmd deleted file mode 100644 index 5627a721f..000000000 --- a/docs/diagrams/architecture/services-6.mmd +++ /dev/null @@ -1,11 +0,0 @@ -sequenceDiagram - participant Controller - participant Service - participant Repository - participant Processor - - Controller->>Service: Update Request - Service->>Repository: Save Entity - Repository-->>Service: Saved Entity - Service->>Processor: postProcess(...) - Service-->>Controller: Response DTO diff --git a/docs/diagrams/architecture/services-7.mmd b/docs/diagrams/architecture/services-7.mmd deleted file mode 100644 index 9ffceb5d9..000000000 --- a/docs/diagrams/architecture/services-7.mmd +++ /dev/null @@ -1,7 +0,0 @@ -flowchart TD - Client["Client Application"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] - ApiCore --> ServicesNode["Services Module"] - ServicesNode --> Mongo[("MongoDB")] - ServicesNode --> Authz["Authorization Service"] - ServicesNode --> Crypto["Encryption Service"] diff --git a/docs/diagrams/architecture/services.mmd b/docs/diagrams/architecture/services.mmd deleted file mode 100644 index b3c9c9b9e..000000000 --- a/docs/diagrams/architecture/services.mmd +++ /dev/null @@ -1,26 +0,0 @@ -flowchart LR - subgraph Controllers["API Layer"] - RestControllers["REST Controllers"] - GraphQL["GraphQL DataFetchers"] - end - - subgraph ServicesLayer["Services Module"] - UserServiceNode["UserService"] - SSOServiceNode["SSOConfigService"] - DomainValidator["DefaultDomainExistenceValidator"] - end - - subgraph Processors["Processors"] - UserProcessorNode["UserProcessor"] - SSOProcessorNode["SSOConfigProcessor"] - end - - subgraph DataLayer["Data Layer"] - UserRepo["UserRepository"] - SSORepo["SSOConfigRepository"] - MongoDocs["Mongo Domain Models"] - end - - Controllers --> ServicesLayer - ServicesLayer --> Processors - ServicesLayer --> DataLayer diff --git a/docs/diagrams/architecture/stream-processing-kafka-2.mmd b/docs/diagrams/architecture/stream-processing-kafka-2.mmd new file mode 100644 index 000000000..2934888be --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka-2.mmd @@ -0,0 +1,6 @@ +flowchart TD + Config["KafkaStreamsConfig"] --> AppId["Application ID"] + Config --> Serdes["Custom JSON Serdes"] + Config --> ConsumerCfg["Consumer Config"] + Config --> ProducerCfg["Producer Config"] + Config --> StateDir["State Store Directory"] diff --git a/docs/diagrams/architecture/stream-processing-kafka-3.mmd b/docs/diagrams/architecture/stream-processing-kafka-3.mmd new file mode 100644 index 000000000..4161fe769 --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka-3.mmd @@ -0,0 +1,3 @@ +flowchart LR + Topics["Inbound Kafka Topics"] --> Listener["JsonKafkaListener"] + Listener --> Processor["GenericJsonMessageProcessor"] diff --git a/docs/diagrams/architecture/stream-processing-kafka-4.mmd b/docs/diagrams/architecture/stream-processing-kafka-4.mmd new file mode 100644 index 000000000..6ea679c95 --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka-4.mmd @@ -0,0 +1,6 @@ +flowchart TD + Source["Tool Event"] --> ToolType["IntegratedToolType"] + Source --> SourceType["Source Event Type"] + ToolType --> Mapper["EventTypeMapper"] + SourceType --> Mapper + Mapper --> Unified["UnifiedEventType"] diff --git a/docs/diagrams/architecture/stream-processing-kafka-5.mmd b/docs/diagrams/architecture/stream-processing-kafka-5.mmd new file mode 100644 index 000000000..c14b6a60f --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka-5.mmd @@ -0,0 +1,5 @@ +flowchart LR + Event["DeserializedDebeziumMessage"] --> Enrichment + Enrichment --> MachineCache["MachineIdCacheService"] + Enrichment --> TenantResolver["Tenant Resolver"] + Enrichment --> Enriched["IntegratedToolEnrichedData"] diff --git a/docs/diagrams/architecture/stream-processing-kafka-6.mmd b/docs/diagrams/architecture/stream-processing-kafka-6.mmd new file mode 100644 index 000000000..aa055b136 --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka-6.mmd @@ -0,0 +1,4 @@ +flowchart TD + DebeziumMsg["DeserializedDebeziumMessage"] --> Transform + Transform --> Unified["UnifiedLogEvent"] + Unified --> Cassandra["Cassandra Repository"] diff --git a/docs/diagrams/architecture/stream-processing-kafka-7.mmd b/docs/diagrams/architecture/stream-processing-kafka-7.mmd new file mode 100644 index 000000000..4d9dc7cfb --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka-7.mmd @@ -0,0 +1,5 @@ +flowchart LR + Activity["Activity Topic"] --> Join + HostActivity["HostActivity Topic"] --> Join + Join --> Enriched["Enriched Activity"] + Enriched --> Output["Fleet MDM Events Topic"] diff --git a/docs/diagrams/architecture/stream-processing-kafka-8.mmd b/docs/diagrams/architecture/stream-processing-kafka-8.mmd new file mode 100644 index 000000000..a19885a6a --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka-8.mmd @@ -0,0 +1,10 @@ +flowchart TD + A["Integrated Tool"] --> B["Debezium CDC"] + B --> C["Kafka Topic"] + C --> D["JsonKafkaListener"] + D --> E["Deserializer"] + E --> F["Data Enrichment"] + F --> G["EventTypeMapper"] + G --> H["Message Handler"] + H --> I["Cassandra"] + H --> J["Outbound Kafka"] diff --git a/docs/diagrams/architecture/stream-processing-kafka.mmd b/docs/diagrams/architecture/stream-processing-kafka.mmd new file mode 100644 index 000000000..7ae199d1c --- /dev/null +++ b/docs/diagrams/architecture/stream-processing-kafka.mmd @@ -0,0 +1,11 @@ +flowchart LR + ToolA["Integrated Tools\nMeshCentral / Tactical / Fleet"] --> Debezium["Debezium CDC"] + Debezium --> KafkaIn["Kafka Inbound Topics"] + KafkaIn --> Listener["JsonKafkaListener"] + Listener --> Processor["GenericJsonMessageProcessor"] + Processor --> Deserializer["Tool-Specific Deserializers"] + Deserializer --> Enrichment["IntegratedToolDataEnrichmentService"] + Enrichment --> Mapper["EventTypeMapper"] + Mapper --> Handler["DebeziumMessageHandler"] + Handler --> Cassandra["Cassandra (UnifiedLogEvent)"] + Handler --> KafkaOut["Outbound Kafka Topics"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-2.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-2.mmd deleted file mode 100644 index 90b3c0db9..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-2.mmd +++ /dev/null @@ -1,10 +0,0 @@ -flowchart TD - A["Kafka Debezium Event"] --> B["JsonKafkaListener"] - B --> C["GenericJsonMessageProcessor"] - C --> D["Tool-Specific Deserializer"] - D --> E["Unified DeserializedDebeziumMessage"] - E --> F["IntegratedToolDataEnrichmentService"] - F --> G["EventTypeMapper"] - G --> H["DebeziumMessageHandler"] - H --> I["Destination Handler"] - I --> J["Cassandra or Kafka"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-3.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-3.mmd deleted file mode 100644 index e3f6fdecd..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-3.mmd +++ /dev/null @@ -1,6 +0,0 @@ -flowchart LR - ActivityTopic["Fleet Activities"] --> Join - HostActivityTopic["Fleet Host Activities"] --> Join - Join["Left Join (5s Window)"] --> Enriched["Enriched ActivityMessage"] - Enriched --> HeaderAdder["Add MESSAGE_TYPE_HEADER"] - HeaderAdder --> OutputTopic["Fleet MDM Events Topic"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-4.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-4.mmd deleted file mode 100644 index 021b220e7..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-4.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart LR - ToolType["IntegratedToolType"] --> Key - SourceType["Source Event Type"] --> Key - Key["tool:sourceType"] --> Mapper["EventTypeMapper"] - Mapper --> Unified["UnifiedEventType"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-5.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-5.mmd deleted file mode 100644 index 260c90f84..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers-5.mmd +++ /dev/null @@ -1,5 +0,0 @@ -flowchart TD - Event["DeserializedDebeziumMessage"] --> MachineLookup["MachineIdCacheService"] - MachineLookup --> OrgLookup["Organization Cache"] - OrgLookup --> TenantResolution["TenantIdProvider or ClusterTenantIdResolver"] - TenantResolution --> Enriched["IntegratedToolEnrichedData"] diff --git a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers.mmd b/docs/diagrams/architecture/stream-service-core-kafka-and-handlers.mmd deleted file mode 100644 index 27fcc8807..000000000 --- a/docs/diagrams/architecture/stream-service-core-kafka-and-handlers.mmd +++ /dev/null @@ -1,43 +0,0 @@ -flowchart LR - subgraph Tools["Integrated Tools"] - MeshCentral["MeshCentral"] - Tactical["Tactical RMM"] - Fleet["Fleet MDM"] - end - - subgraph KafkaIn["Inbound Kafka Topics"] - InTopics["Debezium Topics"] - end - - subgraph StreamCore["Stream Service Core Kafka And Handlers"] - Listener["JsonKafkaListener"] - Processor["GenericJsonMessageProcessor"] - Deserializer["Tool Event Deserializers"] - Enrichment["IntegratedToolDataEnrichmentService"] - Mapper["EventTypeMapper"] - Handler["DebeziumMessageHandler"] - CassandraHandler["DebeziumCassandraMessageHandler"] - TenantKafkaHandler["TenantDebeziumKafkaMessageHandler"] - end - - subgraph Storage["Storage & Downstream"] - Cassandra["Cassandra UnifiedLogEvent"] - KafkaOut["Outbound Kafka Topics"] - end - - MeshCentral --> InTopics - Tactical --> InTopics - Fleet --> InTopics - - InTopics --> Listener - Listener --> Processor - Processor --> Deserializer - Deserializer --> Enrichment - Enrichment --> Mapper - Mapper --> Handler - - Handler --> CassandraHandler - Handler --> TenantKafkaHandler - - CassandraHandler --> Cassandra - TenantKafkaHandler --> KafkaOut diff --git a/docs/getting-started/first-steps.md b/docs/getting-started/first-steps.md index a9bcf6f73..efe5f32f2 100644 --- a/docs/getting-started/first-steps.md +++ b/docs/getting-started/first-steps.md @@ -1,178 +1,142 @@ # First Steps -After completing the [Quick Start](quick-start.md), here are the first five things to explore in **openframe-oss-lib** to become productive quickly. +After cloning and building `openframe-oss-lib`, here are the 5 key things to do to start being productive with the codebase. --- -## 1. Understand the Module Structure +## 1. Explore the Module Structure -The repository is a Maven multi-module project. Each module is independently deployable and follows a consistent pattern: - -```text -openframe-/ -β”œβ”€β”€ src/ -β”‚ β”œβ”€β”€ main/java/com/openframe/... # Production code -β”‚ └── test/java/com/openframe/... # Unit and integration tests -└── pom.xml # Module POM (inherits parent) -``` - -Start by reviewing the parent POM at the repository root to understand the full module list and shared dependency versions: +The repository is a Maven multi-module project. Start by understanding which modules are available: ```bash -cat pom.xml +# List all modules from the parent POM +mvn help:evaluate -Dexpression=project.modules -q -DforceStdout ``` -Key properties to note: +Alternatively, review the top-level `pom.xml` β€” it lists all 29+ modules under the `` section. Key modules to know first: -| Property | Value | -|----------|-------| -| `revision` | Current unified version (`5.79.3`) | -| `java.version` | `21` | -| `spring-boot-starter-parent` | `3.3.0` | -| `spring-cloud.version` | `2023.0.3` | +| Module | What to Explore | +|--------|----------------| +| `openframe-core` | Shared utilities, validation annotations, pagination | +| `openframe-exception` | Exception hierarchy and `ErrorCode` enum | +| `openframe-data-mongo-common` | Core domain document models (User, Device, Ticket, etc.) | +| `openframe-api-lib` | Shared API contract DTOs and filters | +| `openframe-security-core` | JWT configuration and `AuthPrincipal` | --- -## 2. Explore the Core Domain Model +## 2. Understand the Domain Models -The best entry point for understanding the data model is `openframe-data-mongo-common`. This module defines all MongoDB documents: +The `openframe-data-mongo-common` module defines all core MongoDB documents. Spend time reviewing these documents to understand the data model: ```bash +# Browse the document models ls openframe-data-mongo-common/src/main/java/com/openframe/data/document/ ``` -Key domain areas: - -| Package | Domain | -|---------|--------| -| `device/` | `Device`, `Machine`, `DeviceHealth` | -| `organization/` | `Organization`, `ContactInformation` | -| `user/` | `User`, `AuthUser`, `Invitation` | -| `ticket/` | `Ticket`, `TicketNote`, `TicketAttachment` | -| `tool/` | `IntegratedTool`, `ToolConnection`, `ToolCredentials` | -| `notification/` | `Notification`, `NotificationContext`, `ReadStatus` | -| `tenant/` | `Tenant`, `TenantKey`, `SSOPerTenantConfig` | -| `oauth/` | `MongoRegisteredClient`, `OAuthToken` | +Key documents: +- `device/Machine.java` – Physical device / endpoint representation +- `ticket/Ticket.java` – Support ticket with lifecycle states +- `organization/Organization.java` – Client organization record +- `user/User.java` + `auth/AuthUser.java` – Platform user and auth data +- `notification/Notification.java` – In-platform notification system +- `tool/IntegratedTool.java` – MSP tool integration record -Read the domain model reference documentation for a full data-flow diagram and entity relationships: -[./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md](./reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md) +> All domain documents implement `TenantScoped`, meaning every query includes a `tenantId` filter for multi-tenant isolation. --- -## 3. Try the Security Modules +## 3. Run the Module Tests -The security stack is a critical foundation for any OpenFrame service. Explore: +Integration tests use **Testcontainers** β€” they spin up real MongoDB, Redis, and NATS instances in Docker automatically. No manual setup required. -### `openframe-security-core` +```bash +# Run unit tests for core modules (fast, no Docker needed) +mvn test -pl openframe-core +mvn test -pl openframe-exception +mvn test -pl openframe-api-lib -Provides JWT signing and verification: +# Run integration tests for MongoDB sync (requires Docker) +mvn verify -pl openframe-data-mongo-sync -```java -// Inject the JwtService to sign or validate tokens -@Autowired -private JwtService jwtService; +# Run integration tests for NATS module (requires Docker) +mvn verify -pl openframe-data-nats ``` -Properties to configure (in `application.yml`): +> **Tip:** Look at integration test base classes like `BaseMongoIntegrationTest` in `openframe-data-mongo-sync` to understand the test harness patterns used across the project. -```yaml -jwt: - public-key: classpath:keys/public.pem - private-key: classpath:keys/private.pem - issuer: https://your-tenant.openframe.ai - audience: openframe-api -``` +--- -### `openframe-security-oauth` +## 4. Explore the API Contracts -The OAuth BFF module provides ready-made endpoints for browser-based OAuth flows. To enable: +The `openframe-api-lib` module defines the contracts used by all API services. Understanding these DTOs is essential before building any service: -```yaml -openframe: - gateway: - oauth: - enable: true +```bash +ls openframe-api-lib/src/main/java/com/openframe/api/dto/ ``` -Exposed endpoints automatically: +Key contract categories: -- `GET /oauth/login` -- `GET /oauth/callback` -- `POST /oauth/refresh` -- `GET /oauth/logout` +| Package | Contents | +|---------|---------| +| `dto/device/` | Device filter criteria and filter options | +| `dto/organization/` | Organization CRUD request/response | +| `dto/command/` | Remote command dispatch contracts | +| `dto/script/` | Script management inputs and responses | +| `dto/knowledgebase/` | Knowledge base article management | +| `dto/shared/` | Relay pagination: `ConnectionArgs`, `CursorCodec` | +| `service/` | Service interfaces: `DeviceService`, `ToolService`, etc. | --- -## 4. Run Your First Integration Test +## 5. Join the Community and Get Help -The `openframe-data-mongo-sync` module has a comprehensive integration test suite using Testcontainers. Run it to verify your local Docker setup: +OpenFrame is developed in the open. The primary community channel is **Slack**: -```bash -# Start Docker first, then run integration tests -mvn verify -pl openframe-data-mongo-sync -Pfailsafe -``` +[![Join OpenMSP Slack](https://img.shields.io/badge/Slack-OpenMSP-blue?logo=slack)](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) -Testcontainers will automatically: -1. Pull the MongoDB Docker image -2. Start a containerized MongoDB instance -3. Run all `*IT.java` tests against it -4. Tear down the container on completion +All discussions, questions, and feature requests are managed on [OpenMSP Slack](https://www.openmsp.ai/). -Integration test classes follow the `*IT.java` naming convention (configured in the parent `maven-surefire-plugin`). +Also explore: +- **GitHub Repository**: [flamingo-stack/openframe-oss-lib](https://github.com/flamingo-stack/openframe-oss-lib) +- **Flamingo Platform**: [flamingo.run](https://flamingo.run) +- **OpenFrame Platform**: [openframe.ai](https://openframe.ai) --- -## 5. Explore the Gateway Module +## Common Initial Configuration -The gateway is the entry point for all service traffic. Review: +### OSS Single-Tenant Mode + +For OSS deployments (single tenant), the minimum configuration is: ```bash -ls openframe-gateway-service-core/src/main/java/com/openframe/gateway/ +# Set environment variables for single-tenant OSS mode +export TENANT_ID=oss +export SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/openframe +export SPRING_REDIS_HOST=localhost ``` -Key files to read: - -| File | Purpose | -|------|---------| -| `security/GatewaySecurityConfig.java` | Main reactive security filter chain | -| `security/filter/ApiKeyAuthenticationFilter.java` | API key authentication + rate limiting | -| `config/ws/WebSocketGatewayConfig.java` | WebSocket routing configuration | -| `upstream/DefaultToolUpstreamResolver.java` | Tool proxy URL resolution | +### Multi-Tenant SaaS Mode -The gateway supports these route patterns: - -```text -/api/** β†’ ADMIN role required -/tools/agent/** β†’ AGENT role required -/ws/tools/** β†’ WebSocket proxy to integrated tools -/external-api/** β†’ API key authentication -``` +For multi-tenant SaaS deployments, the `TENANT_ID` environment variable is resolved per-request by the `DefaultTenantIdProvider`, and each tenant has isolated MongoDB data. --- -## Where to Get Help +## Explore the Reference Documentation -| Resource | Link | -|----------|------| -| OpenMSP Community (Slack) | [https://www.openmsp.ai/](https://www.openmsp.ai/) | -| OpenFrame Platform | [https://openframe.ai](https://openframe.ai) | -| Flamingo | [https://flamingo.run](https://flamingo.run) | -| Reference Architecture | [./reference/architecture/README.md](./reference/architecture/README.md) | +The reference documentation (generated by CodeWiki) provides detailed architectural information for each major module. Check these resources: -> **Note:** We use Slack for all community support and discussions. Please join the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) instead of creating GitHub Issues. +- [Architecture Overview](../development/architecture/README.md) β€” high-level system diagram +- [Reference Docs](./reference/architecture/README.md) β€” module-by-module deep dives --- -## What to Explore Next - -Once comfortable with the basics: - -- Review the **Authorization Service** to understand multi-tenant JWT issuance -- Explore **Stream Service Core** for Kafka / Debezium event processing -- Look at **Management Service Core** for startup initializers and schedulers -- Check the **External API Service** for integration patterns +## Next Steps -All reference documentation is available under: -[./reference/architecture/](./reference/architecture/) +After completing your first steps: -[![OpenFrame v0.3.7 - Enhanced Developer Experience](https://img.youtube.com/vi/O8hbBO5Mym8/maxresdefault.jpg)](https://www.youtube.com/watch?v=O8hbBO5Mym8) +- Set up your [Development Environment](../development/setup/environment.md) for contributing +- Read the [Local Development Guide](../development/setup/local-development.md) +- Review the [Architecture README](../development/architecture/README.md) to understand data flows +- Check [Contributing Guidelines](../development/contributing/guidelines.md) before submitting code diff --git a/docs/getting-started/introduction.md b/docs/getting-started/introduction.md index 1bcf37af4..44c7d8c51 100644 --- a/docs/getting-started/introduction.md +++ b/docs/getting-started/introduction.md @@ -1,43 +1,34 @@ -# OpenFrame OSS Lib β€” Introduction +# Introduction to OpenFrame OSS Lib -**openframe-oss-lib** is the foundational backend library powering the [OpenFrame](https://openframe.ai) platform β€” the AI-driven, open-source MSP infrastructure stack built by [Flamingo](https://flamingo.run). +**OpenFrame OSS Lib** (`openframe-oss-lib`) is the modular backbone of [OpenFrame](https://openframe.ai) β€” Flamingo's AI-powered MSP platform that replaces expensive proprietary software with open-source alternatives enhanced by intelligent automation. -This repository provides the **shared Spring Boot libraries** that every OpenFrame service is built upon: authentication, API layers, gateway routing, streaming, persistence, and more. +This library provides the shared, composable building blocks that power the entire OpenFrame stack: from multi-tenant identity and OAuth2 authorization, to reactive APIs, event-driven stream processing, and an embeddable AI chat engine. -[![OpenFrame Product Walkthrough (Beta Access)](https://img.youtube.com/vi/awc-yAnkhIo/maxresdefault.jpg)](https://www.youtube.com/watch?v=awc-yAnkhIo) +[![OpenFrame: 5-Minute MSP Platform Walkthrough - Cut Vendor Costs & Automate Ops](https://img.youtube.com/vi/er-z6IUnAps/maxresdefault.jpg)](https://www.youtube.com/watch?v=er-z6IUnAps) --- ## What Is OpenFrame OSS Lib? -OpenFrame OSS Lib is a multi-module Maven project (Java 21, Spring Boot 3.3) that delivers the **core backend infrastructure** for the OpenFrame MSP platform. Rather than a standalone application, it is a set of reusable modules consumed by every service deployed inside an OpenFrame installation. +`openframe-oss-lib` is a multi-module **Spring Boot 3 / Java 21** Maven project (version `6.0.10`) that provides reusable infrastructure libraries for building OpenFrame-compatible services. It is the foundational layer consumed by all services running inside the OpenFrame platform. -The library covers: - -- **Multi-tenant authentication and authorization** β€” OAuth2 Authorization Server, per-tenant RSA key pairs, JWT issuance, SSO integration -- **API service layers** β€” REST controllers and Relay-compliant GraphQL execution -- **Gateway routing and security** β€” reactive edge gateway, API key rate limiting, WebSocket proxying -- **Event ingestion and streaming** β€” Kafka / Debezium CDC processing, Kafka Streams enrichment -- **Real-time messaging** β€” NATS publish/subscribe for device and tool notifications -- **Polyglot persistence** β€” MongoDB (sync + reactive), Redis caching, Apache Pinot analytics, Cassandra log storage -- **Management initializers and schedulers** β€” distributed ShedLock-based cluster-safe schedulers, startup bootstrapping -- **External REST API** β€” API key–authenticated integration surface for third parties -- **SDKs** β€” Fleet MDM and Tactical RMM Java clients +> This repository does **not** ship a standalone application β€” it provides libraries and service-core modules that are embedded into deployable services. --- -## Key Features & Benefits +## Key Features | Feature | Description | |---------|-------------| -| Multi-tenant by default | Every module is designed with tenant isolation: scoped keys, scoped caches, scoped scheduler locks | -| Spring Boot 3.3 / Java 21 | Latest LTS Java with virtual threads ready, modern Spring Security | -| Modular Maven structure | 30+ independent modules β€” include only what you need | -| Open source, GitHub Packages | Published to GitHub Maven Packages with unified versioning | -| Tool integrations | First-class support for Tactical RMM, Fleet MDM, MeshCentral | -| AI-ready event model | Unified event type model enables Mingo AI and Fae agent consumption | -| GraphQL + REST | Relay-compliant GraphQL for the internal API and versioned REST for external integrations | -| Reactive gateway | Spring Cloud Gateway + WebFlux + Netty for high-concurrency proxying | +| **Multi-Tenant Identity** | OAuth2 Authorization Server with per-tenant RSA key pairs and OIDC support | +| **Reactive API Layer** | REST + GraphQL (Netflix DGS) with Relay-style pagination and DataLoader batching | +| **Gateway & WebSocket Routing** | Spring Cloud Gateway with JWT validation, rate limiting, and tool proxying | +| **MongoDB Data Layer** | Reactive + sync repositories, cursor pagination, multi-tenant scoping | +| **Kafka Stream Processing** | Debezium CDC ingestion, event normalization, Kafka Streams enrichment | +| **Management & Operations** | Distributed schedulers, migration support (Mongock), NATS initialization | +| **Frontend UI & AI Chat** | Embeddable React AI assistant (Guide + Mingo modes), Kanban, Notifications | +| **Integrated Tool SDKs** | Native SDKs for MeshCentral, Tactical RMM, and Fleet MDM | +| **Shared API Contracts** | Centralized DTOs, filters, and pagination primitives for all services | --- @@ -45,92 +36,96 @@ The library covers: This library is intended for: -- **Platform engineers** building or extending OpenFrame microservices -- **MSP developers** integrating their tooling with the OpenFrame event model -- **Open-source contributors** improving the Flamingo/OpenFrame ecosystem - -> **Community:** Questions, discussions, and support happen on the [OpenMSP Slack Community](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). We don't use GitHub Issues. +- **MSP Platform Engineers** building services that integrate with OpenFrame +- **Backend Developers** implementing new OpenFrame microservices +- **Open-Source Contributors** extending the OpenFrame ecosystem +- **System Architects** designing multi-tenant SaaS or OSS MSP platforms --- -## High-Level Architecture +## Platform Overview ```mermaid flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service"] - Gateway --> ApiCore["API Service Core (GraphQL + REST)"] - - ApiCore --> Authz["Authorization Service Core"] - ApiCore --> Stream["Stream Service Core (Kafka)"] - ApiCore --> Management["Management Service Core"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - Stream --> Kafka["Kafka / Debezium"] - Stream --> Pinot["Apache Pinot"] - ApiCore --> Redis["Redis"] - Management --> NATS["NATS"] + Frontend["Frontend Core UI and Chat"] + Gateway["Gateway Service Core"] + Auth["Authorization Service Core"] + API["API Service Core (HTTP + GraphQL)"] + Management["Management Service Core"] + Data["Data Models and Repositories Mongo"] + Stream["Stream Processing Kafka"] + Cassandra["Cassandra (Unified Events)"] + Tools["Integrated Tools (MeshCentral, Tactical, Fleet)"] + + Frontend --> Gateway + Gateway --> Auth + Gateway --> API + API --> Data + API --> Stream + Stream --> Cassandra + Stream --> Data + Management --> Data + Management --> Stream + Tools --> Stream ``` --- -## Module Overview - -The library is organized into functional groups: - -```mermaid -graph LR - subgraph apiFoundation["API Foundation"] - A1["openframe-api-lib"] - A2["openframe-api-service-core"] - end - subgraph auth["Authorization"] - B1["openframe-authorization-service-core"] - B2["openframe-security-core"] - B3["openframe-security-oauth"] - end - subgraph gateway["Gateway"] - C1["openframe-gateway-service-core"] - end - subgraph data["Data Layer"] - D1["openframe-data-mongo-*"] - D2["openframe-data-redis"] - D3["openframe-data-kafka"] - D4["openframe-data-nats"] - D5["openframe-data-cassandra"] - D6["openframe-data-pinot"] - end - subgraph tools["Tool SDKs"] - E1["sdk/fleetmdm"] - E2["sdk/tacticalrmm"] - end - subgraph management["Management"] - F1["openframe-management-service-core"] - F2["openframe-stream-service-core"] - end -``` +## Module Ecosystem + +The library is organized into 29+ composable modules, grouped by domain: + +### Core Infrastructure +- `openframe-core` – Shared utilities, constants, validation +- `openframe-exception` – Unified exception hierarchy +- `openframe-core-crypto` – Encryption services +- `openframe-config-core` – Configuration server support + +### Data Layer +- `openframe-data-mongo-common` – Document models and base repositories +- `openframe-data-mongo-sync` – Synchronous MongoDB repositories +- `openframe-data-mongo-reactive` – Reactive MongoDB repositories +- `openframe-data-redis` – Redis caching and rate limiting +- `openframe-data-kafka` – Kafka producer utilities +- `openframe-data-cassandra` – Cassandra persistence for event logs +- `openframe-data-pinot` – Apache Pinot analytics queries +- `openframe-data-nats` – NATS messaging and pub/sub + +### Service Cores +- `openframe-api-service-core` – REST + GraphQL application layer +- `openframe-api-lib` – Shared API contracts and DTOs +- `openframe-authorization-service-core` – OAuth2/OIDC Authorization Server +- `openframe-gateway-service-core` – Reactive edge gateway +- `openframe-management-service-core` – Operational control plane +- `openframe-stream-service-core` – Kafka event ingestion pipeline +- `openframe-client-core` – Agent registration and device management +- `openframe-security-core` – JWT and security primitives +- `openframe-security-oauth` – OAuth2 BFF (Backend For Frontend) + +### Tool Integrations +- `openframe-tactical-sdk` – Tactical RMM integration +- `sdk/fleetmdm` – Fleet MDM client SDK +- `sdk/tacticalrmm` – Tactical RMM client SDK + +### Frontend +- `openframe-frontend-core` – Reusable React UI components and AI Chat --- -## Current Version +## The Flamingo / OpenFrame Platform -The current library version is **5.79.3** β€” published to [GitHub Packages](https://github.com/flamingo-stack/openframe-oss-lib). - ---- +OpenFrame is part of the [Flamingo](https://flamingo.run) AI-powered MSP stack: -## Links & Resources +- **Mingo AI** – AI assistant for technicians +- **Fae** – AI assistant for clients +- **OpenFrame** – Unified platform integrating MSP tools with intelligent automation -- **OpenFrame Platform:** [https://openframe.ai](https://openframe.ai) -- **Flamingo:** [https://flamingo.run](https://flamingo.run) -- **GitHub Repository:** [https://github.com/flamingo-stack/openframe-oss-lib](https://github.com/flamingo-stack/openframe-oss-lib) -- **OpenMSP Community (Slack):** [https://www.openmsp.ai/](https://www.openmsp.ai/) -- **Reference Architecture Docs:** [./reference/architecture/README.md](./reference/architecture/README.md) +Join the community: [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA) --- ## Next Steps -- Review the [Prerequisites Guide](prerequisites.md) for environment requirements -- Follow the [Quick Start Guide](quick-start.md) to set up the library -- Explore [First Steps](first-steps.md) to understand the key modules +- Review the [Prerequisites](prerequisites.md) to set up your environment +- Follow the [Quick Start Guide](quick-start.md) to get up and running in 5 minutes +- Explore [First Steps](first-steps.md) after your initial setup diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index 8a194c1e6..3d420e6e4 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -1,56 +1,57 @@ # Prerequisites -Before working with **openframe-oss-lib**, ensure your development environment meets all requirements below. +Before working with `openframe-oss-lib`, ensure your development environment meets the following requirements. --- ## Required Software -| Tool | Minimum Version | Purpose | -|------|----------------|---------| -| Java (JDK) | 21 | Required by all modules (Spring Boot 3.3 baseline) | -| Apache Maven | 3.9+ | Build and dependency management | -| Git | 2.x | Source code management | -| Docker | 24.x | Running integration test containers | -| Node.js | 20+ | Required for the documentation tooling (`package.json` present) | +| Tool | Minimum Version | Notes | +|------|----------------|-------| +| **Java (JDK)** | 21 | Project uses Java 21 features; LTS recommended | +| **Apache Maven** | 3.8+ | Used for building all modules | +| **Git** | 2.x+ | For cloning the repository | +| **Docker** | 24.x+ | Required for integration tests (Testcontainers) | +| **Docker Compose** | 2.x+ | Available for local infrastructure setup | +| **Node.js** | 18+ | Required for frontend-core module development | ---- - -## Java Version +> **Note:** The project uses the `flatten-maven-plugin` with CI-friendly versioning. Maven Wrapper (`./mvnw`) is recommended if available. -This library targets **Java 21** (LTS). Ensure your `JAVA_HOME` points to a Java 21 JDK: +--- -```bash -java -version -# Should output: openjdk 21.x.x ... -``` +## System Requirements -Recommended distributions: +| Resource | Minimum | Recommended | +|----------|---------|-------------| +| **RAM** | 8 GB | 16 GB+ | +| **CPU Cores** | 4 | 8+ | +| **Disk Space** | 10 GB free | 20 GB free | +| **OS** | Linux / macOS / Windows (WSL2) | Linux / macOS | -- [Eclipse Temurin 21](https://adoptium.net/) -- [Amazon Corretto 21](https://aws.amazon.com/corretto/) -- [GraalVM 21](https://www.graalvm.org/) +> Integration tests use **Testcontainers** to spin up MongoDB and other services in Docker. Docker must be accessible to the running JVM process. --- -## Maven +## Infrastructure Dependencies -Maven 3.9 or higher is required: +When running services locally, the following infrastructure components are required: -```bash -mvn -version -# Apache Maven 3.9.x ... -``` +| Service | Version | Purpose | +|---------|---------|---------| +| **MongoDB** | 6.x+ | Primary data store for all domain documents | +| **Redis** | 7.x+ | Caching, rate limiting, and distributed locks (ShedLock) | +| **Apache Kafka** | 3.x+ | Event streaming and CDC processing | +| **NATS** | 2.x+ | Agent messaging and real-time notifications | +| **Apache Cassandra** | 4.x+ | Long-term unified event log storage | +| **Apache Pinot** | 1.2.0 | Analytics query engine | -The project uses the `flatten-maven-plugin` for CI-friendly versioning (`${revision}`), so Maven 3.9+ is required for proper resolution. +> For unit and module-level tests, these are not required β€” Testcontainers handles in-process infrastructure. --- -## GitHub Packages Access - -The library is published to **GitHub Maven Packages**. To consume it as a dependency in downstream services you need a GitHub Personal Access Token (PAT) with `read:packages` permission. +## GitHub Package Registry Access -Add to `~/.m2/settings.xml`: +The library publishes artifacts to **GitHub Maven Package Registry**. To consume modules as Maven dependencies, configure your `~/.m2/settings.xml`: ```xml @@ -58,106 +59,83 @@ Add to `~/.m2/settings.xml`: github YOUR_GITHUB_USERNAME - YOUR_GITHUB_PAT + YOUR_GITHUB_TOKEN ``` -> Your `GITHUB_PAT` must have the `read:packages` scope to resolve dependencies, and `write:packages` to publish new versions. +Generate a GitHub Personal Access Token (PAT) with `read:packages` scope from [GitHub Settings β†’ Developer Settings](https://github.com/settings/tokens). --- -## Infrastructure Services (for Integration Tests) - -Some modules run integration tests against live services via Testcontainers. Ensure Docker is running and the following images can be pulled: +## Key Environment Variables -| Service | Used By | -|---------|---------| -| MongoDB | `openframe-data-mongo-sync`, `openframe-api-service-core` integration tests | -| NATS | `openframe-data-nats` integration tests | +The following environment variables are used across modules: -Testcontainers will automatically spin up and tear down containers during integration test execution. The only requirement is a running Docker daemon. +| Variable | Module | Description | +|----------|--------|-------------| +| `TENANT_ID` | `openframe-data-mongo-common` | Tenant ID for multi-tenant scoping (defaults to `oss`) | +| `SPRING_DATA_MONGODB_URI` | All data modules | MongoDB connection string | +| `SPRING_REDIS_HOST` | `openframe-data-redis` | Redis host | +| `SPRING_KAFKA_BOOTSTRAP_SERVERS` | `openframe-data-kafka` | Kafka broker addresses | +| `NATS_SERVER_URL` | `openframe-data-nats` | NATS server URL | -```bash -docker info -# Should not return an error -``` - ---- - -## Environment Variables - -The following environment variables may be needed depending on which modules you are developing: - -| Variable | Required For | Description | -|----------|-------------|-------------| -| `GITHUB_ACTOR` | Publishing | GitHub username for package publishing | -| `GITHUB_TOKEN` | Publishing | GitHub PAT for package publishing | - -For local development without publishing, only local Maven settings are needed. - ---- - -## Recommended IDE - -| IDE | Notes | -|-----|-------| -| IntelliJ IDEA (Ultimate or Community) | Best support for Spring Boot, Maven multi-module, Lombok | -| VS Code + Java Extension Pack | Alternative for lighter setup | - -### IntelliJ Setup Checklist - -1. Import as **Maven project** (not Gradle) -2. Set **Project SDK** to Java 21 -3. Enable **Annotation Processors** for Lombok support -4. Set Maven delegate: `Settings β†’ Build Tools β†’ Maven β†’ Runner β†’ Delegate IDE build/run actions to Maven` +> In OSS single-tenant mode, `TENANT_ID` defaults to `oss` if not set. --- ## Verification Commands -Run these checks before beginning development: +Run these commands to verify your environment is ready: ```bash -# Verify Java version +# Verify Java version (must be 21+) java -version -# Verify Maven version +# Verify Maven is available mvn -version # Verify Docker is running docker info -# Verify Git configuration -git config user.name -git config user.email +# Verify Docker Compose +docker compose version + +# Verify Git +git --version + +# Verify Node.js (for frontend-core work) +node --version +npm --version +``` -# Verify GitHub Packages access (requires settings.xml configured) -mvn dependency:resolve -pl openframe-core -q +Expected Java output: +```text +openjdk version "21.x.x" ... ``` --- -## System Requirements +## IDE Recommendations -| Resource | Minimum | Recommended | -|----------|---------|-------------| -| RAM | 8 GB | 16 GB | -| Disk | 5 GB free | 20 GB free | -| CPU | 4 cores | 8+ cores | -| OS | macOS, Linux, Windows (WSL2) | macOS or Linux | +| IDE | Recommended Plugins | +|-----|---------------------| +| **IntelliJ IDEA** (recommended) | Lombok, Spring Boot, GraphQL | +| **VS Code** | Extension Pack for Java, Spring Boot Tools | +| **Eclipse** | Spring Tools Suite 4 | + +> Enable annotation processing in your IDE to support **Lombok** (`@Data`, `@Builder`, `@Slf4j`, etc.), which is used extensively across all modules. -> Building the entire repository (`mvn install`) compiles 30+ modules. Sufficient RAM (especially heap) prevents OOM during compilation. +--- + +## Maven Settings for Building -To increase Maven heap: +To build the full project locally, ensure access to the GitHub Maven Package Registry as described above, then verify your local Maven cache: ```bash -export MAVEN_OPTS="-Xmx4g -XX:MaxMetaspaceSize=512m" +# Verify Maven can resolve dependencies +mvn dependency:resolve -pl openframe-core --quiet ``` ---- - -## Community & Support - -If you run into setup issues, reach out on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). +If dependency resolution fails, check your GitHub token permissions and `settings.xml` configuration. diff --git a/docs/getting-started/quick-start.md b/docs/getting-started/quick-start.md index 8237d6d25..ba992e25e 100644 --- a/docs/getting-started/quick-start.md +++ b/docs/getting-started/quick-start.md @@ -1,6 +1,8 @@ # Quick Start -Get up and running with **openframe-oss-lib** in 5 minutes. +Get up and running with `openframe-oss-lib` in under 5 minutes. + +[![Getting Started with OpenFrame - Organization Setup Basics](https://img.youtube.com/vi/-_56_qYvMWk/maxresdefault.jpg)](https://www.youtube.com/watch?v=-_56_qYvMWk) --- @@ -11,190 +13,178 @@ Get up and running with **openframe-oss-lib** in 5 minutes. git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib -# 2. Build the full library (skip tests for speed) +# 2. Build all modules (skip tests for speed) mvn install -DskipTests -# 3. Verify the build -mvn verify -pl openframe-core -DskipTests +# 3. Run tests for a specific module +mvn test -pl openframe-core + +# 4. Run integration tests (requires Docker) +mvn verify -pl openframe-data-mongo-sync ``` --- -## Step 1 β€” Clone the Repository +## Step 1: Clone the Repository ```bash git clone https://github.com/flamingo-stack/openframe-oss-lib.git cd openframe-oss-lib ``` -The repository contains 30+ Maven modules under a single parent POM at the root. - --- -## Step 2 β€” Configure GitHub Packages (Required for Dependency Resolution) +## Step 2: Verify the Build -Add your GitHub credentials to `~/.m2/settings.xml`: +The project uses a `revision` property for CI-friendly versioning. The current version is **6.0.10**. -```xml - - - - github - YOUR_GITHUB_USERNAME - YOUR_GITHUB_PAT - - - +```bash +# Build all modules, skipping tests +mvn install -DskipTests + +# Expected output: BUILD SUCCESS ``` -> A GitHub Personal Access Token (PAT) with `read:packages` scope is required. See the [Prerequisites Guide](prerequisites.md) for details. +> **Note:** The first build may take several minutes as Maven downloads all dependencies. --- -## Step 3 β€” Build the Library +## Step 3: Use a Module as a Maven Dependency -Build all modules without running tests for the fastest initial setup: +After a successful local install, you can reference any module in your own project: -```bash -mvn install -DskipTests +```xml + + + com.openframe.oss + openframe-core + 6.0.10 + ``` -Expected output: +For the API contracts module: -```text -[INFO] Reactor Summary for OpenFrame OSS Libraries 5.79.3: -[INFO] -[INFO] openframe-exception .................... SUCCESS [ 3.5 s] -[INFO] openframe-core ........................ SUCCESS [ 2.1 s] -[INFO] openframe-core-crypto ................. SUCCESS [ 1.8 s] -[INFO] openframe-data-mongo-common ........... SUCCESS [ 4.2 s] -... -[INFO] BUILD SUCCESS +```xml + + com.openframe.oss + openframe-api-lib + 6.0.10 + ``` --- -## Step 4 β€” Add a Module as a Dependency +## Step 4: Add the Parent POM (Optional) -To use a specific module in your own Spring Boot service, add the parent BOM and the desired dependency to your `pom.xml`: +If you are building an OpenFrame-compatible service, inherit from the parent POM to get aligned dependency management: ```xml - - - - - com.openframe.oss - openframe-oss-lib - 5.79.3 - pom - import - - - - - - - - com.openframe.oss - openframe-core - - - - - com.openframe.oss - openframe-data-mongo-sync - - - - + com.openframe.oss - openframe-api-service-core - - + openframe-oss-lib + 6.0.10 + ``` ---- +This gives you pre-configured: +- Spring Boot 3.3.0 +- Java 21 +- Lombok 1.18.30 +- Spring Cloud 2023.0.3 +- NATS, Kafka, MongoDB, Pinot, gRPC versions -## Step 5 β€” Run Tests for a Specific Module +--- -To run tests for a single module: +## Step 5: Run a Single Module Test ```bash -# Run unit tests only for a specific module -mvn test -pl openframe-core +# Run unit tests for the exception module +mvn test -pl openframe-exception -# Run integration tests (requires Docker) +# Run integration tests for MongoDB sync module (requires Docker) mvn verify -pl openframe-data-mongo-sync ``` -> Integration tests use Testcontainers and require a running Docker daemon. - --- -## Available Modules β€” Quick Reference - -| Module | GroupId | Description | -|--------|---------|-------------| -| `openframe-core` | `com.openframe.oss` | Core utilities, pagination, validation | -| `openframe-exception` | `com.openframe.oss` | Standard exception hierarchy | -| `openframe-core-crypto` | `com.openframe.oss` | Encryption utilities | -| `openframe-security-core` | `com.openframe.oss` | JWT, PKCE, cookie service | -| `openframe-security-oauth` | `com.openframe.oss` | OAuth2 BFF layer | -| `openframe-authorization-service-core` | `com.openframe.oss` | Multi-tenant OAuth2 auth server | -| `openframe-api-lib` | `com.openframe.oss` | API contracts, filter DTOs | -| `openframe-api-service-core` | `com.openframe.oss` | REST + GraphQL API service layer | -| `openframe-gateway-service-core` | `com.openframe.oss` | Reactive gateway, routing, security | -| `openframe-data-mongo-common` | `com.openframe.oss` | MongoDB domain documents | -| `openframe-data-mongo-sync` | `com.openframe.oss` | Synchronous MongoDB repositories | -| `openframe-data-mongo-reactive` | `com.openframe.oss` | Reactive MongoDB repositories | -| `openframe-data-redis` | `com.openframe.oss` | Redis cache configuration | -| `openframe-data-kafka` | `com.openframe.oss` | Kafka multi-tenant configuration | -| `openframe-data-nats` | `com.openframe.oss` | NATS real-time messaging | -| `openframe-data-cassandra` | `com.openframe.oss` | Cassandra log storage | -| `openframe-data-pinot` | `com.openframe.oss` | Apache Pinot analytics | -| `openframe-management-service-core` | `com.openframe.oss` | Schedulers, initializers | -| `openframe-stream-service-core` | `com.openframe.oss` | Kafka streams, event enrichment | -| `openframe-external-api-service-core` | `com.openframe.oss` | External REST API | -| `sdk/fleetmdm` | `com.openframe.oss` | Fleet MDM Java SDK | -| `sdk/tacticalrmm` | `com.openframe.oss` | Tactical RMM Java SDK | +## Hello World: Using Core Utilities + +Here is a minimal example using the shared utilities from `openframe-core`: + +```java +import com.openframe.core.util.SlugUtil; +import com.openframe.core.service.AgentRegistrationSecretGenerator; + +// Generate a URL-safe slug +String slug = SlugUtil.toSlug("My MSP Organization"); +// Result: "my-msp-organization" + +// The AgentRegistrationSecretGenerator produces secure tokens +// for device registration flows +``` --- ## Expected Results -After a successful `mvn install -DskipTests` you should see: +After a successful build, you should see: ```text +[INFO] ------------------------------------------------------------------------ +[INFO] Reactor Summary for OpenFrame OSS Libraries 6.0.10: +[INFO] +[INFO] OpenFrame OSS Libraries ........................ SUCCESS +[INFO] openframe-exception ............................ SUCCESS +[INFO] openframe-core ................................ SUCCESS +[INFO] openframe-core-crypto ......................... SUCCESS +[INFO] openframe-data-mongo-common ................... SUCCESS +[INFO] openframe-data-mongo-sync ..................... SUCCESS +[INFO] openframe-data-mongo-reactive ................. SUCCESS +[INFO] ... [INFO] BUILD SUCCESS -[INFO] Total time: 2-4 minutes (depending on hardware) -[INFO] Finished at: ... +[INFO] ------------------------------------------------------------------------ ``` -All modules are installed into your local Maven repository (`~/.m2/repository/com/openframe/oss/`). - --- -## Video Walkthrough - -[![Getting Started with OpenFrame - Organization Setup Basics](https://img.youtube.com/vi/-_56_qYvMWk/maxresdefault.jpg)](https://www.youtube.com/watch?v=-_56_qYvMWk) +## Quick Module Reference + +| Module | Artifact ID | Use Case | +|--------|------------|----------| +| Core utilities | `openframe-core` | Slugs, pagination, validation | +| Exception handling | `openframe-exception` | Unified error responses | +| Encryption | `openframe-core-crypto` | AES encryption services | +| MongoDB common | `openframe-data-mongo-common` | Document models | +| MongoDB sync | `openframe-data-mongo-sync` | Synchronous repositories | +| MongoDB reactive | `openframe-data-mongo-reactive` | Reactive repositories | +| Redis | `openframe-data-redis` | Caching, rate limits | +| Kafka | `openframe-data-kafka` | Event producers | +| API contracts | `openframe-api-lib` | Shared DTOs and filters | +| Security core | `openframe-security-core` | JWT, auth principals | --- -## Troubleshooting +## Consuming from GitHub Packages + +If you prefer consuming published artifacts rather than building locally, add the GitHub Packages repository to your `pom.xml`: + +```xml + + + github + https://maven.pkg.github.com/flamingo-stack/openframe-oss-lib + + +``` -| Problem | Solution | -|---------|---------| -| `Could not resolve dependencies` | Check `~/.m2/settings.xml` for GitHub credentials | -| `OutOfMemoryError` during build | Set `export MAVEN_OPTS="-Xmx4g"` | -| Integration tests fail | Ensure Docker daemon is running | -| `flatten-maven-plugin` errors | Upgrade Maven to 3.9+ | +Ensure your `~/.m2/settings.xml` has a `server` entry with `id=github` and a valid GitHub token as described in the [Prerequisites](prerequisites.md). --- ## Next Steps -After building successfully, explore: +After your quick start: -- [First Steps Guide](first-steps.md) for key module walkthroughs -- [Development Environment Setup](../development/setup/environment.md) for IDE configuration -- [Architecture Overview](../development/architecture/README.md) for system design patterns +- Follow the [First Steps Guide](first-steps.md) to explore key features and modules +- Review the [Prerequisites](prerequisites.md) for full environment configuration +- Explore the [Architecture Overview](../development/architecture/README.md) to understand how modules fit together diff --git a/docs/reference/architecture/README.md b/docs/reference/architecture/README.md index 3d2a87c5d..c234fea88 100644 --- a/docs/reference/architecture/README.md +++ b/docs/reference/architecture/README.md @@ -1,447 +1,405 @@ # OpenFrame OSS Lib – Repository Overview -The **openframe-oss-lib** repository contains the foundational backend libraries powering the OpenFrame platform. It provides: +The **`openframe-oss-lib`** repository is the modular backbone of the OpenFrame platform β€” Flamingo’s AI-powered MSP stack. -- Multi-tenant authentication and authorization -- API service layers (REST + GraphQL) -- Gateway routing and security -- Event ingestion and streaming (Kafka) -- Real-time messaging (NATS) -- Polyglot persistence (MongoDB, Redis, Pinot) -- Management initializers and distributed schedulers +It provides: -This repository represents the **core backend infrastructure stack** of OpenFrame and is designed to be modular, extensible, and multi-tenant by default. +- βœ… Multi-tenant identity & OAuth2 authorization +- βœ… Reactive API layer (REST + GraphQL) +- βœ… Gateway & WebSocket routing +- βœ… MongoDB data layer (reactive + sync) +- βœ… Kafka-based stream processing & enrichment +- βœ… Operational management services +- βœ… Frontend UI & AI Chat foundation +- βœ… Shared API contract definitions + +This repository is structured as independent but composable modules forming a full end-to-end MSP platform stack. --- # High-Level Architecture -OpenFrame follows a layered, service-oriented architecture with strong separation between: - -- Identity & Authorization -- API Surface (Internal + External) -- Gateway & Edge Security -- Stream Processing -- Data Access Layer -- Caching & Messaging -- Management & Operations +Below is the complete system architecture represented across modules in this repository. ```mermaid flowchart TD - Client["Client / Browser / Agent"] --> Gateway["Gateway Service Core"] - Gateway --> ExternalAPI["External API Service"] - Gateway --> ApiCore["API Service Core"] - - ApiCore --> Authz["Authorization Service Core"] - ApiCore --> Stream["Stream Service Core"] - ApiCore --> Management["Management Service Core"] - - Authz --> Mongo["MongoDB"] - ApiCore --> Mongo - Stream --> Kafka["Kafka"] - Stream --> Pinot["Pinot"] - ApiCore --> Redis["Redis"] - Management --> NATS["NATS"] + Frontend["Frontend Core UI and Chat"] + Gateway["Gateway Service Core"] + Auth["Authorization Service Core"] + API["API Service Core (HTTP + GraphQL)"] + Management["Management Service Core"] + Data["Data Models and Repositories Mongo"] + Stream["Stream Processing Kafka"] + Cassandra["Cassandra (Unified Events)"] + Tools["Integrated Tools (MeshCentral, Tactical, Fleet)"] + + Frontend --> Gateway + Gateway --> Auth + Gateway --> API + API --> Data + API --> Stream + Stream --> Cassandra + Stream --> Data + Management --> Data + Management --> Stream + Tools --> Stream ``` --- -# Core Modules - -Below is a structured overview of the major modules contained in this repository. - ---- - -## 1. API Foundation +# Repository Modules -### πŸ”Ή `api-contracts-and-pagination` -Defines reusable API contracts: +## 1. `api-lib-contracts` -- Relay-style pagination (`ConnectionArgs`, cursors) -- Count-aware query results -- Opaque cursor encoding (`CursorCodec`) -- Standard mutation inputs +**Purpose:** +Defines all shared DTOs, filters, pagination primitives, and API boundary contracts. -πŸ“„ Core Docs: -- `CountedGenericQueryResult` -- `ConnectionArgs` -- `CursorCodec` -- `MutationDeleteInput` +### Responsibilities ---- +- Device, Organization, Script, Tool, KnowledgeBase DTOs +- Relay-style pagination (`ConnectionArgs`, `CursorCodec`) +- Filter criteria objects (DeviceFilterCriteria, LogFilterCriteria, etc.) +- Command dispatch contracts +- Shared mappers (e.g., `OrganizationMapper`) -### πŸ”Ή `api-domain-filters-dtos` -Strongly-typed filter DTOs for: +### Architectural Role -- Devices -- Events -- Logs -- Knowledge Base -- Organizations -- Tools +```mermaid +flowchart LR + Frontend --> API + API --> Contracts["Api Lib Contracts"] + Contracts --> Data +``` -Bridges API input β†’ Mongo query filters. +πŸ“˜ See module documentation: **Api Lib Contracts** --- -### πŸ”Ή `api-lib-core-services` -Reusable domain services: - -- Installed agent resolution -- Tool connections -- Ticket queries -- Knowledge base lifecycle hooks -- Device status processors - -Optimized for DataLoader and GraphQL batching. - ---- +## 2. `api-service-core-http-and-graphql` -## 2. API Service Core +**Purpose:** +Primary application-layer API surface. -### πŸ”Ή REST Controllers -Internal operational endpoints: +Exposes: -- Organizations -- Devices -- Users -- Invitations -- SSO Config -- API Keys -- Release versions -- Agent lifecycle control +- REST endpoints +- GraphQL (Netflix DGS) +- Relay global node resolution +- DataLoader batching +- OAuth2 Resource Server security -πŸ“„ Module: `api-service-core-rest-controllers` +### Key Components ---- +- Controllers (Device, Organization, User, Invitation, SSOConfig, etc.) +- GraphQL DataFetchers +- DataLoaders (N+1 prevention) +- SecurityConfig (JWT validation) +- Relay Node resolvers -### πŸ”Ή GraphQL Layer -Relay-compliant GraphQL execution layer: +### API Flow -- Data fetchers -- DataLoaders -- Relay Node resolution -- Cursor-based pagination -- Polymorphic type resolution +```mermaid +flowchart TD + Client["Frontend / API Client"] --> Gateway + Gateway --> API + API --> Controllers + API --> GraphQL + Controllers --> Services + GraphQL --> Services + Services --> Data +``` -πŸ“„ Modules: -- `api-service-core-graphql-datafetchers` -- `api-service-core-dataloaders` -- `api-service-core-graphql-dtos` -- `api-service-core-relay-type-resolution` +πŸ“˜ See module documentation: **Api Service Core HTTP And GraphQL** --- -### πŸ”Ή Security & Config -Provides: +## 3. `authorization-service-core` -- JWT resource server config -- Multi-issuer support -- Custom GraphQL scalars -- OAuth client initialization -- Argument resolvers +**Purpose:** +Multi-tenant OAuth2 + OIDC Authorization Server. -πŸ“„ Module: `api-service-core-config-and-security` +Handles: ---- - -## 3. Authorization Service Core - -Multi-tenant OAuth2 Authorization Server. +- JWT issuance +- Tenant-aware signing keys +- Dynamic OIDC client registration +- SSO (Google, Microsoft) +- Password reset & invitations +- OAuth2 persistence in Mongo -### Capabilities - -- Per-tenant RSA key pairs -- JWT issuance with `tenant_id` -- Dynamic client registration -- SSO flows (Google, Microsoft) -- PKCE support -- Invitation-based onboarding -- Tenant discovery & registration +### Token Flow ```mermaid flowchart TD - Browser --> AuthController["Auth Controllers"] - AuthController --> TenantContext["TenantContextFilter"] - TenantContext --> AuthServer["AuthorizationServerConfig"] - AuthServer --> TenantKeyService["TenantKeyService"] - TenantKeyService --> Mongo - AuthServer --> JWT["Signed JWT"] + User --> AuthServer["Authorization Service Core"] + AuthServer --> JWT["Tenant-Scoped JWT"] + JWT --> Gateway + Gateway --> API ``` -πŸ“„ Modules: -- `authorization-service-core-server-and-tenant` -- `authorization-service-core-auth-controllers-and-dtos` -- `authorization-service-core-keys-and-authorization-persistence` -- `authorization-service-core-sso-flow-and-utils` +Each tenant has: + +- Independent RSA keypair +- Isolated issuer +- Custom JWT claims + +πŸ“˜ See module documentation: **Authorization Service Core** --- -## 4. Gateway Service Core +## 4. `gateway-service-core` -Reactive edge gateway built on Spring Cloud Gateway + WebFlux. +**Purpose:** +Reactive edge layer (Spring Cloud Gateway + WebFlux). -### Responsibilities +Handles: + +- JWT validation +- API key authentication +- Rate limiting +- REST & WebSocket proxying +- Tool-specific upstream resolution -- JWT validation (multi-issuer) -- Role-based authorization -- API key authentication + rate limiting -- WebSocket proxying -- Tool upstream resolution -- Origin sanitization -- CORS enforcement +### Edge Routing ```mermaid -flowchart TD - Request --> OriginSanitizer - OriginSanitizer --> JwtAuth - JwtAuth --> RoleCheck - RoleCheck --> RouteDecision - RouteDecision --> ToolResolver - ToolResolver --> UpstreamTool +flowchart LR + Browser --> Gateway + Agent --> Gateway + ExternalAPI --> Gateway + + Gateway --> API + Gateway --> Auth + Gateway --> Tools ``` -πŸ“„ Module: `gateway-service-core-security-and-routing` +Supports: + +- MeshCentral +- Tactical RMM +- Fleet MDM +- NATS WebSocket streams + +πŸ“˜ See module documentation: **Gateway Service Core** --- -## 5. Stream Service Core (Kafka) +## 5. `management-service-core` -Event ingestion and normalization backbone. +**Purpose:** +Operational control plane. -### Responsibilities +Handles: -- Consume Debezium CDC events -- Tool-specific deserialization -- Event enrichment (tenant, device, org) -- Unified event type mapping -- Cassandra persistence -- Kafka republishing -- Kafka Streams enrichment +- Cluster version coordination +- Tool lifecycle management +- Initializers (NATS, agents, configs) +- Distributed schedulers (ShedLock + Redis) +- Mongo migrations (Mongock) +- Background health tasks + +### Operational Flow ```mermaid flowchart TD - DebeziumEvent --> Deserializer - Deserializer --> Enrichment - Enrichment --> EventTypeMapper - EventTypeMapper --> Cassandra - EventTypeMapper --> KafkaOut + Startup --> Initializers + Initializers --> Data + Scheduler --> Redis["ShedLock (Redis)"] + Scheduler --> Data + Scheduler --> Stream ``` -πŸ“„ Module: `stream-service-core-kafka-and-handlers` +πŸ“˜ See module documentation: **Management Service Core** --- -## 6. Data Layer +## 6. `data-models-and-repositories-mongo` -### MongoDB +**Purpose:** +MongoDB persistence backbone. -- Domain documents (`User`, `Organization`, `Device`, `Ticket`, etc.) -- Base repositories (sync + reactive) -- Custom query repositories -- Cursor pagination logic -- Index configuration +Includes: -πŸ“„ Modules: -- `data-mongo-domain-model` -- `data-mongo-query-filters` -- `data-mongo-base-repositories` -- `data-mongo-sync-config-and-custom-repositories` -- `data-mongo-reactive-repositories` +- Core documents (User, Device, Ticket, Organization, Event, Notification) +- QueryFilter objects +- Reactive + Sync repositories +- Custom MongoTemplate implementations +- Cursor pagination +- Aggregation metrics ---- +### Data Layer Architecture -### Redis +```mermaid +flowchart TD + Services --> Repositories + Repositories --> CustomImpl["Custom Repository"] + CustomImpl --> MongoTemplate + MongoTemplate --> MongoDB +``` -- Tenant-aware cache key prefixing -- Spring Cache integration -- Reactive + blocking templates +Multi-tenant isolation enforced via `tenantId` field. -πŸ“„ Module: `data-redis-cache` +πŸ“˜ See module documentation: **Data Models And Repositories Mongo** --- -### Kafka +## 7. `stream-processing-kafka` -- Tenant-scoped Kafka configuration -- Topic auto-provisioning -- Debezium message modeling -- Retry & recovery handling +**Purpose:** +Event ingestion & normalization pipeline. -πŸ“„ Module: `data-kafka-configuration-and-retry` +Consumes: ---- - -### NATS (Real-Time Notifications) - -- Persist-first notification strategy -- Read-state tracking -- Per-recipient NATS publish -- Graceful degradation if NATS disabled +- Debezium CDC events +- MeshCentral +- Tactical RMM +- Fleet MDM -πŸ“„ Module: `data-nats-notifications` - ---- +Performs: -### Pinot (Analytics) +- Tool-specific deserialization +- Event normalization +- Tenant resolution +- Kafka Streams enrichment +- Cassandra persistence -- Log exploration -- Device faceted filtering -- High-performance aggregation queries +### Event Pipeline ```mermaid -flowchart LR - StreamService --> PinotCluster - PinotCluster --> PinotRepositories - PinotRepositories --> ApiService +flowchart TD + Tools --> Debezium + Debezium --> KafkaTopic + KafkaTopic --> Listener + Listener --> Deserializer + Deserializer --> Enrichment + Enrichment --> Mapper + Mapper --> Handler + Handler --> Cassandra + Handler --> OutboundKafka ``` -πŸ“„ Module: `data-pinot-repositories` +πŸ“˜ See module documentation: **Stream Processing Kafka** --- -## 7. Management Service Core - -Operational backbone of the platform. +## 8. `frontend-core-ui-and-chat` -### Startup Initializers +**Purpose:** +Reusable UI foundation + AI Chat engine. -- Agent registration secret bootstrap -- NATS stream provisioning -- Client configuration loading -- Tactical RMM script initialization +Includes: -### Distributed Schedulers (ShedLock + Redis) +- Embeddable AI assistant (Guide + Mingo modes) +- Ticket Center +- Kanban Board +- Notifications Drawer +- Data Tables +- Runtime-driven navigation -- Offline device detection -- API key stats sync -- Fleet MDM setup -- Version publish fallback +### Chat Architecture ```mermaid flowchart TD - Startup --> Initializers - Initializers --> StreamsReady - StreamsReady --> Schedulers - Schedulers --> ExternalSystems + EmbeddableChat --> UnifiedChat + UnifiedChat --> GuideMode["SSE Mode"] + UnifiedChat --> MingoMode["NATS Mode"] + GuideMode --> API + MingoMode --> Stream ``` -πŸ“„ Module: `management-service-core-initializers-and-schedulers` - ---- +Designed for: -## 8. External API Service Core +- Platform embedding +- Runtime injection +- Transport abstraction +- Segment-based streaming messages -Public REST interface for third-party integrations. - -- API key–based authentication -- Cursor-based pagination -- Device / Event / Log / Organization APIs -- Tool proxy endpoints -- OpenAPI documentation - -πŸ“„ Module: `external-api-service-core` +πŸ“˜ See module documentation: **Frontend Core UI And Chat** --- -## 9. Security Core & OAuth BFF - -Frontend-safe OAuth2 BFF layer. +# End-to-End Request Flow -- PKCE utilities -- JWT encoder/decoder -- OAuth login flow -- Token refresh & revocation -- HttpOnly cookie management +Below is a full lifecycle example: user logs in, performs an action, and event propagates. ```mermaid sequenceDiagram - participant Browser - participant BFF - participant AuthServer - - Browser->>BFF: GET /oauth/login - BFF->>AuthServer: Redirect with PKCE - AuthServer->>BFF: Callback with code - BFF->>AuthServer: Exchange code - BFF->>Browser: Set cookies + participant User + participant Gateway + participant Auth + participant API + participant Stream + participant Data + + User->>Gateway: Login Request + Gateway->>Auth: OAuth2 Flow + Auth-->>Gateway: JWT + Gateway->>API: Authenticated Request + API->>Data: CRUD Operation + API->>Stream: Emit Event + Stream->>Data: Persist Unified Event ``` -πŸ“„ Module: `security-core-and-oauth-bff` - --- -# End-to-End System View - -```mermaid -flowchart TD - Client --> Gateway - Gateway --> ExternalAPI - Gateway --> ApiCore - - ApiCore --> Authz - ApiCore --> Mongo - ApiCore --> Redis - ApiCore --> Pinot +# Core Design Principles - Authz --> Mongo - Authz --> JWT - - Stream --> Kafka - Stream --> Cassandra - Stream --> Pinot - - Management --> Redis - Management --> NATS -``` +1. **Tenant-first architecture** +2. **Strict module separation** +3. **Reactive edge + streaming core** +4. **Cursor-based pagination** +5. **Tool-agnostic event normalization** +6. **Extensible processors and hooks** +7. **Shared contract spine** --- -# Design Principles - -The repository follows consistent architectural principles: - -1. **Multi-Tenancy First** - - Tenant ID embedded in JWT - - Tenant-scoped keys and locks - - Tenant-aware caching - -2. **Separation of Concerns** - - Domain model isolated from transport - - Query filters separate from API DTOs - - Gateway isolated from business logic +# How Everything Fits Together -3. **Polyglot Persistence** - - MongoDB β†’ transactional - - Pinot β†’ analytical - - Redis β†’ caching - - Kafka β†’ streaming - - NATS β†’ real-time +```mermaid +flowchart LR + Frontend + Gateway + Auth + API + Management + Data + Stream + + Frontend --> Gateway + Gateway --> Auth + Gateway --> API + API --> Data + API --> Stream + Stream --> Data + Management --> Data + Management --> Stream +``` -4. **Event-Driven Architecture** - - Debezium ingestion - - Kafka normalization - - Unified event model +The repository forms: -5. **Extensibility** - - Conditional beans - - Processor hooks - - Strategy patterns - - Pluggable resolvers +- πŸ” Identity & trust layer (Authorization) +- 🌐 Edge & routing layer (Gateway) +- 🧠 Application API layer (API Core) +- πŸ—„ Data persistence layer (Mongo) +- πŸ”„ Event processing layer (Kafka) +- βš™ Operational control plane (Management) +- πŸ’¬ UI & AI interaction layer (Frontend Core) --- # Conclusion -The **openframe-oss-lib** repository is the foundational backend library stack of OpenFrame. It provides: +The **`openframe-oss-lib`** repository is the complete modular backbone of OpenFrame’s open-source MSP platform. + +It delivers: -- A multi-tenant OAuth2 authorization server -- Internal and external API layers -- A reactive gateway -- Event streaming and normalization -- Analytics querying -- Distributed management workflows -- Secure OAuth BFF flows -- Polyglot data infrastructure +- Multi-tenant security +- Unified API contracts +- Reactive gateway & API +- Scalable Mongo persistence +- Event-driven stream processing +- Distributed operational control +- Embeddable AI-powered UI -Together, these modules form a production-grade, scalable, multi-tenant backend architecture that powers the OpenFrame platform end-to-end. \ No newline at end of file +Together, these modules form a production-grade, extensible, and tenant-safe platform for modern MSP automation. \ No newline at end of file diff --git a/docs/reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md b/docs/reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md deleted file mode 100644 index 8072b0290..000000000 --- a/docs/reference/architecture/api-contracts-and-pagination/api-contracts-and-pagination.md +++ /dev/null @@ -1,329 +0,0 @@ -# Api Contracts And Pagination - -## Overview - -The **Api Contracts And Pagination** module defines the foundational API data contracts used across OpenFrame services, with a strong focus on: - -- Relay-style cursor-based pagination -- Count-aware query results -- Opaque cursor encoding and decoding -- Standardized mutation input structures - -This module sits at the boundary between transport layers (REST and GraphQL) and domain/query services. It ensures consistent, predictable API behavior across the platform while hiding internal persistence details (such as MongoDB identifiers or composite keys) from API consumers. - ---- - -## Core Responsibilities - -The Api Contracts And Pagination module provides: - -1. **Generic query result wrappers** with total/filtered counts -2. **Relay-compliant pagination arguments** -3. **Opaque cursor encoding/decoding utilities** -4. **Standard mutation input contracts (e.g., delete by ID)** - -These contracts are reused by: - -- GraphQL data fetchers -- REST controllers -- Domain query services -- MongoDB custom repositories - ---- - -## Architectural Positioning - -The module acts as a shared contract layer between API entry points and data/query layers. - -```mermaid -flowchart TD - Client["Client Application"] --> API["REST Controllers / GraphQL DataFetchers"] - API --> Contracts["Api Contracts And Pagination"] - Contracts --> Services["Domain Query Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("MongoDB")] -``` - -### Key Characteristics - -- **Transport-agnostic**: Works with both REST and GraphQL. -- **Persistence-agnostic**: Does not expose internal database identifiers. -- **Validation-aware**: Uses Jakarta validation annotations for input constraints. -- **Relay-compliant**: Follows the GraphQL Relay Connection specification patterns. - ---- - -# Core Components - -## 1. CountedGenericQueryResult - -**Class:** `CountedGenericQueryResult` -**Extends:** `GenericQueryResult` - -### Purpose - -Adds a `filteredCount` field to a generic query result, allowing APIs to return: - -- The current page of results -- Total elements (from base class) -- Filtered result count (after applying search/filter criteria) - -### Structure - -```text -CountedGenericQueryResult - β”œβ”€ List results - β”œβ”€ int totalCount - └─ int filteredCount -``` - -### Why It Matters - -This structure enables: - -- Efficient UI pagination -- Accurate result counts after filtering -- Consistent response shape across domains (devices, events, users, etc.) - -### Example Usage Flow - -```mermaid -flowchart LR - Request["Query Request with Filters"] --> Service["Query Service"] - Service --> Repo["Custom Repository"] - Repo --> Result["CountedGenericQueryResult"] - Result --> API["Serialized API Response"] -``` - -The `filteredCount` is especially important when: - -- Full dataset size is large -- Filters significantly reduce results -- UI must display: "Showing 10 of 245 filtered results" - ---- - -## 2. ConnectionArgs - -**Class:** `ConnectionArgs` - -### Purpose - -Defines Relay-style pagination arguments used primarily in GraphQL connections. - -Supports: - -- Forward pagination: `first` + `after` -- Backward pagination: `last` + `before` - -### Field Definitions - -```text -ConnectionArgs - β”œβ”€ Integer first (min 1, max 100) - β”œβ”€ String after - β”œβ”€ Integer last (min 1, max 100) - └─ String before -``` - -### Validation Rules - -- `first` and `last` must be between 1 and 100. -- Validation is enforced via Jakarta annotations. -- Prevents excessive data loading and abuse. - -### Pagination Model - -```mermaid -flowchart TD - Client["Client"] --> Args["ConnectionArgs"] - Args --> Decode["CursorCodec.decode()"] - Decode --> Query["Repository Query with Limit"] - Query --> Encode["CursorCodec.encode()"] - Encode --> Response["Connection Response with Edges"] -``` - -### Design Principles - -- Hard limit (100) protects backend resources. -- Supports bi-directional pagination. -- Compatible with Relay Connection pattern. - ---- - -## 3. CursorCodec - -**Class:** `CursorCodec` - -### Purpose - -Encodes and decodes opaque pagination cursors using Base64. - -This ensures that internal cursor values such as: - -- MongoDB ObjectIds -- Composite keys (e.g., "timestamp_eventId") -- Internal database offsets - -are not exposed directly to API consumers. - -### Encoding Flow - -```mermaid -flowchart LR - Raw["Raw Internal Cursor"] --> Encode["Base64 Encode"] - Encode --> Opaque["Opaque Cursor String"] -``` - -### Decoding Flow - -```mermaid -flowchart LR - Opaque["Opaque Cursor"] --> Decode["Base64 Decode"] - Decode --> Raw["Raw Internal Cursor"] -``` - -### Key Behaviors - -- Returns `null` for null or empty input. -- Gracefully handles invalid Base64 input. -- Hides implementation details from clients. - -### Example - -```text -Raw Cursor: 665fae2b3a91c87b12345678 -Encoded: NjY1ZmFlMmIzYTkxYzg3YjEyMzQ1Njc4 -``` - -This pattern guarantees that clients cannot infer database structure or ordering logic. - ---- - -## 4. MutationDeleteInput - -**Class:** `MutationDeleteInput` - -### Purpose - -Defines a standard mutation input structure for delete operations. - -### Structure - -```text -MutationDeleteInput - └─ String id (must not be blank) -``` - -### Design Rationale - -- Enforces explicit ID-based deletion. -- Aligns with GraphQL mutation input patterns. -- Supports validation via `@NotBlank`. - -### Mutation Flow - -```mermaid -flowchart TD - Client["Client Mutation"] --> Input["MutationDeleteInput"] - Input --> Validate["Validation Layer"] - Validate --> Service["Domain Service"] - Service --> Repo["Repository Delete by ID"] - Repo --> Response["Mutation Result"] -``` - ---- - -# Pagination Strategy in the Platform - -The Api Contracts And Pagination module supports two complementary patterns: - -## 1. Cursor-Based Pagination (GraphQL) - -Used for connection-based queries: - -- Forward and backward navigation -- Opaque cursors -- Relay compliance - -Benefits: - -- Stable pagination even with concurrent inserts -- No reliance on numeric offsets -- Efficient indexed queries - -## 2. Counted Query Results (REST or Hybrid) - -Used where: - -- UI requires total/filtered counts -- Offset-based or filtered queries are needed -- Simpler REST-style pagination is sufficient - ---- - -# End-to-End Data Flow Example - -Below is a complete pagination lifecycle example. - -```mermaid -flowchart TD - Step1["Client Requests first 20 after Cursor"] --> Step2["ConnectionArgs Validated"] - Step2 --> Step3["CursorCodec.decode()"] - Step3 --> Step4["Repository Query with Limit 21"] - Step4 --> Step5["Determine hasNextPage"] - Step5 --> Step6["CursorCodec.encode() for Edges"] - Step6 --> Step7["Return Connection Response"] -``` - -### Why Fetch 21 for a Limit of 20? - -To determine if: - -- There is a next page -- `hasNextPage` should be true - -This pattern ensures efficient and correct pagination semantics. - ---- - -# Design Principles - -The Api Contracts And Pagination module follows these core principles: - -1. **Consistency Across Domains** - Devices, events, users, organizations, and tools all use uniform contracts. - -2. **Opaque by Default** - Clients never see internal IDs or composite keys. - -3. **Validation at the Edge** - Input constraints are enforced before hitting service layers. - -4. **Resource Safety** - Pagination limits prevent unbounded queries. - -5. **Transport Layer Neutrality** - Usable by both REST controllers and GraphQL data fetchers. - ---- - -# Summary - -The **Api Contracts And Pagination** module is a foundational contract layer for OpenFrame APIs. - -It standardizes: - -- How lists are queried -- How pagination works -- How cursors are encoded -- How deletions are requested - -By centralizing these patterns, the platform achieves: - -- Predictable API behavior -- Secure data exposure boundaries -- Efficient pagination at scale -- Strong validation guarantees - -This module underpins consistent API design across the entire OpenFrame ecosystem. \ No newline at end of file diff --git a/docs/reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md b/docs/reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md deleted file mode 100644 index 9cead9cef..000000000 --- a/docs/reference/architecture/api-domain-filters-dtos/api-domain-filters-dtos.md +++ /dev/null @@ -1,366 +0,0 @@ -# Api Domain Filters Dtos - -The **Api Domain Filters Dtos** module defines the domain-level Data Transfer Objects (DTOs) used to: - -- Express filtering criteria for core entities (Devices, Events, Logs, Knowledge Base, Organizations, Tools) -- Provide structured filter metadata for UI dropdowns and faceted search -- Standardize list and response payloads shared across REST and GraphQL layers - -This module acts as a **contract layer** between: - -- GraphQL DataFetchers and REST Controllers -- Core services and query layers -- Mongo query filter builders and repositories - -It does not contain business logic. Instead, it provides strongly typed filter contracts that drive query construction in downstream modules. - ---- - -## Architectural Role - -At a high level, Api Domain Filters Dtos sits between API input DTOs and persistence query filters. - -```mermaid -flowchart LR - Client["Client (REST or GraphQL)"] --> ApiInput["GraphQL Input / REST Request"] - ApiInput --> DomainFilter["Api Domain Filters Dtos"] - DomainFilter --> QueryLayer["Mongo Query Filters"] - QueryLayer --> Repository["Mongo Repositories"] - Repository --> Database[("MongoDB")] -``` - -### Responsibilities - -1. **Normalize filter structures** across APIs -2. **Encapsulate filtering intent** in strongly typed criteria classes -3. **Support UI filter metadata responses** (counts, labels, selectable options) -4. **Bridge domain enums and database query filters** - ---- - -# Core Filter Patterns - -Across entities, this module follows consistent patterns: - -## 1. *FilterCriteria Classes* - -These represent the filtering rules submitted by the client. - -Examples: - -- `LogFilterCriteria` -- `DeviceFilterCriteria` -- `EventFilterCriteria` -- `KnowledgeBaseFilterCriteria` -- `ToolFilterCriteria` - -These classes typically include: - -- Date ranges (`startDate`, `endDate`) -- Entity type filters -- Status filters -- Organization scoping -- Tag-based filtering - -They are consumed by service/query layers to build Mongo queries. - ---- - -## 2. *Filters Classes (Faceted Responses)* - -These classes provide filter metadata for UI dropdowns and dashboards. - -Examples: - -- `LogFilters` -- `DeviceFilters` -- `EventFilters` -- `ToolFilters` - -They usually contain: - -- Available filter values -- Display labels -- Optional counts per filter value - -These enable dynamic filtering experiences (faceted search). - ---- - -## 3. *List & Response DTOs* - -Used to standardize list responses and entity projections: - -- `OrganizationList` -- `ToolList` -- `OrganizationResponse` - -These are shared between: - -- GraphQL DTO layer -- REST external API layer - ---- - -# Entity-Specific Filter Structures - -## Log Filtering - -### LogFilterCriteria - -Encapsulates audit log filtering rules: - -- `startDate` / `endDate` -- `eventTypes` -- `toolTypes` -- `severities` -- `organizationIds` -- `deviceId` - -```mermaid -flowchart TD - LogFilterCriteria["LogFilterCriteria"] --> DateRange["Date Range"] - LogFilterCriteria --> EventTypes["Event Types"] - LogFilterCriteria --> ToolTypes["Tool Types"] - LogFilterCriteria --> Severities["Severities"] - LogFilterCriteria --> Organizations["Organization IDs"] - LogFilterCriteria --> Device["Device ID"] -``` - -### LogFilters - -Provides available values for: - -- Tool types -- Event types -- Severities -- Organizations (via `OrganizationFilterOption`) - -### OrganizationFilterOption - -Represents a selectable organization option: - -- `id` -- `name` - -Used primarily for UI dropdown population. - ---- - -## Device Filtering - -### DeviceFilterCriteria - -Supports multi-dimensional filtering: - -- `statuses` (DeviceStatus enum) -- `deviceTypes` (DeviceType enum) -- `osTypes` -- `organizationIds` -- `tagKeys` -- `tagValues` - -```mermaid -flowchart TD - DeviceFilterCriteria["DeviceFilterCriteria"] --> Statuses["Device Statuses"] - DeviceFilterCriteria --> Types["Device Types"] - DeviceFilterCriteria --> OsTypes["OS Types"] - DeviceFilterCriteria --> Orgs["Organization IDs"] - DeviceFilterCriteria --> Tags["Tag Keys / Values"] -``` - -### DeviceFilters - -Faceted filter response containing: - -- `statuses` -- `deviceTypes` -- `osTypes` -- `organizationIds` -- `tagKeys` -- `filteredCount` - -Each option is represented by: - -### DeviceFilterOption - -- `value` -- `label` -- `count` - -### TagFilterOption - -- `key` -- `value` -- `count` - -This enables advanced tag-based device filtering. - ---- - -## Event Filtering - -### EventFilterCriteria - -Filters system/user events by: - -- `userIds` -- `eventTypes` -- `startDate` -- `endDate` - -### EventFilters - -Returns available: - -- User IDs -- Event types - -This supports activity feed filtering and auditing dashboards. - ---- - -## Knowledge Base Filtering - -### KnowledgeBaseFilterCriteria - -Used to query hierarchical content: - -- `parentId` -- `type` (KnowledgeBaseItemType enum) -- `tagIds` -- `statuses` (KnowledgeBaseArticleStatus enum) - -This enables: - -- Folder-level filtering -- Tag-based content queries -- Status-based moderation filtering - ---- - -## Organization Filtering & Responses - -### OrganizationFilterOptions - -Internal filtering structure for: - -- `category` -- `minEmployees` -- `maxEmployees` -- `hasActiveContract` -- `status` - -### OrganizationList - -Wrapper for returning multiple Organization documents. - -### OrganizationResponse - -Shared response DTO across GraphQL and REST. - -Contains: - -- Identity fields (`id`, `organizationId`) -- Business metadata -- Revenue and contract information -- Lifecycle timestamps -- Status fields - -```mermaid -flowchart LR - Organization["Organization (Mongo Document)"] --> OrganizationResponse["OrganizationResponse DTO"] - OrganizationResponse --> RestAPI["REST API"] - OrganizationResponse --> GraphQL["GraphQL API"] -``` - ---- - -## Tool Filtering - -### ToolFilterCriteria - -Filters integrated tools by: - -- `enabled` -- `type` -- `category` -- `platformCategory` - -### ToolFilters - -Provides available: - -- Tool types -- Categories -- Platform categories - -### ToolList - -Wrapper for returning a list of `IntegratedTool` domain documents. - ---- - -# Interaction with Other Layers - -## With GraphQL Layer - -GraphQL input DTOs are transformed into FilterCriteria objects, which are then passed to service/query layers. - -```mermaid -flowchart LR - GraphQLInput["GraphQL Filter Input"] --> Criteria["FilterCriteria DTO"] - Criteria --> Service["Query Service"] - Service --> MongoFilter["Mongo Query Filter"] -``` - -## With Mongo Query Filters - -Each FilterCriteria maps conceptually to a corresponding Mongo query filter class in the query layer. - -Example conceptual mapping: - -```mermaid -flowchart TD - DeviceFilterCriteria --> DeviceQueryFilter["DeviceQueryFilter"] - EventFilterCriteria --> EventQueryFilter["EventQueryFilter"] - ToolFilterCriteria --> ToolQueryFilter["ToolQueryFilter"] -``` - -The DTO layer remains persistence-agnostic, while query modules handle database-specific logic. - ---- - -# Design Principles - -The Api Domain Filters Dtos module follows these architectural principles: - -1. **Separation of Concerns** - DTOs describe intent; services implement logic. - -2. **Strong Typing** - Uses enums and structured lists instead of raw maps. - -3. **UI-Driven Design** - Includes filter option and count DTOs for faceted filtering. - -4. **Cross-API Reusability** - Shared response DTOs prevent duplication across REST and GraphQL. - -5. **Extensibility** - New filter dimensions can be added without impacting existing consumers. - ---- - -# Summary - -The **Api Domain Filters Dtos** module provides the structured filtering and response contracts that power: - -- Device management -- Audit logs -- Event tracking -- Knowledge base content -- Organization management -- Tool integrations - -It forms the backbone of query expressiveness across the OpenFrame API stack, enabling consistent filtering semantics across services, APIs, and persistence layers. \ No newline at end of file diff --git a/docs/reference/architecture/api-lib-contracts/api-lib-contracts.md b/docs/reference/architecture/api-lib-contracts/api-lib-contracts.md new file mode 100644 index 000000000..fa2f21a75 --- /dev/null +++ b/docs/reference/architecture/api-lib-contracts/api-lib-contracts.md @@ -0,0 +1,455 @@ +# Api Lib Contracts + +## Overview + +Api Lib Contracts is the shared contract module for the OpenFrame backend. It defines: + +- Data Transfer Objects (DTOs) for queries, mutations, filters, and responses +- Shared pagination primitives (Relay-style connections and cursors) +- Command input and dispatch contracts +- Filter criteria models for devices, logs, events, tools, scripts, and organizations +- Cross-layer mappers (e.g., OrganizationMapper) + +This module contains **no transport logic** (no controllers, no GraphQL resolvers, no persistence). Instead, it acts as the stable contract layer between: + +- HTTP + GraphQL services +- Management and stream-processing services +- Mongo data models and repositories +- Frontend clients + +By centralizing contracts here, OpenFrame ensures type consistency and reduces duplication across services. + +--- + +## Architectural Role in the System + +```mermaid +flowchart LR + Frontend["Frontend UI and Chat"] -->|"GraphQL / REST"| ApiService["API Service Core"] + ApiService -->|"Uses DTOs"| Contracts["Api Lib Contracts"] + Contracts -->|"Maps to"| DataModels["Mongo Data Models"] + ApiService -->|"Queries"| DataModels + Stream["Stream Processing Kafka"] -->|"Publishes Events"| DataModels + Gateway["Gateway Service Core"] -->|"Secured Requests"| ApiService + Auth["Authorization Service Core"] -->|"JWT / Tenant Context"| ApiService +``` + +### Key Responsibilities + +1. **Defines API boundaries** (inputs and outputs). +2. **Decouples transport from persistence**. +3. **Enforces validation constraints** using Jakarta Validation. +4. **Implements Relay-style pagination primitives**. +5. **Provides shared mapping utilities**. + +--- + +# Core Contract Categories + +## 1. Generic Query & Pagination Contracts + +### CountedGenericQueryResult + +`CountedGenericQueryResult` extends a generic query result and adds: + +- `filteredCount` – total number of items after filters are applied + +This is typically used in filtered list endpoints where: + +- The UI needs total counts +- Filtering and pagination occur simultaneously + +### Relay Pagination Support + +#### ConnectionArgs + +Encapsulates forward and backward pagination arguments: + +- `first` + `after` +- `last` + `before` + +Validation constraints: + +- Minimum: 1 +- Maximum: 100 + +#### CursorCodec + +Utility for encoding and decoding opaque cursors. + +```text +Raw cursor (e.g. "timestamp_eventId") + ↓ Base64 encode +Opaque cursor returned to client + ↓ Base64 decode +Raw internal cursor +``` + +This ensures: + +- Internal identifiers remain hidden +- Pagination logic can evolve without breaking clients + +#### MutationDeleteInput + +Standard delete input wrapper: + +- `id` (required) + +Used for GraphQL mutation patterns requiring explicit input objects. + +--- + +## 2. Device Filtering Contracts + +### DeviceFilterCriteria + +Defines server-side filtering logic: + +- Device statuses +- Device types +- OS types +- Organization IDs +- Tag key/value filtering + +### DeviceFilterOption & TagFilterOption + +Used for UI filter dropdowns: + +- `value` / `label` +- `count` + +### DeviceFilters + +Aggregated filter response model containing: + +- Status options +- Device type options +- OS type options +- Organization options +- Tag keys +- `filteredCount` + +```mermaid +flowchart TD + Client["Client Filters"] --> Criteria["DeviceFilterCriteria"] + Criteria --> Service["Service Layer"] + Service --> Repo["Mongo Repository"] + Repo --> Results["Filtered Devices"] + Results --> Filters["DeviceFilters Response"] +``` + +--- + +## 3. Audit & Event Filtering Contracts + +### LogFilterCriteria + +Supports advanced audit filtering: + +- Date range +- Event types +- Tool types +- Severities +- Organization IDs +- Device ID + +### LogFilters + +Provides UI-ready filter options: + +- Tool types +- Event types +- Severities +- Organization dropdown options + +### EventFilterCriteria & EventFilters + +Used for querying core event documents: + +- User IDs +- Event types +- Date ranges + +These contracts map closely to Mongo query filters in the data layer. + +--- + +## 4. Command Execution Contracts + +### RunCommandInput + +GraphQL input for dispatching ad-hoc commands to agents: + +- `machineId` +- `command` +- `shell` +- `privilegeLevel` +- `timeoutSeconds` + +Validation ensures: + +- Required fields are present +- Timeout is positive + +### CommandDispatchResponse + +Returns: + +- `executionId` + +### CancelExecutionInput & CancelDispatchResponse + +Used to cancel agent executions. + +```mermaid +sequenceDiagram + participant Client + participant ApiService as "API Service" + participant Agent + + Client->>ApiService: RunCommandInput + ApiService->>Agent: Dispatch command + Agent-->>ApiService: executionId + ApiService-->>Client: CommandDispatchResponse +``` + +--- + +## 5. Knowledge Base Contracts + +### CreateArticleCommand + +Defines article creation fields: + +- Name, content, summary +- Parent hierarchy +- Status +- Tag assignments +- Organization / device / ticket links + +### UpdateArticleCommand + +Partial update for article metadata and content. + +### KnowledgeBaseFilterCriteria + +Filtering by: + +- Parent +- Type +- Tags +- Statuses + +### KnowledgeBaseAttachmentUpload + +Returns: + +- Attachment metadata +- Pre-signed upload URL + +--- + +## 6. Script Management Contracts + +### CreateScriptInput + +Defines script creation fields: + +- Name +- Shell +- Script body +- Supported platforms +- Default timeout +- Default arguments +- Environment variables + +### ScriptEnvVarInput + +Symmetric DTO used for: + +- Input +- Output + +Contains: + +- Name +- Value +- `secret` flag + +### ScriptFilterInput + +Supports filtering by: + +- Shells +- Statuses +- Platforms +- Tag + +### UpdateScriptInput + +Implements full-replacement semantics (PUT-style): + +- All fields must be provided +- Null clears optional fields + +### ScriptResponse + +API-facing representation: + +- Omits tenantId +- Serializes enums as names +- Includes lifecycle metadata + +--- + +## 7. Organization Contracts & Mapping + +### OrganizationResponse + +Shared response model for both GraphQL and REST. + +Contains: + +- Business metadata +- Financial metadata +- Contract dates +- Contact information +- Status and lifecycle fields + +### OrganizationList + +Wrapper for returning multiple organizations. + +### OrganizationFilterOptions + +Internal filtering metadata: + +- Category +- Employee range +- Contract status + +### OrganizationMapper + +A Spring component responsible for: + +- Mapping create requests to entities +- Performing partial updates +- Converting entities to OrganizationResponse +- Mapping nested contact information + +```mermaid +flowchart TD + Request["Create / Update Request"] --> Mapper["OrganizationMapper"] + Mapper --> Entity["Organization Entity"] + Entity --> Mapper + Mapper --> Response["OrganizationResponse"] +``` + +Key design decisions: + +- `organizationId` is generated as UUID +- Immutable once created +- Partial updates ignore null fields +- Mailing address may mirror physical address + +--- + +## 8. Tool Filtering Contracts + +### ToolFilterCriteria + +Server-side filtering fields: + +- Enabled +- Type +- Category +- Platform category + +### ToolFilters + +UI-oriented aggregated filter options. + +### ToolList + +Wrapper returning: + +- List of integrated tool entities + +--- + +# Design Principles + +## 1. Separation of Concerns + +- DTOs contain no persistence logic +- No repository or controller code +- No service orchestration + +This ensures portability across services. + +## 2. Validation at the Boundary + +Jakarta validation annotations enforce: + +- Required fields +- Numeric constraints +- Input integrity + +Transport layers trigger validation automatically. + +## 3. Tenant Safety + +Certain fields (e.g., tenantId) are intentionally excluded from API contracts. + +Tenant context is derived from authentication layers, not from client input. + +## 4. Opaque Pagination + +CursorCodec ensures: + +- Internal identifiers remain hidden +- Backward-compatible pagination evolution + +--- + +# How Api Lib Contracts Fits Into OpenFrame + +```mermaid +flowchart TD + Contracts["Api Lib Contracts"] + Api["API Service Core"] + Auth["Authorization Service"] + Data["Mongo Data Layer"] + Gateway["Gateway Service"] + Frontend["Frontend UI"] + + Frontend --> Gateway + Gateway --> Auth + Gateway --> Api + Api --> Contracts + Api --> Data +``` + +Api Lib Contracts is the **contract spine** of the backend. It: + +- Standardizes API inputs and outputs +- Ensures consistent filter and pagination behavior +- Enables both REST and GraphQL to share models +- Decouples transport, business logic, and persistence + +Without this module, every service would redefine overlapping DTOs, increasing coupling and breaking consistency. + +--- + +# Summary + +Api Lib Contracts is a foundational module that: + +- Defines all core API DTOs +- Implements filter and pagination primitives +- Supports device, event, audit, organization, script, tool, and knowledge base domains +- Provides shared mapping utilities +- Enforces validation at API boundaries + +It is intentionally lightweight but architecturally critical β€” serving as the shared language between OpenFrame services and clients. \ No newline at end of file diff --git a/docs/reference/architecture/api-lib-core-services/api-lib-core-services.md b/docs/reference/architecture/api-lib-core-services/api-lib-core-services.md deleted file mode 100644 index f06349115..000000000 --- a/docs/reference/architecture/api-lib-core-services/api-lib-core-services.md +++ /dev/null @@ -1,298 +0,0 @@ -# Api Lib Core Services - -## Overview - -The **Api Lib Core Services** module provides the core business service layer for the OpenFrame API library. It sits between the API-facing layers (REST controllers and GraphQL data fetchers) and the persistence layer (Mongo repositories and domain documents). - -This module encapsulates reusable domain logic related to: - -- Installed agents per machine -- Tool connections and integrations -- Ticket querying and search -- Knowledge base publication lifecycle -- Device status post-processing - -It is designed to be: - -- βœ… Framework-aligned (Spring Boot, Spring Data) -- βœ… Multi-tenant ready (delegating to repository-level filters) -- βœ… GraphQL DataLoader-friendly (batch-oriented methods) -- βœ… Extensible via conditional beans and processors - ---- - -## Architectural Position - -The Api Lib Core Services module acts as a **pure service layer** within the larger OpenFrame architecture. - -```mermaid -flowchart TD - Controllers["REST Controllers"] --> Services["Api Lib Core Services"] - DataFetchers["GraphQL Data Fetchers"] --> Services - Services --> Repositories["Mongo Repositories"] - Repositories --> MongoDB[("MongoDB")] - - Services --> Processors["Domain Processors"] -``` - -### Upstream Consumers - -- REST Controllers (API Service Core) -- GraphQL Data Fetchers -- DataLoaders (batch loading layer) - -### Downstream Dependencies - -- Mongo domain documents (Machine, Ticket, ToolConnection, InstalledAgent) -- Spring Data repositories -- Mongo event lifecycle hooks - -The module intentionally contains **no HTTP or GraphQL annotations**, ensuring reusability across different API surfaces. - ---- - -## Core Components - -### 1. InstalledAgentService - -**Purpose:** -Provides read access and batch-loading logic for installed agents across machines. - -**Key Capabilities:** - -- Fetch installed agents for a single machine -- Fetch installed agents for multiple machines (DataLoader optimized) -- Lookup by ID -- Lookup by machine and agent type - -```mermaid -flowchart LR - DataLoader["InstalledAgentDataLoader"] --> Service["InstalledAgentService"] - Service --> Repo["InstalledAgentRepository"] - Repo --> Mongo[("MongoDB")] -``` - -#### Batch-Oriented Design - -The method: - -```java -getInstalledAgentsForMachines(List machineIds) -``` - -- Fetches all agents using `findByMachineIdIn` -- Groups them in-memory by machine ID -- Returns a list aligned with the original input order - -This structure avoids the **N+1 query problem** in GraphQL environments. - ---- - -### 2. ToolConnectionService - -**Purpose:** -Manages retrieval of tool connection data associated with machines. - -**Responsibilities:** - -- Fetch tool connections by ID -- Fetch tool connections per machine -- Batch-fetch tool connections for multiple machines - -```mermaid -flowchart LR - ToolDataLoader["ToolConnectionDataLoader"] --> ToolService["ToolConnectionService"] - ToolService --> ToolRepo["ToolConnectionRepository"] - ToolRepo --> Mongo[("MongoDB")] -``` - -Similar to InstalledAgentService, this service is optimized for batch resolution in GraphQL environments. - ---- - -### 3. TicketQueryService - -**Purpose:** -Encapsulates ticket search and retrieval logic with filtering and cursor-based pagination support. - -**Key Methods:** - -- `findById(String ticketId)` -- `searchTickets(TicketQueryFilter filter, String search, int limit)` - -```mermaid -flowchart TD - API["API Layer"] --> TicketService["TicketQueryService"] - TicketService --> QueryFilter["TicketQueryFilter"] - TicketService --> Repo["TicketRepository"] - Repo --> Mongo[("MongoDB")] -``` - -#### Query Construction Flow - -1. Build a `Query` object using: - - Structured filter (`TicketQueryFilter`) - - Free-text search -2. Delegate to repository method: - - `findTicketsWithCursor(...)` -3. Apply default sorting: - - Field: `createdAt` - - Direction: `DESC` - -This service centralizes ticket query semantics and ensures consistent sorting behavior. - ---- - -### 4. KnowledgeBasePublishLifecycleListener - -**Purpose:** -Automatically stamps `publishedAt` when a knowledge base article transitions into the `PUBLISHED` state for the first time. - -This is implemented as a: - -- Spring `@Component` -- Extends `AbstractMongoEventListener` -- Hooks into `onBeforeConvert` - -```mermaid -flowchart TD - Save["Save KnowledgeBaseItem"] --> MongoEvent["BeforeConvertEvent"] - MongoEvent --> Listener["KnowledgeBasePublishLifecycleListener"] - Listener --> Stamp["Set publishedAt if null"] - Stamp --> Persist[("MongoDB")] -``` - -### Semantic Guarantees - -- βœ… `publishedAt` is set only once -- βœ… Unpublish/republish cycles do not overwrite the original timestamp -- βœ… Aligns with Schema.org `datePublished` and Atom `published` - -This ensures historical integrity and stable audit semantics. - ---- - -### 5. DefaultDeviceStatusProcessor - -**Purpose:** -Provides a default implementation for device status post-processing logic. - -It implements `DeviceStatusProcessor` and is registered with: - -- `@Component` -- `@ConditionalOnMissingBean` - -```mermaid -flowchart LR - DeviceStatusUpdate["Machine Status Updated"] --> Processor["DeviceStatusProcessor"] - Processor --> DefaultImpl["DefaultDeviceStatusProcessor"] -``` - -### Extensibility Model - -Because it is conditionally registered, applications can: - -- Provide a custom `DeviceStatusProcessor` -- Override default behavior without modifying the core module - -The default implementation only logs status changes, making it safe and side-effect free. - ---- - -## Design Patterns Used - -### 1. Service Layer Pattern -Encapsulates domain logic and shields API layers from persistence details. - -### 2. DataLoader-Friendly Batch APIs -Both InstalledAgentService and ToolConnectionService provide: - -- `List>` aligned responses -- Deterministic ordering -- Single query per batch - -### 3. Repository Delegation -All query logic ultimately delegates to repositories responsible for: - -- Query construction -- Pagination mechanics -- Sorting logic - -### 4. Event-Driven Lifecycle Hooks -KnowledgeBasePublishLifecycleListener demonstrates: - -- Mongo lifecycle event integration -- Declarative business rules -- Immutable publication timestamp semantics - -### 5. Conditional Bean Extension -DefaultDeviceStatusProcessor enables pluggable domain behavior using Spring Boot auto-configuration conventions. - ---- - -## Data Flow Example: GraphQL Machine Query - -When querying machines with nested installed agents and tool connections: - -```mermaid -flowchart TD - Client["GraphQL Client"] --> DataFetcher["MachineDataFetcher"] - DataFetcher --> AgentLoader["InstalledAgentDataLoader"] - DataFetcher --> ToolLoader["ToolConnectionDataLoader"] - - AgentLoader --> AgentService["InstalledAgentService"] - ToolLoader --> ToolService["ToolConnectionService"] - - AgentService --> AgentRepo["InstalledAgentRepository"] - ToolService --> ToolRepo["ToolConnectionRepository"] - - AgentRepo --> Mongo[("MongoDB")] - ToolRepo --> Mongo -``` - -This ensures: - -- βœ… No N+1 queries -- βœ… Deterministic batch resolution -- βœ… Clean separation between data resolution and domain logic - ---- - -## Transaction and Consistency Model - -- Read-only services are annotated with `@Transactional(readOnly = true)` -- Lifecycle listeners operate before persistence conversion -- Repository layer enforces query-level consistency - -The module intentionally avoids complex transaction orchestration and instead relies on: - -- Repository atomic operations -- Mongo consistency guarantees - ---- - -## Extension Guidelines - -When extending Api Lib Core Services: - -1. βœ… Keep services stateless -2. βœ… Delegate persistence to repositories -3. βœ… Provide batch-friendly APIs when used by GraphQL -4. βœ… Use conditional beans for override points -5. βœ… Keep lifecycle logic idempotent - ---- - -## Summary - -The **Api Lib Core Services** module is the domain-centric service backbone of the OpenFrame API stack. - -It provides: - -- Clean separation of concerns -- Batch-optimized data access patterns -- Event-driven lifecycle management -- Extensible processing hooks -- Consistent query semantics - -By centralizing domain service logic in this module, the broader OpenFrame platform maintains a modular, scalable, and extensible API architecture. \ No newline at end of file diff --git a/docs/reference/architecture/api-organization-mapping/api-organization-mapping.md b/docs/reference/architecture/api-organization-mapping/api-organization-mapping.md deleted file mode 100644 index 86fb3e3d8..000000000 --- a/docs/reference/architecture/api-organization-mapping/api-organization-mapping.md +++ /dev/null @@ -1,322 +0,0 @@ -# Api Organization Mapping - -The **Api Organization Mapping** module is responsible for translating between external API data transfer objects (DTOs) and the internal persistence model for organizations. It acts as the boundary layer between: - -- REST and GraphQL API contracts -- Domain entities stored in MongoDB -- Service-layer business logic - -At its core is the `OrganizationMapper`, a Spring component that ensures consistent, safe, and predictable conversion between request objects, domain entities, and response DTOs. - ---- - -## 1. Purpose and Responsibilities - -The Api Organization Mapping module provides: - -- βœ… Conversion from API request DTOs to `Organization` entities -- βœ… Partial update (patch-style) entity updates -- βœ… Conversion from `Organization` entities to response DTOs -- βœ… Nested object mapping (contact information, addresses, contacts) -- βœ… Controlled immutability for system-critical fields (e.g., `organizationId`) - -It centralizes mapping logic to avoid duplication across: - -- REST controllers -- GraphQL data fetchers -- Service layer components - ---- - -## 2. Core Component - -### OrganizationMapper - -Location: - -```text -com.openframe.api.mapper.OrganizationMapper -``` - -Type: - -- Spring `@Component` -- Stateless mapper -- Shared across REST and GraphQL APIs - -### Primary Methods - -| Method | Purpose | -|--------|----------| -| `toEntity(CreateOrganizationRequest)` | Create new `Organization` entity | -| `updateEntity(Organization, UpdateOrganizationRequest)` | Partial update of entity | -| `toResponse(Organization)` | Convert entity to API response | - ---- - -## 3. High-Level Architecture - -The mapper sits between API contracts and the domain model. - -```mermaid -flowchart LR - subgraph ApiLayer["API Layer"] - RestController["OrganizationController"] - GraphQLFetcher["OrganizationDataFetcher"] - end - - Mapper["OrganizationMapper"] - - subgraph DomainLayer["Domain & Persistence"] - OrgEntity["Organization Entity"] - MongoRepo["Mongo Repository"] - end - - RestController -->|"Create/Update Request"| Mapper - GraphQLFetcher -->|"Mutation Input"| Mapper - - Mapper -->|"Entity"| OrgEntity - OrgEntity --> MongoRepo - - MongoRepo --> OrgEntity - OrgEntity -->|"Response DTO"| Mapper - Mapper -->|"OrganizationResponse"| RestController - Mapper -->|"GraphQL DTO"| GraphQLFetcher -``` - -### Related Modules - -- API contracts and DTO definitions: [Api Domain Filters Dtos](api-domain-filters-dtos.md) -- REST layer: [Api Service Core Rest Controllers](api-service-core-rest-controllers.md) -- GraphQL layer: [Api Service Core Graphql Datafetchers](api-service-core-graphql-datafetchers.md) -- Persistence model: [Data Mongo Domain Model](data-mongo-domain-model.md) - ---- - -## 4. Create Flow Mapping - -### 4.1 Entity Creation - -`toEntity(CreateOrganizationRequest request)`: - -- Generates a new `organizationId` using UUID -- Copies request fields into the entity -- Sets `isDefault` to `false` -- Converts nested contact structures - -```mermaid -flowchart TD - Request["CreateOrganizationRequest"] --> MapperCreate["toEntity()"] - MapperCreate --> GenId["Generate UUID"] - MapperCreate --> MapFields["Map Scalar Fields"] - MapperCreate --> MapContact["Map Contact Information"] - MapContact --> MapAddress["Map Address"] - MapContact --> MapContacts["Map Contact Persons"] - GenId --> OrgEntity["Organization Entity"] - MapFields --> OrgEntity - MapContact --> OrgEntity -``` - -### 4.2 UUID Generation - -The organization ID is: - -- Generated internally -- Immutable after creation -- Independent from MongoDB primary key (`id`) - -This ensures: - -- External-safe identifier exposure -- Decoupling from database implementation -- Stable public references - ---- - -## 5. Partial Update Strategy - -### 5.1 Patch-Style Updates - -`updateEntity(existing, request)`: - -- Only updates fields that are **non-null** in the request -- Leaves unspecified fields unchanged -- Does **not** allow updating `organizationId` - -```mermaid -flowchart TD - UpdateRequest["UpdateOrganizationRequest"] --> CheckField["Field != null?"] - CheckField -->|"Yes"| ApplyUpdate["Set Field on Entity"] - CheckField -->|"No"| Skip["Keep Existing Value"] - ApplyUpdate --> OrgEntity["Updated Organization"] - Skip --> OrgEntity -``` - -### 5.2 Immutability Rules - -The following field is intentionally immutable: - -- `organizationId` - -Reason: - -- Prevents identity mutation -- Protects referential integrity across services -- Ensures stable tenant-level references - ---- - -## 6. Response Mapping - -`toResponse(Organization organization)` converts the domain entity into `OrganizationResponse`. - -### 6.1 Field Transformations - -- `status` β†’ converted to string via `status.name()` -- `createdAt` and `updatedAt` preserved -- Nested contact information mapped to DTO equivalents - -```mermaid -flowchart LR - OrgEntity["Organization Entity"] --> MapperResponse["toResponse()"] - MapperResponse --> MapScalars["Map Scalar Fields"] - MapperResponse --> MapStatus["Enum to String"] - MapperResponse --> MapNested["Map Contact Information DTO"] - MapScalars --> Response["OrganizationResponse"] - MapStatus --> Response - MapNested --> Response -``` - ---- - -## 7. Nested Contact Mapping - -The mapper handles complex nested structures: - -### 7.1 ContactInformation - -Structure includes: - -- Contacts (list of `ContactPerson`) -- Physical address -- Mailing address -- `mailingAddressSameAsPhysical` flag - -### 7.2 Mailing Address Copy Logic - -If `mailingAddressSameAsPhysical == true`: - -- Mailing address is a **copy** of physical address -- Avoids shared object references -- Prevents unintended mutation side effects - -```mermaid -flowchart TD - Dto["ContactInformationDto"] --> CheckFlag{{"Same As Physical?"}} - CheckFlag -->|"Yes"| CopyAddr["Copy Physical Address"] - CheckFlag -->|"No"| MapMail["Map Mailing Address"] - CopyAddr --> Entity["ContactInformation Entity"] - MapMail --> Entity -``` - -This defensive copying ensures data integrity when updates occur later. - ---- - -## 8. Interaction with Other Modules - -### 8.1 REST Controllers - -Controllers such as `OrganizationController`: - -- Accept request DTOs -- Call services -- Use `OrganizationMapper` to convert entities to responses - -See: [Api Service Core Rest Controllers](api-service-core-rest-controllers.md) - -### 8.2 GraphQL Data Fetchers - -`OrganizationDataFetcher` uses the same mapper to: - -- Convert mutation inputs -- Map domain entities to GraphQL DTOs - -See: [Api Service Core Graphql Datafetchers](api-service-core-graphql-datafetchers.md) - -### 8.3 Domain Model - -Entity involved: - -- `com.openframe.data.document.organization.Organization` - -Defined in: [Data Mongo Domain Model](data-mongo-domain-model.md) - ---- - -## 9. End-to-End Organization Lifecycle (Simplified) - -```mermaid -flowchart LR - Client["Client App"] --> RestAPI["REST or GraphQL API"] - RestAPI --> MapperCreate["OrganizationMapper"] - MapperCreate --> Service["Organization Service"] - Service --> Repo["Mongo Repository"] - Repo --> DB["MongoDB"] - DB --> Repo - Repo --> Service - Service --> MapperResponse["OrganizationMapper"] - MapperResponse --> RestAPI - RestAPI --> Client -``` - -The Api Organization Mapping module ensures that: - -- API contracts remain stable -- Domain entities remain clean and persistence-focused -- Mapping rules are centralized and consistent - ---- - -## 10. Design Principles - -### 10.1 Single Responsibility - -The mapper: - -- Does not contain business logic -- Does not access repositories -- Does not perform validation - -It strictly transforms data structures. - -### 10.2 Stateless & Thread-Safe - -- No shared mutable state -- No injected dependencies -- Safe for concurrent use - -### 10.3 Explicit Mapping Over Reflection - -All fields are mapped explicitly rather than using reflection-based mapping frameworks. - -Benefits: - -- Better compile-time safety -- Clear documentation of field transformations -- Predictable behavior -- Easier debugging - ---- - -## 11. Key Takeaways - -The **Api Organization Mapping** module: - -- Acts as the transformation boundary between API and domain -- Protects immutable identity fields -- Implements safe partial update semantics -- Handles complex nested structures with defensive copying -- Is shared across REST and GraphQL layers - -By centralizing organization mapping logic, the platform ensures consistent behavior across all API surfaces while keeping the domain model isolated from transport-layer concerns. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md b/docs/reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md deleted file mode 100644 index 0c8137f06..000000000 --- a/docs/reference/architecture/api-service-core-config-and-security/api-service-core-config-and-security.md +++ /dev/null @@ -1,350 +0,0 @@ -# Api Service Core Config And Security - -The **Api Service Core Config And Security** module defines the foundational runtime configuration for the OpenFrame API service. It centralizes: - -- Core Spring Boot bean configuration -- Authentication and argument resolution -- OAuth client bootstrapping -- GraphQL custom scalar definitions -- Outbound HTTP client configuration -- JWT resource server integration with issuer-based caching - -This module is intentionally lightweight in business logic. Its responsibility is to configure and secure the API runtime so that higher-level modules such as REST controllers, GraphQL data fetchers, and service layers can operate consistently and securely. - ---- - -## 1. Architectural Role in the System - -The Api Service Core Config And Security module sits at the heart of the API runtime. It wires together: - -- Spring Boot auto-configuration -- Spring Security (OAuth2 Resource Server) -- GraphQL DGS custom scalars -- OAuth client initialization -- Shared infrastructure beans (PasswordEncoder, RestTemplate) - -### High-Level Placement - -```mermaid -flowchart LR - Gateway["Gateway Service"] -->|"JWT Forwarded"| ApiService["API Service Runtime"] - ApiService -->|"Uses"| ConfigModule["Api Service Core Config And Security"] - ApiService --> Controllers["REST Controllers"] - ApiService --> GraphQL["GraphQL Data Fetchers"] - ApiService --> Services["Domain Services"] - Services --> DataLayer["Mongo / Kafka / Redis"] -``` - -### Security Responsibility Split - -The API service does **not** perform full authentication enforcement. Instead: - -- The **Gateway Service**: - - Validates JWT tokens - - Handles public vs protected routes - - Injects `Authorization` headers from cookies -- The **Api Service Core Config And Security** module: - - Enables OAuth2 Resource Server support - - Resolves JWT issuers dynamically - - Exposes `@AuthenticationPrincipal` support - -This layered approach keeps the API service stateless and focused on business logic. - ---- - -## 2. Core Configuration Components - -### 2.1 ApiApplicationConfig - -**Component:** -- `ApiApplicationConfig` - -Provides shared infrastructure beans. - -#### PasswordEncoder Bean - -```java -@Bean -public PasswordEncoder passwordEncoder() { - return new BCryptPasswordEncoder(); -} -``` - -- Uses BCrypt for secure password hashing. -- Shared across services handling credentials. -- Ensures consistent hashing strategy across modules. - ---- - -### 2.2 AuthenticationConfig - -**Component:** -- `AuthenticationConfig` - -Registers a custom Spring MVC argument resolver: - -- `AuthPrincipalArgumentResolver` - -This enables controller methods like: - -```java -public ResponseEntity getProfile(@AuthenticationPrincipal AuthPrincipal principal) -``` - -#### Flow - -```mermaid -flowchart TD - Request["HTTP Request"] --> Security["OAuth2 Resource Server"] - Security --> Controller["Controller Method"] - Controller --> Resolver["AuthPrincipalArgumentResolver"] - Resolver --> Principal["AuthPrincipal Injected"] -``` - -This ensures: -- Clean controller signatures -- Strong typing for authenticated users -- No manual extraction of JWT claims - ---- - -### 2.3 SecurityConfig - -**Component:** -- `SecurityConfig` - -This is the central Spring Security configuration. - -#### Key Characteristics - -- CSRF disabled (stateless API) -- All routes `permitAll()` at API layer -- OAuth2 Resource Server enabled -- Multi-issuer JWT support -- Caffeine-based JWT provider cache - -### JWT Issuer-Based Authentication - -The module uses: - -- `JwtIssuerAuthenticationManagerResolver` -- A `LoadingCache` - -This allows dynamic resolution of authentication managers based on the JWT issuer. - -```mermaid -flowchart TD - Incoming["Incoming Request with JWT"] --> ExtractIssuer["Extract iss Claim"] - ExtractIssuer --> CacheCheck["Check JwtAuthenticationProvider Cache"] - CacheCheck -->|"Miss"| CreateDecoder["Create JwtDecoder from Issuer"] - CreateDecoder --> StoreCache["Store in Caffeine Cache"] - CacheCheck -->|"Hit"| UseProvider["Reuse Cached Provider"] - StoreCache --> Authenticate["Authenticate JWT"] - UseProvider --> Authenticate - Authenticate --> Principal["SecurityContext Updated"] -``` - -#### Cache Configuration - -The cache is controlled by properties: - -- `openframe.security.jwt.cache.expire-after` -- `openframe.security.jwt.cache.refresh-after` -- `openframe.security.jwt.cache.maximum-size` - -This ensures: -- Efficient decoder reuse -- Controlled memory usage -- Automatic refresh of issuer metadata - ---- - -### 2.4 DataInitializer - -**Component:** -- `DataInitializer` - -Implements a `CommandLineRunner` to initialize OAuth clients at application startup. - -#### Responsibilities - -- Reads properties: - - `oauth.client.default.id` - - `oauth.client.default.secret` -- Checks if client exists -- Updates secret if changed -- Creates client if missing - -```mermaid -flowchart TD - Startup["Application Startup"] --> LoadProps["Read OAuth Properties"] - LoadProps --> QueryRepo["Find Client by ID"] - QueryRepo -->|"Exists"| CompareSecret["Compare Secrets"] - CompareSecret -->|"Different"| UpdateSecret["Update and Save"] - CompareSecret -->|"Same"| Skip["No Action"] - QueryRepo -->|"Missing"| CreateClient["Create OAuthClient"] -``` - -#### Design Rationale - -- Ensures environment-driven configuration. -- Avoids manual database seeding. -- Keeps OAuth client configuration aligned with deployment configuration. - ---- - -### 2.5 GraphQL Custom Scalars - -The module defines three custom DGS scalars. - -#### DateScalarConfig - -- Scalar name: `Date` -- Backed by `LocalDate` -- Format: `yyyy-MM-dd` - -Validation ensures: -- Strict format compliance -- Clear error messages for invalid input - ---- - -#### InstantScalarConfig - -- Scalar name: `Instant` -- Backed by `Instant` -- ISO-8601 format (e.g. `2026-01-01T10:15:30Z`) - -Provides: -- Consistent time serialization -- Accurate timezone handling - ---- - -#### LongScalarConfig - -- Scalar name: `Long` -- Supports 64-bit integers -- Required for values exceeding GraphQL `Int` limits - -Handles: -- Numeric literals -- String-based numeric input -- Safe coercion with validation - -```mermaid -flowchart LR - GraphQLInput["GraphQL Query Input"] --> Scalar["Custom Scalar Coercing"] - Scalar --> Validation["Format Validation"] - Validation --> Parsed["Java Type (LocalDate / Instant / Long)"] - Parsed --> DataFetcher["Data Fetcher Execution"] -``` - -These scalars standardize type handling across all GraphQL data fetchers. - ---- - -### 2.6 RestTemplateConfig - -**Component:** -- `RestTemplateConfig` - -Defines a singleton `RestTemplate` bean. - -```java -@Bean -public RestTemplate restTemplate() { - return new RestTemplate(); -} -``` - -Used for: -- Outbound HTTP calls -- Internal service-to-service communication -- OAuth metadata resolution (indirectly via Spring Security) - -Centralizing the bean allows: -- Future interceptors -- Timeout configuration -- Observability instrumentation - ---- - -## 3. End-to-End Security Flow - -Below is a simplified request lifecycle involving this module. - -```mermaid -sequenceDiagram - participant Client - participant Gateway - participant Api as "API Service" - participant Security as "SecurityConfig" - participant Controller - - Client->>Gateway: Request with Cookie or Token - Gateway->>Gateway: Validate JWT - Gateway->>Api: Forward request with Authorization header - Api->>Security: Resolve issuer and authenticate - Security->>Api: Populate SecurityContext - Api->>Controller: Invoke handler - Controller->>Controller: Inject AuthPrincipal -``` - -### Key Observations - -- Authentication enforcement happens upstream. -- API service trusts Gateway filtering. -- Multi-tenant issuer resolution is supported dynamically. -- Controllers remain clean and declarative. - ---- - -## 4. Design Principles - -The Api Service Core Config And Security module follows these principles: - -### 4.1 Separation of Concerns - -- Gateway: authentication + edge security -- API: resource server support + identity propagation -- Services: business logic - -### 4.2 Statelessness - -- No server-side sessions -- JWT-based identity -- Cache only for issuer metadata - -### 4.3 Extensibility - -- Centralized bean definitions -- Pluggable scalar definitions -- Configurable cache properties - -### 4.4 Environment-Driven Configuration - -- OAuth clients initialized via properties -- Cache behavior driven by configuration -- No hard-coded credentials - ---- - -## 5. Summary - -The **Api Service Core Config And Security** module is the foundation of the OpenFrame API runtime. It: - -- Enables secure JWT-based authentication -- Integrates with multi-issuer OAuth providers -- Standardizes GraphQL scalar behavior -- Provides core infrastructure beans -- Bootstraps OAuth client configuration - -While minimal in business logic, this module is critical for ensuring: - -- Secure request handling -- Consistent authentication context propagation -- Reliable GraphQL type handling -- Clean separation between infrastructure and domain layers - -It acts as the runtime spine of the API service, ensuring that all higher-level modules operate in a secure, predictable, and standardized environment. diff --git a/docs/reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md b/docs/reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md deleted file mode 100644 index 7caecbce5..000000000 --- a/docs/reference/architecture/api-service-core-dataloaders/api-service-core-dataloaders.md +++ /dev/null @@ -1,343 +0,0 @@ -# Api Service Core Dataloaders - -## Overview - -The **Api Service Core Dataloaders** module provides batched, asynchronous data loading capabilities for the GraphQL layer of the OpenFrame API Service Core. - -Built on top of **Netflix DGS** and the standard `org.dataloader` pattern, this module eliminates N+1 query problems by batching and caching entity lookups during a single GraphQL request lifecycle. - -It acts as a bridge between: - -- GraphQL Data Fetchers (resolvers) -- Domain Services -- Mongo repositories -- Underlying domain documents - -This module is a critical performance layer in the API stack. - ---- - -## Architectural Context - -The Api Service Core Dataloaders module sits between the GraphQL Data Fetchers and the service/repository layer. - -```mermaid -flowchart TD - Client["GraphQL Client"] --> GQL["GraphQL Data Fetchers"] - GQL --> DL["Api Service Core Dataloaders"] - DL --> Service["Domain Services"] - DL --> Repo["Mongo Repositories"] - Service --> Mongo[("MongoDB")] - Repo --> Mongo -``` - -### Responsibilities - -- Batch load related entities by IDs -- Preserve request ordering -- Avoid redundant queries within a request -- Delegate to appropriate service or repository -- Return results asynchronously via `CompletableFuture` - ---- - -## Why DataLoaders Are Required - -Without DataLoaders, a query such as: - -```graphql -query { - devices { - id - organization { - name - } - } -} -``` - -would result in: - -- 1 query to load devices -- N queries to load organizations - -With DataLoaders: - -- 1 query to load devices -- 1 batched query to load all required organizations - -This dramatically improves performance and scalability. - ---- - -## DataLoader Execution Model - -All loaders implement: - -```java -BatchLoader -``` - -and are annotated with: - -```java -@DgsDataLoader(name = "...") -``` - -Each loader: - -1. Receives a list of keys -2. Removes nulls -3. Batch queries the database or service -4. Maps results by ID -5. Returns results in the same order as input -6. Executes asynchronously via `CompletableFuture.supplyAsync` - -```mermaid -sequenceDiagram - participant Resolver as "GraphQL Resolver" - participant Loader as "DataLoader" - participant Service as "Service/Repository" - - Resolver->>Loader: load([id1, id2, id3]) - Loader->>Service: Batch fetch entities - Service-->>Loader: List of entities - Loader-->>Resolver: Ordered results -``` - ---- - -# Core DataLoaders - -Below are the loaders provided by this module. - ---- - -## InstalledAgentDataLoader - -**Purpose:** Batch loads InstalledAgent objects grouped by machine ID. - -**Delegates to:** `InstalledAgentService` - -```mermaid -flowchart LR - Resolver --> InstalledAgentDataLoader - InstalledAgentDataLoader --> InstalledAgentService - InstalledAgentService --> Mongo -``` - -- Key: `machineId` -- Value: `List` -- Use case: Resolving installed agents for multiple machines in one request - ---- - -## KnowledgeBaseAttachmentDataLoader - -**Purpose:** Batch loads attachments for knowledge base items. - -**Delegates to:** `KnowledgeBaseAttachmentService` - -- Key: `knowledgeBaseItemId` -- Value: `List` -- Includes debug logging for batch size visibility - ---- - -## KnowledgeBaseItemDataLoader - -**Purpose:** Batch loads KnowledgeBaseItem by ID. - -**Used for:** Polymorphic AssignableTarget resolution for KNOWLEDGE_ARTICLE target type. - -**Delegates to:** `KnowledgeBaseItemRepository` - -### Special Behavior - -- Filters null IDs -- De-duplicates IDs -- Returns `null` for missing items -- Preserves request order - -```mermaid -flowchart TD - Resolver --> KnowledgeBaseItemDataLoader - KnowledgeBaseItemDataLoader --> KnowledgeBaseItemRepository - KnowledgeBaseItemRepository --> Mongo -``` - ---- - -## KnowledgeBaseTagDataLoader - -**Purpose:** Batch loads Tags associated with Knowledge Base items. - -**Delegates to:** `KnowledgeBaseTagService` - -- Key: `knowledgeBaseItemId` -- Value: `List` - ---- - -## MachineDataLoader - -**Purpose:** Batch loads Machine entities by machine ID. - -**Used for:** AssignableTarget resolution for DEVICE target type. - -**Delegates to:** `MachineRepository` - -### Behavior - -- Filters null IDs -- Performs `findByMachineIdIn` -- Maps results by machineId -- Preserves input ordering - ---- - -## OrganizationDataLoader - -**Purpose:** Batch loads Organization entities by organization ID. - -**Key Optimization:** Prevents N+1 queries when resolving organization for many machines. - -**Delegates to:** `OrganizationRepository` - -### Additional Logic - -- Filters out soft-deleted organizations -- Only returns organizations with `ACTIVE` status - -```mermaid -flowchart TD - Resolver --> OrganizationDataLoader - OrganizationDataLoader --> OrganizationRepository - OrganizationRepository --> Mongo -``` - ---- - -## TagDataLoader - -**Purpose:** Batch loads Tags for machines. - -**Delegates to:** `TagService` - -- Key: `machineId` -- Value: `List` - ---- - -## TicketDataLoader - -**Purpose:** Batch loads Ticket entities by ID. - -**Used for:** AssignableTarget resolution for TICKET target type. - -**Delegates to:** `TicketRepository` - -### Behavior - -- Filters null IDs -- Uses `findAllById` -- Preserves input order - ---- - -## ToolConnectionDataLoader - -**Purpose:** Batch loads ToolConnection entities for machines. - -**Delegates to:** `ToolConnectionService` - -- Key: `machineId` -- Value: `List` - ---- - -## UserDataLoader - -**Purpose:** Batch loads UserResponse DTOs by user ID. - -**Delegates to:** `UserService` - -### Important Detail - -The loader goes through `UserService` rather than directly querying a repository. This ensures: - -- Registered UserProcessor implementations enrich responses -- SaaS-specific user augmentation is applied -- Profile image and additional computed fields are included - -**Used by:** KnowledgeBaseItem author resolver - ---- - -# Cross-Module Relationships - -The Api Service Core Dataloaders module works closely with: - -- [Api Service Core GraphQL Datafetchers](../api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md) -- Domain services in the API layer -- Mongo repositories from the data layer - -```mermaid -flowchart TD - subgraph graphql_layer["GraphQL Layer"] - DF["Data Fetchers"] - DL["DataLoaders"] - end - - subgraph service_layer["Service Layer"] - SVC["Domain Services"] - end - - subgraph data_layer["Data Layer"] - REPO["Mongo Repositories"] - DB[("MongoDB")] - end - - DF --> DL - DL --> SVC - DL --> REPO - SVC --> DB - REPO --> DB -``` - ---- - -# Performance Characteristics - -βœ… Eliminates N+1 query issues -βœ… Reduces database round trips -βœ… Maintains deterministic ordering -βœ… Enables async non-blocking resolution -βœ… Scales efficiently with large GraphQL queries - ---- - -# Design Patterns Used - -- **Batch Loading Pattern** -- **Repository Pattern** -- **Service Layer Abstraction** -- **Asynchronous Execution via CompletableFuture** -- **GraphQL DataLoader Pattern** - ---- - -# Summary - -The **Api Service Core Dataloaders** module is a performance-critical layer in the OpenFrame GraphQL architecture. - -It ensures that complex nested queries: - -- Execute efficiently -- Avoid excessive database calls -- Preserve domain encapsulation -- Maintain clean separation between GraphQL and persistence layers - -Without this module, GraphQL queries across devices, organizations, tickets, knowledge base items, and users would not scale effectively. - -This module enables high-performance, production-grade GraphQL execution within the Api Service Core. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md b/docs/reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md deleted file mode 100644 index 6757c6391..000000000 --- a/docs/reference/architecture/api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md +++ /dev/null @@ -1,439 +0,0 @@ -# Api Service Core Graphql Datafetchers - -## Overview - -The **Api Service Core Graphql Datafetchers** module is the GraphQL execution layer of the OpenFrame API Service Core. It exposes domain capabilities (devices, events, organizations, knowledge base, notifications, assignments, tools, logs, and tags) through Netflix DGS-based GraphQL data fetchers. - -This module acts as the orchestration layer between: - -- GraphQL schema and Relay node model -- Application services (DeviceService, EventService, KnowledgeBaseService, etc.) -- DataLoaders for batching and N+1 mitigation -- Security context (JWT-based principals) -- Cursor-based pagination and filter criteria - -It does **not** implement business logic directly. Instead, it: - -- Decodes Relay global IDs -- Maps GraphQL inputs into domain filter criteria -- Delegates to services -- Wraps results into Relay-compatible connection types -- Resolves nested fields using DataLoaders - ---- - -## High-Level Architecture - -```mermaid -flowchart LR - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetchers["DGS DataFetchers"] - DataFetchers --> Mappers["GraphQL Mappers"] - DataFetchers --> Services["Application Services"] - Services --> Repositories["Mongo / Pinot / Cassandra Repositories"] - - DataFetchers --> DataLoaders["DataLoaders (Batching)"] - DataLoaders --> Services - - DataFetchers --> Security["SecurityContext + JWT"] -``` - -### Responsibilities of This Module - -- Implements `@DgsQuery`, `@DgsMutation`, and `@DgsData` -- Converts GraphQL inputs into service-layer DTOs -- Encodes and decodes Relay global IDs -- Builds cursor-based pagination criteria -- Returns Relay-compatible `Connection` and `Edge` types -- Resolves polymorphic types (NotificationContext, Node interface) - ---- - -## Core Design Patterns - -### 1. Relay Global ID Pattern - -All node-based entities use Relay global IDs. - -```text -Global ID = Base64(TypeName:RawId) -Example: "Machine:abc123" -> encoded -``` - -Pattern used everywhere: - -- Decode input ID using `Relay.fromGlobalId()` -- Fetch entity using raw ID -- Encode output ID using `Relay.toGlobalId()` - -This enables: - -- Uniform `node(id: ID!)` resolution -- Type-safe global references -- Cross-entity linking - ---- - -### 2. Cursor-Based Pagination - -Pagination is built using: - -- `ConnectionArgs` -- `CursorPaginationCriteria` -- `CountedGenericQueryResult` or `GenericQueryResult` -- `GenericEdge` -- `CountedGenericConnection` or `GenericConnection` - -```mermaid -flowchart TD - Query["GraphQL Query"] --> Args["ConnectionArgs"] - Args --> Pagination["CursorPaginationCriteria"] - Pagination --> ServiceQuery["Service.query(...)"] - ServiceQuery --> Result["QueryResult"] - Result --> Mapper["GraphQL Mapper"] - Mapper --> Connection["Relay Connection"] -``` - -This ensures consistent pagination behavior across: - -- Devices -- Events -- Organizations -- Logs -- Knowledge base -- Assignments -- Notifications - ---- - -### 3. DataLoader-Based N+1 Prevention - -Nested fields are resolved asynchronously using DGS DataLoaders. - -Example relationships: - -- Machine β†’ Tags -- Machine β†’ InstalledAgents -- Machine β†’ ToolConnections -- Machine β†’ Organization -- KnowledgeBaseItem β†’ Tags -- KnowledgeBaseItem β†’ Attachments -- KnowledgeBaseItem β†’ Author -- Assignment β†’ Target entity - -```mermaid -flowchart LR - Query["devices query"] --> DeviceFetcher - DeviceFetcher --> DeviceService - DeviceFetcher --> MachineNode["Machine"] - MachineNode -->|"tags"| TagLoader - MachineNode -->|"installedAgents"| AgentLoader - TagLoader --> TagService - AgentLoader --> InstalledAgentService -``` - -All DataLoader calls return `CompletableFuture` to ensure batched execution. - ---- - -## Data Fetcher Components - -### AssignmentDataFetcher - -Handles: - -- Assignment queries -- Assign/unassign mutations -- Assignment counts by target type -- Target resolution using DataLoaders - -Key patterns: - -- Uses `AssignmentService` -- Maps results via `GraphQLAssignmentMapper` -- Resolves polymorphic assignment targets (Organization, Machine, Ticket, KnowledgeBaseItem) - ---- - -### DeviceDataFetcher - -Responsible for: - -- Device listing with filtering and pagination -- Device filters -- Device lookup by ID -- Resolving nested relations (tags, agents, organization) - -Key dependencies: - -- `DeviceService` -- `DeviceFilterService` -- `TagService` -- `GraphQLDeviceMapper` - -Implements full Relay connection pattern. - ---- - -### EventDataFetcher - -Provides: - -- Event listing -- Event lookup -- Event creation and update -- Event filter retrieval - -Features: - -- Converts `EventFilterInput` into `EventFilterCriteria` -- Uses `GraphQLEventMapper` -- Returns `GenericConnection` - ---- - -### KnowledgeBaseDataFetcher - -One of the most feature-rich fetchers. - -Supports: - -- Folder and article queries -- Publishing/unpublishing -- Archiving/unarchiving -- Folder tree retrieval -- Tag management -- Attachment uploads (temp + permanent) -- Attachment linking - -Security-aware behavior: - -- Extracts user ID from JWT -- Uses `AuthPrincipal` -- Applies mutation wrapper pattern for success/error payloads - -```mermaid -flowchart TD - Mutation["createArticle"] --> GetUser["Extract JWT User"] - GetUser --> Mapper - Mapper --> Service["KnowledgeBaseService"] - Service --> DB["Mongo Repository"] -``` - -Also resolves nested fields: - -- Tags -- Attachments -- Author -- Parent ID - ---- - -### LogDataFetcher - -Conditionally enabled via: - -```text -spring.data.cassandra.enabled=true -``` - -Responsibilities: - -- Log listing -- Log filters -- Log detail retrieval - -Uses: - -- `LogService` -- `GraphQLLogMapper` - -Provides cursor-based pagination for log events. - ---- - -### NodeDataFetcher - -Implements Relay `node(id: ID!)` and `nodes(ids: [ID!])`. - -```mermaid -flowchart TD - NodeQuery["node(id)"] --> Decode["Relay.fromGlobalId"] - Decode --> TypeSwitch["NodeType enum"] - TypeSwitch --> DeviceService - TypeSwitch --> OrganizationService - TypeSwitch --> EventService - TypeSwitch --> ToolService - TypeSwitch --> TagService -``` - -Supports types such as: - -- Machine -- Organization -- Event -- IntegratedTool -- Tag -- ToolConnection -- InstalledAgent -- Tenant - -This enables universal entity lookup via a single GraphQL entry point. - ---- - -### NotificationDataFetcher - -Handles authenticated notification access. - -Security rules: - -- Requires `ADMIN` or `AGENT` authority -- Determines recipient type (USER or MACHINE) -- Extracts identity from JWT - -Operations: - -- List notifications -- Mark as read -- Mark all as read -- Delete -- Delete all read -- Unread counts by category - -```mermaid -flowchart TD - Query["notifications"] --> CurrentRecipient - CurrentRecipient --> Principal["AuthPrincipal"] - Principal --> NotificationService - NotificationService --> ReadStateService -``` - -Also includes strict ID decoding validation for notification IDs. - ---- - -### OrganizationDataFetcher - -Provides: - -- Organization listing (paginated) -- Organization lookup -- Filter options - -Uses: - -- `OrganizationService` -- `OrganizationQueryService` -- `GraphQLOrganizationMapper` - ---- - -### TagDataFetcher - -Supports: - -- Tag listing -- Key/value suggestions -- Tag creation -- Tag updates -- Tag deletion - -Converts Relay ID for mutation operations. - ---- - -### ToolsDataFetcher - -Provides: - -- Integrated tool listing -- Tool filters - -Uses: - -- `ToolService` -- `GraphQLToolMapper` - ---- - -### NotificationContextGraphQlTypeResolver - -Resolves polymorphic GraphQL types for `NotificationContext`. - -```mermaid -flowchart TD - NotificationContext --> Resolver - Resolver -->|"type discriminator"| TypeMap - TypeMap --> GraphQLType -``` - -- Maps discriminator string β†’ GraphQL type name -- Falls back to `GenericContext` if not found -- Enables GraphQL union/interface resolution - ---- - -## Security Integration - -The module integrates tightly with JWT-based security. - -Key aspects: - -- Uses `SecurityContextHolder` -- Extracts `AuthPrincipal` from JWT -- Determines `ActorType` (USER or AGENT) -- Applies `@PreAuthorize` annotations - -Security is enforced at: - -- Query level (Notification access) -- Mutation level -- User-aware operations (Knowledge base, notifications) - ---- - -## Interaction with Other Layers - -```mermaid -flowchart LR - GraphQL["GraphQL DataFetchers"] --> Services - Services --> Domain["Domain Models"] - Services --> Repositories - Services --> Messaging["NATS / Kafka"] - Services --> Cache["Redis"] -``` - -The Api Service Core Graphql Datafetchers module: - -- Does not access repositories directly -- Does not contain domain logic -- Acts as translation + orchestration layer - ---- - -## Key Architectural Benefits - -1. Clear separation between API and business logic -2. Uniform Relay support across all entities -3. Consistent pagination model -4. Batch loading via DataLoaders -5. Centralized Node resolution -6. Strong JWT-based identity handling -7. Polymorphic type resolution for flexible schemas - ---- - -## Summary - -The **Api Service Core Graphql Datafetchers** module is the central GraphQL execution layer of the OpenFrame platform. - -It: - -- Bridges GraphQL schema to service layer -- Implements Relay global ID and connection patterns -- Uses DataLoaders to prevent N+1 issues -- Enforces JWT-based security -- Provides consistent pagination and filtering - -By keeping business logic in services and focusing purely on API orchestration, this module ensures a clean, scalable, and maintainable GraphQL architecture for the OpenFrame ecosystem. diff --git a/docs/reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md b/docs/reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md deleted file mode 100644 index f118770b9..000000000 --- a/docs/reference/architecture/api-service-core-graphql-dtos/api-service-core-graphql-dtos.md +++ /dev/null @@ -1,434 +0,0 @@ -# Api Service Core Graphql Dtos - -The **Api Service Core Graphql Dtos** module defines the GraphQL-facing Data Transfer Objects (DTOs) used by the Api Service Core. These DTOs represent: - -- GraphQL connection and edge wrappers -- Query filter input types -- Mutation input types -- Structured response models -- Lightweight aggregation models - -This module acts as the **schema contract layer** between GraphQL data fetchers and the underlying domain, service, and persistence layers. - -It does not contain business logic. Instead, it formalizes how data flows into and out of the GraphQL API. - ---- - -## Architectural Role - -Within the overall Api Service Core architecture, this module sits between: - -- GraphQL Data Fetchers (execution layer) -- Domain DTOs and persistence documents (data layer) -- Services and processors (business layer) - -### High-Level Placement - -```mermaid -flowchart TD - Client["GraphQL Client"] --> Schema["GraphQL Schema"] - Schema --> DataFetcher["GraphQL Data Fetchers"] - DataFetcher --> Dtos["Api Service Core Graphql Dtos"] - Dtos --> Services["Core Services"] - Services --> Repositories["Mongo Repositories"] - Repositories --> Database[("Database")] -``` - -The **Api Service Core Graphql Dtos** module defines the types used in: - -- Query arguments -- Mutation inputs -- Relay-style connections -- Filter inputs -- Structured API responses - ---- - -# Module Structure - -The module can be logically grouped into the following categories: - -1. Relay Connection Models -2. Filter Input DTOs -3. Mutation Input DTOs -4. Domain Response DTOs -5. Aggregation and Assignment DTOs - ---- - -# 1. Relay Connection Models - -GraphQL in Api Service Core follows the Relay-style pagination model. This module provides reusable generic wrappers. - -## GenericEdge - -```java -public class GenericEdge { - private T node; - private String cursor; -} -``` - -Responsibilities: - -- Wraps a single node -- Provides a cursor for pagination -- Forms part of a GraphQL Connection response - -## CountedGenericConnection - -```java -public class CountedGenericConnection - extends GenericConnection { - private int filteredCount; -} -``` - -Responsibilities: - -- Extends the base connection model -- Adds `filteredCount` for total result size -- Supports filtered pagination use cases - -### Relay Pagination Flow - -```mermaid -flowchart LR - Query["GraphQL Query"] --> Args["Connection Arguments"] - Args --> Fetcher["Data Fetcher"] - Fetcher --> Service["Service Layer"] - Service --> Edge["GenericEdge"] - Edge --> Connection["CountedGenericConnection"] - Connection --> Client["GraphQL Client"] -``` - -This ensures: - -- Stable cursor-based pagination -- Support for filtering -- Predictable total counts for UI rendering - ---- - -# 2. Filter Input DTOs - -These classes represent GraphQL input types for querying and filtering domain entities. - -They are optimized for client-side flexibility and map to domain-level filter criteria. - -## LogFilterInput - -Supports filtering by: - -- Date range (`startDate`, `endDate`) -- Event types -- Tool types -- Severities -- Organization IDs -- Device ID - -Used in log and audit queries. - ---- - -## DeviceFilterInput - -Supports filtering by: - -- Device status -- Device type -- Operating system types -- Organization IDs -- Tag keys and values - -Enables complex device search and inventory queries. - ---- - -## EventFilterInput - -Supports filtering by: - -- User IDs -- Event types -- Date range - -Used by event-related GraphQL queries. - ---- - -## KnowledgeBaseFilterInput - -Supports filtering by: - -- Parent folder -- Item type -- Tag IDs - -Used to navigate and filter knowledge base structures. - ---- - -## OrganizationFilterInput - -Supports filtering by: - -- Category -- Employee range -- Contract status -- Organization status - ---- - -## ToolFilterInput - -Supports filtering by: - -- Enabled state -- Tool type -- Category -- Platform category - ---- - -## NotificationFilterInput - -Supports filtering by: - -- Read/unread status - ---- - -### Filter Mapping Flow - -```mermaid -flowchart TD - GraphQLInput["GraphQL Filter Input"] --> DataFetcher["Data Fetcher"] - DataFetcher --> Criteria["Domain Filter Criteria"] - Criteria --> Repository["Custom Repository"] - Repository --> Results["Domain Documents"] - Results --> Connection["Connection DTO"] -``` - -Filter inputs are intentionally: - -- Client-oriented -- Flexible -- Decoupled from persistence implementation - ---- - -# 3. Mutation Input DTOs - -These DTOs define structured input contracts for GraphQL mutations. - -They frequently include validation annotations such as: - -- `@NotBlank` -- `@NotNull` -- `@Size` - -## Event Mutations - -### CreateEventInput - -Fields: - -- `userId` (required) -- `type` (required) -- `data` - -Used for creating audit or system events. - ---- - -## Knowledge Base Mutations - -### CreateArticleInput - -Supports: - -- Article metadata -- Status -- Tag assignments -- Cross-entity assignments - -### UpdateArticleInput - -Supports partial updates of: - -- Name -- Parent -- Content -- Summary - -### DeleteFolderInput - -Supports: - -- Folder deletion -- Child handling strategy -- Optional move target - -### CreateKnowledgeBaseAttachmentInput - -Defines: - -- Target article -- File metadata - -### CreateKnowledgeBaseTempAttachmentInput - -Defines temporary upload metadata before final linking. - -### LinkKnowledgeBaseTempAttachmentsInput - -Supports bulk linking of temporary attachments to an article. - ---- - -## User Mutations - -### UpdateUserRequest - -Supports updating: - -- First name -- Last name - -Includes size validation constraints. - ---- - -### Mutation Execution Flow - -```mermaid -flowchart TD - Client["GraphQL Mutation"] --> Input["Mutation Input DTO"] - Input --> Validation["Bean Validation"] - Validation --> Processor["Service or Processor"] - Processor --> Repository["Mongo Repository"] - Repository --> Response["Response DTO"] -``` - -The DTO layer guarantees: - -- Schema consistency -- Validation enforcement -- Clear mutation contracts - ---- - -# 4. Domain Response DTOs - -These classes represent structured GraphQL output types. - -## UserResponse - -Encapsulates: - -- Identity fields -- Roles -- Status -- Profile image -- Audit timestamps - -Designed to: - -- Hide internal domain representation -- Expose only client-relevant fields -- Maintain immutability via builder pattern - ---- - -# 5. Aggregation and Assignment DTOs - -## AssignedItemCount - -```java -public class AssignedItemCount { - private AssignmentTargetType targetType; - private int count; -} -``` - -Purpose: - -- Represents assignment counts grouped by target type -- Used in dashboards and summary queries -- Bridges domain enum types into GraphQL responses - ---- - -# Design Principles - -The **Api Service Core Graphql Dtos** module follows several architectural principles: - -## 1. Clear Separation of Concerns - -- DTOs contain no business logic -- Services implement logic -- Repositories handle persistence - -## 2. GraphQL-Optimized Models - -- Designed for GraphQL schema shape -- Optimized for client flexibility -- Uses input-specific classes instead of reusing domain objects - -## 3. Validation at the Boundary - -Validation annotations ensure: - -- Early error detection -- Strong input guarantees -- Reduced service-layer complexity - -## 4. Relay Compliance - -- Edge-based pagination -- Connection wrappers -- Cursor-based navigation - ---- - -# End-to-End Data Flow Example - -```mermaid -flowchart TD - UI["Frontend UI"] --> Query["GraphQL Query or Mutation"] - Query --> DTOIn["Input DTO"] - DTOIn --> Fetcher["Data Fetcher"] - Fetcher --> Service["Core Service"] - Service --> Repo["Mongo Repository"] - Repo --> Domain["Domain Document"] - Domain --> DTOOut["Edge or Response DTO"] - DTOOut --> Connection["CountedGenericConnection"] - Connection --> UI -``` - -This module ensures the GraphQL API remains: - -- Strongly typed -- Stable -- Decoupled from internal persistence structures -- Easy to evolve without breaking clients - ---- - -# Summary - -The **Api Service Core Graphql Dtos** module defines the GraphQL contract layer of the Api Service Core. It provides: - -- Relay-compatible connection models -- Rich filter input types -- Structured mutation inputs -- Clean response DTOs -- Aggregation models for summary data - -By isolating API-facing structures in a dedicated module, the system maintains: - -- Strong separation between API and domain -- High schema clarity -- Safer refactoring of internal models -- Predictable GraphQL behavior diff --git a/docs/reference/architecture/api-service-core-http-and-graphql/api-service-core-http-and-graphql.md b/docs/reference/architecture/api-service-core-http-and-graphql/api-service-core-http-and-graphql.md new file mode 100644 index 000000000..e1689929d --- /dev/null +++ b/docs/reference/architecture/api-service-core-http-and-graphql/api-service-core-http-and-graphql.md @@ -0,0 +1,358 @@ +# Api Service Core Http And Graphql + +## Overview + +The **Api Service Core Http And Graphql** module is the central application layer of the OpenFrame backend. It exposes: + +- REST endpoints for operational and administrative APIs +- GraphQL APIs (queries, mutations, Relay-style node resolution) +- Security integration as an OAuth2 Resource Server +- DataLoader-based batching to prevent N+1 query problems +- DTOs and connection models aligned with the `api-lib-contracts` module + +This module acts as the orchestration layer between: + +- Gateway (edge routing and JWT handling) +- Authorization Service (OAuth2 / OIDC) +- Mongo repositories and domain services +- Stream-processing and external integrations +- Frontend (GraphQL + REST consumers) + +It does **not** contain core domain persistence logic. Instead, it delegates to services and repositories defined in other modules. + +--- + +## High-Level Architecture + +```mermaid +flowchart TD + Client["Frontend / API Client"] --> Gateway["Gateway Service Core"] + Gateway --> ApiService["Api Service Core Http And Graphql"] + + ApiService --> Controllers["REST Controllers"] + ApiService --> GraphQL["GraphQL DataFetchers"] + + Controllers --> Services["Application Services"] + GraphQL --> Services + + Services --> Mongo["Mongo Repositories"] + Services --> Stream["Kafka / Stream Processing"] + + ApiService --> Authz["Authorization Service Core"] +``` + +### Responsibilities + +| Layer | Responsibility | +|--------|----------------| +| REST Controllers | HTTP endpoints for operational APIs | +| GraphQL DataFetchers | GraphQL queries & mutations via DGS | +| DataLoaders | Batch loading & N+1 prevention | +| DTOs | GraphQL and REST data contracts | +| SecurityConfig | OAuth2 Resource Server integration | +| Services | Business orchestration (delegated to domain services) | + +--- + +## Security Architecture + +Security is intentionally minimal in this module. + +### Key Principle + +The **Gateway Service Core** performs: + +- JWT validation +- Cookie extraction +- Authorization header injection +- PermitAll path filtering + +The Api Service Core Http And Graphql module: + +- Acts as an OAuth2 Resource Server +- Supports `@AuthenticationPrincipal` resolution +- Caches JWT decoders by issuer + +### Security Flow + +```mermaid +sequenceDiagram + participant Client + participant Gateway + participant ApiService + participant AuthServer + + Client->>Gateway: HTTP Request + Gateway->>AuthServer: Validate JWT + AuthServer-->>Gateway: Token valid + Gateway->>ApiService: Forward request with Authorization header + ApiService->>ApiService: Resolve AuthPrincipal + ApiService-->>Client: JSON / GraphQL response +``` + +### Core Security Components + +- `SecurityConfig` – Configures OAuth2 Resource Server and issuer-based JWT resolution +- `AuthenticationConfig` – Registers `AuthPrincipalArgumentResolver` +- `ApiApplicationConfig` – Provides `PasswordEncoder` +- Caffeine cache for `JwtAuthenticationProvider` + +--- + +## REST Layer + +The REST controllers provide operational APIs and administrative endpoints. + +### Key Controllers + +| Controller | Responsibility | +|------------|----------------| +| `HealthController` | Health probe endpoint | +| `MeController` | Authenticated user info | +| `ApiKeyController` | API key lifecycle | +| `OrganizationController` | Organization mutations | +| `UserController` | User management | +| `InvitationController` | User invitation lifecycle | +| `SSOConfigController` | SSO provider configuration | +| `AgentRegistrationSecretController` | Agent bootstrap secrets | +| `ForceAgentController` | Forced tool / agent operations | +| `ReleaseVersionController` | Current release metadata | +| `OpenFrameClientConfigurationController` | Client config exposure | + +### REST Processing Flow + +```mermaid +flowchart LR + Request["HTTP Request"] --> Controller["@RestController"] + Controller --> Service["Application Service"] + Service --> Repository["Mongo Repository"] + Repository --> Service + Service --> Controller + Controller --> Response["HTTP Response"] +``` + +Controllers are thin and delegate logic to service classes. + +--- + +## GraphQL Layer (DGS) + +The GraphQL API is implemented using Netflix DGS. + +### Categories of DataFetchers + +| Category | Examples | +|----------|----------| +| Core Entities | DeviceDataFetcher, OrganizationDataFetcher | +| Events & Logs | EventDataFetcher, LogDataFetcher | +| Knowledge Base | KnowledgeBaseDataFetcher | +| Assignments | AssignmentDataFetcher | +| Notifications | NotificationDataFetcher | +| Tools & Scripts | ToolsDataFetcher, ScriptDataFetcher | +| Relay Node | NodeDataFetcher | + +### GraphQL Query Flow + +```mermaid +flowchart TD + Query["GraphQL Query"] --> DataFetcher["@DgsQuery / @DgsMutation"] + DataFetcher --> Mapper["GraphQL Mapper"] + Mapper --> Service["Domain Service"] + Service --> Repository["Mongo Repository"] + Repository --> Service + Service --> Mapper + Mapper --> Connection["Connection / Edge DTO"] +``` + +--- + +## Relay and Global ID Model + +This module implements Relay-style node resolution. + +### Global ID Handling + +- IDs are encoded using `Relay.toGlobalId(type, id)` +- Resolved using `Relay.fromGlobalId(id)` +- Centralized resolution via `NodeDataFetcher` + +### Node Type Resolution + +```mermaid +flowchart TD + NodeQuery["node(id)"] --> NodeFetcher["NodeDataFetcher"] + NodeFetcher --> TypeSwitch["NodeTypeResolver"] + TypeSwitch --> Machine["Machine"] + TypeSwitch --> Organization["Organization"] + TypeSwitch --> Event["Event"] + TypeSwitch --> Ticket["Ticket"] + TypeSwitch --> KnowledgeBaseItem["KnowledgeBaseItem"] +``` + +This ensures consistent object lookup across all GraphQL types. + +--- + +## Pagination Model + +The module implements cursor-based pagination. + +### Core DTOs + +- `GenericEdge` +- `CountedGenericConnection` +- `ConnectionArgs` +- `CursorPaginationCriteria` + +### Pagination Flow + +```mermaid +flowchart LR + ClientArgs["first / after"] --> ConnectionArgs + ConnectionArgs --> CursorCriteria + CursorCriteria --> ServiceQuery + ServiceQuery --> QueryResult + QueryResult --> ConnectionDTO +``` + +This model provides: + +- Forward and backward pagination +- Filtered count tracking +- Stable cursors + +--- + +## DataLoader Layer + +To prevent N+1 query issues in GraphQL, the module defines multiple DataLoaders. + +### Examples + +- `MachineDataLoader` +- `OrganizationDataLoader` +- `UserDataLoader` +- `KnowledgeBaseAttachmentDataLoader` +- `TagDataLoader` +- `TicketDataLoader` + +### N+1 Prevention Pattern + +```mermaid +flowchart TD + GraphQLField["Resolve field"] --> DataLoader + DataLoader --> BatchLoad["Batch Repository Query"] + BatchLoad --> ResultMap + ResultMap --> Response +``` + +Each DataLoader: + +- Accepts a list of IDs +- Performs a single batched query +- Returns results in original order + +--- + +## Knowledge Base Subsystem + +The Knowledge Base GraphQL implementation is one of the most complex subsystems. + +Capabilities include: + +- Folder and article tree structure +- Tag assignment +- Assignment to devices / tickets / organizations +- Archive and publish lifecycle +- Temporary attachment upload +- Permanent attachment linking + +It demonstrates: + +- Use of Relay global IDs +- DataLoader integration +- Mutation error wrapping +- Author resolution via `UserDataLoader` + +--- + +## Notification Subsystem + +The Notification GraphQL implementation supports: + +- Cursor-based notification listing +- Category-based unread counters +- Actor-type resolution (USER vs AGENT) +- Bulk mark-as-read and delete operations + +Security is enforced using method-level authorization: + +- `@PreAuthorize("hasAnyAuthority('ADMIN', 'AGENT')")` + +Recipient resolution depends on `AuthPrincipal` and JWT claims. + +--- + +## SSO Configuration Management + +The SSOConfigService manages: + +- Provider enable/disable +- Client secret encryption +- Domain validation +- Microsoft-specific tenant validation +- Post-save processing hooks + +The service integrates with: + +- EncryptionService +- SSOConfigRepository +- SSOConfigProcessor + +This allows OSS deployments to override behavior via conditional beans. + +--- + +## Extensibility Model + +Several extension points exist: + +| Extension Type | Mechanism | +|----------------|-----------| +| Post-processing hooks | `@ConditionalOnMissingBean` processors | +| SSO provider strategies | Strategy pattern | +| User enrichment | `UserProcessor` | +| Agent secret lifecycle | `AgentRegistrationSecretProcessor` | + +This enables SaaS or enterprise editions to inject custom behavior. + +--- + +## Relationship to Other Modules + +The Api Service Core Http And Graphql module depends on: + +- Authorization Service Core (JWT issuance and OAuth2 flows) +- Gateway Service Core (edge authentication and routing) +- Data Models and Repositories Mongo (persistence layer) +- Stream Processing Kafka (event ingestion and enrichment) +- Api Lib Contracts (shared DTOs) + +It serves as the primary application API surface consumed by the Frontend Core UI and Chat module. + +--- + +## Summary + +The **Api Service Core Http And Graphql** module is the application-layer backbone of OpenFrame. + +It provides: + +- REST APIs for operational management +- A fully Relay-compliant GraphQL API +- Cursor-based pagination +- DataLoader batching +- Minimal but robust OAuth2 Resource Server security +- Extensible processor-based hooks + +It sits between the Gateway and domain services, translating transport-layer requests into structured domain interactions while maintaining consistent API contracts across the platform. diff --git a/docs/reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md b/docs/reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md deleted file mode 100644 index 5e9693efc..000000000 --- a/docs/reference/architecture/api-service-core-relay-type-resolution/api-service-core-relay-type-resolution.md +++ /dev/null @@ -1,276 +0,0 @@ -# Api Service Core Relay Type Resolution - -## Overview - -The **Api Service Core Relay Type Resolution** module is responsible for resolving polymorphic GraphQL types in the Api Service Core layer. It bridges: - -- Mongo domain model objects (e.g., `Machine`, `Organization`, `Ticket`) -- GraphQL interfaces and unions (e.g., `Node`, `AssignableTarget`) -- The Netflix DGS runtime type resolution mechanism - -In a Relay-compliant GraphQL architecture, interfaces such as `Node` and domain-specific unions such as `AssignableTarget` require runtime type resolution. This module provides that resolution logic via DGS `@DgsTypeResolver` components. - ---- - -## Why This Module Exists - -GraphQL interfaces and unions require a mechanism to determine the concrete GraphQL type for a returned object at runtime. - -For example: - -- A `Node` query may return a `Machine`, `User`, `Ticket`, or `Organization`. -- An `AssignableTarget` may represent an `Organization`, `Machine`, `Ticket`, or `KnowledgeBaseItem`. - -The GraphQL engine must map Java domain objects to their corresponding GraphQL schema type names. - -The **Api Service Core Relay Type Resolution** module provides that mapping in a centralized and explicit way. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Client["GraphQL Client"] --> Query["GraphQL Query"] - Query --> DGS["DGS Runtime"] - DGS --> Resolver["Type Resolver"] - Resolver --> DomainObj["Domain Object"] - DomainObj --> GraphQLType["GraphQL Type Name"] - - subgraph relay_layer["Relay Type Resolution Layer"] - Resolver - end - - subgraph domain_layer["Mongo Domain Model"] - DomainObj - end -``` - -### Flow Summary - -1. A GraphQL query resolves a field returning an interface or union. -2. DGS invokes the registered `@DgsTypeResolver`. -3. The resolver inspects the Java object using `instanceof` checks. -4. It returns the GraphQL type name as a `String`. -5. The runtime binds the object to the correct schema type. - ---- - -## Core Components - -This module contains two DGS type resolvers: - -- `AssignableTargetTypeResolver` -- `NodeTypeResolver` - -Both are annotated with `@DgsComponent` and use `@DgsTypeResolver`. - ---- - -## AssignableTargetTypeResolver - -**Component:** -`openframe-oss-lib.openframe-api-service-core.src.main.java.com.openframe.api.relay.AssignableTargetTypeResolver.AssignableTargetTypeResolver` - -### Responsibility - -Resolves the GraphQL union/interface `AssignableTarget` into a concrete schema type. - -### Supported Domain Types - -| Java Type | GraphQL Type Returned | -|------------|-----------------------| -| `Organization` | `Organization` | -| `Machine` | `Machine` | -| `Ticket` | `Ticket` | -| `KnowledgeBaseItem` | `KnowledgeBaseItem` | - -If an unsupported object is passed, the resolver throws: - -```text -IllegalArgumentException: Unknown AssignableTarget type -``` - -### Resolution Logic - -```mermaid -flowchart TD - Input["AssignableTarget Object"] --> CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| Org["Return Organization"] - CheckOrg -->|"No"| CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| Machine["Return Machine"] - CheckMachine -->|"No"| CheckTicket{"Ticket?"} - CheckTicket -->|"Yes"| Ticket["Return Ticket"] - CheckTicket -->|"No"| CheckKB{"KnowledgeBaseItem?"} - CheckKB -->|"Yes"| KB["Return KnowledgeBaseItem"] - CheckKB -->|"No"| Error["Throw IllegalArgumentException"] -``` - -### Architectural Role - -This resolver enables: - -- Polymorphic assignment targets -- Unified assignment models across multiple entity types -- Strong alignment between domain model and GraphQL schema - ---- - -## NodeTypeResolver - -**Component:** -`openframe-oss-lib.openframe-api-service-core.src.main.java.com.openframe.api.relay.NodeTypeResolver.NodeTypeResolver` - -### Responsibility - -Resolves the Relay `Node` interface to a concrete GraphQL type. - -This is central to: - -- Global object identification -- Relay-style pagination -- Cross-entity querying - -### Supported Domain Types - -| Java Type | GraphQL Type Returned | -|------------|-----------------------| -| `Machine` | `Machine` | -| `Organization` | `Organization` | -| `Event` | `Event` | -| `IntegratedTool` | `IntegratedTool` | -| `Tenant` | `Tenant` | -| `ItemAssignment` | `ItemAssignment` | -| `Ticket` | `Ticket` | -| `KnowledgeBaseItem` | `KnowledgeBaseItem` | -| `User` | `User` | - -If an unsupported object is encountered, an exception is thrown: - -```text -IllegalArgumentException: Unknown Node type -``` - -### Resolution Flow - -```mermaid -flowchart TD - NodeInput["Node Object"] --> CheckMachine{"Machine?"} - CheckMachine -->|"Yes"| ReturnMachine["Return Machine"] - CheckMachine -->|"No"| CheckOrg{"Organization?"} - CheckOrg -->|"Yes"| ReturnOrg["Return Organization"] - CheckOrg -->|"No"| CheckEvent{"Event?"} - CheckEvent -->|"Yes"| ReturnEvent["Return Event"] - CheckEvent -->|"No"| Continue["Other instanceof checks"] - Continue --> ReturnType["Return Matching Type"] -``` - ---- - -## Integration with GraphQL Layer - -This module works closely with: - -- [Api Service Core GraphQL Datafetchers](../api-service-core-graphql-datafetchers/api-service-core-graphql-datafetchers.md) -- [Api Service Core GraphQL Dtos](../api-service-core-graphql-dtos/api-service-core-graphql-dtos.md) - -### Interaction Diagram - -```mermaid -flowchart LR - DataFetcher["DataFetcher"] --> Domain["Domain Object"] - Domain --> Resolver["Node or AssignableTarget Resolver"] - Resolver --> SchemaType["GraphQL Schema Type"] - SchemaType --> Response["GraphQL Response"] -``` - -1. A DataFetcher returns a domain entity. -2. The entity implements or represents a `Node` or `AssignableTarget`. -3. DGS invokes the appropriate resolver. -4. The resolver returns the GraphQL type name. -5. The object is serialized correctly into the GraphQL response. - ---- - -## Relationship to the Domain Model - -This module depends directly on the Mongo domain model defined in: - -- Device entities (`Machine`) -- Organization entities (`Organization`, `Tenant`) -- User entities (`User`) -- Event entities (`Event`) -- Ticket entities (`Ticket`) -- Knowledge base entities (`KnowledgeBaseItem`) -- Tool entities (`IntegratedTool`) -- Assignment entities (`ItemAssignment`) - -These classes originate from the data layer and are not GraphQL-specific. - -The resolvers form a clean boundary between: - -```mermaid -flowchart TB - DomainModel["Mongo Domain Model"] --> RelayResolver["Relay Type Resolver"] - RelayResolver --> GraphQLSchema["GraphQL Schema Types"] -``` - -This keeps domain objects independent from GraphQL schema annotations. - ---- - -## Design Characteristics - -### 1. Explicit Type Mapping - -- No reflection-based resolution -- No schema name inference -- Explicit `instanceof` checks - -This ensures: - -- Predictable behavior -- Easy debugging -- Compile-time safety for supported types - -### 2. Fail-Fast Strategy - -Unknown types result in immediate `IllegalArgumentException`. - -Benefits: - -- Prevents silent schema mismatches -- Forces developers to update resolvers when introducing new Node types - -### 3. Relay Compliance - -By resolving the `Node` interface properly, the module enables: - -- Global ID-based queries -- Cursor-based pagination -- Cross-type collections - ---- - -## When to Update This Module - -You must update **Api Service Core Relay Type Resolution** when: - -- A new domain type implements the `Node` interface in the GraphQL schema -- A new entity becomes assignable under `AssignableTarget` -- A new Mongo document class is exposed through GraphQL - -Failure to update this module will result in runtime exceptions. - ---- - -## Summary - -The **Api Service Core Relay Type Resolution** module is a critical but focused infrastructure component that: - -- Implements GraphQL Relay runtime type resolution -- Maps Mongo domain entities to GraphQL schema types -- Enables polymorphic querying -- Enforces strict type safety and fail-fast behavior - -Although small in size, it is foundational to the correctness of the GraphQL execution layer in Api Service Core. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md b/docs/reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md deleted file mode 100644 index 19b8e7b66..000000000 --- a/docs/reference/architecture/api-service-core-rest-controllers/api-service-core-rest-controllers.md +++ /dev/null @@ -1,425 +0,0 @@ -# Api Service Core Rest Controllers - -The **Api Service Core Rest Controllers** module exposes the primary internal REST endpoints of the OpenFrame API Service Core. It acts as the HTTP boundary layer between clients (UI, agents, internal services) and the underlying application services, command/query services, and domain logic. - -This module is responsible for: - -- Exposing secure REST endpoints for tenant-scoped operations -- Delegating business logic to dedicated service layers -- Translating HTTP semantics into domain/service calls -- Enforcing authentication context via `AuthPrincipal` -- Returning DTO-based responses for consistency and API stability - -It complements the GraphQL data fetchers and external API controllers by providing internal and operational REST endpoints. - ---- - -## Architectural Role in the Platform - -Within the overall OpenFrame architecture, the Api Service Core Rest Controllers module sits at the edge of the API Service Core and depends on: - -- Application services (command/query services) -- Domain services and processors -- Security context (`AuthPrincipal`) -- Mongo-backed persistence modules -- Tenant-aware authorization infrastructure - -```mermaid -flowchart TD - Client["Client / UI / Agent"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] - - subgraph rest_layer["REST Controller Layer"] - Controllers["Api Service Core Rest Controllers"] - end - - subgraph service_layer["Service Layer"] - CommandServices["Command Services"] - QueryServices["Query Services"] - DomainProcessors["Domain Processors"] - end - - subgraph data_layer["Data Layer"] - Mongo["Mongo Repositories"] - Redis["Redis Cache"] - Kafka["Kafka / Events"] - end - - ApiCore --> Controllers - Controllers --> CommandServices - Controllers --> QueryServices - Controllers --> DomainProcessors - - CommandServices --> Mongo - QueryServices --> Mongo - DomainProcessors --> Kafka - DomainProcessors --> Redis -``` - -The controllers themselves contain minimal business logic and primarily orchestrate calls to services. - ---- - -## Controller Overview - -The module contains the following REST controllers: - -- AgentRegistrationSecretController -- ApiKeyController -- DeviceController -- ForceAgentController -- HealthController -- InvitationController -- MeController -- OpenFrameClientConfigurationController -- OrganizationController -- ReleaseVersionController -- SSOConfigController -- UserController - -Each controller is scoped to a specific functional domain. - ---- - -# Endpoint Domains - -## 1. Agent Registration Secret - -**Base Path:** `/agent/registration-secret` - -Controller: `AgentRegistrationSecretController` - -Responsibilities: - -- Retrieve active registration secret -- List all historical secrets -- Generate new registration secret - -```mermaid -sequenceDiagram - participant Admin - participant Controller as AgentRegistrationSecretController - participant Service as AgentRegistrationSecretService - - Admin->>Controller: POST /agent/registration-secret/generate - Controller->>Service: generateNewSecret() - Service-->>Controller: AgentRegistrationSecretResponse - Controller-->>Admin: 201 Created -``` - -This endpoint is typically used during agent provisioning and secure enrollment flows. - ---- - -## 2. API Key Management - -**Base Path:** `/api-keys` - -Controller: `ApiKeyController` - -Key features: - -- List user API keys -- Create new API key -- Update metadata -- Delete key -- Regenerate secret - -Authentication is derived from `AuthPrincipal`, ensuring API keys are scoped to the authenticated user. - -```mermaid -flowchart LR - User["Authenticated User"] --> Controller["ApiKeyController"] - Controller --> Service["ApiKeyService"] - Service --> Repo["BaseApiKeyRepository"] -``` - -Security Characteristics: - -- User-scoped access -- Regeneration rotates secret while preserving key identity -- Creation returns secret only once - ---- - -## 3. Device Status Updates - -**Base Path:** `/devices` - -Controller: `DeviceController` - -Primary responsibility: - -- Update device status via `PATCH /devices/{machineId}` - -This is typically invoked internally by agents or system processes to reflect device health or connectivity state. - -```mermaid -flowchart TD - Agent["Agent"] --> Controller["DeviceController"] - Controller --> Service["DeviceService"] - Service --> DeviceDoc["Device Document"] -``` - ---- - -## 4. Force Agent Operations - -**Base Path:** `/force` - -Controller: `ForceAgentController` - -Supports operational commands such as: - -- Force tool installation -- Force tool reinstallation -- Force tool update -- Force client update -- Bulk operations ("all") - -These endpoints delegate to: - -- ForceToolInstallationService -- ForceClientUpdateService -- ForceToolAgentUpdateService - -```mermaid -flowchart TD - Admin["Admin Action"] --> Controller["ForceAgentController"] - Controller --> InstallSvc["ForceToolInstallationService"] - Controller --> UpdateSvc["ForceToolAgentUpdateService"] - Controller --> ClientSvc["ForceClientUpdateService"] - - InstallSvc --> Kafka["Kafka Event"] - UpdateSvc --> Kafka - ClientSvc --> Kafka -``` - -These operations are typically asynchronous and propagate through event pipelines. - ---- - -## 5. Health Check - -**Path:** `/health` - -Controller: `HealthController` - -- Lightweight liveness endpoint -- Returns `200 OK` with body `OK` -- Used by orchestrators and load balancers - ---- - -## 6. Invitations - -**Base Path:** `/invitations` - -Controller: `InvitationController` - -Supports: - -- Create invitation -- Paginated listing -- Revoke invitation -- Resend invitation - -```mermaid -sequenceDiagram - participant Admin - participant Controller as InvitationController - participant Service as InvitationService - - Admin->>Controller: POST /invitations - Controller->>Service: createInvitation(request) - Service-->>Controller: InvitationResponse - Controller-->>Admin: 201 Created -``` - -Invitation flows integrate with SSO and tenant onboarding subsystems. - ---- - -## 7. Current User Context - -**Path:** `/me` - -Controller: `MeController` - -Purpose: - -- Exposes authenticated user context -- Returns identity, roles, tenant ID -- Returns 401 if no authenticated principal - -```mermaid -flowchart TD - Request["GET /me"] --> AuthCheck["AuthPrincipal Present?"] - AuthCheck -->|"Yes"| Response["Return User Info"] - AuthCheck -->|"No"| Unauthorized["401 Unauthorized"] -``` - -This endpoint is commonly used by frontend applications to bootstrap user state. - ---- - -## 8. OpenFrame Client Configuration - -**Base Path:** `/openframe-client/configuration` - -Controller: `OpenFrameClientConfigurationController` - -Provides configuration metadata used by the OpenFrame client application. - -Delegates to: - -- OpenFrameClientConfigurationQueryService - ---- - -## 9. Organization Mutations - -**Base Path:** `/organizations` - -Controller: `OrganizationController` - -Handles: - -- Create organization -- Update organization -- Update status (ACTIVE / ARCHIVED) -- Check if archivable - -```mermaid -flowchart TD - Admin["Admin"] --> Controller["OrganizationController"] - Controller --> CommandSvc["OrganizationCommandService"] - Controller --> DomainSvc["OrganizationService"] - CommandSvc --> Repo["Organization Repository"] -``` - -Archiving rules: - -- Cannot archive if active devices exist -- May return `409 Conflict` - -Read operations are intentionally separated into external-facing modules. - ---- - -## 10. Release Version - -**Base Path:** `/release-version` - -Controller: `ReleaseVersionController` - -Responsibilities: - -- Return current platform release metadata -- Respond with 404 if not present - -Used by: - -- UI build metadata -- Agent compatibility checks -- Monitoring tools - ---- - -## 11. SSO Configuration - -**Base Path:** `/sso` - -Controller: `SSOConfigController` - -Supports: - -- List enabled providers -- List available providers -- Retrieve configuration -- Create or update provider config -- Toggle enablement -- Delete configuration - -```mermaid -flowchart TD - Admin["Admin"] --> Controller["SSOConfigController"] - Controller --> Service["SSOConfigService"] - Service --> Strategy["Provider Strategy"] - Strategy --> Provider["Google / Microsoft"] -``` - -This integrates with the Authorization Service Core and OAuth infrastructure. - ---- - -## 12. User Management - -**Base Path:** `/users` - -Controller: `UserController` - -Supports: - -- Paginated listing -- Get by ID -- Update user -- Soft delete user - -```mermaid -flowchart LR - Admin["Admin"] --> Controller["UserController"] - Controller --> Service["UserService"] - Service --> Repo["User Repository"] -``` - -Soft deletion ensures audit integrity and traceability. - ---- - -# Security Model - -All controllers (except `/health`) rely on Spring Security and JWT-based authentication. - -Authentication Flow: - -```mermaid -flowchart TD - Request["Incoming HTTP Request"] --> Filter["JWT Filter"] - Filter --> Principal["AuthPrincipal"] - Principal --> Controller["REST Controller"] -``` - -Key characteristics: - -- Tenant-aware security context -- Role-based authorization -- Principal injection via `@AuthenticationPrincipal` -- Clear separation between authentication and business logic - ---- - -# Design Principles - -The Api Service Core Rest Controllers module follows these principles: - -1. Thin controllers (no heavy business logic) -2. Explicit HTTP semantics (correct status codes) -3. DTO-based contract isolation -4. Clear separation of command vs query concerns -5. Tenant-aware multi-organization architecture - ---- - -# Summary - -The **Api Service Core Rest Controllers** module is the internal REST faΓ§ade of the OpenFrame API Service Core. It orchestrates: - -- Identity-scoped user operations -- Organization and tenant management -- API key lifecycle -- SSO configuration -- Agent lifecycle control -- Operational management endpoints - -It serves as a critical integration layer between authenticated clients and the underlying domain, persistence, and event-driven infrastructure, ensuring a clean, secure, and maintainable API boundary. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md b/docs/reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md deleted file mode 100644 index 4943e9a41..000000000 --- a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/api-service-core-user-sso-services-and-processors.md +++ /dev/null @@ -1,183 +0,0 @@ -# Api Service Core User Sso Services And Processors - -## Overview - -The **Api Service Core User Sso Services And Processors** module encapsulates user management, Single Sign-On (SSO) configuration, domain validation, and extensible post-processing hooks for user- and identity-related workflows inside the API Service Core. - -It acts as the orchestration layer between: - -- REST and GraphQL controllers in Api Service Core -- Mongo persistence (User, Invitation, SSOConfig, AgentRegistrationSecret) -- Encryption and domain validation services -- External Authorization Service Core (SSO, tenant, OAuth flows) - -This module is designed for **extensibility**. All major lifecycle events (user updates, SSO config changes, invitation creation, agent secret generation) are delegated to pluggable processor interfaces with default no-op implementations. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Controller["REST / GraphQL Controllers"] --> UserService["UserService"] - Controller --> SSOService["SSOConfigService"] - - UserService --> UserRepo["UserRepository"] - UserService --> UserProcessor["UserProcessor"] - - SSOService --> SSORepo["SSOConfigRepository"] - SSOService --> Encryption["EncryptionService"] - SSOService --> DomainValidation["DomainValidationService"] - SSOService --> SSOProcessor["SSOConfigProcessor"] - - subgraph processors_layer["Post-Processing Layer"] - UserProcessor --> DefaultUserProcessor["DefaultUserProcessor"] - SSOProcessor --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] - InvitationProcessor --> DefaultInvitationProcessor["DefaultInvitationProcessor"] - AgentSecretProcessor --> DefaultAgentProcessor["DefaultAgentRegistrationSecretProcessor"] - end -``` - -### Key Responsibilities - -- Manage users (query, update, soft delete) -- Manage SSO provider configuration (CRUD + enable/disable) -- Enforce domain validation rules for auto-provisioned SSO -- Provide extension points for SaaS or enterprise overrides - ---- - -## Sub-Modules - -To keep responsibilities clean and extensible, this module can be logically divided into two sub-modules: - -### 1. Services - -Core business services handling user and SSO logic. - -πŸ‘‰ See: [Services](api-service-core-user-sso-services-and-processors/services/services.md) - -### 2. Processors - -Lifecycle hooks triggered after core operations (update, delete, toggle, create). - -πŸ‘‰ See: [Processors](api-service-core-user-sso-services-and-processors/processors/processors.md) - ---- - -## Domain Validation Strategy - -Domain validation plays a critical role in SSO auto-provisioning. - -```mermaid -flowchart TD - Request["SSOConfigRequest"] --> Normalize["Normalize Domains"] - Normalize --> GenericCheck["validateGenericPublicDomain()"] - GenericCheck --> ExistsCheck["validateExists()"] - ExistsCheck --> Decision{"Auto Provision?"} - Decision -->|"No"| Save["Save Config"] - Decision -->|"Yes"| RequireDomain["Require At Least One Domain"] - RequireDomain --> MicrosoftCheck{"Microsoft?"} - MicrosoftCheck -->|"Yes"| RequireTenant["Require msTenantId"] - MicrosoftCheck -->|"No"| Save - RequireTenant --> Save -``` - -### DefaultDomainExistenceValidator - -The default implementation always returns `false`, meaning: - -- OSS deployments do not block SSO domain configuration -- SaaS deployments can override with a stricter validator - -This is implemented using `@ConditionalOnMissingBean`, ensuring easy replacement. - ---- - -## User Lifecycle - -```mermaid -flowchart TD - ListUsers["listUsers()"] --> FetchPage["UserRepository.findAll()"] - FetchPage --> MapResponse["UserMapper.toResponse()"] - MapResponse --> PostProcess["UserProcessor.postProcessUserGet()"] - - DeleteUser["softDeleteUser()"] --> ValidateSelf["Prevent Self Delete"] - ValidateSelf --> ValidateOwner["Prevent OWNER Delete"] - ValidateOwner --> MarkDeleted["Set Status DELETED"] - MarkDeleted --> SaveUser["UserRepository.save()"] - SaveUser --> PostDelete["UserProcessor.postProcessUserDeleted()"] -``` - -### Soft Delete Rules - -- Users cannot delete themselves -- OWNER role accounts cannot be deleted -- Deletion sets status to `DELETED` instead of removing the record - -This preserves referential integrity and audit consistency. - ---- - -## SSO Configuration Lifecycle - -```mermaid -flowchart TD - Upsert["upsertConfig()"] --> Exists{"Config Exists?"} - Exists -->|"Yes"| Update["Update Existing"] - Exists -->|"No"| Create["Create New"] - Update --> Encrypt["Encrypt Client Secret"] - Create --> Encrypt - Encrypt --> Save["SSOConfigRepository.save()"] - Save --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] -``` - -### Important Behaviors - -- Client secrets are encrypted before persistence -- Decryption occurs only when returning editable admin configuration -- Enabling/disabling triggers processor hooks -- Deletion triggers processor hooks - ---- - -## Extensibility Model - -All processors use: - -- `@Component` -- `@ConditionalOnMissingBean` - -This allows: - -- OSS default no-op behavior -- SaaS override with custom implementations -- Multi-tenant side effects (auditing, events, provisioning, sync) - -This pattern ensures the module is both **framework-level stable** and **deployment-level customizable**. - ---- - -## Integration With Other Modules - -This module integrates closely with: - -- Authorization Service Core (SSO flows, tenant registration, OAuth) -- Data Mongo Domain Model (User, Invitation, SSOConfig documents) -- Api Service Core REST Controllers (UserController, SSOConfigController) - -It does not implement OAuth flows directly β€” it manages configuration and lifecycle hooks that support those flows. - ---- - -## Summary - -The **Api Service Core User Sso Services And Processors** module is the identity orchestration layer of the API service. It: - -- Centralizes user management -- Governs SSO provider configuration -- Enforces domain safety rules -- Provides structured extension hooks -- Enables SaaS override without modifying core logic - -Its clean separation between services and processors makes it highly maintainable and enterprise-ready. \ No newline at end of file diff --git a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/processors.md b/docs/reference/architecture/api-service-core-user-sso-services-and-processors/processors.md deleted file mode 100644 index a6f72e41b..000000000 --- a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/processors.md +++ /dev/null @@ -1,303 +0,0 @@ -# Processors - -The **Processors** module provides extension points for domain-specific post-processing logic within the API Service Core, specifically around user management, invitations, SSO configuration, and agent registration secrets. - -In OpenFrame’s modular architecture, Processors act as **hooks** that execute after core service operations complete. By default, they provide no-op (logging-only) implementations, but they are designed to be overridden in custom or enterprise deployments. - -This module lives under the *API Service Core – User, SSO Services and Processors* domain and works closely with the sibling Services module. - ---- - -## 1. Purpose and Design Philosophy - -The Processors module exists to: - -- Provide **post-operation hooks** for core domain actions -- Enable **custom business logic injection** without modifying core services -- Maintain **clean separation of concerns** between service logic and side effects -- Support OSS deployments with safe default behavior - -All default implementations are annotated with: - -- `@Component` -- `@ConditionalOnMissingBean` - -This ensures: - -- The default implementation is used in OSS environments -- Custom implementations can transparently override the default behavior - ---- - -## 2. High-Level Architecture - -The Processors module sits between: - -- Core domain services (UserService, SSOConfigService, etc.) -- Downstream side effects (notifications, audit, integrations, etc.) - -```mermaid -flowchart LR - Controller["REST or GraphQL Controller"] --> Service["Core Service"] - Service --> DomainModel["Domain Model"] - Service --> Processor["Processor Interface"] - Processor --> DefaultImpl["Default Implementation"] - Processor --> CustomImpl["Custom Implementation (Optional)"] -``` - -### Key Characteristics - -- Services remain pure and focused on business rules -- Processors encapsulate side effects -- Spring’s conditional bean loading determines the active implementation - ---- - -## 3. Core Components - -The Processors module includes four default implementations: - -- DefaultAgentRegistrationSecretProcessor -- DefaultInvitationProcessor -- DefaultSSOConfigProcessor -- DefaultUserProcessor - -Each implements its respective processor interface and logs events by default. - ---- - -## 4. Agent Registration Secret Processing - -### Class - -`DefaultAgentRegistrationSecretProcessor` - -### Responsibilities - -- Post-process generated agent registration secrets -- Post-process deactivated secrets -- Provide safe logging in OSS mode - -### Interaction Flow - -```mermaid -flowchart TD - Generator["Secret Generation Logic"] --> Persist["Persist AgentRegistrationSecret"] - Persist --> ProcessorCall["postProcessSecretGenerated()"] - ProcessorCall --> DefaultProcessor["DefaultAgentRegistrationSecretProcessor"] -``` - -### Extension Use Cases - -A custom implementation may: - -- Push secrets to an external vault -- Emit audit events -- Trigger management workflows -- Notify external systems - -By default, only debug logging is performed. - ---- - -## 5. Invitation Processing - -### Class - -`DefaultInvitationProcessor` - -### Responsibilities - -- Post-process invitation creation -- Post-process invitation revocation - -### Flow - -```mermaid -flowchart TD - InvitationService["Invitation Service"] --> SaveInvitation["Persist Invitation"] - SaveInvitation --> ProcessorHook["postProcessInvitationCreated()"] - ProcessorHook --> DefaultInvitationProcessor["DefaultInvitationProcessor"] -``` - -### Typical Customization Scenarios - -A custom processor may: - -- Send email notifications -- Publish events to Kafka or NATS -- Integrate with external identity providers -- Trigger onboarding workflows - -The default implementation logs metadata such as invitation ID and email. - ---- - -## 6. SSO Configuration Processing - -### Class - -`DefaultSSOConfigProcessor` - -### Responsibilities - -- Post-process SSO config save -- Post-process SSO config deletion -- Post-process SSO config toggle (enable/disable) - -### Flow - -```mermaid -flowchart TD - SSOService["SSOConfigService"] --> SaveConfig["Persist SSOConfig"] - SaveConfig --> ProcessorHook["postProcessConfigSaved()"] - ProcessorHook --> DefaultSSOProcessor["DefaultSSOConfigProcessor"] -``` - -### Integration Context - -SSO configuration impacts: - -- OAuth/OIDC flows -- Tenant authentication -- Authorization Server behavior - -A custom processor could: - -- Register dynamic OAuth clients -- Synchronize configuration with external IdPs -- Invalidate authentication caches - -The default implementation performs debug logging only. - ---- - -## 7. User Processing - -### Class - -`DefaultUserProcessor` - -### Responsibilities - -- Post-process user deletion -- Post-process user updates -- Post-process user fetch (single) -- Post-process user fetch (paged) - -### Flow Example: User Update - -```mermaid -flowchart TD - UserController["UserController"] --> UserService["UserService"] - UserService --> PersistUser["Persist User"] - PersistUser --> ProcessorHook["postProcessUserUpdated()"] - ProcessorHook --> DefaultUserProcessor["DefaultUserProcessor"] -``` - -### Data Types Involved - -- User (Mongo domain model) -- UserResponse (API DTO) -- UserPageResponse (paginated DTO) - -### Customization Scenarios - -A custom implementation might: - -- Emit audit logs -- Synchronize user state with external systems -- Trigger deprovisioning in downstream tools -- Enforce domain-level policies - -The default behavior is debug-level logging. - ---- - -## 8. Conditional Bean Strategy - -All default processors use: - -- `@ConditionalOnMissingBean(value = Interface.class, ignored = DefaultImpl.class)` - -This enables the following resolution model: - -```mermaid -flowchart TD - SpringContext["Spring Application Context"] --> CheckCustom["Custom Bean Present?"] - CheckCustom -->|"Yes"| UseCustom["Use Custom Implementation"] - CheckCustom -->|"No"| UseDefault["Use Default Implementation"] -``` - -This pattern ensures: - -- Clean override without modifying core modules -- Backward compatibility -- Pluggable architecture for enterprise extensions - ---- - -## 9. Relationship to Services Module - -The Processors module complements the Services module located at: - -- [Services](../services/services.md) - -### Responsibility Split - -```mermaid -flowchart LR - Services["Services Module"] -->|"Business Logic"| Domain["Domain Model"] - Services -->|"Calls"| Processors["Processors Module"] - Processors -->|"Side Effects"| ExternalSystems["External Systems"] -``` - -- Services handle validation, persistence, and domain rules -- Processors handle post-operation side effects - -This clear boundary keeps the architecture maintainable and extensible. - ---- - -## 10. How Processors Fit Into the Overall System - -Within the broader OpenFrame platform: - -- Controllers (REST / GraphQL) invoke Services -- Services mutate domain entities -- Processors execute post-commit logic -- External systems (notifications, auth server, integrations) react - -```mermaid -flowchart TD - API["API Layer"] --> ServiceLayer["Service Layer"] - ServiceLayer --> RepositoryLayer["Repository Layer"] - ServiceLayer --> ProcessorLayer["Processors"] - ProcessorLayer --> Integration["Integrations / Events / Audit"] -``` - -This modular layering: - -- Prevents service bloat -- Encourages testability -- Enables OSS and enterprise feature differentiation - ---- - -## 11. Summary - -The **Processors** module provides structured extension points for: - -- Agent registration secrets -- Invitations -- SSO configuration -- User lifecycle operations - -By default, implementations are safe and logging-only. In advanced deployments, these processors become powerful integration points that allow OpenFrame to plug into broader ecosystems without altering core service logic. - -The design promotes: - -- Clean architecture -- Pluggability -- Multi-tenant extensibility -- Enterprise customization without forking core modules diff --git a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/services.md b/docs/reference/architecture/api-service-core-user-sso-services-and-processors/services.md deleted file mode 100644 index 456083ead..000000000 --- a/docs/reference/architecture/api-service-core-user-sso-services-and-processors/services.md +++ /dev/null @@ -1,302 +0,0 @@ -# Services - -The **Services** module contains the core application services responsible for user management, Single Sign-On (SSO) configuration, and domain validation within the OpenFrame API Service Core. - -These services sit between controllers (REST and GraphQL) and the data layer, encapsulating business logic, validation, orchestration, and post-processing hooks. They integrate with repositories, processors, mappers, encryption utilities, and tenant/domain validation infrastructure. - ---- - -## 1. Module Responsibilities - -The Services module provides: - -- βœ… User lifecycle management (read, update, soft delete) -- βœ… SSO configuration management (CRUD, toggle, validation) -- βœ… Domain validation extension point (SaaS overrides) -- βœ… Post-processing hooks via processors -- βœ… Secure handling of secrets (via encryption service) - -Core components: - -- `DefaultDomainExistenceValidator` -- `SSOConfigService` -- `UserService` - ---- - -## 2. Architectural Position - -The Services module operates inside the API Service Core and collaborates with multiple surrounding modules. - -```mermaid -flowchart LR - subgraph Controllers["API Layer"] - RestControllers["REST Controllers"] - GraphQL["GraphQL DataFetchers"] - end - - subgraph ServicesLayer["Services Module"] - UserServiceNode["UserService"] - SSOServiceNode["SSOConfigService"] - DomainValidator["DefaultDomainExistenceValidator"] - end - - subgraph Processors["Processors"] - UserProcessorNode["UserProcessor"] - SSOProcessorNode["SSOConfigProcessor"] - end - - subgraph DataLayer["Data Layer"] - UserRepo["UserRepository"] - SSORepo["SSOConfigRepository"] - MongoDocs["Mongo Domain Models"] - end - - Controllers --> ServicesLayer - ServicesLayer --> Processors - ServicesLayer --> DataLayer -``` - -### Key Relationships - -- Controllers invoke Services for business operations. -- Services delegate side-effects to Processors. -- Services persist and query through repositories. -- Domain validation is pluggable and environment-aware. - ---- - -## 3. UserService - -`UserService` encapsulates all user-related domain logic for the API layer. - -### 3.1 Responsibilities - -- Retrieve users by ID or email -- Paginated listing -- Bulk fetch by IDs -- Update profile fields -- Soft delete users with safety checks -- Trigger lifecycle processors - -### 3.2 Core Dependencies - -- `UserRepository` -- `UserMapper` -- `UserProcessor` -- Mongo `User` document - -### 3.3 User Lifecycle Flow - -```mermaid -flowchart TD - Request["User Update Request"] --> Load["Load User from Repository"] - Load --> Check{"User Exists?"} - Check -->|"No"| Error["Throw IllegalArgumentException"] - Check -->|"Yes"| Modify["Apply Field Updates"] - Modify --> Save["Save via UserRepository"] - Save --> PostProcess["UserProcessor.postProcessUserUpdated()"] - PostProcess --> Response["Return UserResponse"] -``` - -### 3.4 Soft Delete Safety Rules - -Soft deletion includes strict business constraints: - -- A user **cannot delete themselves** β†’ `UserSelfDeleteNotAllowedException` -- A user with `OWNER` role cannot be deleted β†’ `OperationNotAllowedException` -- Deletion sets status to `DELETED` instead of removing the document - -```mermaid -flowchart TD - DeleteRequest["Soft Delete Request"] --> LoadUser["Load User"] - LoadUser --> SelfCheck{"Requester == Target?"} - SelfCheck -->|"Yes"| SelfError["Throw Self Delete Exception"] - SelfCheck -->|"No"| OwnerCheck{"Has OWNER Role?"} - OwnerCheck -->|"Yes"| OwnerError["Throw OperationNotAllowedException"] - OwnerCheck -->|"No"| MarkDeleted["Set Status = DELETED"] - MarkDeleted --> SaveDeleted["Save User"] - SaveDeleted --> PostDelete["UserProcessor.postProcessUserDeleted()"] -``` - -This ensures organizational integrity and prevents accidental tenant lockout. - ---- - -## 4. SSOConfigService - -`SSOConfigService` manages Single Sign-On provider configuration for the tenant. - -### 4.1 Responsibilities - -- List enabled providers -- Return editable configuration (including decrypted secret) -- Create or update provider config (upsert) -- Toggle provider enabled state -- Delete provider configuration -- Validate domain and provisioning constraints -- Trigger SSO lifecycle processors - -### 4.2 Core Dependencies - -- `SSOConfigRepository` -- `EncryptionService` -- `SSOProperties` -- `SSOConfigProcessor` -- `SSOConfigMapper` -- `DomainValidationService` - -### 4.3 SSO Configuration Flow - -```mermaid -flowchart TD - AdminRequest["Upsert SSOConfigRequest"] --> Validate["validateAutoProvision()"] - Validate --> Normalize["Normalize Domains"] - Normalize --> DomainValidation["DomainValidationService.validate*"] - DomainValidation --> Encrypt["Encrypt Client Secret"] - Encrypt --> SaveConfig["Save via SSOConfigRepository"] - SaveConfig --> PostProcess["SSOConfigProcessor.postProcessConfigSaved()"] - PostProcess --> ReturnResponse["Return SSOConfigResponse"] -``` - -### 4.4 Domain Validation Logic - -When `autoProvisionUsers` is enabled: - -- βœ… `allowedDomains` must not be empty -- βœ… Domains are normalized (trimmed, lowercased, deduplicated) -- βœ… Public/generic domains are rejected -- βœ… Domains must exist (pluggable validator) -- βœ… Microsoft provider requires `msTenantId` when auto-provisioning - -This ensures secure enterprise SSO onboarding. - -### 4.5 Secure Secret Handling - -- Client secrets are encrypted before persistence. -- Decryption occurs only when returning editable configuration. -- Encryption is delegated to `EncryptionService`. - ---- - -## 5. DefaultDomainExistenceValidator - -`DefaultDomainExistenceValidator` provides the baseline implementation for domain existence checks. - -### 5.1 Behavior - -```java -public boolean anyExists(List domains) { - return false; -} -``` - -### 5.2 Design Purpose - -- Acts as a **non-blocking default** implementation. -- Enabled via `@ConditionalOnMissingBean`. -- Allows SaaS or enterprise deployments to override with a stricter validator. - -```mermaid -flowchart LR - DomainService["DomainValidationService"] --> Interface["DomainExistenceValidator"] - Interface --> DefaultImpl["DefaultDomainExistenceValidator"] - Interface --> CustomImpl["SaaS Override Implementation"] -``` - -This pattern supports multi-tenant SaaS deployments where domain ownership must be verified. - ---- - -## 6. Cross-Cutting Patterns - -### 6.1 Processor Hook Pattern - -Services delegate side effects to processors after core operations: - -- `postProcessUserGet` -- `postProcessUserUpdated` -- `postProcessUserDeleted` -- `postProcessConfigSaved` -- `postProcessConfigDeleted` -- `postProcessConfigToggled` - -This keeps services focused on: - -- Validation -- Persistence -- Mapping - -While processors handle: - -- Audit logging -- External sync -- Notifications -- Policy enforcement - -```mermaid -sequenceDiagram - participant Controller - participant Service - participant Repository - participant Processor - - Controller->>Service: Update Request - Service->>Repository: Save Entity - Repository-->>Service: Saved Entity - Service->>Processor: postProcess(...) - Service-->>Controller: Response DTO -``` - ---- - -## 7. Integration with Other Modules - -The Services module integrates with: - -- API Service Core Controllers (REST & GraphQL) -- Mongo domain and repository modules -- Security and OAuth infrastructure -- Authorization Service for SSO flows -- Encryption and crypto services - -High-level integration view: - -```mermaid -flowchart TD - Client["Client Application"] --> Gateway["Gateway Service"] - Gateway --> ApiCore["API Service Core"] - ApiCore --> ServicesNode["Services Module"] - ServicesNode --> Mongo[("MongoDB")] - ServicesNode --> Authz["Authorization Service"] - ServicesNode --> Crypto["Encryption Service"] -``` - ---- - -## 8. Design Principles - -The Services module follows these architectural principles: - -- βœ… Clear separation of concerns -- βœ… Stateless service layer -- βœ… Repository-driven persistence -- βœ… DTO ↔ Entity mapping abstraction -- βœ… Post-operation extension hooks -- βœ… Pluggable domain validation -- βœ… Secure secret handling -- βœ… Defensive business rule enforcement - ---- - -# Summary - -The **Services** module forms the business logic core of user and SSO management within OpenFrame’s API Service Core. It ensures: - -- Secure and validated SSO onboarding -- Safe user lifecycle operations -- Tenant-aware domain validation -- Extensible post-processing -- Clean separation between API, business logic, and persistence layers - -It acts as the orchestration layer that binds controllers, repositories, processors, and security infrastructure into a cohesive, enterprise-ready service architecture. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md b/docs/reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md deleted file mode 100644 index 2445546a4..000000000 --- a/docs/reference/architecture/authorization-service-core-auth-controllers-and-dtos/authorization-service-core-auth-controllers-and-dtos.md +++ /dev/null @@ -1,393 +0,0 @@ -# Authorization Service Core Auth Controllers And Dtos - -## Overview - -The **Authorization Service Core Auth Controllers And Dtos** module exposes the public HTTP interface for tenant onboarding, login, invitation acceptance, password reset, SSO discovery, and tenant discovery within the OpenFrame multi-tenant authorization system. - -It acts as the **edge layer** of the Authorization Service Core, translating HTTP requests into domain-level service calls and orchestrating redirects, cookies, and validation logic. - -This module is tightly integrated with: - -- The authorization server and tenant context layer: - [Authorization Service Core Server And Tenant](../authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md) -- The SSO flow and security utilities layer (redirect helpers, auth state utilities, registration constants) -- The persistence and key management layer (Mongo authorization service, client repositories, tenant keys) - ---- - -## Architectural Role - -This module sits at the boundary between: - -- 🌐 External clients (browser, SPA, CLI, third-party tools) -- πŸ” Spring Security & OAuth2 Authorization Server -- 🏒 Multi-tenant domain services - -It contains: - -- REST controllers -- MVC login controller -- Request/response DTOs -- Validation annotations - -### High-Level Request Flow - -```mermaid -flowchart LR - Client["Browser / Client App"] --> Controllers["Auth Controllers"] - Controllers --> Services["Domain Services"] - Services --> Persistence["Authorization & Tenant Persistence"] - Services --> Security["Spring Security / OAuth2"] - Security --> Client -``` - -The controllers remain intentionally thin. All business logic is delegated to service-layer components such as: - -- `InvitationRegistrationService` -- `SsoInvitationService` -- `TenantRegistrationService` -- `TenantDiscoveryService` -- `PasswordResetService` - ---- - -# Controllers - -## 1. InvitationRegistrationController - -**Base path:** `/invitations` - -Handles invitation-based onboarding, both standard and SSO-based. - -### Endpoints - -#### `POST /invitations/accept` -Registers a user using an invitation token. - -- Input: `InvitationRegistrationRequest` -- Output: `AuthUser` -- Delegates to: `InvitationRegistrationService` - -#### `GET /invitations/accept/sso` -Initiates SSO-based invitation acceptance. - -Flow: - -```mermaid -sequenceDiagram - participant Browser - participant Controller as InvitationRegistrationController - participant SSOService as SsoInvitationService - participant IdP as External IdP - - Browser->>Controller: GET /invitations/accept/sso - Controller->>SSOService: startAccept(request) - SSOService-->>Controller: SsoAuthorizeData - Controller->>Browser: Set-Cookie + 303 Redirect - Browser->>IdP: OAuth2 Authorization Request -``` - -Key behaviors: - -- Clears previous auth state using `AuthStateUtils` -- Sets secure, HTTP-only SSO invitation cookie -- Performs 303 redirect using `Redirects.seeOther` -- Encodes error message and redirects to configured `openframe.auth.error-url` on failure - ---- - -## 2. TenantRegistrationController - -**Base path:** `/oauth` - -Responsible for creating new tenants via: - -- Password-based registration -- SSO-based registration - -### `POST /oauth/register` - -Registers a new tenant and initial user. - -- Input: `TenantRegistrationRequest` -- Output: `Tenant` -- Delegates to: `TenantRegistrationService` - -### `GET /oauth/register/sso` - -Initiates SSO tenant registration. - -```mermaid -flowchart TD - Start["User Clicks Register with SSO"] --> Clear["Clear Auth State"] - Clear --> StartReg["SsoTenantRegistrationService.startRegistration()"] - StartReg --> Cookie["Set COOKIE_SSO_REG"] - Cookie --> Redirect["Redirect to IdP"] -``` - -Security properties: - -- Secure cookies -- HTTP-only flags -- Tenant-scoped redirect path -- Centralized error handling - ---- - -## 3. LoginController - -MVC controller (not REST). - -### Endpoints - -- `GET /login` β†’ renders login page -- `GET /` β†’ renders index page - -Responsibilities: - -- Inject error message when `?error` query param is present -- Expose optional password reset URL from configuration - -This controller integrates with Spring Security login processing. - ---- - -## 4. PasswordResetController - -**Base path:** `/password-reset` - -Handles password recovery flows. - -### `POST /password-reset/request` - -- Input: `ResetRequest` -- Normalizes email to lowercase -- Delegates to `PasswordResetService.createResetToken` -- Returns HTTP 202 (Accepted) - -### `POST /password-reset/confirm` - -- Input: `ResetConfirm` -- Enforces strong password pattern validation -- Delegates to `PasswordResetService.resetPassword` -- Returns HTTP 204 (No Content) - -Password validation rules: - -- Minimum 8 characters -- At least one uppercase -- At least one lowercase -- At least one digit -- At least one special character - ---- - -## 5. SsoDiscoveryController - -**Base path:** `/sso/providers` - -Used by frontend before redirecting to SSO provider. - -### `GET /sso/providers/invite` - -- Requires `invitationId` -- Loads invitation via `InvitationValidator` -- Returns effective providers for tenant - -### `GET /sso/providers/registration` - -- Returns system default SSO providers -- Delegates to `SSOConfigService` - -Response model: - -```text -{ - "providers": ["google", "microsoft"] -} -``` - ---- - -## 6. TenantDiscoveryController - -**Base path:** `/tenant` - -Supports multi-tenant login UX. - -### `GET /tenant/discover?email=` - -Returns: - -- Whether accounts exist -- Associated tenant ID -- Available authentication providers - -Flow: - -```mermaid -flowchart LR - User["User enters email"] --> Controller["TenantDiscoveryController"] - Controller --> Service["TenantDiscoveryService"] - Service --> Response["TenantDiscoveryResponse"] -``` - -Used to determine: - -- Password login vs SSO -- Redirect to tenant-specific issuer - ---- - -# DTO Layer - -The DTOs define the external API contract and enforce validation constraints. - -## InvitationRegistrationRequest - -Extends `CoreUserRequest`. - -Fields: - -- `invitationId` (required) -- `switchTenant` (optional) - ---- - -## TenantRegistrationRequest - -Extends `CoreUserRequest`. - -Fields: - -- `email` -- `tenantName` -- `tenantDomain` -- `accessCode` - -Validation ensures: - -- Proper tenant domain format -- Organization name pattern restrictions - ---- - -## SsoTenantRegistrationInitRequest - -Used before redirecting to IdP. - -Fields: - -- `email` -- `tenantName` -- `tenantDomain` -- `provider` -- `redirectTo` - -This ensures the system has enough context to: - -- Create tenant if needed -- Attach SSO configuration -- Prepare OAuth2 client registration - ---- - -## SsoInvitationAcceptRequest - -Fields: - -- `invitationId` -- `provider` -- `switchTenant` -- `redirectTo` - -Used to bootstrap SSO flow for invited users. - ---- - -## PasswordResetDtos - -Two nested DTOs: - -- `ResetRequest` -- `ResetConfirm` - -Encapsulates strict password validation rules. - ---- - -## TenantDiscoveryResponse - -```text -{ - "email": "user@example.com", - "has_existing_accounts": true, - "tenant_id": "tenant123", - "auth_providers": ["password", "google"] -} -``` - ---- - -## TenantAvailabilityResponse - -Indicates whether a tenant domain is available and suggests alternatives. - ---- - -# Multi-Tenant & Security Integration - -This module works in conjunction with: - -- Tenant context resolution -- OAuth2 authorization server -- SSO provider strategies (Google, Microsoft) -- Registration processors -- Token generation and persistence - -### Cross-Module Interaction - -```mermaid -flowchart TD - Controllers["Auth Controllers"] --> TenantLayer["Tenant Context Layer"] - Controllers --> SSOFlow["SSO Flow & Handlers"] - Controllers --> AuthServer["OAuth2 Authorization Server"] - AuthServer --> Persistence["Mongo Authorization Service"] -``` - -The controllers do not directly manipulate: - -- JWT generation -- Key pairs -- OAuth token storage -- Client registration persistence - -These responsibilities are delegated to the lower layers of the Authorization Service Core. - ---- - -# Design Principles - -βœ… Thin controllers -βœ… Strong DTO validation -βœ… Clear separation of concerns -βœ… Secure cookie handling -βœ… Explicit redirect logic -βœ… Multi-tenant aware flows -βœ… SSO-provider abstraction - ---- - -# Summary - -The **Authorization Service Core Auth Controllers And Dtos** module defines the complete public-facing contract of the OpenFrame Authorization Service. - -It: - -- Exposes login, registration, invitation, and password reset APIs -- Coordinates SSO bootstrap and redirect logic -- Enforces strict validation rules -- Supports multi-tenant discovery and onboarding -- Bridges HTTP requests to secure, tenant-aware domain services - -This module is the entry point into the OpenFrame multi-tenant identity and authorization infrastructure. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md b/docs/reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md deleted file mode 100644 index abca1c21b..000000000 --- a/docs/reference/architecture/authorization-service-core-keys-and-authorization-persistence/authorization-service-core-keys-and-authorization-persistence.md +++ /dev/null @@ -1,355 +0,0 @@ -# Authorization Service Core Keys And Authorization Persistence - -## Overview - -The **Authorization Service Core Keys And Authorization Persistence** module is responsible for: - -- Generating and managing tenant-scoped RSA signing keys -- Converting RSA keys to and from PEM format -- Persisting OAuth2 registered clients -- Persisting OAuth2 authorizations (authorization codes, access tokens, refresh tokens) -- Preserving PKCE metadata across the authorization lifecycle - -This module forms the cryptographic and persistence backbone of the Authorization Server. It ensures that: - -- Each tenant has a secure signing key used for JWT issuance -- OAuth2 clients are stored and reconstructed correctly -- Authorization state (including PKCE parameters) survives restarts and distributed deployments - -It integrates tightly with Spring Authorization Server, MongoDB repositories, and the tenant-aware authorization server configuration. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - ClientApp["OAuth2 Client Application"] --> AuthServer["Authorization Server"] - - subgraph key_layer["Tenant Key Layer"] - TenantKeyService["TenantKeyService"] --> KeyGenerator["RsaAuthenticationKeyPairGenerator"] - TenantKeyService --> PemUtil["PemUtil"] - TenantKeyService --> TenantKeyRepo["TenantKeyRepository"] - TenantKeyService --> EncryptionService["EncryptionService"] - end - - subgraph client_layer["Client Registration Layer"] - RegisteredRepo["MongoRegisteredClientRepository"] --> MongoClientRepo["RegisteredClientMongoRepository"] - end - - subgraph auth_layer["Authorization Persistence Layer"] - AuthService["MongoAuthorizationService"] --> AuthMapper["MongoAuthorizationMapper"] - AuthService --> MongoAuthRepo["MongoOAuth2AuthorizationRepository"] - AuthMapper --> MongoEntity["MongoOAuth2Authorization"] - end - - AuthServer --> TenantKeyService - AuthServer --> RegisteredRepo - AuthServer --> AuthService -``` - -### Responsibilities by Layer - -| Layer | Responsibility | -|--------|----------------| -| Tenant Key Layer | RSA key generation, encryption, and retrieval per tenant | -| Client Registration Layer | Persistent storage of OAuth2 RegisteredClient definitions | -| Authorization Persistence Layer | Persistent storage of authorization codes, access tokens, refresh tokens, and PKCE metadata | - ---- - -## Tenant RSA Key Management - -### Purpose - -The Authorization Service must sign JWT access tokens and ID tokens. In a multi-tenant system, each tenant requires its own signing key to: - -- Isolate cryptographic trust boundaries -- Support per-tenant key rotation -- Prevent cross-tenant token misuse - -### Core Components - -- `PemUtil` -- `RsaAuthenticationKeyPairGenerator` -- `TenantKeyService` - ---- - -### PemUtil - -`PemUtil` is a low-level utility that: - -- Parses PEM-encoded RSA public/private keys -- Serializes RSA keys into PEM format -- Wraps base64 content into 64-character lines - -Supported formats: - -```text ------BEGIN PUBLIC KEY----- -(base64) ------END PUBLIC KEY----- - ------BEGIN PRIVATE KEY----- -(base64) ------END PRIVATE KEY----- -``` - -This class ensures interoperability with external JWK and JWT libraries. - ---- - -### RsaAuthenticationKeyPairGenerator - -This Spring component generates RSA key pairs using configurable properties: - -- Algorithm (for example, RSA) -- Key size (for example, 2048 or 4096) -- SecureRandom algorithm - -Generation flow: - -```mermaid -flowchart TD - Start["Generate Key Pair"] --> InitRandom["Initialize SecureRandom"] - InitRandom --> InitGenerator["Initialize KeyPairGenerator"] - InitGenerator --> Generate["Generate RSA KeyPair"] - Generate --> ConvertPem["Convert to PEM"] - ConvertPem --> CreateKid["Create kid UUID"] - CreateKid --> Result["Return AuthenticationKeyPair"] -``` - -Each generated key pair includes: - -- RSAPublicKey -- RSAPrivateKey -- Public PEM -- Private PEM -- `kid` (Key ID) prefixed with `kid-` - -The `kid` is later embedded into JWT headers. - ---- - -### TenantKeyService - -`TenantKeyService` provides the runtime contract used by the Authorization Server to retrieve signing keys. - -#### Responsibilities - -1. Retrieve active key for a tenant -2. Generate and persist a key if none exists -3. Encrypt private key material before persistence -4. Return a Nimbus `RSAKey` for signing - -#### getOrCreateActiveKey Flow - -```mermaid -flowchart TD - Request["Request RSAKey for Tenant"] --> CheckActive["Check Active Key Count"] - CheckActive --> Found{"Active Key Exists?"} - Found -->|"Yes"| LoadDoc["Load TenantKey Document"] - Found -->|"No"| Create["Generate New Key"] - Create --> Encrypt["Encrypt Private PEM"] - Encrypt --> Save["Save TenantKey"] - Save --> LoadDoc - LoadDoc --> Parse["Parse PEM to RSA Keys"] - Parse --> BuildJwk["Build Nimbus RSAKey"] - BuildJwk --> Return["Return RSAKey"] -``` - -#### Security Model - -- Public key is stored in plaintext PEM. -- Private key is encrypted using `EncryptionService` before persistence. -- Only decrypted in-memory during signing operations. - -If multiple active keys are detected for a tenant, a warning is logged to prevent `kid` mismatches. - ---- - -## OAuth2 Client Persistence - -### MongoRegisteredClientRepository - -This class implements Spring Authorization Server's `RegisteredClientRepository` interface. - -It acts as a bridge between: - -- Spring `RegisteredClient` -- Mongo `MongoRegisteredClient` document - -### Save Mapping - -On save: - -- Authentication methods are converted to string values -- Grant types are converted to string values -- Token TTL values are converted to seconds -- PKCE requirement flags are persisted - -### Reconstruction Mapping - -On retrieval: - -- `ClientSettings` is rebuilt -- `TokenSettings` is rebuilt -- Authentication methods and grant types are rehydrated -- Redirect URIs and scopes are restored - -This ensures full compatibility with Spring Authorization Server runtime expectations. - ---- - -## OAuth2 Authorization Persistence - -### Core Components - -- `MongoAuthorizationService` -- `MongoAuthorizationMapper` - -These classes implement and support Spring's `OAuth2AuthorizationService`. - ---- - -## MongoAuthorizationMapper - -This class converts between: - -- `OAuth2Authorization` (Spring domain model) -- `MongoOAuth2Authorization` (Mongo document) - -It persists: - -- Authorization grant type -- Authorization code -- Access token -- Refresh token -- State -- PKCE parameters -- Authorization request snapshot - -### PKCE Preservation Strategy - -PKCE parameters (`code_challenge`, `code_challenge_method`) are captured from: - -- OAuth2AuthorizationRequest additional parameters -- Authorization code metadata - -They are normalized and stored in: - -- `arAdditional` -- `authorizationCodeMetadata` - -When reconstructing: - -- PKCE keys are restored with underscore format -- Both dot and underscore variants are normalized -- OAuth2AuthorizationRequest is rebuilt - -This ensures PKCE validation works correctly during token exchange. - ---- - -## MongoAuthorizationService - -Implements `OAuth2AuthorizationService`. - -### Save Flow - -```mermaid -flowchart TD - SaveCall["Save Authorization"] --> ExtractPkce["Extract PKCE Parameters"] - ExtractPkce --> MapEntity["Map to Mongo Entity"] - MapEntity --> Persist["Save to Mongo"] - Persist --> Verify["Debug Verify PKCE"] -``` - -### Token Lookup Flow - -```mermaid -flowchart TD - Lookup["Find By Token"] --> TypeCheck{"Token Type?"} - TypeCheck -->|"Access"| AccessQuery["Find By Access Token"] - TypeCheck -->|"Refresh"| RefreshQuery["Find By Refresh Token"] - TypeCheck -->|"Code"| CodeQuery["Find By Authorization Code"] - TypeCheck -->|"Null"| MultiQuery["Try All Token Fields"] - AccessQuery --> MapBack["Map to Domain"] - RefreshQuery --> MapBack - CodeQuery --> MapBack - MultiQuery --> MapBack - MapBack --> ReturnAuth["Return OAuth2Authorization"] -``` - -It supports lookup by: - -- Access token -- Refresh token -- Authorization code -- State - -After retrieval, the entity is mapped back to `OAuth2Authorization`, ensuring: - -- PKCE metadata is restored -- Principal placeholder is recreated -- Tokens are rebuilt with correct expiry - ---- - -## End-to-End Authorization Lifecycle - -```mermaid -sequenceDiagram - participant User - participant AuthServer - participant KeyService - participant AuthService - participant MongoDB - - User->>AuthServer: Authorization Request with PKCE - AuthServer->>AuthService: Save OAuth2Authorization - AuthService->>MongoDB: Persist Authorization Code - AuthServer->>KeyService: Get Tenant Signing Key - KeyService->>MongoDB: Load or Create TenantKey - KeyService->>AuthServer: Return RSAKey - AuthServer->>User: Issue JWT Access Token - User->>AuthServer: Token Exchange - AuthServer->>AuthService: Find Authorization by Code - AuthService->>MongoDB: Retrieve Authorization - AuthServer->>User: Return Access and Refresh Tokens -``` - ---- - -## Security Considerations - -1. Private Key Encryption - - Private RSA keys are encrypted before Mongo persistence. - - Decrypted only when building the runtime RSAKey. - -2. PKCE Integrity - - PKCE values are preserved exactly as provided. - - Underscore normalization avoids validation mismatches. - -3. Token Expiry - - TTL values are enforced through stored expiration timestamps. - - Expired authorizations can be pruned by repository-level TTL logic. - -4. Multi-Tenant Isolation - - Each tenant has an isolated signing key. - - Each authorization record references its registered client. - ---- - -## Summary - -The **Authorization Service Core Keys And Authorization Persistence** module provides: - -- Cryptographically secure tenant-aware RSA key management -- Robust Mongo-backed OAuth2 client persistence -- Reliable authorization and token persistence -- Full PKCE support with metadata normalization -- Seamless integration with Spring Authorization Server - -It is the foundation that guarantees secure JWT issuance, correct OAuth2 behavior, and multi-tenant isolation within the OpenFrame Authorization Server architecture. diff --git a/docs/reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md b/docs/reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md deleted file mode 100644 index 5fbba1505..000000000 --- a/docs/reference/architecture/authorization-service-core-server-and-tenant/authorization-service-core-server-and-tenant.md +++ /dev/null @@ -1,405 +0,0 @@ -# Authorization Service Core Server And Tenant - -## Overview - -The **Authorization Service Core Server And Tenant** module is the heart of OpenFrame's multi-tenant identity and OAuth2 infrastructure. It implements a fully multi-tenant OAuth2 Authorization Server using Spring Authorization Server, with per-tenant key material, dynamic client registration, and tenant-aware authentication flows. - -This module is responsible for: - -- Acting as the OAuth2 / OIDC Authorization Server -- Issuing tenant-scoped JWT access tokens and refresh tokens -- Managing per-tenant signing keys (JWKS) -- Resolving tenant context for every request -- Integrating with SSO and dynamic OAuth2 clients -- Enforcing tenant isolation at the authentication and token layers - -It works closely with: - -- Data Mongo Domain Model (users, OAuth clients, tokens) -- Security Core And OAuth BFF (shared security constants and utilities) -- Gateway Service Core Security And Routing (JWT validation and issuer routing) -- Authorization Service Core SSO Flow And Utils (SSO login, provider strategies) - ---- - -## High-Level Architecture - -The module is composed of five core building blocks: - -- Authorization Server Configuration -- Default Security Configuration -- Dynamic Client Registration -- Tenant Context Management -- Tenant-Aware Key Infrastructure - -```mermaid -flowchart TD - Client["Browser or API Client"] --> Gateway["Gateway Service"] - Gateway --> Authz["Authorization Service Core Server And Tenant"] - - subgraph authz_core["Authorization Layer"] - TenantFilter["TenantContextFilter"] - AuthServerConfig["AuthorizationServerConfig"] - SecurityConfigBean["SecurityConfig"] - DynamicClientRepo["DynamicClientRegistrationRepository"] - TenantKeySvc["TenantKeyService"] - end - - Authz --> TenantFilter - TenantFilter --> AuthServerConfig - TenantFilter --> SecurityConfigBean - AuthServerConfig --> TenantKeySvc - SecurityConfigBean --> DynamicClientRepo - - AuthServerConfig --> JWKS["Per-Tenant JWKS"] - AuthServerConfig --> JWT["Signed JWT Tokens"] -``` - ---- - -## Multi-Tenant Design Principles - -The Authorization Service Core Server And Tenant module is built around strict tenant isolation: - -1. Every request must resolve a tenant identifier. -2. Every JWT contains a `tenant_id` claim. -3. Every tenant has its own RSA key pair. -4. OAuth2 client registrations are resolved dynamically per tenant. -5. User authentication always queries tenant-scoped user records. - -Tenant identity flows through the entire request lifecycle using a thread-local context. - ---- - -## Tenant Context Management - -### TenantContext - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.tenant.TenantContext.TenantContext` - -A lightweight `ThreadLocal` holder storing the current tenant ID for the duration of a request. - -Responsibilities: - -- Store tenant ID per thread -- Provide `setTenantId`, `getTenantId`, and `clear` -- Ensure isolation between concurrent requests - -```mermaid -flowchart LR - Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] - Filter --> ThreadLocal["TenantContext ThreadLocal"] - ThreadLocal --> Services["UserService and Key Services"] -``` - ---- - -### TenantContextFilter - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.tenant.TenantContextFilter.TenantContextFilter` - -This filter executes early in the filter chain and resolves the tenant ID using: - -- URL path prefix (for example `/tenantId/oauth2/authorize`) -- Query parameter `tenant` -- Existing HTTP session attribute - -Key behaviors: - -- Stores tenant ID in both session and `TenantContext` -- Prevents unsafe cross-tenant session reuse -- Allows special onboarding tenant switching -- Clears context after request completion - -Tenant resolution directly influences: - -- JWT signing key selection -- OAuth2 client registration -- User lookup and authentication - ---- - -## Authorization Server Configuration - -### AuthorizationServerConfig - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.AuthorizationServerConfig.AuthorizationServerConfig` - -This class configures Spring Authorization Server with multi-issuer support. - -### Key Responsibilities - -- Enables OIDC support -- Allows multiple issuers (`multipleIssuersAllowed(true)`) -- Configures OAuth2 endpoints -- Sets up JWT encoder and decoder -- Defines token customization logic -- Registers tenant-aware JWK source - -### Security Filter Chain (Order 1) - -```mermaid -flowchart TD - Request["OAuth2 Endpoint Request"] --> Match["Authorization Endpoints Matcher"] - Match --> Authenticated["Require Authentication"] - Authenticated --> JWTValidation["JWT Resource Server Enabled"] - JWTValidation --> Response["OAuth2 Response"] -``` - -This chain applies only to Authorization Server endpoints. - ---- - -## Per-Tenant Key Infrastructure - -### JWKSource and TenantKeyService Integration - -The `jwkSource` bean dynamically selects the RSA key based on the current tenant: - -```mermaid -flowchart LR - JWTRequest["JWKS or Token Signing"] --> TenantId["TenantContext.getTenantId()"] - TenantId --> KeyService["TenantKeyService.getOrCreateActiveKey"] - KeyService --> RSAKey["Tenant RSAKey"] - RSAKey --> JWKSet["JWKSet Returned"] -``` - -Important properties: - -- Each tenant has its own active RSA key -- Key ID (`kid`) is logged and exposed via JWKS -- If tenant ID is missing, the request fails - -This guarantees: - -- Cryptographic tenant isolation -- Independent key rotation per tenant -- Safe multi-issuer JWT validation - ---- - -## JWT Token Customization - -The module injects tenant and user metadata into access tokens. - -### Custom Claims Added - -When issuing an `access_token`, the following claims are added: - -- `tenant_id` -- `userId` -- `roles` - -Role logic: - -- If user has `OWNER`, `ADMIN` is implicitly included -- Roles are emitted as string names - -```mermaid -flowchart TD - Auth["Authenticated Principal"] --> Lookup["UserService.findActiveByEmailAndTenant"] - Lookup --> Claims["Add Claims to JWT"] - Claims --> Token["Signed Access Token"] -``` - -Additionally: - -- Refresh token grants update `lastLogin` -- Claims are derived from tenant-scoped `AuthUser` - -This ensures that downstream services (Gateway, API Service) can enforce tenant and role-based authorization purely from JWT claims. - ---- - -## User Authentication Integration - -### UserDetailsService - -Provides tenant-aware authentication: - -- Looks up users by email + tenant ID -- Converts roles into `ROLE_*` authorities -- Uses BCrypt for password hashing - -### AuthenticationManager - -Uses a `DaoAuthenticationProvider` wired with: - -- Tenant-aware `UserDetailsService` -- BCrypt `PasswordEncoder` - -This enables: - -- Form login -- Programmatic authentication -- Password-based fallback for SSO users - ---- - -## Default Security Configuration - -### SecurityConfig - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.SecurityConfig.SecurityConfig` - -This filter chain handles **non-Authorization Server endpoints** (Order 2). - -Responsibilities: - -- Form login configuration -- OAuth2 login configuration -- SSO failure handling -- Auto-provisioning of users -- Microsoft multi-tenant issuer validation - ---- - -### OAuth2 Login and SSO Integration - -Key integrations: - -- `SsoAuthorizationRequestResolver` -- `AuthSuccessHandler` -- Custom `JwtDecoderFactory` -- Auto-provisioning logic - -```mermaid -flowchart TD - User["User Clicks SSO"] --> OAuthLogin["oauth2Login()"] - OAuthLogin --> Resolver["SsoAuthorizationRequestResolver"] - Resolver --> Provider["External IdP"] - Provider --> Callback["OIDC Callback"] - Callback --> OidcService["Custom OidcUserService"] - OidcService --> AutoProvision["Auto Provision If Needed"] - AutoProvision --> Success["AuthSuccessHandler"] -``` - ---- - -## Auto-Provisioning Logic - -When a user logs in via OIDC: - -1. Resolve tenant from context -2. Extract email and provider -3. Load per-tenant SSO configuration -4. Validate allowed domains -5. Register or reactivate user if necessary -6. Assign ADMIN role during SSO registration - -Safeguards: - -- Does not block login if provisioning fails -- Honors global domain policy fallback -- Avoids duplicate image sync - -This tightly integrates identity provider flows with OpenFrame's tenant-based user model. - ---- - -## Dynamic Client Registration - -### DynamicClientRegistrationRepository - -Component: -- `openframe-oss-lib.openframe-authorization-service-core.src.main.java.com.openframe.authz.config.DynamicClientRegistrationRepository.DynamicClientRegistrationRepository` - -Implements `ClientRegistrationRepository` dynamically per tenant. - -Behavior: - -- Resolves tenant ID from: - - `TenantContext` - - HTTP session -- Delegates to `DynamicClientRegistrationService` -- Returns null if tenant is unresolved - -```mermaid -flowchart LR - OAuthFlow["OAuth2 Login Request"] --> Repo["DynamicClientRegistrationRepository"] - Repo --> TenantCtx["TenantContext"] - TenantCtx --> DynamicSvc["DynamicClientRegistrationService"] - DynamicSvc --> ClientReg["ClientRegistration"] -``` - -This allows: - -- Per-tenant SSO provider configuration -- Dynamic Google and Microsoft client resolution -- Runtime onboarding of new tenants - ---- - -## Multi-Issuer and Gateway Compatibility - -The Authorization Service Core Server And Tenant module is designed for compatibility with the Gateway Service: - -- Each tenant may have its own issuer URL -- JWT includes `tenant_id` -- JWKS endpoint is tenant-aware -- Gateway validates JWT using issuer-specific configuration - -This ensures horizontal scalability and safe routing across multiple tenants. - ---- - -## Security Characteristics - -- Per-tenant RSA key pairs -- BCrypt password hashing -- OIDC ID token validation -- Microsoft multi-tenant issuer pattern validation -- Session invalidation on unsafe tenant switch -- Strict JWT claim injection -- CSRF disabled for OAuth endpoints only - ---- - -## Request Lifecycle Summary - -```mermaid -flowchart TD - Step1["Incoming Request"] --> Step2["TenantContextFilter Resolves Tenant"] - Step2 --> Step3{{"OAuth2 Endpoint?"}} - Step3 -->|"Yes"| Step4["AuthorizationServerConfig"] - Step3 -->|"No"| Step5["SecurityConfig"] - Step4 --> Step6["JWT Issued with Tenant Claims"] - Step5 --> Step7["Authenticated Session or SSO"] - Step6 --> EndNode["Response"] - Step7 --> EndNode -``` - ---- - -## How This Module Fits Into OpenFrame - -Within the broader OpenFrame architecture: - -- It is the authoritative identity provider. -- It enforces tenant boundaries at authentication time. -- It supplies signed JWTs used by: - - API Service Core - - Gateway Service Core - - External API Service Core -- It integrates with Mongo repositories for user and OAuth persistence. - -Without this module, the multi-tenant trust boundary would not exist. - ---- - -## Conclusion - -The **Authorization Service Core Server And Tenant** module provides: - -- Multi-tenant OAuth2 Authorization Server -- Tenant-aware JWT issuance -- Per-tenant RSA key management -- Dynamic client registration -- SSO auto-provisioning -- Strict tenant isolation guarantees - -It forms the cryptographic and identity backbone of OpenFrame, enabling secure, scalable, and tenant-isolated authentication across the entire platform. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md b/docs/reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md deleted file mode 100644 index e5c3b06e0..000000000 --- a/docs/reference/architecture/authorization-service-core-sso-flow-and-utils/authorization-service-core-sso-flow-and-utils.md +++ /dev/null @@ -1,359 +0,0 @@ -# Authorization Service Core Sso Flow And Utils - -## Overview - -The **Authorization Service Core Sso Flow And Utils** module encapsulates the Single Sign-On (SSO) flow orchestration, post-authentication handling, provider registration strategies, extensibility processors, and supporting web/security utilities for the OpenFrame Authorization Server. - -It sits on top of the core authorization server and tenant infrastructure and focuses on: - -- Handling OAuth2/OIDC authentication success -- Driving SSO-based tenant registration and invitation flows -- Dynamically configuring Google and Microsoft OIDC providers -- Providing extensibility hooks for registration and user lifecycle events -- Offering security and redirect utilities for clean session handling - -This module is a critical bridge between: - -- The **Authorization Server Core** (multi-tenant, OAuth2, client registration) -- The **User and Tenant Services** -- External OIDC providers (Google, Microsoft) - ---- - -## Architectural Context - -At a high level, the module enhances the OAuth2 login pipeline and injects SSO-specific behaviors. - -```mermaid -flowchart TD - Browser["Browser"] -->|"OAuth2 Login"| AuthServer["Authorization Server"] - AuthServer -->|"Authentication Success"| AuthSuccessHandler["Auth Success Handler"] - - AuthSuccessHandler -->|"Delegate"| SsoFlowHandlers["SSO Flow Handlers"] - AuthSuccessHandler -->|"Update User"| UserService["User Service"] - - SsoFlowHandlers --> InviteHandler["Invite SSO Handler"] - SsoFlowHandlers --> TenantRegHandler["Tenant Registration SSO Handler"] - - InviteHandler --> InvitationService["Invitation Registration Service"] - TenantRegHandler --> TenantRegService["Tenant Registration Service"] - - AuthSuccessHandler --> RedirectsUtil["Redirect Utilities"] - AuthSuccessHandler --> AuthStateUtils["Auth State Utilities"] -``` - -The flow can be summarized as: - -1. User authenticates via OAuth2 (Google or Microsoft). -2. `AuthSuccessHandler` updates user metadata and delegates to SSO flow handlers. -3. Based on SSO cookies, the appropriate handler executes: - - Invitation-based onboarding - - Tenant self-registration -4. Registration services create or update tenant/user entities. -5. Cookies are cleared and user is redirected to the correct tenant context. - ---- - -## Authentication Success Orchestration - -### AuthSuccessHandler - -**Responsibility:** Central post-authentication entry point. - -Key behaviors: - -- Extracts tenant ID from `TenantContext` -- Resolves email from `Authentication` (OIDC or standard login) -- Updates `lastLogin` via `UserService` -- Optionally marks email as verified for trusted providers -- Delegates to SSO tenant registration success handler - -### Email Resolution Strategy - -Email resolution is provider-aware and handled via `OidcUserUtils`: - -```mermaid -flowchart TD - OidcUser["OIDC User Claims"] --> Email["email"] - OidcUser --> Preferred["preferred_username"] - OidcUser --> Upn["upn"] - OidcUser --> Unique["unique_name"] - - Email --> Resolved["Resolved Email"] - Preferred --> Resolved - Upn --> Resolved - Unique --> Resolved -``` - -Order of precedence: - -1. `email` -2. `preferred_username` -3. `upn` -4. `unique_name` - -This ensures compatibility with Google and Azure AD organizational accounts. - ---- - -## SSO Flow Handling - -The module supports two primary SSO-driven flows using cookie-based state management. - -### SsoRegistrationConstants - -Defines cookie names and special onboarding tenant identifiers: - -- `of_sso_reg` β†’ Tenant self-registration flow -- `of_sso_invite` β†’ Invitation-based onboarding -- `sso-onboarding` β†’ Temporary tenant context - ---- - -### InviteSsoHandler - -**Use Case:** User accepts an invitation and authenticates via SSO. - -Flow: - -```mermaid -flowchart TD - Cookie["Invite Cookie"] --> Decode["Decode Invite Payload"] - Oidc["OIDC User"] --> Extract["Extract Names and Email"] - Decode --> BuildReq["Build InvitationRegistrationRequest"] - Extract --> BuildReq - BuildReq --> Register["InvitationRegistrationService.registerByInvitation"] - Register --> Redirect["Clear Cookie and Redirect"] -``` - -Key characteristics: - -- Validates SSO session via `SsoCookieCodec` -- Generates a random password (UUID-based) -- Uses provider picture (if available) -- Optionally switches tenant -- Clears SSO cookie while preserving OAuth session continuity - ---- - -### TenantRegSsoHandler - -**Use Case:** New tenant self-registration via SSO. - -Flow: - -```mermaid -flowchart TD - Cookie["Registration Cookie"] --> Decode["Decode Tenant Payload"] - Oidc["OIDC User"] --> Email["Resolve Email"] - Decode --> Validate["Validate Tenant Name and Domain"] - Email --> BuildReq["Build TenantRegistrationRequest"] - Validate --> BuildReq - BuildReq --> Register["TenantRegistrationService.registerTenant"] - Register --> Redirect["Clear Cookie and Redirect"] -``` - -Key behaviors: - -- Requires tenant name and domain -- Normalizes domain to lowercase -- Generates secure random password -- Associates authenticated email with tenant - ---- - -## OIDC Provider Strategies - -The module uses a strategy pattern for dynamic client registration. - -### GoogleClientRegistrationStrategy - -- Provider ID: `google` -- Uses `GoogleSSOProperties` -- Extends base OIDC registration strategy -- Integrates with `SSOConfigService` - -### MicrosoftClientRegistrationStrategy - -- Provider ID: `microsoft` -- Uses `MicrosoftSSOProperties` -- Supports Azure AD OIDC configuration - -```mermaid -flowchart LR - SSOConfigService["SSO Config Service"] --> GoogleStrategy["Google Client Registration Strategy"] - SSOConfigService --> MicrosoftStrategy["Microsoft Client Registration Strategy"] - - GoogleStrategy --> GoogleProps["Google SSO Properties"] - MicrosoftStrategy --> MicrosoftProps["Microsoft SSO Properties"] -``` - -This design enables: - -- Per-tenant provider configuration -- Default provider fallbacks -- Clean extension for additional IdPs - ---- - -## Default Provider Configuration - -### GoogleDefaultProviderConfig -### MicrosoftDefaultProviderConfig - -Provide default client ID and secret values when no tenant-specific configuration exists. - -This ensures: - -- Zero-config local development -- Sensible fallback for multi-tenant environments - ---- - -## Domain Policy Lookup - -### NoopGlobalDomainPolicyLookup - -Fallback implementation for domain-based auto-provisioning. - -- Returns empty result by default -- Activated only if no custom `GlobalDomainPolicyLookup` bean exists -- Enables domain-to-tenant auto-binding customization - -This is a key extensibility point for enterprise auto-provisioning. - ---- - -## Registration and User Lifecycle Processors - -The module defines multiple processor extension points with default no-op implementations. - -### DefaultRegistrationProcessor - -Hooks: - -- `preProcessTenantRegistration` -- `postProcessTenantRegistration` -- `postProcessInvitationRegistration` -- `postProcessAutoProvision` - -Allows custom logic such as: - -- Audit logging -- License provisioning -- External system synchronization - -### DefaultUserDeactivationProcessor - -Triggered after user deactivation. - -### DefaultUserEmailVerifiedProcessor - -Triggered when user email is marked verified. - -These processors follow a conditional bean pattern: - -- If no custom bean exists β†’ default implementation is used -- If custom bean exists β†’ default is ignored - -This enables safe override without modifying core logic. - ---- - -## Security and Web Utilities - -### OidcUserUtils - -Provides claim resolution helpers: - -- Email resolution across providers -- Picture URL extraction -- Safe string claim parsing - -Ensures provider inconsistencies do not leak into business logic. - ---- - -### ResetTokenUtil - -Generates secure password reset tokens: - -- 32 random bytes -- URL-safe Base64 encoding without padding -- Uses `SecureRandom` - -This provides cryptographically strong reset tokens. - ---- - -### AuthStateUtils - -Handles authentication state cleanup: - -- Invalidates session -- Clears `JSESSIONID` -- Removes cookies at root and context path - -Used during logout or auth reset scenarios. - ---- - -### Redirects - -Utility for safe HTTP redirects: - -- 302 Found -- 303 See Other -- Root-based redirect support -- Context-aware URL construction - -Ensures: - -- Clean redirect semantics -- No manual URL concatenation -- Context-path safety - ---- - -## Complete SSO Registration Sequence - -```mermaid -sequenceDiagram - participant Browser - participant AuthServer as "Authorization Server" - participant Success as "Auth Success Handler" - participant Flow as "SSO Flow Handler" - participant Service as "Registration Service" - - Browser->>AuthServer: OAuth2 Login (Google or Microsoft) - AuthServer->>Success: onAuthenticationSuccess - Success->>Success: Update lastLogin - Success->>Flow: Delegate to flow handler - Flow->>Service: Register user or tenant - Service->>Flow: Return tenant context - Flow->>Browser: Redirect to target tenant -``` - ---- - -## Key Design Principles - -1. **Separation of concerns** – Authentication success handling is isolated from registration logic. -2. **Cookie-driven flow state** – SSO flow intent is persisted safely across OAuth redirects. -3. **Strategy pattern for IdPs** – Enables clean addition of new providers. -4. **Extensibility via processors** – Business-specific logic can be injected without modifying core. -5. **Provider-agnostic claim resolution** – Normalizes Google and Microsoft differences. -6. **Secure defaults** – Strong random token generation and strict cookie clearing. - ---- - -## Summary - -The **Authorization Service Core Sso Flow And Utils** module is the orchestration layer that transforms raw OAuth2/OIDC authentication into: - -- Tenant-aware onboarding -- Invitation-driven account activation -- Secure and extensible registration flows -- Clean post-authentication lifecycle updates - -It acts as the glue between authentication, tenant management, and user lifecycle processing, ensuring a secure, extensible, and multi-tenant-ready SSO experience across OpenFrame. \ No newline at end of file diff --git a/docs/reference/architecture/authorization-service-core/authorization-service-core.md b/docs/reference/architecture/authorization-service-core/authorization-service-core.md new file mode 100644 index 000000000..e1aca4036 --- /dev/null +++ b/docs/reference/architecture/authorization-service-core/authorization-service-core.md @@ -0,0 +1,380 @@ +# Authorization Service Core + +The Authorization Service Core module is the security backbone of OpenFrame. It implements a **multi-tenant OAuth2 Authorization Server**, supports **OIDC-based Single Sign-On (SSO)** (Google and Microsoft), and manages **tenant-scoped authentication, token issuance, and user lifecycle events**. + +This module is responsible for: + +- Acting as an OAuth2 / OpenID Connect Authorization Server +- Issuing JWT access and refresh tokens +- Managing tenant-aware signing keys +- Supporting dynamic OIDC client registration per tenant +- Handling SSO onboarding and invitation flows +- Managing password resets and user authentication +- Persisting OAuth2 authorizations in MongoDB + +It integrates tightly with the data layer (MongoDB repositories), the Gateway Service Core (JWT validation), and the API Service Core (resource protection). + +--- + +## High-Level Architecture + +```mermaid +flowchart LR + Browser["User Browser"] -->|"Login / OAuth2"| AuthServer["Authorization Service Core"] + AuthServer -->|"JWT Access Token"| Gateway["Gateway Service Core"] + Gateway -->|"Validated Request"| ApiService["API Service Core"] + + AuthServer -->|"Persist Authorizations"| Mongo[("MongoDB")] + AuthServer -->|"Load Users / Tenants"| DataLayer["Data Models & Repositories"] +``` + +### Responsibilities by Boundary + +| Layer | Responsibility | +|--------|---------------| +| Authorization Server | OAuth2, OIDC, JWT issuance | +| Security Layer | Form login, SSO login, JWT validation | +| Tenant Context | Multi-tenant request isolation | +| Key Management | Per-tenant RSA signing keys | +| Persistence | Mongo-based token + client storage | + +--- + +# Core Architectural Concepts + +## 1. Multi-Tenant Isolation + +Each request is bound to a **Tenant Context** stored in a `ThreadLocal`. + +```mermaid +flowchart TD + Request["Incoming HTTP Request"] --> Filter["TenantContextFilter"] + Filter --> Context["TenantContext (ThreadLocal)"] + Context --> AuthFlow["Authentication / Token Flow"] + AuthFlow --> Clear["TenantContext.clear()"] +``` + +### Key Components + +- `TenantContext` – Thread-local storage of current tenant ID +- `TenantContextFilter` – Extracts tenant ID from: + - URL path (e.g. `/sas/{tenant}/oauth2/...`) + - Query parameter `tenant` + - Existing HTTP session +- Session switching logic for SSO onboarding flows + +This ensures: + +- Tokens are tenant-scoped +- Keys are tenant-scoped +- Users are tenant-scoped + +--- + +## 2. OAuth2 Authorization Server Configuration + +The module uses Spring Authorization Server via `AuthorizationServerConfig`. + +### Capabilities + +- Multiple issuers allowed (multi-tenant support) +- OIDC enabled +- JWT resource server configuration +- Custom authentication entry point +- Tenant-aware JWK source + +```mermaid +flowchart TD + Config["AuthorizationServerConfig"] --> SecurityChain["SecurityFilterChain (Order 1)"] + SecurityChain --> OAuth2["OAuth2AuthorizationServerConfigurer"] + OAuth2 --> OIDC["OIDC Enabled"] + OAuth2 --> JWT["JWT Resource Server"] + Config --> JWKSource["Tenant-aware JWKSource"] + Config --> TokenCustomizer["JWT Token Customizer"] +``` + +--- + +## 3. Tenant-Aware JWT Signing + +Each tenant has its own RSA key pair. + +### Flow + +```mermaid +flowchart TD + TokenRequest["Access Token Request"] --> KeyService["TenantKeyService"] + KeyService --> Repo["TenantKeyRepository"] + KeyService --> Generator["RsaAuthenticationKeyPairGenerator"] + KeyService --> Encrypt["EncryptionService"] + KeyService --> JWT["RSAKey (JWK)"] + JWT --> Encoder["NimbusJwtEncoder"] +``` + +### Key Management Components + +- `TenantKeyService` + - Retrieves active key + - Generates key if missing + - Encrypts private key before storage +- `RsaAuthenticationKeyPairGenerator` + - Generates RSA key pair + - Assigns unique `kid` +- `PemUtil` + - Converts RSA keys to/from PEM + +This guarantees: + +- Token isolation between tenants +- Independent key rotation +- JWK endpoint serves tenant-specific key + +--- + +## 4. JWT Customization + +Access tokens are enriched via `OAuth2TokenCustomizer`. + +### Custom Claims Added + +- `tenant_id` +- `userId` +- `roles` + +Role logic: + +- `OWNER` automatically implies `ADMIN` +- Roles are emitted as strings + +```mermaid +flowchart TD + Principal["Authenticated User"] --> Customizer["OAuth2TokenCustomizer"] + Customizer --> Claims["Add Claims"] + Claims --> JWT["Signed JWT"] +``` + +--- + +# Security Configuration + +The `SecurityConfig` handles all **non-authorization-server endpoints**. + +## Responsibilities + +- Form login +- OAuth2 login (SSO) +- Microsoft issuer validation +- Auto-provisioning SSO users +- Authentication failure handling + +```mermaid +flowchart TD + User["User"] -->|"/login"| FormLogin["Form Login"] + User -->|"/oauth2/authorization/{provider}"| OAuthLogin["OIDC Login"] + + OAuthLogin --> DecoderFactory["Microsoft-Aware JWT Decoder"] + OAuthLogin --> OidcUserService["Custom OIDC User Service"] + OidcUserService --> AutoProvision["Auto Provisioning"] + AutoProvision --> UserService["UserService"] +``` + +--- + +# Single Sign-On (SSO) + +## Supported Providers + +- Google +- Microsoft + +Provider configuration includes: + +- Tenant-specific credentials +- Default system-level credentials +- Dynamic client registration + +### Dynamic Client Registration + +`DynamicClientRegistrationRepository` resolves clients at runtime using: + +- Current tenant +- Session fallback + +```mermaid +flowchart TD + OAuthFlow["OAuth2 Login Flow"] --> Repo["DynamicClientRegistrationRepository"] + Repo --> Context["TenantContext"] + Repo --> DynamicService["DynamicClientRegistrationService"] + DynamicService --> Client["ClientRegistration"] +``` + +--- + +# SSO Flows + +## 1. Tenant Registration via SSO + +Handled by: + +- `TenantRegistrationController` +- `TenantRegSsoHandler` + +Flow: + +```mermaid +flowchart TD + User --> Init["/oauth/register/sso"] + Init --> Cookie["Set SSO Registration Cookie"] + Cookie --> Provider["Redirect to Google/Microsoft"] + Provider --> Callback["OIDC Callback"] + Callback --> Handler["TenantRegSsoHandler"] + Handler --> TenantService["TenantRegistrationService"] + TenantService --> Redirect["Redirect to Tenant OAuth Flow"] +``` + +--- + +## 2. Invitation Acceptance via SSO + +Handled by: + +- `InvitationRegistrationController` +- `InviteSsoHandler` + +Flow: + +```mermaid +flowchart TD + User --> Accept["/invitations/accept/sso"] + Accept --> Cookie["Set Invite Cookie"] + Cookie --> Provider["Redirect to Provider"] + Provider --> Callback["OIDC Callback"] + Callback --> InviteHandler["InviteSsoHandler"] + InviteHandler --> InviteService["InvitationRegistrationService"] +``` + +--- + +# Password Reset Flow + +Implemented via `PasswordResetController`. + +```mermaid +flowchart TD + User --> Request["POST /password-reset/request"] + Request --> Service["PasswordResetService"] + Service --> Token["Generate Reset Token"] + + User --> Confirm["POST /password-reset/confirm"] + Confirm --> Validate["Validate Token"] + Validate --> Update["Update Password"] +``` + +Reset tokens are generated using secure random Base64 URL encoding. + +--- + +# Mongo-Based Authorization Persistence + +OAuth2 authorizations are stored in MongoDB. + +## Components + +- `MongoAuthorizationService` +- `MongoAuthorizationMapper` +- `MongoRegisteredClientRepository` + +```mermaid +flowchart TD + AuthServer["OAuth2Authorization"] --> Mapper["MongoAuthorizationMapper"] + Mapper --> Entity["MongoOAuth2Authorization"] + Entity --> Repo["MongoOAuth2AuthorizationRepository"] + Repo --> Mongo[("MongoDB")] +``` + +### Stored Artifacts + +- Authorization codes +- Access tokens +- Refresh tokens +- PKCE parameters +- Client settings + +PKCE parameters are preserved across serialization/deserialization. + +--- + +# Authentication Success Handling + +`AuthSuccessHandler` performs: + +- Update `lastLogin` +- Mark email verified when appropriate +- Delegate to SSO flow continuation handler + +This ensures login success does not disrupt SSO onboarding flows. + +--- + +# Default Extension Points + +The module provides default no-op processors: + +- `DefaultRegistrationProcessor` +- `DefaultUserDeactivationProcessor` +- `DefaultUserEmailVerifiedProcessor` + +These allow customization without modifying core logic. + +--- + +# How It Fits Into OpenFrame + +```mermaid +flowchart LR + Frontend["Frontend"] --> Gateway + Gateway -->|"Bearer Token"| Api + Gateway -->|"JWT Validation"| Authorization + Authorization -->|"Token Issuance"| Gateway + + Authorization --> Data + + subgraph Services["OpenFrame Services"] + Authorization["Authorization Service Core"] + Api["API Service Core"] + Gateway["Gateway Service Core"] + Data["Mongo Data Layer"] + end +``` + +### Summary of Role + +The Authorization Service Core: + +- Issues tenant-scoped JWTs +- Validates and manages OAuth2 flows +- Handles SSO onboarding and invitations +- Provides secure multi-tenant isolation +- Supplies JWK endpoints for downstream verification +- Persists token state in MongoDB + +It is the **identity and trust authority** for the entire OpenFrame platform. + +--- + +# Key Design Principles + +1. Tenant-first architecture +2. Stateless JWT with tenant-aware signing +3. Secure default flows +4. Extensible processors +5. Provider-aware OIDC validation +6. PKCE-first OAuth2 implementation + +--- + +# Conclusion + +The Authorization Service Core is a production-grade, multi-tenant identity provider tailored for OpenFrame. It integrates OAuth2, OIDC, tenant isolation, Mongo persistence, and extensible SSO flows into a cohesive authentication backbone. + +All other services in the platform rely on it for trust, token validation, and identity enforcement. \ No newline at end of file diff --git a/docs/reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md b/docs/reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md deleted file mode 100644 index 195ade979..000000000 --- a/docs/reference/architecture/data-kafka-configuration-and-retry/data-kafka-configuration-and-retry.md +++ /dev/null @@ -1,380 +0,0 @@ -# Data Kafka Configuration And Retry - -## Overview - -The **Data Kafka Configuration And Retry** module provides the foundational Kafka infrastructure for the OpenFrame OSS ecosystem. It encapsulates: - -- Multi-tenant Kafka configuration -- Custom Spring Boot auto-configuration for Kafka -- Topic provisioning and management -- Standardized Debezium message modeling -- Structured recovery handling for failed Kafka operations - -This module is designed to be reusable across services such as Stream Service, Management Service, and other components that rely on event-driven communication. - -It replaces default Spring Boot Kafka auto-configuration with a controlled, tenant-aware setup tailored for OpenFrame OSS. - ---- - -## Architectural Role in the Platform - -Within the overall system, this module acts as the **Kafka infrastructure layer** used by event-driven services. - -```mermaid -flowchart LR - AppService[Application Service] --> KafkaInfra[Data Kafka Configuration And Retry] - KafkaInfra --> KafkaCluster[Kafka Cluster] - KafkaCluster --> StreamService[Stream Service Core] - KafkaCluster --> OtherConsumers[Other Consumers] -``` - -### Responsibilities - -1. Provide tenant-aware Kafka configuration -2. Standardize producer and consumer factories -3. Enable dynamic topic auto-creation -4. Provide a canonical Debezium message wrapper -5. Handle Kafka publishing failures via recovery logic - ---- - -## Module Components - -The module consists of the following core components: - -| Component | Responsibility | -|------------|----------------| -| `KafkaTopicProperties` | Defines topic configuration and auto-creation behavior | -| `OssKafkaConfig` | Enables Kafka and disables default auto-configuration | -| `OssTenantKafkaAutoConfiguration` | Main bean wiring for producers, consumers, listeners, and admin | -| `OssTenantKafkaProperties` | Tenant-scoped Kafka configuration properties | -| `KafkaHeader` | Standardized Kafka header constants | -| `DebeziumMessage` | Generic wrapper for Debezium CDC events | -| `KafkaRecoveryHandlerImpl` | Structured recovery handler for failed Kafka publishes | - ---- - -# Configuration Architecture - -## Custom Kafka Bootstrapping - -The module disables Spring Boot's default `KafkaAutoConfiguration` and replaces it with a controlled configuration: - -```mermaid -flowchart TD - SpringBoot[Spring Boot] --> DefaultKafkaAutoConfig[KafkaAutoConfiguration] - DefaultKafkaAutoConfig -->|Excluded| OssKafkaConfig - OssKafkaConfig --> OssTenantKafkaAutoConfiguration - OssTenantKafkaAutoConfiguration --> ProducerFactory - OssTenantKafkaAutoConfiguration --> ConsumerFactory - OssTenantKafkaAutoConfiguration --> KafkaTemplate - OssTenantKafkaAutoConfiguration --> KafkaAdmin -``` - -### OssKafkaConfig - -- Enables `@EnableKafka` -- Excludes `KafkaAutoConfiguration` -- Ensures only OSS tenant configuration is active - -This prevents accidental configuration conflicts and guarantees predictable Kafka behavior. - ---- - -## Tenant-Aware Kafka Properties - -### OssTenantKafkaProperties - -Bound to: - -```text -spring.oss-tenant -``` - -This class wraps Spring's `KafkaProperties`, allowing full reuse of: - -- Producer properties -- Consumer properties -- Listener configuration -- Template configuration -- Admin configuration - -### Enabling Kafka - -Kafka is conditionally activated using: - -```text -spring.oss-tenant.enabled=true -``` - -If disabled, no Kafka beans are registered. - ---- - -# Topic Management - -## KafkaTopicProperties - -Bound to: - -```text -openframe.oss-tenant.kafka.topics -``` - -It supports: - -- Automatic topic creation -- Partition configuration -- Replication factor configuration - -### Structure - -```text -openframe: - oss-tenant: - kafka: - topics: - inbound: - device-events: - name: device.events - partitions: 3 - replication-factor: 2 -``` - -### Auto-Creation Flow - -```mermaid -flowchart TD - ConfigProps[KafkaTopicProperties] --> AdminBean[KafkaAdmin] - AdminBean --> TopicBuilder - TopicBuilder --> NewTopic - NewTopic --> KafkaCluster -``` - -If topic auto-creation is enabled: - -- `KafkaAdmin` registers `NewTopic` beans -- Topics are created on startup -- Logging confirms partition and replica configuration - ---- - -# Producer and Consumer Infrastructure - -## Producer Side - -### ProducerFactory - -- Key serializer: `StringSerializer` -- Value serializer: `JsonSerializer` -- Backed by Spring `KafkaProperties` - -### KafkaTemplate - -- Default topic configurable -- JSON payload support -- Used by higher-level producers - -### OssTenantKafkaProducer - -Created as a bean to provide a simplified abstraction over `KafkaTemplate`. - ---- - -## Consumer Side - -### ConsumerFactory - -- Key deserializer: `StringDeserializer` -- Value deserializer: `JsonDeserializer` -- Uses Spring-configured properties - -### Listener Container Factory - -Configurable: - -- Concurrency -- AckMode (defaults to RECORD) -- Poll timeout -- Idle event interval -- Container logging - -```mermaid -flowchart LR - KafkaCluster --> ListenerContainer - ListenerContainer --> ConsumerFactory - ConsumerFactory --> JsonDeserializer - ListenerContainer --> AckMode[RECORD Ack Mode] -``` - -The default acknowledgment mode ensures per-record reliability if not explicitly configured. - ---- - -# Debezium Event Modeling - -## DebeziumMessage - -The module defines a generic wrapper for Debezium Change Data Capture events. - -### Structure - -```mermaid -flowchart TD - DebeziumMessage --> Payload - Payload --> Before[Before State] - Payload --> After[After State] - Payload --> Source - Payload --> Operation - Payload --> Timestamp - Source --> Database - Source --> Table - Source --> Connector -``` - -### Supported Fields - -- `before` – state before change -- `after` – state after change -- `operation` – CDC operation (c, u, d) -- `ts_ms` – event timestamp -- `source` – connector metadata - - database - - table - - connector type - - schema - - snapshot flag - -This structure ensures: - -- Strong typing of CDC events -- Cross-service consistency -- Cleaner event enrichment in downstream consumers - ---- - -# Kafka Headers Standardization - -## KafkaHeader - -Defines common header keys used across producers and consumers. - -Currently: - -```text -message-type -``` - -This enables: - -- Event routing -- Polymorphic deserialization -- Handler selection in downstream services - ---- - -# Retry and Recovery Handling - -## KafkaRecoveryHandlerImpl - -This component implements structured error recovery for failed Kafka publish operations. - -### Behavior - -When publishing fails: - -- The exception is captured -- Topic and key are logged -- Error class and message are recorded -- Payload summary is logged -- Full stacktrace is attached - -```mermaid -flowchart TD - Producer --> KafkaCluster - KafkaCluster -->|Failure| RecoveryHandler - RecoveryHandler --> StructuredLog -``` - -### Logging Strategy - -The recovery handler logs: - -- `topic` -- `key` -- `errorClass` -- `errorMsg` -- `payload` - -This ensures observability without introducing immediate dead-letter queue complexity. - -It can later be extended to: - -- Publish to retry topics -- Send to dead-letter queues -- Trigger alerting systems - ---- - -# Integration with Other Modules - -The **Data Kafka Configuration And Retry** module serves as a foundational infrastructure layer for: - -- Stream processing services consuming Debezium events -- Management services publishing operational events -- Notification pipelines -- Cross-service event propagation - -It does not implement business logic itself. Instead, it standardizes: - -- How Kafka is configured -- How topics are created -- How messages are structured -- How failures are handled - ---- - -# Design Principles - -### 1. Tenant Isolation -Kafka configuration is explicitly scoped under: - -```text -spring.oss-tenant -``` - -This prevents accidental mixing with default Kafka configurations. - -### 2. Explicit Auto-Configuration -Default Spring Kafka auto-configuration is excluded to avoid: - -- Bean conflicts -- Hidden configuration overrides -- Environment-specific ambiguity - -### 3. Strong CDC Modeling -Debezium events are modeled as typed structures instead of raw maps. - -### 4. Safe Defaults -- Ack mode defaults to RECORD -- Admin auto-creation enabled unless disabled -- Kafka enabled by default but explicitly configurable - -### 5. Observability First -Failure handling prioritizes structured logging for operational visibility. - ---- - -# Summary - -The **Data Kafka Configuration And Retry** module provides a controlled, tenant-aware Kafka foundation for the OpenFrame OSS platform. - -It ensures: - -- Deterministic Kafka bootstrapping -- Configurable topic lifecycle management -- Structured Debezium message handling -- Safe producer/consumer defaults -- Clear recovery and error logging - -By centralizing Kafka configuration and retry behavior, the module guarantees consistency across all services that rely on event-driven communication. \ No newline at end of file diff --git a/docs/reference/architecture/data-models-and-repositories-mongo/data-models-and-repositories-mongo.md b/docs/reference/architecture/data-models-and-repositories-mongo/data-models-and-repositories-mongo.md new file mode 100644 index 000000000..5db954810 --- /dev/null +++ b/docs/reference/architecture/data-models-and-repositories-mongo/data-models-and-repositories-mongo.md @@ -0,0 +1,441 @@ +# Data Models And Repositories Mongo + +## Overview + +The **Data Models And Repositories Mongo** module provides the MongoDB-backed persistence layer for the OpenFrame platform. It defines: + +- Core MongoDB document models (users, tickets, devices, organizations, events, notifications, etc.) +- Query filter objects for advanced searching +- Base repository abstractions (technology-agnostic) +- Reactive and synchronous repository implementations +- Custom MongoTemplate-based query logic with cursor pagination +- Multi-tenant support via tenant-aware scoping + +This module is the foundation for higher-level modules such as: + +- API Service Core HTTP And GraphQL +- Authorization Service Core +- Management Service Core +- Gateway Service Core (indirectly via upstream services) + +It centralizes all MongoDB persistence logic to keep business services storage-agnostic. + +--- + +## High-Level Architecture + +```mermaid +flowchart TD + Controllers["API / GraphQL / Auth Controllers"] --> Services["Application Services"] + Services --> Repositories["Mongo Repositories"] + Repositories --> CustomImpl["Custom Repository Implementations"] + Repositories --> ReactiveImpl["Reactive Repositories"] + CustomImpl --> MongoTemplate["MongoTemplate"] + ReactiveImpl --> ReactiveMongo["ReactiveMongoRepository"] + MongoTemplate --> MongoDB[("MongoDB")] + ReactiveMongo --> MongoDB +``` + +### Responsibilities by Layer + +- **Document Models** β†’ Define MongoDB collections and indexes +- **Query Filters** β†’ Encapsulate search and filtering logic +- **Base Repositories** β†’ Provide technology-agnostic contracts +- **Reactive Repositories** β†’ Used by reactive services (e.g., auth flows) +- **Custom Repositories** β†’ Advanced queries, aggregation, cursor pagination +- **Tenant Provider** β†’ Injects tenant context into queries + +--- + +## Multi-Tenancy Model + +Most domain documents implement `TenantScoped`, containing a `tenantId` field indexed for isolation. + +```mermaid +flowchart LR + Request["Incoming Request"] --> TenantContext["TenantIdProvider"] + TenantContext --> Repository["Tenant-Scoped Repository"] + Repository --> Query["Mongo Query with tenantId"] + Query --> MongoDB[("MongoDB")] +``` + +### Tenant ID Resolution + +`DefaultTenantIdProvider`: + +- Reads tenant ID from `TENANT_ID` environment variable +- Defaults to `oss` +- Can be overridden by custom `TenantIdProvider` bean + +This enables: + +- SaaS multi-tenant isolation +- OSS single-tenant mode +- Authorization server tenant-aware domain resolution + +--- + +# Core Document Models + +## User and AuthUser + +### User +Collection: `users` + +Fields: +- `email` (indexed, normalized to lowercase) +- `roles` +- `status` +- `emailVerified` +- `createdAt`, `updatedAt` +- `tenantId` + +### AuthUser +Extends `User`. +Collection: `users` (same inheritance hierarchy). + +Additional fields: +- `passwordHash` +- `loginProvider` (LOCAL, GOOGLE, etc.) +- `externalUserId` +- `lastLogin` +- `imageUrl` (SSO profile cache) + +Compound index: + +```text +{ tenantId: 1, email: 1 } (unique) +``` + +Used heavily by: +- Authorization Service Core +- API Service Core + +--- + +## Organization + +Collection: `organizations` + +Key features: +- Unique `organizationId` +- Soft deletion (`status` = ACTIVE, ARCHIVED, DELETED) +- Contract lifecycle tracking +- Indexed search fields + +Business helpers: +- `isContractActive()` +- `isArchived()` +- `isDeleted()` + +Custom queries handled by `CustomOrganizationRepositoryImpl`. + +--- + +## Device + +Collection: `devices` + +Fields include: +- `machineId` +- `serialNumber` +- `status` +- `type` +- `lastCheckin` + +Indexed on `tenantId`. + +Used by: +- Device controllers +- Fleet integrations +- Management schedulers + +--- + +## Ticket + +Collection: `tickets` + +Compound indexes: + +```text +{ tenantId:1, ticketNumber:1 } (unique) +{ status:1, order:1 } +{ statusKind:1 } +{ statusId:1, order:1 } +``` + +Supports: +- Legacy `status` +- Lifecycle-based `statusId` and `statusKind` +- Cursor pagination +- Aggregation metrics +- Bulk updates + +`CustomTicketRepositoryImpl` provides: + +- Cursor-based pagination +- Aggregation counts by status +- Average resolution time +- Bulk status reassignment +- Search across title, device, org, assignee + +--- + +## CoreEvent + +Collection: `events` + +Fields: +- `type` +- `payload` +- `timestamp` +- `userId` +- `status` + +Used by: +- Stream Processing Kafka module +- Activity dashboards +- Audit queries + +`CustomEventRepositoryImpl` supports: +- Date range filtering +- Cursor pagination +- Distinct event types +- Distinct user IDs + +--- + +## Notification + +Collection: `notifications` + +Supports: +- Severity levels +- Context embedding +- Created date auto-population + +`CustomNotificationRepositoryImpl` provides: +- Recipient-based pagination +- Read/unread filtering +- Cursor-based navigation +- Status joining with read state + +--- + +# Query Filter Objects + +Filter objects encapsulate dynamic search logic. + +Examples: + +- `EventQueryFilter` +- `OrganizationQueryFilter` +- `TicketQueryFilter` +- `ToolQueryFilter` +- `UserQueryFilter` + +These objects: +- Avoid controller-level query construction +- Enable reusable MongoTemplate queries +- Support complex combinations of AND / OR criteria + +Example flow: + +```mermaid +flowchart TD + GraphQL["GraphQL Input Filter"] --> FilterObject["QueryFilter DTO"] + FilterObject --> CustomRepo["Custom Repository"] + CustomRepo --> MongoQuery["Mongo Query + Criteria"] + MongoQuery --> MongoDB[("MongoDB")] +``` + +--- + +# Repository Architecture + +## Base Repository Contracts + +Technology-agnostic interfaces: + +- `BaseUserRepository` +- `BaseTenantRepository` +- `BaseApiKeyRepository` +- `BaseIntegratedToolRepository` + +These abstract return types: + +- Blocking (`Optional`, `boolean`, `List`) +- Reactive (`Mono`, `Flux`) + +This enables both reactive and synchronous implementations. + +--- + +## Reactive Repositories + +Located under `openframe-data-mongo-reactive`. + +Examples: +- `ReactiveUserRepository` +- `ReactiveTenantRepository` +- `ReactiveOAuthClientRepository` + +Used primarily by: +- Authorization Service Core +- Reactive authentication flows + +Built on `ReactiveMongoRepository`. + +--- + +## Synchronous Configuration + +Two configurations control repository activation: + +- `MongoSyncConfig` +- `TenantAwareSyncConfig` + +Activation is controlled via properties: + +```text +spring.data.mongodb.enabled +openframe.tenant-isolation.enabled +``` + +This allows: +- Tenant-aware repositories +- Shared cluster mode +- Flexible deployment strategies + +--- + +# Custom Repository Implementations + +Custom implementations use `MongoTemplate` for advanced querying. + +Common capabilities: + +- Cursor-based pagination using `_id` +- Multi-field sorting +- Compound search with `$or` and `$and` +- Aggregation pipelines +- Distinct field retrieval +- Bulk updates + +## Cursor Pagination Pattern + +```mermaid +flowchart TD + Client["Client"] --> QueryBuilder["Build Query"] + QueryBuilder --> ApplyCursor["Apply _id comparison"] + ApplyCursor --> ApplySort["Apply Sort + Limit"] + ApplySort --> Execute["mongoTemplate.find"] + Execute --> Page["Result Page"] +``` + +Behavior: +- If sort field is `_id`, compare directly +- Otherwise: + - Compare primary sort field + - Tie-break using `_id` +- Invalid cursor logs warning and returns first page + +Implemented in: +- CustomMachineRepositoryImpl +- CustomTicketRepositoryImpl +- CustomEventRepositoryImpl +- CustomOrganizationRepositoryImpl +- CustomScriptRepositoryImpl +- CustomNotificationRepositoryImpl +- CustomIntegratedToolRepositoryImpl +- CustomUserRepositoryImpl + +--- + +# Aggregation and Metrics + +Ticket repository provides aggregation-based metrics: + +- Count by `status` +- Count by `statusKind` +- Count by `statusId` +- Average resolution time + +Uses MongoDB Aggregation Framework. + +```mermaid +flowchart LR + Tickets[("tickets collection")] --> Group["$group"] + Group --> Project["$project"] + Project --> Metrics["Aggregated Metrics"] +``` + +--- + +# Integration with Other Modules + +## API Service Core HTTP And GraphQL + +Uses: +- Document models +- Query filters +- Custom repositories +- Pagination utilities + +GraphQL resolvers delegate persistence to this module. + +## Authorization Service Core + +Uses: +- AuthUser document +- ReactiveUserRepository +- ReactiveTenantRepository +- MongoAuthorizationService + +Enables domain-based tenancy and OAuth client storage. + +## Management Service Core + +Uses: +- Device +- Ticket +- IntegratedTool +- Script repositories + +For schedulers, migrations, and background processing. + +## Stream Processing Kafka + +Consumes events and writes: +- CoreEvent documents +- Enriched activity records + +--- + +# Design Principles + +1. Clear separation between domain model and service logic +2. Technology-agnostic repository contracts +3. Multi-tenant safety by default +4. Cursor-based pagination for scalability +5. Aggregation-driven metrics +6. Index-first design +7. Reactive + synchronous support + +--- + +# Summary + +The **Data Models And Repositories Mongo** module is the persistence backbone of OpenFrame. + +It provides: + +- Strongly typed MongoDB documents +- Multi-tenant data isolation +- Reactive and blocking repository layers +- Advanced filtering and pagination +- Aggregation-based analytics +- Custom query implementations for complex domains + +All higher-level services rely on this module for consistent, scalable, and tenant-safe data access. \ No newline at end of file diff --git a/docs/reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md b/docs/reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md deleted file mode 100644 index 85d255373..000000000 --- a/docs/reference/architecture/data-mongo-base-repositories/data-mongo-base-repositories.md +++ /dev/null @@ -1,256 +0,0 @@ -# Data Mongo Base Repositories - -## Overview - -The **Data Mongo Base Repositories** module defines technology-agnostic repository contracts for core MongoDB-backed domain entities in the OpenFrame platform. These base interfaces act as a unifying abstraction layer between: - -- The **domain model** (Mongo documents such as User, Tenant, API Key, Integrated Tool) -- The **blocking (imperative)** repository implementations -- The **reactive (Project Reactor)** repository implementations - -By introducing generic wrapper types (`T`, `B`, `L`, `ID`), this module ensures that the same repository contract can be reused across both blocking and reactive stacks without duplicating business semantics. - ---- - -## Architectural Role in the Data Layer - -Within the broader persistence architecture, this module sits between the domain documents and concrete repository implementations. - -```mermaid -flowchart TD - Domain["Mongo Domain Model"] --> BaseRepo["Data Mongo Base Repositories"] - BaseRepo --> Blocking["Blocking Repositories"] - BaseRepo --> Reactive["Reactive Repositories"] - - Blocking --> Services["Application Services"] - Reactive --> Services -``` - -### Responsibilities - -- Define **common repository operations** for core entities -- Provide **generic signatures** adaptable to Optional / List or Mono / Flux -- Ensure **consistent query semantics** across execution models -- Reduce duplication between blocking and reactive repository layers - ---- - -## Design Philosophy - -Each base repository follows a consistent pattern: - -```text -public interface BaseXRepository { - T findBy...(...); - B existsBy...(...); - L findAllBy...(...); -} -``` - -Where: - -- `T` β†’ Wrapper for single-result queries - - Blocking: `Optional` - - Reactive: `Mono` -- `B` β†’ Wrapper for boolean responses - - Blocking: `boolean` - - Reactive: `Mono` -- `L` β†’ Wrapper for multi-result queries (if applicable) - - Blocking: `List` - - Reactive: `Flux` -- `ID` β†’ Identifier type (typically `String`) - -This allows downstream modules to: - -- Implement blocking repositories using Spring Data Mongo -- Implement reactive repositories using Spring Data Reactive Mongo -- Preserve identical method semantics across implementations - ---- - -# Core Interfaces - -## 1. BaseApiKeyRepository - -**Purpose:** Defines common API key query operations. - -### Key Methods - -- `findByIdAndUserId(String keyId, String userId)` -- `findByUserId(String userId)` -- `findExpiredKeys(Instant currentTime)` - -### Architectural Context - -```mermaid -flowchart LR - ApiKey["API Key Document"] --> ApiKeyRepo["BaseApiKeyRepository"] - ApiKeyRepo --> UserScope["User-Scoped Queries"] - ApiKeyRepo --> Expiry["Expiration Queries"] -``` - -### Design Considerations - -- API keys are **user-scoped**. -- Expiration handling is centralized via `findExpiredKeys`. -- Enables scheduled cleanup and security enforcement logic. - -This repository is critical for authentication flows and API access management. - ---- - -## 2. BaseTenantRepository - -**Purpose:** Defines tenant lookup and domain validation operations. - -### Key Methods - -- `findByDomain(String domain)` -- `existsByDomain(String domain)` - -### Architectural Context - -```mermaid -flowchart TD - Tenant["Tenant Document"] --> TenantRepo["BaseTenantRepository"] - TenantRepo --> DomainLookup["Domain-Based Resolution"] - DomainLookup --> AuthLayer["Authorization & SSO"] -``` - -### Design Considerations - -- Domain-based lookup enables **multi-tenant resolution**. -- Used heavily in authentication and tenant-aware request routing. -- `existsByDomain` supports validation during registration and onboarding flows. - ---- - -## 3. BaseIntegratedToolRepository - -**Purpose:** Provides lookup functionality for integrated tools. - -### Key Methods - -- `findByType(String type)` - -### Architectural Context - -```mermaid -flowchart LR - Tool["Integrated Tool Document"] --> ToolRepo["BaseIntegratedToolRepository"] - ToolRepo --> TypeLookup["Type-Based Resolution"] - TypeLookup --> Gateway["Gateway & Integration Layer"] -``` - -### Design Considerations - -- Tool resolution is based on logical tool type. -- Enables integration routing and upstream configuration. -- Keeps tool lookup semantics consistent across sync and reactive stacks. - ---- - -## 4. BaseUserRepository - -**Purpose:** Defines foundational user lookup and existence checks. - -### Key Methods - -- `findByEmail(String email)` -- `existsByEmail(String email)` -- `existsByEmailAndStatus(String email, UserStatus status)` - -### Architectural Context - -```mermaid -flowchart TD - UserDoc["User Document"] --> UserRepo["BaseUserRepository"] - UserRepo --> EmailLookup["Email-Based Lookup"] - UserRepo --> StatusCheck["Status Validation"] - StatusCheck --> Security["Authentication & Access Control"] -``` - -### Design Considerations - -- Email is the primary identity attribute. -- Status validation supports enforcement of states such as ACTIVE or DISABLED. -- Provides a consistent existence-check abstraction across blocking and reactive implementations. - ---- - -# Cross-Cutting Patterns - -## 1. Technology Agnostic Contracts - -All repositories avoid direct dependencies on: - -- Spring Data interfaces -- Reactive types -- Mongo-specific annotations - -Instead, they rely on generic wrapper types, allowing different execution models without rewriting business contracts. - ---- - -## 2. Multi-Tenancy Alignment - -The BaseTenantRepository and BaseUserRepository are foundational for tenant-aware identity resolution. - -```mermaid -flowchart TD - Request["Incoming Request"] --> DomainExtract["Extract Domain"] - DomainExtract --> TenantRepo["BaseTenantRepository"] - TenantRepo --> TenantResolved["Tenant Context Established"] - TenantResolved --> UserRepo["BaseUserRepository"] - UserRepo --> AuthResult["User Validated"] -``` - -This separation ensures that: - -- Tenant resolution is independent from authentication logic. -- User validation remains consistent regardless of execution model. - ---- - -## 3. Expiration and Lifecycle Handling - -Repositories such as BaseApiKeyRepository expose lifecycle-oriented queries (e.g., expired keys). This supports: - -- Scheduled cleanup tasks -- Security enforcement policies -- Token rotation and revocation strategies - ---- - -# Interaction with Other Data Modules - -Although this module only defines interfaces, it enables: - -- Concrete Mongo repository implementations in synchronization modules. -- Reactive repository implementations in reactive data modules. -- Service-layer logic in API and authorization services. - -```mermaid -flowchart LR - Base["Data Mongo Base Repositories"] --> SyncImpl["Mongo Sync Implementations"] - Base --> ReactiveImpl["Mongo Reactive Implementations"] - SyncImpl --> ServiceLayer["Service Layer"] - ReactiveImpl --> ServiceLayer -``` - -The base repository layer ensures consistent semantics regardless of whether the application is running in blocking or reactive mode. - ---- - -# Summary - -The **Data Mongo Base Repositories** module is a foundational abstraction layer in the OpenFrame persistence architecture. - -It provides: - -- Unified repository contracts -- Execution-model independence (blocking vs reactive) -- Tenant-aware and security-aware query primitives -- Reduced duplication across repository implementations - -By separating repository semantics from implementation details, this module ensures long-term maintainability, flexibility, and architectural consistency across the OpenFrame data stack. diff --git a/docs/reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md b/docs/reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md deleted file mode 100644 index 77d406e97..000000000 --- a/docs/reference/architecture/data-mongo-domain-model/data-mongo-domain-model.md +++ /dev/null @@ -1,471 +0,0 @@ -# Data Mongo Domain Model - -## Overview - -The **Data Mongo Domain Model** module defines the core MongoDB document structures used across the OpenFrame platform. It provides the canonical persistence model for: - -- Multi-tenant users and authentication -- Organizations and tenancy boundaries -- Devices and operational state -- Tickets and PSA workflows -- Events and audit trail -- Notifications and read states -- OAuth clients and tokens -- Tag assignments - -This module acts as the **persistence foundation** for API services, authorization services, stream processing, gateway routing, and management components. All higher-level services depend on these document definitions for consistent storage and indexing strategies. - ---- - -## Architectural Role in the Platform - -The Data Mongo Domain Model sits at the base of the application stack and is shared by: - -- API Service Core (REST + GraphQL) -- Authorization Service Core -- Stream Service Core (Kafka consumers) -- Management Service Core -- External API Service Core -- Gateway and Security components (indirectly via persistence) - -### High-Level Data Flow - -```mermaid -flowchart TD - Gateway["Gateway Service"] --> ApiService["API Service Core"] - ApiService --> DomainModel["Data Mongo Domain Model"] - AuthService["Authorization Service"] --> DomainModel - StreamService["Stream Service"] --> DomainModel - ManagementService["Management Service"] --> DomainModel - ExternalApi["External API Service"] --> DomainModel - - DomainModel --> MongoDB[("MongoDB")] -``` - -The module defines **document schemas and indexing strategies**, while repositories and services (in other modules) implement business logic on top of these models. - ---- - -## Core Domain Areas - -The domain model can be grouped into the following bounded contexts: - -1. Identity & Authentication -2. Organization & Tenancy -3. Device Management -4. Ticketing (PSA) -5. Events & Audit -6. Notifications -7. OAuth & Client Registration -8. Tagging System - ---- - -# 1. Identity & Authentication - -## User - -**Collection:** `users` - -Represents an application user within a tenant. - -Key characteristics: - -- Indexed by `email` -- Supports multiple roles (`UserRole`) -- Email normalization (lowercased, trimmed) -- Soft status control via `UserStatus` -- Audit fields (`createdAt`, `updatedAt`) - -### Responsibilities - -- Technician identity -- Role-based authorization support -- Assignment target for tickets -- Reporter reference for tickets - ---- - -## AuthUser - -Extends `User` for multi-tenant authorization server use. - -**Compound Index:** `(tenantId, email)` unique when tenant exists. - -Additional fields: - -- `tenantId` (multi-tenancy boundary) -- `passwordHash` -- `loginProvider` (LOCAL, GOOGLE, etc.) -- `externalUserId` -- `lastLogin` -- `imageUrl` (cached profile picture) - -### Purpose - -- Used by the Authorization Service -- Supports SSO and external identity providers -- Maintains domain-based tenancy - -### Identity Model Diagram - -```mermaid -classDiagram - class User { - id - email - firstName - lastName - roles - status - createdAt - updatedAt - } - - class AuthUser { - tenantId - passwordHash - loginProvider - externalUserId - lastLogin - imageUrl - } - - User <|-- AuthUser -``` - ---- - -# 2. Organization & Tenancy - -## Organization - -**Collection:** `organizations` - -Represents a company or tenant-scoped business entity. - -Key features: - -- Unique `organizationId` -- Indexed `name` -- `isDefault` flag -- Business metadata (revenue, employees, contract dates) -- Soft-delete via `OrganizationStatus` (ACTIVE, ARCHIVED, DELETED) -- Contract validation logic - -### Important Design Decision - -Organizations are **never hard-deleted** to preserve device and ticket references. Instead: - -- `ARCHIVED` β†’ hidden from standard queries -- `DELETED` β†’ soft-deleted but still referentially valid - ---- - -# 3. Device Management - -## Device - -**Collection:** `devices` - -Represents a managed endpoint. - -Key fields: - -- `machineId` (link to external Machine entity) -- `serialNumber`, `model`, `osVersion` -- `status` (ACTIVE, OFFLINE, MAINTENANCE) -- `DeviceType` -- `lastCheckin` -- `DeviceConfiguration` -- `DeviceHealth` - -### Device Lifecycle - -```mermaid -stateDiagram-v2 - [*] --> ACTIVE - ACTIVE --> OFFLINE - OFFLINE --> ACTIVE - ACTIVE --> MAINTENANCE - MAINTENANCE --> ACTIVE -``` - -Devices are heavily used by: - -- Ticketing -- Stream event ingestion -- Management schedulers -- External API integrations - ---- - -# 4. Ticketing (PSA Domain) - -Ticketing is modeled as a primary entity with related collections. - -## Ticket - -**Collection:** `tickets` - -Primary PSA entity. - -Key design elements: - -- Unique `ticketNumber` (auto-increment per tenant) -- Indexed status and assignment fields -- Linked to: - - `deviceId` - - `organizationId` - - `assignedTo` (User) - - `reporterId` -- Soft resolution via `resolvedAt` - -### Index Strategy - -Compound indexes support: - -- Status-based dashboards -- Assignment filtering -- Organization filtering -- Device filtering - ---- - -## TicketNote - -**Collection:** `ticket_notes` - -Technician-only internal notes. - -- Indexed by `ticketId` -- Sorted by `createdAt` -- Editable (tracked via `updatedAt`) - ---- - -## TicketAttachment - -**Collection:** `ticket_attachments` - -Metadata-only storage. - -Files are stored externally (S3/MinIO), while this document stores: - -- `ticketId` -- `fileName` -- `contentType` -- `fileSize` -- `storagePath` - ---- - -### Ticket Aggregate Model - -```mermaid -flowchart TD - Ticket["Ticket"] --> Note["TicketNote"] - Ticket --> Attachment["TicketAttachment"] - Ticket --> User["User (Assigned)"] - Ticket --> Device["Device"] - Ticket --> Org["Organization"] -``` - -Ticket owns metadata; attachments and notes are independent collections for scalability. - ---- - -# 5. Events & Audit - -## CoreEvent - -**Collection:** `events` - -Represents system-level or domain events. - -Fields: - -- `type` -- `payload` -- `timestamp` -- `userId` -- `status` (CREATED, PROCESSING, COMPLETED, FAILED) - -### Usage - -- Stream ingestion persistence -- Audit logs -- Async processing workflows - ---- - -# 6. Notifications - -## Notification - -**Collection:** `notifications` - -Represents a system notification. - -Fields: - -- `severity` -- `title` -- `description` -- `createdAt` -- `context` - ---- - -## NotificationReadState - -**Collection:** `notification_read_states` - -Tracks per-recipient state. - -Compound indexes ensure: - -- Unique (recipientId, recipientType, notificationId) -- Fast unread queries by status -- Filtering by category - -### Notification Model - -```mermaid -flowchart TD - Notification["Notification"] --> ReadState["NotificationReadState"] - ReadState --> Recipient["User or Organization"] -``` - -Notifications are immutable; read state is tracked separately. - ---- - -# 7. OAuth & Client Registration - -## MongoRegisteredClient - -**Collection:** `oauth_registered_clients` - -Represents OAuth2 clients. - -Key features: - -- Unique `clientId` -- Grant types -- Redirect URIs -- Scopes -- PKCE support -- Token TTL configuration - ---- - -## OAuthToken - -**Collection:** `oauth_tokens` - -Stores issued tokens. - -Fields: - -- `userId` -- `accessToken` -- `refreshToken` -- Expiry timestamps -- `clientId` -- `scopes` - -Used by Authorization Service for token validation and revocation. - ---- - -# 8. Tagging System - -## TagAssignment - -**Collection:** `tag_assignments` - -Provides a unified tagging mechanism across entities. - -Unique compound index: - -- `(entityId, tagId, entityType)` - -Supports: - -- Label-style tags -- Key-value tags with `values` -- Timestamped tagging (`taggedAt`) -- Attribution (`taggedBy`) - -### Cross-Entity Tag Model - -```mermaid -flowchart LR - TagAssignment["TagAssignment"] --> Device - TagAssignment --> Ticket - TagAssignment --> Organization -``` - -This design avoids embedding tags inside each entity and enables consistent querying. - ---- - -# Multi-Tenancy Considerations - -Multi-tenancy is enforced through: - -- `tenantId` in AuthUser -- Tenant-scoped repositories in other modules -- Unique constraints within tenant boundaries -- Per-tenant ticket numbering - -The domain model avoids hard coupling to tenant logic; enforcement is handled at repository and service layers. - ---- - -# Indexing Strategy Summary - -The module heavily leverages: - -- `@Indexed` for query acceleration -- `@CompoundIndex` for dashboard queries -- Unique constraints for integrity -- Partial indexes for multi-tenant isolation - -This ensures: - -- Fast filtering -- Scalable dashboard queries -- Safe uniqueness guarantees -- Reduced cross-tenant data leakage risk - ---- - -# Design Principles - -The Data Mongo Domain Model follows these principles: - -1. Document-per-aggregate boundary -2. Soft deletes over hard deletes -3. External storage for large binary data -4. Event-driven compatibility -5. Strong indexing strategy -6. Multi-tenant safe uniqueness -7. Clear separation of identity vs domain users - ---- - -# Conclusion - -The **Data Mongo Domain Model** is the foundational persistence layer of OpenFrame. It defines the canonical MongoDB structures that power: - -- Authentication and SSO -- Tenant isolation -- Device monitoring -- Ticketing workflows -- Notifications -- Event processing -- OAuth client management - -All higher-level services rely on these documents to provide consistent, scalable, and multi-tenant safe behavior across the platform. diff --git a/docs/reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md b/docs/reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md deleted file mode 100644 index da7d8fd77..000000000 --- a/docs/reference/architecture/data-mongo-query-filters/data-mongo-query-filters.md +++ /dev/null @@ -1,316 +0,0 @@ -# Data Mongo Query Filters - -## Overview - -The **Data Mongo Query Filters** module defines strongly-typed filter objects used to construct dynamic MongoDB queries across the OpenFrame platform. These filters act as the boundary between higher-level API input (REST and GraphQL) and low-level repository query execution. - -Rather than embedding query logic directly in controllers or services, this module provides composable filter models that: - -- Encapsulate domain-specific filtering criteria -- Support time-based and enum-based constraints -- Enable clean separation between API DTOs and MongoDB query construction -- Promote consistency across sync and reactive repositories - -This module is part of the data layer and works closely with: - -- [Data Mongo Domain Model](../data-mongo-domain-model/data-mongo-domain-model.md) -- [Data Mongo Base Repositories](../data-mongo-base-repositories/data-mongo-base-repositories.md) -- [Data Mongo Sync Config and Custom Repositories](../data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md) -- [Data Mongo Reactive Repositories](../data-mongo-reactive-repositories/data-mongo-reactive-repositories.md) - ---- - -## Architectural Role in the System - -The Data Mongo Query Filters module sits between API-level filter inputs and repository-level query execution. - -```mermaid -flowchart LR - ApiLayer["API Layer\nREST + GraphQL"] --> FilterDTOs["Filter Input DTOs"] - FilterDTOs --> QueryFilters["Data Mongo Query Filters"] - QueryFilters --> CustomRepos["Custom Mongo Repositories"] - CustomRepos --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> CustomRepos -``` - -### Flow Explanation - -1. **API Layer** receives filter inputs (e.g., `TicketFilterInput`, `UserFilterInput`). -2. These inputs are mapped into internal **Query Filter** objects defined in this module. -3. Custom repositories interpret the filter object and build a MongoDB `Criteria` / query. -4. Queries are executed against collections defined in the domain model. - -This separation ensures: - -- API contracts can evolve without tightly coupling to MongoDB internals. -- Query construction remains centralized and testable. -- Business services remain independent from persistence logic. - ---- - -## Design Principles - -### 1. Immutable, Builder-Based Construction - -All filter classes use Lombok annotations: - -- `@Data` -- `@Builder` -- `@NoArgsConstructor` -- `@AllArgsConstructor` - -This enables safe, expressive creation: - -```java -TicketQueryFilter filter = TicketQueryFilter.builder() - .statuses(List.of(TicketStatus.OPEN)) - .organizationIds(List.of("org-1")) - .createdAtFrom(Instant.now().minus(30, ChronoUnit.DAYS)) - .build(); -``` - -### 2. Domain-Oriented Filtering - -Each filter aligns directly with a Mongo document type from the domain model: - -- Events β†’ `EventQueryFilter` -- Organizations β†’ `OrganizationQueryFilter` -- Tickets β†’ `TicketQueryFilter` -- Tools β†’ `ToolQueryFilter` -- Users β†’ `UserQueryFilter` - -### 3. Null-Safe Optional Criteria - -All fields are optional. Repositories interpret `null` values as "no constraint". - -This allows flexible query composition without requiring multiple overloaded methods. - ---- - -# Filter Classes - -## EventQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.event.filter.EventQueryFilter.EventQueryFilter` - -### Purpose - -Encapsulates filtering logic for querying event documents. - -### Fields - -- `List userIds` – Filter by user IDs -- `List eventTypes` – Restrict by event categories -- `LocalDate startDate` – Inclusive lower bound -- `LocalDate endDate` – Inclusive upper bound - -### Usage Context - -Used by event-related repositories to construct date-range and type-based queries. - -```mermaid -flowchart TD - Filter["EventQueryFilter"] --> ByUser["userIds"] - Filter --> ByType["eventTypes"] - Filter --> ByStart["startDate"] - Filter --> ByEnd["endDate"] -``` - ---- - -## OrganizationQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.organization.filter.OrganizationQueryFilter.OrganizationQueryFilter` - -### Purpose - -Defines structured filtering for organization queries. - -### Fields - -- `String category` -- `Integer minEmployees` -- `Integer maxEmployees` -- `Boolean hasActiveContract` -- `String status` - -### Typical Query Logic - -Repositories may: - -- Apply numeric range constraints for employee size -- Apply boolean checks for contract state -- Apply equality matching on category and status - -```mermaid -flowchart LR - OrgFilter["OrganizationQueryFilter"] --> SizeRange["Employee Range"] - OrgFilter --> Contract["Active Contract"] - OrgFilter --> Status["Organization Status"] -``` - ---- - -## TicketQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.ticket.filter.TicketQueryFilter.TicketQueryFilter` - -### Purpose - -Provides comprehensive filtering for ticket documents. - -### Fields - -- `List statuses` -- `List organizationIds` -- `List assigneeIds` -- `List labelIds` -- `List deviceIds` -- `List creationSources` -- `Instant createdAtFrom` -- `Instant createdAtTo` - -### Capabilities - -- Multi-status filtering -- Organization-scoped filtering -- Assignee-based filtering -- Label and device constraints -- Creation source classification -- Time-window filtering using `Instant` - -```mermaid -flowchart TD - TicketFilter["TicketQueryFilter"] --> Statuses["TicketStatus List"] - TicketFilter --> OrgScope["Organization IDs"] - TicketFilter --> Assignees["Assignee IDs"] - TicketFilter --> Labels["Label IDs"] - TicketFilter --> Devices["Device IDs"] - TicketFilter --> Source["Creation Sources"] - TicketFilter --> TimeFrom["createdAtFrom"] - TicketFilter --> TimeTo["createdAtTo"] -``` - -This filter is commonly used in custom repository implementations for advanced aggregation and dashboard queries. - ---- - -## ToolQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.tool.filter.ToolQueryFilter.ToolQueryFilter` - -### Purpose - -Filters integrated tool documents based on enablement and classification. - -### Fields - -- `Boolean enabled` -- `String type` -- `String category` -- `String platformCategory` - -### Common Use Cases - -- Retrieve only enabled tools -- Filter by tool type (e.g., RMM, MDM) -- Segment tools by platform category - ---- - -## UserQueryFilter - -**Component:** -`openframe-oss-lib.openframe-data-mongo-common.src.main.java.com.openframe.data.document.user.filter.UserQueryFilter.UserQueryFilter` - -### Purpose - -Defines flexible filtering for user queries. - -### Fields - -- `String emailRegex` -- `String nameRegex` -- `UserStatus status` - -### Query Semantics - -- Regex-based filtering for search capabilities -- Status-based filtering (e.g., ACTIVE, DISABLED) - -```mermaid -flowchart LR - UserFilter["UserQueryFilter"] --> Email["emailRegex"] - UserFilter --> Name["nameRegex"] - UserFilter --> Status["UserStatus"] -``` - ---- - -# Integration with Repositories - -The filters are interpreted by repository implementations in the sync and reactive Mongo modules. - -```mermaid -flowchart TD - ServiceLayer["Service Layer"] --> QueryFilter["Query Filter"] - QueryFilter --> CustomRepository["Custom Repository Impl"] - CustomRepository --> CriteriaBuilder["Mongo Criteria Builder"] - CriteriaBuilder --> Mongo[("MongoDB Collection")] -``` - -Repositories typically: - -1. Inspect each non-null filter field. -2. Add corresponding `Criteria` conditions. -3. Combine conditions using `andOperator` / `orOperator`. -4. Apply pagination and sorting. - ---- - -# Benefits to the Platform - -### Clean Separation of Concerns - -- API DTOs define external contract. -- Query filters define internal query semantics. -- Repositories handle persistence mechanics. - -### Extensibility - -Adding a new filter field: - -1. Extend the filter class. -2. Update repository criteria logic. -3. Expose through API input DTO if needed. - -No cross-layer coupling is introduced. - -### Consistency Across Domains - -All filters follow a uniform pattern: - -- Builder-based construction -- Optional fields -- Domain-aligned attributes -- Used exclusively by data layer - ---- - -# Summary - -The **Data Mongo Query Filters** module provides the structured, domain-driven filtering layer for MongoDB queries across OpenFrame. - -It: - -- Encapsulates filtering logic for events, organizations, tickets, tools, and users. -- Bridges API input models with Mongo repository implementations. -- Supports flexible, composable, and maintainable query construction. -- Enforces consistent filtering patterns across the data access layer. - -This module is a foundational building block in the persistence architecture, enabling scalable, expressive, and maintainable MongoDB querying throughout the platform. \ No newline at end of file diff --git a/docs/reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md b/docs/reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md deleted file mode 100644 index 618500f92..000000000 --- a/docs/reference/architecture/data-mongo-reactive-repositories/data-mongo-reactive-repositories.md +++ /dev/null @@ -1,287 +0,0 @@ -# Data Mongo Reactive Repositories - -## Overview - -The **Data Mongo Reactive Repositories** module provides reactive, non-blocking access to MongoDB for selected core domain entities in the OpenFrame platform. Built on **Spring Data Reactive MongoDB** and **Project Reactor**, this module enables asynchronous data access patterns using `Mono` and `Flux` types. - -It complements the synchronous repository layer by offering reactive alternatives for high-concurrency or event-driven use cases, particularly around: - -- OAuth client persistence -- User lookups and existence checks -- Security and authentication flows - -This module integrates closely with: - -- The domain model defined in **Data Mongo Domain Model** -- Shared base repository contracts from the synchronous layer -- Spring Boot auto-configuration and repository scanning - ---- - -## Core Components - -This module contains three primary components: - -- `MongoReactiveConfig` -- `ReactiveOAuthClientRepository` -- `ReactiveUserRepository` - ---- - -## Architecture Overview - -```mermaid -flowchart TD - App["Application Context"] --> Config["MongoReactiveConfig"] - Config -->|"@EnableReactiveMongoRepositories"| Scan["Reactive Repository Scan"] - - Scan --> OAuthRepo["ReactiveOAuthClientRepository"] - Scan --> UserRepo["ReactiveUserRepository"] - - OAuthRepo --> OAuthClient["OAuthClient Document"] - UserRepo --> UserDoc["User Document"] - - OAuthClient --> MongoDB[("MongoDB")] - UserDoc --> MongoDB -``` - -The configuration class enables reactive repository scanning. Spring automatically wires implementations for repository interfaces extending `ReactiveMongoRepository`. - ---- - -## Reactive Programming Model - -This module uses **Project Reactor** primitives: - -- `Mono` β€” zero or one result -- `Flux` β€” zero to many results - -Example semantics: - -```text -Mono β†’ Asynchronous single user lookup -Mono β†’ Asynchronous existence check -``` - -Unlike synchronous repositories that block on I/O, reactive repositories: - -- Use non-blocking MongoDB drivers -- Integrate with WebFlux and reactive security chains -- Support high-concurrency workloads efficiently - ---- - -## Component Details - -### MongoReactiveConfig - -**Class:** `MongoReactiveConfig` -**Package:** `com.openframe.data.config` - -```java -@Configuration -@EnableReactiveMongoRepositories(basePackages = "com.openframe.data.reactive.repository") -public class MongoReactiveConfig { -} -``` - -#### Responsibilities - -- Enables reactive MongoDB repository scanning -- Registers Spring Data reactive repository infrastructure -- Configures repositories located under: - -```text -com.openframe.data.reactive.repository -``` - -This class acts as the entry point for the reactive persistence layer. - ---- - -### ReactiveOAuthClientRepository - -**Interface:** `ReactiveOAuthClientRepository` -**Extends:** `ReactiveMongoRepository` - -```java -@Repository -public interface ReactiveOAuthClientRepository - extends ReactiveMongoRepository { - - Mono findByClientId(String clientId); -} -``` - -#### Responsibilities - -- Reactive CRUD operations for `OAuthClient` -- Lookup by `clientId` -- Used by authentication and OAuth flows - -#### Data Flow - -```mermaid -sequenceDiagram - participant Service as "Auth Service" - participant Repo as "ReactiveOAuthClientRepository" - participant DB as "MongoDB" - - Service->>Repo: findByClientId(clientId) - Repo->>DB: Reactive query - DB-->>Repo: OAuthClient document - Repo-->>Service: Mono -``` - -#### Typical Use Cases - -- OAuth client validation -- Client credentials flow -- Token issuance validation -- Dynamic client lookup - ---- - -### ReactiveUserRepository - -**Interface:** `ReactiveUserRepository` -**Extends:** -- `ReactiveMongoRepository` -- `BaseUserRepository, Mono, String>` - -```java -@Repository -public interface ReactiveUserRepository - extends ReactiveMongoRepository, - BaseUserRepository, Mono, String> { - - Mono findByEmail(String email); - - Mono existsByEmail(String email); - - Mono existsByEmailAndStatus(String email, UserStatus status); -} -``` - -#### Responsibilities - -- Reactive user retrieval by email -- Email existence checks -- Status-aware existence validation -- Compliance with shared `BaseUserRepository` contract - -#### Inheritance Model - -```mermaid -flowchart LR - BaseRepo["BaseUserRepository"] --> ReactiveRepo["ReactiveUserRepository"] - ReactiveMongo["ReactiveMongoRepository"] --> ReactiveRepo - ReactiveRepo --> UserDoc["User Document"] -``` - -The repository: - -- Inherits generic user contract behavior -- Adapts return types to `Mono` -- Ensures consistency across reactive and synchronous layers - -#### Example Reactive Pattern - -```text -findByEmail(email) - β†’ returns Mono - β†’ may emit value or complete empty -``` - -This integrates directly with: - -- Reactive security filters -- OAuth BFF layers -- WebFlux controllers - ---- - -## Relationship to Other Modules - -### Data Mongo Domain Model - -The reactive repositories operate on domain documents defined in: - -- `User` -- `OAuthClient` - -These are defined in the **Data Mongo Domain Model** module. - -(See the corresponding domain model documentation for detailed schema information.) - ---- - -### Data Mongo Base Repositories - -`ReactiveUserRepository` implements the generic `BaseUserRepository` contract, ensuring: - -- Cross-layer API consistency -- Shared query method definitions -- Type-safe generics for both blocking and reactive variants - -This pattern allows the platform to maintain consistent repository contracts while supporting different execution models. - ---- - -## Reactive vs Synchronous Repositories - -| Aspect | Reactive | Synchronous | -|--------|----------|------------| -| Driver | Reactive MongoDB driver | Blocking MongoDB driver | -| Return Types | `Mono`, `Flux` | `T`, `List` | -| Thread Model | Event-loop friendly | Thread-per-request | -| Backpressure | Supported | Not supported | - -The reactive repositories are ideal for: - -- High-concurrency authentication workloads -- Streaming integrations -- Event-driven services - ---- - -## Security and OAuth Integration - -Reactive repositories are particularly important in: - -- OAuth client validation -- Email-based login checks -- Account existence validation -- Status-based user access control - -Because these flows are frequently executed and latency-sensitive, non-blocking access improves scalability and throughput. - ---- - -## Extension Guidelines - -When adding new reactive repositories: - -1. Place them under: - -```text -com.openframe.data.reactive.repository -``` - -2. Extend `ReactiveMongoRepository` -3. Return `Mono` or `Flux` -4. If applicable, implement shared base repository contracts -5. Ensure the entity exists in the domain model module - ---- - -## Summary - -The **Data Mongo Reactive Repositories** module provides: - -- Spring Data reactive MongoDB integration -- Reactive user and OAuth client persistence -- Contract alignment with base repository interfaces -- Non-blocking access patterns for authentication-critical flows - -It forms a lightweight but critical foundation for scalable security, authentication, and identity-related operations across the OpenFrame platform. \ No newline at end of file diff --git a/docs/reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md b/docs/reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md deleted file mode 100644 index 4c8b166e1..000000000 --- a/docs/reference/architecture/data-mongo-sync-config-and-custom-repositories/data-mongo-sync-config-and-custom-repositories.md +++ /dev/null @@ -1,434 +0,0 @@ -# Data Mongo Sync Config And Custom Repositories - -## Overview - -The **Data Mongo Sync Config And Custom Repositories** module provides the synchronous MongoDB configuration and advanced repository implementations used across the OpenFrame platform. - -It is responsible for: - -- Bootstrapping synchronous MongoDB repositories -- Customizing Mongo mapping and auditing behavior -- Managing indexes and legacy index cleanup -- Implementing advanced filtering, cursor-based pagination, sorting, and aggregation logic -- Providing bulk and optimized update operations -- Supporting optimistic locking retry observability - -This module builds on: - -- [Data Mongo Domain Model](../data-mongo-domain-model/data-mongo-domain-model.md) -- [Data Mongo Query Filters](../data-mongo-query-filters/data-mongo-query-filters.md) -- [Data Mongo Base Repositories](../data-mongo-base-repositories/data-mongo-base-repositories.md) - -It acts as the **data-access execution layer** for API services, management services, and background processors. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - ApiLayer["API Service Layer"] --> RepositoryLayer["Custom Repositories"] - RepositoryLayer --> MongoTemplate["MongoTemplate"] - MongoTemplate --> MongoDB[("MongoDB")] - - DomainModel["Domain Documents"] --> RepositoryLayer - QueryFilters["Query Filter DTOs"] --> RepositoryLayer - - Config["Mongo Sync Config"] --> RepositoryLayer - IndexConfig["Mongo Index Config"] --> MongoDB -``` - -### Key Responsibilities - -| Layer | Responsibility | -|--------|----------------| -| Mongo Sync Config | Enables repositories, auditing, mapping customization | -| Mongo Index Config | Ensures required indexes, removes legacy indexes | -| Custom Repositories | Filtering, cursor pagination, aggregations, bulk updates | -| Retry Listener | Observability for optimistic locking retries | - ---- - -# Configuration Layer - -## Mongo Sync Config - -**Component:** `MongoSyncConfig` - -Enables synchronous Mongo repositories and configures Mongo mapping behavior. - -### Features - -- Conditional activation via `spring.data.mongodb.enabled` -- Enables `@EnableMongoRepositories` for `com.openframe.data.repository` -- Enables `@EnableMongoAuditing` -- Custom `MappingMongoConverter` - -### Mapping Customization - -```mermaid -flowchart LR - Factory["MongoDatabaseFactory"] --> Resolver["DefaultDbRefResolver"] - Resolver --> Converter["MappingMongoConverter"] - Conversions["MongoCustomConversions"] --> Converter -``` - -Notable configuration: - -- `setMapKeyDotReplacement("__dot__")` - - Allows storing map keys containing dots (`.`) safely - -This is critical for metadata-heavy documents like events and external application integrations. - ---- - -## Mongo Index Config - -**Component:** `MongoIndexConfig` - -Executed at application startup via `@PostConstruct`. - -### Index Responsibilities - -- Ensures compound indexes on `application_events` -- Drops stale legacy indexes - -### Example Indexes - -- `{ userId: ASC, timestamp: DESC }` -- `{ type: ASC, metadata.tags: ASC }` - -### Legacy Cleanup - -```mermaid -flowchart TD - Startup["Application Startup"] --> EnsureIndexes["Ensure Required Indexes"] - EnsureIndexes --> DropLegacy["Drop Stale Indexes"] - DropLegacy --> LogResult["Log Success Or Skip"] -``` - -The cleanup ensures: - -- Schema evolution safety -- Removal of org-scoped uniqueness constraints -- Tenant-wide tag uniqueness enforcement consistency - ---- - -# Custom Repository Implementations - -This module implements advanced data-access behavior beyond standard Spring Data repositories. - -Common patterns across repositories: - -- Cursor-based pagination using `_id` -- Secondary sort field with `_id` tie-breaker -- Mongo `Criteria`-based dynamic filtering -- Aggregation pipelines for grouped metrics -- Distinct queries for filter options -- Bulk operations for efficiency - ---- - -## Cursor-Based Pagination Pattern - -Most repositories follow this strategy: - -```mermaid -flowchart TD - ClientRequest["Client Request With Cursor"] --> ParseCursor["Parse ObjectId"] - ParseCursor --> LoadCursorDoc["Load Cursor Document"] - LoadCursorDoc --> CompareSortField["Compare Sort Field"] - CompareSortField --> AddCriteria["Add $or Criteria"] - AddCriteria --> ApplySort["Sort By Field And _id"] - ApplySort --> Limit["Apply Limit"] -``` - -This ensures: - -- Stable ordering -- Deterministic pagination -- No duplicates or gaps -- Proper tie-breaking when sort field values are equal - ---- - -# Repository Categories - -## 1. Assignment Repository - -**Component:** `CustomItemAssignmentRepositoryImpl` - -Features: - -- Cursor-based pagination -- Search by display name -- Group-by aggregation per `AssignmentTargetType` - -Aggregation example: - -```mermaid -flowchart LR - Match["Match itemId"] --> Group["Group By targetType"] - Group --> Count["Count"] - Count --> MapResult["Map To EnumMap"] -``` - ---- - -## 2. Machine Repository - -**Component:** `CustomMachineRepositoryImpl` - -Capabilities: - -- Multi-field filtering (status, type, OS, organization) -- Regex-based search (hostname, IP, serial, model) -- Cursor pagination with compound sorting -- Count queries for pagination metadata - ---- - -## 3. Event Repositories - -### External Application Event Repository - -**Component:** `ExternalApplicationEventRepository` - -- Spring Data `MongoRepository` -- Time-range queries -- Custom `@Query` for metadata tag filtering - -### Custom Event Repository - -**Component:** `CustomEventRepositoryImpl` - -Features: - -- Date range filtering -- Distinct queries (userId, event type) -- Cursor pagination -- Search over type and data fields - ---- - -## 4. Knowledge Base Repository - -**Component:** `CustomKnowledgeBaseItemRepositoryImpl` - -Advanced behaviors: - -- Folder vs article separation -- Archived vs active filtering -- Combined `$or` search and cursor criteria using `$and` -- Draft visibility model - -### Composite Criteria Strategy - -Spring Data does not allow multiple root `$or` conditions. - -Solution: - -```mermaid -flowchart TD - SearchCriteria["Search OR Criteria"] --> Combine - CursorCriteria["Cursor OR Criteria"] --> Combine - Combine["Combine Using $and"] --> FinalQuery["Final Query"] -``` - -This avoids null-key collisions in the Mongo query builder. - ---- - -## 5. Notification Repositories - -### Custom Notification Repository - -**Component:** `CustomNotificationRepositoryImpl` - -Implements: - -- Recipient-based filtering -- Read/unread filtering -- Title search -- Forward/backward pagination -- Two-phase fetch (read states first, then notifications) - -```mermaid -flowchart TD - FetchReadStates["Fetch NotificationReadState"] --> ExtractIds - ExtractIds["Extract Notification IDs"] --> FetchNotifications - FetchNotifications["Fetch Notification Documents"] --> Merge["Merge With Status"] -``` - -### Custom Notification Read State Repository - -**Component:** `CustomNotificationReadStateRepositoryImpl` - -- Bulk unordered insert -- Gracefully swallows duplicate-key errors -- Improves idempotency in concurrent flows - ---- - -## 6. Ticket Repository - -**Component:** `CustomTicketRepositoryImpl` - -One of the most feature-rich repositories. - -Capabilities: - -- Complex filtering (status, org, assignee, device) -- Created-at range filtering -- Cursor-based pagination with tie-breaker -- Aggregation metrics: - - Count by status - - Average resolution time -- Bulk status update -- Partial field updates (title) - -### Average Resolution Time Aggregation - -```mermaid -flowchart LR - MatchResolved["Match Resolved Tickets"] --> ProjectDiff["Compute resolvedAt - createdAt"] - ProjectDiff --> GroupAvg["Average Resolution Time"] -``` - ---- - -## 7. Organization Repository - -**Component:** `CustomOrganizationRepositoryImpl` - -Features: - -- Default ACTIVE status filtering -- Contract validity evaluation -- Employee range filters -- Regex category matching -- Cursor pagination with multi-field sorting - ---- - -## 8. Integrated Tool Repository - -**Component:** `CustomIntegratedToolRepositoryImpl` - -Capabilities: - -- Enabled/type/category filtering -- Search over name and description -- Distinct queries for: - - type - - category - - platformCategory - ---- - -## 9. User Repository - -**Component:** `CustomUserRepositoryImpl` - -Focused on search: - -- Status filtering -- Email regex -- Name regex (first OR last name) -- Sorted by createdAt descending - ---- - -## 10. OAuth Token Repository - -**Component:** `OAuthTokenRepository` - -Simple Spring Data repository with: - -- `findByAccessToken` -- `findByRefreshToken` - -Used by authentication and token lifecycle management flows. - ---- - -## 11. Ticket Attachments And Notes - -- `TicketAttachmentRepository` -- `TicketNoteRepository` - -Provide standard CRUD plus: - -- Find by ticket ID -- Batch lookup -- Delete by ticket ID -- Sorted retrieval (notes) - ---- - -# Retry Observability - -## Optimistic Locking Retry Listener - -**Component:** `OptimisticLockingRetryListener` - -Provides structured logging for: - -- Retry attempts -- Retry success -- Retry exhaustion - -```mermaid -flowchart TD - RetryStart["Retry Attempt"] --> OnError["onError()"] - OnError --> LogWarn["Log Warning"] - RetryEnd["Retry Complete"] --> Close - Close --> SuccessOrFail["Log Success Or Error"] -``` - -This improves: - -- Visibility into concurrency conflicts -- Operational debugging -- Retry exhaustion tracing - ---- - -# Cross-Module Relationships - -The **Data Mongo Sync Config And Custom Repositories** module interacts heavily with: - -- **Domain documents** from Data Mongo Domain Model -- **Query filter DTOs** from Data Mongo Query Filters -- **API Services** and **Management Services** that orchestrate business logic -- **Authorization Service** when dealing with OAuth tokens - -It serves as the **core synchronous persistence engine** of the OpenFrame platform. - ---- - -# Design Principles - -1. Database-Level Filtering First -2. Deterministic Cursor Pagination -3. Safe Aggregations With Type Mapping -4. Bulk Operations For Performance -5. Graceful Handling Of Legacy Indexes -6. Idempotent Insert Strategies -7. Clear Separation Between Query Building And Execution - ---- - -# Summary - -The **Data Mongo Sync Config And Custom Repositories** module: - -- Configures MongoDB for synchronous workloads -- Ensures proper indexing and schema evolution safety -- Implements advanced query logic across all major domain entities -- Provides efficient aggregation and bulk operations -- Enables stable cursor-based pagination across the platform -- Improves concurrency observability via retry listeners - -It is a foundational infrastructure module that underpins all synchronous data access within the OpenFrame ecosystem. \ No newline at end of file diff --git a/docs/reference/architecture/data-nats-notifications/data-nats-notifications.md b/docs/reference/architecture/data-nats-notifications/data-nats-notifications.md deleted file mode 100644 index 5f5202f3a..000000000 --- a/docs/reference/architecture/data-nats-notifications/data-nats-notifications.md +++ /dev/null @@ -1,365 +0,0 @@ -# Data Nats Notifications - -## Overview - -The **Data Nats Notifications** module is responsible for broadcasting real-time notifications over NATS while ensuring durable persistence in MongoDB. It acts as the bridge between: - -- The persistent notification domain model (Mongo documents) -- The read-state tracking subsystem -- The NATS messaging infrastructure -- Downstream consumers such as WebSocket gateways, clients, and machine agents - -This module guarantees that: - -- Notifications are **persisted first** (source of truth). -- Read states are created for each recipient. -- Real-time delivery over NATS is **best-effort and non-blocking**. -- Clients can reconcile missed events via GraphQL or REST catch-up. - -It is feature-flagged and can operate in persistence-only mode when NATS is disabled. - ---- - -## Core Responsibilities - -The Data Nats Notifications module provides: - -1. βœ… Command validation and audience sanitation -2. βœ… Durable notification persistence -3. βœ… Read-state creation for users and machines -4. βœ… NATS subject-based publishing -5. βœ… Safe failure handling with rollback protection -6. βœ… Graceful degradation when NATS is unavailable - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Caller["Application Service"] --> Command["NotificationCommand"] - Command --> Broadcaster["NotificationBroadcaster"] - - Broadcaster --> Repo["NotificationRepository"] - Broadcaster --> ReadState["NotificationReadStateService"] - Broadcaster --> Registry["NotificationContextDescriptorRegistry"] - - Broadcaster -->|"optional"| Publisher["NotificationNatsPublisher"] - Publisher --> Nats["NATS Server"] - - Repo --> Mongo[("MongoDB")] - ReadState --> Mongo -``` - -### Key Design Principle - -**Persistence is the source of truth.** - -NATS delivery is best-effort. If publishing fails, the notification remains persisted and recipients reconcile via API queries. - ---- - -## Notification Lifecycle - -```mermaid -flowchart TD - Start["Broadcast Request"] --> Validate["Validate NotificationCommand"] - Validate --> Persist["Persist Notification"] - Persist --> CreateReadStates["Create Read States"] - CreateReadStates --> Publish["Publish to NATS"] - Publish --> End["Complete"] - - CreateReadStates -->|"failure"| Rollback["Delete Notification"] -``` - -### Transactional Strategy - -- If **read-state creation fails**, the notification document is deleted. -- If **NATS publish fails**, no rollback occurs. -- Recipients catch up using stored notifications. - ---- - -# Core Components - -## 1. NotificationCommand - -Immutable command object representing a validated broadcast request. - -### Responsibilities - -- Enforces required fields: - - `title` - - `severity` - - `context` -- Ensures `context.type` is non-blank -- Sanitizes and validates: - - `adminAudience` - - `machineAudience` -- Requires at least one non-empty audience -- Produces immutable recipient sets - -### Validation Rules - -```text -- title must not be blank -- severity must not be null -- context must not be null -- context.type must not be blank -- at least one audience must be non-empty -- no blank entries in audience sets -``` - -This prevents invalid broadcasts from reaching persistence or NATS. - ---- - -## 2. NotificationBroadcaster - -The orchestration service of the module. - -### Dependencies - -- `NotificationRepository` -- `NotificationReadStateService` -- `NotificationContextDescriptorRegistry` -- Optional `NotificationNatsPublisher` - -### Feature Flag - -```text -openframe.features.notifications.enabled=false -``` - -If disabled: -- No persistence -- No read states -- No NATS publish -- Broadcast returns `null` - ---- - -### Broadcast Flow - -```mermaid -flowchart LR - Cmd["NotificationCommand"] --> Build["Build Notification"] - Build --> Save["Save to Repository"] - Save --> Category["Resolve Category"] - Category --> RS["Create Read States"] - RS --> Publish["Publish Per Recipient"] -``` - -### Read-State Creation - -Recipients are separated by type: - -- `RecipientType.USER` -- `RecipientType.MACHINE` - -The category is derived from: - -- `NotificationContextDescriptorRegistry` - -If read-state creation throws an exception: - -1. The persisted notification is deleted. -2. The exception is rethrown. -3. Caller must retry. - ---- - -### NATS Publishing Strategy - -Publishing is performed per-recipient: - -- Each admin user β†’ `user.{userId}.notification` -- Each machine β†’ `machine.{machineId}.notification` - -Failures are handled individually: - -- Logged as warnings -- No rollback -- Recipient reconciles via API - ---- - -## 3. NotificationNatsPublisher - -Encapsulates subject generation and NATS publishing. - -### Conditional Activation - -```text -@ConditionalOnProperty("spring.cloud.stream.enabled") -``` - -If Spring Cloud Stream is disabled: -- Publisher bean is not created -- Broadcaster logs persistence-only mode - ---- - -### Subject Templates - -```text -user.{userId}.notification -machine.{machineId}.notification -``` - -### Publish Safety Rules - -- `userId` and `machineId` must not be blank -- Notification must be persisted (must have `id`) -- `NatsException` is caught and logged -- No exception propagates to caller - ---- - -### Message Construction - -The domain `Notification` document is mapped to a lightweight transport model: - -```mermaid -flowchart TD - Doc["Notification Document"] --> Msg["NotificationMessage"] - - Msg --> Id["id"] - Msg --> Severity["severity"] - Msg --> Title["title"] - Msg --> Desc["description"] - Msg --> Created["createdAt"] - Msg --> Context["context"] -``` - -Only fields required by clients are included. - ---- - -## 4. NotificationMessage - -Transport DTO published to NATS. - -### Fields - -```text -id : String -severity : NotificationSeverity -title : String -description : String -createdAt : Instant -context : NotificationContext -``` - -Designed to: - -- Be serialization-friendly -- Avoid heavy domain coupling -- Provide enough context for real-time rendering - ---- - -# Reliability Model - -The module intentionally separates: - -- **Storage reliability** (strong) -- **Delivery reliability** (eventual) - -```mermaid -flowchart TD - Persist["Persist Notification"] --> Durable["Durable Storage"] - Durable --> RealTime["Real-Time Publish"] - - RealTime -->|"failure"| CatchUp["Client Catch-Up via API"] -``` - -### Guarantees - -| Concern | Guarantee | -|----------|-----------| -| Persistence | Strong (Mongo) | -| Read State | Strong (created before publish) | -| NATS Delivery | Best effort | -| Consistency | Eventual for clients | - ---- - -# Integration with the Platform - -Although self-contained, this module integrates with: - -- Mongo notification domain model -- Notification read-state services -- NATS infrastructure -- GraphQL notification queries -- Gateway WebSocket or client subscribers - -### Typical End-to-End Flow - -```mermaid -sequenceDiagram - participant Service - participant Broadcaster - participant Mongo - participant NATS - participant Client - - Service->>Broadcaster: broadcast(command) - Broadcaster->>Mongo: save(notification) - Broadcaster->>Mongo: createReadStates() - Broadcaster->>NATS: publish(subject, message) - NATS->>Client: deliver real-time event - Client->>Mongo: query for reconciliation -``` - ---- - -# Design Principles - -## 1. Fail-Safe Persistence - -No notification is published without being persisted. - -## 2. Controlled Rollback - -Only read-state failures trigger deletion. - -## 3. Optional Real-Time Layer - -System works fully without NATS. - -## 4. Recipient Isolation - -Failures per-recipient do not block others. - -## 5. Immutable Command Pattern - -Validation happens before orchestration. - ---- - -# Configuration Summary - -```text -openframe.features.notifications.enabled -spring.cloud.stream.enabled -``` - -| Property | Purpose | -|----------|----------| -| openframe.features.notifications.enabled | Master feature flag | -| spring.cloud.stream.enabled | Enables NATS publisher bean | - ---- - -# Conclusion - -The **Data Nats Notifications** module provides a robust, fault-tolerant notification broadcasting mechanism built on: - -- Strong persistence guarantees -- Read-state tracking -- Optional real-time messaging via NATS -- Safe error handling and graceful degradation - -It ensures that OpenFrame can deliver reliable notifications to both administrators and machine agents while maintaining consistency and operational safety across distributed services. \ No newline at end of file diff --git a/docs/reference/architecture/data-pinot-repositories/data-pinot-repositories.md b/docs/reference/architecture/data-pinot-repositories/data-pinot-repositories.md deleted file mode 100644 index c3a1457da..000000000 --- a/docs/reference/architecture/data-pinot-repositories/data-pinot-repositories.md +++ /dev/null @@ -1,308 +0,0 @@ -# Data Pinot Repositories - -## Overview - -The **Data Pinot Repositories** module provides analytical data access capabilities using **Apache Pinot** for high-performance, real-time OLAP queries. While transactional and operational data is stored in MongoDB, this module is responsible for: - -- Real-time log analytics -- Device-level aggregations and facet queries -- High-volume, time-series filtering -- Efficient count and distinct queries for UI filters - -It acts as the **analytics read layer** of the OpenFrame platform, optimized for dashboards, filters, and log exploration workloads. - ---- - -## Architectural Role in the Platform - -The platform follows a polyglot persistence architecture: - -- **MongoDB** β†’ transactional and domain persistence -- **Kafka / Stream Service** β†’ event ingestion and enrichment -- **Apache Pinot** β†’ analytical querying and aggregations - -The Data Pinot Repositories module sits on top of Pinot and exposes structured repository abstractions used by API and management services. - -```mermaid -flowchart LR - StreamService["Stream Service Core Kafka And Handlers"] -->|"Ingest Events"| PinotCluster["Apache Pinot Cluster"] - MongoDB["Data Mongo Domain Model"] -->|"Transactional Data"| Services["API & Management Services"] - PinotCluster -->|"Analytical Queries"| PinotRepos["Data Pinot Repositories"] - PinotRepos -->|"Facet Counts & Logs"| Services -``` - -### Upstream Dependencies - -- Event ingestion from **Stream Service Core Kafka And Handlers** -- Pinot cluster availability and schema configuration - -### Downstream Consumers - -- API Service (GraphQL and REST) -- External API Service -- Management Service (resync operations) - ---- - -## Core Components - -The module contains the following core components: - -| Component | Responsibility | -|------------|----------------| -| `PinotConfig` | Configures Pinot broker and controller connections | -| `PinotEventEntity` | Domain marker for Pinot-backed event projections | -| `PinotClientDeviceRepository` | Device analytics and facet queries | -| `PinotClientLogRepository` | Log search, filtering, and projection queries | - ---- - -## Configuration Layer - -### PinotConfig - -`PinotConfig` provides Spring-managed `Connection` beans for interacting with Pinot: - -- **Broker Connection** β†’ Used for query execution -- **Controller Connection** β†’ Used for cluster/controller operations - -Configuration properties: - -- `pinot.broker.url` -- `pinot.controller.url` -- `pinot.tables.devices.name` -- `pinot.tables.logs.name` - -```mermaid -flowchart TD - AppConfig["Spring Context"] --> PinotConfig["PinotConfig"] - PinotConfig --> BrokerConn["pinotBrokerConnection()"] - PinotConfig --> ControllerConn["pinotControllerConnection()"] - BrokerConn --> PinotCluster["Pinot Broker"] - ControllerConn --> PinotController["Pinot Controller"] -``` - -The broker connection is injected into repository implementations using `@Qualifier("pinotBrokerConnection")`. - ---- - -## Device Analytics Repository - -### PinotClientDeviceRepository - -The **PinotClientDeviceRepository** provides analytical queries over the `devices` Pinot table. - -It supports: - -- Faceted filter queries -- Aggregated device counts -- Multi-dimensional filtering -- Tag-based filtering - -### Supported Filters - -- Status (excluding `DELETED`) -- Device type -- OS type -- Organization -- Tags -- Tag key-value pairs - -### Facet Query Pattern - -Facet queries dynamically exclude the field being aggregated to ensure correct filter option computation. - -```mermaid -flowchart TD - Start["Facet Request"] --> BuildQuery["PinotQueryBuilder"] - BuildQuery --> ApplyFilters["applyDeviceFilters()"] - ApplyFilters --> ExcludeField{"Exclude Facet Field?"} - ExcludeField -->|"Yes"| SkipFilter["Skip That Filter"] - ExcludeField -->|"No"| ApplyFilter["Apply Filter"] - SkipFilter --> GroupBy["GROUP BY facetField"] - ApplyFilter --> GroupBy - GroupBy --> Execute["executeKeyCountQuery()"] -``` - -### Key Behaviors - -1. Always excludes devices with status `DELETED`. -2. Dynamically removes the active facet field from filters. -3. Returns `Map` for UI-ready filter counts. -4. Uses count queries for total filtered device count. - -This repository is optimized for dashboard filters and device inventory analytics. - ---- - -## Log Analytics Repository - -### PinotClientLogRepository - -The **PinotClientLogRepository** supports time-series log exploration and advanced filtering. - -### Capabilities - -- Date range filtering -- Cursor-based pagination -- Relevance search -- Sortable column validation -- Distinct filter options -- Organization option projections - -### Query Construction Flow - -```mermaid -flowchart LR - Request["Log Query Request"] --> Builder["PinotQueryBuilder"] - Builder --> DateRange["whereDateRange()"] - DateRange --> Filters["whereIn() / whereEquals()"] - Filters --> Search["whereRelevanceLogSearch()"] - Search --> Cursor["whereCursor()"] - Cursor --> Sorting["orderBySortInput()"] - Sorting --> Limit["limit()"] - Limit --> Execute["executeLogQuery()"] -``` - -### Sorting Controls - -The repository enforces a whitelist of sortable columns: - -- `eventTimestamp` -- `severity` -- `eventType` -- `toolType` -- `organizationId` -- `deviceId` -- `ingestDay` - -If a field is not sortable: - -- It falls back to `eventTimestamp`. - -This protects against invalid or unsafe query construction. - -### Projection Mapping - -Results are mapped into `LogProjection` objects using column index mapping derived from Pinot result sets. - -Important fields include: - -- `toolEventId` -- `eventTimestamp` -- `toolType` -- `eventType` -- `severity` -- `organizationId` -- `summary` - -The `eventTimestamp` is converted from epoch milliseconds to `Instant`. - ---- - -## Multi-Tenant Isolation - -All queries are built using a `tenantId` parameter via `PinotQueryBuilder`. - -This ensures: - -- Logical data isolation -- Tenant-safe aggregations -- Secure filtering across shared Pinot tables - -```mermaid -flowchart TD - Query["Incoming Query"] --> TenantFilter["Apply tenantId"] - TenantFilter --> PinotExec["Execute on Shared Table"] - PinotExec --> TenantScoped["Tenant Scoped Result"] -``` - ---- - -## Interaction with Other Modules - -### Stream Ingestion - -Events are ingested via Kafka and enriched by the stream layer before being indexed into Pinot. - -See: - -- [Stream Service Core Kafka And Handlers](../stream-service-core-kafka-and-handlers.md) - -### Device Pinot Resynchronization - -Management workflows may trigger re-synchronization of device data into Pinot. - -See: - -- [Management Service Core Initializers And Schedulers](../management-service-core-initializers-and-schedulers.md) - -### API Layer Consumption - -The API layer consumes Pinot repositories to provide: - -- Filter option endpoints -- Log search endpoints -- Aggregated dashboard counts - ---- - -## Design Principles - -### 1. Separation of Concerns - -- MongoDB β†’ transactional storage -- Pinot β†’ analytical read queries - -### 2. Query Builder Abstraction - -All query logic flows through a `PinotQueryBuilder`, ensuring: - -- Safe dynamic query construction -- Consistent filter application -- Centralized cursor logic - -### 3. Facet-First Design - -Repositories are optimized for UI filter experiences: - -- Count per facet value -- Distinct option lists -- Filter exclusion logic - -### 4. Performance-Oriented - -- Aggregations pushed to Pinot -- Minimal application-side computation -- Projection-based mapping - ---- - -## End-to-End Log Analytics Flow - -```mermaid -flowchart LR - Client["UI / API Request"] --> ApiService["API Service"] - ApiService --> LogRepo["PinotClientLogRepository"] - LogRepo --> PinotBroker["Pinot Broker"] - PinotBroker --> PinotServer["Pinot Servers"] - PinotServer --> PinotBroker - PinotBroker --> LogRepo - LogRepo --> ApiService - ApiService --> Client -``` - ---- - -## Summary - -The **Data Pinot Repositories** module provides the analytical backbone of the OpenFrame platform. - -It enables: - -- Real-time log exploration -- High-performance device filtering -- Multi-dimensional facet queries -- Secure tenant-scoped analytics - -By leveraging Apache Pinot, it ensures scalable, low-latency query execution for dashboard and filtering workloads while keeping transactional systems isolated in MongoDB. \ No newline at end of file diff --git a/docs/reference/architecture/data-redis-cache/data-redis-cache.md b/docs/reference/architecture/data-redis-cache/data-redis-cache.md deleted file mode 100644 index cb2f0e676..000000000 --- a/docs/reference/architecture/data-redis-cache/data-redis-cache.md +++ /dev/null @@ -1,240 +0,0 @@ -# Data Redis Cache - -The **Data Redis Cache** module provides Redis-based caching and key infrastructure for the OpenFrame platform. It integrates Spring Cache with Redis, configures synchronous and reactive Redis templates, and enforces tenant-aware key prefixing to ensure strict data isolation across organizations. - -This module acts as the foundational caching layer used by API services, management services, and other components that require high-performance, low-latency data access. - ---- - -## Purpose and Responsibilities - -The Data Redis Cache module is responsible for: - -- Enabling and configuring Spring Cache backed by Redis -- Providing a tenant-aware cache key prefix strategy -- Supplying `RedisTemplate` and `ReactiveRedisTemplate` beans -- Enforcing standardized key/value serialization -- Supporting conditional activation based on configuration - -Redis is enabled only when the property `spring.redis.enabled=true` is set, allowing flexible deployment topologies. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - AppServices["Application Services"] --> CacheAbstraction["Spring Cache Abstraction"] - CacheAbstraction --> CacheManager["RedisCacheManager"] - CacheManager --> RedisConnection["RedisConnectionFactory"] - CacheManager --> KeyBuilder["OpenframeRedisKeyBuilder"] - RedisConnection --> RedisServer[("Redis Server")] - - ReactiveServices["Reactive Services"] --> ReactiveTemplate["ReactiveRedisTemplate"] - ReactiveTemplate --> ReactiveConnection["ReactiveRedisConnectionFactory"] - ReactiveConnection --> RedisServer -``` - -### Key Components - -| Component | Responsibility | -|------------|----------------| -| CacheConfig | Configures Spring Cache and RedisCacheManager | -| RedisConfig | Provides RedisTemplate and ReactiveRedisTemplate beans | -| OpenframeRedisKeyConfiguration | Registers OpenframeRedisKeyBuilder for tenant-aware key prefixes | - ---- - -# Core Configuration Classes - -## CacheConfig - -**Class:** `CacheConfig` - -This class enables Spring caching and configures the Redis-backed `CacheManager`. - -### Activation - -The configuration is loaded only if: - -```text -spring.redis.enabled=true -``` - -### Default Cache Behavior - -The default cache configuration includes: - -- Time-to-live (TTL): 6 hours -- Null values disabled -- String key serialization -- JSON value serialization using `GenericJackson2JsonRedisSerializer` -- Tenant-aware key prefixing - -### Tenant-Aware Key Prefixing - -All cache keys follow this structure: - -```text -::: -``` - -The prefix is computed via `OpenframeRedisKeyBuilder`, ensuring isolation between tenants. - -### Custom TTL Overrides - -Certain caches use shorter TTL values: - -| Cache Name | TTL | -|------------|------| -| fleetPolicyCache | 1 hour | -| fleetQueryCache | 1 hour | - -This prevents stale policy or query data from persisting too long. - -### Cache Initialization Flow - -```mermaid -flowchart TD - Start["Application Startup"] --> RedisEnabled{"Redis Enabled?"} - RedisEnabled -->|Yes| CreateManager["Create RedisCacheManager"] - RedisEnabled -->|No| Skip["Skip Redis Cache Config"] - CreateManager --> DefaultConfig["Apply Default TTL 6h"] - DefaultConfig --> FleetOverride["Override Fleet TTL 1h"] - FleetOverride --> Ready["CacheManager Ready"] -``` - ---- - -## RedisConfig - -**Class:** `RedisConfig` - -This class provides low-level Redis beans for both blocking and reactive usage. - -### Provided Beans - -| Bean | Type | Purpose | -|------|------|----------| -| redisTemplate | RedisTemplate | Standard Redis operations | -| reactiveStringRedisTemplate | ReactiveStringRedisTemplate | Reactive string-based operations | -| reactiveRedisTemplate | ReactiveRedisTemplate | Reactive key-value operations | - -### Serialization Strategy - -All templates use: - -- String serialization for keys -- String serialization for values (in template beans) -- JSON serialization for Spring Cache values (via CacheConfig) - -This ensures consistent behavior across imperative and reactive flows. - -### Repository Support - -`@EnableRedisRepositories` enables Redis-backed repositories under: - -```text -com.openframe.data.repository.redis -``` - -This allows future extensions such as token stores, distributed locks, or transient state persistence. - ---- - -## OpenframeRedisKeyConfiguration - -**Class:** `OpenframeRedisKeyConfiguration` - -This configuration registers the `OpenframeRedisKeyBuilder` bean. - -### Responsibilities - -- Binds `OpenframeRedisProperties` -- Constructs a reusable key builder -- Ensures consistent prefix generation across the platform - -The key builder centralizes key naming conventions, preventing: - -- Cross-tenant collisions -- Environment conflicts -- Inconsistent naming patterns - ---- - -# Multi-Tenant Key Strategy - -The Data Redis Cache module enforces strict tenant-aware isolation. - -```mermaid -flowchart LR - TenantA["Tenant A"] --> KeyBuilder - TenantB["Tenant B"] --> KeyBuilder - KeyBuilder --> RedisKeyA["tenantA:deviceCache::123"] - KeyBuilder --> RedisKeyB["tenantB:deviceCache::123"] - RedisKeyA --> RedisServer[("Redis")] - RedisKeyB --> RedisServer -``` - -Even when logical cache keys are identical, the computed prefix ensures physical separation in Redis. - ---- - -# Integration Within the Platform - -The Data Redis Cache module supports multiple platform layers: - -- API services for query result caching -- Authorization services for token or metadata caching -- Management services for transient synchronization data -- Gateway services for rate limiting or session metadata - -It works alongside: - -- Mongo repositories (primary persistence layer) -- Kafka streaming services (event-driven updates) -- Security modules (authentication and token validation) - -Redis acts as a performance optimization layer β€” never the source of truth. - ---- - -# Conditional Activation and Deployment - -Redis caching is optional and controlled by configuration. - -```text -spring.redis.enabled=true -``` - -If disabled: - -- CacheConfig is not loaded -- RedisConfig is not loaded -- No Redis repositories are enabled - -This design supports: - -- Local development without Redis -- Staging environments with partial caching -- Production environments with full caching enabled - ---- - -# Design Principles - -The Data Redis Cache module follows these principles: - -1. Tenant Isolation by Default -2. Safe Serialization Strategies -3. Sensible TTL Defaults with Domain Overrides -4. Reactive and Imperative Support -5. Conditional Bootstrapping - ---- - -# Summary - -The **Data Redis Cache** module provides a structured, tenant-aware, and extensible Redis caching layer for the OpenFrame platform. By centralizing Redis configuration, serialization policies, and key naming strategies, it ensures consistent behavior across services while preserving strong multi-tenant boundaries. - -It serves as the high-performance caching backbone of the platform while maintaining strict separation from the system's primary data stores. \ No newline at end of file diff --git a/docs/reference/architecture/external-api-service-core/external-api-service-core.md b/docs/reference/architecture/external-api-service-core/external-api-service-core.md deleted file mode 100644 index 8a65298ac..000000000 --- a/docs/reference/architecture/external-api-service-core/external-api-service-core.md +++ /dev/null @@ -1,436 +0,0 @@ -# External Api Service Core - -## Overview - -The **External Api Service Core** module exposes a secure, API key–based REST interface for external integrations with the OpenFrame platform. - -It provides: - -- Public REST endpoints under `/api/v1/**` -- Tool proxy endpoints under `/tools/**` -- OpenAPI (Swagger) documentation -- Cursor-based pagination, filtering, and sorting -- API key authentication via `X-API-Key` - -Unlike the internal GraphQL-based API service, this module is purpose-built for third-party systems, automation scripts, and integration partners. - ---- - -## Architectural Positioning - -The External Api Service Core sits at the boundary between external consumers and the internal platform services. - -```mermaid -flowchart LR - Client["External Client"] -->|"X-API-Key"| ExternalAPI["External Api Service Core"] - - ExternalAPI --> DeviceService["Device Service"] - ExternalAPI --> EventService["Event Service"] - ExternalAPI --> LogService["Log Service"] - ExternalAPI --> OrganizationService["Organization Services"] - ExternalAPI --> ToolService["Tool Service"] - - ExternalAPI --> ProxyService["Rest Proxy Service"] - ProxyService --> IntegratedTool["Integrated Tool"] - - DeviceService --> MongoDB[("MongoDB")] - EventService --> MongoDB - LogService --> Pinot[("Apache Pinot")] - ToolService --> MongoDB -``` - -### Responsibilities - -- Authenticate requests using API keys -- Translate REST query parameters into internal filter criteria -- Apply cursor-based pagination -- Map domain models into external DTOs -- Proxy tool-specific requests to integrated tools - ---- - -## Authentication Model - -All endpoints require an API key provided in the `X-API-Key` header. - -```text -X-API-Key: ak_keyId.sk_secretKey -``` - -Internally: - -- The API key is validated by upstream security filters -- `X-User-Id` and `X-API-Key-Id` headers are injected -- Controllers use those headers for auditing and authorization - -The module does **not** perform token-based OAuth authentication. It is explicitly designed for machine-to-machine API key usage. - ---- - -## OpenAPI Configuration - -### OpenApiConfig - -The `OpenApiConfig` class configures: - -- API metadata (title, version, license) -- API key security scheme -- Grouped OpenAPI paths -- Server base path `/external-api` - -Documented path groups: - -```text -Included: -- /tools/** -- /api/v1/** - -Excluded: -- /actuator/** -- /api/core/** -``` - -Security scheme definition: - -```text -Type: APIKEY -In: HEADER -Header name: X-API-Key -``` - ---- - -# REST Controllers - -All REST endpoints are versioned under `/api/v1` except integration proxy endpoints (`/tools/**`). - ---- - -## DeviceController - -**Base Path:** `/api/v1/devices` - -### Capabilities - -- List devices with filtering and pagination -- Retrieve a device by machine ID -- Retrieve device filter options -- Update device status - -### Query Features - -The controller converts query parameters into `DeviceFilterCriteria`: - -- Status filters -- Device type filters -- OS type filters -- Organization filters -- Tag filters -- Search -- Sorting -- Cursor-based pagination - -```mermaid -flowchart TD - Request["GET /api/v1/devices"] --> Criteria["DeviceFilterCriteria"] - Criteria --> Pagination["CursorPaginationCriteria"] - Pagination --> Query["DeviceService.queryDevices()"] - Query --> Result["Query Result"] - Result --> Mapper["DeviceMapper"] - Mapper --> Response["DevicesResponse"] -``` - -### Tag Enrichment - -When `includeTags=true`, the controller: - -1. Extracts machine IDs -2. Loads tags via `TagService` -3. Returns enriched response - -Failures in tag loading fall back to non-enriched responses. - ---- - -## EventController - -**Base Path:** `/api/v1/events` - -### Capabilities - -- Query events with filtering -- Retrieve event by ID -- Create event -- Update event -- Retrieve filter options - -### Filtering Dimensions - -- User IDs -- Event types -- Date range -- Search term -- Sorting -- Cursor pagination - -```mermaid -flowchart TD - EventRequest["GET /api/v1/events"] --> Filter["EventFilterCriteria"] - Filter --> Service["EventService.queryEvents()"] - Service --> Mapper["EventMapper"] - Mapper --> EventsResponse["EventsResponse"] -``` - -Create and update operations directly delegate to `EventService`. - ---- - -## LogController - -**Base Path:** `/api/v1/logs` - -### Capabilities - -- Query logs with filtering -- Retrieve log filter options -- Retrieve detailed log entry - -### Filtering Dimensions - -- Date range -- Tool type -- Event type -- Severity -- Organization -- Device ID -- Search -- Sorting -- Cursor pagination - -Log queries are executed via `LogService`, which may retrieve data from analytics stores such as Apache Pinot. - -Detailed log retrieval requires composite identifiers: - -```text -ingestDay -toolType -eventType -timestamp -toolEventId -``` - ---- - -## OrganizationController - -**Base Path:** `/api/v1/organizations` - -### Capabilities - -- List organizations with filtering -- Retrieve organization by database ID -- Retrieve organization by business identifier -- Create organization -- Update organization -- Update status (ACTIVE / ARCHIVED) -- Check archive eligibility - -### Query Delegation - -The controller delegates: - -- Reads β†’ `OrganizationQueryService` -- Writes β†’ `OrganizationCommandService` -- Archival checks β†’ `OrganizationService` - -```mermaid -flowchart LR - OrgRequest["Organization Request"] --> QueryService["OrganizationQueryService"] - OrgRequest --> CommandService["OrganizationCommandService"] - CommandService --> Validation["Archive Rules"] -``` - -Archiving is blocked when active devices exist. - ---- - -## ToolController - -**Base Path:** `/api/v1/tools` - -### Capabilities - -- List integrated tools -- Retrieve tool filter options - -Filtering includes: - -- Enabled status -- Tool type -- Category -- Search -- Sorting - -Delegates to `ToolService` and maps results using `ToolMapper`. - ---- - -## IntegrationController - -**Base Path:** `/tools/{toolId}/**` - -This controller proxies arbitrary HTTP requests to integrated tools. - -Supported methods: - -- GET -- POST -- PUT -- PATCH -- DELETE -- OPTIONS - -```mermaid -flowchart TD - Client["External Client"] --> ProxyController["IntegrationController"] - ProxyController --> RestProxyService["RestProxyService"] - RestProxyService --> ToolRepo["IntegratedToolRepository"] - RestProxyService --> Resolver["ProxyUrlResolver"] - Resolver --> TargetTool["Integrated Tool API"] -``` - ---- - -# Rest Proxy Service - -The `RestProxyService` performs secure HTTP forwarding. - -## Responsibilities - -1. Validate tool existence -2. Verify tool is enabled -3. Resolve upstream URL -4. Attach tool credentials -5. Forward request -6. Return upstream response - -## Credential Injection - -Based on `APIKeyType`: - -```text -HEADER β†’ Custom header injection -BEARER_TOKEN β†’ Authorization: Bearer -NONE β†’ No credential -``` - -## HTTP Client Configuration - -- Connection timeout: 10 seconds -- Response timeout: 60 seconds -- Apache HttpClient 5 - -The service preserves: - -- HTTP method -- Request body -- Response status code -- Response body - ---- - -# Pagination Model - -All list endpoints use cursor-based pagination. - -Components: - -- `CursorPaginationCriteria.fromRest(cursor, limit)` -- `SortInput.from(sortField, sortDirection)` - -Benefits: - -- Stable pagination -- Scalable large dataset traversal -- No offset-based performance degradation - ---- - -# Error Handling - -Standard HTTP status codes are used: - -```text -200 Success -201 Created -204 No Content -400 Bad Request -401 Unauthorized -403 Forbidden -404 Not Found -409 Conflict -429 Too Many Requests -500 Internal Server Error -``` - -Domain-specific exceptions (e.g., `DeviceNotFoundException`, `OrganizationNotFoundException`) are translated into structured error responses. - ---- - -# Data Sources and Dependencies - -The module integrates with: - -- MongoDB (devices, organizations, tools) -- Apache Pinot (log analytics) -- Integrated tool APIs (via proxy) -- Core domain services from the API layer - -It does not directly manage persistence; instead, it orchestrates existing services. - ---- - -# Key Design Characteristics - -## 1. Separation of Concerns - -- Controllers handle HTTP concerns -- Services perform business logic -- Mappers transform domain β†’ external DTO -- Proxy service handles external tool routing - -## 2. External-First Contract - -The REST surface is optimized for: - -- Predictable filtering -- Explicit query parameters -- Stable versioning (`/api/v1`) -- API key–based automation - -## 3. Observability - -All controllers log: - -- Request parameters -- User ID -- API key ID -- Pagination and sorting - -This enables auditability and traceability. - ---- - -# Summary - -The **External Api Service Core** module provides a secure, API key–driven REST interface for third-party integrations. - -It: - -- Exposes device, event, log, organization, and tool APIs -- Implements cursor-based pagination and rich filtering -- Proxies requests to integrated tools -- Enforces API key authentication -- Publishes OpenAPI documentation - -It acts as the official external integration boundary of the OpenFrame platform, enabling automation, ecosystem integrations, and partner access without exposing internal GraphQL or domain-layer complexity. diff --git a/docs/reference/architecture/frontend-core-ui-and-chat/frontend-core-ui-and-chat.md b/docs/reference/architecture/frontend-core-ui-and-chat/frontend-core-ui-and-chat.md new file mode 100644 index 000000000..d23e8d657 --- /dev/null +++ b/docs/reference/architecture/frontend-core-ui-and-chat/frontend-core-ui-and-chat.md @@ -0,0 +1,444 @@ +# Frontend Core Ui And Chat + +The **Frontend Core Ui And Chat** module provides the reusable UI foundation and real-time AI chat experience for the OpenFrame platform. + +It is designed to be: + +- βœ… **Embeddable** – works inside Hub, standalone apps, or third-party platforms +- βœ… **Runtime-driven** – all navigation, identity, and endpoints flow through runtime context +- βœ… **Transport-agnostic** – supports SSE (Guide mode) and NATS/WebSocket (Mingo mode) +- βœ… **Design-system aligned** – built entirely on ODS tokens and shared UI primitives + +This module powers: + +- The floating **"Ask AI"** assistant panel +- AI-driven conversations (Guide + Mingo modes) +- Ticket Center +- Board (Kanban-style) views +- Notifications Drawer +- Data Tables and filtering surfaces +- Navigation and layout primitives + +--- + +## 1. Architectural Overview + +At a high level, the module sits on top of platform runtime and backend services while remaining UI-focused and transport-agnostic. + +```mermaid +flowchart LR + User["User"] --> UI["Frontend Core Ui And Chat"] + + subgraph ChatLayer["Chat System"] + EmbeddableChat["EmbeddableChat"] + Composer["ChatComposer"] + MessageRow["ChatMessageRow"] + DialogHistory["MingoChatHistory"] + end + + subgraph FeatureLayer["Feature Surfaces"] + TicketCenter["TicketCenter"] + Board["Board"] + Notifications["NotificationDrawer"] + DataTable["DataTable Components"] + end + + UI --> ChatLayer + UI --> FeatureLayer + + ChatLayer --> Runtime["Chat Runtime Context"] + ChatLayer --> Transport["SSE / NATS / WebSocket"] + + Transport --> Backend["API Service + Stream Processing"] +``` + +### Key Concepts + +- **Runtime-first design**: `useRequiredChatRuntime()` injects identity, endpoints, platform source, and navigation behavior. +- **Mode-based chat engine**: Guide (SSE/RAG) and Mingo (NATS/agent) are pluggable via `useUnifiedChat`. +- **Composable UI primitives**: All surfaces share ODS-based components (Button, Card, Drawer, Tag, etc.). +- **Strict separation of concerns**: + - Transport + state: hooks + - Rendering: components + - Platform integration: runtime context + +--- + +# 2. Chat System + +The Chat system is the core of this module and is built around `EmbeddableChat`. + +## 2.1 EmbeddableChat + +**Core component:** + +- `EmbeddableChatProps` + +`EmbeddableChat` is a portable AI assistant panel that: + +- Supports controlled and uncontrolled open state +- Can render inside a Drawer shell or as a shell-less body +- Switches between Guide and Mingo modes +- Manages dialog history, archive view, rename/archive modals +- Renders streaming messages with inline entity cards + +### Mode Architecture + +```mermaid +flowchart TD + EmbeddableChat --> UnifiedChat["useUnifiedChat"] + + UnifiedChat --> GuideMode["Guide Mode (SSE)"] + UnifiedChat --> MingoMode["Mingo Mode (NATS)"] + + GuideMode --> SSEAdapter["useSseChatAdapter"] + MingoMode --> NatsAdapter["useNatsChatAdapter"] + + SSEAdapter --> SSEBackend["API Service (SSE)"] + NatsAdapter --> NatsBackend["Stream Processing (NATS)"] +``` + +### Responsibilities + +- Floating trigger button +- Drawer rendering (Radix-based) +- Identity-aware greeting +- Slash command onboarding +- Attachment support (Guide-only) +- Source citation chips +- Archive page +- Inline entity card dispatch + +--- + +## 2.2 Message Model + +The chat uses a **segment-based message architecture**. + +Core message types: + +- `TextMessageData` +- `ExecutedToolMessageData` +- `AIMetadataMessageData` +- `ApprovalRequest` +- `ApprovalBatchData` + +Messages are rendered using: + +- `ChatMessageRow` +- `ChatMessageList` +- Segment-aware renderers (tool execution, approvals, thinking, etc.) + +### Segment Flow + +```mermaid +flowchart LR + Chunk["ChunkData"] --> Processor["useRealtimeChunkProcessor"] + Processor --> Segments["MessageSegment[]"] + Segments --> Message["Message"] + Message --> UI["ChatMessageRow"] +``` + +This design allows: + +- Streaming token updates +- Inline tool execution state +- Approval workflows +- Context compaction visualization + +--- + +## 2.3 Chat Composer + +**Core component:** + +- `ChatComposerProps` + +`ChatComposer` renders: + +- `ChatInput` +- Model usage row (`ModelDisplay`) +- Attachment add button (Guide mode only) +- Archived state placeholder + +It integrates with: + +- Slash commands +- Token usage reporting +- Streaming stop control +- Attachment staging and upload state + +--- + +## 2.4 Dialog History & Archive + +Core components: + +- `MingoChatHistoryProps` +- `ChatArchivePageProps` + +Features: + +- Grouping: Today / Yesterday / Older +- Infinite scroll with sentinel +- Rename / Archive actions +- Restore archived dialogs +- Scroll fade affordances + +```mermaid +flowchart TD + DialogList --> History["MingoChatHistory"] + History --> Row["History Row"] + Row --> Rename["Rename Modal"] + Row --> Archive["Archive Modal"] + Archive --> ArchivePage["ChatArchivePage"] +``` + +--- + +# 3. Ticket Center + +**Core component:** + +- `TicketCenterProps` + +The Ticket Center provides a customer-facing support surface tightly integrated with chat identity. + +### Identity Gate + +```mermaid +flowchart TD + Identity["useChatIdentity"] --> Check{Auth Tier?} + Check -->|anon| Empty["Sign-in EmptyState"] + Check -->|authenticated| TicketCenterAuthed +``` + +### Features + +- Optimistic ticket creation +- Inline conversation threads +- Close / reopen actions +- Refetch and cache management +- Support system health awareness + +TicketCenter intentionally does not bundle providers; embedders supply: + +- QueryClientProvider +- ChatRuntimeContext + +--- + +# 4. Board (Kanban Surface) + +Core components: + +- `BoardProps` +- `TicketCardProps` + +The Board implements a drag-and-drop Kanban system using `@dnd-kit`. + +### Drag Flow + +```mermaid +flowchart LR + Ticket["TicketCard"] --> DndContext + DndContext --> DragStart + DragStart --> DragOver + DragOver --> DragEnd + DragEnd --> OnChange["BoardChange"] +``` + +Capabilities: + +- Cross-column drag with allowedFromColumns rules +- Optimistic reordering +- Infinite scroll per column +- Overlay preview during drag +- Horizontal scrollbar with custom thumb + +--- + +# 5. Notifications Drawer + +**Core component:** + +- `NotificationDrawerProps` + +Features: + +- Right-side Drawer +- Infinite scroll unread list +- "Complete All" bulk action +- Live pop-up toggle +- History navigation callback + +```mermaid +flowchart TD + NotificationContext --> Drawer + Drawer --> UnreadList + UnreadList --> Tile["NotificationTile"] + Tile --> MarkRead +``` + +--- + +# 6. Data Table System + +Core components: + +- `DataTableHeaderProps` +- `DataTableRowProps` +- `TableProps` (deprecated legacy types) + +Built on: + +- `@tanstack/react-table` + +### Header Capabilities + +- Consumer-driven sorting +- Filter dropdown integration +- Sticky header support +- Responsive column hiding + +### Row Behavior + +- Full-row link mode +- Action bubbling guard via `data-no-row-click` +- Portal-safe click handling + +```mermaid +flowchart LR + Table --> Header + Table --> Row + Header --> SortChange + Row --> RowClick +``` + +--- + +# 7. Navigation & Layout Primitives + +Core components: + +- `HeaderProps` +- `NavigationSidebarProps` +- `PageLayoutProps` + +### Sidebar Behavior + +```mermaid +flowchart TD + Viewport --> Breakpoint{md / lg?} + Breakpoint -->|Desktop| PersistentSidebar + Breakpoint -->|Tablet| OverlaySidebar + OverlaySidebar --> Backdrop +``` + +Features: + +- Persisted minimized state (localStorage) +- Tablet overlay mode +- Escape-to-close +- Disabled state for full navigation lock + +--- + +# 8. Shared UI Components + +Additional reusable UI primitives included: + +- `OrganizationCard` +- `TagsManager` +- `FilterModal` +- `ChatTicketList` + +These are fully ODS-aligned and used across: + +- Board +- Ticket Center +- Admin surfaces +- Data listing pages + +--- + +# 9. Network & Transport Types + +Core transport types: + +- `NetworkResponse` +- `PaginatedResponse` +- `WebSocketConfig` +- `WebSocketMessage` +- `ChunkData` + +### Real-Time Pipeline + +```mermaid +flowchart LR + Backend --> NATS["NATS / JetStream"] + Backend --> SSE["SSE"] + + NATS --> Subscription["useNatsDialogSubscription"] + SSE --> Adapter["useSseChatAdapter"] + + Subscription --> Processor + Adapter --> Processor +``` + +The real-time system supports: + +- Reconnection backoff tuning +- Chunk catch-up +- Token usage telemetry +- Approval lifecycle events + +--- + +# 10. How This Module Fits Into OpenFrame + +The **Frontend Core Ui And Chat** module is the presentation and interaction layer for: + +- `api-service-core-http-and-graphql` (SSE Guide mode) +- `stream-processing-kafka` (via NATS agent streaming) +- `authorization-service-core` (identity via runtime) +- `gateway-service-core` (WebSocket + routing) + +It does **not**: + +- Own authentication +- Own data persistence +- Own API contracts + +Instead, it consumes them via: + +- Runtime context +- Hooks +- Transport adapters + +--- + +# 11. Design Principles + +1. **Runtime abstraction over hard imports** +2. **Segment-first message rendering** +3. **Feature-flag-driven extensibility** +4. **Platform-neutral embedding** +5. **Strict separation of transport and presentation** + +--- + +# Conclusion + +The **Frontend Core Ui And Chat** module is the reusable UI and AI interaction layer of OpenFrame. + +It provides: + +- A fully embeddable AI assistant +- Real-time streaming with approval workflows +- Ticket management +- Drag-and-drop board workflows +- Data-table and filtering primitives +- Responsive navigation framework + +By centralizing chat logic, UI components, and interaction patterns in a single portable module, OpenFrame ensures consistent UX across Hub, customer portals, and embedded third-party environments. \ No newline at end of file diff --git a/docs/reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md b/docs/reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md deleted file mode 100644 index 8920d6d19..000000000 --- a/docs/reference/architecture/gateway-service-core-security-and-routing/gateway-service-core-security-and-routing.md +++ /dev/null @@ -1,368 +0,0 @@ -# Gateway Service Core Security And Routing - -## Overview - -The **Gateway Service Core Security And Routing** module is the reactive edge layer of the OpenFrame platform. It is responsible for: - -- Acting as the **entry point** for REST and WebSocket traffic -- Enforcing **JWT-based authentication and role-based authorization** -- Providing **API key authentication and rate limiting** for external APIs -- Dynamically resolving **multi-tenant issuers** -- Routing and proxying requests to integrated tools (e.g., MeshCentral, Tactical RMM) -- Handling WebSocket proxying and session decoration - -Built on **Spring Cloud Gateway + WebFlux + Netty**, it provides a fully reactive, non-blocking gateway optimized for high concurrency and real-time integrations. - ---- - -## High-Level Architecture - -```mermaid -flowchart TD - Client["Client / Agent / UI"] --> Gateway["Gateway Service Core Security And Routing"] - - subgraph security_layer["Security Layer"] - AuthHeader["AddAuthorizationHeaderFilter"] - JwtConfig["JwtAuthConfig"] - SecurityConfig["GatewaySecurityConfig"] - ApiKeyFilter["ApiKeyAuthenticationFilter"] - OriginFilter["OriginSanitizerFilter"] - end - - subgraph routing_layer["Routing & Proxy Layer"] - IntegrationCtrl["IntegrationController"] - WsConfig["WebSocketGatewayConfig"] - UpstreamResolvers["ToolUpstreamResolvers"] - end - - Gateway --> security_layer - security_layer --> routing_layer - routing_layer --> Upstream["Integrated Tools / NATS / Services"] -``` - -The module is divided into four major concerns: - -1. **Reactive Server & Client Configuration (Netty + WebClient)** -2. **Security (JWT, API Keys, Roles, CORS, Origin)** -3. **WebSocket Routing & Decoration** -4. **Tool Upstream Resolution & Proxying** - ---- - -## Reactive Infrastructure Configuration - -### NettySocketConfig - -Configures low-level TCP behavior for both: - -- Embedded Netty web server -- Gateway outbound HTTP client -- Reactor Netty WebSocket client - -Key socket options: - -- `SO_LINGER = 0` (fast connection teardown) -- `TCP_NODELAY = true` (low-latency communication) - -This ensures optimal performance for high-throughput proxy and WebSocket traffic. - ---- - -### WebClientConfig - -Provides a preconfigured `WebClient.Builder` with: - -- 30s connection timeout -- 30s response timeout -- Read/write timeout handlers - -Used internally for REST proxying to upstream tools. - ---- - -## Security Architecture - -Security is layered and reactive. It supports: - -- Multi-tenant JWT validation -- Role-based access control -- API key authentication for external APIs -- Bearer token resolution from multiple sources -- CORS control -- Origin header sanitization - -### Security Flow - -```mermaid -flowchart TD - Request["Incoming Request"] --> Origin["OriginSanitizerFilter"] - Origin --> AuthHeader["AddAuthorizationHeaderFilter"] - AuthHeader --> JwtValidation["JWT Authentication"] - JwtValidation --> RoleCheck["Role Authorization Rules"] - RoleCheck --> ApiKeyCheck{"External API Path?"} - ApiKeyCheck -->|Yes| ApiKeyFilter["ApiKeyAuthenticationFilter"] - ApiKeyCheck -->|No| Continue["Continue Routing"] - ApiKeyFilter --> Continue - Continue --> Controller["Controller / Route"] -``` - ---- - -## JWT Authentication & Multi-Tenant Support - -### JwtAuthConfig - -- Uses a **Caffeine cache** to store `ReactiveAuthenticationManager` instances per issuer -- Dynamically builds JWT decoders per tenant -- Supports: - - Platform issuer (local public key) - - External issuers via OIDC discovery - -### IssuerUrlProvider - -Resolves allowed issuers dynamically from tenant configuration. - -- Reads tenants reactively -- Builds issuer URLs using: - - `allowed-issuer-base` - - Tenant ID - - Optional super-tenant ID -- Caches issuer URLs - -This enables strict issuer validation in multi-tenant deployments. - ---- - -## GatewaySecurityConfig - -Defines the reactive security filter chain. - -Key behavior: - -- Disables CSRF, HTTP Basic, form login -- Enables OAuth2 Resource Server -- Uses `ReactiveAuthenticationManagerResolver` for issuer-based validation -- Injects `AddAuthorizationHeaderFilter` before authentication - -### Role Rules - -- `/api/**` β†’ ADMIN -- `/tools/agent/**` β†’ AGENT -- `/ws/tools/agent/**` β†’ AGENT -- `/ws/nats` β†’ AGENT or ADMIN -- `/guide/**` β†’ ADMIN -- `/clients/**` β†’ AGENT - -All other paths default to `permitAll()` unless matched earlier. - ---- - -## AddAuthorizationHeaderFilter - -Ensures a Bearer token exists by resolving it from: - -1. Access token cookie -2. Custom header -3. Query parameter - -If found, it injects: - -```text -Authorization: Bearer -``` - -This enables WebSocket and browser flows that cannot always set headers directly. - ---- - -## API Key Authentication & Rate Limiting - -### ApiKeyAuthenticationFilter - -A `GlobalFilter` applied to `/external-api/**` endpoints. - -Flow: - -```mermaid -flowchart TD - Req["/external-api/**"] --> CheckKey["Check X-API-Key Header"] - CheckKey --> Valid{"Valid Key?"} - Valid -->|No| Reject401["Return 401"] - Valid -->|Yes| RateCheck["RateLimitService.isAllowed"] - RateCheck --> Allowed{"Allowed?"} - Allowed -->|No| Reject429["Return 429 + Retry-After"] - Allowed -->|Yes| AddHeaders["Add RateLimit Headers"] - AddHeaders --> AddContext["Inject X-User-Id + X-Api-Key-Id"] - AddContext --> Forward["Forward to External API"] -``` - -Capabilities: - -- Validates API key -- Increments usage stats -- Enforces minute/hour/day limits -- Adds rate limit headers -- Injects user context headers - ---- - -## WebSocket Gateway Configuration - -### WebSocketGatewayConfig - -Defines WebSocket routing using Spring Cloud Gateway. - -Routes: - -- `/ws/tools/agent/{toolId}/**` -- `/ws/tools/{toolId}/**` -- `/ws/nats` -- `/ws/nats-api` - -It also decorates: - -- `WebSocketClient` (optional proxy cleanup) -- `WebSocketService` (security + metrics decorator) - ---- - -### WebSocket Proxy URL Filters - -#### ToolAgentWebSocketProxyUrlFilter - -- Extracts `toolId` from agent path -- Resolves upstream WebSocket URL - -#### ToolApiWebSocketProxyUrlFilter - -- Extracts `toolId` from API path -- Removes `Origin` header -- Injects tool-specific API key headers - -These filters delegate actual resolution to the upstream resolver registry. - ---- - -## Tool Upstream Resolution - -The gateway uses a strategy pattern for resolving tool destinations. - -```mermaid -flowchart LR - Request["Tool Request"] --> Registry["ToolUpstreamResolverRegistry"] - Registry -->|Specific Tool| Mesh["MeshCentralUpstreamResolver"] - Registry -->|Specific Tool| Tactical["TacticalRmmUpstreamResolver"] - Registry -->|Fallback| Default["DefaultToolUpstreamResolver"] - Mesh --> Proxy - Tactical --> Proxy - Default --> Proxy -``` - -### DefaultToolUpstreamResolver - -- Reads tool URLs from MongoDB -- Uses `ToolUrlService` -- Resolves API and WS URLs - -### MeshCentralUpstreamResolver - -- Uses configuration properties -- Avoids DB lookup -- Supports path prefix injection - -### TacticalRmmUpstreamResolver - -- Routes REST to backend -- Routes WebSocket to: - - NATS (if path matches) - - Daphne (otherwise) - -This allows fine-grained routing without an intermediate nginx layer. - ---- - -## CORS & Origin Handling - -### CorsConfig - -- Enabled unless explicitly disabled -- Uses Spring Cloud Gateway global CORS properties - -### OriginSanitizerFilter - -- Removes `Origin: null` headers -- Prevents invalid origin propagation to upstream services - ---- - -## Internal Auth Probe - -### InternalAuthProbeController - -Conditional endpoint: - -``` -GET /internal/authz/probe -``` - -Used for health checks in internal deployments. - ---- - -## Path Constants - -`PathConstants` centralizes prefix definitions: - -- `/clients` -- `/api` -- `/tools` -- `/ws/tools` -- `/chat` -- `/guide` - -Ensures consistent routing and security matching. - ---- - -## Request Lifecycle Summary - -```mermaid -flowchart TD - Client --> Gateway - Gateway --> OriginFilter - OriginFilter --> AuthHeader - AuthHeader --> JwtResolver - JwtResolver --> RoleAuth - RoleAuth --> RouteDecision - RouteDecision -->|REST| IntegrationController - RouteDecision -->|WebSocket| WebSocketGatewayConfig - IntegrationController --> UpstreamResolver - WebSocketGatewayConfig --> UpstreamResolver - UpstreamResolver --> IntegratedTool -``` - ---- - -## Key Design Principles - -- Fully reactive (WebFlux + Reactor) -- Strict multi-tenant issuer validation -- Pluggable upstream resolution per tool -- Centralized security enforcement at the edge -- API key rate limiting for external APIs -- Optimized Netty socket configuration -- Clean separation between security, routing, and proxy concerns - ---- - -## Conclusion - -The **Gateway Service Core Security And Routing** module is the secure, reactive edge of the OpenFrame platform. It combines: - -- JWT-based multi-tenant authentication -- Role-based authorization -- API key validation and rate limiting -- WebSocket proxying and decoration -- Intelligent upstream resolution - -It ensures that all traffic entering the platform is authenticated, authorized, rate-limited, and routed efficiently to the correct internal or integrated service. \ No newline at end of file diff --git a/docs/reference/architecture/gateway-service-core/gateway-service-core.md b/docs/reference/architecture/gateway-service-core/gateway-service-core.md new file mode 100644 index 000000000..d7ddf2952 --- /dev/null +++ b/docs/reference/architecture/gateway-service-core/gateway-service-core.md @@ -0,0 +1,367 @@ +# Gateway Service Core + +The **Gateway Service Core** module is the reactive edge layer of the OpenFrame platform. It acts as the unified entry point for: + +- REST API traffic +- WebSocket connections +- Tool integrations (MeshCentral, Tactical RMM, and others) +- External API access with API keys and rate limiting +- Multi-tenant JWT validation and security enforcement + +Built on **Spring Cloud Gateway + WebFlux (Netty)**, this module provides high-performance, non-blocking request routing and security orchestration between frontend clients, agents, and downstream services. + +--- + +## 1. Architectural Overview + +The Gateway Service Core sits between clients (UI, agents, integrations) and internal services or integrated tools. + +```mermaid +flowchart LR + Browser["Frontend UI"] --> Gateway["Gateway Service Core"] + Agent["OpenFrame Agent"] --> Gateway + ExternalAPI["External API Client"] --> Gateway + + Gateway --> ApiService["API Service Core"] + Gateway --> AuthService["Authorization Service Core"] + Gateway --> Management["Management Service Core"] + Gateway --> ToolMesh["MeshCentral"] + Gateway --> ToolTactical["Tactical RMM"] + Gateway --> Nats["NATS WebSocket"] +``` + +### Responsibilities + +1. **Authentication & Authorization** (JWT, roles, scopes) +2. **API Key validation and rate limiting** +3. **Tenant-aware issuer resolution** +4. **REST and WebSocket proxying** +5. **Tool-specific upstream resolution strategies** +6. **CORS and security header enforcement** +7. **Reactive performance tuning (Netty + WebClient)** + +--- + +## 2. Core Layers Inside the Gateway + +```mermaid +flowchart TD + Netty["Netty & WebFlux Runtime"] --> Filters["Security & Global Filters"] + Filters --> Controllers["REST Controllers"] + Filters --> WsRoutes["WebSocket Routes"] + + Controllers --> RestProxy["REST Proxy Layer"] + WsRoutes --> WsProxy["WebSocket Proxy Layer"] + + RestProxy --> UpstreamResolvers["Tool Upstream Resolvers"] + WsProxy --> UpstreamResolvers + + UpstreamResolvers --> Downstream["Integrated Tools / Services"] +``` + +The module is structured around five main concerns: + +- **Runtime & Networking Configuration** +- **Security & JWT Infrastructure** +- **API Key & Rate Limiting** +- **Tool Routing & Upstream Resolution** +- **WebSocket Proxying** + +--- + +# 3. Runtime & Networking Configuration + +### NettySocketConfig + +Optimizes the embedded Netty server and client: + +- Disables `SO_LINGER` +- Enables `TCP_NODELAY` +- Customizes both server and HTTP client behavior +- Provides a dedicated `ReactorNettyWebSocketClient` + +This ensures low-latency proxy behavior for high-frequency WebSocket and REST traffic. + +### WebClientConfig + +Defines a tuned `WebClient.Builder` with: + +- 30s connect timeout +- 30s read/write timeout +- Response timeout configuration + +Used internally by proxy and integration services. + +--- + +# 4. Security Architecture + +Security in the Gateway Service Core is fully reactive and role-driven. + +## 4.1 GatewaySecurityConfig + +Configures: + +- OAuth2 Resource Server (JWT validation) +- Role-based access rules +- WebFlux security filter chain +- Endpoint-level role constraints + +### Role Model + +- `ROLE_ADMIN` +- `ROLE_AGENT` +- `SCOPE_*` authorities + +### Path-Based Access + +Path constants define security zones: + +- `/api/**` β†’ ADMIN +- `/tools/agent/**` β†’ AGENT +- `/ws/tools/**` β†’ ADMIN or AGENT (depending on route) +- `/clients/**` β†’ AGENT +- `/chat/**` β†’ AGENT or ADMIN + +--- + +## 4.2 JWT Multi-Tenant Validation + +### JwtAuthConfig + +Implements: + +- Dynamic issuer-based authentication manager resolution +- Caffeine cache of `ReactiveAuthenticationManager` +- Strict issuer validation + +```mermaid +flowchart TD + Request["Incoming JWT"] --> IssuerResolver["Issuer Resolver"] + IssuerResolver --> Cache["Caffeine Cache"] + Cache --> JwtDecoder["JWT Decoder"] + JwtDecoder --> Validator["Issuer + Default Validators"] + Validator --> Authenticated["Authenticated Principal"] +``` + +### DefaultIssuerUrlProvider + +OSS fallback for single-tenant deployments: + +- Uses `allowed-issuer-base + super-tenant-id` +- Avoids database lookup + +--- + +## 4.3 Authorization Header Injection + +### AddAuthorizationHeaderFilter + +Pre-auth filter that ensures an `Authorization` header exists by resolving the token from: + +1. `access_token` cookie +2. Custom header +3. Query parameter + +This enables: + +- WebSocket auth +- Browser cookie-based sessions +- Backward compatibility with legacy clients + +--- + +## 4.4 CORS Handling + +Two configurations: + +- **CorsConfig** β†’ Standard configurable CORS +- **CorsDisableConfig** β†’ Permissive CORS (SaaS mode) + +Controlled via `openframe.gateway.disable-cors` property. + +--- + +# 5. API Key Authentication & Rate Limiting + +The gateway protects `/external-api/**` endpoints via a global filter. + +## ApiKeyAuthenticationFilter + +Flow: + +```mermaid +flowchart TD + Req["Request /external-api/**"] --> CheckHeader["X-API-Key Present?"] + CheckHeader -->|"No"| Reject["401 Unauthorized"] + CheckHeader -->|"Yes"| Validate["Validate API Key"] + Validate -->|"Invalid"| Reject + Validate -->|"Valid"| RateCheck["Rate Limit Check"] + RateCheck -->|"Exceeded"| TooMany["429 Too Many Requests"] + RateCheck -->|"Allowed"| Forward["Forward Request"] +``` + +### Responsibilities + +- Validates API key +- Increments usage statistics +- Applies minute/hour/day limits +- Adds headers: + - `X-RateLimit-Limit-*` + - `X-RateLimit-Remaining-*` +- Injects user context headers + +This enables secure external tool integrations and third-party automation. + +--- + +# 6. REST Tool Proxying + +## IntegrationController + +Handles tool-based routing under: + +- `/tools/{toolId}/**` +- `/tools/agent/{toolId}/**` + +Capabilities: + +- Health checks +- Integration tests +- Transparent REST proxying + +Delegates to: + +- `IntegrationService` +- `RestProxyService` + +--- + +# 7. WebSocket Gateway + +WebSocket routing is handled via Spring Cloud Gateway route definitions. + +## WebSocketGatewayConfig + +Defines routes for: + +- `/ws/tools/agent/{toolId}/**` +- `/ws/tools/{toolId}/**` +- `/ws/nats` +- `/ws/nats-api` + +Also provides: + +- Optional proxy session cleanup +- WebSocket security decorator +- Traffic metrics integration + +```mermaid +flowchart LR + ClientWS["WebSocket Client"] --> GatewayWS["WebSocket Gateway"] + GatewayWS --> ToolResolver["Tool Upstream Resolver"] + ToolResolver --> Mesh["MeshCentral"] + ToolResolver --> Tactical["Tactical RMM"] + GatewayWS --> Nats["NATS"] +``` + +--- + +# 8. Tool Upstream Resolution Strategy + +The gateway uses a pluggable `ToolUpstreamResolver` strategy. + +## DefaultToolUpstreamResolver + +- Reads tool URLs from Mongo +- Supports REST and WS +- Fallback for all tools + +## MeshCentralUpstreamResolver + +Optimized routing: + +- Avoids Mongo lookup per request +- Reads config from `openframe.tools.meshcentral.*` +- Supports path prefix injection for tenant scoping + +## TacticalRmmUpstreamResolver + +Path-aware routing: + +- REST β†’ backend +- WS β†’ Daphne +- WS with NATS path prefix β†’ NATS upstream + +```mermaid +flowchart TD + Request["Tool Request"] --> ResolverRegistry["Resolver Registry"] + ResolverRegistry -->|"meshcentral-server"| MeshResolver["MeshCentral Resolver"] + ResolverRegistry -->|"tactical-rmm"| TacticalResolver["Tactical RMM Resolver"] + ResolverRegistry -->|"other"| DefaultResolver["Default Resolver"] +``` + +This design allows each tool to define: + +- Independent REST upstream +- Independent WebSocket upstream +- Path-based internal fan-out + +Without modifying core gateway logic. + +--- + +# 9. Origin Sanitization & Internal Probes + +## OriginSanitizerFilter + +Removes `Origin: null` headers to avoid WebSocket/CORS edge-case failures. + +## InternalAuthProbeController + +Optional internal probe endpoint: + +- `/internal/authz/probe` +- Enabled only via configuration + +Used for health or sidecar verification. + +--- + +# 10. Interaction with Other Modules + +The Gateway Service Core coordinates with: + +- **Authorization Service Core** β†’ Issues JWT tokens +- **API Service Core (HTTP & GraphQL)** β†’ Business APIs +- **Management Service Core** β†’ Tool initialization and lifecycle +- **Data Models & Mongo Repositories** β†’ Tool and API key metadata +- **Stream Processing (Kafka)** β†’ Event-driven backend processing +- **Frontend Core UI & Chat** β†’ Consumes REST and WebSocket endpoints + +The gateway itself remains stateless and reactive, delegating persistence and business logic to downstream modules. + +--- + +# 11. Design Principles + +1. **Reactive First** – Non-blocking I/O everywhere +2. **Multi-Tenant Secure by Default** +3. **Pluggable Tool Routing** +4. **Protocol-Agnostic Proxy (REST + WS)** +5. **Clear Role Segmentation (ADMIN vs AGENT)** +6. **OSS and SaaS Compatibility** + +--- + +# Conclusion + +The **Gateway Service Core** is the high-performance security and routing backbone of OpenFrame. It: + +- Enforces authentication and authorization +- Validates API keys and rate limits +- Dynamically resolves JWT issuers per tenant +- Transparently proxies REST and WebSocket traffic +- Implements tool-specific upstream routing strategies + +It enables OpenFrame to integrate diverse IT tools, agents, and clients behind a unified, secure, and reactive edge layer. \ No newline at end of file diff --git a/docs/reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md b/docs/reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md deleted file mode 100644 index 2d3ed0a48..000000000 --- a/docs/reference/architecture/management-service-core-initializers-and-schedulers/management-service-core-initializers-and-schedulers.md +++ /dev/null @@ -1,433 +0,0 @@ -# Management Service Core Initializers And Schedulers - -The **Management Service Core Initializers And Schedulers** module is responsible for bootstrapping, orchestrating, and maintaining background operational workflows across the OpenFrame platform. It ensures that required configuration artifacts, streams, secrets, scripts, and distributed jobs are initialized correctly at startup and remain consistent throughout runtime. - -This module acts as the operational backbone for: - -- System bootstrapping (initial secrets, client configuration, tool agents) -- Distributed scheduling with cluster-safe locking -- Stream and messaging initialization (NATS) -- External tool orchestration (e.g., Tactical RMM) -- Maintenance and retry workflows -- Pinot resynchronization and fleet setup tasks - ---- - -## Architectural Overview - -The module is structured into five major concerns: - -1. **Configuration Layer** – Enables scheduling, retries, and distributed locks. -2. **Initializers** – Bootstrap critical data and external integrations at startup. -3. **Schedulers** – Periodic maintenance and retry workflows. -4. **Controllers** – Operational endpoints for manual triggers. -5. **Processors & Services** – Extension points and domain orchestration logic. - -```mermaid -flowchart TD - subgraph ConfigLayer["Configuration Layer"] - MgmtConfig["ManagementConfiguration"] - RetryConfig["RetryConfiguration"] - ShedLock["ShedLockConfig"] - end - - subgraph Initializers["Startup Initializers"] - SecretInit["AgentRegistrationSecretInitializer"] - AgentInit["IntegratedToolAgentInitializer"] - NatsInit["NatsStreamConfigurationInitializer"] - ClientInit["OpenFrameClientConfigurationInitializer"] - TacticalInit["TacticalRmmScriptsInitializer"] - end - - subgraph Schedulers["Background Schedulers"] - VersionFallback["AgentVersionUpdatePublishFallbackScheduler"] - ApiKeySync["ApiKeyStatsSyncScheduler"] - Heartbeat["DeviceHeartbeatOfflineDetectionScheduler"] - FleetMdm["FleetMdmSetupScheduler"] - end - - subgraph Controllers["Operational Controllers"] - ToolCtrl["IntegratedToolController"] - PinotCtrl["DevicePinotResyncController"] - ReleaseCtrl["ReleaseVersionController"] - end - - MgmtConfig --> Initializers - ShedLock --> Schedulers - RetryConfig --> Schedulers - Controllers --> Schedulers - Initializers --> Schedulers -``` - ---- - -# 1. Configuration Layer - -## ManagementConfiguration - -Provides the base Spring configuration for the module. - -Key responsibilities: -- Component scanning across `com.openframe` -- Excludes `CassandraHealthIndicator` to avoid unnecessary health wiring -- Exposes a `PasswordEncoder` using BCrypt for secure hashing - -This ensures secure secret generation and compatibility with other security modules. - ---- - -## RetryConfiguration - -Enables Spring Retry using: - -```text -@EnableRetry -``` - -This allows services and schedulers to use retry semantics for transient failures (e.g., network, external systems, Redis, NATS). - ---- - -## ShedLockConfig - -Enables: - -- `@EnableScheduling` -- `@EnableSchedulerLock` - -Provides a Redis-backed distributed `LockProvider`. - -### Tenant-Scoped Locking - -Lock keys follow the pattern: - -```text -of:{tenantId}:job-lock:: -``` - -This ensures: -- No duplicate scheduler execution across clustered nodes -- Isolation per tenant -- Environment separation - -```mermaid -flowchart LR - Scheduler["Scheduled Job"] --> LockProvider["RedisLockProvider"] - LockProvider --> Redis[("Redis")] - Redis --> LockKey["Tenant Scoped Lock Key"] -``` - ---- - -# 2. Startup Initializers - -All initializers implement `ApplicationRunner`, meaning they execute after Spring Boot startup. - ---- - -## AgentRegistrationSecretInitializer - -Ensures an initial `AgentRegistrationSecret` exists at startup. - -Flow: - -```mermaid -flowchart TD - Start["Application Startup"] --> SecretInit["AgentRegistrationSecretInitializer"] - SecretInit --> Service["AgentRegistrationSecretManagementService"] - Service --> Create["createInitialSecret()"] - Create --> Processor["DefaultAgentRegistrationSecretManagementProcessor"] -``` - -Features: -- Idempotent secret creation -- Pluggable post-processing via `AgentRegistrationSecretManagementProcessor` -- Safe failure handling - ---- - -## IntegratedToolAgentInitializer - -Loads agent configurations from classpath resources. - -Responsibilities: -- Reads JSON configuration files -- Maps to `IntegratedToolAgentConfiguration` -- Updates persisted configuration via `IntegratedToolAgentService` -- Fails fast if no configuration is defined - -This guarantees consistent tool-agent configuration across environments. - ---- - -## NatsStreamConfigurationInitializer - -Pre-provisions required NATS streams: - -- TOOL_INSTALLATION -- CLIENT_UPDATE -- TOOL_UPDATE -- TOOL_CONNECTIONS -- INSTALLED_AGENTS - -Each stream defines: -- Subjects (wildcard-based routing) -- File storage -- Retention policy - -Also supports extension via `AdditionalStreamConfigurationProvider`. - -```mermaid -flowchart TD - Init["NatsStreamConfigurationInitializer"] --> DefaultStreams["Default Stream Configurations"] - Init --> Additional["AdditionalStreamConfigurationProvider"] - DefaultStreams --> NatsService["NatsStreamManagementService"] - Additional --> NatsService -``` - ---- - -## OpenFrameClientConfigurationInitializer - -Loads the default client configuration from: - -```text -agent-configurations/client-configuration.json -``` - -Steps: -1. Deserialize JSON -2. Assign default ID -3. Persist via `OpenFrameClientConfigurationService` - -Ensures the client always has a canonical configuration baseline. - ---- - -## TacticalRmmScriptsInitializer - -Bootstraps PowerShell scripts inside Tactical RMM. - -Workflow: - -```mermaid -flowchart TD - Startup["Application Startup"] --> LoadTool["Load Tactical RMM Tool"] - LoadTool --> FetchScripts["Fetch Existing Scripts"] - FetchScripts --> Compare["Compare by Name"] - Compare -->|"Missing"| Create["Create Script"] - Compare -->|"Exists"| Update["Update Script"] -``` - -Features: -- Loads script bodies from classpath -- Idempotent create-or-update behavior -- Uses API key stored in IntegratedTool -- Logs per-script success/failure - -This guarantees consistent automation availability for OpenFrame client updates. - ---- - -# 3. Background Schedulers - -Schedulers run conditionally based on configuration properties and use ShedLock for distributed safety when needed. - ---- - -## AgentVersionUpdatePublishFallbackScheduler - -Purpose: Retry publishing client and tool-agent updates when previous attempts failed. - -Key concepts: -- Checks `PublishState` -- Retries until `maxPublishAttempts` -- Publishes via NATS publishers - -```mermaid -flowchart TD - Tick["Scheduled Tick"] --> ClientCheck["Check Client PublishState"] - ClientCheck --> RetryClient["Publish If Needed"] - Tick --> AgentCheck["Check Tool Agents"] - AgentCheck --> RetryAgent["Publish If Needed"] -``` - -This ensures eventual consistency for version update propagation. - ---- - -## ApiKeyStatsSyncScheduler - -Synchronizes API key statistics from Redis to MongoDB. - -Features: -- Distributed lock (`apiKeyStatsSync`) -- Configurable interval -- Safe retry behavior - -Ensures durable persistence of usage metrics. - ---- - -## DeviceHeartbeatOfflineDetectionScheduler - -Periodically marks stale devices as offline. - -Flow: - -```mermaid -flowchart LR - Scheduler["Heartbeat Scheduler"] --> Service["DeviceHeartbeatOfflineDetectionService"] - Service --> Mongo[("MongoDB Device Records")] -``` - -This keeps device health states accurate. - ---- - -## FleetMdmSetupScheduler - -Ensures Fleet MDM server is configured and API tokens are generated when necessary. - -Logic: -- Looks up `fleetmdm-server` tool -- Runs setup logic if present -- Retries on failure - -This supports automated fleet provisioning. - ---- - -# 4. Operational Controllers - -These controllers provide operational endpoints for manual intervention. - ---- - -## IntegratedToolController - -Endpoint base: `/v1/tools` - -Responsibilities: -- Retrieve tool configurations -- Save tool configuration -- Conditionally create/update Debezium connectors -- Execute post-save hooks - -```mermaid -flowchart TD - Client["API Client"] --> Controller["IntegratedToolController"] - Controller --> Save["IntegratedToolService.saveTool()"] - Save --> TenantCheck["TenantIdProvider"] - TenantCheck -->|"Registered"| Debezium["DebeziumService"] - Save --> Hooks["IntegratedToolPostSaveHook"] -``` - -Key design detail: -- Debezium connectors are deferred until tenant registration. - ---- - -## DevicePinotResyncController - -Endpoint: `/v1/devices/pinot-resync` - -Purpose: -- Replays all machines to Pinot -- Uses `MachineTagEventService.processMachineSaveAll` - -This is typically used for analytics rehydration or recovery. - ---- - -## ReleaseVersionController - -Endpoint: `/v1/cluster-registrations` - -Delegates version processing to `ReleaseVersionService`. - -Intended to coordinate cluster-level version awareness and propagation. - ---- - -# 5. Services and Extension Points - -## OpenFrameClientVersionUpdateService - -Currently serves as a coordination entry point for client version publishing. - -Uses: -- `OpenFrameClientUpdatePublisher` - -Designed for future expansion to centralize version update orchestration. - ---- - -## IntegratedToolPostSaveHook - -A lightweight extension mechanism: - -```text -void onToolSaved(String toolId, IntegratedTool tool) -``` - -Advantages: -- No Spring event overhead -- Simple service-specific side effects -- Clean separation of concerns - ---- - -## DefaultAgentRegistrationSecretManagementProcessor - -Default implementation used when no custom processor is defined. - -Behavior: -- Logs secret creation -- Supports override via Spring bean replacement - ---- - -# Runtime Interaction Summary - -The module ensures: - -1. Startup is safe and idempotent. -2. Distributed jobs execute only once per cluster. -3. External systems (NATS, Tactical RMM, Debezium) are reconciled. -4. Retry and fallback logic guarantee eventual consistency. -5. Operational APIs allow manual recovery and resync. - -```mermaid -flowchart TD - Boot["Application Boot"] --> Initializers - Initializers --> Messaging["NATS Streams Ready"] - Messaging --> Schedulers - Schedulers --> External["External Systems"] - Controllers --> External -``` - ---- - -# Key Design Principles - -- **Idempotency First** – Initializers and schedulers tolerate repeated execution. -- **Distributed Safety** – ShedLock prevents duplicate execution. -- **Tenant Isolation** – Lock keys and processing are tenant-scoped. -- **Eventual Consistency** – Fallback schedulers ensure propagation. -- **Extensibility** – Hooks and processors enable modular extension. - ---- - -# Conclusion - -The **Management Service Core Initializers And Schedulers** module is the operational orchestrator of OpenFrame. It ensures that the system boots correctly, remains consistent across distributed nodes, maintains external integrations, and continuously reconciles state through scheduled processes. - -Without this module, OpenFrame would lack: - -- Safe distributed background processing -- Automated stream and tool provisioning -- Reliable publish retry mechanisms -- Operational recovery endpoints - -It is a foundational module enabling stable, self-healing, multi-tenant management operations. \ No newline at end of file diff --git a/docs/reference/architecture/management-service-core/management-service-core.md b/docs/reference/architecture/management-service-core/management-service-core.md new file mode 100644 index 000000000..5089d81ed --- /dev/null +++ b/docs/reference/architecture/management-service-core/management-service-core.md @@ -0,0 +1,421 @@ +# Management Service Core + +## Overview + +The **Management Service Core** module is the operational backbone of the OpenFrame platform. It is responsible for: + +- Cluster and release version coordination +- Integrated tool lifecycle management +- Agent and client configuration bootstrapping +- Scheduled background maintenance tasks +- MongoDB migrations and data backfills +- Distributed job coordination via Redis-based locking + +While the API and Authorization modules expose external interfaces, the Management Service Core ensures that the platform is correctly initialized, synchronized, and maintained over time. + +--- + +## Architectural Role in the Platform + +The Management Service Core sits between infrastructure-level components (MongoDB, Redis, NATS, Kafka Connect) and higher-level services such as API, Gateway, and Stream Processing. + +```mermaid +flowchart TD + Gateway["Gateway Service Core"] --> API["API Service Core HTTP and GraphQL"] + API --> Data["Data Models and Repositories Mongo"] + API --> Stream["Stream Processing Kafka"] + + Management["Management Service Core"] --> Data + Management --> Redis["Redis"] + Management --> NATS["NATS Streams"] + Management --> KafkaConnect["Kafka Connect / Debezium"] + + Stream --> Data +``` + +### Responsibilities by Layer + +| Layer | Responsibility | +|--------|----------------| +| Configuration | Bootstraps encoders, retry, scheduling, locking | +| Initialization | Seeds secrets, tools, streams, client configs | +| Controllers | Administrative operational endpoints | +| Migrations | Schema and data evolution (Mongock) | +| Schedulers | Periodic distributed background tasks | +| Services | Domain-specific operational logic | +| Hooks | Extensibility points after tool persistence | + +--- + +# Core Configuration + +## ManagementConfiguration + +Provides the base Spring configuration and defines a `PasswordEncoder` bean using BCrypt for secure hashing. + +Key aspects: + +- Component scanning across `com.openframe` +- Excludes Cassandra health indicator +- Registers BCrypt password encoder + +This configuration ensures Management Service Core can operate independently without requiring Cassandra dependencies. + +--- + +## RetryConfiguration + +Enables Spring Retry across the module. + +```text +@Configuration +@EnableRetry +``` + +This allows service methods to use retry semantics for: + +- External integrations +- Transient infrastructure failures +- Messaging retries + +--- + +## ShedLockConfig + +Enables distributed scheduling using Redis-backed locks. + +```mermaid +flowchart LR + Scheduler["Scheduled Task"] --> ShedLock["ShedLock"] + ShedLock --> Redis["Redis Lock Provider"] +``` + +Key characteristics: + +- Multi-instance safe scheduling +- Tenant-scoped lock keys +- Environment-aware locking namespace + +Lock keys follow a tenant-scoped pattern: + +```text +of:{tenantId}:job-lock:{environment}:{lockName} +``` + +This guarantees only one instance of a scheduled job runs across the cluster. + +--- + +# Controllers + +The Management Service Core exposes operational endpoints that are typically invoked by internal services or cluster agents. + +--- + +## DevicePinotResyncController + +**Endpoint:** `POST /v1/devices/pinot-resync` + +Purpose: + +- Reads all machines from MongoDB +- Triggers `MachineTagEventService.processMachineSaveAll(...)` +- Forces re-indexing into analytical storage (e.g., Pinot) + +```mermaid +flowchart TD + Controller["DevicePinotResyncController"] --> Repo["MachineRepository"] + Controller --> EventService["MachineTagEventService"] + Repo --> Mongo["MongoDB"] + EventService --> Stream["Event Processing"] +``` + +This is primarily used for recovery and consistency repair. + +--- + +## IntegratedToolController + +**Base path:** `/v1/tools` + +Responsibilities: + +- Retrieve all integrated tools +- Retrieve tool by ID +- Create or update tool configuration + +### Save Flow + +```mermaid +flowchart TD + Request["Save Tool Request"] --> Service["IntegratedToolService"] + Service --> Mongo["MongoDB"] + Service --> Debezium["DebeziumService"] + Service --> Hooks["Post Save Hooks"] +``` + +Important behavior: + +- Preserves existing UUID if tool exists +- Ensures tenant scoping +- Enables tool automatically +- Conditionally applies Debezium connectors only after tenant registration +- Invokes all `IntegratedToolPostSaveHook` implementations + +### Extension Point + +`IntegratedToolPostSaveHook` allows lightweight side effects after persistence without using Spring events. + +--- + +## ReleaseVersionController + +**Endpoint:** `POST /v1/cluster-registrations` + +Accepts a `ReleaseVersionRequest` containing: + +```text +imageTagVersion: string +``` + +Delegates to `ReleaseVersionService` to process cluster version updates. + +This supports: + +- Cluster self-registration +- Rolling upgrade awareness +- Version-based feature control + +--- + +# Application Initializers + +Initializers run at application startup using `ApplicationRunner`. + +```mermaid +flowchart TD + Startup["Application Startup"] --> SecretInit["Agent Registration Secret"] + Startup --> ToolAgentInit["Integrated Tool Agent Config"] + Startup --> NatsInit["NATS Stream Config"] + Startup --> ClientInit["Client Configuration"] + Startup --> TacticalInit["Tactical RMM Scripts"] +``` + +--- + +## AgentRegistrationSecretInitializer + +- Ensures an initial agent registration secret exists +- Delegates to `AgentRegistrationSecretManagementService` +- Uses optional processor hook for post-processing + +Default behavior is implemented by `DefaultAgentRegistrationSecretManagementProcessor`. + +--- + +## IntegratedToolAgentInitializer + +- Loads agent configurations from classpath JSON files +- Parses via Jackson +- Applies configuration updates through `IntegratedToolAgentService` + +Fails fast if no configuration paths are provided. + +--- + +## NatsStreamConfigurationInitializer + +Bootstraps NATS streams: + +- TOOL_INSTALLATION +- CLIENT_UPDATE +- TOOL_UPDATE +- TOOL_CONNECTIONS +- INSTALLED_AGENTS + +```mermaid +flowchart LR + Init["NATS Stream Initializer"] --> Stream1["TOOL_INSTALLATION"] + Init --> Stream2["CLIENT_UPDATE"] + Init --> Stream3["TOOL_UPDATE"] + Init --> Stream4["TOOL_CONNECTIONS"] + Init --> Stream5["INSTALLED_AGENTS"] +``` + +Allows additional providers to contribute extra stream definitions. + +--- + +## OpenFrameClientConfigurationInitializer + +- Loads `client-configuration.json` +- Applies configuration through `OpenFrameClientConfigurationService` + +Ensures clients receive consistent update metadata. + +--- + +## TacticalRmmScriptsInitializer + +Automates Tactical RMM script lifecycle: + +- Reads PowerShell scripts from resources +- Checks existing scripts via TacticalRmmClient +- Creates or updates scripts + +This ensures remote management automation remains consistent across deployments. + +--- + +# Database Migrations (Mongock) + +The module uses Mongock `@ChangeUnit` migrations for safe schema evolution. + +```mermaid +flowchart TD + Migration["Mongock Migration"] --> BackfillVersion["Backfill Document Version"] + Migration --> BackfillOrder["Backfill Ticket Order"] + Migration --> StatusMigration["Migrate Ticket Status Model"] +``` + +## BackfillDocumentVersionChangeUnit + +Adds `documentVersion = 0` where missing in: + +- integrated_tool_agents +- openframe_client_configuration +- release_versions + +Tenant-scoped execution. + +--- + +## BackfillTicketOrdersChangeUnit + +- Assigns LexoRank ordering to tickets missing order +- Processes per `TicketStatus` +- Sorts by `createdAt` descending + +Ensures stable column ordering in ticket boards. + +--- + +## MigrateTicketStatusesChangeUnit + +Feature-flagged migration that: + +- Seeds system status definitions +- Converts legacy `status` field +- Writes `statusId` and `statusKind` +- Removes legacy field + +Supports lifecycle model rollout safely. + +--- + +# Scheduled Background Jobs + +Schedulers provide continuous system maintenance. + +```mermaid +flowchart TD + Scheduler["Schedulers"] --> AgentVersion["Agent Version Fallback"] + Scheduler --> ApiKeySync["API Key Stats Sync"] + Scheduler --> Heartbeat["Device Offline Detection"] + Scheduler --> FleetMdm["Fleet MDM Setup"] +``` + +--- + +## AgentVersionUpdatePublishFallbackScheduler + +- Publishes OpenFrame client updates +- Publishes tool agent updates +- Retries until `maxPublishAttempts` +- Uses distributed ShedLock + +Ensures reliable version propagation via NATS. + +--- + +## ApiKeyStatsSyncScheduler + +- Synchronizes API key usage statistics +- Syncs from Redis to MongoDB +- Protected by distributed lock + +Provides consistent billing/analytics state. + +--- + +## DeviceHeartbeatOfflineDetectionScheduler + +- Periodically marks stale devices offline +- Delegates to `DeviceHeartbeatOfflineDetectionService` + +Maintains accurate fleet health state. + +--- + +## FleetMdmSetupScheduler + +- Detects Fleet MDM tool presence +- Executes setup and token provisioning +- Retries automatically on failure + +Enables automated MDM provisioning. + +--- + +# Services + +## OpenFrameClientVersionUpdateService + +Designed to coordinate version update processing for OpenFrame clients. +Currently acts as a structural extension point for publishing version updates. + +--- + +## DefaultAgentRegistrationSecretManagementProcessor + +Fallback processor for agent registration secrets. + +- Executes after initial secret creation +- Intended for logging or minimal side effects +- Replaceable via custom bean implementation + +--- + +# Operational Characteristics + +## Multi-Tenancy + +- All migrations and schedulers operate with `TenantIdProvider` +- Redis locks are tenant-scoped +- Tool and configuration updates preserve tenant boundaries + +## Distributed Safety + +- ShedLock prevents duplicate execution +- Retry mechanisms handle transient failures +- Feature flags gate risky migrations + +## Extensibility + +- Post-save hooks for tools +- Additional NATS stream providers +- Replaceable secret processors + +--- + +# Summary + +The **Management Service Core** module ensures that OpenFrame environments: + +- Start in a consistent state +- Remain synchronized across distributed instances +- Safely evolve database schemas +- Maintain external tool integrations +- Execute reliable background operations + +It is the operational control plane of the platform β€” coordinating lifecycle, infrastructure, and system integrity across all services. \ No newline at end of file diff --git a/docs/reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md b/docs/reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md deleted file mode 100644 index 95246761a..000000000 --- a/docs/reference/architecture/security-core-and-oauth-bff/security-core-and-oauth-bff.md +++ /dev/null @@ -1,351 +0,0 @@ -# Security Core And Oauth Bff - -## Overview - -The **Security Core And Oauth Bff** module provides the foundational security building blocks for the OpenFrame platform. It combines: - -- JWT encoding and decoding infrastructure -- RSA key configuration and loading -- PKCE (Proof Key for Code Exchange) utilities -- OAuth2 Backend-for-Frontend (BFF) endpoints -- Redirect resolution and state management - -This module acts as the bridge between the platform’s Authorization Server, Gateway, and frontend clients. It centralizes token handling, cookie management, and OAuth flows while remaining configurable and extensible. - ---- - -## Architectural Role in the Platform - -At runtime, Security Core And Oauth Bff sits between: - -- Frontend clients (browser-based applications) -- The Authorization Server -- The Gateway layer - -It is responsible for: - -- Initiating OAuth2 authorization flows -- Managing PKCE and state -- Exchanging authorization codes for tokens -- Storing tokens securely in HTTP-only cookies -- Refreshing and revoking tokens -- Providing JWT encoder/decoder beans for internal services - -### High-Level Architecture - -```mermaid -flowchart LR - Browser["Browser Client"] -->|"GET /oauth/login"| Bff["OAuth BFF Controller"] - Bff -->|"Redirect to authorize"| AuthServer["Authorization Server"] - AuthServer -->|"Callback with code"| Bff - Bff -->|"Exchange code for tokens"| AuthServer - Bff -->|"Set HttpOnly Cookies"| Browser - - Browser -->|"API calls with cookies"| Gateway["Gateway Service"] - Gateway -->|"Validate JWT"| JwtDecoder["JwtDecoder Bean"] -``` - ---- - -## Core Components - -### 1. JwtSecurityConfig - -**Class:** `JwtSecurityConfig` - -This Spring configuration class provides the core JWT infrastructure: - -- `JwtEncoder` bean using `NimbusJwtEncoder` -- `JwtDecoder` bean using `NimbusJwtDecoder` -- RSA-based signing and verification - -#### Responsibilities - -- Build a JWK set from configured RSA keys -- Expose a `JwtEncoder` for signing tokens -- Expose a `JwtDecoder` for validating tokens - -### JWT Bean Configuration Flow - -```mermaid -flowchart TD - JwtConfig["JwtConfig"] -->|"loadPublicKey()"| PublicKey["RSAPublicKey"] - JwtConfig -->|"loadPrivateKey()"| PrivateKey["RSAPrivateKey"] - - PublicKey --> RsaKey["RSAKey Builder"] - PrivateKey --> RsaKey - - RsaKey --> JwkSet["JWKSet"] - JwkSet --> JwtEncoder["NimbusJwtEncoder"] - - PublicKey --> JwtDecoder["NimbusJwtDecoder"] -``` - -This design ensures: - -- Strong asymmetric cryptography (RSA) -- Compatibility with Spring Security OAuth2 Resource Server -- Clean separation of configuration and usage - ---- - -### 2. JwtConfig - -**Class:** `JwtConfig` - -`JwtConfig` is a Spring `@ConfigurationProperties` service that binds properties under the `jwt` prefix. - -#### Properties - -- `publicKey` -- `privateKey` -- `issuer` -- `audience` - -#### Key Responsibilities - -- Load RSA public key -- Parse and decode PEM-encoded private key -- Produce `RSAPublicKey` and `RSAPrivateKey` instances - -The private key loading process: - -```mermaid -flowchart TD - PemString["PEM Private Key"] --> StripHeaders["Remove BEGIN/END markers"] - StripHeaders --> Base64Decode["Base64 Decode"] - Base64Decode --> KeySpec["PKCS8EncodedKeySpec"] - KeySpec --> KeyFactory["KeyFactory RSA"] - KeyFactory --> RsaPrivate["RSAPrivateKey"] -``` - -This encapsulation prevents direct cryptographic handling throughout the codebase and centralizes key logic. - ---- - -### 3. SecurityConstants - -**Class:** `SecurityConstants` - -Defines standardized names used across OAuth flows: - -- `authorization` query parameter -- `access_token` -- `refresh_token` -- `Access-Token` header -- `Refresh-Token` header - -This prevents string duplication and ensures consistent token propagation between: - -- BFF controller -- Cookie service -- Gateway filters - ---- - -### 4. PKCEUtils - -**Class:** `PKCEUtils` - -Utility class implementing PKCE for secure OAuth2 Authorization Code flows. - -#### Provided Methods - -- `generateState()` – 128-bit random value -- `generateCodeVerifier()` – 256-bit random value -- `generateCodeChallenge()` – SHA-256 based challenge -- `urlEncode()` – Safe redirect parameter encoding - -### PKCE Flow - -```mermaid -flowchart TD - Verifier["Code Verifier (random 32 bytes)"] --> Hash["SHA-256"] - Hash --> Challenge["Base64URL Encode"] - - Challenge -->|"Sent to Authorization Server"| AuthServer["Authorization Server"] - Verifier -->|"Stored by BFF"| BffStore["State JWT / Cookie"] -``` - -This ensures: - -- Protection against authorization code interception -- CSRF mitigation via state parameter -- Strong cryptographic randomness via `SecureRandom` - ---- - -## OAuth Backend-For-Frontend (BFF) - -### OAuthBffController - -**Class:** `OAuthBffController` - -This is the central HTTP interface for browser-based authentication. - -It is conditionally enabled via: - -- `openframe.gateway.oauth.enable=true` - -### Exposed Endpoints - -| Endpoint | Method | Purpose | -|-----------|--------|----------| -| `/oauth/login` | GET | Start OAuth flow | -| `/oauth/continue` | GET | Continue flow without clearing session | -| `/oauth/callback` | GET | Handle authorization code | -| `/oauth/refresh` | POST | Refresh tokens | -| `/oauth/logout` | GET | Revoke refresh token and clear cookies | -| `/oauth/dev-exchange` | GET | Exchange development ticket | - ---- - -## End-to-End Login Flow - -```mermaid -sequenceDiagram - participant Browser - participant BFF as OAuth BFF Controller - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/login - BFF->>Auth: Redirect with state and PKCE challenge - Auth->>Browser: Login UI - Auth->>BFF: Callback with code and state - BFF->>Auth: Exchange code for tokens - Auth->>BFF: Access and Refresh tokens - BFF->>Browser: Set HttpOnly cookies and redirect -``` - -### Key Mechanics - -1. State is signed as a JWT. -2. State cookie is stored with configurable TTL. -3. Tokens are returned as HTTP-only cookies. -4. Optional development ticket injection. -5. Errors redirect to configured error URL. - ---- - -## Token Refresh Flow - -```mermaid -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: POST /oauth/refresh - BFF->>Auth: Refresh token request - Auth->>BFF: New tokens - BFF->>Browser: Set new cookies -``` - -Features: - -- Supports refresh via cookie or header -- Supports tenant-based lookup -- Returns 401 if token missing or invalid - ---- - -## Logout Flow - -```mermaid -sequenceDiagram - participant Browser - participant BFF - participant Auth as Authorization Server - - Browser->>BFF: GET /oauth/logout - BFF->>Auth: Revoke refresh token - BFF->>Browser: Clear auth cookies -``` - -Logout ensures: - -- Refresh token revocation -- Cookie invalidation -- Stateless cleanup - ---- - -## Redirect Resolution - -### DefaultRedirectTargetResolver - -Provides fallback redirect resolution logic: - -1. Use explicit `redirectTo` parameter if provided -2. Fallback to `Referer` header -3. Default to `/` - -```mermaid -flowchart TD - Requested["Requested redirectTo"] -->|"Has value"| UseRequested["Use requested value"] - Requested -->|"Empty"| RefererCheck["Check Referer header"] - RefererCheck -->|"Present"| UseReferer["Use Referer"] - RefererCheck -->|"Missing"| DefaultRoot["Use /"] -``` - -This allows safe continuation after login without tight frontend coupling. - ---- - -## Security Design Principles - -Security Core And Oauth Bff follows these principles: - -- **Asymmetric cryptography** for JWT signing -- **Short-lived state tokens** for CSRF prevention -- **PKCE enforcement** for public clients -- **HttpOnly cookie storage** for tokens -- **Tenant-aware flows** -- **Reactive non-blocking controllers** using Project Reactor - ---- - -## Configuration Overview - -### JWT Properties - -```text -jwt.publicKey.value= -jwt.privateKey.value= -jwt.issuer=openframe -jwt.audience=openframe-api -``` - -### OAuth BFF Properties - -```text -openframe.gateway.oauth.enable=true -openframe.gateway.oauth.state-cookie-ttl-seconds=180 -openframe.gateway.oauth.dev-ticket-enabled=true -openframe.auth.error-url=/auth/error -``` - ---- - -## Extension Points - -Security Core And Oauth Bff is designed to be extended: - -- Replace `RedirectTargetResolver` -- Customize cookie behavior via `CookieService` -- Extend `OAuthBffService` -- Plug in alternative key loading mechanisms - ---- - -## Summary - -The **Security Core And Oauth Bff** module provides: - -- JWT cryptographic infrastructure -- OAuth2 BFF endpoints -- PKCE and CSRF protection -- Token refresh and revocation logic -- Redirect safety and cookie management - -It is a foundational module that secures the entire OpenFrame request lifecycle, enabling multi-tenant, OAuth2-compliant, and browser-safe authentication flows. \ No newline at end of file diff --git a/docs/reference/architecture/stream-processing-kafka/stream-processing-kafka.md b/docs/reference/architecture/stream-processing-kafka/stream-processing-kafka.md new file mode 100644 index 000000000..1eaf09fe5 --- /dev/null +++ b/docs/reference/architecture/stream-processing-kafka/stream-processing-kafka.md @@ -0,0 +1,353 @@ +# Stream Processing Kafka + +The **Stream Processing Kafka** module is the event ingestion and stream-processing backbone of OpenFrame. +It consumes Change Data Capture (CDC) events from integrated tools (MeshCentral, Tactical RMM, Fleet MDM), enriches and normalizes them, maps them to a unified event model, and forwards them to downstream systems such as Cassandra and Kafka topics. + +This module enables: + +- Multi-tenant event ingestion via Kafka +- Debezium-based CDC processing +- Tool-specific event deserialization +- Unified event type mapping +- Machine, organization, and tenant enrichment +- Optional Kafka Streams–based enrichment pipelines +- Cassandra persistence for long-term log storage + +--- + +## 1. Architectural Overview + +The Stream Processing Kafka module sits between integrated tools and the unified data layer. + +```mermaid +flowchart LR + ToolA["Integrated Tools\nMeshCentral / Tactical / Fleet"] --> Debezium["Debezium CDC"] + Debezium --> KafkaIn["Kafka Inbound Topics"] + KafkaIn --> Listener["JsonKafkaListener"] + Listener --> Processor["GenericJsonMessageProcessor"] + Processor --> Deserializer["Tool-Specific Deserializers"] + Deserializer --> Enrichment["IntegratedToolDataEnrichmentService"] + Enrichment --> Mapper["EventTypeMapper"] + Mapper --> Handler["DebeziumMessageHandler"] + Handler --> Cassandra["Cassandra (UnifiedLogEvent)"] + Handler --> KafkaOut["Outbound Kafka Topics"] +``` + +### Core Responsibilities + +| Layer | Responsibility | +|--------|---------------| +| Kafka Listener | Consumes multi-topic CDC events | +| Deserializers | Convert tool-specific payloads into normalized structures | +| Enrichment | Resolve tenant, machine, organization metadata | +| Mapping | Convert source event types to `UnifiedEventType` | +| Handlers | Persist to Cassandra or republish to Kafka | +| Kafka Streams | Perform real-time stream joins (Fleet activity enrichment) | + +--- + +## 2. Kafka Configuration Layer + +### 2.1 `KafkaConfig` + +Provides a `Converter` to convert Kafka headers into strongly typed `MessageType` enums. + +This allows the listener to route events correctly based on header metadata. + +### 2.2 `KafkaStreamsConfig` + +Enables and configures Kafka Streams processing. + +Key configuration aspects: + +- Application ID namespaced by cluster ID +- At-least-once processing guarantee +- Explicit key/value Serdes +- Controlled stream threads (1) +- State directory configuration +- Windowing tolerance via `MAX_TASK_IDLE_MS` + +```mermaid +flowchart TD + Config["KafkaStreamsConfig"] --> AppId["Application ID"] + Config --> Serdes["Custom JSON Serdes"] + Config --> ConsumerCfg["Consumer Config"] + Config --> ProducerCfg["Producer Config"] + Config --> StateDir["State Store Directory"] +``` + +--- + +## 3. Event Ingestion Pipeline + +### 3.1 JsonKafkaListener + +`JsonKafkaListener` consumes multiple inbound integrated-tool topics: + +- MeshCentral events +- Tactical RMM events +- Fleet MDM events +- Fleet query result events +- Fleet policy membership events + +It forwards: + +- `CommonDebeziumMessage` +- `MessageType` (from header) + +To: + +- `GenericJsonMessageProcessor` + +```mermaid +flowchart LR + Topics["Inbound Kafka Topics"] --> Listener["JsonKafkaListener"] + Listener --> Processor["GenericJsonMessageProcessor"] +``` + +Activation is conditional on cluster mode (`tenant`), ensuring proper multi-tenant isolation. + +--- + +## 4. Deserialization Layer + +All tool-specific deserializers extend a shared base (via `IntegratedToolEventDeserializer`). + +They extract: + +- Agent ID +- Tool event ID +- Source event type +- Message summary +- Timestamp +- Result / Error payloads + +### Supported Tool Deserializers + +| Tool | Deserializer | +|------|-------------| +| MeshCentral | `MeshCentralEventDeserializer` | +| Tactical RMM (audit) | `TrmmAuditEventDeserializer` | +| Tactical RMM (history) | `TrmmAgentHistoryEventDeserializer` | +| Tactical RMM (task result) | `TrmmTaskResultEventDeserializer` | +| Fleet MDM (activity) | `FleetEventDeserializer` | +| Fleet MDM (policy activity) | `FleetPolicyActivityDeserializer` | +| Fleet MDM (policy membership) | `FleetPolicyMembershipEventDeserializer` | +| Fleet MDM (query result) | `FleetQueryResultEventDeserializer` | + +Each deserializer: + +- Extracts structured fields from CDC payload +- Uses `TimestampParser` for ISO-8601 normalization +- Optionally consults cache services +- Produces a unified intermediate event model + +--- + +## 5. Unified Event Type Mapping + +### EventTypeMapper + +Maps: + +- `(IntegratedToolType, sourceEventType)` +To: +- `UnifiedEventType` + +If no mapping exists β†’ defaults to `UNKNOWN`. + +```mermaid +flowchart TD + Source["Tool Event"] --> ToolType["IntegratedToolType"] + Source --> SourceType["Source Event Type"] + ToolType --> Mapper["EventTypeMapper"] + SourceType --> Mapper + Mapper --> Unified["UnifiedEventType"] +``` + +This creates a single normalized event vocabulary across all tools. + +--- + +## 6. Data Enrichment Layer + +### 6.1 IntegratedToolDataEnrichmentService + +Enriches events with: + +- Machine ID +- Hostname +- Organization ID +- Organization Name +- Tenant ID + +Sources: + +- Redis machine cache +- Organization cache +- `TenantIdProvider` +- Optional `ClusterTenantIdResolver` + +```mermaid +flowchart LR + Event["DeserializedDebeziumMessage"] --> Enrichment + Enrichment --> MachineCache["MachineIdCacheService"] + Enrichment --> TenantResolver["Tenant Resolver"] + Enrichment --> Enriched["IntegratedToolEnrichedData"] +``` + +If machine or tenant cannot be resolved, warnings are logged but processing continues. + +--- + +## 7. Message Handling Layer + +### 7.1 GenericMessageHandler + +Provides: + +- Validation hook +- OperationType routing (CREATE / UPDATE / DELETE / READ) +- Lifecycle dispatch (`handleCreate`, `handleUpdate`, etc.) + +### 7.2 DebeziumMessageHandler + +Adds Debezium operation extraction logic (`c`, `u`, `d`, `r`). + +### 7.3 DebeziumCassandraMessageHandler + +Transforms enriched events into `UnifiedLogEvent` and persists them to Cassandra. + +```mermaid +flowchart TD + DebeziumMsg["DeserializedDebeziumMessage"] --> Transform + Transform --> Unified["UnifiedLogEvent"] + Unified --> Cassandra["Cassandra Repository"] +``` + +Key fields set: + +- Tenant ID +- Tool Type +- Unified Event Type +- Event Timestamp +- Severity +- Message +- Details + +### 7.4 TenantDebeziumKafkaMessageHandler + +Republishes enriched Debezium events to outbound Kafka topics using `OssTenantRetryingKafkaProducer`. + +Includes `TenantIdRequiredDebeziumEventValidator` which drops events without tenant context. + +--- + +## 8. Kafka Streams: Fleet Activity Enrichment + +`ActivityEnrichmentService` performs stream joins between: + +- `fleet-mdm-activities` +- `fleet-mdm-host-activities` + +It: + +1. Re-keys streams by activity ID +2. Performs a left join (5-second window) +3. Injects `hostId` and `agentId` +4. Adds appropriate `MessageType` headers +5. Publishes enriched events + +```mermaid +flowchart LR + Activity["Activity Topic"] --> Join + HostActivity["HostActivity Topic"] --> Join + Join --> Enriched["Enriched Activity"] + Enriched --> Output["Fleet MDM Events Topic"] +``` + +Window configuration: + +- Duration: 5 seconds +- No grace period +- At-least-once semantics + +--- + +## 9. Timestamp Handling + +`TimestampParser` standardizes timestamps using: + +- `Instant.parse()` +- ISO-8601 format +- Returns epoch milliseconds + +Ensures consistent temporal ordering across heterogeneous tools. + +--- + +## 10. Multi-Tenancy Model + +Tenant resolution follows two modes: + +| Mode | Tenant Resolution | +|------|-------------------| +| Tenant Cluster | `TenantIdProvider` (single tenant per cluster) | +| Shared Cluster | `ClusterTenantIdResolver` maps tool-scoped ID to canonical tenant ID | + +Events without tenant ID are rejected by `TenantIdRequiredDebeziumEventValidator`. + +--- + +## 11. End-to-End Event Flow + +```mermaid +flowchart TD + A["Integrated Tool"] --> B["Debezium CDC"] + B --> C["Kafka Topic"] + C --> D["JsonKafkaListener"] + D --> E["Deserializer"] + E --> F["Data Enrichment"] + F --> G["EventTypeMapper"] + G --> H["Message Handler"] + H --> I["Cassandra"] + H --> J["Outbound Kafka"] +``` + +--- + +## 12. Design Principles + +- **Tool-Agnostic Normalization** – Unified event model across MSP tools +- **Strong Multi-Tenancy Isolation** – Tenant-aware routing and validation +- **Extensibility** – Add new deserializer + mapping without modifying core flow +- **Observability** – Extensive logging on missing mappings or enrichment failures +- **Backpressure Safe** – Kafka-based buffering and retry logic +- **Separation of Concerns** – Listener β†’ Deserializer β†’ Enrichment β†’ Mapping β†’ Handler + +--- + +## 13. How It Fits into OpenFrame + +The Stream Processing Kafka module: + +- Feeds unified events into Cassandra for analytics and dashboards +- Supplies enriched events to downstream automation services +- Enables audit history for devices, users, scripts, and policies +- Bridges integrated tools into a single normalized event pipeline + +It is a foundational infrastructure module powering: + +- Device timeline views +- Automation history +- Compliance tracking +- Security monitoring +- Tenant-isolated event analytics + +--- + +# Summary + +The **Stream Processing Kafka** module transforms raw integrated-tool CDC events into normalized, enriched, tenant-aware unified events. + +Through Kafka, Debezium, enrichment services, mapping logic, and optional Kafka Streams processing, it provides a scalable, extensible, and multi-tenant event ingestion backbone for OpenFrame. \ No newline at end of file diff --git a/docs/reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md b/docs/reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md deleted file mode 100644 index cc6d0eb16..000000000 --- a/docs/reference/architecture/stream-service-core-kafka-and-handlers/stream-service-core-kafka-and-handlers.md +++ /dev/null @@ -1,356 +0,0 @@ -# Stream Service Core Kafka And Handlers - -## Overview - -The **Stream Service Core Kafka And Handlers** module is the event ingestion, normalization, enrichment, and distribution backbone of the OpenFrame platform. - -It is responsible for: - -- Consuming Debezium CDC events from integrated tools (MeshCentral, Tactical RMM, Fleet MDM) -- Deserializing tool-specific payloads into unified internal models -- Enriching events with tenant, organization, and device metadata -- Mapping tool-specific event types to unified event types -- Persisting events (e.g., Cassandra) -- Republishing normalized events to outbound Kafka topics -- Running Kafka Streams pipelines for activity enrichment - -This module acts as the **bridge between external tool ecosystems and the unified OpenFrame event model** used across API, analytics, notifications, and management services. - ---- - -## High-Level Architecture - -```mermaid -flowchart LR - subgraph Tools["Integrated Tools"] - MeshCentral["MeshCentral"] - Tactical["Tactical RMM"] - Fleet["Fleet MDM"] - end - - subgraph KafkaIn["Inbound Kafka Topics"] - InTopics["Debezium Topics"] - end - - subgraph StreamCore["Stream Service Core Kafka And Handlers"] - Listener["JsonKafkaListener"] - Processor["GenericJsonMessageProcessor"] - Deserializer["Tool Event Deserializers"] - Enrichment["IntegratedToolDataEnrichmentService"] - Mapper["EventTypeMapper"] - Handler["DebeziumMessageHandler"] - CassandraHandler["DebeziumCassandraMessageHandler"] - TenantKafkaHandler["TenantDebeziumKafkaMessageHandler"] - end - - subgraph Storage["Storage & Downstream"] - Cassandra["Cassandra UnifiedLogEvent"] - KafkaOut["Outbound Kafka Topics"] - end - - MeshCentral --> InTopics - Tactical --> InTopics - Fleet --> InTopics - - InTopics --> Listener - Listener --> Processor - Processor --> Deserializer - Deserializer --> Enrichment - Enrichment --> Mapper - Mapper --> Handler - - Handler --> CassandraHandler - Handler --> TenantKafkaHandler - - CassandraHandler --> Cassandra - TenantKafkaHandler --> KafkaOut -``` - ---- - -## Processing Flow - -The module follows a structured event pipeline: - -```mermaid -flowchart TD - A["Kafka Debezium Event"] --> B["JsonKafkaListener"] - B --> C["GenericJsonMessageProcessor"] - C --> D["Tool-Specific Deserializer"] - D --> E["Unified DeserializedDebeziumMessage"] - E --> F["IntegratedToolDataEnrichmentService"] - F --> G["EventTypeMapper"] - G --> H["DebeziumMessageHandler"] - H --> I["Destination Handler"] - I --> J["Cassandra or Kafka"] -``` - -### Key Stages - -1. **Kafka Consumption** – `JsonKafkaListener` consumes multi-topic integrated tool events. -2. **Deserialization** – Tool-specific deserializers convert raw CDC JSON into structured internal messages. -3. **Enrichment** – Tenant, machine, and organization metadata are resolved via Redis cache services. -4. **Type Mapping** – `EventTypeMapper` maps tool-specific source types into `UnifiedEventType`. -5. **Handling & Dispatch** – Generic and specialized handlers route events to storage or outbound topics. - ---- - -## Kafka Configuration Layer - -### KafkaConfig - -Provides: - -- `Converter` for resolving message type headers. -- Integration with Spring Kafka infrastructure. - -This enables dynamic routing of heterogeneous tool events through a unified processing layer. - -### KafkaStreamsConfig - -Enables Kafka Streams processing when `kafka.stream.enabled=true`. - -Key responsibilities: - -- Configures `application.id` (namespaced by cluster ID) -- Defines custom `Serde` for: - - `ActivityMessage` - - `HostActivityMessage` -- Configures state store, threading, and processing guarantees - -Processing guarantee: `AT_LEAST_ONCE` - ---- - -## Kafka Streams: Activity Enrichment - -`ActivityEnrichmentService` builds a Kafka Streams topology that: - -- Consumes: - - Fleet activity topic - - Fleet host activity topic -- Performs a time-windowed left join -- Enriches activity events with host information -- Adds required Kafka headers -- Publishes enriched events back to an inbound topic - -```mermaid -flowchart LR - ActivityTopic["Fleet Activities"] --> Join - HostActivityTopic["Fleet Host Activities"] --> Join - Join["Left Join (5s Window)"] --> Enriched["Enriched ActivityMessage"] - Enriched --> HeaderAdder["Add MESSAGE_TYPE_HEADER"] - HeaderAdder --> OutputTopic["Fleet MDM Events Topic"] -``` - -This allows Fleet policy and activity events to be normalized before entering the main Debezium event processing path. - ---- - -## Tool-Specific Deserializers - -Each integrated tool has a dedicated deserializer extending `IntegratedToolEventDeserializer`. - -### Fleet MDM - -- `FleetEventDeserializer` -- `FleetPolicyActivityDeserializer` -- `FleetPolicyMembershipEventDeserializer` -- `FleetQueryResultEventDeserializer` - -Responsibilities: - -- Extract agent ID -- Resolve policy/query metadata via cache services -- Build structured `result` or `error` JSON -- Normalize timestamps using `TimestampParser` -- Assign correct `MessageType` - -### Tactical RMM - -- `TrmmAgentHistoryEventDeserializer` -- `TrmmAuditEventDeserializer` -- `TrmmTaskResultEventDeserializer` - -Responsibilities: - -- Resolve primary key IDs to logical agent IDs via cache -- Parse script results and command outputs -- Convert execution results into structured JSON payloads - -### MeshCentral - -- `MeshCentralEventDeserializer` - -Responsibilities: - -- Parse nested JSON payloads -- Extract `etype.action` composite source type -- Extract tenant domain for shared cluster resolution - ---- - -## Unified Event Mapping - -`EventTypeMapper` maps: - -- `IntegratedToolType` -- Source event type string - -To: - -- `UnifiedEventType` - -```mermaid -flowchart LR - ToolType["IntegratedToolType"] --> Key - SourceType["Source Event Type"] --> Key - Key["tool:sourceType"] --> Mapper["EventTypeMapper"] - Mapper --> Unified["UnifiedEventType"] -``` - -If no mapping exists, the system falls back to `UNKNOWN`. - -This ensures: - -- Cross-tool normalization -- Consistent severity assignment -- Unified downstream analytics - ---- - -## Data Enrichment Layer - -`IntegratedToolDataEnrichmentService` enriches deserialized events with: - -- Machine ID -- Hostname -- Organization ID and name -- Tenant ID - -It integrates with: - -- Redis cache via `MachineIdCacheService` -- Tenant resolution via `TenantIdProvider` -- Optional `ClusterTenantIdResolver` (shared cluster mode) - -```mermaid -flowchart TD - Event["DeserializedDebeziumMessage"] --> MachineLookup["MachineIdCacheService"] - MachineLookup --> OrgLookup["Organization Cache"] - OrgLookup --> TenantResolution["TenantIdProvider or ClusterTenantIdResolver"] - TenantResolution --> Enriched["IntegratedToolEnrichedData"] -``` - -This ensures that all downstream consumers receive fully contextualized events. - ---- - -## Message Handling Framework - -### GenericMessageHandler - -Provides a template pattern: - -- Validate message -- Transform to destination model -- Resolve operation type (CREATE, UPDATE, DELETE, READ) -- Route to appropriate handler method - -### DebeziumMessageHandler - -Specialization for CDC events: - -- Extracts operation code from Debezium payload (`c`, `u`, `d`, `r`) -- Maps to `OperationType` - -### DebeziumCassandraMessageHandler - -Transforms events into `UnifiedLogEvent` and writes to Cassandra. - -Responsibilities: - -- Construct partition key -- Populate severity and unified type -- Persist to `UnifiedLogEventRepository` - -### TenantDebeziumKafkaMessageHandler - -Publishes validated events to tenant-scoped outbound Kafka topics using `OssTenantRetryingKafkaProducer`. - ---- - -## Validation Layer - -`TenantIdRequiredDebeziumEventValidator` ensures: - -- All events have a resolved tenant ID -- Events without tenant context are dropped - -This prevents cross-tenant contamination and guarantees multi-tenant isolation. - ---- - -## Timestamp Normalization - -`TimestampParser` standardizes ISO-8601 timestamps into epoch milliseconds. - -All integrated tools rely on Debezium ISO format, ensuring: - -- Consistent event ordering -- Reliable partitioning -- Accurate time-based queries - ---- - -## Multi-Tenant and Cluster Modes - -The module supports two deployment modes: - -1. **Tenant Cluster Mode** - - One tenant per Kafka cluster - - `TenantIdProvider` resolves tenant ID directly - -2. **Shared Cluster Mode** - - Multiple tenants share Kafka - - `ClusterTenantIdResolver` maps tool-specific domain identifiers to canonical tenant IDs - -This design ensures compatibility across SaaS and dedicated deployments. - ---- - -## Integration with Other Modules - -This module integrates tightly with: - -- Data Mongo domain model (for unified event storage references) -- Data Cassandra repositories (for log persistence) -- Data Kafka configuration and retry (for producer reliability) -- Data Redis cache (for machine and organization lookups) -- Management and API services (downstream consumers of normalized events) - -It serves as the **event normalization boundary** between external tools and the internal OpenFrame domain model. - ---- - -## Summary - -The **Stream Service Core Kafka And Handlers** module provides: - -- Reliable Kafka ingestion -- Tool-specific deserialization -- Unified event type normalization -- Tenant-aware enrichment -- Cassandra persistence -- Kafka republishing -- Kafka Streams activity joins - -It is a critical infrastructure layer enabling: - -- Cross-tool observability -- Unified audit trails -- Analytics and reporting -- Real-time notification pipelines -- Multi-tenant isolation guarantees - -Without this module, the OpenFrame platform would lack a consistent, normalized, and tenant-safe event backbone. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelDispatchResponse.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelDispatchResponse.md new file mode 100644 index 000000000..248abff13 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelDispatchResponse.md @@ -0,0 +1,16 @@ + +A response DTO representing the result of a cancellation request for a dispatched command execution. + +## Key Components + +- **`executionId`** β€” The unique identifier of the execution that was cancelled + +## Usage Example + +```java +CancelDispatchResponse response = CancelDispatchResponse.builder() + .executionId("exec-abc-123") + .build(); + +String id = response.getExecutionId(); +``` \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelExecutionInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelExecutionInput.md new file mode 100644 index 000000000..51f1939d2 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CancelExecutionInput.md @@ -0,0 +1,22 @@ + +Data Transfer Object (DTO) for carrying the required identifiers to cancel an ongoing execution on a target machine. + +## Key Components + +| Field | Type | Constraint | Description | +|-------|------|-----------|-------------| +| `machineId` | `String` | `@NotBlank` | Identifier of the target machine running the execution | +| `executionId` | `String` | `@NotBlank` | Identifier of the specific execution to cancel | + +## Usage Example + +```java +CancelExecutionInput input = new CancelExecutionInput(); +input.setMachineId("machine-abc123"); +input.setExecutionId("exec-xyz789"); + +// Pass to command handler or REST controller +commandService.cancelExecution(input); +``` + +> Both fields are required β€” a blank `machineId` or `executionId` will trigger a validation error before the cancellation request is processed. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CommandDispatchResponse.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CommandDispatchResponse.md new file mode 100644 index 000000000..c9ef597b3 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.CommandDispatchResponse.md @@ -0,0 +1,22 @@ + +A DTO representing the server's response after dispatching a command for execution. + +## Key Components + +- **`executionId`** (`String`) β€” Unique identifier for the dispatched command execution, used to track or query execution status. + +Annotations: +- `@Data` β€” Generates getters, setters, `equals`, `hashCode`, and `toString` via Lombok. +- `@Builder` β€” Provides a fluent builder pattern for object construction via Lombok. + +## Usage Example + +```java +// Building a response after dispatching a command +CommandDispatchResponse response = CommandDispatchResponse.builder() + .executionId("exec-8f3a2c1d-4b56-7890") + .build(); + +// Accessing the execution ID +String id = response.getExecutionId(); +``` \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.RunCommandInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.RunCommandInput.md new file mode 100644 index 000000000..692cf1e85 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/command/.RunCommandInput.md @@ -0,0 +1,25 @@ + +DTO representing the GraphQL input payload for dispatching an ad-hoc shell command to a remote agent. + +## Key Components + +| Field | Type | Constraint | Description | +|-------|------|------------|-------------| +| `machineId` | `String` | `@NotBlank` | Target machine identifier | +| `command` | `String` | `@NotBlank` | Shell command to execute | +| `shell` | `ScriptShell` | `@NotNull` | Shell interpreter (e.g. `POWERSHELL`, `CMD`, `BASH`) | +| `privilegeLevel` | `PrivilegeLevel` | `@NotNull` | Execution privilege context (e.g. `SYSTEM`, `USER`) | +| `timeoutSeconds` | `Integer` | `@Positive` | Optional execution timeout in seconds | + +## Usage Example + +```java +RunCommandInput input = new RunCommandInput(); +input.setMachineId("machine-abc123"); +input.setCommand("Get-Process | Select-Object Name, CPU"); +input.setShell(ScriptShell.POWERSHELL); +input.setPrivilegeLevel(PrivilegeLevel.SYSTEM); +input.setTimeoutSeconds(30); +``` + +> Bean validation (`@NotBlank`, `@NotNull`, `@Positive`) is enforced automatically at the GraphQL layer before the command is dispatched to the agent. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.CreateScriptInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.CreateScriptInput.md new file mode 100644 index 000000000..a21ea38ed --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.CreateScriptInput.md @@ -0,0 +1,35 @@ + +DTO for capturing client-supplied fields when creating a new script resource. Tenant attribution is intentionally excluded and resolved from the authenticated principal server-side. + +## Key Components + +| Field | Type | Constraint | Description | +|-------|------|------------|-------------| +| `name` | `String` | `@NotBlank` | Human-readable script name | +| `shell` | `ScriptShell` | `@NotNull` | Target shell/interpreter (e.g. PowerShell, Bash) | +| `scriptBody` | `String` | `@NotBlank` | Raw script content | +| `description` | `String` | β€” | Optional human-readable description | +| `tag` | `String` | β€” | Organizational tag | +| `supportedPlatforms` | `List` | β€” | Target OS/platform list | +| `defaultTimeoutSeconds` | `Integer` | `@Positive` | Execution timeout override | +| `defaultArgs` | `List` | β€” | Pre-filled argument defaults | +| `envVars` | `List` | `@Valid` | Environment variable definitions (cascading validation) | + +## Usage Example + +```java +CreateScriptInput input = new CreateScriptInput(); +input.setName("Disk Cleanup"); +input.setShell(ScriptShell.POWERSHELL); +input.setScriptBody("Remove-Item -Path $env:TEMP\\* -Recurse -Force"); +input.setDescription("Clears the Windows temp directory"); +input.setSupportedPlatforms(List.of(ScriptPlatform.WINDOWS)); +input.setDefaultTimeoutSeconds(120); + +ScriptEnvVarInput envVar = new ScriptEnvVarInput(); +envVar.setKey("LOG_LEVEL"); +envVar.setValue("INFO"); +input.setEnvVars(List.of(envVar)); +``` + +> **Note:** Do not supply a tenant ID in this payload. The API resolves tenant attribution automatically from the authenticated principal at the controller/resolver layer. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptEnvVarInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptEnvVarInput.md new file mode 100644 index 000000000..e5ca0ff29 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptEnvVarInput.md @@ -0,0 +1,30 @@ + +Symmetric DTO representing a script environment variable, used for both create/update input and response output payloads. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `name` | `String` | Environment variable name β€” required, validated with `@NotBlank` | +| `value` | `String` | Environment variable value β€” optional | +| `secret` | `boolean` | Marks the value as sensitive (passwords, API keys, tokens) | + +## Usage Example + +```java +// Build a plain environment variable +ScriptEnvVarInput envVar = ScriptEnvVarInput.builder() + .name("NODE_ENV") + .value("production") + .secret(false) + .build(); + +// Build a sensitive environment variable +ScriptEnvVarInput secretVar = ScriptEnvVarInput.builder() + .name("API_KEY") + .value("sk-abc123") + .secret(true) + .build(); +``` + +> **⚠️ Security Note:** Secret values are currently stored in **plaintext** pending a full secret-management implementation (encryption at rest + secure agent delivery). UI layers, logs, and audit trails are responsible for masking and redacting sensitive values in the interim. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptFilterInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptFilterInput.md new file mode 100644 index 000000000..3fb7841b5 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptFilterInput.md @@ -0,0 +1,32 @@ + +Data Transfer Object for filtering scripts in the OpenFrame RMM module, supporting multi-value filtering by shell type, status, platform compatibility, and tag. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `shells` | `List` | Filter scripts matching any of the specified shell types | +| `statuses` | `List` | Filter by script status; defaults to excluding `DELETED` when null/empty | +| `supportedPlatforms` | `List` | Filter scripts compatible with ANY of the specified platforms | +| `tag` | `String` | Case-insensitive exact match on a single tag | + +All fields are optional β€” omitting a field applies no constraint for that dimension. + +## Usage Example + +```java +// Filter for active PowerShell scripts targeting Windows +ScriptFilterInput filter = ScriptFilterInput.builder() + .shells(List.of(ScriptShell.POWERSHELL)) + .statuses(List.of(ScriptStatus.ACTIVE)) + .supportedPlatforms(List.of(ScriptPlatform.WINDOWS)) + .tag("patching") + .build(); + +// Filter for all Bash/ZSH scripts regardless of status +ScriptFilterInput shellOnly = ScriptFilterInput.builder() + .shells(List.of(ScriptShell.BASH, ScriptShell.ZSH)) + .build(); +``` + +> **Note:** When `statuses` is `null` or empty, the service layer automatically excludes `DELETED` scripts β€” pass `ScriptStatus.DELETED` explicitly only if soft-deleted scripts are intentionally required. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptResponse.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptResponse.md new file mode 100644 index 000000000..cc3069d0b --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.ScriptResponse.md @@ -0,0 +1,40 @@ + +Data Transfer Object (DTO) representing a script resource in API responses, exposing all script metadata while intentionally omitting the internal `tenantId` field. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `id` | `String` | Unique script identifier | +| `name` / `description` | `String` | Human-readable script metadata | +| `shell` | `String` | Shell enum name (e.g. `POWERSHELL`, `BASH`) | +| `scriptBody` | `String` | Raw script content | +| `supportedPlatforms` | `List` | Platform enum names the script targets | +| `defaultTimeoutSeconds` | `Integer` | Execution timeout default | +| `defaultArgs` | `List` | Pre-configured argument defaults | +| `envVars` | `List` | Environment variable definitions | +| `status` | `String` | Lifecycle enum name (e.g. `ACTIVE`, `DISABLED`) | +| `statusChangedAt` / `createdAt` / `updatedAt` | `Instant` | Audit timestamps | + +Uses Lombok `@Data` and `@Builder` for boilerplate-free construction and accessor generation. + +## Usage Example + +```java +ScriptResponse response = ScriptResponse.builder() + .id("script-abc123") + .name("Disk Cleanup") + .description("Removes temp files from Windows endpoints") + .shell("POWERSHELL") + .scriptBody("Remove-Item -Path $env:TEMP\\* -Recurse -Force") + .tag("maintenance") + .supportedPlatforms(List.of("WINDOWS")) + .defaultTimeoutSeconds(120) + .defaultArgs(List.of("-Verbose")) + .status("ACTIVE") + .createdAt(Instant.now()) + .updatedAt(Instant.now()) + .build(); +``` + +> **Security note:** `tenantId` is deliberately excluded β€” tenant context is resolved server-side from the authenticated session and never returned to the caller. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.UpdateScriptInput.md b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.UpdateScriptInput.md new file mode 100644 index 000000000..f301d4be0 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/dto/script/.UpdateScriptInput.md @@ -0,0 +1,39 @@ + +Data Transfer Object (DTO) representing the full-replacement (PUT) request payload for updating an existing script resource. + +## Key Components + +| Field | Type | Constraint | Description | +|---|---|---|---| +| `name` | `String` | `@NotBlank` | Script display name | +| `description` | `String` | optional | Human-readable description | +| `shell` | `ScriptShell` | `@NotNull` | Target shell/interpreter | +| `scriptBody` | `String` | `@NotBlank` | Full script source code | +| `tag` | `String` | optional | Categorization tag | +| `supportedPlatforms` | `List` | optional | Target OS/platform list | +| `defaultTimeoutSeconds` | `Integer` | `@Positive` | Execution timeout | +| `defaultArgs` | `List` | optional | Default runtime arguments | +| `envVars` | `List` | `@Valid` | Environment variable definitions | + +## Key Behavior + +- **PUT semantics** β€” every field overwrites the stored document; optional fields sent as `null` **clear** the stored value +- Partial updates are **not supported** β€” callers must send the complete resource on every request +- Required fields (`name`, `shell`, `scriptBody`) mirror `CreateScriptInput` constraints, preventing the document from reaching an invalid state + +## Usage Example + +```java +UpdateScriptInput input = new UpdateScriptInput(); +input.setName("Disk Cleanup"); +input.setShell(ScriptShell.POWERSHELL); +input.setScriptBody("Get-PSDrive -PSProvider FileSystem | ..."); +input.setDefaultTimeoutSeconds(120); +input.setSupportedPlatforms(List.of(ScriptPlatform.WINDOWS)); + +// Null optional fields clear stored values +input.setDescription(null); // clears existing description +input.setTag(null); // clears existing tag +``` + +> **Note:** Because this is a full-replacement operation, always populate all fields you wish to retain β€” omitting optional fields will erase previously stored data. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/mapper/.ScriptMapper.md b/openframe-api-lib/src/main/java/com/openframe/api/mapper/.ScriptMapper.md new file mode 100644 index 000000000..195f9306c --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/mapper/.ScriptMapper.md @@ -0,0 +1,42 @@ + +Handles bidirectional mapping between `Script` entities and their corresponding DTOs (`CreateScriptInput`, `UpdateScriptInput`, `ScriptResponse`) for the OpenFrame RMM script management domain. + +## Key Components + +| Method | Description | +|--------|-------------| +| `toEntity(tenantId, input)` | Maps a `CreateScriptInput` DTO to a new `Script` document, scoped to the given tenant | +| `updateEntity(existing, input)` | Mutates an existing `Script` entity in-place from an `UpdateScriptInput` DTO | +| `toResponse(entity)` | Converts a `Script` entity to a `ScriptResponse` DTO, normalizing enums to their string names | +| `mapEnvVarsToEntity(...)` | Private helper β€” maps `ScriptEnvVarInput` list to `ScriptEnvVar` document list | +| `mapEnvVarsToResponse(...)` | Private helper β€” maps `ScriptEnvVar` document list back to `ScriptEnvVarInput` list | +| `mapPlatformsToResponse(...)` | Private helper β€” converts `ScriptPlatform` enum list to `List` | + +> **Note:** Secret masking for env vars (`secret == true`) is pending implementation once secret-management support lands (see inline TODO). + +## Usage Example + +```java +@Service +public class ScriptService { + + private final ScriptMapper scriptMapper; + private final ScriptRepository scriptRepository; + + // Create a new script + public ScriptResponse createScript(String tenantId, CreateScriptInput input) { + Script entity = scriptMapper.toEntity(tenantId, input); + Script saved = scriptRepository.save(entity); + return scriptMapper.toResponse(saved); + } + + // Update an existing script + public ScriptResponse updateScript(String id, UpdateScriptInput input) { + Script existing = scriptRepository.findById(id).orElseThrow(); + scriptMapper.updateEntity(existing, input); + return scriptMapper.toResponse(scriptRepository.save(existing)); + } +} +``` + +> This mapper is transport-agnostic β€” GraphQL-specific concerns (cursor pagination, Relay Connection/Edge envelopes) are handled separately in `GraphQLScriptMapper`. \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md b/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md index 6df506d03..8b2023e54 100644 --- a/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md +++ b/openframe-api-lib/src/main/java/com/openframe/api/service/.KnowledgeBaseTempAttachmentService.md @@ -1,52 +1,57 @@ - -Manages temporary file uploads for Knowledge Base articles before they are permanently saved, mirroring the ticket-based `TempAttachmentService` pattern. + +Manages temporary file uploads for Knowledge Base articles before they are permanently saved, mirroring the ticket-based `TempAttachmentService` pattern with GCS-backed presigned URL support. ## Key Components | Method | Description | |--------|-------------| -| `createUploadUrl()` | Creates a `TempAttachment` record and generates a unique GCS storage path for a pending upload | -| `generateUploadUrl()` | Returns a presigned GCS PUT URL for direct frontend-to-storage upload | +| `createUploadUrl()` | Creates a `TempAttachment` record with a unique GCS temp path and returns it for presigned URL generation | +| `generateUploadUrl()` | Generates a time-limited GCS presigned PUT URL for direct frontend-to-GCS upload | | `deleteTempAttachment()` | Deletes a temp attachment from GCS and the database, enforcing uploader ownership | -| `linkTempAttachmentsToArticle()` | Moves temp files to permanent `kb-attachments/` paths and creates `KnowledgeBaseItemAttachment` records | -| `moveToArticle()` *(private)* | Handles per-file GCS move, temp record cleanup, and permanent attachment creation | - -**Configuration property:** `openframe.kb.presigned-url-expiration-minutes` (default: `15`) +| `linkTempAttachmentsToArticle()` | Promotes temp files to permanent `kb-attachments/` storage, creates `KnowledgeBaseItemAttachment` records, and cleans up temp entries | +| `moveToArticle()` *(private)* | Moves a single file from `temp/{id}/` to `kb-attachments/{articleId}/` via GCS, with per-file error isolation | ## Upload Flow ```mermaid sequenceDiagram - participant U as User - participant S as KBTempAttachmentService - participant GCS as GCS Storage - participant DB as Database - - U->>S: createUploadUrl(fileName, contentType) - S->>DB: Save TempAttachment (temp/{id}/file) - S->>U: Return TempAttachment + presigned PUT URL - U->>GCS: Upload file directly - U->>S: linkTempAttachmentsToArticle(articleId, tempIds) - S->>GCS: moveFile(temp path β†’ kb-attachments path) - S->>DB: Delete TempAttachment - S->>DB: Save KnowledgeBaseItemAttachment + participant Frontend + participant Service + participant GCS + participant DB + + Frontend->>Service: createUploadUrl(uploaderId, fileName, ...) + Service->>DB: Save TempAttachment + Service->>Frontend: TempAttachment + presigned PUT URL + Frontend->>GCS: PUT file directly + Frontend->>Service: linkTempAttachmentsToArticle(articleId, tempIds) + Service->>GCS: moveFile(temp/ β†’ kb-attachments/) + Service->>DB: Delete TempAttachment + Service->>DB: Save KnowledgeBaseItemAttachment ``` ## Usage Example ```java -// Step 1: Request an upload URL -TempAttachment temp = kbTempAttachmentService.createUploadUrl( - userId, "diagram.png", "image/png", 204800L -); -String presignedUrl = kbTempAttachmentService.generateUploadUrl(temp); -// Frontend uploads directly to presignedUrl via HTTP PUT +// Step 1: Create temp upload record and get presigned URL +TempAttachment temp = knowledgeBaseTempAttachmentService + .createUploadUrl("user-123", "diagram.png", "image/png", 204800L); + +String uploadUrl = knowledgeBaseTempAttachmentService + .generateUploadUrl(temp); +// Frontend uploads directly to uploadUrl via HTTP PUT // Step 2: On article save, promote temp files to permanent storage List attachments = - kbTempAttachmentService.linkTempAttachmentsToArticle( - articleId, + knowledgeBaseTempAttachmentService.linkTempAttachmentsToArticle( + "article-456", List.of(temp.getId()), - userId + "user-123" ); -``` \ No newline at end of file +``` + +## Configuration + +| Property | Default | Description | +|----------|---------|-------------| +| `openframe.kb.presigned-url-expiration-minutes` | `15` | Lifetime of the GCS presigned upload URL | \ No newline at end of file diff --git a/openframe-api-lib/src/main/java/com/openframe/api/service/rmm/.ScriptService.md b/openframe-api-lib/src/main/java/com/openframe/api/service/rmm/.ScriptService.md new file mode 100644 index 000000000..ff0fb5cd1 --- /dev/null +++ b/openframe-api-lib/src/main/java/com/openframe/api/service/rmm/.ScriptService.md @@ -0,0 +1,45 @@ + +Manages CRUD operations for RMM scripts within a tenant-scoped context, enforcing name uniqueness, soft-delete semantics, and cursor-based pagination. + +## Key Components + +| Member | Description | +|---|---| +| `create(CreateScriptInput)` | Creates a new script; throws `ConflictException` on duplicate name | +| `get(String id)` | Fetches a single visible (non-deleted) script by ID | +| `list(...)` | Returns a cursor-paginated, filterable, sortable list of scripts | +| `update(String id, UpdateScriptInput)` | Full replacement (PUT semantics); validates name uniqueness | +| `delete(String id)` | Soft-deletes by setting status to `DELETED`; idempotent | +| `loadVisibleOrThrow(...)` | Internal helper treating `DELETED` documents as not found | +| `loadOrThrow(...)` | Internal helper loading any document regardless of status | +| `toQueryFilter(ScriptFilterInput)` | Translates API-layer filter DTO to repository-layer `ScriptQueryFilter` | + +**Tenant scoping** is resolved via `TenantIdProvider` β€” JWT claims are deliberately not trusted for tenant scoping to support super-admin tokens safely. + +## Usage Example + +```java +// Create +ScriptResponse created = scriptService.create( + CreateScriptInput.builder().name("patch-windows").build() +); + +// Read (throws NotFoundException if deleted or missing) +ScriptResponse script = scriptService.get("script-id-123"); + +// Paginated list with filter and sort +GenericQueryResult page = scriptService.list( + ScriptFilterInput.builder().tag("security").build(), + "patch", + SortInput.builder().field("name").direction(SortDirection.ASC).build(), + CursorPaginationCriteria.builder().limit(20).build() +); + +// Update +ScriptResponse updated = scriptService.update("script-id-123", + UpdateScriptInput.builder().name("patch-windows-v2").build() +); + +// Soft-delete (idempotent) +scriptService.delete("script-id-123"); +``` \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.CommandDataFetcher.md b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.CommandDataFetcher.md new file mode 100644 index 000000000..9a16bd700 --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.CommandDataFetcher.md @@ -0,0 +1,50 @@ + +GraphQL mutation resolver for RMM (Remote Monitoring & Management) ad-hoc command dispatch operations. Exposes two mutations for running and cancelling commands on managed devices. + +## Key Components + +| Member | Type | Description | +|--------|------|-------------| +| `commandDispatchService` | `CommandDispatchService` | Injected service handling command dispatch logic | +| `runCommand` | `@DgsMutation` | Dispatches an ad-hoc command to a remote device | +| `cancelExecution` | `@DgsMutation` | Cancels a previously dispatched command execution | + +## Usage Example + +```java +// GraphQL mutation β€” run a command +mutation { + runCommand(input: { + deviceId: "device-123", + command: "ipconfig /all" + }) { + dispatchId + status + } +} + +// GraphQL mutation β€” cancel an execution +mutation { + cancelExecution(input: { + dispatchId: "dispatch-456" + }) { + success + message + } +} +``` + +```java +// Direct service delegation (internal) +RunCommandInput input = new RunCommandInput("device-123", "ipconfig /all"); +CommandDispatchResponse response = commandDispatchService.runCommand(input); + +CancelExecutionInput cancelInput = new CancelExecutionInput("dispatch-456"); +CancelDispatchResponse cancelResponse = commandDispatchService.cancelExecution(cancelInput); +``` + +## Notes + +- Inputs are validated via Jakarta Bean Validation (`@Valid`) and Spring's `@Validated` +- All business logic is delegated to `CommandDispatchService` β€” this class is a thin resolver layer +- Registered as a DGS component, automatically discovered by the Netflix DGS framework \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md index bbd5f16df..d7317df9b 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.NotificationDataFetcher.md @@ -1,53 +1,49 @@ - -GraphQL DGS data fetcher that handles all notification-related queries and mutations for authenticated ADMIN and AGENT users, including listing, read-state management, and deletion operations. + +GraphQL DGS data fetcher that exposes notification queries and mutations for authenticated `ADMIN` and `AGENT` actors, resolving the current recipient from the JWT principal and delegating to `NotificationService` and `NotificationReadStateService`. ## Key Components | Member | Type | Description | -|--------|------|-------------| -| `notificationNodeId` | `@DgsData` | Encodes the notification's raw ID into a Relay-compliant global ID | -| `notifications` | `@DgsQuery` | Returns a paginated, cursor-based connection of `NotificationView` items with optional filter and search | -| `hasUnreadNotifications` | `@DgsQuery` | Returns `true` if the current recipient has any unread notifications | -| `unreadCountsByCategory` | `@DgsQuery` | Returns per-category unread counts as a list of `UnreadCategoryCount` | +|---|---|---| +| `notifications` | `@DgsQuery` | Paginated, filterable list of notifications for the current user or machine | +| `hasUnreadNotifications` | `@DgsQuery` | Returns `true` if the recipient has any unread notifications | +| `unreadCountsByCategory` | `@DgsQuery` | Returns per-category unread counts as `List` | | `markNotificationAsRead` | `@DgsMutation` | Marks a single notification (by Relay global ID) as read | -| `markAllNotificationsAsRead` | `@DgsMutation` | Marks all notifications as read; returns count affected | +| `markAllNotificationsAsRead` | `@DgsMutation` | Marks every notification as read; returns the affected count | | `deleteNotification` | `@DgsMutation` | Deletes a single notification by Relay global ID | -| `deleteAllReadNotifications` | `@DgsMutation` | Deletes all read notifications; returns count deleted | -| `currentRecipient()` | Private helper | Resolves the caller to a `Recipient` record β€” `MACHINE` for AGENT actors, `USER` otherwise | -| `decodeNotificationId()` | Private helper | Decodes and validates a Relay global ID, asserting the `Notification` type | +| `deleteAllReadNotifications` | `@DgsMutation` | Deletes all read notifications; returns the affected count | +| `notificationNodeId` | `@DgsData` | Resolves the Relay-spec global `id` field on the `Notification` type | +| `currentRecipient()` | Private helper | Extracts `RecipientType.USER` or `RecipientType.MACHINE` from the JWT principal | +| `decodeNotificationId()` | Private helper | Decodes and validates a Relay global ID, asserting it references the `Notification` type | ## Usage Example ```java -# GraphQL β€” List notifications with pagination and filter +// GraphQL query β€” list paginated notifications with an unread filter query { - notifications(first: 10, filter: { read: false }) { + notifications(filter: { read: false }, first: 20, sort: { field: "createdAt", direction: DESC }) { edges { node { id title - category + createdAt } } - pageInfo { - hasNextPage - endCursor - } + pageInfo { hasNextPage endCursor } } + hasUnreadNotifications + unreadCountsByCategory { category count } } -# GraphQL β€” Mark a single notification as read +// GraphQL mutation β€” mark a single notification as read mutation { - markNotificationAsRead(notificationId: "Notification:abc123") + markNotificationAsRead(notificationId: "Tm90aWZpY2F0aW9uOjEyMw==") } -# GraphQL β€” Unread counts per category -query { - unreadCountsByCategory { - category - count - } +// GraphQL mutation β€” bulk delete all read notifications +mutation { + deleteAllReadNotifications } ``` -> All operations require a valid JWT and the `ADMIN` or `AGENT` authority. AGENT actors are resolved to their `machine_id` claim; all other authenticated users resolve to their `userId`. \ No newline at end of file +> All operations require a valid JWT with the `ADMIN` or `AGENT` authority. `AGENT` principals must carry a `machine_id` claim; otherwise an `UnauthorizedException` is thrown. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.ScriptDataFetcher.md b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.ScriptDataFetcher.md new file mode 100644 index 000000000..bb50361a6 --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/datafetcher/.ScriptDataFetcher.md @@ -0,0 +1,48 @@ + +GraphQL data fetcher (DGS component) for RMM script CRUD operations, acting as a thin passthrough layer between the GraphQL layer and `ScriptService`. + +## Key Components + +| Member | Type | Description | +|--------|------|-------------| +| `script` | `@DgsQuery` | Fetches a single script by ID | +| `scripts` | `@DgsQuery` | Returns a paginated, filterable, sortable connection of scripts | +| `createScript` | `@DgsMutation` | Creates a new RMM script | +| `updateScript` | `@DgsMutation` | Updates an existing script by ID | +| `deleteScript` | `@DgsMutation` | Deletes a script by ID, returns `true` on success | +| `ScriptService` | Dependency | Handles business logic and tenant scoping via `TenantIdProvider` | +| `GraphQLScriptMapper` | Dependency | Maps connection args to pagination criteria and results to GraphQL connections | + +## Usage Example + +```java +// Query β€” fetch a single script +ScriptResponse s = scriptDataFetcher.script("script-uuid-123"); + +// Query β€” paginated list with filter and sort +GenericConnection> page = scriptDataFetcher.scripts( + ScriptFilterInput.builder().category("maintenance").build(), + "disk cleanup", // free-text search + SortInput.builder().field("name").direction(SortDirection.ASC).build(), + 10, // first + null, // after cursor + null, // last + null // before cursor +); + +// Mutation β€” create +ScriptResponse created = scriptDataFetcher.createScript( + CreateScriptInput.builder().name("Disk Cleanup").content("...").build() +); + +// Mutation β€” update +ScriptResponse updated = scriptDataFetcher.updateScript( + "script-uuid-123", + UpdateScriptInput.builder().name("Disk Cleanup v2").build() +); + +// Mutation β€” delete +boolean deleted = scriptDataFetcher.deleteScript("script-uuid-123"); // β†’ true +``` + +> **Note:** Tenant scoping is resolved inside `ScriptService`. Role-based authorization is not yet enforced at this layer and is planned for a future security pass once the RMM role model is finalized. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/mapper/.GraphQLScriptMapper.md b/openframe-api-service-core/src/main/java/com/openframe/api/mapper/.GraphQLScriptMapper.md new file mode 100644 index 000000000..2eae1d212 --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/mapper/.GraphQLScriptMapper.md @@ -0,0 +1,33 @@ + +Assembles Relay-compliant GraphQL Connection envelopes for script queries and converts `ConnectionArgs` into `CursorPaginationCriteria` for the GraphQL layer. + +## Key Components + +| Method | Description | +|--------|-------------| +| `toCursorPaginationCriteria(ConnectionArgs)` | Converts Relay-style connection arguments into cursor pagination criteria via `CursorPaginationCriteria.fromConnectionArgs()` | +| `toConnection(GenericQueryResult)` | Builds a `GenericConnection` envelope by mapping each `ScriptResponse` item to a `GenericEdge` with a Base64-encoded cursor derived from the script's ID | + +## Design Notes + +- **Separation of concerns:** Pure entity ↔ DTO mapping is delegated to `ScriptMapper` in `openframe-api-lib`, keeping non-GraphQL callers free from DGS/Relay dependencies. +- **Mirrors the pattern** established by `GraphQLDeviceMapper` and `GraphQLNotificationMapper`. + +## Usage Example + +```java +@Component +public class ScriptDataFetcher { + + private final GraphQLScriptMapper graphQLScriptMapper; + + public GenericConnection> getScripts(ConnectionArgs args) { + CursorPaginationCriteria criteria = + graphQLScriptMapper.toCursorPaginationCriteria(args); + + GenericQueryResult result = scriptService.findAll(criteria); + + return graphQLScriptMapper.toConnection(result); + } +} +``` \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md index fecda7810..0e347584b 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolAgentUpdateService.md @@ -1,38 +1,37 @@ - -Orchestrates forced tool agent update operations across managed machines by validating requests, fetching agent configurations, and publishing update events via NATS messaging. + +Handles forced update dispatch for integrated tool agents across one or more managed machines, publishing update messages via NATS and aggregating per-machine results. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `process()` | Public Method | Handles update requests for a specific list of machine IDs | -| `processAll()` | Public Method | Triggers updates for all registered machines in the repository | -| `processMachine()` | Private Method | Fetches the tool agent config and publishes the update event for a single machine | -| `buildResponseItem()` | Private Method | Constructs a `ForceToolAgentUpdateResponseItem` with machine ID, agent ID, and status | -| `validateToolAgentId()` | Private Method | Guards against blank tool agent IDs | -| `validateMachineIds()` | Private Method | Guards against empty machine ID lists | -| `toolAgentUpdateUpdatePublisher` | Dependency | NATS publisher that dispatches the update message to the tool agent | +| `process(ForceToolAgentUpdateRequest)` | `public` | Dispatches a forced update for a tool agent to a specific list of machine IDs | +| `processAll(String toolAgentId)` | `public` | Dispatches a forced update for a tool agent to **all** registered machines | +| `processMachines(List, String)` | `private` | Iterates machine IDs and delegates to `processMachine` | +| `processMachine(String, String)` | `private` | Looks up the tool agent config, publishes the update via NATS, and returns a `PROCESSED` or `FAILED` status item | +| `buildResponseItem(...)` | `private` | Constructs a `ForceToolAgentUpdateResponseItem` with machine ID, tool agent ID, and status | +| `validateToolAgentId` / `validateMachineIds` | `private` | Guards against blank/empty inputs, throwing `IllegalArgumentException` | ## Usage Example ```java -// Force update a specific set of machines +// Targeted update β€” specific machines ForceToolAgentUpdateRequest request = new ForceToolAgentUpdateRequest(); -request.setToolAgentId("agent-123"); +request.setToolAgentId("zabbix-agent"); request.setMachineIds(List.of("machine-001", "machine-002")); ForceToolAgentUpdateResponse response = forceToolAgentUpdateService.process(request); response.getItems().forEach(item -> - log.info("Machine: {}, Status: {}", item.getMachineId(), item.getStatus()) + log.info("Machine {} β†’ {}", item.getMachineId(), item.getStatus()) ); -// Force update ALL registered machines for a given agent -ForceToolAgentUpdateResponse allResponse = forceToolAgentUpdateService.processAll("agent-123"); +// Broadcast update β€” all registered machines +ForceToolAgentUpdateResponse broadcastResponse = + forceToolAgentUpdateService.processAll("zabbix-agent"); ``` -## Status Outcomes +## Behavior Notes -Each processed machine returns one of two statuses from `ForceAgentStatus`: - -- **`PROCESSED`** β€” Update event successfully published via NATS -- **`FAILED`** β€” Exception occurred (agent not found or publish error); logged and gracefully captured per machine \ No newline at end of file +- Each machine is processed independently β€” a failure on one machine (`FAILED`) does not halt processing of others +- The update is published via `ToolAgentUpdateUpdatePublisher` (NATS), using the resolved `IntegratedToolAgent` configuration +- If the tool agent ID is not found in the repository, the machine item is marked `FAILED` and the error is logged \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md index 043ad134b..eb0100c62 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ForceToolInstallationService.md @@ -1,35 +1,38 @@ - -A Spring service that orchestrates forced tool agent installations and reinstallations across machines, providing batch processing capabilities for both specific machine sets and all available machines. + +Orchestrates forced tool agent installation and reinstallation across managed machines, validating inputs and delegating to underlying installation services per machine. ## Key Components -- **`process(ForceToolInstallationRequest)`** - Installs a tool agent on specified machines -- **`processAll(String toolAgentId)`** - Installs a tool agent on all available machines -- **`processReinstall(ForceToolInstallationRequest)`** - Forces reinstallation on specified machines -- **`processReinstallAll(String toolAgentId)`** - Forces reinstallation on all machines -- **`processMachines(List, String, boolean)`** - Internal batch processor for machine operations -- **`processMachine(String, String, boolean)`** - Handles individual machine installations with error handling +| Method | Description | +|--------|-------------| +| `process(request)` | Triggers installation for a specific list of machine IDs | +| `processAll(toolAgentId)` | Triggers installation across all machines in the repository | +| `processAll(toolAgentId, reinstall)` | Same as above with optional reinstall flag | +| `processReinstall(request)` | Forces reinstallation for a specific list of machine IDs | +| `processReinstallAll(toolAgentId)` | Forces reinstallation across all machines | +| `processMachine(...)` | Per-machine handler; resolves agent config and delegates to `ToolInstallationService` | +| `buildResponseItem(...)` | Constructs a response item with `PROCESSED` or `FAILED` status | ## Usage Example ```java -@Autowired -private ForceToolInstallationService forceInstallationService; - -// Install tool on specific machines +// Force install a tool agent on specific machines ForceToolInstallationRequest request = new ForceToolInstallationRequest(); -request.setToolAgentId("my-tool-agent"); -request.setMachineIds(Arrays.asList("machine-1", "machine-2")); - -ForceToolAgentInstallationResponse response = forceInstallationService.process(request); +request.setToolAgentId("zabbix-agent"); +request.setMachineIds(List.of("machine-001", "machine-002")); -// Install on all machines -ForceToolAgentInstallationResponse allResponse = - forceInstallationService.processAll("my-tool-agent"); +ForceToolAgentInstallationResponse response = forceToolInstallationService.process(request); +response.getItems().forEach(item -> + log.info("Machine {} β†’ {}", item.getMachineId(), item.getStatus()) +); -// Force reinstall on specific machines -ForceToolAgentInstallationResponse reinstallResponse = - forceInstallationService.processReinstall(request); +// Force reinstall across all machines +ForceToolAgentInstallationResponse allResponse = + forceToolInstallationService.processReinstallAll("zabbix-agent"); ``` -The service validates inputs, retrieves tool agent configurations, and delegates to `ToolInstallationService` while providing comprehensive error handling and status reporting through response items. \ No newline at end of file +## Notes + +- Each machine is processed independently; failures are caught and returned as `ForceAgentStatus.FAILED` without aborting the batch. +- Throws `IllegalArgumentException` if `toolAgentId` is blank or `machineIds` is empty. +- Throws `IllegalStateException` if no agent configuration is found for the given `toolAgentId`. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md index c48e59efe..324a20f85 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.NotificationService.md @@ -1,44 +1,40 @@ - -Provides paginated listing of notifications for a recipient, with support for read/unread filtering, search normalization, and cursor-based pagination. + +Handles paginated retrieval of notifications for a given recipient, supporting cursor-based pagination, read/unread filtering, text search, and configurable sort direction. ## Key Components -| Member | Type | Description | -|--------|------|-------------| -| `SEARCH_MIN_LENGTH` | Constant | Minimum character threshold (2) before a search term is applied | -| `list()` | Method | Fetches a cursor-paginated page of `NotificationView` items for a given recipient | -| `normalizeSearch()` | Private Method | Trims and nullifies search strings shorter than `SEARCH_MIN_LENGTH` | -| `buildPageInfo()` | Private Method | Constructs `PageInfo` with cursor positions and forward/backward pagination flags | +| Member | Description | +|--------|-------------| +| `list(...)` | Overloaded public method that fetches a paginated `GenericQueryResult` for a recipient, with optional `SortInput` | +| `normalizeSearch(String)` | Trims and nullifies search strings shorter than `SEARCH_MIN_LENGTH` (2 chars) | +| `resolveSortDirection(SortInput)` | Maps a `SortInput` to `Sort.Direction`; logs a warning and falls back to `DESC` if an invalid field is requested | +| `buildPageInfo(...)` | Constructs cursor-based `PageInfo` (start/end cursors, `hasNextPage`, `hasPreviousPage`) from the result slice | +| `ALLOWED_SORT_FIELDS` | Whitelist of valid sort fields: `"id"` and `"createdAt"` | ## Usage Example ```java -// Fetch first page of unread notifications for a user -NotificationFilter filter = new NotificationFilter(/* read= */ false, /* search= */ null); -CursorPaginationCriteria pagination = CursorPaginationCriteria.builder() - .limit(20) - .build(); +// Inject the service +@Autowired +private NotificationService notificationService; +// Build filter and pagination +NotificationFilter filter = new NotificationFilter(/* read= */ false, /* search= */ "alert"); +CursorPaginationCriteria pagination = CursorPaginationCriteria.of(20, null, false); +SortInput sort = new SortInput("createdAt", SortDirection.DESC); + +// Fetch notifications for a user recipient GenericQueryResult result = notificationService.list( - "user-123", - RecipientType.USER, - filter, - pagination + "user-123", + RecipientType.USER, + filter, + pagination, + sort ); -// Access results List notifications = result.getItems(); PageInfo pageInfo = result.getPageInfo(); - -// Fetch next page using end cursor -CursorPaginationCriteria nextPage = CursorPaginationCriteria.builder() - .limit(20) - .cursor(pageInfo.getEndCursor()) - .build(); +// pageInfo.isHasNextPage(), pageInfo.getEndCursor(), etc. ``` -## Pagination Behavior - -- Uses a **limit + 1** fetch strategy to determine if a next/previous page exists without a separate count query. -- Result order is **reversed** automatically when paginating backward. -- Cursors are encoded via `CursorCodec` using the notification's ID as the key. \ No newline at end of file +> **Note:** When paginating backwards, the result list is automatically reversed before mapping, ensuring consistent chronological ordering regardless of traversal direction. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md index bbf54cb82..dd43a1dec 100644 --- a/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/.ReleaseVersionQueryService.md @@ -1,13 +1,13 @@ - -Provides read-only access to the current release version of the OpenFrame platform by querying the underlying data store. + +Provides read-only access to the current release version by querying the underlying repository. ## Key Components | Element | Description | |---|---| -| `ReleaseVersionQueryService` | Spring `@Service` bean for querying release version data | -| `getReleaseVersion()` | Fetches the singleton `ReleaseVersion` document using `ReleaseVersion.DEFAULT_ID` | -| `ReleaseVersionRepository` | Injected repository (via `@RequiredArgsConstructor`) used to perform the lookup | +| `ReleaseVersionQueryService` | Spring service bean for retrieving release version metadata | +| `getReleaseVersion()` | Returns the first available `ReleaseVersion` document wrapped in an `Optional` | +| `ReleaseVersionRepository` | Injected repository dependency used to query the version data store | ## Usage Example @@ -27,4 +27,4 @@ public class VersionController { } ``` -> **Note:** `getReleaseVersion()` returns an `Optional`, so callers should handle the case where no version document has been persisted yet (e.g., on a fresh deployment). \ No newline at end of file +> **Note:** `getReleaseVersion()` uses `findFirstBy()` under the hood, meaning it assumes a single version document exists in the data store at any given time. If no document is present, an empty `Optional` is returned rather than throwing an exception. \ No newline at end of file diff --git a/openframe-api-service-core/src/main/java/com/openframe/api/service/rmm/.CommandDispatchService.md b/openframe-api-service-core/src/main/java/com/openframe/api/service/rmm/.CommandDispatchService.md new file mode 100644 index 000000000..24532b1b9 --- /dev/null +++ b/openframe-api-service-core/src/main/java/com/openframe/api/service/rmm/.CommandDispatchService.md @@ -0,0 +1,38 @@ + +Handles dispatching ad-hoc shell commands and cancellation requests from the dashboard to remote agents via core NATS (fire-and-forget), with no backend persistence. + +## Key Components + +| Method | Description | +|--------|-------------| +| `runCommand(RunCommandInput)` | Generates a unique `executionId`, builds a `CommandMessage`, and publishes it to the target agent via NATS | +| `cancelExecution(CancelExecutionInput)` | Builds a `CancelMessage` and publishes a cancel request for an in-flight execution | + +**Dependencies:** +- `CommandNatsPublisher` β€” handles the underlying NATS publish for both command and cancel messages + +## Usage Example + +```java +// Dispatch a shell command to a remote machine +RunCommandInput input = RunCommandInput.builder() + .machineId("agent-abc123") + .command("Get-Process") + .shell("powershell") + .privilegeLevel("admin") + .timeoutSeconds(30) + .build(); + +CommandDispatchResponse response = commandDispatchService.runCommand(input); +String executionId = response.getExecutionId(); // track async result separately + +// Cancel an in-flight execution +CancelExecutionInput cancelInput = CancelExecutionInput.builder() + .machineId("agent-abc123") + .executionId(executionId) + .build(); + +CancelDispatchResponse cancelResponse = commandDispatchService.cancelExecution(cancelInput); +``` + +> **Note:** This service is pure transport β€” it assigns an `executionId` for correlation but does not store results. Responses from the agent arrive on a separate NATS subject and are handled by the execution service. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md index 302f985bb..6acededed 100644 --- a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md +++ b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.NotificationDataFetcherIT.md @@ -1,46 +1,37 @@ - -Integration test suite for the `NotificationDataFetcher` GraphQL layer, verifying end-to-end notification query and mutation behavior against a real MongoDB instance using DGS and Spring Boot test infrastructure. + +Integration test suite that verifies the `NotificationDataFetcher` GraphQL layer end-to-end against a real MongoDB instance, covering queries, mutations, and JWT-based principal resolution for both admin users and machine agents. ## Key Components -| Component | Description | -|-----------|-------------| -| `NotificationDataFetcherIT` | Main test class extending `BaseMongoIntegrationTest`; runs only when `-Dintegration.tests=true` is set | -| `DgsQueryExecutor` | Executes raw GraphQL query/mutation strings against the full DGS schema | -| `NotificationReadStateService` | Service under test; used to seed `NotificationReadState` documents before assertions | -| `MongoTemplate` | Used in `@BeforeEach` to drop and reset `Notification` and `NotificationReadState` collections | -| `loginAsAdmin` / `loginAsAgent` | Helpers that inject a synthetic `JwtAuthenticationToken` into the `SecurityContextHolder` for role-based test isolation | - -## Test Coverage - -| Test | Scenario | -|------|----------| -| `admin_lists_own_rows` | ADMIN JWT resolves USER-addressed notification rows via `notifications` query | -| `agent_lists_machine_rows` | AGENT JWT with `machine_id` claim resolves MACHINE-addressed rows | -| `has_unread` | `hasUnreadNotifications` flips from `true` β†’ `false` after `markRead` | -| `mark_as_read` | `markNotificationAsRead` mutation accepts a Relay global ID and returns `true` | -| `mark_all_as_read` | `markAllNotificationsAsRead` returns count of UNREAD rows updated | -| `delete_notification` | `deleteNotification` mutation soft-deletes a single row by Relay global ID | -| `delete_all_read` | `deleteAllReadNotifications` transitions only READ rows to DELETED | -| `unread_counts_by_category` | `unreadCountsByCategory` returns one entry per category in UNREAD rows | +- **`admin_lists_own_rows`** β€” Confirms an ADMIN JWT principal retrieves only their own `USER`-addressed read-state rows via the `notifications` query. +- **`agent_lists_machine_rows`** β€” Confirms an AGENT JWT principal with a `machine_id` claim retrieves `MACHINE`-addressed rows without requiring an explicit schema argument. +- **`has_unread`** β€” Verifies `hasUnreadNotifications` returns `true` for unread rows and flips to `false` after `markRead` is called. +- **`mark_as_read`** β€” Exercises the `markNotificationAsRead` mutation using a Relay global ID and asserts the mutation returns `true`. +- **`mark_all_as_read`** β€” Exercises `markAllNotificationsAsRead` and asserts the returned count equals the number of unread rows. +- **`delete_notification`** β€” Exercises `deleteNotification` with a Relay global ID and asserts soft-deletion (returns `true`). +- **`delete_all_read`** β€” Exercises `deleteAllReadNotifications` and asserts only READ rows are transitioned to DELETED. +- **`admin_lists_oldest_first_with_sort_asc`** β€” Verifies `sort: { direction: ASC }` returns edges oldest-first. +- **`admin_lists_newest_first_by_default`** β€” Verifies default sort order is newest-first (DESC) when no `sort` argument is provided. ## Usage Example ```java -// Run integration tests (requires a running MongoDB; enable via system property) -./mvnw verify -Dintegration.tests=true +// Run integration tests with the required system property +// mvn verify -Dintegration.tests=true + +// Typical test pattern used throughout the suite +loginAsAdmin(ALICE); +Notification n = mongoTemplate.save(NotificationFixtures.basic("welcome")); +readStateService.createForAudience( + n.getId(), NotificationCategory.TICKETS, "title", + RecipientType.USER, Set.of(ALICE) +); -// Example query exercised in tests -queryExecutor.execute(""" +ExecutionResult res = queryExecutor.execute(""" query { notifications(first: 10) { edges { node { id title read } } } } """); - -// Example mutation with Relay global ID -String globalId = new Relay().toGlobalId("Notification", n.getId()); -queryExecutor.execute( - "mutation($id: ID!) { markNotificationAsRead(notificationId: $id) }", - Map.of("id", globalId) -); +assertThat(res.getErrors()).isEmpty(); +assertThat(edges(res)).hasSize(1); ``` -> **Note:** Tests are gated by `@EnabledIfSystemProperty(named = "integration.tests", matches = "true")` and tagged `@Tag("integration")`. Cloud Stream auto-configurations are excluded to prevent messaging infrastructure from being required at test startup. \ No newline at end of file +> **Note:** Tests are gated behind `-Dintegration.tests=true` and require a running MongoDB instance provided by `BaseMongoIntegrationTest`. Spring Cloud Stream auto-configurations are excluded to keep the test context lightweight. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md index 9cc6371c9..e670a1ef3 100644 --- a/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md +++ b/openframe-api-service-core/src/test/java/com/openframe/api/integration/datafetcher/notification/.TestApprovalContext.md @@ -1,24 +1,33 @@ - -A data model class representing the context payload for test approval notifications, extending the base `NotificationContext` with ticket and approval request identifiers. + +A notification context payload for test approval workflow events, extending `NotificationContext` to carry ticket and approval request identifiers. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `ticketId` | `String` | Identifier of the associated support ticket | -| `approvalRequestId` | `String` | Identifier of the approval request being tested | - -Inherits all fields and behavior from `NotificationContext` via Lombok's `@SuperBuilder`, `@EqualsAndHashCode(callSuper = true)`, and `@ToString(callSuper = true)`. +| `TYPE` | `String` constant | Discriminator value sourced from `TestApprovalContextDescriptor.TYPE` | +| `ticketId` | `String` | Reference to the associated support ticket | +| `approvalRequestId` | `String` | Reference to the specific approval request | +| `getType()` | `@Override` method | Returns the fixed `TYPE` discriminator for polymorphic serialization | ## Usage Example ```java -// Build a TestApprovalContext using the generated builder -TestApprovalContext context = TestApprovalContext.builder() - .ticketId("TKT-1234") +TestApprovalContext ctx = TestApprovalContext.builder() + .ticketId("TICK-1234") .approvalRequestId("APR-5678") .build(); -// Pass context to a notification dispatcher -notificationService.sendTestApproval(context); -``` \ No newline at end of file +// Type discriminator is readable but excluded from deserialization input +String type = ctx.getType(); // β†’ TestApprovalContextDescriptor.TYPE + +// Typically attached to a Notification before dispatch +Notification notification = Notification.builder() + .context(ctx) + .build(); +``` + +## Notes + +- `@JsonIgnoreProperties(value = "type", allowGetters = true)` ensures the `type` field is **serialized** (readable) but **ignored on deserialization**, preventing conflicts during polymorphic JSON mapping. +- Inherits base fields and behavior from `NotificationContext` via `@SuperBuilder` and `callSuper = true` on `@EqualsAndHashCode` / `@ToString`. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md b/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md index 06ca94d6d..92883be28 100644 --- a/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md +++ b/openframe-api-service-core/src/test/java/com/openframe/api/integration/service/.NotificationServiceIT.md @@ -1,50 +1,47 @@ - -Integration test suite for `NotificationService.list()`, validating pagination behavior, read-flag merging, audience isolation, search normalization, and soft-delete exclusion against a real MongoDB instance. + +Integration test suite for `NotificationService.list()`, verifying pagination, read-flag merging, audience isolation, search normalization, soft-delete exclusion, and sort behavior against a real MongoDB instance. ## Key Components | Component | Role | -|---|---| -| `NotificationService` | Service under test β€” handles `list()` with filters and cursor pagination | -| `NotificationReadStateService` | Creates and mutates read-state records used by `list()` | -| `BaseMongoIntegrationTest` | Base class providing embedded/real MongoDB wiring | -| `NotificationFixtures` | Helper that constructs `Notification` documents for seeding | -| `NotificationFilter` | Filter DTO accepting `read` boolean and search string | -| `CursorPaginationCriteria` | Pagination input (limit-based cursor) | - -## Test Coverage - -| Test | Scenario | -|---|---| -| `list_user_with_read_flag` | Read state is merged into `NotificationView.read` | -| `list_machine_recipient` | `RecipientType.MACHINE` audience visibility works | -| `audience_isolation_by_id` | Notifications scoped to other users are hidden | -| `audience_isolation_by_type` | Same ID, different `RecipientType` β†’ isolated pages | -| `read_filter_narrows_page` | `read=true` / `read=false` filters return correct rows | -| `short_search_normalized_to_no_search` | 1-char search terms are ignored | -| `whitespace_search_normalized_to_no_search` | Whitespace-only search is treated as no-op | -| `search_no_match_returns_empty_page` | Non-matching search returns empty page with `hasNextPage=false` | -| `deleted_excluded` | Soft-deleted read states are excluded from listing | +|-----------|------| +| `NotificationServiceIT` | Main test class extending `BaseMongoIntegrationTest` | +| `NotificationService` | Service under test β€” `list()` method with filtering, pagination, and sort | +| `NotificationReadStateService` | Supporting service for seeding read-state rows and marking notifications | +| `NotificationFixtures` | Test data factory for building `Notification` documents | +| `NotificationFilter` | Filter DTO controlling `read` flag and search term | +| `CursorPaginationCriteria` | Cursor-based pagination input (limit, cursor) | +| `SortInput` / `SortDirection` | Sort configuration passed to the 5-arg `list()` overload | ## Usage Example ```java -// Seed a notification and mark it read for a user +// Seed a notification and a read-state row, then assert the view reflects the read flag Notification n = mongoTemplate.save(NotificationFixtures.basic("welcome")); -readStateService.createForAudience( - n.getId(), NotificationCategory.TICKETS, "title", RecipientType.USER, Set.of("user-alice") -); -readStateService.markRead("user-alice", RecipientType.USER, n.getId()); - -// List with no filter β€” returns the notification with read=true -GenericQueryResult result = notificationService.list( - "user-alice", - RecipientType.USER, - NotificationFilter.EMPTY, - CursorPaginationCriteria.builder().limit(10).build() -); +readStateService.createForAudience(n.getId(), NotificationCategory.TICKETS, "title", U, Set.of(ALICE)); +readStateService.markRead(ALICE, U, n.getId()); +GenericQueryResult result = + notificationService.list(ALICE, U, NotificationFilter.EMPTY, page10()); + +assertThat(result.getItems()).hasSize(1); assertThat(result.getItems().get(0).read()).isTrue(); ``` -> **Note:** Tests are gated by the system property `integration.tests=true` and require a live MongoDB connection. Operations on `markRead`, `markAllAsRead`, `delete`, `hasUnread`, and counts are covered in `NotificationReadStateServiceIT`. \ No newline at end of file +## Test Coverage + +| Scenario | Assertion | +|----------|-----------| +| Read flag merge | `NotificationView.read()` reflects `ReadStatus` | +| Audience isolation by ID | Caller only sees their own `read_state` rows | +| Audience isolation by type | `USER` vs `MACHINE` rows are non-overlapping | +| Read/unread filter | `NotificationFilter(true/false, null)` narrows results correctly | +| Short search term (`< 2` chars) | Normalized to no-search; full page returned | +| Whitespace-only search | Trimmed to empty; full page returned | +| Search with no match | Empty page, `hasNextPage=false` | +| Soft-deleted notifications | Excluded from default listings | +| Sort `ASC` | Oldest-first ordering | +| Sort `null` | Defaults to newest-first (`DESC`) | +| Sort with unsupported field | Unknown field ignored; direction still honored | + +> **Note:** `markRead`, `markAllAsRead`, `delete`, `hasUnread`, and counts are covered in `NotificationReadStateServiceIT`, not here. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/mapper/.ScriptMapperTest.md b/openframe-api-service-core/src/test/java/com/openframe/api/mapper/.ScriptMapperTest.md new file mode 100644 index 000000000..44d9d9e75 --- /dev/null +++ b/openframe-api-service-core/src/test/java/com/openframe/api/mapper/.ScriptMapperTest.md @@ -0,0 +1,39 @@ + +Unit tests for `ScriptMapper#updateEntity`, verifying PUT semantics β€” that every writable field is correctly overwritten on the entity, including explicit `null` values that must clear previously-set data. + +## Key Components + +| Test | Description | +|------|-------------| +| `updateEntity_nullsInInput_clearFieldsOnEntity` | Asserts all writable fields are set to `null` when input has no values set | +| `updateEntity_doesNotTouchInternalFields` | Confirms `id` and `tenantId` are never modified by the mapper | +| `updateEntity_fullyPopulatedInput_overwritesAllFields` | Validates a fully populated `UpdateScriptInput` overwrites every writable field | +| `updateEntity_emptyListInput_clearsListField` | Ensures empty lists (`[]`) explicitly clear list fields rather than being ignored | +| `fullyPopulated()` | Private builder helper that constructs a `Script` with all fields set, used as the baseline entity across tests | + +## Usage Example + +```java +// The mapper is instantiated directly (no Spring context needed) +ScriptMapper mapper = new ScriptMapper(); + +// Build a pre-existing entity +Script existing = Script.builder() + .id("65f4a8000000000000000001") + .tenantId("tenant-1") + .name("Restart Spooler") + .shell(ScriptShell.POWERSHELL) + .build(); + +// Apply a partial update β€” null fields will clear existing values (PUT semantics) +UpdateScriptInput input = new UpdateScriptInput(); +input.setName("New Name"); +// description left null β†’ will clear existing description + +mapper.updateEntity(existing, input); +// existing.getName() β†’ "New Name" +// existing.getDescription() β†’ null (cleared) +// existing.getId() β†’ "65f4a8000000000000000001" (untouched) +``` + +> **Note:** `ScriptServiceTest` mocks the mapper; these tests exercise the real mapper instance to validate actual field-overwrite behaviour, including the distinction between `null` and empty `List.of()` inputs. \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.CommandDispatchServiceTest.md b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.CommandDispatchServiceTest.md new file mode 100644 index 000000000..bc57507db --- /dev/null +++ b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.CommandDispatchServiceTest.md @@ -0,0 +1,43 @@ + +Unit test suite for `CommandDispatchService`, verifying command dispatch and cancellation behavior via NATS messaging in the OpenFrame RMM platform. + +## Key Components + +- **`runCommand` tests** β€” Validates that `CommandDispatchService.runCommand()` generates a unique `executionId`, correctly maps all fields from `RunCommandInput` (shell, command, privilege level, optional timeout) into a `CommandMessage`, and publishes it to the correct machine subject via `CommandNatsPublisher`. +- **`cancelExecution` tests** β€” Validates that `cancelExecution()` relays the caller-supplied `executionId` verbatim in a `CancelMessage`, routes through `publishCancel` (not `publishCommand`), and never mints a new execution ID. +- **`CommandNatsPublisher` (mock)** β€” Injected mock used with `ArgumentCaptor` to assert exact wire-payload contents without a live NATS broker. + +## Usage Example + +```java +// Typical test pattern: capture and assert the published CommandMessage +@Test +void runCommand_publishesAndReturnsExecutionId() { + CommandDispatchResponse response = commandDispatchService.runCommand(input); + + // Execution ID is non-blank and echoed in the response + assertThat(response.getExecutionId()).isNotBlank(); + + // Capture the exact message sent over NATS + ArgumentCaptor captor = ArgumentCaptor.forClass(CommandMessage.class); + verify(commandNatsPublisher).publishCommand(eq(MACHINE_ID), captor.capture()); + + CommandMessage sent = captor.getValue(); + assertThat(sent.getExecutionId()).isEqualTo(response.getExecutionId()); + assertThat(sent.getShell()).isEqualTo(ScriptShell.BASH); + assertThat(sent.getPrivilegeLevel()).isEqualTo(PrivilegeLevel.ADMIN); + assertThat(sent.getTimeout()).isNull(); // agent default applies +} +``` + +## Test Coverage Summary + +| Scenario | Assertion Focus | +|---|---| +| `runCommand` basic dispatch | Non-blank ID, correct NATS subject, full payload mapping | +| Privilege level forwarding | `USER` vs `ADMIN` not collapsed to a backend default | +| Optional timeout forwarding | Timeout passed verbatim, not modified | +| Distinct execution IDs | No global state leakage across invocations | +| `cancelExecution` relay | Caller ID echoed verbatim, `CancelMessage` published | +| Cancel idempotency | Same ID relayed on repeated calls | +| Routing isolation | `publishCancel` and `publishCommand` use separate paths | \ No newline at end of file diff --git a/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.ScriptServiceTest.md b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.ScriptServiceTest.md new file mode 100644 index 000000000..2e11ee374 --- /dev/null +++ b/openframe-api-service-core/src/test/java/com/openframe/api/service/rmm/.ScriptServiceTest.md @@ -0,0 +1,47 @@ + +Unit test suite for `ScriptService`, covering CRUD operations, cursor-based pagination, and tenant-scoped data access using Mockito and AssertJ. + +## Key Components + +| Component | Role | +|-----------|------| +| `ScriptServiceTest` | Main test class wiring `ScriptService` under test via `@InjectMocks` | +| `ScriptRepository` (mock) | Simulates MongoDB document persistence and paginated queries | +| `ScriptMapper` (mock) | Validates entity↔DTO mapping is delegated correctly | +| `TenantIdProvider` (mock) | Provides tenant isolation (`tenant-1`) across all test scenarios | + +**Test methods:** + +- `create_whenNameUnique_persistsAndReturnsResponse` β€” happy path: entity is saved and response returned +- `create_whenNameAlreadyExists_throwsConflict` β€” duplicate name within tenant throws `ConflictException` +- `create_whenInputHasEnvVars_persistsThemThroughTheMapper` β€” env vars (including secrets) flow through mapper +- `get_whenExists_returnsResponse` β€” tenant-scoped lookup returns mapped DTO +- `get_whenNotFound_throwsNotFound` β€” missing script throws `NotFoundException` +- `list_forwardFirstPage_dropsSentinelAndReportsHasNext` β€” forward pagination drops `limit+1` sentinel, sets `hasNextPage=true` +- `list_forwardLastPage_hasNoNext` β€” last page correctly sets `hasNextPage=false` +- `list_backwardPagination_reversesItemsAndFlipsHasMore` β€” backward cursor reverses item order and maps `hasMore` to `hasPreviousPage` +- `list_emptyResult_returnsEmptyPageInfo` β€” empty dataset returns null cursors and `false` flags +- `list_normalisesPaginationCriteria` β€” null `limit` is normalized before repository call + +## Usage Example + +```java +// Typical forward pagination test pattern used in this suite +CursorPaginationCriteria criteria = CursorPaginationCriteria.builder() + .limit(2) + .cursor(null) + .backward(false) + .build(); + +when(scriptRepository.findPageForTenant( + eq(TENANT_ID), eq(null), eq(null), + eq("_id"), eq(Sort.Direction.DESC), + eq(null), eq(false), eq(3))) // limit+1 sentinel fetch + .thenReturn(List.of(s1, s2, s3)); + +GenericQueryResult result = scriptService.list(null, null, null, criteria); + +assertThat(result.getItems()).hasSize(2); // sentinel dropped +assertThat(result.getPageInfo().isHasNextPage()).isTrue(); +assertThat(result.getPageInfo().isHasPreviousPage()).isFalse(); // first page, no cursor supplied +``` \ No newline at end of file diff --git a/openframe-authorization-service-core/src/main/java/com/openframe/authz/exception/.OwnerCannotSwitchTenantException.md b/openframe-authorization-service-core/src/main/java/com/openframe/authz/exception/.OwnerCannotSwitchTenantException.md new file mode 100644 index 000000000..1f7da713f --- /dev/null +++ b/openframe-authorization-service-core/src/main/java/com/openframe/authz/exception/.OwnerCannotSwitchTenantException.md @@ -0,0 +1,19 @@ + +A specialized exception thrown when a tenant owner attempts to switch to a different tenant, which is a restricted operation in OpenFrame's authorization model. + +## Key Components + +- **`OwnerCannotSwitchTenantException`** β€” Extends `ConflictException` (HTTP 409), signaling a business rule violation rather than a system error. +- **`ErrorCode.OWNER_CANNOT_SWITCH_TENANT`** β€” Structured error code passed to the parent for consistent API error responses. +- **Constructor parameter:** `email` β€” The owner's email address, appended to the message for traceability. + +## Usage Example + +```java +// Thrown during tenant-switch validation in the authorization service +if (user.isOwner()) { + throw new OwnerCannotSwitchTenantException(user.getEmail()); +} +``` + +> Tenant owners are bound to their originating tenant. Attempting to switch tenants results in a `409 Conflict` response with error code `OWNER_CANNOT_SWITCH_TENANT`. \ No newline at end of file diff --git a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md index 28862a828..8842df0bd 100644 --- a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md +++ b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/processor/.RegistrationProcessor.md @@ -1,43 +1,38 @@ - -Defines a hook-based interface for intercepting registration lifecycle events across tenant, invitation, and SSO auto-provisioning flows. + +Defines the `RegistrationProcessor` interface, providing pre/post processing hooks for tenant registration, invitation-based registration, SSO auto-provisioning, and tenant ID reservation. ## Key Components -| Method | Trigger Point | +| Method | Description | |---|---| -| `preProcessTenantRegistration` | Before tenant registration logic executes | -| `postProcessTenantRegistration` | After a tenant and user are successfully created | -| `postProcessInvitationRegistration` | After a user registers via invitation link | -| `postProcessAutoProvision` | After SSO first-login provisioning or subsequent SSO profile refresh | +| `preProcessTenantRegistration` | Hook called before tenant registration logic executes | +| `postProcessTenantRegistration` | Hook called after successful tenant registration, receives the created `Tenant` and `AuthUser` | +| `postProcessInvitationRegistration` | Hook called after a user registers via an invitation link | +| `postProcessAutoProvision` | Hook for SSO first-login provisioning and subsequent profile refresh (supports optional `pictureUrl`) | +| `reserveTenantIdForRegistration` | Returns the `tenantId` for the new tenant β€” defaults to a fresh UUID; SaaS implementations atomically claim a pre-generated ID from a READY cluster | -All methods provide default no-op implementations, making partial implementation safe β€” only override the hooks relevant to your use case. +All methods ship as **no-op defaults**, so implementors only override what they need. ## Usage Example ```java @Component -public class AuditRegistrationProcessor implements RegistrationProcessor { - - private final AuditService auditService; +public class CustomRegistrationProcessor implements RegistrationProcessor { @Override public void postProcessTenantRegistration( - Tenant tenant, - AuthUser user, - TenantRegistrationRequest request) { - auditService.logTenantCreated(tenant.getId(), user.getId()); + Tenant tenant, AuthUser user, TenantRegistrationRequest request) { + // Send welcome email, provision default resources, etc. + notificationService.sendWelcome(user.getEmail(), tenant.getTenantId()); } @Override - public void postProcessAutoProvision(AuthUser user, String pictureUrl) { - if (pictureUrl != null) { - auditService.logProfilePictureUpdated(user.getId(), pictureUrl); - } + public String reserveTenantIdForRegistration(TenantRegistrationRequest request) { + // SaaS: claim a pre-provisioned cluster ID + return clusterPoolService.claimNextReadyTenantId(); } // preProcessTenantRegistration and postProcessInvitationRegistration - // intentionally left as no-ops via default implementations + // remain as no-ops via default interface methods } -``` - -> Implement this interface to inject custom business logic β€” such as audit logging, welcome emails, or CRM sync β€” at any stage of the registration pipeline without modifying core auth flows. \ No newline at end of file +``` \ No newline at end of file diff --git a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md index 38704311e..78fc48180 100644 --- a/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md +++ b/openframe-authorization-service-core/src/main/java/com/openframe/authz/service/user/.InvitationRegistrationService.md @@ -1,36 +1,57 @@ - -Service that handles user registration through invitation links in the OpenFrame multi-tenant authentication system. It manages tenant switching, user deactivation, and invitation acceptance workflows. + +Handles the full registration flow for invitation-based user sign-ups, resolving tenant assignment, managing existing active users, and finalizing invitation acceptance. ## Key Components -- **registerByInvitation()** - Main entry point that processes invitation registration requests -- **handleExistingActiveUser()** - Manages scenarios where user already exists in same or different tenant -- **createUserForInvitation()** - Creates new user account from invitation details -- **acceptInvitation()** - Marks invitation as accepted and triggers post-processing -- **resolveTargetTenantId()** - Determines target tenant based on local tenant configuration +| Member | Type | Description | +|--------|------|-------------| +| `registerByInvitation` | `public` | Entry point β€” validates invitation, resolves tenant, creates or reuses user, and accepts the invitation | +| `resolveTargetTenantId` | `private` | Returns the first local tenant ID when `localTenant=true`, otherwise uses the tenant from the invitation | +| `handleExistingActiveUser` | `private` | Handles collision when an active user already exists for the invited email β€” reuse, deactivate, or throw | +| `createUserForInvitation` | `private` | Delegates new user creation to `UserService` using invitation and request data | +| `acceptInvitation` | `private` | Marks invitation as `ACCEPTED`, persists it, and triggers post-processing | +| `markVerifiedQuietly` | `private` | Silently marks email as verified; swallows exceptions to avoid blocking acceptance | + +## Configuration + +| Property | Default | Description | +|----------|---------|-------------| +| `openframe.tenancy.local-tenant` | `false` | When `true`, forces all invitations to resolve to the first tenant in the repository | + +## Tenant Switch Logic + +```mermaid +graph TD + A["registerByInvitation()"] --> B["loadAndEnsureAcceptable()"] + B --> C["findActiveByEmail()"] + C --> D{"Existing user?"} + D -->|"No"| E["createUserForInvitation()"] + D -->|"Yes, same tenant"| F["reuse existing user"] + D -->|"Yes, different tenant"| G{"switchTenant=true?"} + G -->|"No"| H["UserActiveInAnotherTenantException"] + G -->|"Yes"| I{"Has OWNER role?"} + I -->|"Yes"| J["OwnerCannotSwitchTenantException"] + I -->|"No"| K["deactivateUser() + create new"] + E --> L["acceptInvitation()"] + F --> L + K --> L +``` ## Usage Example ```java -@Autowired -private InvitationRegistrationService registrationService; - -public void processUserInvitation() { - InvitationRegistrationRequest request = InvitationRegistrationRequest.builder() - .invitationId("inv-123") - .firstName("John") - .lastName("Doe") - .password("securePassword123") - .switchTenant(true) // Allow switching from current tenant - .build(); - - try { - AuthUser newUser = registrationService.registerByInvitation(request); - log.info("User registered successfully: {}", newUser.getEmail()); - } catch (UserActiveInAnotherTenantException e) { - log.error("User already active in different tenant: {}", e.getMessage()); - } -} +// Called from a registration controller after user submits invitation form +InvitationRegistrationRequest request = InvitationRegistrationRequest.builder() + .invitationId("inv_abc123") + .firstName("Jane") + .lastName("Doe") + .password("s3cur3P@ss") + .switchTenant(false) + .build(); + +AuthUser registeredUser = invitationRegistrationService.registerByInvitation(request); +// registeredUser is now active in the invitation's target tenant +// invitation status set to ACCEPTED ``` -The service supports tenant switching scenarios - if a user is already active in another tenant and `switchTenant=true`, it deactivates the existing user account before creating a new one in the target tenant. For local tenant deployments, it automatically resolves to the first available tenant. \ No newline at end of file +> **Note:** Setting `switchTenant=true` deactivates the user's current tenant membership before registering under the new one. Users with the `OWNER` role are never permitted to switch tenants and will always throw `OwnerCannotSwitchTenantException`. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/config/.AsyncConfig.md b/openframe-client-core/src/main/java/com/openframe/client/config/.AsyncConfig.md new file mode 100644 index 000000000..9c6d10a95 --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/config/.AsyncConfig.md @@ -0,0 +1,32 @@ + +Configures asynchronous task execution for the OpenFrame client, enabling Spring's async processing with a virtual-thread-based executor optimized for tool installation tasks. + +## Key Components + +| Element | Description | +|---------|-------------| +| `AsyncConfig` | Spring `@Configuration` class that enables async processing via `@EnableAsync` | +| `TOOL_INSTALL_EXECUTOR` | Constant (`"toolInstallExecutor"`) used to reference the named executor bean | +| `toolInstallExecutor()` | Bean that wraps Java 21 virtual threads (`Executors.newVirtualThreadPerTaskExecutor()`) via `TaskExecutorAdapter` | + +## Usage Example + +Inject the executor by name into any Spring component to run tool installation tasks asynchronously on lightweight virtual threads: + +```java +import org.springframework.scheduling.annotation.Async; +import org.springframework.stereotype.Service; + +@Service +public class ToolInstallService { + + @Async(AsyncConfig.TOOL_INSTALL_EXECUTOR) + public CompletableFuture installTool(String toolId) { + // Runs on a virtual thread - ideal for I/O-bound install operations + performInstallation(toolId); + return CompletableFuture.completedFuture(null); + } +} +``` + +> **Note:** Uses Java 21 virtual threads (`Project Loom`), making it well-suited for I/O-bound workloads like downloading, installing, or configuring MSP tools β€” where high concurrency is needed without the overhead of platform threads. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md b/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md index 2303b4c4c..5b4fc8e4a 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md +++ b/openframe-client-core/src/main/java/com/openframe/client/controller/.AgentController.md @@ -1,31 +1,39 @@ - -A Spring Boot REST controller that handles agent registration functionality for the OpenFrame client system through a secure API endpoint. + +REST controller handling agent registration and reinstallation endpoints for the OpenFrame platform, exposing two authenticated operations under `/api/agents`. ## Key Components -- **AgentController**: Main REST controller class that exposes agent-related endpoints -- **register()**: POST endpoint that handles agent registration with initial key authentication -- **AgentRegistrationService**: Injected service that contains the business logic for agent registration -- **Security Headers**: Uses `X-Initial-Key` header for authentication during registration +| Element | Description | +|---|---| +| `AgentController` | `@RestController` mapped to `/api/agents` | +| `POST /register` | Registers a new agent using an initial key and registration payload | +| `POST /reinstall` | Reinstalls an existing agent using initial key, machine ID, and client secret | +| `AgentRegistrationService` | Delegated service handling registration business logic | + +### Request Headers + +| Header | Endpoint(s) | Purpose | +|---|---|---| +| `X-Initial-Key` | Both | Bootstrap authentication key | +| `X-Machine-Id` | `/reinstall` only | Identifies the existing machine | +| `X-Client-Secret` | `/reinstall` only | Authenticates the existing agent | ## Usage Example ```java -// Example request to register an agent -@PostMapping("/register") -public ResponseEntity register( - @RequestHeader("X-Initial-Key") String initialKey, - @Valid @RequestBody AgentRegistrationRequest request) { - - // Service handles validation and registration logic - AgentRegistrationResponse response = agentRegistrationService.register(initialKey, request); - return ResponseEntity.ok(response); -} - -// Example client usage (conceptual) -// POST /api/agents/register -// Headers: X-Initial-Key: your-initial-key -// Body: AgentRegistrationRequest JSON payload +// Register a new agent +POST /api/agents/register +Headers: + X-Initial-Key: +Body: AgentRegistrationRequest { ... } + +// Reinstall an existing agent +POST /api/agents/reinstall +Headers: + X-Initial-Key: + X-Machine-Id: + X-Client-Secret: +Body: AgentRegistrationRequest { ... } ``` -The controller uses constructor injection via Lombok's `@RequiredArgsConstructor` and validates incoming requests with `@Valid`. All agent registration requests require an initial authentication key provided through the `X-Initial-Key` header for security purposes. \ No newline at end of file +Both endpoints return an `AgentRegistrationResponse` and enforce `@Valid` bean validation on the request body. Authentication credentials are passed exclusively via HTTP headers, keeping them out of the request body. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md b/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md index e1d54a2d3..5eff5f666 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md +++ b/openframe-client-core/src/main/java/com/openframe/client/dto/agent/.AgentRegistrationRequest.md @@ -1,59 +1,48 @@ - -Data Transfer Object (DTO) used to capture all device metadata submitted by an agent during the registration process with the OpenFrame platform. + +Data Transfer Object (DTO) for registering a new agent/device with the OpenFrame platform, capturing device identity, network details, hardware specs, OS information, and optional tags. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `hostname` | `String` | Device hostname | -| `organizationId` | `String` | Target organization for the device | -| `ip` | `String` | Device IP address | -| `macAddress` | `String` | Network MAC address | -| `osUuid` | `String` | OS-level unique identifier | -| `agentVersion` | `String` | **Required** β€” agent version string | -| `status` | `DeviceStatus` | Current device status enum | -| `displayName` | `String` | Human-readable device name | -| `serialNumber` | `String` | Hardware serial number | -| `manufacturer` | `String` | Device manufacturer | -| `model` | `String` | Device model identifier | -| `type` | `DeviceType` | Device type enum | -| `osType` / `osVersion` / `osBuild` | `String` | OS details | -| `timezone` | `String` | Device timezone | -| `tags` | `List` | Tags to create and assign at registration | +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `hostname` | `String` | βœ… | Unique device hostname | +| `organizationId` | `String` | βœ… | Owning organization | +| `agentVersion` | `String` | βœ… | Version of the installed agent | +| `osType` | `String` | βœ… | Operating system type identifier | +| `ip` | `String` | ❌ | Device IP address | +| `macAddress` | `String` | ❌ | Network MAC address | +| `osUuid` | `String` | ❌ | OS-level unique identifier | +| `status` | `DeviceStatus` | ❌ | Initial device status enum | +| `type` | `DeviceType` | ❌ | Device type classification enum | +| `tags` | `List` | ❌ | Tags to create and assign at registration | ## Usage Example ```java AgentRegistrationRequest request = new AgentRegistrationRequest(); -// Core identification +// Required fields request.setHostname("workstation-01"); request.setOrganizationId("org_abc123"); +request.setAgentVersion("2.4.1"); +request.setOsType("WINDOWS"); -// Network -request.setIp("192.168.1.100"); +// Optional hardware/network fields +request.setIp("192.168.1.42"); request.setMacAddress("AA:BB:CC:DD:EE:FF"); -request.setAgentVersion("2.4.1"); // @NotBlank β€” required - -// Hardware request.setManufacturer("Dell"); -request.setModel("Latitude 5520"); -request.setSerialNumber("SN-987654"); - -// OS -request.setType(DeviceType.WORKSTATION); -request.setOsType("Windows"); +request.setModel("OptiPlex 7090"); +request.setSerialNumber("SN123456"); request.setOsVersion("11"); request.setOsBuild("22H2"); request.setTimezone("America/New_York"); +request.setStatus(DeviceStatus.ONLINE); +request.setType(DeviceType.WORKSTATION); -// Tags (validated with @Valid) +// Optional tags AgentRegistrationTagInput tag = new AgentRegistrationTagInput(); -tag.setName("managed"); +tag.setName("office-nyc"); request.setTags(List.of(tag)); ``` -## Validation Notes - -- `agentVersion` is the only `@NotBlank` enforced field β€” registration will be rejected if omitted. -- `tags` list is validated recursively via `@Valid`, delegating constraint checks to `AgentRegistrationTagInput`. \ No newline at end of file +> Validation is enforced via Jakarta Bean Validation β€” `@NotBlank` guards required string fields, and `@Valid` cascades validation into each `AgentRegistrationTagInput` entry in the `tags` list. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/exception/.InvalidClientSecretException.md b/openframe-client-core/src/main/java/com/openframe/client/exception/.InvalidClientSecretException.md new file mode 100644 index 000000000..915075e01 --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/exception/.InvalidClientSecretException.md @@ -0,0 +1,18 @@ + +Signals that a client has provided an invalid or unrecognized secret during authentication, extending `UnauthorizedException` to indicate an HTTP 401-level access denial. + +## Key Components + +- **`InvalidClientSecretException`** β€” Concrete exception class extending `UnauthorizedException`, used specifically when client secret validation fails. +- **Constructor** β€” Accepts an `ErrorCode` (for machine-readable error classification) and a `String message` (for human-readable detail), delegating both to the parent via `super()`. + +## Usage Example + +```java +if (!secretEncoder.matches(providedSecret, storedSecret)) { + throw new InvalidClientSecretException( + ErrorCode.INVALID_CLIENT_SECRET, + "The client secret provided does not match our records." + ); +} +``` \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md b/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md index 35c1b778c..c279018fa 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md +++ b/openframe-client-core/src/main/java/com/openframe/client/listener/.MachineHeartbeatListener.md @@ -1,36 +1,29 @@ - -A Spring component that listens for machine heartbeat messages via NATS messaging system and processes them to track machine status. + +NATS message listener that subscribes to machine heartbeat events and delegates processing to `MachineStatusService`. It subscribes on application startup and gracefully drains the dispatcher on shutdown. ## Key Components -- **`@EventListener(ApplicationReadyEvent.class)`** - Automatically subscribes to heartbeat messages when the application starts -- **`subscribeToMachineHeartbeats()`** - Sets up NATS dispatcher to listen on `machine.*.heartbeat` subject pattern -- **`handleMessage(Message)`** - Processes incoming heartbeat messages by extracting machine ID and timestamp -- **`@PreDestroy cleanup()`** - Gracefully shuts down the NATS dispatcher on application shutdown - -## Dependencies - -- **`MachineStatusService`** - Processes heartbeat data for machine status tracking -- **`NatsTopicMachineIdExtractor`** - Extracts machine ID from NATS topic patterns -- **NATS Connection** - Handles message subscription and delivery +| Member | Description | +|--------|-------------| +| `SUBJECT` | Wildcard topic pattern `machine.*.heartbeat` matching all machine heartbeat messages | +| `subscribeToMachineHeartbeats()` | Initializes a NATS `Dispatcher` and subscribes on `ApplicationReadyEvent` | +| `handleMessage(Message)` | Callback invoked per message; extracts machine ID, stamps the event time, and calls `machineStatusService.processHeartbeat()` | +| `cleanup()` | `@PreDestroy` hook that drains the dispatcher with a 5-second timeout for graceful shutdown | ## Usage Example +The listener is auto-registered as a Spring `@Component`. No manual wiring is needed β€” it activates when the application context is fully started. + ```java -// The listener automatically starts when the Spring application is ready -// It subscribes to subjects like: -// - machine.001.heartbeat -// - machine.production-line-A.heartbeat -// - machine.sensor-xyz.heartbeat - -// When a heartbeat message arrives, it: -// 1. Extracts machine ID from the subject -// 2. Generates current timestamp -// 3. Calls machineStatusService.processHeartbeat(machineId, timestamp) - -// Configuration in application.yml: -// nats: -// url: nats://localhost:4222 +// Triggered automatically on ApplicationReadyEvent. +// To simulate the flow in a test: +Message mockMessage = Mockito.mock(Message.class); +when(mockMessage.getSubject()).thenReturn("machine.abc-123.heartbeat"); + +machineHeartbeatListener.handleMessage(mockMessage); + +// MachineStatusService.processHeartbeat("abc-123", ) is called internally +verify(machineStatusService).processHeartbeat(eq("abc-123"), any(Instant.class)); ``` -The component uses Spring's lifecycle management to ensure clean startup and shutdown, with proper error handling and logging throughout the message processing pipeline. \ No newline at end of file +> The `Dispatcher` is thread-managed by the NATS client internally. `handleMessage` runs on a NATS-owned thread, so `MachineStatusService` implementations must be thread-safe. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md b/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md index 953be7312..c66e46dc2 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md +++ b/openframe-client-core/src/main/java/com/openframe/client/listener/.ToolConnectionListener.md @@ -1,42 +1,39 @@ - -A Spring component that listens for tool connection events on a NATS JetStream subject, managing durable consumer lifecycle and delegating processing to `ToolConnectionService`. + +A Spring component that subscribes to a NATS JetStream subject to process tool connection events for OpenFrame machines, handling consumer lifecycle management including creation, update, and graceful shutdown. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `STREAM_NAME` | Constant | Target JetStream stream (`TOOL_CONNECTIONS`) | -| `SUBJECT` | Constant | Wildcard subject pattern (`machine.*.tool-connection`) | -| `CONSUMER_NAME` | Constant | Durable consumer identifier (`tool-connection-processor-v2`) | -| `MAX_DELIVER` | Constant | Maximum redelivery attempts before message is dead-lettered (50) | -| `ACK_WAIT` | Constant | Acknowledgement timeout window (30 seconds) | +| `STREAM_NAME` | Constant | JetStream stream identifier: `TOOL_CONNECTIONS` | +| `SUBJECT` | Constant | Wildcard subject pattern: `machine.*.tool-connection` | +| `CONSUMER_NAME` | Constant | Durable consumer name: `tool-connection-processor-v2` | +| `MAX_DELIVER` | Constant | Maximum redelivery attempts before abandoning (`50`) | +| `ACK_WAIT` | Constant | Acknowledgement timeout (`30s`) | | `subscribeToToolConnections()` | Method | Bootstraps JetStream push subscription on `ApplicationReadyEvent` | -| `buildConsumerConfig()` | Method | Creates or updates the durable consumer with delivery group support | -| `handleMessage()` | Method | Deserializes and routes each `ToolConnectionMessage` | -| `cleanup()` | Method | Gracefully unsubscribes and drains the dispatcher on shutdown | +| `buildConsumerConfig()` | Method | Creates or updates a durable consumer with explicit ack and delivery group | +| `handleMessage()` | Method | Deserializes and dispatches each `ToolConnectionMessage` to `ToolConnectionService` | +| `isLastAttempt()` | Method | Checks if current delivery equals `MAX_DELIVER` to signal final processing attempt | +| `cleanup()` | Method | Gracefully unsubscribes and drains the dispatcher on application shutdown (`@PreDestroy`) | ## Usage Example ```java -// Bean is auto-configured by Spring on startup. -// Subscription is triggered automatically via ApplicationReadyEvent. +// Bean is auto-registered via @Component and triggers on startup +// No manual instantiation needed. Internally it behaves as follows: -// Internally, messages are published to the subject pattern: -// machine..tool-connection +// 1. On ApplicationReadyEvent, subscription is established: +// Subject : machine.*.tool-connection +// Consumer : tool-connection-processor-v2 (durable, explicit ack) +// Group : tool-connection (load-balanced delivery group) -// Example payload consumed by this listener: -{ - "toolType": "antivirus", - "agentToolId": "tool-abc-123" -} +// 2. Each received message is handled: +ToolConnectionMessage msg = objectMapper.readValue(payload, ToolConnectionMessage.class); +toolConnectionService.addToolConnection(machineId, msg.getToolType(), msg.getAgentToolId(), lastAttempt); +message.ack(); -// On success: message.ack() is called. -// On failure: message is left unacked for redelivery (up to MAX_DELIVER=50 attempts). -// On last attempt: lastAttempt=true is passed to ToolConnectionService. +// 3. On failure, the message is left unacked for redelivery up to MAX_DELIVER (50) times +// 4. On shutdown, the subscription is cleanly drained over a 5-second window ``` -## Notes - -- The `v2` consumer suffix was introduced during a hotfix to support delivery groups, which cannot be applied to existing consumers retroactively. -- Uses explicit ACK policy β€” failed processing never silently discards messages. -- `@PreDestroy` ensures clean drain of in-flight messages during shutdown with a 5-second grace period. \ No newline at end of file +> **Note:** The `-v2` consumer name suffix was introduced during a hotfix to enable delivery group support, as existing consumers without a delivery group cannot be migrated in-place. The previous consumer (`tool-connection-processor`) is deprecated. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md b/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md index 5ddc7d454..b8686162f 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md +++ b/openframe-client-core/src/main/java/com/openframe/client/service/.MachineStatusService.md @@ -1,23 +1,17 @@ - -Manages machine (device) connectivity status transitions within the OpenFrame platform, handling online/offline updates, heartbeat processing, and first-connection event detection with stale-event protection. + +Manages machine connectivity status transitions by processing online/offline events and heartbeats, with stale event protection and first-connection detection. ## Key Components -| Method | Description | -|--------|-------------| -| `updateToOnline()` | Marks a machine as `ONLINE` for a given timestamp | -| `updateToOffline()` | Marks a machine as `OFFLINE` for a given timestamp | -| `processHeartbeat()` | Treats an incoming heartbeat as an `ONLINE` status update | -| `update()` *(private)* | Core logic: loads the machine, validates event freshness, applies status change | -| `isEventNewer()` *(private)* | Guards against out-of-order events by comparing timestamps | -| `applyStatusUpdate()` *(private)* | Persists the new status and fires `DeviceFirstConnectedEvent` on `PENDING β†’ ONLINE/OFFLINE` transition | -| `logStaleEvent()` *(private)* | Warns when an older event arrives after a newer one has already been applied | - -## Behavior Notes - -- **Stale-event protection** β€” events with a timestamp older than `machine.lastSeen` are silently dropped with a warning log. -- **First-connection detection** β€” when a device transitions out of `PENDING` status for the first time, a `DeviceFirstConnectedEvent` is published via Spring's `ApplicationEventPublisher`. -- Throws `MachineNotFoundException` if the `machineId` does not exist in the repository. +| Member | Type | Description | +|--------|------|-------------| +| `updateToOnline` | `public void` | Marks a machine as `ONLINE` for a given timestamp | +| `updateToOffline` | `public void` | Marks a machine as `OFFLINE` for a given timestamp | +| `processHeartbeat` | `public void` | Treats a heartbeat as an `ONLINE` status update | +| `update` | `private void` | Core logic: resolves the machine, guards against stale events, delegates to `applyStatusUpdate` | +| `isEventNewer` | `private boolean` | Returns `true` if `eventTimestamp` is after `lastSeen` (or `lastSeen` is null) | +| `applyStatusUpdate` | `private void` | Persists the new status/timestamp and publishes `DeviceFirstConnectedEvent` on `PENDING β†’ ONLINE/OFFLINE` transition | +| `logStaleEvent` | `private void` | Emits a `WARN` log when an out-of-order event is discarded | ## Usage Example @@ -26,12 +20,18 @@ Manages machine (device) connectivity status transitions within the OpenFrame pl @Autowired private MachineStatusService machineStatusService; -// Device comes online -machineStatusService.updateToOnline("machine-abc-123", Instant.now()); +// Agent comes online +machineStatusService.updateToOnline("machine-abc123", Instant.now()); + +// Agent sends a heartbeat +machineStatusService.processHeartbeat("machine-abc123", Instant.now()); + +// Agent goes offline +machineStatusService.updateToOffline("machine-abc123", Instant.now()); +``` -// Device goes offline -machineStatusService.updateToOffline("machine-abc-123", Instant.now()); +## Notes -// Heartbeat received from agent -machineStatusService.processHeartbeat("machine-abc-123", Instant.now()); -``` \ No newline at end of file +- Throws `MachineNotFoundException` if the `machineId` is not found in `MachineRepository`. +- Stale events (timestamp ≀ `lastSeen`) are silently discarded with a `WARN` log β€” no exception is raised. +- The `DeviceFirstConnectedEvent` fires **once** per machine lifecycle, on the first `PENDING β†’ ONLINE/OFFLINE` transition, enabling downstream provisioning or notification workflows. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md index 9a9a43fe5..bd08f1bd0 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md +++ b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationService.md @@ -1,35 +1,38 @@ - -Handles the end-to-end registration of OpenFrame agents, orchestrating OAuth client creation, machine persistence, tag assignment, tool installation, and post-processing within a single transaction. + +Handles agent registration and reinstallation for OpenFrame client machines, managing OAuth client creation, machine record persistence, and post-registration processing workflows. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `register()` | Method | Entry point β€” validates the initial key, generates credentials, resolves the org, persists all entities, and returns the agent credentials | -| `saveOAuthClient()` | Method | Creates and persists an `OAuthClient` with hashed secret and `AGENT` role using client-credentials grant | -| `saveMachine()` | Method | Builds and persists a `Machine` document with `PENDING` status and `DESKTOP` type | -| `resolveOrganizationId()` | Method | Uses the requested org ID if it exists, otherwise falls back to the tenant's default organization | -| `saveInstalledAgent()` | Method | Optionally records the agent version via `InstalledAgentService` (feature-flagged by `openframe.feature.save-installed-agent-on-registration`) | -| `AGENT_ROLE` | Constant | Role assigned to all registered agent OAuth clients (`"AGENT"`) | -| `CLIENT_ID_TEMPLATE` | Constant | Pattern for client IDs: `agent_{machineId}` | +| `register()` | Method | Registers a new agent: validates the initial key, generates IDs/secrets, creates OAuth client and machine records, then triggers post-processing | +| `reinstall()` | Method | Reinstalls an existing agent: validates credentials, updates the existing machine record, and reruns post-processing without generating new credentials | +| `createOAuthClient()` | Private Method | Persists a new `OAuthClient` with encoded secret, `client_credentials` grant type, and `AGENT` role | +| `createMachine()` | Private Method | Persists a new `Machine` document with `PENDING` status and resolved organization ID | +| `postProcessMachine()` | Private Method | Orchestrates post-registration steps: saves installed agent info, assigns tags, triggers tool installation, and runs the registration processor | +| `updateMachine()` | Private Method | Overwrites machine fields and re-resolves the organization ID during reinstall | ## Usage Example ```java -// Called from a registration endpoint, typically with a pre-shared initial key +// New agent registration AgentRegistrationRequest request = new AgentRegistrationRequest(); request.setHostname("workstation-01"); -request.setIp("192.168.1.50"); request.setOsType("WINDOWS"); -request.setAgentVersion("1.4.2"); -request.setOrganizationId("org-abc123"); // optional; falls back to default org +request.setAgentVersion("2.1.0"); +request.setOrganizationId("org-abc"); +request.setTags(List.of("office", "finance")); -AgentRegistrationResponse response = agentRegistrationService.register(initialKey, request); +AgentRegistrationResponse response = agentRegistrationService.register("initial-secret-key", request); +// response contains: machineId, clientId (e.g. "agent_"), clientSecret -// Response contains credentials the agent uses for subsequent API calls -String machineId = response.getMachineId(); -String clientId = response.getClientId(); // e.g. "agent_" -String clientSecret = response.getClientSecret(); +// Agent reinstall (retains existing credentials) +AgentRegistrationResponse reinstallResponse = agentRegistrationService.reinstall( + "initial-secret-key", + response.getMachineId(), + response.getClientSecret(), + request +); ``` -> **Note:** The entire `register()` flow runs inside a `@Transactional` boundary. A TODO marks a pending two-phase commit strategy for the NATS integration to handle distributed consistency. \ No newline at end of file +> **Note:** The `register` flow enforces uniqueness of generated machine IDs. If a collision is detected, an `IllegalStateException` is thrown before persisting. The `reinstall` flow throws `InvalidClientSecretException` if the provided `machineId`/`clientSecret` pair cannot be validated against an existing `OAuthClient`. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md index 0a4107be7..534bf303a 100644 --- a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md +++ b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.AgentRegistrationToolInstallationService.md @@ -1,34 +1,22 @@ - -A service that coordinates tool agent installation during the agent registration process by retrieving all enabled tool agents and installing them on a specified machine. + +Handles asynchronous tool installation for newly registered agents by retrieving all enabled tool agents and triggering their installation on a specified machine. ## Key Components -- **process(String machineId)** - Main orchestration method that retrieves all enabled tool agents and triggers installation for each one -- **integratedToolAgentService** - Service dependency for managing integrated tool agent data -- **toolInstallationService** - Service dependency that handles the actual tool installation process +- **`process(String machineId)`** β€” Async method (runs on `TOOL_INSTALL_EXECUTOR` thread pool) that fetches all enabled `IntegratedToolAgent` records and dispatches installation for each on the target machine. ## Usage Example ```java -@Autowired -private AgentRegistrationToolInstallationService installationService; - -// Install all enabled tools on a newly registered machine -public void registerNewAgent(String machineId) { - // ... other registration logic - - // Install all available tools - installationService.process(machineId); - - log.info("Tool installation completed for machine: {}", machineId); -} - -// Example in a registration controller -@PostMapping("/register") -public ResponseEntity registerAgent(@RequestParam String machineId) { - installationService.process(machineId); - return ResponseEntity.ok("Agent registered with tools installed"); -} +// Triggered during agent registration flow +agentRegistrationToolInstallationService.process("machine-abc-123"); +// Runs asynchronously on TOOL_INSTALL_EXECUTOR; returns immediately ``` -The service follows a simple orchestration pattern where it retrieves all enabled tool agents from the data layer and delegates the actual installation work to the `ToolInstallationService` for each tool-machine combination. \ No newline at end of file +## Dependencies + +| Dependency | Role | +|---|---| +| `IntegratedToolAgentService` | Retrieves all enabled tool agent configurations | +| `ToolInstallationService` | Executes the per-agent installation logic on the target machine | +| `AsyncConfig.TOOL_INSTALL_EXECUTOR` | Dedicated thread pool for non-blocking tool installs | \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.OrganizationIdResolver.md b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.OrganizationIdResolver.md new file mode 100644 index 000000000..edfce4b31 --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/service/agentregistration/.OrganizationIdResolver.md @@ -0,0 +1,29 @@ + +Resolves an organization ID during agent registration by validating a requested ID against the data store, falling back to the default organization if the requested ID is absent or not found. + +## Key Components + +- **`resolve(String requestedOrganizationId)`** β€” Primary entry point. Returns the requested organization ID if it exists, otherwise logs a warning and falls back to the default organization ID. +- **`existsRequested(String)`** β€” Private helper that checks the ID is non-blank and exists via `OrganizationService`. +- **`getDefaultOrganizationId()`** β€” Private helper that fetches the default organization; throws `IllegalStateException` if none is configured. + +## Usage Example + +```java +@Component +@RequiredArgsConstructor +public class AgentRegistrationService { + + private final OrganizationIdResolver organizationIdResolver; + + public void registerAgent(AgentRegistrationRequest request) { + // Resolves to provided ID if valid, otherwise falls back to default + String organizationId = organizationIdResolver.resolve(request.getOrganizationId()); + + // Proceed with registration using the resolved organization ID + ... + } +} +``` + +> **Note:** If no default organization exists (e.g., tenant setup was incomplete), `resolve()` will throw an `IllegalStateException`. Ensure the default organization is created during tenant registration to avoid this failure path. \ No newline at end of file diff --git a/openframe-client-core/src/main/java/com/openframe/client/service/validator/.ClientSecretValidator.md b/openframe-client-core/src/main/java/com/openframe/client/service/validator/.ClientSecretValidator.md new file mode 100644 index 000000000..03056a4c2 --- /dev/null +++ b/openframe-client-core/src/main/java/com/openframe/client/service/validator/.ClientSecretValidator.md @@ -0,0 +1,35 @@ + +Validates the client secret provided during OAuth authentication against the stored encoded secret for an `OAuthClient`. + +## Key Components + +| Element | Description | +|---|---| +| `ClientSecretValidator` | Spring `@Component` responsible for client secret validation | +| `validate(OAuthClient, String)` | Checks secret is non-empty, then verifies it against the BCrypt-encoded stored value | +| `PasswordEncoder` | Injected Spring Security encoder used for secure hash comparison | +| `InvalidClientSecretException` | Thrown with specific `ErrorCode` on empty (`CLIENT_SECRET_EMPTY`) or mismatched (`CLIENT_SECRET_INVALID`) secrets | + +## Usage Example + +```java +@Service +@RequiredArgsConstructor +public class OAuthTokenService { + + private final ClientSecretValidator clientSecretValidator; + private final OAuthClientRepository clientRepository; + + public TokenResponse issueToken(TokenRequest request) { + OAuthClient client = clientRepository.findByClientId(request.getClientId()) + .orElseThrow(() -> new ClientNotFoundException(...)); + + // Throws InvalidClientSecretException if empty or invalid + clientSecretValidator.validate(client, request.getClientSecret()); + + return generateToken(client); + } +} +``` + +> **Note:** The raw secret is never stored β€” comparison is delegated entirely to `PasswordEncoder.matches()`, ensuring the plain-text secret is only transiently held in memory during the request lifecycle. \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md b/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md index 13e32af0b..7ca9bab01 100644 --- a/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md +++ b/openframe-client-core/src/test/java/com/openframe/client/controller/.AgentControllerTest.md @@ -1,47 +1,49 @@ - -Unit test suite for the `AgentController`, validating HTTP behavior of the `/api/agents/register` endpoint using MockMvc with mocked service dependencies. + +Unit tests for `AgentController`, verifying HTTP request/response behavior for agent registration and reinstallation endpoints using MockMvc and Mockito. ## Key Components | Component | Description | |-----------|-------------| -| `AgentController` | Controller under test, wired with a mocked `AgentRegistrationService` | -| `AgentRegistrationService` | Mocked service handling agent registration logic | -| `AgentRegistrationRequest` | DTO with hostname, IP, MAC address, OS UUID, and agent version | -| `AgentRegistrationResponse` | DTO returning `machineId`, `clientId`, and `clientSecret` | -| `BaseGlobalExceptionHandler` | Applied via `setControllerAdvice` to validate error response shapes | -| `TestAuthenticationManager` | Stub authentication manager injected via `BasicAuthenticationFilter` | - -## Test Coverage - -| Test | Expected Outcome | -|------|-----------------| -| `register_WithValidRequest_ReturnsOk` | `200 OK` with full credential payload | -| `register_MissingHeader_ReturnsBadRequest` | `400 BAD_REQUEST` when `X-Initial-Key` is absent | -| `register_WithoutInitialKey_ReturnsBadRequest` | `400 BAD_REQUEST` (duplicate missing-header scenario) | -| `register_WithInvalidInitialKey_ReturnsUnauthorized` | `401 UNAUTHORIZED` on secret validation failure | -| `register_WithDuplicateMachineId_ReturnsConflict` | `409 CONFLICT` when machine is already registered | -| `register_WithValidRequest_ReturnsCredentials` | `200 OK` with `clientId` and `clientSecret` | -| `register_WithValidRequest_StoresAgentInfo` | Verifies service receives all request fields correctly | +| `AgentController` | Controller under test, wrapping `AgentRegistrationService` | +| `AgentRegistrationService` | Mocked service handling registration/reinstall logic | +| `AgentRegistrationRequest` | Request DTO with fields: `hostname`, `ip`, `macAddress`, `osUuid`, `osType`, `agentVersion` | +| `AgentRegistrationResponse` | Response DTO returning `machineId`, `clientId`, `clientSecret` | +| `TestAuthenticationManager` | Test utility simulating Basic Auth filter | +| `BaseGlobalExceptionHandler` | Shared exception handler mapping domain exceptions to HTTP status codes | + +## Covered Test Cases + +**`POST /api/agents/register`** +- Valid request β†’ `200 OK` with credentials +- Missing `X-Initial-Key` header β†’ `400 Bad Request` +- Invalid initial key β†’ `401 Unauthorized` +- Duplicate machine β†’ `409 Conflict` +- Missing mandatory `hostname` field β†’ `400 VALIDATION_ERROR` +- Verifies service receives correct request payload + +**`POST /api/agents/reinstall`** +- Valid headers + body β†’ `200 OK` +- Missing `X-Initial-Key` β†’ `400 Bad Request` +- Missing `X-Machine-Id` β†’ `400 Bad Request` +- Missing `X-Client-Secret` β†’ `400 Bad Request` +- Invalid client secret β†’ *(truncated in source)* ## Usage Example ```java -// Example: how the service mock is configured and verified -when(agentRegistrationService.register(eq("test-key"), any(AgentRegistrationRequest.class))) - .thenReturn(registrationResponse); - +// Example: how the controller under test is exercised mockMvc.perform(post("/api/agents/register") .header("X-Initial-Key", "test-key") .contentType(MediaType.APPLICATION_JSON) .content(objectMapper.writeValueAsString(registrationRequest))) .andExpect(status().isOk()) - .andExpect(jsonPath("$.machineId").value("test-machine-id")); + .andExpect(jsonPath("$.machineId").value("test-machine-id")) + .andExpect(jsonPath("$.clientSecret").value("client-secret")); -// Verify all fields are passed through to the service +// Verify the service received the correct payload verify(agentRegistrationService).register( eq("test-key"), - argThat(req -> req.getHostname().equals("test-host") && - req.getMacAddress().equals("00:11:22:33:44:55")) + argThat(req -> req.getHostname().equals("test-host")) ); ``` \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md b/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md index 71c0c8f8e..c8518d1cb 100644 --- a/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md +++ b/openframe-client-core/src/test/java/com/openframe/client/service/.AgentRegistrationServiceTest.md @@ -1,40 +1,36 @@ - -Unit test suite for `AgentRegistrationService`, validating agent registration flows including new machine creation, duplicate detection, and tag assignment. + +Unit test suite for `AgentRegistrationService`, validating agent registration and reinstallation workflows including credential generation, machine persistence, tag assignment, and secret validation ordering. ## Key Components | Test Method | Purpose | |---|---| -| `registerAgent_WithNewMachine_ReturnsCredentials` | Verifies full registration lifecycle: credential generation, OAuth client persistence, and machine document creation | -| `registerAgent_WithExistingMachine_ThrowsException` | Ensures duplicate machine detection throws `IllegalStateException` and prevents any saves | -| `registerAgent_WithTags_AssignsTagsToDevice` | Confirms tag assignment is delegated to `RegistrationTagAssignmentService` when tags are present | -| `registerAgent_WithoutTags_DoesNotAssignTags` | Confirms `assignTags` is still called with a `null` tag list when no tags are provided | - -**Mocked dependencies:** `OAuthClientRepository`, `MachineRepository`, `OrganizationService`, `AgentRegistrationSecretValidator`, `AgentSecretGenerator`, `MachineIdGenerator`, `PasswordEncoder`, `RegistrationTagAssignmentService`, `InstalledAgentService`, `AgentRegistrationToolInstallationService`, `AgentRegistrationProcessor` +| `registerAgent_WithNewMachine_ReturnsCredentials` | Verifies full registration flow: ID generation, OAuth client creation, machine persistence, and credential response | +| `registerAgent_WithExistingMachine_ThrowsException` | Ensures duplicate machine IDs throw `IllegalStateException` and prevent any saves | +| `registerAgent_WithTags_AssignsTagsToDevice` | Confirms tag assignment is invoked with correct tag inputs when tags are present | +| `registerAgent_WithoutTags_DoesNotAssignTags` | Confirms `assignTags` is still called with `null` when no tags are provided | +| `reinstall_WithValidKeyAndSecret_OverwritesMachineAndReturnsExistingCreds` | Validates reinstall path: reuses existing OAuth credentials, overwrites machine data, resets status to `PENDING`, and enforces validation ordering | ## Usage Example ```java -// Simulate a new agent registration with tags -AgentRegistrationRequest request = new AgentRegistrationRequest(); -request.setHostname("workstation-01"); -request.setIp("10.0.0.5"); -request.setMacAddress("AA:BB:CC:DD:EE:FF"); -request.setOsUuid("os-uuid-xyz"); -request.setAgentVersion("2.1.0"); -request.setTags(List.of( - AgentRegistrationTagInput.builder() - .key("site") - .values(List.of("NEW_YORK")) - .build() -)); - -// Expected: response contains machineId, clientId ("agent_"), and clientSecret -AgentRegistrationResponse response = agentRegistrationService.register("initial-key", request); +// Typical mock setup pattern used across tests +when(machineIdGenerator.generate()).thenReturn(MACHINE_ID); +when(oauthClientRepository.existsByMachineId(MACHINE_ID)).thenReturn(false); +when(agentSecretGenerator.generate()).thenReturn(CLIENT_SECRET); +when(passwordEncoder.encode(CLIENT_SECRET)).thenReturn("encoded-secret"); +when(oauthClientRepository.save(any())).thenAnswer(i -> i.getArguments()[0]); + +AgentRegistrationResponse response = agentRegistrationService.register(INITIAL_KEY, request); + +// Assert credentials returned correctly +assertEquals(MACHINE_ID, response.getMachineId()); +assertEquals("agent_" + MACHINE_ID, response.getClientId()); + +// Assert validation ordering on reinstall (initial key before client secret) +InOrder inOrder = inOrder(agentRegistrationSecretValidator, clientSecretValidator); +inOrder.verify(agentRegistrationSecretValidator).validate(INITIAL_KEY); +inOrder.verify(clientSecretValidator).validate(existingClient, CLIENT_SECRET); ``` -**Key assertions validated across tests:** -- `clientId` is always prefixed as `agent_` -- Saved `OAuthClient` uses `client_credentials` grant type and `AGENT` role -- Saved `Machine` defaults to `DeviceStatus.PENDING` with a non-null `lastSeen` timestamp -- Secret validator is always invoked before any persistence \ No newline at end of file +**Notable patterns:** Uses `ArgumentCaptor` to assert saved entity field values directly, `InOrder` to enforce validation sequence, and `never()` to assert no re-creation of IDs or OAuth clients during reinstall. \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/service/agentregistration/.OrganizationIdResolverTest.md b/openframe-client-core/src/test/java/com/openframe/client/service/agentregistration/.OrganizationIdResolverTest.md new file mode 100644 index 000000000..d1c9ddea8 --- /dev/null +++ b/openframe-client-core/src/test/java/com/openframe/client/service/agentregistration/.OrganizationIdResolverTest.md @@ -0,0 +1,45 @@ + +Unit tests for `OrganizationIdResolver`, verifying organization ID resolution logic including fallback-to-default and error handling behaviors. + +## Key Components + +| Element | Description | +|---------|-------------| +| `resolve_WithExistingRequestedOrg_ReturnsIt` | Confirms a valid, known org ID is returned directly without querying the default | +| `resolve_WithUnknownRequestedOrg_FallsBackToDefault` | Ensures an unrecognized org ID falls back to the default organization | +| `resolve_WithBlank_ReturnsDefaultWithoutLookup` | Validates that blank/whitespace input skips the org lookup and returns the default | +| `resolve_WhenNoDefaultOrganization_Throws` | Asserts an `IllegalStateException` is thrown when no default organization exists | +| `organization()` | Helper method that constructs a minimal `Organization` document for test fixtures | + +## Usage Example + +```java +// Scenario: requested org exists β†’ returned directly +when(organizationService.getOrganizationByOrganizationId("org-1")) + .thenReturn(Optional.of(organization("org-1"))); + +String result = resolver.resolve("org-1"); // β†’ "org-1" +verify(organizationService, never()).getDefaultOrganization(); + +// Scenario: blank input β†’ default org returned, no ID lookup performed +when(organizationService.getDefaultOrganization()) + .thenReturn(Optional.of(organization("default-uuid"))); + +String result = resolver.resolve(" "); // β†’ "default-uuid" +verify(organizationService, never()).getOrganizationByOrganizationId(any()); + +// Scenario: no default configured β†’ throws +when(organizationService.getDefaultOrganization()).thenReturn(Optional.empty()); +assertThrows(IllegalStateException.class, () -> resolver.resolve(null)); +``` + +## Coverage Summary + +```text +Input | Org Found? | Default Found? | Outcome +-----------------------|------------|----------------|------------------------- +Valid org ID | βœ… Yes | Not queried | Returns requested org ID +Unknown org ID | ❌ No | βœ… Yes | Returns default org ID +Blank / whitespace | Not queried| βœ… Yes | Returns default org ID +Null / blank | Not queried| ❌ No | Throws IllegalStateException +``` \ No newline at end of file diff --git a/openframe-client-core/src/test/java/com/openframe/client/service/validator/.ClientSecretValidatorTest.md b/openframe-client-core/src/test/java/com/openframe/client/service/validator/.ClientSecretValidatorTest.md new file mode 100644 index 000000000..f6866090d --- /dev/null +++ b/openframe-client-core/src/test/java/com/openframe/client/service/validator/.ClientSecretValidatorTest.md @@ -0,0 +1,43 @@ + +Unit tests for `ClientSecretValidator`, verifying secret validation logic including blank input rejection and BCrypt-based secret matching against an `OAuthClient`. + +## Key Components + +| Component | Role | +|-----------|------| +| `ClientSecretValidator` | Class under test β€” validates raw client secrets against stored encoded values | +| `PasswordEncoder` | Mocked Spring Security component used to compare raw vs. encoded secrets | +| `InvalidClientSecretException` | Expected exception carrying an `ErrorCode` on validation failure | +| `ErrorCode.CLIENT_SECRET_EMPTY` | Error code asserted when the provided secret is blank/whitespace | +| `ErrorCode.CLIENT_SECRET_INVALID` | Error code asserted when the secret does not match the stored hash | + +## Test Cases + +| Test | Scenario | Expected Outcome | +|------|----------|-----------------| +| `validate_WithMatchingSecret_Passes` | Raw secret matches encoded secret via `PasswordEncoder` | No exception thrown | +| `validate_WithBlankSecret_ThrowsEmpty` | Secret is blank/whitespace | `InvalidClientSecretException` with `CLIENT_SECRET_EMPTY`; `PasswordEncoder` never invoked | +| `validate_WithWrongSecret_ThrowsInvalid` | Raw secret does not match encoded secret | `InvalidClientSecretException` with `CLIENT_SECRET_INVALID` | + +## Usage Example + +```java +// Arrange +OAuthClient client = new OAuthClient(); +client.setClientSecret("$2a$10$encodedHash..."); + +// Blank secret β€” fast-fail before encoder is called +assertThrows(InvalidClientSecretException.class, + () -> validator.validate(client, " ")); + +// Wrong secret β€” encoder consulted, mismatch detected +when(passwordEncoder.matches("wrong", "$2a$10$encodedHash...")).thenReturn(false); +assertThrows(InvalidClientSecretException.class, + () -> validator.validate(client, "wrong")); + +// Correct secret β€” passes silently +when(passwordEncoder.matches("correct", "$2a$10$encodedHash...")).thenReturn(true); +assertDoesNotThrow(() -> validator.validate(client, "correct")); +``` + +> **Note:** Blank secret validation short-circuits before `PasswordEncoder.matches()` is called β€” verified explicitly with `verify(passwordEncoder, never()).matches(any(), any())` β€” preventing unnecessary crypto operations on empty input. \ No newline at end of file diff --git a/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md b/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md index 9c3540bb2..813b0ae7f 100644 --- a/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md +++ b/openframe-data-device-aspect/src/main/java/com/openframe/data/service/impl/.MachineTagEventServiceImpl.md @@ -1,18 +1,30 @@ - -Handles machine tag lifecycle events by processing save and delete operations for machines, tag assignments, and tags, then publishing enriched `MachinePinotMessage` payloads to a Kafka topic for downstream analytics. + +Handles machine tag event processing by listening for save/delete events on `Machine`, `Tag`, and `TagAssignment` entities, then publishing enriched `MachinePinotMessage` payloads to a Kafka topic for downstream Pinot ingestion. ## Key Components | Method | Description | -|---|---| -| `processMachineSave` | Publishes a single machine event to Kafka with its current tag state | +|--------|-------------| +| `processMachineSave` | Processes a single machine save event and publishes to Kafka | | `processMachineSaveAll` | Iterates and publishes Kafka events for a batch of machines | -| `processTagAssignmentSave` | Handles a single `DEVICE`-type tag assignment, publishing the updated machine state | -| `processTagAssignmentSaveAll` | Batch processes tag assignments, deduplicating by `entityId` to avoid redundant messages | -| `processTagSave` | Propagates a tag update to all machines that carry that tag | -| `processTagSaveAll` | Batch processes tag updates across all affected machines | -| `processTagAssignmentDelete` | Rebuilds and publishes a machine's tag state after a specific tag assignment is removed | -| `processTagAssignmentDeleteByTagId` | Bulk recomputes and publishes updated state for all machines affected by a tag deletion | +| `processTagAssignmentSave` | Handles a single tag assignment save; skips non-`DEVICE` entity types | +| `processTagAssignmentSaveAll` | Batch processes tag assignments, deduplicating by `entityId` | +| `processTagSave` | Propagates a tag update to all machines that carry the tag | +| `processTagSaveAll` | Batch version of `processTagSave` | +| `processTagAssignmentDelete` | Rebuilds and publishes a machine message excluding the deleted tag assignment | +| `processTagAssignmentDeleteByTagId` | Bulk delete handler β€” finds all affected machines and republishes each | + +## Conditional Activation + +The service is only registered when the property `openframe.device.aspect.enabled=true` (defaults to `true` if missing): + +```java +@ConditionalOnProperty( + name = "openframe.device.aspect.enabled", + havingValue = "true", + matchIfMissing = true +) +``` ## Usage Example @@ -21,15 +33,28 @@ Handles machine tag lifecycle events by processing save and delete operations fo @Autowired private MachineTagEventService machineTagEventService; -// Publish a machine event after save -Machine machine = machineRepository.save(newMachine); +// On machine upsert machineTagEventService.processMachineSave(machine); -// Publish updated state after removing a tag from a machine +// On tag assignment removal machineTagEventService.processTagAssignmentDelete(machineId, tagId); -// Propagate tag metadata changes to all machines using that tag -machineTagEventService.processTagSave(updatedTag); +// On tag entity deletion (cascades to all assigned machines) +machineTagEventService.processTagAssignmentDeleteByTagId(tagId); +``` + +## Configuration + +```yaml +openframe: + device: + aspect: + enabled: true + oss-tenant: + kafka: + topics: + outbound: + devices-topic: machines-events ``` -> **Note:** This service is conditionally enabled via `openframe.device.aspect.enabled=true` (defaults to `true`). It is typically invoked from AOP aspects or repository event listeners rather than called directly. \ No newline at end of file +> All public methods are wrapped in try/catch blocks β€” errors are logged but do not propagate, ensuring Kafka publishing failures do not interrupt the caller's transaction. \ No newline at end of file diff --git a/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md b/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md index ae0551fdc..9e26e9f49 100644 --- a/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md +++ b/openframe-data-kafka/src/main/java/com/openframe/kafka/producer/.GenericKafkaProducer.md @@ -1,16 +1,28 @@ - -Abstract base class for Kafka message producers, providing both async (fire-and-forget) and synchronous (blocking) send operations with structured error classification, logging, and exception handling. + +Abstract base class providing async and synchronous Kafka message production with structured error classification, payload logging, and exception routing for retryable vs. non-retryable failures. ## Key Components -| Member | Description | -|---|---| -| `sendAsync()` | Non-blocking send that returns a `CompletableFuture`; logs success/failure via `whenComplete` without altering the future's state | -| `sendAndAwait()` | Blocking send that joins the async future and classifies exceptions into `NonRetryableKafkaException` (fatal) or `TransientKafkaSendException` (retryable) | -| `handleSendFailure()` | Pattern-matched switch that logs authorization errors, oversized records, timeouts, retriable failures, and generic errors with appropriate log levels | -| `validateTopic()` | Guards against null/blank topic names, throwing `NonRetryableKafkaException` immediately | -| `redact()` | Masks message keys in logs, exposing only the first 3 characters to avoid leaking sensitive data | -| `abbreviate()` | Truncates payload strings to 500 characters in log output | +| Member | Type | Description | +|--------|------|-------------| +| `sendAsync()` | `public` | Fire-and-forget send returning a `CompletableFuture`; logs success/failure via `whenComplete` without altering future state | +| `sendAndAwait()` | `public` | Blocking send that joins the future and classifies `CompletionException` causes into `NonRetryableKafkaException` or `TransientKafkaSendException` | +| `validateTopic()` | `private` | Guards against null/blank topic names, throwing `NonRetryableKafkaException` immediately | +| `handleSendFailure()` | `private` | Pattern-matches exception types (`TopicAuthorizationException`, `RecordTooLargeException`, `TimeoutException`, `RetriableException`) for structured log output | +| `redact()` | `private static` | Masks message keys in logs β€” keys ≀ 6 chars become `***`, longer keys show first 3 chars + ellipsis | +| `abbreviate()` | `private static` | Truncates payload strings to 500 chars for safe log output | + +## Error Classification + +```text +CompletionException cause +β”œβ”€β”€ RecordTooLargeException β†’ NonRetryableKafkaException (fail fast) +β”œβ”€β”€ AuthorizationException β†’ NonRetryableKafkaException (fail fast) +β”œβ”€β”€ SerializationException β†’ NonRetryableKafkaException (fail fast) +β”œβ”€β”€ InvalidTopicException β†’ NonRetryableKafkaException (fail fast) +β”œβ”€β”€ UnknownTopicOrPartitionException β†’ TransientKafkaSendException (auto-create pending) +└── All others β†’ TransientKafkaSendException (retryable) +``` ## Usage Example @@ -25,26 +37,16 @@ public class OrderEventProducer extends GenericKafkaProducer { } // Fire-and-forget - public CompletableFuture> publishAsync(OrderEvent event) { - return sendAsync(TOPIC, event.getOrderId(), event); + public void publishAsync(OrderCreatedEvent event) { + sendAsync(TOPIC, event.getOrderId(), event); } - // Blocking β€” suitable for use with @Retryable + // Blocking β€” pair with @Retryable(TransientKafkaSendException.class) @Retryable(retryFor = TransientKafkaSendException.class, maxAttempts = 3) - public void publishAndAwait(OrderEvent event) { + public void publishSync(OrderCreatedEvent event) { sendAndAwait(TOPIC, event.getOrderId(), event); } } ``` -## Exception Classification - -```text -CompletionException cause -β”œβ”€β”€ RecordTooLargeException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ AuthorizationException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ SerializationException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ InvalidTopicException β†’ NonRetryableKafkaException (fatal, do not retry) -β”œβ”€β”€ UnknownTopicOrPartitionException β†’ TransientKafkaSendException (topic may be auto-creating) -└── Everything else β†’ TransientKafkaSendException (retryable) -``` \ No newline at end of file +> **Note:** `sendAsync()` callbacks are logging-only β€” never throw inside `whenComplete` as it cannot change the returned future's state. Use `sendAndAwait()` with Spring Retry for controlled retry semantics. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/.TenantScoped.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/.TenantScoped.md new file mode 100644 index 000000000..f9a631077 --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/.TenantScoped.md @@ -0,0 +1,34 @@ + +Marker interface that enforces tenant isolation on domain objects within the OpenFrame multi-tenant data layer. Any document requiring tenant-scoped access control should implement this interface. + +## Key Components + +| Member | Type | Description | +|--------|------|-------------| +| `getTenantId()` | `String` | Returns the tenant identifier associated with the document | +| `setTenantId(String)` | `void` | Assigns a tenant identifier to the document | + +## Usage Example + +```java +@Document(collection = "devices") +public class Device implements TenantScoped { + + @Id + private String id; + + private String tenantId; + + @Override + public String getTenantId() { + return tenantId; + } + + @Override + public void setTenantId(String tenantId) { + this.tenantId = tenantId; + } +} +``` + +Repositories and query filters can then rely on `TenantScoped` to automatically scope queries to the current tenant context, preventing cross-tenant data leakage across the OpenFrame platform. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md index 72f96fed2..ac98fdcff 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/agent/.AgentRegistrationSecret.md @@ -1,33 +1,27 @@ - -A MongoDB document entity representing a registration secret used to authenticate and register agents within the OpenFrame platform. + +MongoDB document entity representing a secret key used for agent registration, scoped to a specific tenant. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | Primary identifier (`@Id`) for the document | -| `secretKey` | `String` | Unique indexed secret key used for agent registration | -| `createdAt` | `Instant` | Timestamp recording when the secret was created | -| `active` | `boolean` | Flag indicating whether the registration secret is currently valid | - -- **Collection:** `agent_registration_secrets` -- **Annotations:** Lombok `@Data` auto-generates getters, setters, `equals`, `hashCode`, and `toString` +| Element | Description | +|---|---| +| `@Document` | Maps to the `agent_registration_secrets` MongoDB collection | +| `@CompoundIndex` | Unique index on `(tenantId, secretKey)` to prevent duplicate secrets per tenant | +| `TenantScoped` | Interface ensuring all queries are tenant-isolated | +| `secretKey` | The registration secret value used to authenticate agent enrollment | +| `active` | Flag to enable/disable a secret without deletion | +| `createdAt` | Timestamp (`Instant`) recording when the secret was issued | ## Usage Example ```java -// Creating a new agent registration secret AgentRegistrationSecret secret = new AgentRegistrationSecret(); +secret.setTenantId("tenant-abc"); secret.setSecretKey(UUID.randomUUID().toString()); secret.setCreatedAt(Instant.now()); secret.setActive(true); -// Persisting via a Spring Data MongoDB repository -AgentRegistrationSecret saved = agentRegistrationSecretRepository.save(secret); - -// Deactivating an existing secret -saved.setActive(false); -agentRegistrationSecretRepository.save(saved); +agentRegistrationSecretRepository.save(secret); ``` -> **Note:** The `secretKey` field is enforced as unique at the database index level, preventing duplicate registration secrets across agents in the OpenFrame platform. \ No newline at end of file +> Compound uniqueness on `(tenantId, secretKey)` ensures no two active secrets share the same key within a tenant, preventing cross-tenant collisions. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md index d1bd1d4a8..da3951eeb 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKey.md @@ -1,42 +1,45 @@ - -MongoDB document entity representing an API key stored in the `api_keys` collection, including metadata, permission scopes, and expiry logic. + +MongoDB document model representing an API key entity within the OpenFrame platform, scoped to a specific tenant and user. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `keyId` | `String` | MongoDB `@Id` β€” primary identifier for the key | -| `hashedKey` | `String` | Bcrypt/hashed representation of the raw API key | -| `name` / `description` | `String` | Human-readable label and description | -| `userId` | `String` | Indexed owner reference linking the key to a user | -| `scopes` | `List` | Fine-grained permission scopes *(TODO: enforced)* | -| `roles` | `List` | Role assignments *(TODO: enforced)* | +| `keyId` | `String` (`@Id`) | Primary identifier for the API key document | +| `tenantId` | `String` (`@Indexed`) | Tenant scope reference β€” implements `TenantScoped` | +| `hashedKey` | `String` | Stored hash of the raw API key secret | +| `userId` | `String` (`@Indexed`) | Owning user reference | +| `scopes` / `roles` | `List` | Permission model (⚠️ TODO β€” not yet enforced) | | `enabled` | `boolean` | Soft-disable flag, defaults to `true` | -| `createdAt` / `updatedAt` / `expiresAt` | `Instant` | Lifecycle timestamps | +| `expiresAt` | `Instant` | Optional expiry timestamp | | `isExpired()` | `boolean` | Returns `true` if `expiresAt` is set and in the past | -| `isActive()` | `boolean` | Returns `true` if `enabled && !isExpired()` | +| `isActive()` | `boolean` | Returns `true` if `enabled == true` AND `!isExpired()` | ## Usage Example ```java -// Build a new API key document +// Create a new API key document ApiKey apiKey = ApiKey.builder() - .keyId(UUID.randomUUID().toString()) - .hashedKey(hashService.hash(rawKey)) + .tenantId("tenant-001") + .userId("user-abc") .name("CI Integration Key") - .description("Used by the CI pipeline") - .userId("user_abc123") - .scopes(List.of("read:tickets", "write:tickets")) - .roles(List.of("technician")) + .description("Used by GitHub Actions pipeline") + .hashedKey(hashService.hash(rawKey)) + .scopes(List.of("read:devices", "write:tickets")) .enabled(true) .createdAt(Instant.now()) - .expiresAt(Instant.now().plus(Duration.ofDays(90))) + .expiresAt(Instant.now().plus(90, ChronoUnit.DAYS)) .build(); -// Check validity before processing a request +// Validate before processing a request if (!apiKey.isActive()) { throw new UnauthorizedException("API key is disabled or expired"); } ``` -> **Note:** `scopes` and `roles` fields are defined but not yet enforced β€” permission checks are marked as TODO and will be wired into the authorization layer in a future release. \ No newline at end of file +## Notes + +- Stored in the `api_keys` MongoDB collection +- Raw key secrets are **never persisted** β€” only the `hashedKey` is stored +- `scopes` and `roles` fields are present but not yet enforced (marked TODO) +- `tenantId` and `userId` are both indexed for efficient lookup \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md index bea8c95e2..17b77c45d 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/apikey/.ApiKeyStats.md @@ -1,35 +1,38 @@ - -MongoDB document entity for tracking API key usage statistics, storing request counts and last activity timestamp in the `api_key_stats` collection. + +MongoDB document entity for tracking API key usage statistics per tenant, including request counts and last usage timestamp. ## Key Components | Field | Type | Description | |-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `totalRequests` | `Long` | Cumulative count of all requests made with the API key | -| `successfulRequests` | `Long` | Count of requests that completed successfully | -| `failedRequests` | `Long` | Count of requests that resulted in errors | +| `id` | `String` | MongoDB document ID (`@Id`) | +| `tenantId` | `String` | Indexed tenant identifier (implements `TenantScoped`) | +| `totalRequests` | `Long` | Total number of API requests made | +| `successfulRequests` | `Long` | Count of successful requests | +| `failedRequests` | `Long` | Count of failed requests | | `lastUsed` | `LocalDateTime` | Timestamp of the most recent API key usage | +**Collection:** `api_key_stats` + ## Usage Example ```java -// Build a new stats document +// Build a new stats document for a tenant's API key ApiKeyStats stats = ApiKeyStats.builder() - .totalRequests(150L) - .successfulRequests(142L) - .failedRequests(8L) + .tenantId("tenant-abc-123") + .totalRequests(1500L) + .successfulRequests(1480L) + .failedRequests(20L) .lastUsed(LocalDateTime.now()) .build(); -// Update on each API call +// Access tenant-scoped query support via TenantScoped +String tenantId = stats.getTenantId(); // "tenant-abc-123" + +// Update request counters stats.setTotalRequests(stats.getTotalRequests() + 1); -stats.setSuccessfulRequests(stats.getSuccessfulRequests() + 1); +stats.setFailedRequests(stats.getFailedRequests() + 1); stats.setLastUsed(LocalDateTime.now()); ``` -## Notes - -- Lombok annotations (`@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor`) auto-generate boilerplate (getters, setters, builder pattern, constructors). -- Persisted via Spring Data MongoDB; the `@Document` annotation maps this class to the `api_key_stats` collection. -- Designed to complement an API key entity, providing a dedicated stats document to avoid bloating the primary key record. \ No newline at end of file +> **Note:** The `tenantId` field is indexed (`@Indexed`) for efficient per-tenant queries across the `api_key_stats` collection. Implements `TenantScoped` to ensure consistent multi-tenancy enforcement across the OpenFrame platform. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md index 7db0e2778..47e05fc68 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/assignment/.ItemAssignment.md @@ -1,42 +1,42 @@ - -MongoDB document model representing the assignment relationship between items (e.g., scripts, templates) and their targets (e.g., users, devices), stored in the `item_assignments` collection. + +MongoDB document entity representing the assignment of an item (e.g., a configuration or resource) to a specific target within a tenant-scoped context. ## Key Components -| Member | Type | Description | -|---|---|---| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `itemId` | `String` | Reference to the assigned item | -| `itemType` | `AssignmentItemType` | Enum classifying the item (e.g., script, template) | -| `targetType` | `AssignmentTargetType` | Enum classifying the assignment target (e.g., user, device) | -| `targetId` | `String` | Reference to the target entity (indexed) | -| `displayName` | `String` | Human-readable label for the assignment | -| `createdAt` | `Instant` | Auto-populated creation timestamp via `@CreatedDate` | +| Element | Description | +|---|---| +| `ItemAssignment` | Main document class mapped to the `item_assignments` MongoDB collection | +| `TenantScoped` | Interface implemented to enforce multi-tenancy isolation via `tenantId` | +| `AssignmentItemType` | Enum defining the type of item being assigned | +| `AssignmentTargetType` | Enum defining the target receiving the assignment | +| `createdAt` | Auto-populated timestamp via Spring's `@CreatedDate` | ### Indexes -| Index Name | Fields | Unique | +| Index | Fields | Purpose | |---|---|---| -| `item_target_unique` | `itemId`, `targetType`, `targetId` | βœ… Yes β€” prevents duplicate assignments | -| `target_lookup` | `targetType`, `targetId` | ❌ No β€” optimizes queries by target | -| *(single field)* | `targetId` | ❌ No β€” supports direct target lookups | +| `item_target_unique` | `itemId`, `targetType`, `targetId` | Prevents duplicate assignments | +| `target_lookup` | `targetType`, `targetId` | Optimizes lookups by target | +| Single field | `tenantId`, `targetId` | Fast tenant and target queries | ## Usage Example ```java -// Building a new item assignment +// Build and persist a new item assignment ItemAssignment assignment = ItemAssignment.builder() - .itemId("script-abc123") + .tenantId("tenant-abc") + .itemId("script-001") .itemType(AssignmentItemType.SCRIPT) .targetType(AssignmentTargetType.DEVICE) - .targetId("device-xyz789") - .displayName("Patch Tuesday Script β†’ Workstation 42") + .targetId("device-xyz") + .displayName("Startup Script") .build(); -// Querying assignments for a specific target -List assignments = mongoTemplate.find( - Query.query(Criteria.where("targetType").is(AssignmentTargetType.DEVICE) - .and("targetId").is("device-xyz789")), - ItemAssignment.class -); -``` \ No newline at end of file +itemAssignmentRepository.save(assignment); + +// Query assignments by target +List assignments = itemAssignmentRepository + .findByTargetTypeAndTargetId(AssignmentTargetType.DEVICE, "device-xyz"); +``` + +> **Note:** The compound unique index on `itemId + targetType + targetId` ensures an item cannot be assigned to the same target more than once. Duplicate save attempts will throw a `DuplicateKeyException`. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md index d624d9f79..d0332d90f 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthInvitation.md @@ -1,30 +1,25 @@ - -Represents a MongoDB document for authentication-based invitations, extending the base `Invitation` class with tenant-scoped uniqueness enforcement. + +MongoDB document entity representing an authentication invitation scoped to a tenant, combining tenant-based indexing with user invitation data. ## Key Components -| Component | Description | -|-----------|-------------| -| `AuthInvitation` | MongoDB document class extending `Invitation` for auth-specific invite records | -| `tenantId` | Indexed field scoping invitations to a specific tenant | -| `@CompoundIndex` | Enforces unique `tenantId + email` combinations where `tenantId` exists | +- **`AuthInvitation`** β€” Extends `Invitation` and `TenantScoped`, inheriting invitation fields alongside tenant context +- **Compound Index** β€” `{'tenantId': 1, 'email': 1}` ensures efficient tenant-scoped lookups by email +- **Lombok Annotations** β€” `@Data`, `@SuperBuilder`, `@NoArgsConstructor`, and `@EqualsAndHashCode(callSuper = true)` handle boilerplate generation while preserving parent class equality logic ## Usage Example ```java -// Build a new auth invitation for a specific tenant -AuthInvitation invitation = AuthInvitation.builder() - .tenantId("tenant-abc-123") - .email("user@example.com") - .build(); +AuthInvitation invite = AuthInvitation.builder() + .tenantId("tenant-123") + .email("user@example.com") + .build(); -// Compound index ensures no duplicate email per tenant -// { tenantId: "tenant-abc-123", email: "user@example.com" } β†’ unique -``` +authInvitationRepository.save(invite); -## Notes +// Lookup by tenant + email (uses compound index) +Optional found = authInvitationRepository + .findByTenantIdAndEmail("tenant-123", "user@example.com"); +``` -- Inherits base fields (e.g., `email`, invite metadata) from `Invitation` -- The `@CompoundIndex` uses a **partial filter** (`tenantId: { $exists: true }`), meaning global invitations without a `tenantId` are excluded from the uniqueness constraint -- `@Indexed` on `tenantId` enables efficient tenant-scoped queries -- Uses Lombok `@SuperBuilder` to support builder inheritance from the parent `Invitation` class \ No newline at end of file +> **Note:** The compound index on `tenantId` + `email` enforces fast scoped queries and should align with your repository query patterns to avoid full collection scans. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md index 8c5c82c6b..bc21a9ee2 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/auth/.AuthUser.md @@ -1,47 +1,41 @@ - -MongoDB document model representing an authenticated user in OpenFrame's multi-tenant Authorization Server, extending the base `User` document with auth-specific fields and tenant isolation. + +Represents an authenticated user in OpenFrame's multi-tenant Authorization Server, extending the base `User` document with authentication-specific fields such as password credentials, OAuth provider info, and login metadata. ## Key Components -| Member | Type | Description | -|--------|------|-------------| -| `tenantId` | `String` | Indexed tenant identifier; combined with `email` as a unique compound index | -| `passwordHash` | `String` | Hashed password for `LOCAL` authentication | -| `loginProvider` | `String` | Authentication provider (e.g., `LOCAL`, `GOOGLE`) | -| `externalUserId` | `String` | User ID from the external OAuth/SSO provider | -| `lastLogin` | `Instant` | Timestamp of the user's most recent login | -| `imageUrl` | `String` | Cached profile picture URL synced from tenant cluster via `USER_UPDATED` events | -| `getFullName()` | `String` | Derives display name from first/last name, falling back to email | +| Field / Method | Description | +|---|---| +| `passwordHash` | Hashed password for local authentication | +| `loginProvider` | Auth strategy identifier (e.g., `LOCAL`, `GOOGLE`) | +| `externalUserId` | User ID from an external OAuth provider | +| `lastLogin` | Timestamp of the most recent login | +| `imageUrl` | Cached profile picture URL synced via `USER_UPDATED` events for SSO `picture` claim propagation | +| `getFullName()` | Resolves display name from first/last name, falling back to email | + +**Compound Index:** Enforces uniqueness on `{tenantId, email}`, supporting domain-based multi-tenancy. ## Usage Example ```java -// Build a new local AuthUser for a specific tenant AuthUser authUser = AuthUser.builder() - .email("jane@acme.com") - .firstName("Jane") - .lastName("Doe") - .tenantId("acme-corp") - .passwordHash(hashService.hash("s3cur3P@ss")) - .loginProvider("LOCAL") - .build(); - -// Build an SSO user (e.g., Google) -AuthUser ssoUser = AuthUser.builder() - .email("john@acme.com") - .tenantId("acme-corp") - .loginProvider("GOOGLE") - .externalUserId("google-uid-9876") - .imageUrl("https://lh3.googleusercontent.com/photo.jpg") - .build(); - -// Resolve display name with fallback chain + .email("jane@acme.com") + .firstName("Jane") + .lastName("Doe") + .tenantId("acme-corp") + .loginProvider("GOOGLE") + .externalUserId("google-uid-12345") + .lastLogin(Instant.now()) + .imageUrl("https://cdn.example.com/avatar.png") + .build(); + +// Resolves to "Jane Doe" String displayName = authUser.getFullName(); -// β†’ "Jane Doe" -``` -## Notes +// Falls back to email if no name is set +AuthUser noName = AuthUser.builder() + .email("unknown@acme.com") + .build(); +String fallback = noName.getFullName(); // returns "unknown@acme.com" +``` -- The compound index on `{tenantId, email}` enforces per-tenant email uniqueness with a partial filter (`$exists: true`), allowing global users without a `tenantId`. -- `imageUrl` is a cache β€” a `null` value indicates no picture has been synced yet, not that the tenant cluster lacks one. -- Inherits base fields (`email`, `firstName`, `lastName`, etc.) from the `User` superclass via `@SuperBuilder`. \ No newline at end of file +> **Note:** `imageUrl` is a cache synced from the tenant cluster β€” a `null` value means the picture is unknown to the auth server, not that the user has no profile image. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md index f133e40f4..a2d39a145 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/clientconfiguration/.OpenFrameClientConfiguration.md @@ -1,39 +1,33 @@ - -MongoDB document entity representing a versioned OpenFrame client configuration record, stored in the `openframe_client_configuration` collection. + +Represents a MongoDB document storing per-tenant OpenFrame client configuration, including versioning, download settings, and publish state lifecycle metadata. ## Key Components | Field | Type | Description | -|---|---|---| +|-------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | +| `tenantId` | `String` | Unique tenant identifier (`@Indexed(unique = true)`) | | `documentVersion` | `Long` | Optimistic locking version (`@Version`) | -| `version` | `String` | Client configuration version label | -| `downloadConfiguration` | `List` | List of download configuration entries | -| `createdAt` | `LocalDateTime` | Auto-populated creation timestamp (`@CreatedDate`) | -| `updatedAt` | `LocalDateTime` | Auto-populated last modified timestamp (`@LastModifiedDate`) | -| `publishState` | `PublishState` | Current publish state of the configuration | +| `version` | `String` | Client configuration version string | +| `downloadConfiguration` | `List` | List of download configuration entries for the client | +| `publishState` | `PublishState` | Current publish lifecycle state of the configuration | +| `createdAt` / `updatedAt` | `LocalDateTime` | Auto-managed audit timestamps | + +Implements `TenantScoped` to enforce tenant isolation across the platform. ## Usage Example ```java -// Build and save a new client configuration OpenFrameClientConfiguration config = OpenFrameClientConfiguration.builder() + .tenantId("tenant-abc-123") .version("1.0.0") - .publishState(PublishState.DRAFT) .downloadConfiguration(List.of( - DownloadConfiguration.builder() - .platform("windows") - .url("https://downloads.openframe.ai/client/1.0.0/win") - .build() + new DownloadConfiguration(/* ... */) )) + .publishState(PublishState.DRAFT) .build(); -mongoTemplate.save(config); - -// Retrieve and inspect state -OpenFrameClientConfiguration saved = mongoTemplate.findById(id, OpenFrameClientConfiguration.class); -System.out.println(saved.getPublishState()); // DRAFT -System.out.println(saved.getDocumentVersion()); // 1 (auto-incremented) +openFrameClientConfigurationRepository.save(config); ``` -> **Note:** This entity relies on Spring Data MongoDB auditing (`@CreatedDate`, `@LastModifiedDate`) being enabled via `@EnableMongoAuditing` in the application context. Optimistic concurrency control is enforced through the `@Version` field. \ No newline at end of file +> **Note:** `tenantId` has a unique index β€” only one active configuration document is permitted per tenant. Optimistic locking via `documentVersion` prevents concurrent write conflicts. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md index f7d7d4ace..a018a2468 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/connector/.ConnectorAlert.md @@ -1,38 +1,41 @@ - -A MongoDB document model representing an alert generated by a connector, tracking error state, retry attempts, and resolution status within the OpenFrame platform. + +MongoDB document entity representing an alert generated by a connector, tracking error state, retry attempts, and resolution status within a tenant-scoped context. ## Key Components +| Element | Description | +|---|---| +| `ConnectorAlert` | Main document class mapped to the `connector_alerts` MongoDB collection | +| `TenantScoped` | Interface implemented to enforce multi-tenant data isolation | +| `ConnectorAlertType` | Enum (external) categorizing the type of connector error | +| `tenantId` | Indexed field for efficient tenant-based querying | + +## Fields + | Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | +|---|---|---| +| `id` | `String` | MongoDB document identifier | +| `tenantId` | `String` | Tenant owner (indexed) | | `connectorName` | `String` | Name of the connector that triggered the alert | -| `errorType` | `ConnectorAlertType` | Enum categorizing the alert type | -| `errorMessage` | `String` | Human-readable error description | +| `errorType` | `ConnectorAlertType` | Categorized error type | +| `errorMessage` | `String` | Human-readable error detail | | `attempts` | `int` | Number of retry attempts made | -| `createdAt` | `Instant` | Timestamp when the alert was first raised | +| `createdAt` | `Instant` | Timestamp when the alert was created | | `resolved` | `boolean` | Whether the alert has been resolved | -| `resolvedAt` | `Instant` | Timestamp when the alert was resolved | - -Stored in the `connector_alerts` MongoDB collection. Lombok annotations (`@Data`, `@Builder`, `@NoArgsConstructor`, `@AllArgsConstructor`) handle boilerplate generation. +| `resolvedAt` | `Instant` | Timestamp of resolution | ## Usage Example ```java -// Creating a new unresolved connector alert ConnectorAlert alert = ConnectorAlert.builder() - .connectorName("psa-autotask") - .errorType(ConnectorAlertType.AUTH_FAILURE) - .errorMessage("Invalid API credentials provided") + .tenantId("tenant-abc") + .connectorName("zabbix-primary") + .errorType(ConnectorAlertType.CONNECTION_FAILURE) + .errorMessage("Unable to reach Zabbix host at 10.0.0.5") .attempts(3) .createdAt(Instant.now()) .resolved(false) .build(); -// Marking an existing alert as resolved -alert.setResolved(true); -alert.setResolvedAt(Instant.now()); - -// Saving via Spring Data MongoDB repository connectorAlertRepository.save(alert); ``` \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md index 4b5795118..139d8ff65 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Device.md @@ -1,36 +1,37 @@ - -Represents a managed device entity stored in the MongoDB `devices` collection, serving as the core data model for tracking physical and virtual machines across the OpenFrame platform. + +MongoDB document entity representing a managed device within a tenant's infrastructure, storing hardware identity, operational status, and health metadata for the OpenFrame platform. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `machineId` | `String` | Foreign reference linking to the Machine entity | -| `serialNumber` | `String` | Unique hardware serial identifier | -| `model` | `String` | Device model name/number | -| `osVersion` | `String` | Operating system version string | -| `status` | `String` | Lifecycle state: `ACTIVE`, `OFFLINE`, or `MAINTENANCE` | -| `type` | `DeviceType` | Enum classifying device category (e.g., `DESKTOP`, `LAPTOP`, `SERVER`) | -| `lastCheckin` | `Instant` | UTC timestamp of the most recent agent heartbeat | -| `configuration` | `DeviceConfiguration` | Embedded configuration details | -| `health` | `DeviceHealth` | Embedded health metrics and diagnostics | +| Element | Type | Description | +|---|---|---| +| `Device` | `@Document` class | Root MongoDB entity mapped to the `devices` collection | +| `TenantScoped` | Interface | Enforces multi-tenancy isolation via `tenantId` | +| `DeviceType` | Enum | Classifies devices β€” `DESKTOP`, `LAPTOP`, `SERVER`, etc. | +| `DeviceConfiguration` | Embedded | Stores device-level configuration settings | +| `DeviceHealth` | Embedded | Tracks health metrics and diagnostics | +| `lastCheckin` | `Instant` | Timestamp of the most recent device check-in | +| `status` | `String` | Lifecycle state β€” `ACTIVE`, `OFFLINE`, `MAINTENANCE` | ## Usage Example ```java +// Create and persist a new managed device Device device = new Device(); -device.setMachineId("machine-abc123"); -device.setSerialNumber("SN-987654"); +device.setTenantId("tenant-abc"); +device.setMachineId("machine-xyz-001"); +device.setSerialNumber("SN-20240101-001"); device.setModel("Dell OptiPlex 7090"); -device.setOsVersion("Windows 11 22H2"); -device.setStatus("ACTIVE"); +device.setOsVersion("Windows 11 Pro 23H2"); device.setType(DeviceType.DESKTOP); +device.setStatus("ACTIVE"); device.setLastCheckin(Instant.now()); -device.setConfiguration(new DeviceConfiguration()); -device.setHealth(new DeviceHealth()); deviceRepository.save(device); + +// Query all active devices for a tenant +List activeDevices = deviceRepository + .findByTenantIdAndStatus("tenant-abc", "ACTIVE"); ``` -> **Note:** Lombok's `@Data` auto-generates getters, setters, `equals()`, `hashCode()`, and `toString()`. The `lastCheckin` field should be updated on every agent heartbeat to accurately reflect device availability within the Flamingo monitoring stack. \ No newline at end of file +> **Note:** `tenantId` is indexed (`@Indexed`) to optimize multi-tenant query performance across large device fleets. Always filter by `tenantId` first in repository queries to leverage this index. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md index 899082efa..04f24b3ea 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/device/.Machine.md @@ -1,34 +1,35 @@ - -Represents a managed device (machine) in the OpenFrame platform, stored in the MongoDB `machines` collection with full lifecycle, security, and compliance tracking. + +MongoDB document entity representing a managed device (machine) within a tenant's infrastructure, storing hardware identity, agent status, OS details, and security/compliance state. ## Key Components | Field | Type | Description | -|---|---|---| -| `id` | `String` | MongoDB document ID (`@Id`) | -| `machineId` | `String` | Primary auth identifier, mirrors `OAuthClient` | -| `status` | `DeviceStatus` | Current device status (indexed) | -| `organizationId` | `String` | Tenant scoping key (indexed) | +|-------|------|-------------| +| `machineId` | `String` | Primary device identifier, shared with `OAuthClient` for authentication | +| `tenantId` | `String` | Tenant scope (indexed) β€” implements `TenantScoped` | +| `organizationId` | `String` | Organizational grouping within a tenant (indexed) | +| `status` | `DeviceStatus` | Current device connectivity/operational status (indexed) | | `type` | `DeviceType` | Device category β€” workstation, server, etc. (indexed) | -| `osType` / `osVersion` | `String` | OS platform and version details | -| `securityState` | `SecurityState` | Current security posture | -| `complianceState` | `ComplianceState` | Compliance evaluation result | -| `securityAlerts` | `List` | Active security alerts on the device | -| `complianceRequirements` | `List` | Evaluated compliance rules | -| `registeredAt` | `Instant` | Auto-set on first registration (`@CreatedDate`) | -| `updatedAt` | `Instant` | Auto-updated on every save (`@LastModifiedDate`) | -| `stuckNotifiedAt` | `Instant` | Tracks if a `DEVICE_STUCK` event has been dispatched | +| `osType` | `String` | Operating system family (indexed) | +| `securityState` | `SecurityState` | Aggregated security posture | +| `complianceState` | `ComplianceState` | Current compliance evaluation result | +| `securityAlerts` | `List` | Active security findings on the device | +| `complianceRequirements` | `List` | Policy requirements evaluated against the device | +| `registeredAt` | `Instant` | First registration timestamp (`@CreatedDate`) | +| `stuckNotifiedAt` | `Instant` | Tracks whether a `DEVICE_STUCK` event has been dispatched | ## Usage Example ```java Machine machine = new Machine(); -machine.setMachineId("mch-abc123"); -machine.setOrganizationId("org-xyz"); -machine.setHostname("workstation-01"); +machine.setMachineId("machine-abc123"); +machine.setTenantId("tenant-001"); +machine.setOrganizationId("org-42"); +machine.setHostname("workstation-nyc-01"); +machine.setDisplayName("Alice's MacBook"); machine.setType(DeviceType.WORKSTATION); -machine.setOsType("WINDOWS"); -machine.setOsVersion("11"); +machine.setOsType("macOS"); +machine.setOsVersion("14.4.1"); machine.setStatus(DeviceStatus.ONLINE); machine.setLastSeen(Instant.now()); machine.setSecurityState(SecurityState.SECURE); @@ -37,4 +38,4 @@ machine.setComplianceState(ComplianceState.COMPLIANT); machineRepository.save(machine); ``` -> **Note:** `machineId` is shared with the `OAuthClient` record and serves as the primary authentication identity for agent-to-platform communication. The `stuckNotifiedAt` field remains `null` until a stuck-device event is triggered, preventing duplicate alerts. \ No newline at end of file +> `machineId` doubles as the OAuth client identifier β€” always ensure it matches the corresponding `OAuthClient` record to maintain consistent device authentication. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md index 3f44653a0..a337c3c10 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.CoreEvent.md @@ -1,38 +1,41 @@ - -MongoDB document model representing a system event within the OpenFrame core service, used to track and persist event lifecycle states. + +MongoDB document model representing a tenant-scoped event record in the OpenFrame core event system. ## Key Components | Member | Type | Description | |--------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `type` | `String` | Categorizes the event (e.g., ticket created, alert triggered) | +| `tenantId` | `String` | Indexed tenant reference (implements `TenantScoped`) | +| `type` | `String` | Event category/name identifier | | `payload` | `String` | Serialized event data (typically JSON) | | `timestamp` | `Instant` | UTC time the event was recorded | -| `userId` | `String` | Reference to the user who triggered the event | -| `status` | `EventStatus` | Current lifecycle state of the event | +| `userId` | `String` | User who triggered the event | +| `status` | `EventStatus` | Current processing state | | `EventStatus` | `enum` | `CREATED` β†’ `PROCESSING` β†’ `COMPLETED` / `FAILED` | ## Usage Example ```java +// Creating and persisting a new event CoreEvent event = new CoreEvent(); +event.setTenantId("tenant-abc"); event.setType("TICKET_CREATED"); -event.setPayload("{\"ticketId\": \"T-1042\", \"priority\": \"HIGH\"}"); +event.setPayload("{\"ticketId\": \"T-1001\", \"priority\": \"HIGH\"}"); event.setTimestamp(Instant.now()); -event.setUserId("user-abc123"); +event.setUserId("user-xyz"); event.setStatus(CoreEvent.EventStatus.CREATED); -// Persist via Spring Data MongoDB repository -eventRepository.save(event); +mongoTemplate.save(event); // persisted to "events" collection -// Query events by status -List pending = eventRepository.findByStatus(CoreEvent.EventStatus.PROCESSING); +// Checking terminal states +boolean isDone = event.getStatus() == CoreEvent.EventStatus.COMPLETED + || event.getStatus() == CoreEvent.EventStatus.FAILED; ``` ## Notes -- Stored in the `events` MongoDB collection via `@Document(collection = "events")` -- Uses Lombok `@Data` for auto-generated getters, setters, `equals`, `hashCode`, and `toString` -- `payload` is stored as a raw `String`; callers are responsible for serialization/deserialization -- Lifecycle flows: `CREATED` β†’ `PROCESSING` β†’ `COMPLETED` or `FAILED` \ No newline at end of file +- Stored in the `events` MongoDB collection via `@Document` +- `tenantId` is indexed for efficient per-tenant queries, enforcing data isolation across MSP clients +- `payload` is untyped (`String`) β€” callers are responsible for serialization/deserialization +- Lombok `@Data` generates getters, setters, `equals`, `hashCode`, and `toString` \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md index ac9b28aa0..0d846e129 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.Event.md @@ -1,37 +1,41 @@ - -MongoDB document entity representing a system event with metadata including type, payload, timestamp, and user association. + +Represents a tenant-scoped event document persisted in the MongoDB `events` collection, capturing audit/activity records with a typed payload and timestamp. ## Key Components | Field | Type | Description | |-------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `type` | `String` | Event category or action name | -| `payload` | `String` | Serialized event data | +| `tenantId` | `String` | Indexed tenant reference for multi-tenancy scoping | +| `type` | `String` | Event category/name (e.g., `TICKET_CREATED`) | +| `payload` | `String` | Serialized event data (typically JSON) | | `timestamp` | `Instant` | UTC time the event occurred | -| `userId` | `String` | ID of the user who triggered the event | +| `userId` | `String` | Originating user reference | + +Implements `TenantScoped` to enforce multi-tenant data isolation across the platform. ## Usage Example ```java -// Create a new Event using the Lombok builder Event event = Event.builder() + .tenantId("tenant-123") .type("TICKET_CREATED") - .payload("{\"ticketId\": \"T-1042\", \"priority\": \"high\"}") + .payload("{\"ticketId\":\"t-456\",\"priority\":\"HIGH\"}") .timestamp(Instant.now()) - .userId("user-abc123") + .userId("user-789") .build(); -// Save to MongoDB via a repository -eventRepository.save(event); - -// Retrieve events by type -List ticketEvents = eventRepository.findByType("TICKET_CREATED"); +mongoTemplate.save(event); ``` -## Notes +Querying by tenant: + +```java +Query query = new Query( + Criteria.where("tenantId").is("tenant-123") + .and("type").is("TICKET_CREATED") +); +List events = mongoTemplate.find(query, Event.class); +``` -- Stored in the `events` MongoDB collection via `@Document(collection = "events")` -- `@Data` generates getters, setters, `equals`, `hashCode`, and `toString` -- `@Builder` enables the fluent builder pattern for construction -- `payload` is stored as a raw `String`; consider JSON serialization for structured data \ No newline at end of file +> The `tenantId` index ensures efficient per-tenant event queries at scale, critical for Flamingo's multi-tenant MSP architecture. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md index 661578b11..d23622ac5 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/event/.ExternalApplicationEvent.md @@ -1,41 +1,40 @@ - -A MongoDB document model representing events received from external applications, stored in the `external_application_events` collection. + +Represents a MongoDB document for storing external application events scoped to a specific tenant, capturing event type, payload, metadata, and user context. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `type` | `String` | Event type classifier | -| `payload` | `String` | Raw event payload content | -| `timestamp` | `Instant` | UTC timestamp of the event | -| `userId` | `String` | Associated user identifier | -| `metadata` | `EventMetadata` | Nested metadata object | - -### `EventMetadata` (nested class) - -| Field | Type | Description | -|-------|------|-------------| -| `source` | `String` | Origin system or application name | -| `version` | `String` | Event schema or API version | -| `tags` | `Map` | Arbitrary key-value tags for filtering/routing | +| Element | Type | Description | +|---------|------|-------------| +| `ExternalApplicationEvent` | `@Document` class | Root MongoDB document mapped to `external_application_events` collection | +| `TenantScoped` | Interface | Enforces multi-tenancy by requiring a `tenantId` field | +| `EventMetadata` | Nested `@Data` class | Holds auxiliary event info: `source`, `version`, and `tags` | +| `tenantId` | `@Indexed` field | Indexed for efficient per-tenant queries | +| `timestamp` | `Instant` | UTC-based event occurrence time | +| `payload` | `String` | Raw event payload (typically serialized JSON) | ## Usage Example ```java +// Build and persist an external application event +ExternalApplicationEvent.EventMetadata metadata = new ExternalApplicationEvent.EventMetadata(); +metadata.setSource("zendesk"); +metadata.setVersion("v2"); +metadata.setTags(Map.of("env", "production", "region", "us-east-1")); + ExternalApplicationEvent event = new ExternalApplicationEvent(); +event.setTenantId("tenant-abc"); event.setType("ticket.created"); -event.setPayload("{\"ticketId\": \"T-123\", \"priority\": \"high\"}"); +event.setPayload("{\"ticketId\":\"12345\",\"priority\":\"high\"}"); +event.setUserId("user-xyz"); event.setTimestamp(Instant.now()); -event.setUserId("user-456"); - -ExternalApplicationEvent.EventMetadata metadata = new ExternalApplicationEvent.EventMetadata(); -metadata.setSource("connectwise"); -metadata.setVersion("1.0"); -metadata.setTags(Map.of("env", "production", "region", "us-east")); - event.setMetadata(metadata); + mongoTemplate.save(event); + +// Query events by tenant +Query query = new Query(Criteria.where("tenantId").is("tenant-abc") + .and("type").is("ticket.created")); +List events = mongoTemplate.find(query, ExternalApplicationEvent.class); ``` -> **Note:** Uses Lombok `@Data` for automatic generation of getters, setters, `equals`, `hashCode`, and `toString` on both the parent and nested `EventMetadata` class. \ No newline at end of file +> The `tenantId` index ensures queries are isolated per tenant, which is critical for MSP multi-tenant environments where event streams from tools like Zendesk, ConnectWise, or Autotask must remain segregated. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md index 5423924f6..9fb041808 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/installedagents/.InstalledAgent.md @@ -1,36 +1,30 @@ - -Represents a MongoDB document model for tracking agents installed on managed machines within the OpenFrame platform. + +Represents a MongoDB document entity for tracking agents installed on managed machines within a tenant's environment. ## Key Components -| Field | Type | Description | -|-------|------|-------------| +| Member | Type | Description | +|--------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `machineId` | `String` | Reference to the managed machine | -| `agentType` | `String` | Type/category of the installed agent | +| `tenantId` | `String` | Indexed tenant reference (implements `TenantScoped`) | +| `machineId` | `String` | Reference to the machine hosting the agent | +| `agentType` | `String` | Classification/type of the installed agent | | `version` | `String` | Installed agent version string | -| `createdAt` | `String` | Timestamp of agent installation record creation | -| `updatedAt` | `String` | Timestamp of last record update | - -**Annotations:** -- `@Data` β€” Lombok-generated getters, setters, `equals`, `hashCode`, and `toString` -- `@Document(collection = "installed_agents")` β€” Maps to the `installed_agents` MongoDB collection +| `createdAt` | `String` | Record creation timestamp | +| `updatedAt` | `String` | Record last-update timestamp | ## Usage Example ```java -// Create and persist a new InstalledAgent record InstalledAgent agent = new InstalledAgent(); -agent.setMachineId("machine-abc-123"); +agent.setTenantId("tenant-abc"); +agent.setMachineId("machine-001"); agent.setAgentType("monitoring"); agent.setVersion("2.1.0"); agent.setCreatedAt(Instant.now().toString()); agent.setUpdatedAt(Instant.now().toString()); installedAgentRepository.save(agent); - -// Retrieve agents by machine -List agents = installedAgentRepository.findByMachineId("machine-abc-123"); ``` -> **Note:** `createdAt` and `updatedAt` are stored as `String` types. Consider using `Instant` or `LocalDateTime` with a MongoDB converter for timezone-safe timestamp handling in production. \ No newline at end of file +> **Note:** `tenantId` is indexed for efficient multi-tenant queries. All queries against the `installed_agents` collection should scope by `tenantId` to enforce tenant isolation per the `TenantScoped` contract. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md index 062db8168..efd82ecdb 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.GenericContext.md @@ -1,37 +1,30 @@ - -A generic notification context implementation that extends `NotificationContext`, carrying an arbitrary string payload for flexible notification data transport. + +A flexible, generic implementation of `NotificationContext` that carries an arbitrary string payload and type identifier for notifications that don't require a specialized context structure. ## Key Components -| Element | Type | Description | -|---|---|---| -| `GenericContext` | Class | Concrete subclass of `NotificationContext` | -| `payload` | `String` | Arbitrary string data attached to the notification | +| Member | Type | Description | +|--------|------|-------------| +| `type` | `String` | Identifies the category or kind of the generic notification | +| `payload` | `String` | Arbitrary string content (e.g., JSON, plain text) associated with the notification | -Lombok annotations handle boilerplate automatically: - -- `@Data` β€” generates getters, setters, `equals`, `hashCode`, and `toString` -- `@SuperBuilder` β€” enables builder pattern with parent class field inclusion -- `@NoArgsConstructor` / `@AllArgsConstructor` β€” generates both constructors -- `@EqualsAndHashCode(callSuper = true)` β€” includes parent fields in equality checks -- `@ToString(callSuper = true)` β€” includes parent fields in string representation +Inherits all fields and behavior from `NotificationContext` via Lombok's `@SuperBuilder`, `@EqualsAndHashCode(callSuper = true)`, and `@ToString(callSuper = true)`. ## Usage Example ```java -// Builder pattern (includes parent NotificationContext fields) +// Build a generic notification context with a JSON payload GenericContext context = GenericContext.builder() - .payload("{\"event\":\"ticket.created\",\"ticketId\":\"123\"}") + .type("ALERT") + .payload("{\"message\": \"Disk usage exceeded 90%\"}") .build(); -// All-args constructor -GenericContext context = new GenericContext("{\"event\":\"ticket.updated\"}"); - -// Accessing the payload -String payload = context.getPayload(); +// Access fields +String notificationType = context.getType(); // "ALERT" +String notificationData = context.getPayload(); // "{\"message\": ...}" -// Updating the payload -context.setPayload("{\"event\":\"ticket.closed\"}"); +// Use with a notification service +notificationService.send(context); ``` -> **Tip:** Use `GenericContext` when no strongly-typed context subclass exists for your notification type β€” simply serialize your data to JSON and pass it as the `payload` string. \ No newline at end of file +> **Note:** `payload` is intentionally untyped β€” callers are responsible for serializing/deserializing the content. For strongly typed notifications, consider extending `NotificationContext` directly with dedicated fields instead. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md index 2d8c599e1..8dacfa668 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/notification/.NotificationContext.md @@ -1,47 +1,46 @@ - -Abstract base class for notification context payloads, enabling polymorphic JSON deserialization based on a `type` discriminator field. + +Abstract base class for polymorphic notification context objects, enabling Jackson to deserialize notification payloads into their correct subtype based on a `type` discriminator field. ## Key Components | Element | Description | |---|---| -| `type` | String discriminator field used by Jackson to resolve the concrete subclass at deserialization time | -| `@JsonTypeInfo` | Configures polymorphic type handling using `Id.NAME`, reading from the existing `type` property | -| `defaultImpl = GenericContext.class` | Falls back to `GenericContext` when no matching type name is registered | -| `@SuperBuilder` | Enables builder inheritance across subclasses | +| `NotificationContext` | Abstract base class all notification context types must extend | +| `getType()` | Abstract method subclasses must implement to declare their discriminator value | +| `@JsonTypeInfo` | Configures polymorphic deserialization using the `type` property on the JSON object | +| `defaultImpl = GenericContext.class` | Falls back to `GenericContext` when no matching subtype is found | +| `@SuperBuilder` | Enables Lombok builder inheritance across subclass hierarchies | +| `@NoArgsConstructor` | Provides a no-arg constructor required for Jackson deserialization | ## Usage Example ```java -// Define a concrete subclass -@JsonTypeName("ticket") -@Data +// Define a concrete subtype +@JsonTypeName("ticket_created") @SuperBuilder @NoArgsConstructor -@AllArgsConstructor -public class TicketContext extends NotificationContext { +public class TicketCreatedContext extends NotificationContext { + private String ticketId; private String summary; + + @Override + public String getType() { + return "ticket_created"; + } } -// JSON deserialization β€” Jackson resolves to TicketContext +// Jackson will deserialize to the correct subtype automatically String json = """ { - "type": "ticket", + "type": "ticket_created", "ticketId": "TKT-001", "summary": "Server is down" } """; NotificationContext ctx = objectMapper.readValue(json, NotificationContext.class); -// ctx instanceof TicketContext β†’ true - -// Builder usage -TicketContext context = TicketContext.builder() - .type("ticket") - .ticketId("TKT-001") - .summary("Server is down") - .build(); +// ctx is a TicketCreatedContext instance ``` -> Subclasses must register their type name (e.g. via `@JsonTypeName`) and extend `NotificationContext` to participate in polymorphic deserialization. Unknown types fall back to `GenericContext`. \ No newline at end of file +> **Note:** Subclasses must register themselves with `@JsonTypeName` and be discoverable by Jackson (via `@JsonSubTypes` on this class or classpath scanning) for polymorphic resolution to work correctly. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md index a9681de42..84ab045aa 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoOAuth2Authorization.md @@ -1,40 +1,40 @@ - -MongoDB document entity for persisting OAuth2 authorization state, including authorization codes, access tokens, and refresh tokens, with automatic TTL-based expiry cleanup. + +MongoDB document entity representing an OAuth2 authorization record within the OpenFrame platform, storing all token state (authorization code, access token, refresh token) alongside PKCE reconstruction data and automatic TTL-based expiry. ## Key Components -| Field / Method | Description | -|---|---| -| `id` | Primary key (`@Id`) for the authorization document | -| `registeredClientId` | Indexed reference to the OAuth2 registered client | -| `principalName` | Indexed name of the authenticated principal | -| `authorizationGrantType` | Grant type used (e.g., `authorization_code`) | -| `ar*` fields | Minimal PKCE snapshot fields (client ID, redirect URI, scopes, state, code challenge) stored as flat strings to avoid MongoDB dotted-key issues | -| `state` | Indexed OAuth2 state parameter | -| `authorizationCode*` | Authorization code value, issuance/expiry timestamps, and metadata | -| `accessToken*` | Access token value, timestamps, type, scopes, and metadata | -| `refreshToken*` | Refresh token value, timestamps, and metadata | -| `expiresAt` | TTL index field (`expireAfterSeconds = 0`) β€” MongoDB auto-deletes the document when this timestamp passes | -| `updateExpiresAt()` | Derives `expiresAt` as the maximum non-null expiry across all token types | +| Field Group | Fields | Purpose | +|---|---|---| +| Identity | `id`, `tenantId`, `registeredClientId`, `principalName` | Multi-tenant authorization ownership | +| PKCE Snapshot | `arClientId`, `arRedirectUri`, `arScopes`, `arState`, `arAdditional` | Minimal flat fields for PKCE flow reconstruction (avoids dotted-key issues) | +| Authorization Code | `authorizationCodeValue`, `*IssuedAt`, `*ExpiresAt`, `*Metadata` | Authorization code grant state | +| Access Token | `accessTokenValue`, `*IssuedAt`, `*ExpiresAt`, `accessTokenType`, `accessTokenScopes` | Issued access token lifecycle | +| Refresh Token | `refreshTokenValue`, `*IssuedAt`, `*ExpiresAt`, `*Metadata` | Refresh token lifecycle | +| TTL | `expiresAt` | MongoDB TTL index β€” auto-deletes expired documents | -## Usage Example - -```java -MongoOAuth2Authorization authorization = new MongoOAuth2Authorization(); -authorization.setId(UUID.randomUUID().toString()); -authorization.setRegisteredClientId("my-client"); -authorization.setPrincipalName("user@example.com"); -authorization.setAuthorizationGrantType("authorization_code"); +**`updateExpiresAt()`** β€” Computes the latest expiration across all three token types and sets `expiresAt`, driving the MongoDB TTL index for automatic cleanup. -// Set token details -authorization.setAccessTokenValue(tokenValue); -authorization.setAccessTokenIssuedAt(Instant.now()); -authorization.setAccessTokenExpiresAt(Instant.now().plusSeconds(3600)); +**Indexes:** +- Compound: `registeredClientId + principalName` +- Single: `tenantId`, `state`, `accessTokenValue`, `refreshTokenValue` +- TTL: `expiresAt` (`expireAfterSeconds = 0`) -// Compute TTL β€” must be called after any token expiry field is updated -authorization.updateExpiresAt(); +## Usage Example -mongoOAuth2AuthorizationRepository.save(authorization); +```java +MongoOAuth2Authorization auth = new MongoOAuth2Authorization(); +auth.setId(UUID.randomUUID().toString()); +auth.setTenantId("tenant-abc"); +auth.setRegisteredClientId("client-123"); +auth.setPrincipalName("user@example.com"); +auth.setAuthorizationGrantType("authorization_code"); + +// Set token expiry fields, then compute TTL +auth.setAccessTokenExpiresAt(Instant.now().plusSeconds(3600)); +auth.setRefreshTokenExpiresAt(Instant.now().plusSeconds(86400)); +auth.updateExpiresAt(); // expiresAt = refreshTokenExpiresAt (latest) + +mongoTemplate.save(auth); ``` -> **Note:** Always call `updateExpiresAt()` after modifying any token expiry field to ensure the MongoDB TTL index remains accurate and stale documents are cleaned up automatically. \ No newline at end of file +> **Note:** Always call `updateExpiresAt()` after modifying any token expiration field to ensure the MongoDB TTL index reflects the correct document lifetime. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md index 8c8fb4717..191e40274 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.MongoRegisteredClient.md @@ -1,34 +1,35 @@ - -MongoDB document entity representing an OAuth 2.0 registered client, mapped to the `oauth_registered_clients` collection for persisting client credentials and authorization configuration. + +MongoDB document entity representing an OAuth 2.0 registered client, stored in the `oauth_registered_clients` collection with multi-tenant isolation via a compound unique index on `tenantId` and `clientId`. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | Primary document identifier (`@Id`) | -| `clientId` | `String` | Unique OAuth client identifier (`@Indexed(unique = true)`) | +| Member | Type | Description | +|--------|------|-------------| +| `id` | `String` | MongoDB document `_id` | +| `tenantId` | `String` | Tenant identifier (enforces `TenantScoped`) | +| `clientId` | `String` | OAuth 2.0 client identifier | | `clientSecret` | `String` | Hashed client secret | -| `authenticationMethods` | `Set` | Supported authentication methods (e.g., `client_secret_basic`) | -| `grantTypes` | `Set` | Allowed grant types (e.g., `authorization_code`, `refresh_token`) | -| `redirectUris` | `Set` | Permitted redirect URIs post-authorization | -| `scopes` | `Set` | OAuth scopes the client may request | +| `authenticationMethods` | `Set` | Allowed auth methods (e.g. `client_secret_basic`) | +| `grantTypes` | `Set` | Allowed grant types (e.g. `authorization_code`, `refresh_token`) | +| `redirectUris` | `Set` | Whitelisted redirect URIs | +| `scopes` | `Set` | Permitted OAuth scopes | | `requireProofKey` | `boolean` | Enforces PKCE (Proof Key for Code Exchange) | -| `requireAuthorizationConsent` | `boolean` | Requires explicit user consent screen | -| `accessTokenTtlSeconds` | `long` | Access token time-to-live in seconds | -| `refreshTokenTtlSeconds` | `long` | Refresh token time-to-live in seconds | -| `reuseRefreshTokens` | `boolean` | Whether refresh tokens are reused on rotation | +| `requireAuthorizationConsent` | `boolean` | Prompts user consent screen | +| `accessTokenTtlSeconds` | `long` | Access token lifetime | +| `refreshTokenTtlSeconds` | `long` | Refresh token lifetime | +| `reuseRefreshTokens` | `boolean` | Controls refresh token rotation | ## Usage Example ```java MongoRegisteredClient client = MongoRegisteredClient.builder() - .id(UUID.randomUUID().toString()) - .clientId("openframe-web") + .tenantId("tenant-abc") + .clientId("my-app") .clientSecret("{bcrypt}$2a$10$...") .authenticationMethods(Set.of("client_secret_basic")) .grantTypes(Set.of("authorization_code", "refresh_token")) - .redirectUris(Set.of("https://app.openframe.ai/callback")) - .scopes(Set.of("openid", "profile", "email")) + .redirectUris(Set.of("https://app.example.com/callback")) + .scopes(Set.of("openid", "profile", "read:tickets")) .requireProofKey(true) .requireAuthorizationConsent(false) .accessTokenTtlSeconds(3600) @@ -36,7 +37,7 @@ MongoRegisteredClient client = MongoRegisteredClient.builder() .reuseRefreshTokens(false) .build(); -mongoRegisteredClientRepository.save(client); +mongoTemplate.save(client); ``` -> **Note:** `clientId` is enforced as unique at the database index level, preventing duplicate client registrations across the OpenFrame OAuth authorization server. \ No newline at end of file +> **Note:** The compound index `tenant_clientId_idx` guarantees `clientId` uniqueness is scoped per tenant, preventing cross-tenant `clientId` collisions. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md index 418fc38c6..93691b35a 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthClient.md @@ -1,38 +1,36 @@ - -MongoDB document entity representing an OAuth 2.0 client registration, mapped to the `oauth_clients` collection. + +MongoDB document entity representing an OAuth 2.0 client registration, scoped to a specific tenant within the OpenFrame platform. ## Key Components -| Field | Type | Description | -|-------|------|-------------| -| `id` | `String` | MongoDB document identifier (`@Id`) | -| `clientId` | `String` | Public OAuth client identifier | -| `clientSecret` | `String` | Confidential client secret | -| `machineId` | `String` | Associated machine/tenant reference | +| Member | Type | Description | +|--------|------|-------------| +| `id` | `String` | MongoDB document primary key (`@Id`) | +| `tenantId` | `String` | Indexed tenant identifier for multi-tenancy isolation | +| `clientId` | `String` | OAuth 2.0 client identifier | +| `clientSecret` | `String` | OAuth 2.0 client secret | +| `machineId` | `String` | Optional machine/device association | | `redirectUris` | `String[]` | Allowed redirect URIs for authorization flows | -| `grantTypes` | `String[]` | Permitted OAuth grant types | -| `scopes` | `String[]` | Access scopes the client may request | +| `grantTypes` | `String[]` | Permitted OAuth grant types (`authorization_code`, `password`, `client_credentials`, `refresh_token`) | +| `scopes` | `String[]` | Permitted OAuth scopes | | `roles` | `String[]` | Assigned roles (defaults to empty array) | | `enabled` | `boolean` | Whether the client is active (defaults to `true`) | -**Supported Grant Types:** `authorization_code`, `password`, `client_credentials`, `refresh_token` - ## Usage Example ```java -// Creating a new machine-to-machine OAuth client OAuthClient client = new OAuthClient(); -client.setClientId("openframe-service"); +client.setTenantId("tenant-abc"); +client.setClientId("my-service-client"); client.setClientSecret("s3cr3t"); -client.setMachineId("machine-abc123"); -client.setGrantTypes(new String[]{"client_credentials"}); +client.setGrantTypes(new String[]{"client_credentials", "refresh_token"}); client.setScopes(new String[]{"read", "write"}); -client.setRoles(new String[]{"ROLE_SERVICE"}); -client.setRedirectUris(new String[]{}); +client.setRedirectUris(new String[]{"https://app.example.com/callback"}); +client.setRoles(new String[]{"ROLE_API"}); client.setEnabled(true); -// Persisted via Spring Data MongoDB repository -oAuthClientRepository.save(client); +// Persisted to MongoDB collection: oauth_clients +mongoTemplate.save(client); ``` -> **Note:** `clientSecret` should always be stored hashed. The `machineId` field links this client to a specific OpenFrame machine context, enabling per-tenant OAuth isolation. \ No newline at end of file +> **Note:** Implements `TenantScoped` to enforce tenant isolation across all queries. The `tenantId` field is indexed for efficient per-tenant lookups across the `oauth_clients` collection. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md index 5fb46df11..fed77578d 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/oauth/.OAuthToken.md @@ -1,39 +1,36 @@ - -A MongoDB document entity representing an OAuth 2.0 token record, storing access and refresh tokens alongside their expiry timestamps, client identity, and granted scopes. + +MongoDB document entity representing an OAuth token stored in the `oauth_tokens` collection, scoped to a specific tenant. ## Key Components | Field | Type | Description | -|---|---|---| +|-------|------|-------------| | `id` | `String` | MongoDB document identifier (`@Id`) | -| `userId` | `String` | Reference to the owning user | +| `tenantId` | `String` | Indexed tenant reference for multi-tenancy scoping | +| `userId` | `String` | Associated user identifier | | `accessToken` | `String` | OAuth access token value | | `refreshToken` | `String` | OAuth refresh token value | -| `accessTokenExpiry` | `Instant` | UTC expiry timestamp for the access token | -| `refreshTokenExpiry` | `Instant` | UTC expiry timestamp for the refresh token | +| `accessTokenExpiry` | `Instant` | Access token expiration timestamp | +| `refreshTokenExpiry` | `Instant` | Refresh token expiration timestamp | | `clientId` | `String` | OAuth client application identifier | -| `scopes` | `String[]` | Granted permission scopes for this token | +| `scopes` | `String[]` | Granted permission scopes | -- **Collection:** `oauth_tokens` -- **Annotations:** `@Data` (Lombok) generates getters, setters, `equals`, `hashCode`, and `toString` +Implements `TenantScoped` to enforce multi-tenant data isolation across the platform. ## Usage Example ```java -// Persisting a new OAuth token OAuthToken token = new OAuthToken(); +token.setTenantId("tenant-abc"); token.setUserId("user-123"); token.setClientId("openframe-client"); token.setAccessToken("eyJhbGciOiJSUzI1NiJ9..."); token.setRefreshToken("dGhpcyBpcyBhIHJlZnJlc2g..."); token.setAccessTokenExpiry(Instant.now().plusSeconds(3600)); token.setRefreshTokenExpiry(Instant.now().plusSeconds(86400)); -token.setScopes(new String[]{"read", "write", "admin"}); +token.setScopes(new String[]{"read:tickets", "write:tickets"}); mongoTemplate.save(token); - -// Checking token expiry -boolean isExpired = token.getAccessTokenExpiry().isBefore(Instant.now()); ``` -> **Note:** Token values should be encrypted at rest. Expiry checks should always use `Instant.now()` to ensure timezone-safe comparisons. \ No newline at end of file +> **Note:** The `tenantId` field is indexed for efficient per-tenant token lookups. Always ensure `tenantId` is populated before persisting to maintain proper multi-tenant isolation. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md index 2c8694154..37f419aa3 100644 --- a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/organization/.Organization.md @@ -1,49 +1,51 @@ - -MongoDB document entity representing a company or client organization within the OpenFrame platform, storing business metadata, contract details, and lifecycle status. + +MongoDB document entity representing a company or client organization within a tenant's scope, storing business details, contract information, and lifecycle status. ## Key Components | Member | Type | Description | |--------|------|-------------| -| `organizationId` | `String` | Immutable UUID-based unique identifier (indexed, unique) | -| `isDefault` | `Boolean` | Flags the tenant's default organization | +| `id` | `String` | MongoDB document ID | +| `tenantId` | `String` | Tenant scope identifier (indexed) | +| `organizationId` | `String` | Immutable UUID-based unique identifier | +| `isDefault` | `Boolean` | Marks the default organization for a tenant | | `status` | `OrganizationStatus` | Lifecycle state: `ACTIVE`, `ARCHIVED`, or `DELETED` | -| `contactInformation` | `ContactInformation` | Nested contacts and address data | -| `monthlyRevenue` | `BigDecimal` | Revenue figure in the organization's currency | +| `contactInformation` | `ContactInformation` | Embedded contacts and addresses | +| `monthlyRevenue` | `BigDecimal` | Revenue figure in organization's currency | | `contractStartDate` / `contractEndDate` | `LocalDate` | Contract validity window | -| `createdAt` / `updatedAt` | `Instant` | Auto-managed audit timestamps via Spring Data | | `isContractActive()` | `boolean` | Returns `true` if today falls within the contract window | -| `isDeleted()` | `boolean` | Soft-delete check against `OrganizationStatus.DELETED` | -| `isArchived()` | `boolean` | Checks against `OrganizationStatus.ARCHIVED` | +| `isDeleted()` | `boolean` | Soft-delete check | +| `isArchived()` | `boolean` | Archive state check | ## Usage Example ```java -// Build a new organization Organization org = Organization.builder() + .tenantId("tenant-abc") .organizationId(UUID.randomUUID().toString()) .name("Acme Corp") - .category("IT Services") - .numberOfEmployees(150) + .category("Technology") + .numberOfEmployees(250) .websiteUrl("https://acme.example.com") - .monthlyRevenue(new BigDecimal("25000.00")) + .monthlyRevenue(new BigDecimal("15000.00")) .contractStartDate(LocalDate.of(2024, 1, 1)) - .contractEndDate(LocalDate.of(2025, 1, 1)) + .contractEndDate(LocalDate.of(2025, 12, 31)) .isDefault(true) .build(); // Lifecycle checks if (org.isContractActive()) { - System.out.println("Contract is currently active"); + // Contract is currently valid } -if (org.isArchived()) { - System.out.println("Organization is hidden but retained for device references"); +if (!org.isDeleted() && !org.isArchived()) { + // Organization is visible and active } ``` ## Notes -- **Soft deletes only** β€” `DELETED` status is preferred over physical removal to preserve references from devices that may reconnect after going offline. -- `ARCHIVED` hides the organization from normal queries while keeping the document intact. -- `organizationId` is treated as immutable; only `id` (MongoDB `_id`) is managed by the database engine. \ No newline at end of file +- Implements `TenantScoped` β€” all queries should be filtered by `tenantId` +- Uses **soft deletion** (`ARCHIVED`/`DELETED`) instead of hard deletes to preserve references from devices that may come back online +- `organizationId` is unique and **immutable** after creation; `id` is the internal MongoDB `_id` +- Auditing fields (`createdAt`, `updatedAt`) are managed automatically via Spring Data annotations \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.Script.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.Script.md new file mode 100644 index 000000000..3dbc4a876 --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.Script.md @@ -0,0 +1,38 @@ + +MongoDB document entity for RMM scripts within the OpenFrame platform, representing reusable scripts executable on managed agents via ad-hoc dispatch, scheduling, or check integration. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `tenantId` | `String` | Multi-tenant isolation key (compound-indexed with `name`) | +| `shell` | `ScriptShell` | Interpreter type (PowerShell, CMD, Bash, Python) | +| `scriptBody` | `String` | Raw script source stored inline | +| `supportedPlatforms` | `List` | OS targets to prevent cross-platform dispatch errors | +| `defaultTimeoutSeconds` | `Integer` | Agent-enforced execution timeout | +| `defaultArgs` | `List` | Default positional CLI arguments (`argv`) | +| `envVars` | `List` | Pre-execution environment variables (secret encryption pipeline pending) | +| `status` | `ScriptStatus` | Lifecycle state; defaults to `ACTIVE`, uses `DELETED` for soft-delete | + +**Indexes:** +- Compound unique index on `(tenantId, name)` β€” enforces per-tenant name uniqueness +- Single-field index on `status` β€” supports filtered queries by lifecycle state + +## Usage Example + +```java +Script script = Script.builder() + .tenantId("tenant-abc") + .name("Disk Cleanup") + .description("Removes temp files older than 30 days") + .shell(ScriptShell.POWERSHELL) + .scriptBody("Get-ChildItem $env:TEMP | Remove-Item -Recurse -Force") + .tag("maintenance") + .supportedPlatforms(List.of(ScriptPlatform.WINDOWS)) + .defaultTimeoutSeconds(120) + .defaultArgs(List.of("--silent")) + .envVars(List.of(new ScriptEnvVar("LOG_LEVEL", "info", false))) + .build(); +``` + +> **Note:** `description`, `tag`, `defaultArgs`, and `envVars` are optional. Category normalization (trim/lowercase) is enforced at the service layer. Large `scriptBody` values are stored inline β€” object storage offload is planned for a future iteration. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptEnvVar.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptEnvVar.md new file mode 100644 index 000000000..6b6c252a0 --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptEnvVar.md @@ -0,0 +1,30 @@ + +Represents a single environment variable to be exported on an RMM agent prior to script execution. This class is embedded within the `Script` document and is not stored as a standalone MongoDB collection. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `name` | `String` | The environment variable name as exported on the agent | +| `value` | `String` | The variable's value; intended to hold ciphertext when `secret` is `true`, but currently stored in plaintext | +| `secret` | `boolean` | Marks the variable as sensitive; the service layer rejects `secret = true` on write until the secret-management pipeline is implemented | + +## Usage Example + +```java +// Plain environment variable +ScriptEnvVar plain = ScriptEnvVar.builder() + .name("API_BASE_URL") + .value("https://api.example.com") + .secret(false) + .build(); + +// Secret variable β€” currently rejected at the service layer +ScriptEnvVar secret = ScriptEnvVar.builder() + .name("API_KEY") + .value("supersecret") + .secret(true) // ⚠️ write will be rejected until secret-management lands + .build(); +``` + +> **⚠️ Known Limitation:** Secret management (encryption at rest + secure agent delivery) is not yet implemented. The service layer currently rejects any `ScriptEnvVar` with `secret = true` on write, and all values are persisted to MongoDB in plaintext regardless of the `secret` flag. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptPlatform.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptPlatform.md new file mode 100644 index 000000000..6f23394be --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptPlatform.md @@ -0,0 +1,30 @@ + +Enum representing the supported operating-system platforms for RMM script execution targeting. + +## Key Components + +| Value | Description | +|-------|-------------| +| `WINDOWS` | Microsoft Windows environments | +| `LINUX` | Linux-based operating systems | +| `MACOS` | Apple macOS environments | + +## Usage Example + +```java +Script script = new Script(); +script.setPlatform(ScriptPlatform.LINUX); + +if (script.getPlatform() == ScriptPlatform.WINDOWS) { + // Windows-specific execution logic +} + +// Switch-based dispatch +switch (script.getPlatform()) { + case WINDOWS -> executeOnWindows(script); + case LINUX -> executeOnLinux(script); + case MACOS -> executeOnMacOS(script); +} +``` + +> **Note:** Platform granularity is intentionally coarse. Version-specific behavior (e.g. Ubuntu vs. Debian, Windows 10 vs. Server 2022) should be handled within the script body or via runtime arguments rather than by adding new enum values. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptShell.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptShell.md new file mode 100644 index 000000000..ddeaae58d --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptShell.md @@ -0,0 +1,25 @@ + +Enum defining the supported shell interpreters available for executing scripts via the RMM agent. + +## Key Components + +| Value | Description | +|-------|-------------| +| `POWERSHELL` | Windows PowerShell interpreter | +| `CMD` | Windows Command Prompt | +| `BASH` | Unix/Linux Bash shell | +| `PYTHON` | Python interpreter | +| `NUSHELL` | Nu shell interpreter | +| `SHELL` | Generic POSIX shell | + +## Usage Example + +```java +Script script = new Script(); +script.setShell(ScriptShell.POWERSHELL); +script.setContent("Get-Process | Select-Object Name, CPU"); +``` + +## Notes + +The enum values intentionally mirror the shell options exposed by Tactical RMM, allowing existing scripts to be migrated to OpenFrame without modifying their shebangs or interpreter declarations. Additional interpreters may be added as agent-side support expands. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptStatus.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptStatus.md new file mode 100644 index 000000000..c13b35224 --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/.ScriptStatus.md @@ -0,0 +1,30 @@ + +Enum defining the three lifecycle states for a `Script` document in the RMM (Remote Monitoring & Management) data model, supporting soft-delete semantics. + +## Key Components + +| Value | Description | +|-------|-------------| +| `ACTIVE` | Script is live and accessible via standard API surfaces | +| `ARCHIVED` | Script is inactive but retained for reference | +| `DELETED` | Soft-deleted; hidden from API but retained in MongoDB for historical execution record integrity | + +## Usage Example + +```java +Script script = scriptRepository.findById(id); + +if (script.getStatus() == ScriptStatus.DELETED) { + throw new ResourceNotFoundException("Script not found"); +} + +// Soft-delete a script +script.setStatus(ScriptStatus.DELETED); +scriptRepository.save(script); +``` + +## Important Notes + +- **Soft-delete behavior:** `DELETED` documents remain in MongoDB so historic execution records continue to reference a valid parent script. +- **Name reservation:** The compound unique index on `(tenantId, name)` includes `DELETED` documents β€” a soft-deleted script name stays **occupied**. Hard-deletion via admin tooling is required to free the name for reuse. +- **Pattern consistency:** Mirrors the `OrganizationStatus` enum pattern used elsewhere in the OpenFrame data model. \ No newline at end of file diff --git a/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/filter/.ScriptQueryFilter.md b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/filter/.ScriptQueryFilter.md new file mode 100644 index 000000000..f0755a237 --- /dev/null +++ b/openframe-data-mongo-common/src/main/java/com/openframe/data/document/rmm/filter/.ScriptQueryFilter.md @@ -0,0 +1,27 @@ + +Data-layer filter criteria for querying `Script` entities, providing a repository-friendly alternative to the API-layer `ScriptFilterInput` to maintain module independence. + +## Key Components + +| Field | Type | Description | +|-------|------|-------------| +| `shells` | `List` | Filter by one or more script shell types | +| `statuses` | `List` | Filter by one or more script statuses | +| `supportedPlatforms` | `List` | Filter by supported platforms | +| `tag` | `String` | Filter by a specific tag | + +## Usage Example + +```java +// Build a filter for active PowerShell scripts targeting Windows +ScriptQueryFilter filter = ScriptQueryFilter.builder() + .shells(List.of(ScriptShell.POWERSHELL)) + .statuses(List.of(ScriptStatus.ACTIVE)) + .supportedPlatforms(List.of(ScriptPlatform.WINDOWS)) + .tag("patch-management") + .build(); + +List" β†’ stripped -// " tag" β†’ escaped to <their> -// "javascript:alert(1)" href β†’ removed +// The component internally applies: +// 1. escapeUnknownHtmlTags (text pre-pass) +// 2. rehype-raw (parse inline HTML) +// 3. rehypeStripUnsafe (HAST sanitizer) +// 4. rehype-highlight (syntax coloring) ``` -> **Note:** This renderer is designed specifically for untrusted LLM output. The sanitization pipeline runs in layers: text pre-processing β†’ `rehype-raw` β†’ `rehypeStripUnsafe` β†’ `rehype-highlight`, ensuring no XSS vectors survive even if one layer is bypassed. Community support is available on the [OpenMSP Slack](https://join.slack.com/t/openmsp/shared_invite/zt-36bl7mx0h-3~U2nFH6nqHqoTPXMaHEHA). \ No newline at end of file +> **Security note:** `rehypeStripUnsafe` is the authoritative security boundary. `escapeUnknownHtmlTags` is a best-effort pre-pass for React compatibility, not a security control. \ No newline at end of file diff --git a/openframe-frontend-core/src/components/ui/.square-avatar.md b/openframe-frontend-core/src/components/ui/.square-avatar.md index 058e7d479..433312ea2 100644 --- a/openframe-frontend-core/src/components/ui/.square-avatar.md +++ b/openframe-frontend-core/src/components/ui/.square-avatar.md @@ -1,49 +1,45 @@ - -A flexible avatar component that displays a user image or falls back to initials, supporting both square and round variants with multiple size options. + +A flexible avatar component that renders a user image with an automatic initials-based fallback, supporting square and round variants across four sizes. ## Key Components -### `SquareAvatarProps` -Extends `React.HTMLAttributes` with: +### `SquareAvatar` + +A memoized, forwarded-ref React component with the following props: | Prop | Type | Default | Description | |------|------|---------|-------------| -| `src` | `string` | β€” | Image URL | -| `alt` | `string` | β€” | Image alt text (also used for initials fallback) | +| `src` | `string` | β€” | Image URL to display | +| `alt` | `string` | β€” | Alt text; also used to derive initials | | `size` | `'sm' \| 'md' \| 'lg' \| 'xl'` | `'md'` | Controls dimensions (32/40/48/64px) | -| `fallback` | `string` | β€” | Preferred string for generating initials | +| `fallback` | `string` | β€” | Preferred string for deriving initials (overrides `alt`) | | `variant` | `'square' \| 'round'` | `'square'` | Border radius style | +| `initialsClassName` | `string` | β€” | Tailwind overrides for initials font size/color | +| `className` | `string` | β€” | Additional classes for the outer wrapper | -### `SquareAvatar` -A memoized, forwarded-ref component that renders an image with automatic graceful degradation to initials on load error or missing `src`. +**Fallback behavior:** When `src` is absent or fails to load, the component displays initials extracted via `getFirstLastInitials()` (from `fallback` β†’ `alt` β†’ `'?'`). On image error, the `` is hidden and the initials layer is revealed. -**Fallback behavior:** Uses `getFirstLastInitials()` on `fallback ?? alt`, rendering `'?'` if neither yields initials. The initials layer uses `text-ods-text-primary` to maintain WCAG AA contrast across accent-fill backgrounds (e.g. Mingo cyan, current-user pink). +**Accessibility note:** Initials use `text-ods-text-primary` to maintain WCAG AA contrast on both default and accent-fill backgrounds (e.g. `bg-ods-flamingo-pink`). ## Usage Example ```typescript -// Image avatar - +// Basic image avatar + -// Initials-only avatar (no src) +// Round variant with explicit fallback text -// With custom className and forwarded ref -const ref = React.useRef(null); +// Compact initials-only avatar with custom text size ``` \ No newline at end of file diff --git a/openframe-frontend-core/src/components/ui/.tags-input.md b/openframe-frontend-core/src/components/ui/.tags-input.md index 0d2cc4ba2..5c011eb90 100644 --- a/openframe-frontend-core/src/components/ui/.tags-input.md +++ b/openframe-frontend-core/src/components/ui/.tags-input.md @@ -1,51 +1,47 @@ - -A React component that provides an interactive tags input interface, allowing users to add, remove, and manage a collection of string tags with keyboard shortcuts and validation. + +A controlled tags input component that allows users to add and remove string tags via keyboard or button interaction. ## Key Components -- **TagsInput**: Main component that manages tag state and user interactions -- **handleAddTag**: Adds new tags with duplicate checking and max limit validation -- **handleRemoveTag**: Removes tags from the collection -- **handleKeyPress**: Enables Enter key for adding tags -- **TagsInputProps**: Interface defining component configuration options +### `TagsInput` +Main export β€” a form component managing a list of string tags with add/remove functionality. + +**Props (`TagsInputProps`):** + +| Prop | Type | Default | Description | +|------|------|---------|-------------| +| `value` | `string[]` | `[]` | Controlled list of current tags | +| `onChange` | `(tags: string[]) => void` | β€” | Callback fired on add/remove | +| `placeholder` | `string` | `"Add a tag..."` | Input placeholder text | +| `disabled` | `boolean` | `false` | Disables all interactions | +| `maxTags` | `number` | β€” | Optional cap on number of tags | +| `label` | `string` | β€” | Optional field label | +| `inputClassName` | `string` | β€” | Extra classes for the text input | +| `badgeClassName` | `string` | β€” | Extra classes for tag badges | + +**Behavior:** +- Tags are added via **Enter** key or **Add** button +- Duplicate tags are silently ignored +- Tags are removed via the `X` button on each badge +- Input and Add button disable automatically when `maxTags` is reached +- Displays a `{current}/{max}` counter when `maxTags` is set ## Usage Example ```typescript -import { TagsInput } from "./tags-input" -import { useState } from "react" +import { TagsInput } from "@/components/ui/tags-input" -function MyForm() { - const [tags, setTags] = useState([]) +function TicketForm() { + const [tags, setTags] = React.useState(["urgent", "network"]) return ( ) } - -// With initial tags -function PrefilledTags() { - const [technologies, setTechnologies] = useState([ - "React", "TypeScript", "Node.js" - ]) - - return ( - - ) -} -``` - -The component features duplicate prevention, optional tag limits, keyboard navigation, customizable styling, and accessibility support with proper ARIA labels. \ No newline at end of file +``` \ No newline at end of file diff --git a/openframe-frontend-core/src/components/ui/.textarea.md b/openframe-frontend-core/src/components/ui/.textarea.md index 70e6a7d89..67d27b09f 100644 --- a/openframe-frontend-core/src/components/ui/.textarea.md +++ b/openframe-frontend-core/src/components/ui/.textarea.md @@ -1,52 +1,54 @@ - -A client-side textarea component supporting labels, validation states, and optional end icons (as decorative spans or interactive buttons), built on top of `FieldWrapper`. + +A client-side textarea component that extends the native HTML `