feat: update index

[shibadityadeb]

Summary by CodeRabbit

• New Features
  • Added a knowledge library for uploading, processing, organizing, reindexing, deleting, and searching documents.
  • Added hybrid, vector, and keyword search with document metadata and processing status visibility.
  • Added Google Workspace connection, synchronization, resource browsing, logs, and connection management.
  • Added navigation links for Knowledge, Upload, Library, Search, and Connectors.
• Documentation
  • Added setup and integration guidance for connectors and expanded repository documentation.
• Security
  • Added protected OAuth handling and encrypted credential storage.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2883cf3e-de1d-4ec4-b920-2171be75ac46

📥 Commits

Reviewing files that changed from the base of the PR and between f38535e and cdd89f8.

⛔ Files ignored due to path filters (1)

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (120)

• .env.example
• README.md
• apps/api/package.json
• apps/api/prisma/migrations/20260714075643_knowledge_brain/migration.sql
• apps/api/prisma/migrations/20260714080100_chunk_fulltext_index/migration.sql
• apps/api/prisma/migrations/20260714103641_connector_platform/migration.sql
• apps/api/prisma/schema.prisma
• apps/api/src/app.ts
• apps/api/src/config/env.ts
• apps/api/src/config/index.ts
• apps/api/src/modules/connectors/connector.routes.ts
• apps/api/src/modules/connectors/connector.schemas.ts
• apps/api/src/modules/connectors/connector.service.ts
• apps/api/src/modules/knowledge/fusion.ts
• apps/api/src/modules/knowledge/knowledge.routes.ts
• apps/api/src/modules/knowledge/knowledge.schemas.ts
• apps/api/src/modules/knowledge/knowledge.service.ts
• apps/api/src/plugins/services.ts
• apps/api/src/services/temporal.service.ts
• apps/api/src/services/vector.service.ts
• apps/api/tests/connector.schemas.test.ts
• apps/api/tests/fusion.test.ts
• apps/api/tests/knowledge.schemas.test.ts
• apps/web/src/app/(app)/connectors/[id]/page.tsx
• apps/web/src/app/(app)/connectors/page.tsx
• apps/web/src/app/(app)/knowledge/documents/[id]/page.tsx
• apps/web/src/app/(app)/knowledge/library/page.tsx
• apps/web/src/app/(app)/knowledge/page.tsx
• apps/web/src/app/(app)/knowledge/search/page.tsx
• apps/web/src/app/(app)/knowledge/upload/page.tsx
• apps/web/src/app/(app)/layout.tsx
• apps/web/src/components/knowledge/status-badge.tsx
• apps/web/src/lib/api.ts
• docs/connectors.md
• package.json
• packages/activities/package.json
• packages/activities/src/index.ts
• packages/activities/src/knowledge.activities.ts
• packages/activities/src/knowledge.context.ts
• packages/auth/eslint.config.mjs
• packages/auth/package.json
• packages/auth/src/auth.test.ts
• packages/auth/src/crypto.ts
• packages/auth/src/index.ts
• packages/auth/src/oauth2.ts
• packages/auth/tsconfig.json
• packages/connectors/core/eslint.config.mjs
• packages/connectors/core/package.json
• packages/connectors/core/src/base.ts
• packages/connectors/core/src/core.test.ts
• packages/connectors/core/src/errors.ts
• packages/connectors/core/src/index.ts
• packages/connectors/core/src/registry.ts
• packages/connectors/core/src/types.ts
• packages/connectors/core/tsconfig.json
• packages/connectors/google/eslint.config.mjs
• packages/connectors/google/package.json
• packages/connectors/google/src/config.ts
• packages/connectors/google/src/connector.ts
• packages/connectors/google/src/google.test.ts
• packages/connectors/google/src/http.ts
• packages/connectors/google/src/index.ts
• packages/connectors/google/src/mappers.ts
• packages/connectors/google/src/services/calendar.ts
• packages/connectors/google/src/services/drive.ts
• packages/connectors/google/src/services/gmail.ts
• packages/connectors/google/tsconfig.json
• packages/events/eslint.config.mjs
• packages/events/package.json
• packages/events/src/bus.ts
• packages/events/src/events.test.ts
• packages/events/src/index.ts
• packages/events/src/types.ts
• packages/events/tsconfig.json
• packages/knowledge/eslint.config.mjs
• packages/knowledge/package.json
• packages/knowledge/src/chunker.test.ts
• packages/knowledge/src/chunker.ts
• packages/knowledge/src/cleaner.ts
• packages/knowledge/src/embeddings.test.ts
• packages/knowledge/src/embeddings/gemini.provider.ts
• packages/knowledge/src/embeddings/index.ts
• packages/knowledge/src/embeddings/local.provider.ts
• packages/knowledge/src/embeddings/openai.provider.ts
• packages/knowledge/src/embeddings/types.ts
• packages/knowledge/src/embeddings/voyage.provider.ts
• packages/knowledge/src/index.ts
• packages/knowledge/src/metadata.ts
• packages/knowledge/src/parsers.test.ts
• packages/knowledge/src/parsers/csv.parser.ts
• packages/knowledge/src/parsers/docx.parser.ts
• packages/knowledge/src/parsers/html.parser.ts
• packages/knowledge/src/parsers/index.ts
• packages/knowledge/src/parsers/json.parser.ts
• packages/knowledge/src/parsers/markdown.parser.ts
• packages/knowledge/src/parsers/pdf.parser.ts
• packages/knowledge/src/parsers/text.parser.ts
• packages/knowledge/src/pdf-parse.d.ts
• packages/knowledge/src/tokens.ts
• packages/knowledge/src/types.ts
• packages/knowledge/tsconfig.json
• packages/types/src/auth.ts
• packages/workflows/src/connector-contract.ts
• packages/workflows/src/constants.ts
• packages/workflows/src/definitions.ts
• packages/workflows/src/index.ts
• packages/workflows/src/workflows/connector-sync.workflow.ts
• packages/workflows/src/workflows/document-ingestion.workflow.ts
• pnpm-workspace.yaml
• services/connector-worker/eslint.config.mjs
• services/connector-worker/package.json
• services/connector-worker/src/activities.ts
• services/connector-worker/src/config.ts
• services/connector-worker/src/context.ts
• services/connector-worker/src/index.ts
• services/connector-worker/src/token-manager.ts
• services/connector-worker/tsconfig.build.json
• services/connector-worker/tsconfig.json
• services/temporal-worker/src/config.ts
• services/temporal-worker/src/index.ts

---

📝 Walkthrough

Walkthrough

This PR adds a knowledge document pipeline and hybrid search, a Google Workspace connector platform, shared Temporal workflows, persistence models, worker services, API routes, client APIs, web pages, configuration, and supporting tests and documentation.

Changes

Knowledge and connector platform

Layer / File(s)	Summary
Data models and runtime configuration apps/api/prisma/*, apps/api/src/config/*, packages/types/*, package.json, pnpm-workspace.yaml	Adds knowledge and connector database schemas, permissions, environment validation, workspace packages, and development scripts.
Authentication and connector SDK packages/auth/*, packages/connectors/core/*, packages/connectors/google/src/config.ts, packages/connectors/google/src/mappers.ts	Adds AES-GCM secret encryption, OAuth helpers, connector contracts, registries, typed errors, Google resource mappings, and tests.
Google synchronization packages/connectors/google/src/{http.ts,connector.ts}, packages/connectors/google/src/services/*	Implements Google API access, Drive/Gmail/Calendar full synchronization, incremental cursors, error mapping, and provider tests.
Events and Temporal orchestration packages/events/*, packages/workflows/*, apps/api/src/services/temporal.service.ts	Adds Redis events, document ingestion workflows, connector sync workflows, progress queries, activity contracts, and workflow termination support.
Knowledge parsing and embeddings packages/knowledge/src/*	Adds parsers, metadata extraction, cleaning, chunking, local and remote embeddings, hybrid-fusion support, and tests.
Knowledge ingestion runtime packages/activities/src/knowledge.*, services/temporal-worker/*, apps/api/src/plugins/services.ts, apps/api/src/services/vector.service.ts	Connects ingestion activities to storage, Prisma, Qdrant, embeddings, document state updates, and vector cleanup.
Connector worker runtime services/connector-worker/*	Adds worker configuration, token caching and refresh, connector activity persistence, event publication, Temporal execution, health checks, and graceful shutdown.
API modules and service flows apps/api/src/app.ts, apps/api/src/modules/{knowledge,connectors}/*, apps/api/src/config/*	Registers authenticated knowledge and connector endpoints for uploads, search, OAuth, synchronization, resources, logs, and processing status.
Web application experience apps/web/src/app/(app)/*, apps/web/src/components/knowledge/*, apps/web/src/lib/api.ts, README.md, docs/connectors.md	Adds knowledge and connector dashboards, document management, search and upload pages, connector monitoring, client API models, navigation, and platform documentation.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

• shibadityadeb/Context-Brain#1: Both changes extend the Temporal integration layer, including the TemporalService wrapper.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch oauth-connector

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint install failed: private package registry requires authentication. Disable ESLint in CodeRabbit settings or use public packages.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}