Skip to content

Update dependencies, refactor Dockerfiles, and enhance security - #154

Open
Chromeninja wants to merge 26 commits into
v2.2.xfrom
v2.2.x_localfix
Open

Update dependencies, refactor Dockerfiles, and enhance security#154
Chromeninja wants to merge 26 commits into
v2.2.xfrom
v2.2.x_localfix

Conversation

@Chromeninja

Copy link
Copy Markdown

This update includes dependency upgrades, Dockerfile refactoring for consistency, and security enhancements such as real JWT verification and authorization checks. It also consolidates environment variable files and removes deprecated components to streamline the project structure.

Chromeninja and others added 19 commits June 12, 2026 09:50
- Updated gRPC and psycopg2-binary versions in requirements.txt for video_proxy_module and workflow_core_module.
- Refactored Dockerfiles for workflow_core_module, video_proxy_module, presence_module, googlechat_module, mattermost_module, and teams_module to maintain consistency and improve readability.
- Removed credential-manager service from docker-compose.yml.
- Adjusted port mappings and healthcheck URLs in docker-compose.yml for various services.
- Added .env.local.example file for ai_interaction_module to provide a template for local development environment variables.
Removed deprecated docker-compose.updated.yml and docker-compose.yml.backup.
Updated test-deployment.sh to reference Kubernetes and make dev instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaced .env.example (131 lines, minimal) with content from .env-example
(380 lines, annotated with REQUIRED/OPTIONAL flags and documentation).
Deleted .env-example as .env.example is now the canonical file.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removed legacy archive directory containing outdated module implementations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…root compose as deprecated

Deleted docker-compose files from module directories:
- action/pushing/{discord,youtube,lambda,twitch,slack}_action_module/docker-compose.*
- config/observability/docker-compose.observability.yml

Root docker-compose.yml already annotated with deprecation notice directing to K8s.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e/ and helm/)

Deleted outdated raw Kubernetes manifests directory. Modern deployments use:
- k8s/kustomize/ for alpha (MicroK8s)
- k8s/helm/ for beta and production

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ards)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…; add SECRETS_SETUP.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nly targets

- test-integration: call scripts/test-api-all.sh
- test-e2e: call scripts/e2e-test-alpha.sh
- test-functional: error message (not yet implemented)
- smoke-test: call tests/alpha-smoke-test.sh
- seed-mock-data: call scripts/seed-admin.sh
- deploy-dev: call scripts/deploy-alpha.sh
- deploy-prod: error message (CI-only)
- docker-push: error message (CI-only)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uild sources

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… Debian bookworm variants

- PostgreSQL: 16/15 → 16/15-bookworm@sha256
- Redis: 7-alpine → 7-bookworm@sha256
- MinIO, Ollama, Qdrant: latest → latest@sha256
- Updated Helm values (values.yaml, values-beta.yaml, values-local.yaml)
- Updated Kustomize components and alpha overlay imagePatches
- Updated k8s/infrastructure.yaml reference manifests
- All external images now pinned to multi-arch manifest digests
- No Alpine images remain in K8s manifests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n core-identity gRPC handler

Three critical security bugs fixed in IdentityServiceServicer:

1. verify_token(): replaced auth bypass (any non-empty string accepted) with real
   JWT verification via flask_core.auth.verify_jwt_token (PyJWT HS256). Adds
   mandatory tenant claim check — tokens without a non-empty tenant are rejected.
   Secret key read from SECRET_KEY env var / Config.SECRET_KEY; never hardcoded.

2. LookupIdentity(): replaced hardcoded hub_user_id=1/username="test_user" with
   real DB query against hub_user_identities JOIN hub_users. Returns NOT_FOUND
   when identity does not exist. Validates platform and platform_user_id required.

3. GetLinkedPlatforms(): replaced hardcoded static [twitch, discord] list with
   real DB query against hub_user_identities WHERE hub_user_id = %s, ordered by
   is_primary DESC, linked_at ASC. Returns empty list (not error) for no platforms.

Both copies fixed identically:
- core/identity_core_module/services/grpc_handler.py (canonical — built by Dockerfile)
- services/core-identity/identity_core_module/services/grpc_handler.py (service mirror)

JWT util reused: flask_core.auth.verify_jwt_token (libs/flask_core, already installed
in container via Dockerfile). DB pattern mirrors security_service.py executesql usage.

Tests: core/identity_core_module/services/test_grpc_handler.py
15 tests, 15 passed — covers empty/None token, tampered sig, expired, missing tenant,
empty tenant, missing params, identity not found, DB-backed lookup, empty platforms,
DB-backed platform list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Any authenticated user could previously call POST /api/v1/payments/refunds
and refund any transaction — no ownership or role check existed (TODO comment
left since initial implementation). Fixed two gaps:

1. Route lacked requireAuth middleware; unauthenticated callers had full access.
2. Controller had no ownership or role check before delegating to paymentService.

Fix: add requireAuth to the route, then in the controller verify the caller
is either a super_admin/platform-admin or the original payment owner
(payment.metadata.userId) before proceeding. Returns 403 otherwise.

Tests added covering: unauthenticated (403), non-owner non-admin (403),
payment owner (200), super_admin (200), platform-admin (200), missing payment (404).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes commit 1bb601b — the refund authorization test added there
requires this jest.config.js to run, and package-lock.json is tracked
per dependency-pinning standards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…p silently dropping official emotes

- Implements OAuth client-credentials flow to fetch Twitch app access tokens
- Caches tokens with expiry tracking, refreshes when < 60s remaining
- Logs clear warning if credentials unconfigured (graceful degradation)
- Logs errors on network/auth failures (no silent failures)
- Tests: 7 unit tests covering all paths (token fetch, cache, refresh, errors)
- Applied identically to both files across container build trees (router_module & translate_interaction_module)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…per dependency-pinning standard

- Pin 81 ^/~ ranges across 9 package.json files
- Create 4 new package-lock.json for test directories
- Add reusable scripts/pin-npm-versions.py for future maintenance
- All dependencies now use exact versions for supply chain security

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ir volumes across all deployments

Set readOnlyRootFilesystem: true on all 23 Kustomize workloads and all 19 Helm
chart deployments (via values.yaml securityContext — single source of truth for
templates using toYaml). Added emptyDir volume named `tmp` mounted at /tmp to
every workload to prevent crashloops from Python hypercorn/uvicorn worker temp
files. All other existing emptyDir volumes (logs, databases, nginx-cache,
nginx-run) preserved and unchanged.

values-local.yaml is intentionally left with readOnlyRootFilesystem: false for
local development convenience. All other environments (alpha via kustomize,
beta/prod via helm) are now hardened.

Runtime validation via alpha smoke-test is required before promoting to beta/prod.
Read-only root can surface unexpected write paths that are only detectable at
runtime — flag any crashloops in alpha for investigation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @Chromeninja, your pull request is larger than the review limit of 150000 diff characters

@socket-security

socket-security Bot commented Jul 25, 2026

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn Medium
Low adoption: npm @humanfs/types

Location: Package overview

From: admin/hub_module/frontend/package-lock.jsonnpm/eslint@9.39.2npm/@humanfs/types@0.15.0

ℹ Read more on: This package | This alert | What are unpopular packages?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Unpopular packages may have less maintenance and contain other problems.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/@humanfs/types@0.15.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Unmaintained: npm buffer-writer was last published 8 years ago

Last Publish: 11/7/2018, 2:17:24 PM

From: tests/integration/package-lock.jsonnpm/pg@8.11.3npm/buffer-writer@2.0.0

ℹ Read more on: This package | This alert | What are unmaintained packages?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Package should publish periodic maintenance releases if they are maintained, or deprecate if they have no intention in further maintenance.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/buffer-writer@2.0.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Unmaintained: npm json-stringify-safe was last published 11 years ago

Last Publish: 5/19/2015, 1:42:09 AM

From: tests/api/hub-backend/package-lock.jsonnpm/nock@14.0.16npm/json-stringify-safe@5.0.1

ℹ Read more on: This package | This alert | What are unmaintained packages?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Package should publish periodic maintenance releases if they are maintained, or deprecate if they have no intention in further maintenance.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/json-stringify-safe@5.0.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Filesystem access: npm package-json-from-dist with module node:fs

Module: node:fs

Location: Package overview

From: tests/api/hub-backend/package-lock.jsonnpm/jest@30.4.2npm/package-json-from-dist@1.0.1

ℹ Read more on: This package | This alert | What is filesystem access?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: If a package must read the file system, clarify what it will read and ensure it reads only what it claims to. If appropriate, packages can leave file system access to consumers and operate on data passed to it instead.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/package-json-from-dist@1.0.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Unmaintained: npm packet-reader was last published 7 years ago

Last Publish: 2/20/2019, 3:41:20 PM

From: tests/integration/package-lock.jsonnpm/pg@8.11.3npm/packet-reader@1.0.0

ℹ Read more on: This package | This alert | What are unmaintained packages?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Package should publish periodic maintenance releases if they are maintained, or deprecate if they have no intention in further maintenance.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/packet-reader@1.0.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Filesystem access: npm path-scurry with module fs

Module: fs

Location: Package overview

From: tests/api/hub-backend/package-lock.jsonnpm/jest@30.4.2npm/path-scurry@1.11.1

ℹ Read more on: This package | This alert | What is filesystem access?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: If a package must read the file system, clarify what it will read and ensure it reads only what it claims to. If appropriate, packages can leave file system access to consumers and operate on data passed to it instead.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/path-scurry@1.11.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Filesystem access: npm path-scurry with module node:fs

Module: node:fs

Location: Package overview

From: tests/api/hub-backend/package-lock.jsonnpm/jest@30.4.2npm/path-scurry@1.11.1

ℹ Read more on: This package | This alert | What is filesystem access?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: If a package must read the file system, clarify what it will read and ensure it reads only what it claims to. If appropriate, packages can leave file system access to consumers and operate on data passed to it instead.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/path-scurry@1.11.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Filesystem access: npm path-scurry with module node:fs/promises

Module: node:fs/promises

Location: Package overview

From: tests/api/hub-backend/package-lock.jsonnpm/jest@30.4.2npm/path-scurry@1.11.1

ℹ Read more on: This package | This alert | What is filesystem access?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: If a package must read the file system, clarify what it will read and ensure it reads only what it claims to. If appropriate, packages can leave file system access to consumers and operate on data passed to it instead.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/path-scurry@1.11.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn Low
Unmaintained: npm propagate was last published 7 years ago

Last Publish: 4/12/2019, 5:19:11 PM

From: tests/api/hub-backend/package-lock.jsonnpm/nock@14.0.16npm/propagate@2.0.1

ℹ Read more on: This package | This alert | What are unmaintained packages?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Package should publish periodic maintenance releases if they are maintained, or deprecate if they have no intention in further maintenance.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore npm/propagate@2.0.1. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Ignoring alerts on:

  • npm/@mswjs/interceptors@0.41.9
  • npm/nock@14.0.16
  • npm/js-yaml@4.2.0
  • npm/next@15.5.21

View full report

@Chromeninja Chromeninja self-assigned this Jul 25, 2026
- Updated Next.js and eslint-config-next to the latest versions in package.json.
- Added npm-audit.mjs script to automate npm audit checks for high and critical vulnerabilities.
- Created a GitHub Actions workflow (pr-validation.yml) for pull request validation, including Python unit tests and Node.js builds.
- Added ESLint configuration files for backend modules to enforce coding standards and prevent unused variables.
Comment thread action/pushing/mattermost_action_module/requirements.txt Fixed
Comment thread action/pushing/mattermost_action_module/requirements.txt Fixed
Comment thread action/pushing/mattermost_action_module/requirements.txt Fixed
Comment thread action/pushing/mattermost_action_module/requirements.txt Fixed
Comment thread action/pushing/mattermost_action_module/requirements.txt Fixed
Comment thread action/pushing/mattermost_action_module/requirements.txt Fixed
Comment thread action/pushing/lambda_action_module/requirements.txt Fixed
Comment thread action/pushing/gcp_functions_action_module/requirements.txt Fixed
Comment thread action/pushing/discord_action_module/requirements.txt Fixed
Comment thread action/pushing/mattermost_action_module/requirements.txt Fixed
- Added CSRF protection middleware to handle cookie-authenticated requests.
- Implemented double-submit token pattern and origin validation.
- Created tests for CSRF middleware to ensure proper functionality.
- Updated requirements for additional dependencies.
Comment thread admin/marketplace_module/backend/src/index.js Fixed
… consistency

chore: add .gitattributes to ensure executable POSIX shebang for gradlew
@Chromeninja

Copy link
Copy Markdown
Author

@SocketSecurity ignore npm/@mswjs/interceptors@0.41.9 npm/js-yaml@4.2.0 npm/next@15.5.21 npm/nock@14.0.16

ChromeNinja and others added 3 commits July 25, 2026 10:33
- Added gRPC container port (50030) to core-identity deployment and service configurations.
- Updated environment variables to include GRPC_PORT in core-identity.
- Modified SQL queries in sharding.py and command_processor.py to use community_servers table with status checks.
- Enhanced migration scripts to check for required tables and grant necessary permissions to credential_manager role.
- Introduced new RegisterPage and PasswordRecoveryUnavailablePage components for user registration and password recovery.
- Updated authentication tests to reflect changes in registration flow and UI elements.
- Added protobuf definitions and gRPC service implementations for identity management.
@Chromeninja
Chromeninja enabled auto-merge July 26, 2026 03:13
@Chromeninja
Chromeninja disabled auto-merge July 26, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants