Update dependencies, refactor Dockerfiles, and enhance security - #154
Update dependencies, refactor Dockerfiles, and enhance security#154Chromeninja wants to merge 26 commits into
Conversation
- Updated gRPC and psycopg2-binary versions in requirements.txt for video_proxy_module and workflow_core_module. - Refactored Dockerfiles for workflow_core_module, video_proxy_module, presence_module, googlechat_module, mattermost_module, and teams_module to maintain consistency and improve readability. - Removed credential-manager service from docker-compose.yml. - Adjusted port mappings and healthcheck URLs in docker-compose.yml for various services. - Added .env.local.example file for ai_interaction_module to provide a template for local development environment variables.
Removed deprecated docker-compose.updated.yml and docker-compose.yml.backup. Updated test-deployment.sh to reference Kubernetes and make dev instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaced .env.example (131 lines, minimal) with content from .env-example (380 lines, annotated with REQUIRED/OPTIONAL flags and documentation). Deleted .env-example as .env.example is now the canonical file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removed legacy archive directory containing outdated module implementations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…root compose as deprecated
Deleted docker-compose files from module directories:
- action/pushing/{discord,youtube,lambda,twitch,slack}_action_module/docker-compose.*
- config/observability/docker-compose.observability.yml
Root docker-compose.yml already annotated with deprecation notice directing to K8s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e/ and helm/) Deleted outdated raw Kubernetes manifests directory. Modern deployments use: - k8s/kustomize/ for alpha (MicroK8s) - k8s/helm/ for beta and production Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ards) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…; add SECRETS_SETUP.md Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nly targets - test-integration: call scripts/test-api-all.sh - test-e2e: call scripts/e2e-test-alpha.sh - test-functional: error message (not yet implemented) - smoke-test: call tests/alpha-smoke-test.sh - seed-mock-data: call scripts/seed-admin.sh - deploy-dev: call scripts/deploy-alpha.sh - deploy-prod: error message (CI-only) - docker-push: error message (CI-only) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uild sources Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… Debian bookworm variants - PostgreSQL: 16/15 → 16/15-bookworm@sha256 - Redis: 7-alpine → 7-bookworm@sha256 - MinIO, Ollama, Qdrant: latest → latest@sha256 - Updated Helm values (values.yaml, values-beta.yaml, values-local.yaml) - Updated Kustomize components and alpha overlay imagePatches - Updated k8s/infrastructure.yaml reference manifests - All external images now pinned to multi-arch manifest digests - No Alpine images remain in K8s manifests Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n core-identity gRPC handler Three critical security bugs fixed in IdentityServiceServicer: 1. verify_token(): replaced auth bypass (any non-empty string accepted) with real JWT verification via flask_core.auth.verify_jwt_token (PyJWT HS256). Adds mandatory tenant claim check — tokens without a non-empty tenant are rejected. Secret key read from SECRET_KEY env var / Config.SECRET_KEY; never hardcoded. 2. LookupIdentity(): replaced hardcoded hub_user_id=1/username="test_user" with real DB query against hub_user_identities JOIN hub_users. Returns NOT_FOUND when identity does not exist. Validates platform and platform_user_id required. 3. GetLinkedPlatforms(): replaced hardcoded static [twitch, discord] list with real DB query against hub_user_identities WHERE hub_user_id = %s, ordered by is_primary DESC, linked_at ASC. Returns empty list (not error) for no platforms. Both copies fixed identically: - core/identity_core_module/services/grpc_handler.py (canonical — built by Dockerfile) - services/core-identity/identity_core_module/services/grpc_handler.py (service mirror) JWT util reused: flask_core.auth.verify_jwt_token (libs/flask_core, already installed in container via Dockerfile). DB pattern mirrors security_service.py executesql usage. Tests: core/identity_core_module/services/test_grpc_handler.py 15 tests, 15 passed — covers empty/None token, tampered sig, expired, missing tenant, empty tenant, missing params, identity not found, DB-backed lookup, empty platforms, DB-backed platform list. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Any authenticated user could previously call POST /api/v1/payments/refunds and refund any transaction — no ownership or role check existed (TODO comment left since initial implementation). Fixed two gaps: 1. Route lacked requireAuth middleware; unauthenticated callers had full access. 2. Controller had no ownership or role check before delegating to paymentService. Fix: add requireAuth to the route, then in the controller verify the caller is either a super_admin/platform-admin or the original payment owner (payment.metadata.userId) before proceeding. Returns 403 otherwise. Tests added covering: unauthenticated (403), non-owner non-admin (403), payment owner (200), super_admin (200), platform-admin (200), missing payment (404). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes commit 1bb601b — the refund authorization test added there requires this jest.config.js to run, and package-lock.json is tracked per dependency-pinning standards. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…p silently dropping official emotes - Implements OAuth client-credentials flow to fetch Twitch app access tokens - Caches tokens with expiry tracking, refreshes when < 60s remaining - Logs clear warning if credentials unconfigured (graceful degradation) - Logs errors on network/auth failures (no silent failures) - Tests: 7 unit tests covering all paths (token fetch, cache, refresh, errors) - Applied identically to both files across container build trees (router_module & translate_interaction_module) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…per dependency-pinning standard - Pin 81 ^/~ ranges across 9 package.json files - Create 4 new package-lock.json for test directories - Add reusable scripts/pin-npm-versions.py for future maintenance - All dependencies now use exact versions for supply chain security Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ir volumes across all deployments Set readOnlyRootFilesystem: true on all 23 Kustomize workloads and all 19 Helm chart deployments (via values.yaml securityContext — single source of truth for templates using toYaml). Added emptyDir volume named `tmp` mounted at /tmp to every workload to prevent crashloops from Python hypercorn/uvicorn worker temp files. All other existing emptyDir volumes (logs, databases, nginx-cache, nginx-run) preserved and unchanged. values-local.yaml is intentionally left with readOnlyRootFilesystem: false for local development convenience. All other environments (alpha via kustomize, beta/prod via helm) are now hardened. Runtime validation via alpha smoke-test is required before promoting to beta/prod. Read-only root can surface unexpected write paths that are only detectable at runtime — flag any crashloops in alpha for investigation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Sorry @Chromeninja, your pull request is larger than the review limit of 150000 diff characters
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
Ignoring alerts on:
|
- Updated Next.js and eslint-config-next to the latest versions in package.json. - Added npm-audit.mjs script to automate npm audit checks for high and critical vulnerabilities. - Created a GitHub Actions workflow (pr-validation.yml) for pull request validation, including Python unit tests and Node.js builds. - Added ESLint configuration files for backend modules to enforce coding standards and prevent unused variables.
- Added CSRF protection middleware to handle cookie-authenticated requests. - Implemented double-submit token pattern and origin validation. - Created tests for CSRF middleware to ensure proper functionality. - Updated requirements for additional dependencies.
… consistency chore: add .gitattributes to ensure executable POSIX shebang for gradlew
|
@SocketSecurity ignore npm/@mswjs/interceptors@0.41.9 npm/js-yaml@4.2.0 npm/next@15.5.21 npm/nock@14.0.16 |
- Added gRPC container port (50030) to core-identity deployment and service configurations. - Updated environment variables to include GRPC_PORT in core-identity. - Modified SQL queries in sharding.py and command_processor.py to use community_servers table with status checks. - Enhanced migration scripts to check for required tables and grant necessary permissions to credential_manager role. - Introduced new RegisterPage and PasswordRecoveryUnavailablePage components for user registration and password recovery. - Updated authentication tests to reflect changes in registration flow and UI elements. - Added protobuf definitions and gRPC service implementations for identity management.
This update includes dependency upgrades, Dockerfile refactoring for consistency, and security enhancements such as real JWT verification and authorization checks. It also consolidates environment variable files and removes deprecated components to streamline the project structure.