Conversation
fix(security): hardening phase 2 (oracle schema refs + dependency cleanup)
fix(security): hardening phase 3 docker exposure and kafka tls plan
feat(security): add optional kafka sasl-ssl deployment mode
feat(security): automate kafka tls and scram bootstrap workflow
feat(security): add kafka secure-mode preflight and troubleshooting
feat(security): add kafka secure-mode smoke test and acceptance checklist
docs(security): publish hardening readiness report for findings 8-14
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Files Reviewed (15 files)
Verification
All security findings from #8-#14 are adequately addressed. Reviewed by minimax-m2.5-20260211 · 538,152 tokens |
There was a problem hiding this comment.
Pull request overview
Promotes the accumulated security-hardening work from develop to main, focusing on Kafka secure-mode rollout controls (SASL_SSL/TLS), tighter Docker runtime exposure defaults, and a formal readiness report for findings #8–#14.
Changes:
- Added Kafka secure-mode overlay (
docker-compose.secure.yml), Spring Kafka client security properties, and operational scripts (preflight/enable/smoke) to support SASL_SSL rollout. - Reduced accidental runtime exposure by binding docker-compose published ports to localhost by default and expanding
.gitignorerules for secret materials. - Added/updated readiness and operations documentation, plus Oracle init.sql parameterization and a pom cleanup (duplicate security dependency removal).
Reviewed changes
Copilot reviewed 19 out of 21 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
src/main/resources/application.yml |
Adds Spring Kafka security properties sourced from env for SASL/TLS rollout. |
scripts/kafka-secure-smoke.sh |
Adds a smoke test for secure-mode env + quick log scanning. |
scripts/kafka-secure-preflight.sh |
Adds preflight checks for required env vars and secret artifacts. |
scripts/kafka-generate-secrets.sh |
Generates local TLS keystores/truststores and helper credential files. |
scripts/kafka-enable-secure-mode.sh |
Automates the secure-mode enablement flow (generate, bootstrap, overlay). |
scripts/kafka-bootstrap-scram-users.sh |
Bootstraps SCRAM users in plaintext bootstrap phase via ZooKeeper. |
pom.xml |
Removes a duplicate spring-boot-starter-security dependency entry. |
docs/reports/SECURITY-HARDENING-READINESS.md |
Adds a readiness report and evidence checklist for findings #8–#14. |
docs/README.md |
Adds references to security report documents (EN/ES sections). |
docs/operations/README.md |
Adds index entries for Kafka secure-mode operational docs (EN/ES). |
docs/operations/kafka-tls-sasl-hardening-plan.md |
Documents an overall rollout plan for migrating off PLAINTEXT (finding #12). |
docs/operations/kafka-sasl-ssl-troubleshooting.md |
Adds troubleshooting guide for SASL/SSL failure patterns. |
docs/operations/kafka-sasl-ssl-quickstart.md |
Adds operator quickstart steps for enabling Kafka secure overlay. |
docs/operations/kafka-sasl-ssl-acceptance-checklist.md |
Adds acceptance checklist for operational closure of finding #12. |
docker/oracle/init.sql |
Parameterizes grants/schema usage via &MONITOR_APP_USER and removes hardcoded schema qualifiers. |
docker/kafka/secrets/README.md |
Documents expected local secret artifacts and commit guidance. |
docker/kafka/secrets/.gitkeep |
Keeps the secrets directory present without committing secrets. |
docker-compose.yml |
Binds published ports to 127.0.0.1 by default (override via *_BIND_HOST). |
docker-compose.secure.yml |
Introduces SASL_SSL listener configuration and mounts TLS materials for broker/clients. |
.gitignore |
Ignores Kafka secret artifacts while allowing tracked placeholders/docs in the secrets dir. |
.env.example |
Adds example env vars for Kafka secure-mode overlay and secret passwords. |
| KAFKA_INTER_BROKER_LISTENER_NAME: SASL_SSL | ||
| KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512 | ||
| KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: SCRAM-SHA-512 | ||
| KAFKA_SSL_CLIENT_AUTH: required |
There was a problem hiding this comment.
In the secure overlay the broker sets KAFKA_SSL_CLIENT_AUTH: required, but neither kafka-connect nor backend are configured with client keystores/certs (only truststores). With required, TLS handshakes will fail unless you also provision and configure client keystores, or change this to none/requested for SASL/SCRAM-only auth.
| KAFKA_SSL_CLIENT_AUTH: required | |
| KAFKA_SSL_CLIENT_AUTH: none |
| environment: | ||
| KAFKA_LISTENERS: SASL_SSL://0.0.0.0:29092,SASL_SSL_HOST://0.0.0.0:9092 | ||
| KAFKA_ADVERTISED_LISTENERS: SASL_SSL://kafka:29092,SASL_SSL_HOST://localhost:9092 | ||
| KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SASL_SSL:SASL_SSL,SASL_SSL_HOST:SASL_SSL | ||
| KAFKA_INTER_BROKER_LISTENER_NAME: SASL_SSL | ||
| KAFKA_SASL_ENABLED_MECHANISMS: SCRAM-SHA-512 |
There was a problem hiding this comment.
The base docker-compose.yml Kafka healthcheck uses kafka-topics --bootstrap-server localhost:9092 --list (PLAINTEXT). With this overlay, 9092 becomes SASL_SSL, so the inherited healthcheck will likely fail and keep kafka unhealthy (blocking kafka-connect which depends on service_healthy). Consider overriding the healthcheck here with a TLS/SASL-aware check (or a simple port-open check) for secure mode.
| properties: | ||
| security.protocol: ${KAFKA_SECURITY_PROTOCOL:PLAINTEXT} | ||
| sasl.mechanism: ${KAFKA_SASL_MECHANISM:} | ||
| sasl.jaas.config: ${KAFKA_SASL_JAAS_CONFIG:} |
There was a problem hiding this comment.
security.protocol defaults to PLAINTEXT here. The PR description’s production condition says to remove PLAINTEXT listener usage from production runtime configuration; to enforce that, consider overriding this in the prod profile to default to SASL_SSL (or fail fast by omitting a default) so production can’t silently run in PLAINTEXT if env vars are missing.
| sasl.jaas.config: ${KAFKA_SASL_JAAS_CONFIG:} | ||
| ssl.truststore.location: ${KAFKA_SSL_TRUSTSTORE_LOCATION:} | ||
| ssl.truststore.password: ${KAFKA_SSL_TRUSTSTORE_PASSWORD:} | ||
| ssl.endpoint.identification.algorithm: ${KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM:HTTPS} |
There was a problem hiding this comment.
ssl.endpoint.identification.algorithm defaults to HTTPS, which enforces hostname verification. With the current kafka-generate-secrets.sh self-signed cert (CN=monitor-kafka, no SANs), clients connecting to kafka / localhost will likely fail hostname verification. Either generate certificates with appropriate SANs (preferred) or document/override this default for the local secure overlay.
| ssl.endpoint.identification.algorithm: ${KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM:HTTPS} | |
| # Default to disabled for local self-signed/no-SAN certificates; set to HTTPS in environments with SAN-valid certs. | |
| ssl.endpoint.identification.algorithm: ${KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM:} |
| echo "[1/5] Generating broker keystore" | ||
| keytool -genkeypair \ | ||
| -alias kafka-broker \ | ||
| -keyalg RSA \ | ||
| -keysize 2048 \ | ||
| -validity "$VALIDITY_DAYS" \ | ||
| -dname "$DNAME" \ | ||
| -keystore "$SECRETS_DIR/kafka.keystore.jks" \ | ||
| -storepass "$STOREPASS" \ | ||
| -keypass "$STOREPASS" \ | ||
| -storetype JKS |
There was a problem hiding this comment.
The generated broker certificate does not include Subject Alternative Names (SAN). Modern Java TLS stacks typically require SAN for hostname verification, and your compose advertises kafka / localhost. Update the keytool -genkeypair invocation to include SANs for kafka, localhost, and 127.0.0.1 (or make SANs configurable) so SASL_SSL works with hostname verification enabled.
| echo "Step 1/4: generate TLS materials" | ||
| ./scripts/kafka-generate-secrets.sh | ||
|
|
||
| echo "Step 2/4: start base stack (plaintext bootstrap)" | ||
| docker compose up -d kafka zookeeper | ||
|
|
||
| echo "Step 3/4: create SCRAM users" | ||
| ./scripts/kafka-bootstrap-scram-users.sh | ||
|
|
||
| echo "Step 4/5: preflight check" | ||
| ./scripts/kafka-secure-preflight.sh | ||
|
|
||
| echo "Step 5/5: apply secure override" | ||
| docker compose -f docker-compose.yml -f docker-compose.secure.yml up -d kafka kafka-connect backend |
There was a problem hiding this comment.
The step counters are inconsistent (Step 1/4 … Step 3/4, then Step 4/5 / Step 5/5). This is minor but confusing when operators follow the output; make the numbering consistent.
| ### Resumen Ejecutivo | ||
|
|
||
| Esta carpeta es el punto de entrada operativo para validar y diagnosticar `Monitor`. | ||
| Esta pensada para responder rapidamente una pregunta: **�la plataforma esta sana, observable y conectada de extremo a extremo?** | ||
| Esta pensada para responder rapidamente una pregunta: **�la plataforma esta sana, observable y conectada de extremo a extremo?** | ||
|
|
There was a problem hiding this comment.
The Spanish text contains mojibake/encoding artifacts (e.g., **�la plataforma...** should use ¿), and is missing accents (e.g., está). Since this line was modified, please correct the punctuation/accents so the bilingual doc renders correctly.
| - `kafka-sasl-ssl-quickstart.md` - habilitacion de Kafka seguro (TLS/SASL) con scripts de bootstrap | ||
| - `kafka-sasl-ssl-troubleshooting.md` - guia de diagnostico para fallas de Kafka en modo seguro | ||
| - `kafka-sasl-ssl-acceptance-checklist.md` - checklist de aceptacion para cierre operativo de Kafka seguro |
There was a problem hiding this comment.
The newly added Spanish index entries are missing accents (habilitacion, guia, aceptacion). Please update to proper Spanish spelling (habilitación, guía, aceptación) for consistency with bilingual docs.
| - `kafka-sasl-ssl-quickstart.md` - habilitacion de Kafka seguro (TLS/SASL) con scripts de bootstrap | |
| - `kafka-sasl-ssl-troubleshooting.md` - guia de diagnostico para fallas de Kafka en modo seguro | |
| - `kafka-sasl-ssl-acceptance-checklist.md` - checklist de aceptacion para cierre operativo de Kafka seguro | |
| - `kafka-sasl-ssl-quickstart.md` - habilitación de Kafka seguro (TLS/SASL) con scripts de bootstrap | |
| - `kafka-sasl-ssl-troubleshooting.md` - guía de diagnostico para fallas de Kafka en modo seguro | |
| - `kafka-sasl-ssl-acceptance-checklist.md` - checklist de aceptación para cierre operativo de Kafka seguro |
| #### Reportes de Seguridad | ||
|
|
||
| - `docs/reports/SECURITY-AUDIT.md` - evaluacion base de seguridad y catalogo de hallazgos | ||
| - `docs/reports/SECURITY-HARDENING-READINESS.md` - estado actual de cierre/readiness para hallazgos #8-#14 | ||
|
|
There was a problem hiding this comment.
The Spanish bullets added here are missing accents (evaluacion, catalogo) and mix English (readiness) into Spanish. Please correct accents and consider a consistent Spanish term (e.g., preparación/alistamiento) so this section reads cleanly.
| ## 4) Verify secure mode | ||
| - Kafka broker listener map includes `SASL_SSL` | ||
| - Backend starts with `KAFKA_SECURITY_PROTOCOL=SASL_SSL` | ||
| - Debezium connect worker starts with SASL settings | ||
|
|
There was a problem hiding this comment.
The quickstart doesn’t mention hostname verification constraints. With the current secret generation (self-signed cert without SANs) and ssl.endpoint.identification.algorithm=HTTPS default, secure mode may fail with hostname/handshake errors unless cert SANs are added or endpoint identification is overridden. Please add a note here (or adjust the scripts) so the documented flow works end-to-end.
fix(security): stabilize kafka secure bootstrap readiness
|
Security evidence update (post phase9):
Key fixes validated:
|
Release Promotion: develop -> main
Scope
Promote the accumulated security hardening work into main.
Included stream
Readiness reference
See docs/reports/SECURITY-HARDENING-READINESS.md
Release decision
Condition to fully close finding #12 in production:
Evidence checklist (attach in PR comments)