Skip to content

Development #728

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jbpenrath merged 9 commits into from
Jul 6, 2026
Merged

Development #728

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
76dc87f
👔(backend) add X-Mailer header at outbound
jbpenrath Jun 25, 2026
a1ffcf3
🔥(frontend) remove react-email component
jbpenrath Jun 25, 2026
632f38d
✨(mta-in) rewrite MTA-in in pure Python to remove dep on Postfix (#692)
sylvinus Jun 29, 2026
d8fb367
🚨(lint) lint subprojects
sylvinus Jun 29, 2026
114e537
👔(backend) use builtin method to generate message-id (#730)
jbpenrath Jun 29, 2026
23a63d9
✨(inbound) add webhooks, message postmarks, spam fixes
sylvinus Jul 1, 2026
85dd0b4
🐛(pymta) save the origin IP across STARTTLS restarts
sylvinus Jul 1, 2026
109211e
✨(ssrf) add a host allowlist to bypass SSRF in internal networks
sylvinus Jul 1, 2026
c668326
🐛(inbound) improve re-processing of inboundmessages from the admin
sylvinus Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 26 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ create-env-files: \
env.d/development/backend.local \
env.d/development/frontend.local \
env.d/development/mta-in.local \
env.d/development/mta-in-py.local \
env.d/development/mta-out.local \
env.d/development/socks-proxy.local
.PHONY: create-env-files
Expand Down Expand Up @@ -127,6 +128,17 @@ test-back-distroless: build-back-distroless ## build and smoke-test the distrole
print(f'OK: Python {sys.version.split()[0]}, {ssl.OPENSSL_VERSION}')"
.PHONY: test-back-distroless

build-pymta-distroless: ## build the pymta distroless production image
@docker build --target runtime-distroless-prod -t messages-pymta-distroless -f src/mta-in/Dockerfile.pymta src/mta-in/
.PHONY: build-pymta-distroless

test-pymta-distroless: build-pymta-distroless ## build and smoke-test the pymta distroless production image
@docker run --rm messages-pymta-distroless python -c " \
import sys, ssl; \
import pymta.settings; \
print(f'OK: Python {sys.version.split()[0]}, {ssl.OPENSSL_VERSION}, pymta.settings loaded')"
.PHONY: test-pymta-distroless

down: ## stop and remove containers, networks, images, and volumes
@$(COMPOSE) down
.PHONY: down
Expand Down Expand Up @@ -181,6 +193,7 @@ lint: \
lint-front \
typecheck-front \
lint-mta-in \
lint-mta-in-py \
lint-mta-out
.PHONY: lint

Expand Down Expand Up @@ -228,12 +241,17 @@ lint-front: ## run the frontend linter
@$(COMPOSE) run --rm frontend-tools npm run lint
.PHONY: lint-front

lint-mta-in: ## lint mta-in python sources
lint-mta-in: ## lint mta-in python sources (Postfix milter implementation)
$(COMPOSE_RUN) --rm -e EXEC_CMD_ONLY=true mta-in-test ruff format .
#$(COMPOSE_RUN) --rm -e EXEC_CMD_ONLY=true mta-in-test ruff check . --fix
#$(COMPOSE_RUN) --rm -e EXEC_CMD_ONLY=true mta-in-test pylint .
.PHONY: lint-mta-in

lint-mta-in-py: ## lint mta-in python sources (pure-Python pymta implementation)
$(COMPOSE_RUN) --rm -e EXEC_CMD_ONLY=true mta-in-py-test ruff format .
$(COMPOSE_RUN) --rm -e EXEC_CMD_ONLY=true mta-in-py-test ruff check . --fix
.PHONY: lint-mta-in-py

lint-mta-out: ## lint mta-out python sources
$(COMPOSE_RUN) --rm -e EXEC_CMD_ONLY=true mta-out-test ruff format .
.PHONY: lint-mta-out
Expand All @@ -245,6 +263,7 @@ test: \
test-back \
test-front \
test-mta-in \
test-mta-in-py \
test-mta-out \
test-mpa \
test-socks-proxy
Expand Down Expand Up @@ -285,10 +304,14 @@ test-front-amd64: ## run the frontend tests in amd64
$(COMPOSE) run --rm frontend-tools-amd64 npm run test -- $${args:-${1}}
.PHONY: test-front-amd64

test-mta-in: ## run the mta-in tests
test-mta-in: ## run the mta-in tests against the Postfix milter implementation
@$(COMPOSE) run --build --rm mta-in-test
.PHONY: test-mta-in

test-mta-in-py: ## run the mta-in tests against the pure-Python (aiosmtpd) implementation
@$(COMPOSE) run --build --rm mta-in-py-test
.PHONY: test-mta-in-py

test-mta-out: ## run the mta-out tests
@$(COMPOSE) run --build --rm mta-out-test
.PHONY: test-mta-out
Expand Down Expand Up @@ -630,7 +653,7 @@ test-keycloak: ## run all Keycloak provider tests (builds JARs, brings up Keyclo
@bin/test-keycloak
.PHONY: test-keycloak

deps-lock-mta-in: ## lock the dependencies
deps-lock-mta-in: ## lock the dependencies for mta-in (shared between both implementations)
@$(COMPOSE) run --rm --build mta-in-uv uv lock
.PHONY: deps-lock-mta-in

Expand Down
67 changes: 67 additions & 0 deletions compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,8 @@ services:
- EXEC_CMD=true
- MDA_API_BASE_URL=http://localhost:8000/api/mail/
- MTA_HOST=localhost
- MTA_PORT=25
- MTA_IMPL=postfix
command: pytest -vvs tests/
volumes:
- ./src/mta-in:/app
Expand All @@ -277,6 +279,71 @@ services:
target: uv
pull_policy: build

# ---- Pure-Python (aiosmtpd) inbound MTA --------------------------------
# Runs side-by-side with the Postfix-based `mta-in` service on a different
# host port (8920 vs 8910). Both implementations share the same MDA
# contract, env vars, and test suite. Toggle which one is the public-facing
# MTA at the edge by switching the upstream pool.

mta-in-py:
build:
context: src/mta-in
dockerfile: Dockerfile.pymta
target: runtime-distroless-prod
args:
DOCKER_USER: ${DOCKER_USER:-65532}
user: ${DOCKER_USER:-65532}
env_file:
- env.d/development/mta-in.defaults
- env.d/development/mta-in.local
- env.d/development/mta-in-py.defaults
- env.d/development/mta-in-py.local
ports:
- "8920:25"
- "9120:9100" # Prometheus metrics
# Defence-in-depth: pymta needs no on-disk writes at runtime. Read-only
# rootfs + dropped capabilities + no-new-privileges mirror the posture
# a production k8s pod-spec should run with.
read_only: true
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
tmpfs:
- /tmp:rw,noexec,nosuid,size=16m
depends_on:
- backend-dev

mta-in-py-test:
profiles:
- tools
build:
context: src/mta-in
dockerfile: Dockerfile.pymta
target: runtime-dev
args:
DOCKER_USER: ${DOCKER_USER:-65532}
user: ${DOCKER_USER:-65532}
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
env_file:
- env.d/development/mta-in.defaults
- env.d/development/mta-in.local
- env.d/development/mta-in-py.defaults
- env.d/development/mta-in-py.local
environment:
- EXEC_CMD=true
- MDA_API_BASE_URL=http://localhost:8000/api/mail/
- MTA_HOST=localhost
- MTA_PORT=25
- MTA_IMPL=pymta
- MTA_METRICS_URL=http://localhost:9100/metrics
command: pytest -vvs tests/
volumes:
- ./src/mta-in:/app

Comment thread
coderabbitai[bot] marked this conversation as resolved.
Comment on lines +288 to +346

@coderabbitai coderabbitai Bot Jul 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== Files of interest ==\n'
git ls-files | rg '^(compose\.yaml|src/mta-in/|env\.d/development/mta-in-py\.|env\.d/development/mta-in\.)'

printf '\n== compose.yaml around mta-in-py ==\n'
python3 - <<'PY'
from pathlib import Path
p = Path('compose.yaml')
lines = p.read_text().splitlines()
for start, end in [(260, 360)]:
    print(f'--- {p}:{start}-{end} ---')
    for i, line in enumerate(lines[start-1:end], start):
        print(f'{i:4d}: {line}')
PY

printf '\n== Dockerfile.pymta outline ==\n'
if command -v ast-grep >/dev/null 2>&1; then
  ast-grep outline src/mta-in/Dockerfile.pymta || true
else
  sed -n '1,240p' src/mta-in/Dockerfile.pymta
fi

printf '\n== Search for port config / 25 / 2525 / PYMTA_SMTP_PORT ==\n'
rg -n 'PYMTA_SMTP_PORT|2525|:25\b|port\s*=?\s*25|SMTP_PORT|bind.*25|listen.*25' src/mta-in env.d/development compose.yaml

Repository: suitenumerique/messages

Length of output: 7494

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path

files = [
    'src/mta-in/Dockerfile.pymta',
    'src/mta-in/entrypoint.pymta.sh',
    'src/mta-in/src/pymta/settings.py',
    'src/mta-in/src/pymta/server.py',
    'src/mta-in/README.md',
    'env.d/development/mta-in-py.defaults',
]
for file in files:
    p = Path(file)
    print(f'\n== {file} ({sum(1 for _ in p.open())} lines) ==')
    lines = p.read_text().splitlines()
    # print focused slices where useful
    if file.endswith('Dockerfile.pymta'):
        ranges = [(1, 240)]
    elif file.endswith('entrypoint.pymta.sh'):
        ranges = [(1, 220)]
    elif file.endswith('settings.py'):
        ranges = [(1, 160)]
    elif file.endswith('server.py'):
        ranges = [(1, 180)]
    elif file.endswith('README.md'):
        ranges = [(50, 90)]
    else:
        ranges = [(1, 80)]
    for start, end in ranges:
        print(f'--- {file}:{start}-{end} ---')
        for i, line in enumerate(lines[start-1:end], start):
            print(f'{i:4d}: {line}')
PY

Repository: suitenumerique/messages

Length of output: 26369

Restore a portable bind path for mta-in-py

mta-in-py and mta-in-py-test run as uid 65532 with cap_drop: ALL, but they still default to SMTP port 25 and grant no NET_BIND_SERVICE. That makes startup depend on the runtime’s privileged-port sysctl. Add cap_add: NET_BIND_SERVICE or move the container listener to an unprivileged port and remap it from Compose.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@compose.yaml` around lines 288 - 346, The mta-in-py and mta-in-py-test
services still rely on binding to privileged port 25 while running as uid 65532
with cap_drop: ALL, so startup depends on host sysctl behavior. Update the
Compose configuration for these services by either adding NET_BIND_SERVICE to
their capabilities or changing the container-side SMTP listener to an
unprivileged port and keeping the host port mapping in place. Use the mta-in-py
and mta-in-py-test service definitions to make the change consistently.

mta-out:
build:
context: src/mta-out
Expand Down
Loading
Loading