diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e7f401e..ba30584 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -9,7 +9,13 @@ Thanks for contributing. This project is security-adjacent; changes should be de git checkout -b feature/ ``` -2. Make changes with well described commits. (Run formatting and basic checks.) +2. Make changes with well described commits. Before opening the PR, run: + ```bash + gofmt -l . # must print nothing + go vet ./... + go test ./... + ./build-tailwind.sh # only if you touched templates or Tailwind classes + ``` 3. Open a pull request: @@ -23,6 +29,7 @@ Thanks for contributing. This project is security-adjacent; changes should be de * Keep public-facing behavior documented (docs or inline help). * Do not introduce new external network calls by default; make them opt-in where possible. * For changes affecting auth or callback handling, include a short threat-model note in the PR description. +* To add a UI language, copy `pkg/web/locales/en.json`, translate every value, and save it as `pkg/web/locales/.json` with a lowercase locale code (for example `ch`, `ch_de`, `es`, `pt_br`). ## Reporting security issues diff --git a/README.md b/README.md index 40d21c5..a8fd782 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ Fail2Ban UI does not replace Fail2Ban. Ban decisions are still made by the Fail2 * Centralized search, ban, and unban operations across all jails and servers * Remote jail and filter configuration management (depending on connector capabilities) * Filter debugging with live log-pattern testing -* Ban insights, including country-level analytics on an interactive 3D globe +* Ban insights with a Kibana-like ban/unban timeline, incident comparison to isolate repeat attackers, one-click permanent blocking of a whole time range, and country-level analytics * Recurring-offender handling with permanent blocks on MikroTik, pfSense, and OPNsense * Persistent event history and permanent-block records, with data management built in * Configurable alerts over Email (SMTP), Webhook, and Elasticsearch, with GeoIP/Whois enrichment and country filtering @@ -44,7 +44,7 @@ See [docs/architecture.md](docs/architecture.md) for the data-flow description. | Connector | Typical use | Notes | |-----------|-------------|-------| | Local | Fail2Ban runs on the same host as the UI | Uses the Fail2Ban Unix socket and local files | -| SSH | Manage remote hosts without installing an agent | Key-based SSH and remote `fail2ban-client`; requires a dedicated service account with minimal sudo rules | +| SSH | Manage remote hosts without installing an agent | Key-based SSH and remote `fail2ban-client`; requires a dedicated service account with minimal sudo rules. Can optionally open a reverse SSH tunnel so ban callbacks reach the UI from hosts that cannot connect to it directly (NAT, isolated DMZ) | | Agent | Environments where SSH from the UI host is not desired | HTTP agent runs on the Fail2Ban host; see [fail2ban-ui-agent](https://github.com/swissmakers/fail2ban-ui-agent) and the prebuilt image [swissmakers/fail2ban-ui-agent](https://hub.docker.com/r/swissmakers/fail2ban-ui-agent) | ## Quick start (container) @@ -124,7 +124,7 @@ Overview of all active jails, banned IPs, and real-time statistics, total bans, #### Unban IP ![Unban IP](screenshots/0.1_Dashboard_unban_IP.png) -Unbanning a IP addresses directly from the dashboard. Shows the unban confirmation dialog. +Unban an IP address directly from the dashboard. A confirmation dialog shows the affected jail before the unban runs. ### Server Management ![Manage Servers](screenshots/1_Dashboard_Manage_Servers.png) @@ -136,11 +136,11 @@ Overview of all configured jails with their enabled/disabled status. Allows cent #### Edit Jail Configuration ![Edit Jail](screenshots/1.2_Dashboard_Manage_Jails_Edit.png) -When clicking on "Edit Filter / Jail" the Jail configuration editor is opened. It shows the current filter and jail configuration with all options to modify the settings, test or add / modify the logpaths, and save changes. +Clicking **Edit Filter / Jail** opens the configuration editor. Edit the jail and filter settings, add or change log paths and test them, then save -> the change is pushed to the managed host and Fail2Ban is reloaded. #### Logpath Test ![Logpath Test](screenshots/1.3_Dashboard_Manage_Jails_Edit_Logpathtest.png) -Logpath testing functionality that verifies log file paths and checks if files are accessible. Shows test results with visual indicators (yes/no) for each log path. +Logpath testing resolves the configured paths, including wildcards and Fail2Ban variables, and reports one of three results per path: found (green, with the matched files), not found (red), or cannot verify (yellow). The yellow state means the connector's SSH user cannot read the log directory. It is a warning, not a failure. Fail2Ban runs as root on the managed host and will still read the path, so the jail can be enabled. #### Create new Filter ![Create Filter](screenshots/1.4_Dashboard_Manage_Jails_Create_Filter.png) @@ -152,7 +152,7 @@ The second button opens the jail creation modal for setting up new jails. It sup ### Search Functionality ![Search](screenshots/1.6_Dashboard_search.png) -Search for a specific IPs, that where blocked in a specific jail, searches in all active jails. Provides a quick and painless filtering. +Search for an IP address across all active jails of every configured server. The result lists each jail that currently bans the address, so it can be unbanned in the right place. ### Internal Log Overview ![Log Overview](screenshots/2_Dashboard_Log_Overview.png) @@ -166,6 +166,19 @@ Whois lookup modal displaying detailed information about banned IP addresses, in ![Ban Logs](screenshots/2.2_Dashboard_Log_Overview_BanLogs.png) Detailed ban log view showing log lines that triggered the ban, timestamps, and context information for each security event. +#### Ban Insights + +[![Ban Insights](screenshots/2.3_Dashboard_Log_Overview_BanInsights.png)](screenshots/2.3_Dashboard_Log_Overview_BanInsights.png) + +Ban Insights brings SIEM-like analysis to environments without a full SIEM. The ban and unban activity timeline works similar like the one in Kibana -> drag to zoom into a time range, use presets from 8 hours to 30 days, or define a custom range of up to 12 months, so spikes and attack patterns become visible at a glance. + +Typical workflows: + +* **Correlate two incidents.** Pin one spike as Incident A and another as Incident B -> for example, last week's attack and this week's. The incident compare lists the IPs present in both. Addresses that return across incidents are repeat attackers rather than false positives, and can be banned permanently right to the recurring-offenders list. +* **Block a botnet in one click.** Select a time range in which no legitimate user is normally active, such as a nightly spike, and permanently block every IP in it on the configured firewall integration (MikroTik, pfSense, OPNsense). + +The modal also suggests similar past periods, exports IP lists as CSV or JSON, and shows country-level analytics on an interactive 3D globe together with the top recurring IPs. + ### Filter Debugging ![Filter Debug](screenshots/3_Filter_Debug.png) Filter debugging interface for testing Fail2Ban filter regex patterns against log lines. Helps validate filter configurations before deployment. @@ -180,7 +193,7 @@ Main settings page with sections for different configuration categories includin #### Debug Console ![Debug Console](screenshots/4.1_Settings_DebugConsole.png) -When enabled the Debug console showing real-time application logs, system messages, and debugging information. Useful for troubleshooting and monitoring without the need to query the container logs manually everytime. +When enabled, the debug console streams the live application log into the browser -> useful for troubleshooting without querying the container logs manually. Leave it disabled in normal operation. #### Advanced ban actions diff --git a/deployment/container/README.md b/deployment/container/README.md index c5306dd..61ffdad 100644 --- a/deployment/container/README.md +++ b/deployment/container/README.md @@ -6,22 +6,23 @@ This guide covers building and running Fail2Ban UI with Podman or Docker. ### Pull the image -From Docker Hub (default): +The image is published to three registries. Use whichever your environment prefers: ```bash +# Docker Hub (default) podman pull swissmakers/fail2ban-ui:latest -# or with Docker: -docker pull swissmakers/fail2ban-ui:latest -``` -From the Swissmakers registry (fallback): +# GitHub Container Registry +podman pull ghcr.io/swissmakers/fail2ban-ui:latest -```bash +# Swissmakers registry (fallback) podman pull registry.swissmakers.ch/infra/fail2ban-ui:latest -# or with Docker: -docker pull registry.swissmakers.ch/infra/fail2ban-ui:latest ``` +Replace `podman` with `docker` if you use Docker. + +All three receive the same multi-arch manifest (`linux/amd64` and `linux/arm64`) from the same CI build, and each push is tagged three ways: `latest`, the release version (for example `v1.5.3`), and the build commit SHA. Pin to a version tag for reproducible deployments. The images carry OCI metadata labels (`org.opencontainers.image.version`, `.revision`, `.created`, `.source`, `.licenses`), so `podman inspect` or `docker inspect` tells you exactly which build you are running. + ### Run the container ```bash @@ -206,6 +207,9 @@ services: # Pre-built image from Docker Hub (default) image: swissmakers/fail2ban-ui:latest + # Alternative: GitHub Container Registry + # image: ghcr.io/swissmakers/fail2ban-ui:latest + # Alternative: Swissmakers registry (fallback) # image: registry.swissmakers.ch/infra/fail2ban-ui:latest diff --git a/development/README.md b/development/README.md index 94b0b25..3d5dc14 100644 --- a/development/README.md +++ b/development/README.md @@ -13,7 +13,7 @@ Complete OIDC authentication setup with Keycloak and Pocket-ID for testing authe ### 2. For SSH, Local and Agent connector testing (`ssh_and_local/`) Stack for testing Fail2ban UI with: -- Local Fail2ban instance (container) +- Local Fail2ban instances (two containers: primary on the host network, secondary on bridge) - Remote Fail2ban instance via SSH (container) - **Fail2ban Agent:** prebuilt image [`swissmakers/fail2ban-ui-agent`](https://hub.docker.com/r/swissmakers/fail2ban-ui-agent) (sources: [`github.com/swissmakers/fail2ban-ui-agent`](https://github.com/swissmakers/fail2ban-ui-agent)) @@ -40,7 +40,7 @@ Stack for testing Fail2ban UI with: ``` 4. **Access the services:** - - Fail2ban UI: `http://localhost:3080` (or configured port) + - Fail2ban UI: `http://localhost:3080` (or configured port). The `ssh_and_local` stack serves it under `BASE_PATH=/dev`, so use `http://localhost:3080/dev/` there. - OIDC Provider (Pocket-ID): `http://localhost:3000` (if using OIDC setup) ## Notes diff --git a/development/ssh_and_local/README.md b/development/ssh_and_local/README.md index da2fd5f..c1d7600 100644 --- a/development/ssh_and_local/README.md +++ b/development/ssh_and_local/README.md @@ -2,7 +2,7 @@ This stack provides a complete test environment covering all three connector types: -- a **local Fail2Ban instance** (container) for the local connector, +- **two local Fail2Ban instances** (containers) for testing one or two local connectors side by side, - a **remote Fail2Ban instance over SSH** (container) for the SSH connector, - **Fail2Ban plus fail2ban-ui-agent** (container) for the HTTP agent connector. @@ -22,6 +22,22 @@ This stack provides a complete test environment covering all three connector typ | Socket | `./f2b-run-local/` | +### Fail2ban-Local-Secondary + + +| Property | Value | +| ------------------------------- | ------------------------------------------------------------------------ | +| Container | `DEV_fail2ban-local-secondary` | +| Purpose | Second local Fail2Ban instance, for testing two independent local connectors at once | +| Network | bridge mode, unlike the primary, which uses host mode | +| Config | `./fail2ban-config-local-secondary/` | +| Socket | `./f2b-run-local-secondary/` | +| Paths inside `DEV_fail2ban-ui` | `/etc/fail2ban-secondary` (config) and `/var/run/fail2ban-secondary` (socket) | + + +Configure it in the UI as a second server of type **Local**, pointing at the `-secondary` config and socket paths above. The primary instance uses the default `/etc/fail2ban` and `/var/run/fail2ban`. + + ### Fail2ban-SSH @@ -51,12 +67,13 @@ This stack provides a complete test environment covering all three connector typ ### Fail2ban-UI -| Property | Value | -| --------- | ------------------------------------------------------------- | -| Container | `DEV_fail2ban-ui` | -| Port | `3080` | -| URL | `http://172.16.10.18:3080`, or your configured `BIND_ADDRESS` | -| Purpose | The application under test, managing all three instances | +| Property | Value | +| --------- | ------------------------------------------------------------------------------------------------------------ | +| Container | `DEV_fail2ban-ui` | +| Port | `3080`, bound to `0.0.0.0` | +| Base path | `/dev` (`BASE_PATH=/dev`), so the UI lives at `http://localhost:3080/dev/`; visiting `/` redirects there | +| Callback | `CALLBACK_URL=http://10.88.0.1:3080/dev` - the Podman default bridge gateway; change it if your bridge differs | +| Purpose | The application under test, managing all local, SSH and agent instances | ## Setup @@ -101,7 +118,7 @@ podman logs DEV_fail2ban-ssh ### 4. Configure Fail2Ban UI -1. **Open the UI** at `http://172.16.10.18:3080` (or your configured `BIND_ADDRESS:PORT`; with host networking also `http://localhost:3080`). +1. **Open the UI** at `http://localhost:3080/dev/`. The stack sets `BASE_PATH=/dev`, so visiting `http://localhost:3080/` redirects there. With `BIND_ADDRESS=0.0.0.0` and host networking, the UI is also reachable on the host's other addresses. 2. **Add the local server.** Under **Manage Servers**, the local Fail2Ban instance should be auto-detected; enable the local connector. 3. **Add the SSH server.** Under **Manage Servers**, click **Add Server** and configure: @@ -149,11 +166,16 @@ Edit `container-compose.yml` to customize: ```yaml environment: - PORT=3080 - - BIND_ADDRESS=172.16.10.18 # Change to your IP, or 0.0.0.0 - # OIDC settings (when testing OIDC) - - OIDC_ENABLED=false # Set to true to enable OIDC + - BIND_ADDRESS=0.0.0.0 + - BASE_PATH=/dev # remove to serve the UI at the root + - CALLBACK_URL=http://10.88.0.1:3080/dev # must include BASE_PATH + - CALLBACK_SECRET=same-as-ui-callback-secret + - AUTODARK=true + # - OIDC_ENABLED=true # use development/oidc/ for a full OIDC stack ``` +If you remove `BASE_PATH`, drop the `/dev` suffix from `CALLBACK_URL` as well. See [docs/configuration.md](../../docs/configuration.md#http-base-path-subpath-deployment). + ### SSH container The SSH container is pre-configured with: @@ -170,10 +192,12 @@ To modify the SSH configuration, edit the `command` section in `container-compos ``` ./config/ # Fail2Ban UI configuration and database ./ssh-keys/ # SSH key pair (shared between containers) -./fail2ban-config-local/ # Local Fail2Ban configuration -./f2b-run-local/ # Local Fail2Ban socket directory -./fail2ban-config-ssh/ # SSH Fail2Ban configuration -./fail2ban-config-agent/ # Agent Fail2Ban configuration +./fail2ban-config-local/ # Local Fail2Ban configuration (primary) +./f2b-run-local/ # Local Fail2Ban socket directory (primary) +./fail2ban-config-local-secondary/ # Local Fail2Ban configuration (secondary) +./f2b-run-local-secondary/ # Local Fail2Ban socket directory (secondary) +./fail2ban-config-ssh/ # SSH Fail2Ban configuration +./fail2ban-config-agent/ # Agent Fail2Ban configuration ``` ## Test scenarios @@ -213,6 +237,7 @@ To modify the SSH configuration, edit the `command` section in `container-compos 3. Switch between servers. 4. Verify each server's jails are isolated. 5. Test operations on each server independently. +6. Add the secondary local instance as a third server (type **Local**, config `/etc/fail2ban-secondary`, socket `/var/run/fail2ban-secondary`) and verify that jail lists, bans, and configuration edits stay isolated between the two local connectors. ## Troubleshooting diff --git a/docker-compose-allinone.example.yml b/docker-compose-allinone.example.yml index 8e94fb2..40ed56d 100644 --- a/docker-compose-allinone.example.yml +++ b/docker-compose-allinone.example.yml @@ -28,6 +28,8 @@ services: fail2ban-ui: # Use pre-built image from Docker Hub (default) image: swissmakers/fail2ban-ui:latest + # Alternative: Use GitHub Container Registry + # image: ghcr.io/swissmakers/fail2ban-ui:latest # Alternative: Use Swissmakers registry (fallback) # image: registry.swissmakers.ch/infra/fail2ban-ui:latest # Or build from source (uncomment to use): diff --git a/docker-compose.example.yml b/docker-compose.example.yml index 34d56c0..24ba1bd 100644 --- a/docker-compose.example.yml +++ b/docker-compose.example.yml @@ -3,6 +3,9 @@ services: # Use pre-built image from Docker Hub (default) image: swissmakers/fail2ban-ui:latest + # Alternative: Use GitHub Container Registry + # image: ghcr.io/swissmakers/fail2ban-ui:latest + # Alternative: Use Swissmakers registry (fallback) # image: registry.swissmakers.ch/infra/fail2ban-ui:latest diff --git a/docs/api.md b/docs/api.md index b507262..757f8c1 100644 --- a/docs/api.md +++ b/docs/api.md @@ -4,9 +4,9 @@ This is a practical endpoint index for operators. The web frontend uses these en ## Authentication -* When OIDC is enabled, all `/api/*` endpoints, including the WebSocket, require an authenticated session - except the callback endpoints. +* When OIDC is enabled, all `/api/*` endpoints, including the WebSocket, require an authenticated session - except the machine-to-machine endpoints listed below. * Optional OIDC role-based access control can further restrict authenticated users. `admin` users can access everything; `support` users can view operational dashboard/event data and manually ban/unban IPs. -* The callback endpoints (`/api/ban`, `/api/unban`) are authenticated through the `X-Callback-Secret` header. +* The machine-to-machine endpoints (`POST /api/ban`, `POST /api/unban`, `GET /api/healthcheck/callback`) are exempt from the session check and are authenticated through the `X-Callback-Secret` header instead. ## Input validation @@ -46,24 +46,72 @@ Every endpoint that accepts an IP address validates it server-side with Go's `ne | `POST /api/jails/:jail/logpath/test` | Test log path accessibility | | `POST /api/jails/:jail/ban/:ip` | Ban an IP in a jail | | `POST /api/jails/:jail/unban/:ip` | Unban an IP from a jail | +| `GET /api/ips/:ip/search` | Search every configured server for jails that currently ban this IP | + +`GET /api/ips/:ip/search` queries all connectors in parallel against the *live* Fail2Ban state, not the stored event history, with a 15-second timeout per server. Response: + +```json +{ + "ip": "203.0.113.10", + "banned": true, + "matches": [{ "serverId": "...", "serverName": "...", "jail": "sshd" }], + "errors": [{ "serverId": "...", "serverName": "...", "error": "..." }] +} +``` + +A server that is unreachable is reported in `errors` and does not fail the request. + +`POST /api/jails/:jail/logpath/test` accepts an optional body `{"logpath": "..."}`; when omitted, the log path stored in the jail configuration is used. Paths separated by spaces or newlines are tested individually and each returns one result object: + +| Field | Meaning | +|-------|---------| +| `logpath` | The path as configured | +| `resolved_path` | The path after variable and wildcard resolution | +| `found` | `true` when at least one matching file exists | +| `files` | The matched files | +| `error` | Non-empty when the test genuinely failed | +| `inaccessible` | `true` when the log directory could not be read by the connector's SSH user | +| `message` | Human-readable explanation, currently set for the `inaccessible` case | + +`inaccessible: true` is a warning, not a failure: Fail2Ban runs as root on the managed host and will read the path regardless, so the jail can still be enabled. The UI renders this state in yellow, distinct from the red not-found and error states. ### Events and analytics | Method and path | Description | |-----------------|-------------| | `GET /api/events/bans` | List ban and unban events, paginated and filterable | +| `GET /api/events/bans/:id` | Read a single ban event, including the stored whois text and log lines | | `DELETE /api/events/bans` | Delete all stored ban events | +| `GET /api/events/bans/countries` | Distinct country codes present in the stored events; populates the country filter | | `GET /api/events/bans/stats` | Ban statistics: counts and time series | -| `GET /api/events/bans/insights` | Ban insights: countries, top IPs, top jails | +| `GET /api/events/bans/insights` | Ban insights: country distribution and recurring IPs (`minCount`, `limit` query parameters) | +| `GET /api/events/bans/timeline` | Time-bucketed ban/unban counts for the insights timeline | +| `GET /api/events/bans/ips` | Per-IP ban aggregates for a time range | +| `GET /api/events/bans/ips/activity` | Per-IP activity periods, for the insights activity view | + +**Range and filter parameters.** `timeline`, `ips`, and `ips/activity` accept `since` and `until` as RFC3339 timestamps. The default range is the last 8 hours (`until` = now, `since` = 8 hours earlier). All three also accept the filters `serverId`, `jail`, `country`, and `search`. An unparseable timestamp, or `until` not after `since`, returns `400`. + +* `timeline` accepts an optional `bucket` in seconds. The server chooses the bucket size automatically and raises any override that would produce too many buckets. Response: `{ since, until, bucketSeconds, buckets, totals: { bans, unbans } }`. +* `ips` accepts `limit` (default `2000`, maximum `10000`) and returns `{ ips, total, truncated, since, until }`. +* `ips/activity` accepts `minOverlap` (default `3`) and returns `{ periods }`. + +**Paging and search on `GET /api/events/bans`.** Text search uses an SQLite FTS5 index and falls back to `LIKE` matching if the index is unavailable. Result counts are capped server-side at 5000: a `total` of `5001` means "more than 5000 matches", not an exact count. + +Stored events are pruned on a schedule; see [configuration.md](configuration.md#event-retention-ui-managed). ### Advanced actions | Method and path | Description | |-----------------|-------------| | `GET /api/advanced-actions/blocks` | List permanent block records | +| `POST /api/advanced-actions/blocks` | Bulk permanent block for a list of IPs, used by the ban-insights modal | | `DELETE /api/advanced-actions/blocks` | Delete all permanent block records | | `POST /api/advanced-actions/test` | Manually test block/unblock on the configured integration | +`POST /api/advanced-actions/blocks` takes `{"ips": ["203.0.113.10", "..."]}`, at most 500 addresses per request. It requires a configured and valid advanced-actions integration (MikroTik, pfSense, or OPNsense); otherwise it returns `400`. Duplicates are removed while preserving order, and loopback, link-local, multicast, unspecified, and private addresses are skipped. After five consecutive identical integration errors the run aborts and the remaining addresses are returned with status `aborted`. + +Response: `{ "results": [{ "ip", "status", "message" }], "summary": { "requested", "blocked", "alreadyBlocked", "skipped", "invalid", "failed", "aborted" } }`, where `status` is one of `blocked`, `already_blocked`, `skipped_private`, `invalid`, `error`, or `aborted`. `summary.requested` counts the addresses left after deduplication. + ### Settings | Method and path | Description | @@ -113,6 +161,14 @@ See [threat-intel.md](threat-intel.md) for setup and full behavior. |-----------------|-------------| | `GET /api/version` | Running version, with an optional update check (GitHub request when `UPDATE_CHECK` is enabled) | +### Health check + +| Method and path | Description | +|-----------------|-------------| +| `GET /api/healthcheck/callback` | Validates the `X-Callback-Secret` header without side effects and returns `{"ok": true}` | + +This endpoint lets a managed host - primarily the [fail2ban-ui-agent](https://github.com/swissmakers/fail2ban-ui-agent) - confirm that it holds the correct callback secret before it starts posting events. Like `/api/ban` and `/api/unban`, it is exempt from the OIDC session check and is protected only by the callback secret. A missing, unconfigured, or mismatched secret returns `401`. + ### WebSocket | Method and path | Description | @@ -127,6 +183,9 @@ The connection streams real-time events to the frontend: | `console_log` | Debug console log lines, when debug mode is enabled | | `ban_event` | Real-time ban event broadcast | | `unban_event` | Real-time unban event broadcast | +| `ban_event_update` | Enrichment update for an event that was already delivered, sent once the asynchronous Whois/GeoIP lookup completes | + +All message types are carried on the single `/api/ws` connection; there is no second WebSocket endpoint. `ban_event_update` is emitted after the callback response has already been returned: the event is stored and broadcast immediately, while the Whois and GeoIP lookups, which can take several seconds, run in the background. When they finish, the enriched record is broadcast, with the `whois` and `logs` fields stripped from the payload, so the frontend can merge the resolved country into the row it is already displaying. The WebSocket enforces a same-origin policy through the `Origin` header and requires authentication when OIDC is enabled. diff --git a/docs/architecture.md b/docs/architecture.md index 0ccfc0b..c608bc3 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -28,7 +28,7 @@ A *connector* is the mechanism the UI uses to control a single Fail2Ban instance | Connector | Transport | Typical use | Requirements on the managed host | | --------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | Local | Unix socket and direct file access | Fail2Ban runs on the same host as the UI | Read/write access to `/var/run/fail2ban/fail2ban.sock` and `/etc/fail2ban`; read access to the monitored log files | -| SSH | SSH with key-based authentication | Remote hosts where installing additional software is not wanted | A dedicated service account with `sudo fail2ban-client `* and `sudo systemctl restart fail2ban` | +| SSH | SSH with key-based authentication; optional reverse tunnel (`-R`) for the event path | Remote hosts where installing additional software is not wanted | A dedicated service account with `sudo fail2ban-client `* and `sudo systemctl restart fail2ban` | | Agent | HTTP to the [fail2ban-ui-agent](https://github.com/swissmakers/fail2ban-ui-agent) API | Environments where SSH access from the UI host is not desired or not possible | The agent service, with local access to the Fail2Ban socket and configuration | @@ -69,6 +69,10 @@ The backend processes each callback in the following order: 7. Evaluate advanced ban actions. If a recurring-offender threshold is reached, the address is pushed as a permanent block to the configured edge firewall (MikroTik, pfSense, or OPNsense) and recorded in `permanent_blocks`. 8. Return `200 OK`. +Steps 3, 6, and 7 are partially asynchronous. The callback returns `200 OK` as soon as the event has been stored and broadcast; Whois and GeoIP enrichment and alert dispatch continue in a background goroutine. When enrichment completes, the stored row is updated and a `ban_event_update` message is broadcast so already-rendered rows pick up the resolved country. + +For SSH servers with *reverse tunnel for events* enabled, the action posts to `http://localhost:` instead of the global `CALLBACK_URL`, and the tunnel forwards that connection to the UI's HTTP port over the existing SSH session. The UI re-checks each tunnel every 45 seconds and rebuilds it after a failure. See [configuration.md](configuration.md#reverse-ssh-tunnel-for-callbacks). + **Note:** The callback endpoints (`/api/ban`, `/api/unban`) are intentionally reachable without an OIDC session, because they are called by machines, not by users. They are protected exclusively by the callback secret. Treat the secret like a credential and only transport callbacks over TLS or a trusted network. ### Browser to Fail2Ban-UI @@ -76,7 +80,7 @@ The backend processes each callback in the following order: The browser communicates with the backend over HTTPS (REST) and a WebSocket connection (`GET /api/ws`): - When OIDC is enabled, the index page, all `/api/`* routes except the callbacks, and the WebSocket upgrade require an authenticated session. The login flow (`/auth/login`, `/auth/callback`, `/auth/logout`, `/auth/status`, `/auth/user`) and static assets remain public. -- The WebSocket hub validates the `Origin` header against the request `Host` and rejects cross-site connections. Connected clients receive `heartbeat` (about every 30 seconds), `console_log` (debug console), `ban_event`, and `unban_event` messages. +- The WebSocket hub validates the `Origin` header against the request `Host` and rejects cross-site connections. Connected clients receive `heartbeat` (about every 30 seconds), `console_log` (debug console), `ban_event`, `unban_event`, and `ban_event_update` messages. The last one carries the result of the asynchronous Whois/GeoIP enrichment for an event that was already broadcast, so the browser can fill in the country without refetching. ## Backend components @@ -85,7 +89,7 @@ The browser communicates with the backend over HTTPS (REST) and a WebSocket conn | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | REST API (`/api`) | Server management, jail and filter configuration, ban/unban actions, settings, event queries and insights, data management (clearing events and blocks), version and update check | | WebSocket hub | Client registration, origin validation, broadcast of heartbeat, console, and ban/unban messages | -| SQLite storage | `ban_events`, `app_settings`, `servers`, `permanent_blocks` | +| SQLite storage | `ban_events`, `app_settings`, `servers`, `permanent_blocks`; `ban_events` is pruned daily according to the configured event retention window | | Connector manager | One connector instance per configured server; installs the callback action on new servers | | Alert dispatcher | Pluggable providers: Email (SMTP), Webhook, Elasticsearch; per-event toggles and country-based filtering | | GeoIP / Whois | IP-to-country and hostname resolution through MaxMind databases or ip-api.com; used in the UI, in alerts, and in ban insights | @@ -101,6 +105,7 @@ The browser communicates with the backend over HTTPS (REST) and a WebSocket conn | Fail2ban-UI -> SSH-connected host | SSH, port 22 | outbound from Fail2ban-UI | SSH key, dedicated service account | | Fail2ban-UI -> agent-connected host | HTTP(S), agent port | outbound from Fail2ban-UI | Agent token | | Fail2Ban host -> Fail2ban-UI | HTTP(S) `POST /api/ban`, `/api/unban` | inbound to Fail2ban-UI | `X-Callback-Secret` header | +| Fail2Ban host -> Fail2ban-UI (tunneled) | HTTP inside the existing SSH connection (`ssh -R :localhost:`) | no additional inbound port; reuses the outbound SSH session | `X-Callback-Secret` header, transport encrypted by SSH | | Fail2ban-UI -> alert providers | SMTP / HTTPS / Elasticsearch API | outbound from Fail2ban-UI | Provider-specific | | Fail2ban-UI -> edge firewall | SSH (MikroTik) or HTTPS (pfSense, OPNsense) | outbound from Fail2ban-UI | Device credentials / API token | diff --git a/docs/configuration.md b/docs/configuration.md index df70e5c..63d7fae 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -89,6 +89,12 @@ The UI checks every tunnel's SSH master connection every 45 seconds and automati |----------|---------|-------------| | `AUTODARK` | `false` | When `true`, enables automatic dark mode based on the browser or OS preference. The default remains light mode. | +## Container runtime marker + +| Variable | Default | Description | +|----------|---------|-------------| +| `CONTAINER` | set to `true` in the official image | Marks a containerized run. It is set by the Dockerfile - do not set or unset it manually. When present, the UI reads and writes SSH private keys and `known_hosts` under `/config/.ssh` instead of `$HOME/.ssh`, and applies container SSH host-key handling (`StrictHostKeyChecking=accept-new` against the persistent `known_hosts` next to the configured key). | + ## Fail2Ban configuration migration | Variable | Description | @@ -121,6 +127,23 @@ Configure under **Settings -> Alert Settings**: For provider behavior and payloads, see [alert-providers.md](alert-providers.md) and [webhooks.md](webhooks.md). +## Event retention + +Configure under **Settings -> Alert Settings**, field *Event Retention (Days)*. + +| Setting | JSON field | Default | Range | +|---------|-----------|---------|-------| +| Event Retention (Days) | `eventRetentionDays` | `180` | `0` - `36500` | + +Ban and unban events older than the configured number of days are deleted from the SQLite database. Pruning runs once at startup and then every 24 hours; the number of deleted rows is written to the application log (`Pruned N ban events older than D days`). + +Set the value to `0` to keep events forever. The database then grows without bound on busy hosts `ban_events` is by far the largest table, so only do this if you actively need long-term history and monitor disk usage. + +Notes: + +* Retention only affects `ban_events`. Permanent-block records are never pruned automatically; clear them from the data-management controls or through `DELETE /api/advanced-actions/blocks`. +* After a prune the write-ahead log is checkpointed and truncated, but the main database file keeps its allocated size. Run `VACUUM` manually, with the service stopped, if you need to reclaim disk space after shrinking the retention window. + ## Threat intelligence settings (UI-managed) Configure under **Settings -> Alert Settings**: @@ -160,6 +183,8 @@ Common optional variables: | `OIDC_USERNAME_CLAIM` | `preferred_username` | Claim used as the display username | | `OIDC_SKIP_VERIFY` | `false` | Skips TLS verification toward the provider. Development only. | | `OIDC_SKIP_LOGINPAGE` | `false` | Skips the UI login page and redirects to the provider directly | +| `OIDC_CLIENT_SECRET_FILE` | unset | Only consulted when `OIDC_CLIENT_SECRET=auto-configured`: the client secret is read from this file instead of the environment, for example `/config/keycloak-client-secret`. Startup fails if the file cannot be read. | +| `OIDC_LOGOUT_URL` | auto-derived | Provider end-session endpoint, used after the local session is cleared. When unset, the UI derives it from `OIDC_ISSUER_URL` per provider, so set it only when your provider needs a different URL. Example: `https://keycloak.example.com/realms/your-realm/protocol/openid-connect/logout` | OIDC role-based access control is optional. When no role variables are set, every authenticated OIDC user keeps the previous full-access behavior. diff --git a/docs/installation.md b/docs/installation.md index 40feb56..52008be 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -23,6 +23,14 @@ podman run -d --name fail2ban-ui --network=host \ swissmakers/fail2ban-ui:latest ``` +The same image is published to three registries; use whichever your environment prefers: + +* `docker.io/swissmakers/fail2ban-ui` +* `ghcr.io/swissmakers/fail2ban-ui` +* `registry.swissmakers.ch/infra/fail2ban-ui` + +Each carries `latest`, a version tag (for example `v1.5.3`), and a commit-SHA tag. Pin to a version tag for reproducible deployments. + Notes on the mounts: * `/config` stores the SQLite database, settings, and the SSH keys used by the UI. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 7148b2c..02cb7c4 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -427,6 +427,22 @@ sqlite3 /opt/fail2ban-ui/fail2ban-ui.db "PRAGMA integrity_check;" Expected output: `ok`. Any other output indicates a problem - investigate filesystem errors and restore from backup if needed. +### Database growing large + +The `ban_events` table dominates the file size. Check the configured retention window under **Settings -> Alert Settings -> Event Retention (Days)** - the default is 180, and `0` disables pruning entirely. Pruning runs at startup and then every 24 hours; look for `Pruned N ban events older than D days` in the log. + +Reducing the window does not shrink the file on its own. To reclaim the space, stop the service and run: + +```bash +sqlite3 /opt/fail2ban-ui/fail2ban-ui.db "VACUUM;" +``` + +See [configuration.md](configuration.md#event-retention-ui-managed). + +### Event searches return nothing or are slow + +The event list uses an SQLite FTS5 index, which is rebuilt automatically when it falls out of sync with `ban_events`. If SQLite was built without FTS5, the UI silently falls back to `LIKE` matching, which is noticeably slower on large tables. Search result counts are capped at 5000, so a total of `5001` means "more than 5000 matches" rather than an exact count. + ## Reverse proxy checks If the UI loads but real-time updates fail: diff --git a/internal/fail2ban/filter_includes_test.go b/internal/fail2ban/filter_includes_test.go index 8ee5bce..bd8ea44 100644 --- a/internal/fail2ban/filter_includes_test.go +++ b/internal/fail2ban/filter_includes_test.go @@ -23,11 +23,6 @@ import ( "testing" ) -// Characterization tests for the filter [INCLUDES] resolution. The output of -// resolveFilterIncludes is fed straight into fail2ban-regex, so ordering and -// variable shadowing are load-bearing: these pin the behaviour so the shared -// implementation used by the local and SSH connectors cannot drift. - func writeFilterFile(t *testing.T, dir, name, content string) { t.Helper() if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o600); err != nil { diff --git a/screenshots/0.1_Dashboard_unban_IP.png b/screenshots/0.1_Dashboard_unban_IP.png index 9e2b171..4e1f974 100644 Binary files a/screenshots/0.1_Dashboard_unban_IP.png and b/screenshots/0.1_Dashboard_unban_IP.png differ diff --git a/screenshots/0_Dashboard.png b/screenshots/0_Dashboard.png index 6c254f9..7f24853 100644 Binary files a/screenshots/0_Dashboard.png and b/screenshots/0_Dashboard.png differ diff --git a/screenshots/1.1_Dashboard_Manage_Jails.png b/screenshots/1.1_Dashboard_Manage_Jails.png index 728d8f3..002c052 100644 Binary files a/screenshots/1.1_Dashboard_Manage_Jails.png and b/screenshots/1.1_Dashboard_Manage_Jails.png differ diff --git a/screenshots/1.2_Dashboard_Manage_Jails_Edit.png b/screenshots/1.2_Dashboard_Manage_Jails_Edit.png index 3740ef7..ec782fc 100644 Binary files a/screenshots/1.2_Dashboard_Manage_Jails_Edit.png and b/screenshots/1.2_Dashboard_Manage_Jails_Edit.png differ diff --git a/screenshots/1.3_Dashboard_Manage_Jails_Edit_Logpathtest.png b/screenshots/1.3_Dashboard_Manage_Jails_Edit_Logpathtest.png index b8ec070..fe5978c 100644 Binary files a/screenshots/1.3_Dashboard_Manage_Jails_Edit_Logpathtest.png and b/screenshots/1.3_Dashboard_Manage_Jails_Edit_Logpathtest.png differ diff --git a/screenshots/1.4_Dashboard_Manage_Jails_Create_Filter.png b/screenshots/1.4_Dashboard_Manage_Jails_Create_Filter.png index 28899d2..1c87884 100644 Binary files a/screenshots/1.4_Dashboard_Manage_Jails_Create_Filter.png and b/screenshots/1.4_Dashboard_Manage_Jails_Create_Filter.png differ diff --git a/screenshots/1.5_Dashboard_Manage_Jails_Create_Jail.png b/screenshots/1.5_Dashboard_Manage_Jails_Create_Jail.png index bcc6447..0d207fa 100644 Binary files a/screenshots/1.5_Dashboard_Manage_Jails_Create_Jail.png and b/screenshots/1.5_Dashboard_Manage_Jails_Create_Jail.png differ diff --git a/screenshots/1.6_Dashboard_search.png b/screenshots/1.6_Dashboard_search.png index a0d11e5..bcfb1dc 100644 Binary files a/screenshots/1.6_Dashboard_search.png and b/screenshots/1.6_Dashboard_search.png differ diff --git a/screenshots/1_Dashboard_Manage_Servers.png b/screenshots/1_Dashboard_Manage_Servers.png index 1707bb8..79daa43 100644 Binary files a/screenshots/1_Dashboard_Manage_Servers.png and b/screenshots/1_Dashboard_Manage_Servers.png differ diff --git a/screenshots/2.1_Dashboard_Log_Overview_Whois.png b/screenshots/2.1_Dashboard_Log_Overview_Whois.png index 68dfe48..4833b4c 100644 Binary files a/screenshots/2.1_Dashboard_Log_Overview_Whois.png and b/screenshots/2.1_Dashboard_Log_Overview_Whois.png differ diff --git a/screenshots/2.2_Dashboard_Log_Overview_BanLogs.png b/screenshots/2.2_Dashboard_Log_Overview_BanLogs.png index 25c1631..efbe9a7 100644 Binary files a/screenshots/2.2_Dashboard_Log_Overview_BanLogs.png and b/screenshots/2.2_Dashboard_Log_Overview_BanLogs.png differ diff --git a/screenshots/2.3_Dashboard_Log_Overview_BanInsights.png b/screenshots/2.3_Dashboard_Log_Overview_BanInsights.png new file mode 100644 index 0000000..0054fd5 Binary files /dev/null and b/screenshots/2.3_Dashboard_Log_Overview_BanInsights.png differ diff --git a/screenshots/2.4_Dashboard_Threat_Intel.png b/screenshots/2.4_Dashboard_Threat_Intel.png new file mode 100644 index 0000000..83afc66 Binary files /dev/null and b/screenshots/2.4_Dashboard_Threat_Intel.png differ diff --git a/screenshots/2_Dashboard_Log_Overview.png b/screenshots/2_Dashboard_Log_Overview.png index 48eb3a9..e18d3d8 100644 Binary files a/screenshots/2_Dashboard_Log_Overview.png and b/screenshots/2_Dashboard_Log_Overview.png differ diff --git a/screenshots/3.1_Filter_Debug_Testresult.png b/screenshots/3.1_Filter_Debug_Testresult.png index f23213c..90e810c 100644 Binary files a/screenshots/3.1_Filter_Debug_Testresult.png and b/screenshots/3.1_Filter_Debug_Testresult.png differ diff --git a/screenshots/3_Filter_Debug.png b/screenshots/3_Filter_Debug.png index ddb4a6c..d55d5e9 100644 Binary files a/screenshots/3_Filter_Debug.png and b/screenshots/3_Filter_Debug.png differ diff --git a/screenshots/4.1_Settings_DebugConsole.png b/screenshots/4.1_Settings_DebugConsole.png index 5ef50a8..080b95c 100644 Binary files a/screenshots/4.1_Settings_DebugConsole.png and b/screenshots/4.1_Settings_DebugConsole.png differ diff --git a/screenshots/4.2_Settings_AdvancedBanActions.png b/screenshots/4.2_Settings_AdvancedBanActions.png index 3afa995..430043d 100644 Binary files a/screenshots/4.2_Settings_AdvancedBanActions.png and b/screenshots/4.2_Settings_AdvancedBanActions.png differ diff --git a/screenshots/4.3_Settings_AlertSettings.png b/screenshots/4.3_Settings_AlertSettings.png index b36f858..674c9c1 100644 Binary files a/screenshots/4.3_Settings_AlertSettings.png and b/screenshots/4.3_Settings_AlertSettings.png differ diff --git a/screenshots/4.4_Settings_GlobalSettings.png b/screenshots/4.4_Settings_GlobalSettings.png index cc1a34b..858ad8b 100644 Binary files a/screenshots/4.4_Settings_GlobalSettings.png and b/screenshots/4.4_Settings_GlobalSettings.png differ diff --git a/screenshots/4.4_Settings_GlobalSettings2.png b/screenshots/4.4_Settings_GlobalSettings2.png new file mode 100644 index 0000000..90da297 Binary files /dev/null and b/screenshots/4.4_Settings_GlobalSettings2.png differ diff --git a/screenshots/4_Settings.png b/screenshots/4_Settings.png index 338a950..c3e115a 100644 Binary files a/screenshots/4_Settings.png and b/screenshots/4_Settings.png differ diff --git a/screenshots/README.md b/screenshots/README.md index 5f0fe97..e109d26 100644 --- a/screenshots/README.md +++ b/screenshots/README.md @@ -8,7 +8,7 @@ This directory contains screenshots showcasing the features and the interface of ## Unban IP ![Unban IP](0.1_Dashboard_unban_IP.png) -**Description:** Unbanning a IP addresses directly from the dashboard. Shows the unban confirmation dialog. +**Description:** Unban an IP address directly from the dashboard. A confirmation dialog shows the affected jail before the unban runs. ## Server Management ![Manage Servers](1_Dashboard_Manage_Servers.png) @@ -20,11 +20,11 @@ This directory contains screenshots showcasing the features and the interface of ### Edit Jail Configuration ![Edit Jail](1.2_Dashboard_Manage_Jails_Edit.png) -**Description:** When clicking on "Edit Filter / Jail" the Jail configuration editor is opened. It shows the current filter and jail configuration with all options to modify the settings, test or add / modify the logpaths, and save changes. +**Description:** Clicking **Edit Filter / Jail** opens the configuration editor. Edit the jail and filter settings, add or change log paths and test them, then save - the change is pushed to the managed host and Fail2Ban is reloaded. ### Logpath Test ![Logpath Test](1.3_Dashboard_Manage_Jails_Edit_Logpathtest.png) -**Description:** Logpath testing functionality that verifies log file paths and checks if files are accessible. Shows test results with visual indicators (✓/✗) for each log path. +**Description:** Logpath testing resolves the configured paths, including wildcards and Fail2Ban variables, and checks whether matching files exist. Each path is reported as ✓ found, ✗ not found, or ⚠ cannot verify. ### Create Filter ![Create Filter](1.4_Dashboard_Manage_Jails_Create_Filter.png) @@ -36,7 +36,7 @@ This directory contains screenshots showcasing the features and the interface of ## Search Functionality ![Search](1.6_Dashboard_search.png) -**Description:** Search for a specific IPs, that where blocked in a specific jail - searches in all active jails. Provides a quick and painless filtering. +**Description:** Search for an IP address across all active jails of every configured server. The result lists each jail that currently bans the address, so it can be unbanned in the right place. ## Internal Log Overview ![Log Overview](2_Dashboard_Log_Overview.png) @@ -50,6 +50,10 @@ This directory contains screenshots showcasing the features and the interface of ![Ban Logs](2.2_Dashboard_Log_Overview_BanLogs.png) **Description:** Detailed ban log view showing log lines that triggered the ban, timestamps, and context information for each security event. +### Ban Insights +![Ban Insights](2.3_Dashboard_Log_Overview_BanInsights.png) +**Description:** SIEM-like analysis without a full SIEM -> a ban/unban timeline (drag to zoom, presets from 8 hours to 30 days, custom range up to 12 months) makes spikes and attack patterns visible. Pin two spikes as Incident A and B to list the IPs present in both -> repeat attackers, not false positives and ban them permanently to the recurring-offenders list, or select a time range where no legitimate user is active and block every IP in it with one click. + ## Filter Debugging ![Filter Debug](3_Filter_Debug.png) **Description:** Filter debugging interface for testing Fail2Ban filter regex patterns against log lines. Helps validate filter configurations before deployment. @@ -64,7 +68,7 @@ This directory contains screenshots showcasing the features and the interface of ### Debug Console ![Debug Console](4.1_Settings_DebugConsole.png) -**Description:** When enabled the Debug console showing real-time application logs, system messages, and debugging information. Useful for troubleshooting and monitoring without the need to query the container logs manually everytime. +**Description:** When enabled, the debug console streams the live application log into the browser -> useful for troubleshooting without querying the container logs manually. Leave it disabled in normal operation. ### Advanced Ban Actions ![Advanced Ban Actions](4.2_Settings_AdvancedBanActions.png) @@ -72,8 +76,11 @@ This directory contains screenshots showcasing the features and the interface of ### Alert Settings ![Alert Settings](4.3_Settings_AlertSettings.png) -**Description:** Email alert configuration with SMTP settings, country-based filtering (blocks from what country to raport), GeoIP provider selection, and alert preferences for bans and unbans. +**Description:** Alert provider configuration. One of three mutually exclusive providers is active at a time - Email (SMTP), Webhook, or Elasticsearch with per-event toggles for bans and unbans, country-based filtering (which countries to report on), GeoIP provider selection, the maximum number of log lines per alert, and the event retention window. See [docs/alert-providers.md](../docs/alert-providers.md). ### Global Settings ![Global Settings](4.4_Settings_GlobalSettings.png) -**Description:** Global Fail2Ban settings including default bantime, findtime, maxretry, banaction configuration (nftables/firewalld/iptables) and so on. +**Description:** Global Fail2Ban defaults: `bantime`, `findtime`, `maxretry`, `ignoreip`, and the `banaction` backend (nftables, firewalld, iptables). When bantime increment is enabled, the escalation can be tuned with `bantime.rndtime`, `bantime.maxtime`, `bantime.factor`, and `bantime.overalljails`. + +## Example Alert Email +**Description:** [`Example_block_email.pdf`](Example_block_email.pdf) - a rendered example of the Email (SMTP) ban alert, for reference when evaluating the modern template.