diff --git a/README.md b/README.md index 29cd036..46b2d71 100644 --- a/README.md +++ b/README.md @@ -2,20 +2,28 @@ # Alertbot -This bot uses the webhook functionality of monitoring/alerting solutions like Grafana or Alertmanager to forward alerts to matrix rooms. -This means that you no longer have to use E-Mail or Slack to receive alerts. +A [maubot](https://github.com/maubot/maubot) plugin that receives webhooks from monitoring systems and forwards them as formatted messages to [Matrix](https://matrix.org) rooms. + +## Supported webhook types + +| Type | Detection | Source | +|------|-----------|--------| +| Alertmanager | `alerts[].labels.alertname` | Prometheus, custom rules, any Alertmanager source | +| Grafana | `alerts[].labels.grafana_folder` | Grafana alerting | +| Uptime Kuma | `heartbeat.status` | Uptime Kuma | +| Slack webhook | `text` + `attachments` | Slack-format payloads | ## Getting Started You can either [invite](#invite-the-official-instance) an official instance of the bot, or [self-host](#self-host-an-instance) it on your own matrix server. -## Invite the Official Instance +### Invite the Official Instance * Create a Matrix room and invite @alertbot:hyteck.de * Send `!url` to the room. The bot will answer with the webhook URL * Put the Webhook URL into your monitoring solution (see below) -## Self-host an Instance +### Self-host an Instance **Prerequisites:** * A Matrix server where you have access to a maubot instance @@ -35,7 +43,7 @@ cd alertbot mbc build ``` -It's possible to upload the build to you maubot instance from the CLI. This is especially helpful when developing. +It's possible to upload the build to your maubot instance from the CLI. This is especially helpful when developing. First login to your instance, then add the `-u` flag to upload after build. ```shell mbc login @@ -44,30 +52,55 @@ mbc build -u **Configure the Bot** -Next, once the plugin is installed, set up a [client](https://docs.mau.fi/maubot/usage/basic.html#creating-clients) and then create an [instance](https://docs.mau.fi/maubot/usage/basic.html#creating-instances) which connects the client and plugin. +Next, once the plugin is installed, set up a [client](https://docs.mau.fi/maubot/usage/basic.html#creating-clients) and then create an [instance](https://docs.mau.fi/maubot/usage/basic.html#creating-instances) which connects the client and plugin. -Finally, invite the bot to an encrypted room where alerts should be sent and query the bot for the room id with the command `!roomid`. +Finally, invite the bot to a Matrix room where alerts should be sent and query the bot for the room id with the command `!roomid`. ## Setup Alertmanager -This configuration will send all your alerts to the room `!zOcbWjsWzdREnihgeC:example.com` (if the bot has access to it). -Put in your own room-id (`!roomid`) behind the webhook base url (`!url`): -```yaml +This configuration will send all your alerts to a Matrix room. Put in your own room-id (`!roomid`) behind the webhook base url (`!url`): +```yaml receivers: -- name: alertbot - webhook_configs: - - url: https://synapse.hyteck.de/_matrix/maubot/plugin/alertbot/webhook/!zOcbWjsWzdREnihgeC:example.com + - name: alertbot + webhook_configs: + - url: 'https://maubot.example.com/_matrix/maubot/plugin/alertbot/webhook/!roomid:example.com' + send_resolved: true + route: - group_by: - - alertname - - cluster - - service - group_interval: 5m - group_wait: 30s receiver: alertbot + group_by: ['alertname'] + group_wait: 30s + group_interval: 5m repeat_interval: 3h +``` + +**Example messages:** + +Firing: +``` +🔥 **InstanceDown** · critical + +webserver.example.com of job node_exporter has been down for more than 5 minutes. + +• **Instance:** webserver.example.com +• **Job:** node_exporter +• **Since:** 2024-03-30 08:00 UTC + +Source · Silence +``` + +Resolved: +``` +✅ **InstanceDown** · resolved + +webserver.example.com of job node_exporter has been down for more than 5 minutes. + +• **Instance:** webserver.example.com +• **Job:** node_exporter +• **Duration:** 2h 15m · **Resolved:** 2024-03-30 10:15 UTC +Source ``` ## Setup Grafana @@ -78,14 +111,19 @@ The grafana setup is fairly simple and can be used to forward grafana alerts to ## Send Test Alerts -Use an example from the `alert_examples/` to test your setup +Use an example from `alert_examples/` to test your setup: + ```shell curl --header "Content-Type: application/json" \ --request POST \ - --data "@alert_examples/prometheus.json" \ - https://webhook.example.com/_matrix/maubot/plugin/maubot/webhook/!zOcbWjsWzdREnihreC:example.com + --data "@alert_examples/prometheus_alert.json" \ + https://maubot.example.com/_matrix/maubot/plugin/alertbot/webhook/!roomid:example.com ``` +## Dependencies + +- `python-dateutil>=2.8.2` (for timestamp parsing and duration calculation) + ## Matrix Room This project has a dedicated [chat room](https://matrix.to/#/#alertbot:hyteck.de) diff --git a/alertbot.py b/alertbot.py index 705e0f2..f8cf6b5 100644 --- a/alertbot.py +++ b/alertbot.py @@ -1,4 +1,5 @@ import json +from urllib.parse import quote import dateutil.parser from aiohttp.web import Request, Response, json_response @@ -58,10 +59,15 @@ def convert_slack_webhook_to_markdown(data): def get_alert_type(data): - """ - Currently supported are ["grafana-alert", "grafana-resolved", "prometheus-alert", "not-found"] - - :return: alert type + """Detect the type of incoming webhook payload. + + Supported types: + - slack-webhook: Slack-format payload with text + attachments + - uptime-kuma-alert / uptime-kuma-resolved: Uptime Kuma heartbeat + - grafana-alert / grafana-resolved: Grafana alert with grafana_folder label + - alertmanager-alert / alertmanager-resolved: Any Alertmanager webhook + (includes Prometheus-originated alerts — the "job" label is NOT required) + - not-found: Unrecognized format """ if ("text" in data) and ("attachments" in data): @@ -73,7 +79,7 @@ def get_alert_type(data): return "uptime-kuma-alert" elif data["heartbeat"]["status"] == 1: return "uptime-kuma-resolved" - except KeyError: + except (KeyError, TypeError): pass # Grafana @@ -83,22 +89,160 @@ def get_alert_type(data): return "grafana-alert" else: return "grafana-resolved" - except KeyError: + except (KeyError, IndexError, TypeError): pass - # Prometheus + # Generic Alertmanager webhook: has alerts[] with labels.alertname. + # This covers Prometheus-originated alerts, custom alerts, and any other + # source that sends through Alertmanager. No "job" label required. try: - if data["alerts"][0]["labels"]["job"]: + if data["alerts"][0]["labels"]["alertname"]: if data['status'] == "firing": - return "prometheus-alert" + return "alertmanager-alert" else: - return "prometheus-resolved" - except KeyError: + return "alertmanager-resolved" + except (KeyError, IndexError, TypeError): pass return "not-found" +def _format_duration(start_str, end_str): + """Calculate human-readable duration between two ISO 8601 timestamps.""" + try: + start = dateutil.parser.isoparse(start_str) + end = dateutil.parser.isoparse(end_str) + delta = end - start + total_seconds = int(delta.total_seconds()) + if total_seconds < 0: + return None + + days, remainder = divmod(total_seconds, 86400) + hours, remainder = divmod(remainder, 3600) + minutes, _ = divmod(remainder, 60) + + parts = [] + if days: + parts.append(f"{days}d") + if hours: + parts.append(f"{hours}h") + if minutes or not parts: + parts.append(f"{minutes}m") + return " ".join(parts) + except (ValueError, TypeError): + return None + + +def _format_timestamp(iso_str): + """Format an ISO 8601 timestamp as 'YYYY-MM-DD HH:MM UTC'.""" + try: + dt = dateutil.parser.isoparse(iso_str) + return dt.strftime("%Y-%m-%d %H:%M UTC") + except (ValueError, TypeError): + return iso_str + + +def _build_silence_url(external_url, alertname): + """Build a pre-filled Alertmanager silence URL for a given alertname.""" + if not external_url: + return None + base = external_url.rstrip("/") + filter_param = quote('{alertname="' + alertname + '"}') + return f"{base}/#/silences/new?filter={filter_param}" + + +# Labels already shown in the title line — no need to repeat in metadata +_SKIP_LABELS = {"alertname", "severity"} + + +def alertmanager_to_markdown(alert_data: dict) -> list: + """Convert an Alertmanager webhook payload to formatted markdown messages. + + Produces one message per alert in the payload. Each message follows + a structured format optimized for mobile notification clients: + + 1. Status emoji + alert name + severity (= push notification text) + 2. Description + 3. Label metadata (one per line) + 4. Timestamps + 5. Source + silence links + + Args: + alert_data: The full Alertmanager webhook payload dict. + + Returns: + List of markdown-formatted message strings. + """ + messages = [] + external_url = alert_data.get("externalURL", "") + + for alert in alert_data.get("alerts", []): + status = alert.get("status", "unknown") + labels = alert.get("labels", {}) + annotations = alert.get("annotations", {}) + + alertname = labels.get("alertname", "Unknown") + severity = labels.get("severity", "") + + # --- Title line --- + emoji = "\u2705" if status == "resolved" else "\U0001f525" + severity_text = f" \u00b7 {severity}" if severity and status != "resolved" else "" + status_text = " \u00b7 resolved" if status == "resolved" else "" + title = f"{emoji} **{alertname}**{severity_text}{status_text}" + + # --- Description --- + description = annotations.get("description") or annotations.get("summary", "") + + # --- Label metadata --- + # Use • character (not - markdown list marker) so CommonMark + # emits

+
instead of