feat: silence detection – alert when jobs stop running entirely (v4.5.0)

API.md

@@ -316,6 +316,9 @@ List all cron jobs.
       "active": true,
       "notify_on_failure": true,
       "notify_on_recovery": false,
+      "notify_on_silence": false,
+      "silence_grace_minutes": null,
+      "last_silence_alert_at": null,
       "execution_limit_seconds": 300,
       "auto_kill_on_limit": false,
       "singleton": false,
@@ -372,6 +375,8 @@ Create a new cron job.  Scope: **`jobs:write`**
   "tags":                      ["backup"],
   "notify_on_failure":         true,
   "notify_on_recovery":        false,
+  "notify_on_silence":         false,
+  "silence_grace_minutes":     null,
   "execution_limit_seconds":   0,
   "auto_kill_on_limit":        false,
   "singleton":                 false,
@@ -951,6 +956,7 @@ to 60 seconds of delay before the daemon picks up the entry.
 
 | Version | Change |
 |---|---|
+| 4.5.0 | Added `notify_on_silence` (bool), `silence_grace_minutes` (int\|null), `last_silence_alert_at` (string\|null, read-only) to job objects; `GET /health` extended with `silent_jobs` (int\|null) and `last_execution_at` (string\|null) |
 | 4.3.4 | Added `GET /api/v1/audit` endpoint (`audit:read` scope, admin-only); added §15 Audit Log |
 | 4.2.0 | Added `GET /api/v1/agents` endpoint (`settings:read` scope; respects `agent_ids` restriction; omits sensitive fields) |
 | 4.1.0 | Initial external REST API with API key authentication and scope-based authorization |

CHANGELOG.md

@@ -6,6 +6,39 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ---
 
+## [4.5.0] – branch: `feature/silence-detection`
+
+### Added
+
+- **Silence Detection**: Aktive Jobs mit `notify_on_silence = 1` werden von `check-limits.php`
+  minütlich überwacht. Wenn ein Job seinen letzten geplanten Startzeitpunkt (berechnet via
+  `CronExpression::getPreviousRunDate()`) plus Toleranzzeit überschritten hat, ohne einen echten
+  Start (exit_code ≠ -4) zu verzeichnen, wird eine Benachrichtigung per Mail und/oder Telegram
+  ausgelöst. Erkennt stumme Komplettausfälle, die keine `execution_log`-Einträge erzeugen.
+- **Drei Maintenance-Guards** gegen False Positives:
+  - Guard 1: Agent-weite Maintenance → Silence-Detection übersprungen.
+  - Guard 2: Alle Targets des Jobs in Maintenance → kein Alert.
+  - Guard 3: Letzter `execution_log`-Eintrag war ein Maintenance-Sentinel (exit_code = -4) →
+    kein Alert (Maintenance gerade erst beendet).
+- **Dedup**: Alert wird höchstens einmal pro Stunde pro Job gesendet (`last_silence_alert_at`).
+  Das Feld wird in `ExecutionStartEndpoint` automatisch auf `NULL` zurückgesetzt, wenn der Job
+  wieder startet.
+- **Neue Job-Felder**: `notify_on_silence` (bool, opt-in, Default 0), `silence_grace_minutes`
+  (int|null, überschreibt globalen `silence.grace_minutes`-Wert aus `config.json`),
+  `last_silence_alert_at` (datetime|null, system-only, read-only über API).
+- **MailNotifier::sendSilenceAlert()** und **TelegramNotifier::sendSilenceAlert()**: neue Methoden
+  mit amber/gelber Farbgebung (Mail-Template) und 🔇-Emoji (Telegram).
+- **`send-notification.php`**: neuer Zweig `type = "silence"` für asynchronen Dispatch.
+- **`GET /health`** erweitert um `silent_jobs` (Anzahl aktiver Jobs mit überschrittenem
+  Silence-Schwellwert, `null` bei DB-Fehler) und `last_execution_at` (letzter `started_at`-Wert
+  aus `execution_log`, ermöglicht externe Uptime-Monitore ohne Per-Job-Konfiguration).
+- **REST API**: `notify_on_silence` und `silence_grace_minutes` in GET/POST/PUT für Jobs;
+  `last_silence_alert_at` ist read-only.
+- **Web-Formular**: Checkbox „Silence-Alarm aktivieren" und optional Toleranzzeit-Feld im
+  Benachrichtigungs-Tab; Grace-Feld wird per JS ein-/ausgeblendet.
+
+---
+
 ## [4.4.4] – branch: `fix/hmac-agent-username-default`
 
 ### Fixed

README.md

@@ -70,6 +70,7 @@ history, email failure alerts, execution limits, multi-host support, and SSO int
 | **Email alerts** | Receive an email when a job exits with a non-zero status or exceeds its execution limit |
 | **Telegram alerts** | Receive a Telegram message for the same events via the Bot API |
 | **Recovery notifications** | Optionally receive an email and/or Telegram message when a job succeeds again after a failure streak that triggered an alert |
+| **Silence detection** | Opt-in per job: `check-limits.php` uses the cron schedule to calculate the last expected start time and alerts (email + Telegram) if no real execution has been recorded within the schedule interval plus a configurable grace period. Three maintenance-window guards prevent false positives. `GET /health` exposes a `silent_jobs` counter for external monitors |
 | **Maintenance Windows** | Define per-target scheduled maintenance windows; jobs are either skipped (exit code −4) or executed silently depending on the per-job setting. A special **"Cronmanager Agent"** target blocks all executions host-wide (useful for VM maintenance cycles). Conflict icons (⚠ amber / ✕ red) appear in the job list and detail view |
 | **SSH connectivity test** | A **Test** button on the Maintenance Windows page verifies that the agent can reach an SSH target via key-based auth (`BatchMode=yes`, 10 s timeout). The result (Connected / Failed) is shown inline without a page reload |
 | **Startup orphan cleanup** | On agent restart, executions still marked as "running" with no live process are automatically resolved to exit code −5 ("Interrupted by system restart") |

agent/VERSION

@@ -1 +1 @@
-4.4.4
+4.5.0

agent/agent.php

@@ -171,16 +171,50 @@ function jsonResponse(int $statusCode, array $data): void
 
     $router = new Router();
 
-    // -- Health (fully functional, HMAC-exempt) --------------------------------
+    // -- Crons ----------------------------------------------------------------
+
+    // -- Health (HMAC-exempt, registered after PDO init to include DB metrics) -
 
     $router->addRoute('GET', '/health', function (array $params) use ($logger): void {
         $logger->info('Health check requested');
         $versionFile = __DIR__ . '/VERSION';
+
+        $silentJobs      = null;
+        $lastExecutionAt = null;
+
+        try {
+            $healthPdo = \Cronmanager\Agent\Database\Connection::getInstance()->getPdo();
+
+            $graceMinutes = 10;  // same default used by check-limits.php silence detection
+
+            $silentStmt = $healthPdo->query(
+                "SELECT COUNT(*) FROM cronjobs c
+                  WHERE c.active = 1
+                    AND c.notify_on_silence = 1
+                    AND (
+                        SELECT MAX(CASE WHEN e.exit_code != -4 THEN e.started_at END)
+                        FROM execution_log e
+                        WHERE e.cronjob_id = c.id
+                    ) < DATE_SUB(NOW(), INTERVAL COALESCE(c.silence_grace_minutes, {$graceMinutes}) MINUTE)"
+            );
+            $silentJobs = (int) $silentStmt->fetchColumn();
+
+            $lastExecStmt = $healthPdo->query(
+                'SELECT MAX(started_at) FROM execution_log'
+            );
+            $raw = $lastExecStmt->fetchColumn();
+            $lastExecutionAt = ($raw !== false && $raw !== null) ? (string) $raw : null;
+        } catch (\Throwable $e) {
+            $logger->warning('Health: DB query failed', ['message' => $e->getMessage()]);
+        }
+
         jsonResponse(200, [
             'status'            => 'ok',
             'version'           => is_readable($versionFile) ? trim((string) file_get_contents($versionFile)) : 'unknown',
             'container_version' => getenv('APP_VERSION') ?: 'unknown',
             'timestamp'         => date('c'),
+            'silent_jobs'       => $silentJobs,
+            'last_execution_at' => $lastExecutionAt,
         ]);
     });