feat(macos): process and file telemetry via Endpoint Security#46
feat(macos): process and file telemetry via Endpoint Security#46mostafa wants to merge 10 commits into
Conversation
Add the endpoint-sec dependency and an EsfSensor that owns the ES client on a dedicated thread, subscribes to process exec and exit, and surfaces client init failures (not root, missing entitlement, TCC denial) synchronously. Replace the Phase 0 placeholder sensor in the macOS runtime. Event translation is added in following commits.
Translate NOTIFY_EXEC into a process-start SensorEvent: executable path, joined argv, pid, parent pid, working directory, and resolved user, keyed by audit-token pid and process start time. Extract an FFI-free assembly step so the mapping is unit-tested without a live client, and enable the libc username lookup on macOS.
Translate NOTIFY_EXIT into a process-stop SensorEvent carrying pid and user, mirroring the Linux terminate event, with a unit test for the assembly.
Add the com.apple.developer.endpoint-security.client entitlement plist and document the macOS build, ad-hoc signing, and SIP/root requirements for running the sensor locally.
Subscribe to NOTIFY_CREATE and NOTIFY_UNLINK and translate them into file SensorEvents with Sysmon-compatible create and delete IDs, resolving the create destination from either an existing file or a new directory plus filename. The assembler is unit-tested without a live client.
Subscribe to NOTIFY_RENAME and translate it into a file SensorEvent carrying both the source and destination paths, resolving the destination from an existing file or a new directory plus filename.
Subscribe to NOTIFY_CLOSE and emit a file-change SensorEvent only when the closed file was modified, which keeps the high-volume close stream down to actual content changes.
Assert that a macOS file-create event maps through the normalizer and ECS encoder to the expected dataset, category, type, action, file path, and the macos/darwin host.os fields. Uses the repo's field-level ECS contract style rather than a byte-exact golden, since host and timestamp fields vary.
Karib0u
left a comment
There was a problem hiding this comment.
Thanks for this — the threading model (client created/dropped on a dedicated thread, NOTIFY-only subscriptions, non-blocking try_send with drop-on-full) and the FFI-free RawExec/RawFile split are clean and genuinely testable. Two things I'd like addressed before merge (both verified against the Linux sensor), plus a couple of minor notes inline.
| FileAction::Create => (SensorAction::Create, EVENT_ID_FILE_CREATE, 64), | ||
| FileAction::Delete => (SensorAction::Delete, EVENT_ID_FILE_DELETE, 70), | ||
| FileAction::Rename => (SensorAction::Rename, EVENT_ID_FILE_RENAME, 71), | ||
| FileAction::Modify => (SensorAction::Modify, EVENT_ID_FILE_CHANGE, 65), |
There was a problem hiding this comment.
File-modify event_id doesn't match the Linux sensor — breaks Sigma parity.
This maps modify to EVENT_ID_FILE_CHANGE (65), but the Linux sensor maps modify to event_id 11 with action_code 65 (linux/events.rs:207 → 4 => (SensorAction::Modify, 11, 65)). Create/delete/rename all match Linux exactly; only modify diverges, and the value 65 is identical to its own action_code — which looks like the event_id was set to the action_code by mistake.
This matters because Sigma matches EventID against event_id_string (models/event.rs:84), with no downstream remap (ECS event.action keys on action_code instead, which is why the ECS contract test passes). Sysmon has no dedicated file-modify event, so file-change detections key on EventID: 11 — a rule that fires on Linux modify will silently not fire on macOS modify, contradicting the PR's claim that existing Sigma detection works.
Suggested fix:
FileAction::Modify => (SensorAction::Modify, EVENT_ID_FILE_CREATE, 65), // 11, matching LinuxThe test at line ~626 currently asserts event_id == EVENT_ID_FILE_CHANGE, so it locks in the wrong value — worth updating it to assert 11 so the parity is enforced.
| } | ||
|
|
||
| /// Acting process context shared by all file events: pid, executable, user. | ||
| fn actor(msg: &Message) -> (u32, Option<String>, String) { |
There was a problem hiding this comment.
Per-event getpwuid_r on the file hot path.
actor() calls resolved_user(token.ruid()) → lookup_username_by_uid, which runs sysconf + getpwuid_r (a Directory Services / opendirectoryd lookup on macOS) and allocates a buffer on every event — including every unlink and every modified close, both high-volume. This runs inside the mapping on ES's dispatch queue, so under heavy file activity it risks the handler falling behind and events being dropped via the try_send-full path.
The Linux sensor avoids this entirely on the file path — it passes the raw uid.to_string() and never does a directory lookup (resolve_user_field in the normalizer only resolves Windows S-1- SIDs, so it's a no-op for macOS uids).
Suggestion: memoize uid→username (a small Mutex<HashMap<u32, String>>), or defer resolution downstream the way Linux does. (Minor: for create/rename the destination()? early-return runs before actor(), so those skip the lookup when there's no destination — but unlink and modified-close still hit it unconditionally.)
Report file-modify under FileCreate (event id 11), matching the Linux sensor, instead of 65. Sysmon has no dedicated file-modify event, so file-change Sigma rules key on EventID 11; the previous value silently broke that parity. The action code stays 65. Test now asserts 11.
Emit the raw uid as a string for process and file events and defer username resolution to the normalizer, matching the Linux sensor. This removes a getpwuid_r directory-services lookup that ran on Endpoint Security's dispatch queue for every event, including high-volume unlink and modified-close, where it risked the handler falling behind and dropping events. lookup_username_by_uid goes back to Linux-only since the macOS sensor no longer calls it.
|
Thanks, both good catches. Addressed against 1. File-modify event id parity ( 2. Per-event Verified on macOS arm64: |
Summary
Adds the macOS Endpoint Security sensor and wires it into the runtime in place of the placeholder. Process and file events map into the shared event model with Sysmon-compatible identifiers, so existing Sigma, YARA, and IOC detection works on macOS process and file activity.
endpoint-secdependency and anEsfSensorthat owns the ES client on a dedicated thread and surfaces client init failures (not root, missing entitlement, TCC denial) synchronously.Process and file events emit the raw uid and defer username resolution to the normalizer, matching the Linux sensor.
Type of change
feat/enhancement- new featureTest plan
cargo test)Note for maintainers: creating an Endpoint Security client requires root and the
com.apple.developer.endpoint-security.cliententitlement on signed and notarized builds. For local testing you can run with SIP and AMFI relaxed. The live client path was exercised manually under those conditions; the mappings are covered by unit tests.Checklist
Notes
Second of four in the #41 series, on top of #42 (merged). Verified on macOS arm64:
cargo fmt --check,cargo clippy --locked --all-targets -- -D clippy::all, andcargo test --locked(134 passed) are all clean.Part of #41.