SIEM export

What this is. A SIEM (Security Information and Event Management system) is the searchable central database where a SOC (security operations center — the team watching for attacks) collects security events from every tool it runs. A SOC runs its own SIEM (Splunk, Sentinel, Elastic, Chronicle) and wants probectl's security-relevant events flowing into it. This feature forwards two streams — probectl's audit log and its threat-plane signals — into that SIEM, rendered in a standard wire format and pushed over hardened TLS.

The important framing: it is a forwarder, not a SIEM. probectl does not store, search, or correlate these events for you, and it never blocks traffic or acts as an IPS. A threat finding is a confidence-scored signal that the SIEM correlates (detection is a signal, never an enforcement action — one of probectl's non-negotiables). The code is internal/siem (pure — formatters, senders, a buffered forwarder), driven by internal/control which maps audit.Event + incident.Signal onto the canonical siem.Event.

Off by default. Enabling it opens an outbound connection to the operator's SIEM, so it is explicit and config-gated — off unless PROBECTL_SIEM_ENABLED=true (sovereignty / no-phone-home).

Inbound syslog is a separate collection path. probectl can also accept device/security syslog as an input for correlation, but that does not make it the operator's log-search SIEM. The inbound receiver is tenant-bound by source configuration, parses RFC 5424 and RFC 3164 lines as untrusted input, rate-limits each source, stores provenance with the normalized event, and requires either an HMAC signature or a TLS client-certificate subject. The listener is TLS-only: missing TLS config fails closed.

What is forwarded

Both map onto one canonical siem.Event, so every output format carries the same fields: time, tenant, category, action, severity, actor, target, outcome, message, and attributes. One canonical record, four renderings — the formatters only change the costume, never the facts.

Wire formats

A wire format is the exact byte layout the receiving SIEM expects. Selectable via PROBECTL_SIEM_FORMAT (or the preset's default):

• syslog — RFC 5424, the classic line-based log protocol, with structured data ([probectl@32473 tenant="…" …]).
• cef — ArcSight CEF (the pipe-delimited Common Event Format: CEF:0|probectl|probectl|…), tenant in cs1.
• ecs — Elastic Common Schema JSON (Elastic's standard field naming: event.*, organization.id = tenant).
• otlp — OTLP/HTTP logs JSON (the OpenTelemetry log protocol; resource attr probectl.tenant_id).

Presets

PROBECTL_SIEM_PRESET adapts the auth header + the default format to a target SIEM. The endpoint is operator-supplied (the HEC / ingest / Elasticsearch URL — HEC is Splunk's HTTP Event Collector, its token-authenticated ingest endpoint):

The scheme in that table is the preset's default, not a fixed property of the preset: PROBECTL_SIEM_AUTH_SCHEME pins apikey, bearer, basic, splunk or none when your target disagrees. The case that matters in practice is OpenSearch, the Elastic-compatible engine most self-hosters run: it speaks ECS and the same ingest API, but it authenticates with HTTP Basic, not with an Elasticsearch API key. Point it at

PROBECTL_SIEM_PRESET=elastic
PROBECTL_SIEM_AUTH_SCHEME=basic
PROBECTL_SIEM_TOKEN=<user>:<password>     # encoded for you; inject it as a secret
PROBECTL_SIEM_ENDPOINT=https://opensearch.example:9200/probectl-events/_doc

The token is never logged and never appears in a response; basic base64-encodes it so an operator does not have to pre-encode a credential into configuration.

Delivery guarantees (no drops)

A SIEM is a security audit destination, so the design rule is: never silently drop an event — a gap in the security record is indistinguishable from an attacker erasing tracks. The two streams reach that guarantee by different means.

%%{init: {'theme':'base','themeVariables':{'background':'#0d1117','primaryColor':'#161b22','primaryTextColor':'#e6edf3','primaryBorderColor':'#3b82f6','lineColor':'#8b949e','secondaryColor':'#21262d','tertiaryColor':'#0d1117','clusterBkg':'#161b22','clusterBorder':'#30363d','fontFamily':'ui-monospace, SFMono-Regular, Menlo, monospace'},'flowchart':{'curve':'basis','nodeSpacing':55,'rankSpacing':55,'padding':12}}}%%
flowchart LR
  subgraph control[Control plane]
    A[Audit chain<br/>per tenant] -->|Drain from cursor| P[SIEM audit poller]
    T[Threat consumers<br/>TLS / IOC] -->|Enqueue| B[(bounded buffer)]
    P -->|Deliver| F[Forwarder<br/>format + retry]
    B --> F
    C[(siem_delivery<br/>per-tenant cursor)] <-->|advance only past delivered| P
  end
  F -->|HTTPS + auth| S[Operator SIEM]

• Audit path — SIEMAuditPoller drains each tenant's audit events from a durable per-tenant cursor (siem_delivery, RLS-scoped). A cursor is a persisted bookmark: "everything before this point was delivered." It drains one bounded page transaction and advances the committed cursor only past events the SIEM acknowledged — like a registered-mail clerk who crosses an item off the ledger only when the signed receipt is in hand. So a restart resumes exactly where it paused — no drops, and (outside a narrow crash window) no re-sends.
• Replica concurrency — each page transaction creates the cursor row when needed and locks it with SELECT ... FOR UPDATE before reading audit events. A second replica waits, then observes the committed cursor, so overlapping pollers do not forward the same page twice. A process crash after remote acknowledgment but before commit can still replay that final event; SIEM receivers should therefore keep their documented (tenant_id, audit.seq) idempotency key. This is at-least-once delivery with no gaps, not a false exactly-once claim.
• Threat path — consumers enqueue signals into a bounded buffer; when it is full, producers block (backpressure — the pipeline slows down rather than throwing events away) instead of dropping. A worker delivers with exponential-backoff retry (each failed attempt waits twice as long, up to a ceiling, so a struggling SIEM is never hammered).
• Outage handling — a SIEM outage pauses the cursor (the poller commits whatever was delivered and resumes next tick); the buffer applies backpressure. Nothing is silently discarded.

Governance & redaction

Exported audit events are scrubbed of secrets / PII before they leave the network — the SIEM gets the security record, never a copy of the credentials inside it. A built-in case-insensitive denylist (password, passwd, secret, token, api_key, apikey, authorization, cookie, private_key, client_secret, ssn) plus any keys in PROBECTL_SIEM_REDACT_KEYS are matched on the audit record's data keys. A redacted value becomes [redacted] — the key is kept, so the SIEM still sees the shape of the event without the sensitive value. After key redaction, the audit-to-SIEM mapper also runs the core governance PII scanner over actor, action, target, outcome, and every remaining data value before any syslog/CEF/ECS/OTLP formatter sees the event. That scanner masks emails and IPs, strips URL query strings/fragments and path IDs, and blanks embedded bearer tokens or token=... / password=...-style secrets.

Security

• TLS out — delivery uses the hardened, certificate-validating HTTP client (crypto.HardenedHTTPClient); validation is never disabled. The ingest token is sent only as an auth header, never in a URL (URLs land in proxy and access logs; headers do not).
• Tenant isolation — the audit poller drains inside each tenant's RLS scope; the tenant stamped on every exported record is the drained scope's tenant, never a value from the event body. One tenant's data can never be forwarded under another's id.
• Secrets — the ingest token is runtime config; inject it from a secret manager, never commit it, and it is never logged.
• Syslog in — inbound syslog is stamped with the tenant from the registered source, not from the payload. The source must authenticate with sha256=<hmac> over the exact line or with a configured TLS client subject; malformed, oversized, unsigned, forged, or rate-limited messages are rejected before storage.

Checking delivery posture

Three equivalent surfaces answer "is this actually being forwarded, and to what":

• UI — Admin → SIEM export posture. Says whether the exporter is running, which streams are forwarded, the resolved format and preset, the destination host:port, the back-pressure policy, the poll cadence and buffer size, and how many redaction keys are in effect. When it is not delivering it shows the server's own reason (disabled, missing_endpoint, insecure_endpoint, invalid_format) rather than an empty panel.
• CLI — probectl siem status.
• API — GET /v1/siem/status (requires threat.read).

All three are posture, never values. The endpoint path, its query string and the ingest token are never returned by the API, so no surface can display them; the endpoint appears as scheme-free host:port and the credential as configured / not configured. Read that as designed, not as missing detail — see Security above.

Configuration

See configuration.md for the full key table. Minimal Splunk HEC example:

PROBECTL_SIEM_ENABLED=true
PROBECTL_SIEM_PRESET=splunk
PROBECTL_SIEM_ENDPOINT=https://splunk.example:8088/services/collector/raw
PROBECTL_SIEM_TOKEN=<hec-token>