.agents/skills/query-agent-events/AE_FIELDS.md
Verified field map for the agent-events journal namespace.
Every claim here is traceable to producer source at
<repo>/src/daemon/status-file.c (the schema is
STATUS_FILE_VERSION = 29, src/daemon/status-file.h:14).
The .local draft agent-events-journals.md was found to have
14 high-severity divergences (wrong enums, missing fields,
misattributed semantics). This file supersedes it.
Producer JSON path -> journal field name:
agent.profile[0]).--prefix 'AE_' (literal prepend; the
prefix is NOT transliterated --
src/collectors/log2journal/log2journal-help.c:108-109)._. For arrays, it appends
_<index> (log2journal-json.c:477-511).log2journal.c:8-61): lowercase -> uppercase, digits and
uppercase pass through, everything else (including .,
-, :, /, @, _, [, ]) maps to _. Consecutive
underscores are collapsed
(log2journal-json.c:395-396).So:
agent.id -> AE_AGENT_IDagent.profile[0] -> AE_AGENT_PROFILE_0host.boot.id -> AE_HOST_BOOT_ID@timestamp -> AE__TIMESTAMP (note: double underscore
because @ -> _)fatal.errno -> AE_FATAL_ERRNO (NOT from a top-level
fatal_errno; the path is nested)Added by post_status_file() at status-file.c:967-976. These
sit at the JSON top level, so they have NO nested-path
prefix in the journal field name.
| JSON path | Journal field | Type | Always? | Triage meaning |
|---|---|---|---|---|
exit_cause | AE_EXIT_CAUSE | string | yes | Human-readable label for why this session ended. The first thing to look at. See enum below. |
message | AE_MESSAGE | string | yes | One-line summary; subject of FTS search. |
priority | AE_PRIORITY | int | yes | Syslog priority (lower = more severe). |
version_saved | AE_VERSION_SAVED | uint | yes | The status file's own version (typically 29). |
agent_version_now | AE_AGENT_VERSION_NOW | string | yes | Posting agent's version (the agent that did the POST = next session, NOT the one that crashed). |
agent_pid_now | AE_AGENT_PID_NOW | uint | yes | PID of the agent that POSTed. |
host_memory_critical | AE_HOST_MEMORY_CRITICAL | bool | yes | Was the host under memory pressure at POST time? |
host_memory_free_percent | AE_HOST_MEMORY_FREE_PERCENT | uint | yes | % free RAM at POST time. |
agent_health | AE_AGENT_HEALTH | string | yes | Health classification across restart history. See enum below. |
@timestamp | AE__TIMESTAMP | RFC3339 | yes | When the captured session ended. |
version | AE_VERSION | uint | yes | The schema version (STATUS_FILE_VERSION). Use to slice by schema. |
These fields are always present on every record and are
the safest scoping anchors. AE_EXIT_CAUSE, AE_AGENT_HEALTH,
AE_VERSION are all good index-friendly facets.
agent.* fields (dsf_json_agent, status-file.c:99-156)The previous (crashed) session's agent state.
| JSON path | Journal field | Type | Version-gating | Triage meaning |
|---|---|---|---|---|
agent.id | AE_AGENT_ID | UUID | always | Netdata machine GUID (per-install, persistent). The "agent identity". DIFFERENT from host.id. |
agent.since | AE_AGENT_SINCE | RFC3339 | v>=24 | When this install was first registered. |
agent.ephemeral_id | AE_AGENT_EPHEMERAL_ID | UUID | always | Unique per Netdata invocation (changes on every restart). Use to group multiple events from the same crashed session. |
agent.version | AE_AGENT_VERSION | string | always | The version of the crashed session. Slice on this for regression-spotting. |
agent.uptime | AE_AGENT_UPTIME | int seconds | always | Duration the crashed session ran. Short uptime + crash = startup bug. |
agent.node_id | AE_AGENT_NODE_ID | UUID | always | Cloud node UUID (empty when agent isn't claimed). |
agent.claim_id | AE_AGENT_CLAIM_ID | UUID | always | Cloud claim UUID (empty when not claimed). Presence -> cloud-connected agent. |
agent.restarts | AE_AGENT_RESTARTS | uint | always | Total restart count for this install. High value + recent crash = agent loop. |
agent.crashes | AE_AGENT_CRASHES | uint | v>=24 | Total crash count. |
agent.pid | AE_AGENT_PID | uint | v>=27 | PID of the crashed session. |
agent.posts | AE_AGENT_POSTS | uint | v>=22 | Total POSTs from this install. |
agent.aclk | AE_AGENT_ACLK | enum | v>=22 | Cloud connection state. See enum. |
agent.profile[N] | AE_AGENT_PROFILE_0..N | enum array | always | Bitmap rendered as array. Slice by parent vs child vs iot. See enum. |
agent.status | AE_AGENT_STATUS | enum | always | DAEMON_STATUS at the time of capture. See enum. |
agent.exit_reason[N] | AE_AGENT_EXIT_REASON_0..N | enum array | always | EXIT_REASON bitmap rendered as array. Empty array = no specific reason. See enum. |
agent.install_type | AE_AGENT_INSTALL_TYPE | string | always | kickstart, binpkg, static, etc. Slice on this for "is this a packaging issue?" |
agent.db_mode | AE_AGENT_DB_MODE | string | v>=14 | dbengine memory mode. |
agent.db_tiers | AE_AGENT_DB_TIERS | uint | v>=14 | Number of dbengine tiers. |
agent.kubernetes | AE_AGENT_KUBERNETES | bool | v>=14 | Kubernetes deployment? Slice on this for k8s-specific issues. |
agent.sentry_available | AE_AGENT_SENTRY_AVAILABLE | bool | v>=16 | Is Sentry enabled? |
agent.reliability | AE_AGENT_RELIABILITY | int | always | Signed reliability counter (positive = healthy run streak; negative = crash streak). <= -2 -> crash-loop. |
agent.stack_traces | AE_AGENT_STACK_TRACES | string | always | Backtrace backend name (libbacktrace, none). |
agent.timings.init | AE_AGENT_TIMINGS_INIT | int seconds | always | How long startup took. Long init + crash = startup bug. |
agent.timings.exit | AE_AGENT_TIMINGS_EXIT | int seconds | always | How long shutdown took. |
metrics.* fields (dsf_json_metrics, :158-193)Snapshot of the database at the time of capture. Useful for "big-database crashes" investigations.
| JSON path | Journal field |
|---|---|
metrics.nodes.total | AE_METRICS_NODES_TOTAL |
metrics.nodes.receiving | AE_METRICS_NODES_RECEIVING |
metrics.nodes.sending | AE_METRICS_NODES_SENDING |
metrics.nodes.archived | AE_METRICS_NODES_ARCHIVED |
metrics.metrics.collected | AE_METRICS_METRICS_COLLECTED |
metrics.metrics.available | AE_METRICS_METRICS_AVAILABLE |
metrics.instances.collected | AE_METRICS_INSTANCES_COLLECTED |
metrics.instances.available | AE_METRICS_INSTANCES_AVAILABLE |
metrics.contexts.collected | AE_METRICS_CONTEXTS_COLLECTED |
metrics.contexts.available | AE_METRICS_CONTEXTS_AVAILABLE |
host.* fields (dsf_json_host, :195-253)Host-level info, mostly stable across crashes on the same host.
| JSON path | Journal field | Triage meaning |
|---|---|---|
host.id | AE_HOST_ID | OS-level /etc/machine-id. Different from AE_AGENT_ID (Netdata's own identifier). Use to spot multiple agents on the same host. |
host.architecture | AE_HOST_ARCHITECTURE | x86_64, aarch64, armv7l, ... Slice for arch-specific bugs. |
host.virtualization | AE_HOST_VIRTUALIZATION | none, kvm, vmware, lxc, docker, ... |
host.container | AE_HOST_CONTAINER | none, docker, kubernetes, ... |
host.uptime | AE_HOST_UPTIME | MISLEADING NAME. Stores boottime EPOCH (status-file.c:202 writes ds->boottime from now_boottime_sec()). NOT a duration. Compute uptime via now - AE_HOST_UPTIME. |
host.timezone | AE_HOST_TIMEZONE | string, v>=20 |
host.cloud_provider | AE_HOST_CLOUD_PROVIDER | aws, gcp, azure, ..., v>=20 |
host.cloud_instance | AE_HOST_CLOUD_INSTANCE | EC2 instance type etc., v>=20 |
host.cloud_region | AE_HOST_CLOUD_REGION | v>=20 |
host.system_cpus | AE_HOST_SYSTEM_CPUS | uint. Slice for "low-cpu environment" bugs. |
host.boot.id | AE_HOST_BOOT_ID | UUID, changes on every host boot. |
host.memory.total | AE_HOST_MEMORY_TOTAL | bytes, only when OS_SYSTEM_MEMORY_OK |
host.memory.free | AE_HOST_MEMORY_FREE | bytes |
host.memory.netdata | AE_HOST_MEMORY_NETDATA | bytes used by netdata, v>=21 |
host.memory.oom_protection | AE_HOST_MEMORY_OOM_PROTECTION | uint, v>=21 |
host.disk.db.total | AE_HOST_DISK_DB_TOTAL | bytes available to dbengine |
host.disk.db.free | AE_HOST_DISK_DB_FREE | bytes free |
host.disk.db.inodes_total | AE_HOST_DISK_DB_INODES_TOTAL | uint |
host.disk.db.inodes_free | AE_HOST_DISK_DB_INODES_FREE | uint |
host.disk.db.read_only | AE_HOST_DISK_DB_READ_ONLY | bool. True + crash -> "disk read-only" cause. |
host.disk.netdata.dbengine | AE_HOST_DISK_NETDATA_DBENGINE | bytes used by dbengine files. |
host.disk.netdata.sqlite | AE_HOST_DISK_NETDATA_SQLITE | bytes used by SQLite files. |
host.disk.netdata.other | AE_HOST_DISK_NETDATA_OTHER | bytes used by other files. |
host.disk.netdata.last_updated | AE_HOST_DISK_NETDATA_LAST_UPDATED | RFC3339. |
os.* fields (dsf_json_os, :255-266)| JSON path | Journal field | Triage meaning |
|---|---|---|
os.type | AE_OS_TYPE | enum: unknown, linux, freebsd, macos, windows. |
os.kernel | AE_OS_KERNEL | Kernel version string. |
os.name | AE_OS_NAME | Distro name (e.g. Ubuntu, CentOS Stream). |
os.version | AE_OS_VERSION | Distro version. |
os.family | AE_OS_FAMILY | os_id (e.g. ubuntu). Slice for distro-specific issues. |
os.platform | AE_OS_PLATFORM | os_id_like (parent distro family, e.g. debian). NOT a rewrite of AE_OS_FAMILY -- they're independent producer fields. |
hw.* fields (dsf_json_hw, :268-319)DMI / SMBIOS data. Useful for hardware-specific bug
investigation. Privacy-sensitive serials and asset_tags are
commented out at the producer side (status-file.c:275-276, :294-295, :304-305) and never reach the journal.
| JSON path | Journal field | Notes |
|---|---|---|
hw.sys.vendor | AE_HW_SYS_VENDOR | BIOS / system vendor. |
hw.sys.uuid | AE_HW_SYS_UUID | System UUID. |
hw.product.id | AE_HW_PRODUCT_ID | Raw platform product/model identifier (e.g. Mac16,10). |
hw.product.name | AE_HW_PRODUCT_NAME | Human-friendly product name when available (e.g. Mac mini (M4, 2024)). |
hw.product.version | AE_HW_PRODUCT_VERSION | |
hw.product.sku | AE_HW_PRODUCT_SKU | |
hw.product.family | AE_HW_PRODUCT_FAMILY | |
hw.board.name | AE_HW_BOARD_NAME | |
hw.board.version | AE_HW_BOARD_VERSION | |
hw.board.vendor | AE_HW_BOARD_VENDOR | |
hw.chassis.type | AE_HW_CHASSIS_TYPE | Numeric (e.g. 6 = desktop, 9 = laptop). |
hw.chassis.vendor | AE_HW_CHASSIS_VENDOR | |
hw.chassis.version | AE_HW_CHASSIS_VERSION | |
hw.bios.date | AE_HW_BIOS_DATE | |
hw.bios.release | AE_HW_BIOS_RELEASE | |
hw.bios.version | AE_HW_BIOS_VERSION | |
hw.bios.vendor | AE_HW_BIOS_VENDOR |
product.* fields (dsf_json_product, :321-330)| JSON path | Journal field |
|---|---|
product.vendor | AE_PRODUCT_VENDOR |
product.id | AE_PRODUCT_ID |
product.name | AE_PRODUCT_NAME |
product.type | AE_PRODUCT_TYPE |
fatal.* fields (dsf_json_fatal, :331-367)Present on crashes and deliberate fatal conditions. Empty on graceful exits.
| JSON path | Journal field | Version-gating | Triage meaning |
|---|---|---|---|
fatal.line | AE_FATAL_LINE | always | Source line of the panic. Combine with FILENAME and FUNCTION for de-dup. |
fatal.filename | AE_FATAL_FILENAME | always | Source file. Slice on this for "this file is buggy". |
fatal.function | AE_FATAL_FUNCTION | always | Function name (with demangled symbol). Slice on this for "this function is buggy". |
fatal.message | AE_FATAL_MESSAGE | always | Panic message. Subject of FTS. |
fatal.errno | AE_FATAL_ERRNO | always | errno string at panic. |
fatal.thread | AE_FATAL_THREAD | always | Worker thread name (e.g. CTXLOAD, STREAM:63). |
fatal.thread_id | AE_FATAL_THREAD_ID | always | POSIX TID. |
fatal.stack_trace | AE_FATAL_STACK_TRACE | always | Backtrace. Real addresses preserved (anonymization is dedup-only, status-file-dedup.c:26-36). |
fatal.signal_code | AE_FATAL_SIGNAL_CODE | v>=16 | SIGNAL/SI_CODE formatted (e.g. SIGSEGV/SEGV_MAPERR). Empty -> not a signal crash. Primary signal-crash predicate. See enum. |
fatal.sentry | AE_FATAL_SENTRY | v>=17 | Was a Sentry submission attempted? |
fatal.fault_address | AE_FATAL_FAULT_ADDRESS | v>=18 | Hex address of the fault. Empty when signal_code == 0. |
fatal.worker_job_id | AE_FATAL_WORKER_JOB_ID | v>=23 | Worker job ID at panic. |
AE_AGENT_STATUS (DAEMON_STATUS)Source: src/daemon/status-file.c:23-33.
| Value | Meaning for triage |
|---|---|
none | No prior status (very first session). |
initializing | Crashed during startup -> startup bug. Combine with agent.timings.init for context. |
running | Crashed during normal operation -> the most "interesting" class. |
exiting | Crashed during shutdown -> shutdown-path bug. |
exited | Graceful exit (no crash). |
AE_AGENT_ACLK (CLOUD_STATUS)Source: src/claim/cloud-status.c:5-15.
| Value | Meaning for triage |
|---|---|
available | Default; not yet attempted. |
online | Connected to Cloud (ACLK up). |
indirect | Connected via parent. |
banned | Cloud rejected (claim issue). |
offline | Disconnected (network or shutdown). |
(The .local draft listed a disabled value -- it does NOT
exist in the producer source.)
AE_AGENT_HEALTHSource: src/daemon/status-file.c:929-952. Computed by the
agent (not the ingestion server) at POST time across
restart history. Used to isolate crash classes.
| Value | Meaning for triage |
|---|---|
healthy-first | First run, no prior crashes. Boring (filter out). |
healthy-repeated | Multiple healthy runs in a row. |
healthy-loop | Reliability >= 2 consecutive healthy runs. |
healthy-recovered | Was unhealthy, now healthy. |
crash-first | First crash ever on this install. Interesting -- new bug? |
crash-entered | Single crash, then recovered. |
crash-loop | Reliability <= -2 (repeated crashes). Highest-priority class. |
crash-repeated | Two or more crashes. |
To find ALL crashes: (AE_AGENT_HEALTH in crash-first, crash-loop, crash-repeated, crash-entered).
AE_AGENT_PROFILE_* (ND_PROFILE bitmap)Source: src/daemon/config/netdata-conf-profile.c:7-15.
| Value | Meaning |
|---|---|
standalone | Single-node deployment. |
parent | Streaming parent. Slice for "parent-only" bugs. |
child | Streaming child. Slice for "child-only" bugs. |
iot | IoT / lightweight profile. |
(The .local draft listed dopple and store-child -- they do
NOT exist; iot was missing.)
AE_AGENT_EXIT_REASON_* (EXIT_REASON bitmap)Source: src/libnetdata/exit/exit_initiated.c:7-38. The
EXIT_REASON bitmap renders as a JSON array. Empty bitmap ->
empty array (no none element).
20 distinct strings:
| Value | Meaning |
|---|---|
signal-segmentation-fault | SIGSEGV received. |
signal-bus-error | SIGBUS received. |
signal-floating-point-exception | SIGFPE received. |
signal-illegal-instruction | SIGILL received. |
signal-abort | SIGABRT received (assertion / abort()). |
signal-bad-system-call | SIGSYS received. |
signal-cpu-time-limit-exceeded | SIGXCPU received. |
signal-file-size-limit-exceeded | SIGXFSZ received. |
signal-quit | SIGQUIT received. |
signal-terminate | SIGTERM received (graceful kill). |
signal-interrupt | SIGINT received (Ctrl-C). |
out-of-memory | OOM panic. |
already-running | Another instance held the listen socket. |
fatal | Generic fatal() call. |
api-quit | API endpoint requested exit. |
cmd-exit | Explicit netdata --exit invocation. |
service-stop | Service manager (systemd) sent stop. |
system-shutdown | Host shutting down. |
update | Replaced by a new version. |
shutdown-timeout | Shutdown took too long. |
(The .local draft was significantly wrong here -- listed
~10 invented values like exit-called, exit-and-update,
cannot-allocate, oom, assertion-failed, none of which
exist in producer source.)
AE_EXIT_CAUSE (top-level)Source: src/daemon/status-file.c:1097-1286. Computed by the
agent, NOT the ingestion server. The most useful field for
classifying records.
26 distinct strings:
Initial / no prior state (1):
| Value | Meaning |
|---|---|
no last status | First-ever start; no prior status file readable. |
Prior was EXITED (graceful) (7):
| Value | Meaning |
|---|---|
exit no reason | Prior exited cleanly with no reason recorded. |
deadly signal and exit | Got a deadly signal but exited normally. |
fatal and exit | Hit a fatal but managed to exit. |
exit on system shutdown | Host shutting down; agent stopped gracefully. |
exit to update | Stopped to allow an update. |
exit and updated | Stopped and was replaced by a new version. |
exit instructed | netdata --exit or service stop. |
Prior was INITIALIZING (8):
| Value | Meaning |
|---|---|
abnormal power off | Power loss during startup. |
deadly signal on start | Signal during startup. |
out of memory | OOM during startup. (.local draft says cannot allocate -- wrong.) |
already running | Listen socket conflict at init. |
disk read-only | Filesystem read-only at init. |
disk full | Disk full at init. |
disk almost full | Disk near capacity at init. |
fatal on start | fatal() during startup. |
killed hard on start | SIGKILL/SIGTERM during startup. |
Prior was EXITING (5):
| Value | Meaning |
|---|---|
deadly signal on exit | Signal during shutdown. |
exit timeout | Shutdown didn't complete in time. |
fatal on exit | fatal() during shutdown. |
killed hard on shutdown | SIGKILL during shutdown (host shutdown). |
killed hard on update | SIGKILL during hot update. |
killed hard on exit | SIGKILL during exit. |
Prior was RUNNING (6):
| Value | Meaning |
|---|---|
abnormal power off | Power loss during normal operation. |
out of memory | OOM during normal operation. |
deadly signal | Signal received during normal operation. |
killed fatal | SIGKILL after a fatal. |
killed hard low ram | OOM-killed (RAM pressure). |
killed hard | SIGKILL/SIGTERM from outside. |
AE_OS_TYPE (DAEMON_OS_TYPE)Source: src/daemon/status-file.c:35-45.
unknown, linux, freebsd, macos, windows.
AE_FATAL_SIGNAL_CODEFormat: SIGNAL/SI_CODE (e.g. SIGSEGV/SEGV_MAPERR). Sources:
src/libnetdata/signals/signal-code.c:12-53 (signal name map),
:97-184 (per-signal SI_CODE map).
Most relevant for crash triage:
| Value | Meaning |
|---|---|
SIGSEGV/SEGV_MAPERR | Invalid memory map (NULL pointer, freed memory). |
SIGSEGV/SEGV_ACCERR | Access violation (write to read-only page). |
SIGSEGV/SEGV_BNDERR | Address bound check fault. |
SIGSEGV/SEGV_PKUERR | Protection key fault. |
SIGBUS/BUS_ADRALN | Alignment error. |
SIGBUS/BUS_ADRERR | Non-existent physical address. |
SIGBUS/BUS_OBJERR | Object-specific bus error. |
SIGFPE/FPE_INTDIV | Integer divide by zero. |
SIGFPE/FPE_INTOVF | Integer overflow. |
SIGFPE/FPE_FLTDIV | Float divide by zero. |
SIGABRT/SI_TKILL | abort() / assertion failure (typical SI_CODE for abort()). |
SIGTRAP/TRAP_BRKPT | Breakpoint trap. |
SIGTRAP/TRAP_TRACE | Trace trap. |
Empty AE_FATAL_SIGNAL_CODE -> not a signal crash (it's a
deliberate fatal or a graceful exit).
(The .local draft had SIGABRT/ABRT which is wrong: ABRT
is not a valid SI_CODE token. And SIGTRAP/TRAP Trace should
be SIGTRAP/TRAP_TRACE. And SIGVTALRM/VTALRM does not
exist as a per-signal SI_CODE.)
These fields are the first-pass slicers for queries.
Always include at least 1-2 of these in selections before
falling back to FTS:
AE_AGENT_VERSION -- regression / fix-detection.AE_AGENT_HEALTH -- crash class.AE_EXIT_CAUSE -- exit class.AE_FATAL_SIGNAL_CODE -- signal type.AE_FATAL_FUNCTION -- localize to a function.AE_FATAL_FILENAME -- localize to a file.AE_HOST_ARCHITECTURE -- arch-specific bugs.AE_OS_FAMILY -- distro-specific bugs.AE_AGENT_PROFILE_0 (and _1, _2) -- parent / child / iot.AE_AGENT_KUBERNETES -- k8s-specific.AE_AGENT_INSTALL_TYPE -- packaging-specific.Treat these as identifying. The redact-events.sh opt-in
filter masks them when sharing:
AE_AGENT_ID (machine GUID).AE_HOST_ID (OS machine-id).AE_AGENT_NODE_ID, AE_AGENT_CLAIM_ID (Cloud identifiers).AE_HOST_BOOT_ID, AE_AGENT_EPHEMERAL_ID.AE_HW_SYS_UUID.AE_HW_*) when correlated with serial-equivalent
identifiers.(Privacy-sensitive serials and asset_tags are already
commented out at the producer side and never reach the
journal -- status-file.c:275-276, :294-295, :304-305.)
Several producer fields are intentionally redacted at the
producer side (commented out in dsf_json_hw):
hw.sys.serial, hw.sys.asset_taghw.board.serial, hw.board.asset_taghw.chassis.serial, hw.chassis.asset_tagThere is no agent.happiness field in the producer source at
any version. The .local draft mentioned it -- the field has
never existed.
status-file-dedup.c:26-36 zeroes out hex addresses ONLY when
computing the dedup hash. The journal-emitted
AE_FATAL_STACK_TRACE retains real addresses. Useful for
bug investigation; sensitive when sharing externally.