Output flushing is controlled by values in the configuration section ``heartbeat``. By default, Suricata's
output is synchronous with little possibility that written data will not be persisted. However, if ``output.buffer-size``
has a non-zero value, then some data may be written for the output, but not actually flushed. ``buffer-size`` bytes
may be held in memory and written a short time later opening the possibility -- but limited -- for output data
loss.

Hence, a heartbeat mechanism is introduced to limit the amount of time buffered data may exist before being
flushed.  Control is provided to instruct Suricata's detection threads to flush their EVE output. With default
values, there is no change in output buffering and flushing behavior. ``output-flush-interval`` controls
how often Suricata's detect threads will flush output in a heartbeat fashion. A value of ``0`` means
"never"; non-zero values must be in ``[1-60]`` seconds.

Flushing should be considered when ``outputs.buffer-size`` is greater than 0 to limit the amount and
age of buffered, but not persisted, output data.  Flushing is never needed when ``buffer-size`` is ``0``.


::

   heartbeat:
     #output-flush-interval: 0



Output types
~~~~~~~~~~~~

EVE can output to multiple methods. ``regular`` is a normal file. Other
options are ``syslog``, ``unix_dgram``, ``unix_stream`` and ``redis``.

Output types::

      filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
      filename: eve.json
      # Enable for multi-threaded eve.json output; output files are amended
      # with an identifier, e.g., eve.9.json. Default: off
      #threaded: off
      # Specify the amount of buffering, in bytes, for
      # this output type. The default value 0 means "no
      # buffering".
      #buffer-size: 0
      #prefix: "@cee: " # prefix to prepend to each log entry
      # the following are valid when type: syslog above
      #identity: "suricata"
      #facility: local5
      #level: Info ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug
      #ethernet: no # log ethernet header in events when available
      #suricata-version: no # include suricata version. Default no.
      #redis:
      #  server: 127.0.0.1
      #  port: 6379
      #  username: null ## ACL username; if null (default), no username is used
      #  password: null ## AUTH password; if null (default), authentication is disabled
      #  async: true ## if redis replies are read asynchronously
      #  mode: list ## possible values: list|lpush (default), rpush, channel|publish, xadd|stream
      #             ## lpush and rpush are using a Redis list. "list" is an alias for lpush
      #             ## publish is using a Redis channel. "channel" is an alias for publish
      #             ## xadd is using a Redis stream. "stream" is an alias for xadd
      #  key: suricata ## string denoting the key/channel/stream to use (default to suricata)
      #  stream-maxlen: 100000        ## Automatically trims the stream length to at most
                                      ## this number of events. Set to 0 to disable trimming.
                                      ## Only used when mode is set to xadd/stream.
      #  stream-trim-exact: false     ## Trim exactly to the maximum stream length above.
                                      ## Default: use inexact trimming (inexact by a few
                                      ## tens of items)
                                      ## Only used when mode is set to xadd/stream.
      # Redis pipelining set up. This will enable to only do a query every
      # 'batch-size' events. This should lower the latency induced by network
      # connection at the cost of some memory. There is no flushing implemented
      # so this setting as to be reserved to high traffic suricata.
      #  pipelining:
      #    enabled: yes ## set enable to yes to enable query pipelining
      #    batch-size: 10 ## number of entry to keep in buffer

.. _eve-output-alert:

Alerts
~~~~~~

Alerts are event records for rule matches. They can be amended with
metadata, such as the application layer record (HTTP, DNS, etc) an
alert was generated for, and elements of the rule.

The alert is amended with application layer metadata for signatures
using application layer keywords. It is also the case for protocols
over UDP as each single packet is expected to contain a PDU.

For other signatures, the option ``guess-applayer-tx``
can be used to force the detect engine to tie a transaction
to an alert.
This transaction is not guaranteed to be the relevant one,
depending on your use case and how you define relevant here.
**WARNING: If there are multiple live transactions, none will get
picked up.** This is to reduce the chances of logging unrelated data, and may
lead to alerts being logged without metadata, in some cases.
The alert event will have ``tx_guessed: true`` to recognize
such alerts.

Metadata::

        - alert:
            #payload: yes             # enable dumping payload in Base64
            #payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
            #payload-printable: yes   # enable dumping payload in printable (lossy) format
            #payload-length: yes      # enable dumping payload length, including the gaps
            #packet: yes              # enable dumping of packet (without stream segments)
            #http-body: yes           # Requires metadata; enable dumping of http body in Base64
            #http-body-printable: yes # Requires metadata; enable dumping of http body in printable format

            # metadata:

              # Include the decoded application layer (ie. http, dns)
              #app-layer: true

              # Log the current state of the flow record.
              #flow: true

              #rule:
                # Log the metadata field from the rule in a structured
                # format.
                #metadata: true

                # Log the raw rule text.
                #raw: false

                # Include the rule reference information
                #reference: false

Anomaly
~~~~~~~

Anomalies are event records created when packets with unexpected or anomalous
values are handled. These events include conditions such as incorrect protocol
values, incorrect protocol length values, and other conditions which render the
packet suspect. Other conditions may occur during the normal progression of a stream;
these are termed ``stream`` events are include control sequences with incorrect
values or that occur out of expected sequence.

Anomalies are reported by and configured by type:

- Decode
- Stream
- Application layer

Metadata::

    - anomaly:
        # Anomaly log records describe unexpected conditions such as truncated packets,
        # packets with invalid IP/UDP/TCP length values, and other events that render
        # the packet invalid for further processing or describe unexpected behavior on
        # an established stream. Networks which experience high occurrences of
        # anomalies may experience packet processing degradation.
        #
        # Anomalies are reported for the following:
        # 1. Decode: Values and conditions that are detected while decoding individual
        #    packets. This includes invalid or unexpected values for low-level protocol
        #    lengths as well.
        # 2. Stream: This includes stream related events (TCP 3-way handshake issues,
        #    unexpected sequence number, etc).
        # 3. Application layer: These denote application layer specific conditions that
        #    are unexpected, invalid or are unexpected given the application monitoring
        #    state.
        #
        # By default, anomaly logging is disabled. When anomaly logging is enabled,
        # application-layer anomaly reporting is enabled.
        #
        # Choose one or both types of anomaly logging and whether to enable
        # logging of the packet header for packet anomalies.
        types:
          #decode: no
          #stream: no
          #applayer: yes
        #packethdr: no

HTTP
~~~~

HTTP transaction logging.

Config::

    - http:
        extended: yes     # enable this for extended logging information
        # custom allows additional http fields to be included in eve-log
        # the example below adds three additional fields when uncommented
        #custom: [Accept-Encoding, Accept-Language, Authorization]
        # set this value to one among {both, request, response} to dump all
        # http headers for every http request and/or response
        # dump-all-headers: [both, request, response]

List of custom fields:

======================  ======================
Yaml Option             HTTP Header
======================  ======================
accept                  accept
accept_charset          accept-charset
accept_encoding         accept-encoding
accept_language         accept-language
accept_datetime         accept-datetime
authorization           authorization
cache_control           cache-control
cookie                  cookie
from                    from
max_forwards            max-forwards
origin                  origin
pragma                  pragma
proxy_authorization     proxy-authorization
range                   range
te                      te
via                     via
x_requested_with        x-requested-with
dnt                     dnt
x_forwarded_proto       x-forwarded-proto
x_authenticated_user    x-authenticated-user
x_flash_version         x-flash-version
accept_range            accept-range
age                     age
allow                   allow
connection              connection
content_encoding        content-encoding
content_language        content-language
content_length          content-length
content_location        content-location
content_md5             content-md5
content_range           content-range
content_type            content-type
date                    date
etag                    etags
expires                 expires
last_modified           last-modified
link                    link
location                location
proxy_authenticate      proxy-authenticate
referer                 referer
refresh                 refresh
retry_after             retry-after
server                  server
set_cookie              set-cookie
trailer                 trailer
transfer_encoding       transfer-encoding
upgrade                 upgrade
vary                    vary
warning                 warning
www_authenticate        www-authenticate
true_client_ip          true-client-ip
org_src_ip              org-src-ip
x_bluecoat_via          x-bluecoat-via
======================  ======================

In the ``custom`` option values from both columns can be used. The
``HTTP Header`` column is case insensitive.

.. _output-eve-dns:

DNS
~~~

.. note::

   As of Suricata 7.0 the v1 EVE DNS format has been removed.

   Version 2 EVE DNS will be removed in Suricata 9.

DNS records are logged as one entry for the request, and one entry for
the response.

YAML::

        - dns:
            #version: 3

            # Enable/disable this logger. Default: enabled.
            #enabled: yes

            # Control logging of requests and responses:
            # - requests: enable logging of DNS queries
            # - responses: enable logging of DNS answers
            # By default both requests and responses are logged.
            #requests: no
            #responses: no

            # Format of answer logging:
            # - detailed: array item per answer
            # - grouped: answers aggregated by type
            # Default: all
            #formats: [detailed, grouped]

            # Types to log, based on the query type.
            # Default: all.
            #types: [a, aaaa, cname, mx, ns, ptr, txt]

TLS
~~~

TLS records are logged one record per session.

YAML::

        - tls:
            extended: yes     # enable this for extended logging information
            # custom allows to control which tls fields that are included
            # in eve-log
            #custom: [subject, issuer, serial, fingerprint, sni, version, not_before, not_after, certificate, chain, ja3, ja3s, ja4]

The default is to log certificate subject and issuer. If ``extended`` is
enabled, then the log gets more verbose.

By using ``custom`` it is possible to select which TLS fields to log.
**Note that this will disable ``extended`` logging.**

ARP
~~~

ARP records are logged as one entry for the request, and one entry for
the response.

YAML::

        - arp:
            enabled: no

The logger is disabled by default since ARP can generate a large
number of events.

Netflow
~~~~~~~

Netflow records closely relate to flow records except that they are unidirectional while flow records
are bidirectional. This means that there will be twice as many netflow records as there are flow records.

Netflow records are disabled by default.

YAML::

      #- netflow

To enable netflow, change this to::

      - netflow:
          enabled: yes

MQTT
~~~~

EVE-JSON output for MQTT consists of one object per MQTT transaction, with some common and various type-specific fields.
Two aspects can be configured:

YAML::

        - mqtt:
            # passwords: yes           # enable output of passwords
            # string-log-limit: 1kb    # limit size of logged strings in bytes.
                                       # Can be specified in kb, mb, gb. Just a number
                                       # is parsed as bytes. Default is 1KB.
                                       # Use a value of 0 to disable limiting.
                                       # Note that the size is also bounded by
                                       # the maximum parsed message size (see
                                       # app-layer configuration)

The default is to output passwords in cleartext and not to limit the size of
message payloads. Depending on the kind of context the parser is used in (public
output, frequent binary transmissions, ...) this can be configured for regular
``mqtt`` events.

Drops
~~~~~

Drops are event types logged when the engine drops a packet.

Config::

    - drop:
        alerts: yes      # log alerts that caused drops
        flows: all       # start or all: 'start' logs only a single drop
                         # per flow direction. All logs each dropped pkt.
        # Enable logging the final action taken on a packet by the engine
        # (will show more information in case of a drop caused by 'reject')
        verdict: yes

.. _eve-json-output-stats:

Stats
~~~~~

Zero-valued Counters
""""""""""""""""""""

While the human-friendly `stats.log` output will only log out non-zeroed
counters, by default EVE Stats logs output all enabled counters, which may lead
to fairly verbose logs.

To reduce log file size, one may set `null-values` to false. Do note
that this may impact on the visibility of information for which a stats counter
as zero is relevant.

Config::

    - stats:
        # Don't log stats counters that are zero. Default: true
        #null-values: false    # False will NOT log stats counters: 0

Date modifiers in filename

By default, all output is written to the named filename in the outputs section. The ``threaded`` option enables
each output thread to write to individual files. In this case, the ``filename`` will include a unique identifier.

With ``threaded`` enabled, the output will be split among many files -- and
the aggregate of each file's contents must be treated together.

::

   outputs:
     - eve-log:
         filename: eve.json
         threaded: on

This example will cause each Suricata thread to write to its own "eve.json" file. Filenames are constructed
by adding a unique identifier to the filename.  For example, ``eve.7.json``.


Rotate log file
~~~~~~~~~~~~~~~

Eve-log can be configured to rotate based on time.

::

  outputs:
    - eve-log:
        filename: eve-%Y-%m-%d-%H:%M.json
        rotate-interval: minute

The example above creates a new log file each minute, where the filename contains
a timestamp. Other supported ``rotate-interval`` values are ``hour`` and ``day``.

In addition to this, it is also possible to specify the ``rotate-interval`` as a
relative value. One example is to rotate the log file each X seconds.

::

  outputs:
    - eve-log:
        filename: eve-%Y-%m-%d-%H:%M:%S.json
        rotate-interval: 30s

The example above rotates eve-log each 30 seconds. This could be replaced with
``30m`` to rotate every 30 minutes, ``30h`` to rotate every 30 hours, ``30d``
to rotate every 30 days, or ``30w`` to rotate every 30 weeks.

.. _multiple-eve-instances:

Multiple Logger Instances

- eve-log:
    enabled: yes
    type: file
    filename: eve-nsm.json
    types:
      - http
      - dns
      - tls

Log file permissions can be set individually for each logger. ``filemode`` can be used to
control the permissions of a log file, e.g.:

::

  outputs:
    - eve-log:
        enabled: yes
        filename: eve.json
        filemode: 600

The example above sets the file permissions on ``eve.json`` to 600, which means that it is
only readable and writable by the owner of the file.

JSON flags
~~~~~~~~~~

Several flags can be specified to control the JSON output in EVE:

::

  outputs:
    - eve-log:
        json:
          # Sort object keys in the same order as they were inserted
          preserve-order: yes

          # Make the output more compact
          compact: yes

          # Escape all unicode characters outside the ASCII range
          ensure-ascii: yes

          # Escape the '/' characters in string with '\/'
          escape-slash: yes

All these flags are enabled by default, and can be modified per EVE instance.

Community Flow ID

{
  "timestamp": "2003-12-16T13:21:44.891921+0000",
  "flow_id": 1332028388187153,
  "pcap_cnt": 1,
  "event_type": "alert",
  ...
  "community_id": "1:LQU9qZlK+B5F3KDmev6m5PMibrg=",
  "alert": {
    "action": "allowed",
    "gid": 1,
    "signature_id": 1,
  },
}
{
  "timestamp": "2003-12-16T13:21:45.037333+0000",
  "flow_id": 1332028388187153,
  "event_type": "flow",
  "flow": {
    "pkts_toserver": 5,
    "pkts_toclient": 4,
    "bytes_toserver": 338,
    "bytes_toclient": 272,
    "start": "2003-12-16T13:21:44.891921+0000",
    "end": "2003-12-16T13:21:45.346457+0000",
    "age": 1,
    "state": "closed",
    "reason": "shutdown",
    "alerted": true
  },
  "community_id": "1:LQU9qZlK+B5F3KDmev6m5PMibrg=",
}