Adding New Metrics ##################

Scripts allow users to create their own scripts and custom pages_ as long as long as defining their own timeseries schemas_. This is often enough to add custom metrics and visualize them. Some user however need to export internal metrics from the ntopng core. The information that follow should give a good starting point to do this.

.. _custom pages: ../../scripts/custom_pages.html .. _timeseries schemas: ../../scripts/timeseries_schemas.html

General Overview

Traffic elements (such as local hosts and interfaces) are iterated periodically and by some Lua scripts and their statistics are dumped in the form of timeseries. Traffic elements are handled in some standard ways:

Most traffic elements are implemented in C, and their statistics are passed to Lua via the :code:::lua method. For example, :code:AutonomousSystem::lua dumps the autonomous system statistics to Lua. Important if the element has a ::tsLua method check out the case 2 below.
Some other traffic elements are implemented in C, but their statistics are hold on a :code:TimeseriesPoint rather then the element itself. For example, the local hosts data is stored into the :code:HostTimeseriesPoint class. In order to add new timeseries for a local host, the :code:HostTimeseriesPoint is the class to modify (and related :code:::lua method).
Some traffic elements are implemented in Lua. Their state is stored in Redis usually in JSON form. This includes, for example, the SNMP devices.

In order to add a custom timeseries it's necessary to identify the correct case above. It's also important to note that not all the traffic elements can be exported. Remote Hosts and Flows timeseries, for example, cannot be exported anyway due to the design of ntopng.

Case 1: Adding metrics via the `::lua` method

The new metric should be exposed into the :code:::lua method. Then, you can simply add the metric to the custom timeseries scripts_. For example, the autonomous systems num_hosts field, exposed into the :code:AutonomousSystem::lua method, can be written as a timeseries in this way.

Please note that if the metric is already exposed into the :code:::lua method, you can keep compatibility with the standard ntopng and update it normally (no ntopng fork needed).

Case 2: Adding metrics inside a `TimeseriesPoint`

In order to add a new metric to a LocalHost or NetworkInterface, the corresponding :code:TimeseriesPoint should be modified instead:

For LocalHost, modify the :code:HostTimeseriesPoint
For NetworkInterface, modify the :code:NetworkInterfaceTsPoint

Things to keep in mind:

The new metric should be added to the header file (e.g. HostTimeseriesPoint.h).
The metric should be written to the :code:TimeseriesPoint, (e.g. in :code:LocalHostStats::makeTsPoint)
The metric should be exposed to Lua in the :code:TimeseriesPoint:lua method (e.g. in :code:HostTimeseriesPoint::lua)

After this, the metric should now be available in Lua. Use the custom timeseries scripts_ to export it as a timeseries. Since this requires modifications of the C source code, compatibility with the standard ntopng cannot be preserved.

Case 3: Adding metrics for Lua only objects

This really depends on the specific element to be added. Compatibility may or may not be assured.

.. _custom timeseries scripts: #custom-timeseries-scripts

Custom Timeseries Scripts

Once the new metrics are available in Lua via one of the methods discussed above, it's necessary to export such metrics as timeseries. In order to do so, two actions are required:

The metric format should be declared in a timeseries schema
The metric should be written to the timeseries driver

Both actions can be implemented inside the custom timeseries scripts.

ntopng handles custom timeseries with updates every:

1 minute for interfaces
5 minutes for local hosts

This means that custom timeseries with a point every minute and a point every 5 minutes can be generated for interfaces and local hosts, respectively.

ntopng looks for custom timeseries in the following Lua files under :code:scripts/lua/modules/timeseries/custom/:

:code:ts_minute_custom.lua for local hosts timeseries with 5-minute updates
:code:ts_5min_custom.lua for interface timeseries with 1-minute updates

If file :code:ts_5min_custom.lua does not exist, ntopng will skip the creation of custom timeseries with 5-minute updates. Similarly, if file :code:ts_minute_custom.lua does not exist, ntopng will skip the creation of custom timeseries with 1-minute updates.

Sample files :code:ts_5min_custom.lua.sample and :code:ts_minute_custom.lua.sample are created automatically upon ntopng installation with some example contents. Those files are ignored by ntopng. However, it is safe to copy them to :code:ts_5min_custom.lua and :code:ts_minute_custom.lua and modify the copies when it is necessary to add custom timeseries.

Structure of Custom Timeseries Scripts

Every custom file must contain a method :code:setup which defines one or more schemas. Every custom timeseries needs a schema to function. A schema defines the timeseries in terms of tags and metrics. The documentation describes what is a schema in detail. Later in this section an example schema will be shown.

File :code:ts_5min_custom.lua must contain a callback :code:ts_custom.host_update_stats which is called by ntopng every 5 minutes for every active local host. This callback accepts the following arguments:

:code:when The time (expressed as a Unix Epoch) of the call
:code:hostname The IP address of the host, possibly followed by a VLAN tag
:code:host The host metrics in a Lua table
:code:ifstats The interface stats of the host interface
:code:verbose and extra flag passed when ntopng is working in verbose mode

File :code:ts_minute_custom.lua must contain a callback :code:ts_custom.iface_update_stats which is called by ntopng every minute for every monitored interface. This callback accepts the following arguments:

:code:when The time (expressed as a Unix Epoch) of the call
:code:_ifname The name of the monitored interface
:code:ifstats The interface stats of the monitored interface
:code:verbose and extra flag passed when ntopng is working in verbose mode

Callbacks can be used to append points to the timeseries. Indeed, once the schema is defined, it is necessary to :code:append points to the timeseries. The function used to append points to the timeseries is the :code:ts_utils.append documented later in this section of the documentation.

Example

Let's see how to add an interface timeseries that counts the number of issues detected when analyzing sequence numbers. The total issues detected when analyzing sequence numbers is considered as the sum of TCP retransmitted, out-of-order and lost packets.

The first thing to do is to add a schema to the :code:setup function of :code:ts_minute_custom.lua. The schema is created as as follows:

.. code-block:: lua

schema = ts_utils.newSchema("iface:tcp_seq_errors", {step = 60}) schema:addTag("ifid") schema:addMetric("packets")

The first argument of :code:newSchema specifies the timeseries name :code:"iface:tcp_seq_errors". Timeseries interfaces must start with prefix :code:iface:. The second argument is a table that must contain argument :code:step which tells how frequently the timeseries will be updated. As we are in the 1-minute local hosts timeseries Lua script, a value of :code:60 must be specified here.

Then, function :code:addTag is used to indicate an interface id :code:ifid that will be used to uniquely identify the timeseries when multiple interfaces are monitored. Finally, :code:addMetric is called with an argument :code:packets to indicate the metric name. Note that both the :code:ifid and :code:packets are just plain strings here, their actual values will be set in the :code:ts_custom.iface_update_stats when updating the timeseries with new points.

The number of issues detected when analyzing sequence numbers is a counter, that is, is an always-increasing function of time. By default, schemas consider metrics as counters so there is no need to specify this type upon schema addition. For gauges, one has to indicate an extra :code:metrics_type in the table containing the :code:step. So for example, to create a 1-minute timeseries for the number of active flows of a given host, one can use the following syntax :code:ts_utils.newSchema("host:flows", {step=60, metrics_type=ts_utils.metrics.gauge}).

To update the timeseries with new points, callback :code:ts_custom.iface_update_stats is extended with a :code:ts_utils.append call as follows.

.. code-block:: lua

ts_utils.append("iface:tcp_seq_errors", {ifid = ifstats.id, packets = ifstats.tcpPacketStats.retransmissions + ifstats.tcpPacketStats.out_of_order + ifstats.tcpPacketStats.lost}, when, verbose)

The first argument of :code:ts_utils.append is the timeseries name and must be equal to the one specified when defining the schema. The second argument is a table which must contain the tag (:code:ifid) and the metric (:code:packets) which must be set to their actual values. As it can be seen from the example above, the field :code:id of table :code:ifstats is used to set tag :code:ifid, whereas the sum of :code:ifstats.tcpPacketStats table fields :code:retransmissions, :code:out_of_order and :code:lost are used as value for the metric :code:packets.

Finally, the third argument :code:when is the time of the call, and the latest argument :code:verbose indicates whether ntopng is operating in verbose mode.

From that point on, the timeseries will be consistently updated by ntopng.

Multiple schemas and multiple :code:ts_utils.append can be added in the same file.

The full example can be seen at: https://github.com/ntop/ntopng/blob/dev/scripts/lua/modules/timeseries/custom/ts_minute_custom.lua.sample

Another example that creates 5-minute timeseries of local hosts total bytes can be seen at https://github.com/ntop/ntopng/blob/dev/scripts/lua/modules/timeseries/custom/ts_5min_custom.lua.sample

Charting New Metrics

After exporting the new metrics to the timeseries driver (e.g. InfluxDB), the generated timeseries can be charted inside the traffic element page. The particular script to modify depends on the specific traffic element, here are some examples:

For local hosts, modify host_details.lua
For network interfaces, modify if_stats.lua

The script should contain a call to :code:graph_utils.drawGraphs with a :code:timeseries field. The new timeseries should be added to it. Here is for example a modified host_stats.lua with a new host:low_goodput_flows metric:

.. code:: lua

graph_utils.drawGraphs(ifId, schema, tags, _GET["zoom"], url, selected_epoch, { top_protocols = "top:host:ndpi", ... timeseries = { {schema="host:traffic", label=i18n("traffic")}, {schema="host:flows", label=i18n("graphs.active_flows")}, {schema="host:misbehaving_flows", label=i18n("graphs.total_misbehaving_flows")},

     -- The new metric is added here in order to be shown into the charts
     {schema="host:low_goodput_flows",      label="Low Goodput Flows"},

... } })

The metric will appear with the "Low Goodput Flows" into the timeseries dropdown after the timeseries points are available.

Adding New Metrics