IBM i (AS/400) collector

Overview

Monitors IBM i (AS/400) systems using SQL services and CL commands to expose CPU, memory, storage, job, and subsystem activity.

Dependencies:

• unixODBC 2.3+ with IBM i Access ODBC driver
• IBM i 7.2 or later with SQL services enabled

Required Libraries:

• libodbc.so (provided by unixODBC)
• IBM i Access Client Solutions

Collection paths

The collector executes queries in multiple tracks:

• Fast path (5s): lightweight system status queries remain sequential on the main plugin thread.
• Slow path (10s beat): heavier queries (per-queue metrics, subsystems, plan cache, etc.) run in a background worker with bounded concurrency.
• Batch path (≥60s beat): optional long-period worker used for expensive aggregate queries such as queue totals. Disabled by default unless queue totals are explicitly enabled.

CPU Collection Methods:

The collector uses a hybrid approach for CPU utilization metrics to handle IBM i 7.4+ where AVERAGE_CPU_* columns were deprecated:

1. Primary Method - TOTAL_CPU_TIME: Uses the monotonic TOTAL_CPU_TIME counter from QSYS2.SYSTEM_STATUS() to calculate CPU utilization via delta-based calculation. This is the most accurate method but requires *JOBCTL special authority. TOTAL_CPU_TIME is a cumulative counter in nanoseconds representing CPU-seconds consumed, naturally in per-core scale.

2. Fallback Method - ELAPSED_CPU_USED: If *JOBCTL authority is not available, falls back to ELAPSED_CPU_USED with automatic reset detection. This method tracks when IBM i statistics are reset (either manually or via reset_statistics configuration) and re-establishes a baseline after detecting resets. The values are already in per-core scale.

3. Legacy Method - AVERAGE_CPU_UTILIZATION: For IBM i versions before 7.4, uses the now- deprecated AVERAGE_CPU_UTILIZATION column, which IBM reports in the same per-core scale.

The collector automatically selects the appropriate method based on available permissions and logs which method is being used.

CPU Metric Scale:

CPU utilization is reported using the "100% = 1 CPU core" semantic. This means:

• 100% indicates one CPU core is fully utilized
• 400% indicates four CPU cores are fully utilized
• Values are limited to 100% × ConfiguredCPUs, matching the partition's configured capacity

For shared LPARs, the metrics show absolute CPU consumption in per-core scale, not relative to entitled capacity. For example, a shared LPAR entitled to 0.20 cores can show 150% utilization when bursting above entitlement.

Statistics Reset Behavior:

The reset_statistics configuration option controls whether the collector resets IBM i system statistics on each query via SYSTEM_STATUS(RESET_STATISTICS=>'YES'). When enabled:

• System-level statistics (CPU, memory pools, etc.) are reset after each collection cycle
• Matches legacy behavior but clears global statistics that other tools may rely on
• The ELAPSED_CPU_USED fallback method will detect and handle these resets automatically
• Caution: Enabling this affects all users and applications on the IBM i system

Default: false (statistics are not reset, using RESET_STATISTICS=>'NO')

Chart Gaps During Baseline Resets:

The as400.system_activity_cpu_rate and as400.system_activity_cpu_utilization charts rely on delta calculations. When the collector detects that IBM i reset these statistics—or when it is still establishing the initial baseline—it intentionally skips a sample instead of emitting a zero or spike. Netdata renders those skipped samples as small gaps, which is expected behaviour.

Cardinality Management:

To prevent performance issues from excessive metric creation, the collector enforces cardinality limits on per-instance metrics (disks, subsystems, job queues, message queues, output queues, active jobs, network interfaces, HTTP servers).

How Limits Work:

• The collector counts instances before collecting metrics
• If count exceeds the configured max_* limit, collection is skipped entirely for that category
• The collector logs a warning: "[category] count (X) exceeds limit (Y), skipping collection"
• No metrics are collected for that category until you adjust the configuration

Configuration Options:

Use both limit and selector options together to manage high-cardinality environments:

Optional batch-path controls:

Warning: queue totals require scanning IBM i catalog views and can be very expensive on large systems. Leave these options disabled unless aggregate counts are absolutely necessary.

Example Workflow:

1. System has 500 disks, collector skips disk metrics (exceeds default limit of 100)
2. Check logs: "disk count (500) exceeds limit (100), skipping per-disk metrics"
3. Two options:
   • Option A: Increase limit: max_disks: 500 (collects all 500 disks)
   • Option B: Use selector: collect_disks_matching: "00[1-5]*" (cherry-pick specific disks)

Best Practices:

• Use selectors to monitor only business-critical objects in large environments
• Set limits based on your Netdata server's capacity (each instance = multiple charts)
• Start with defaults and adjust based on actual usage patterns

IBM i 7.2–7.3 Behavior Note (Message Queues):

IBM i 7.4 introduced a message-queue table function that returns only the live backlog. On 7.2–7.3 systems we fall back to the QSYS2.MESSAGE_QUEUE_INFO view, which includes all recorded messages (even those already processed/cleared from the queue). Aggregations—especially MAX(SEVERITY)—therefore reflect the historical log, not just the outstanding backlog. This behaviour is inherent to the IBM SQL service and can lead to higher-than-expected max severity values on pre-7.4 systems.

Network interface metrics have a fixed internal limit of 50 instances, and HTTP server metrics are capped at 200 instances; these limits are currently not configurable.

This collector is part of the Netdata monitoring solution.

Collected metrics

Metrics grouped by scope.

The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.

Per IBM i (AS/400) instance

These metrics refer to the entire monitored IBM i (AS/400) instance.

This scope has no labels.

Metrics:

These metrics refer to the entire monitored IBM i (AS/400) instance.

This scope has no labels.

Metrics:

Per activejob

These metrics refer to individual activejob instances.

Labels:

Metrics:

Per disk

These metrics refer to individual disk instances.

Labels:

Metrics:

Per httpserver

These metrics refer to individual httpserver instances.

Labels:

Metrics:

Per jobqueue

These metrics refer to individual jobqueue instances.

Labels:

Metrics:

Per messagequeue

These metrics refer to individual messagequeue instances.

Labels:

Metrics:

Per networkinterface

These metrics refer to individual networkinterface instances.

Labels:

Metrics:

Per outputqueue

These metrics refer to individual outputqueue instances.

Labels:

Metrics:

Per plancache

These metrics refer to individual plancache instances.

Labels:

Metrics:

Per queueoverview

These metrics refer to individual queueoverview instances.

Labels:

Metrics:

Per subsystem

These metrics refer to individual subsystem instances.

Labels:

Metrics:

Per tempstoragebucket

These metrics refer to individual tempstoragebucket instances.

Labels:

Metrics:

Configuration

File

The configuration file name for this integration is ibm.d/as400.conf.

You can edit the configuration file using the edit-config script from the Netdata config directory.

cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config ibm.d/as400.conf

Options

The following options can be defined globally or per job.

Examples

Basic configuration

IBM i (AS/400) monitoring with default settings.

jobs:
  - name: local
    endpoint: dummy://localhost

Troubleshooting

Debug Mode

To troubleshoot issues with the as400 collector, run the ibm.d.plugin with the debug option enabled. The output should give you clues as to why the collector isn't working.

• Navigate to the plugins.d directory, usually at /usr/libexec/netdata/plugins.d/
• Switch to the netdata user
• Run the ibm.d.plugin to debug the collector:

sudo -u netdata ./ibm.d.plugin -d -m as400

Getting Logs

If you're encountering problems with the as400 collector, follow these steps to retrieve logs and identify potential issues:

• Run the command specific to your system (systemd, non-systemd, or Docker container).
• Examine the output for any warnings or error messages that might indicate issues. These messages will typically provide clues about the root cause of the problem.

For systemd systems (most Linux distributions)

sudo journalctl -u netdata --reverse | grep as400

For non-systemd systems

sudo grep as400 /var/log/netdata/error.log
sudo grep as400 /var/log/netdata/collector.log

For Docker containers

sudo docker logs netdata 2>&1 | grep as400