Logger Metadata: Automatic source code parser

The scripts in this folder are used to automatically generate details of the log messages generated by ArduPilot in "DataFlash" log files (.BIN).

Execution

The script is run as part of the automated testing, to allow the website to be updated automatically with any new changes. The latest files can be found here, with specific subfolders for each vehicle.

To run the script manually for local testing, use the following command:

bash

python3 parse.py --vehicle <vehicle>

Where <vehicle> can be one of: Plane, Copter, Sub, Blimp, Rover or Tracker.

For each vehicle type, the following files are generated:

LogMessages.html - HTML file (not used)
LogMessages.md - Markdown format file (not used)
LogMessages.rst - reStructuredText format file - used for populating the Log Message

pages of the Ardupilot website.

LogMessages.xml - XML file - used by tools such as MavExplorer to provide

additional information such as field descriptions.

Populating data

The data to populate the message details should be placed into the relevant .cpp or .h file, alongside the use of the message. The basic format is as follows:

cpp

// @LoggerMessage: <message name>
// @Description: <message description>
// @Field: <field name>: <field description>
// @Field ...

If the same field set is used by multiple messages (for example the PIDx messages), then the @LoggerMessage and @Description lines may be repeated multiple times before the start of the fields, to provide the name and description for each. For example:

cpp

// @LoggerMessage: PIDR
// @Description: Proportional/Integral/Derivative gain values for Roll rate
// @LoggerMessage: PIDN
// @Description: Proportional/Integral/Derivative gain values for North/South velocity
// @Field: TimeUS: Time since system startup
// @Field: Tar: desired value
// ...

Enumerations and Bitmasks

A @Field definition may be followed by one of the following, to link it to an enumeration or bitmask as follows:

cpp

// @FieldBits: <field name>: <bit 0 name>,<bit 1 name>,...
// @FieldBitmaskEnum: <field name>: <enum name>
// @FieldValueEnum: <field name>: <enum name>

The names provided in the @FieldBits definition are free for the user to set, whereas, the @FieldBitmaskEnum and @FieldValueEnum definitions are populated with the names of enumeration sets which are parsed out of the source code.

Typically, all enum and enum class definitions are available for use, and need to be specified with their fully qualified names. If comments are provided for each enum entry, these are are also extracted to form the description. For example:

cpp

class <class name> {
  enum class <enum name>: <data type> {
    <enum entry name> = <enum entry value>, // <enum entry description>
    ...
  }
}

Or for a more simple case:

cpp

enum <enum name> {
    <enum entry name>,
    ...
}

Additionally, groups of #define statements can be extracted as an enumeration if surrounded by @LoggerEnum and @LoggerEnumEnd tags, as follows:

cpp

// @LoggerEnum: <enum name>
#define <enum entry name> <enum entry value>
...
// @LoggerEnumEnd

To print out a list of all discovered enums, the following command can be used:

bash

python3 enum_parse.py --verbose --vehicle <vehicle>

Units and Multipliers

To supplement the field data provided in the comment markup, unit and multiplier data is also extracted from the source code.

For each message, the format, unit and multiplier characters are extracted from either:

Calls to the Write, WriteStreaming or WriteCritical methods on APLogger.
LogStructure definitions.

When displaying the units on the log message documentation, the multiplier character is used to add a SI prefix to the unit. For example if a field has a unit of "A" and a multiplier of 0.001, the documentation will show this as "mA". This aligns with the way in which MAVExplorer decodes the data, but may differ from other tools, as any tool author is at liberty to convert values as best suits their usage.