CPU template helper tool

The cpu-template-helper tool is a program designed to assist users with creating and managing their custom CPU templates.

Usage

The cpu-template-helper tool has two sets of commands: template-related commands and fingerprint-related commands.

Template-related commands

Dump command

This command dumps guest CPU configuration in the custom CPU template JSON format.

cpu-template-helper template dump \
    --output <cpu-config> \
    [--template <cpu-template>] \
    [--config <firecracker-config>]

Users can utilize this as an entry point of a custom CPU template creation to comprehend what CPU configuration are exposed to guests.

The guest CPU configuration consists of the following entities:

• x86_64
  • CPUID
  • MSRs (Model Specific Registers)
• aarch64
  • ARM registers

It retrieves the above entities exposed to a guest by applying the same preboot process as Firecacker and capturing them in the state just before booting a guest. More details about the preboot process can be found here and here.

[!NOTE]

Some MSRs and ARM registers are not included in the output, since they are not reasonable to modify with CPU templates. The full list of them can be found in Appendix.

[!NOTE]

Since the output depends on underlying hardware and software stack (BIOS, CPU, kernel, Firecracker), it is required to dump guest CPU configuration on each combination when creating a custom CPU template targeting them all.

Strip command

This command strips identical entries from multiple guest CPU configuration files generated with the dump command.

cpu-template-helper template strip \
    --paths <cpu-config-1> <cpu-config-2> [..<cpu-config-N>] \
    --suffix <suffix>

One practical use case of the CPU template feature is to provide a consistent CPU feature set to guests running on multiple CPU models. When creating a custom CPU template for this purpose, it is efficient to focus on the differences in guest CPU configurations across those CPU models. Given that a dumped guest CPU configuration typically amounts to approximately 1,000 lines, this command considerably narrows down the scope to consider.

Verify command

This command verifies that the given custom CPU template is applied correctly.

cpu-template-helper template verify \
    --template <cpu-template> \
    [--config <firecracker-config>]

Firecracker modifies the guest CPU configuration after the CPU template is applied. Occasionally, due to hardware and/or software limitations, KVM might not set the given configuration. Since Firecracker does not check them at runtime, it is required to ensure that these situations don't happen with their custom CPU templates before deploying it.

When a template is specified both through --template and in Firecracker configuration file provided via --config, the template specified with --template takes precedence.

[!NOTE]

This command does not ensure that the contents of the template are sensible. Thus, users need to make sure that the template does not have any inconsistent entries and does not crash guests.

Fingerprint-related commands

Dump command

This command not only dumps the guest CPU configuration, but also host information that could affect the validity of custom CPU templates.

cpu-template-helper fingerprint dump \
    --output <output-path> \
    [--template <cpu-template>] \
    [--config <firecracker-config>]

Keeping the underlying hardware and software stack updated is essential for maintaining security and leveraging new technologies. On the other hand, since the guest CPU configuration can vary depending on the infrastructure, updating it could lead to a situation where a custom CPU template loses its validity. In addition, even if values of the guest CPU configuration don't change, its internal behavior or semantics could still change. For instance, a kernel version update may introduce changes to KVM emulation and a microcode update may alter the behavior of CPU instructions.

To ensure awareness of these changes, it is strongly recommended to store the fingerprint file at the time of creating a custom CPU template and to continuously compare it with the current one.

Compare command

This command compares two fingerprint files: one was taken at the time of custom CPU template creation and the other is taken currently.

cpu-template-helper fingerprint compare \
    --prev <prev-fingerprint> \
    --curr <curr-fingerprint> \
    --filters <field-1> [..<field-N>]

By continously comparing fingerprint files, users can ensure they are aware of any changes that could require revising the custom CPU template. However, it is worth noting that not all of these changes necessarily require a revision, and some changes could be inconsequential to the custom CPU template depending on its use case. To provide users with flexibility in comparing fingerprint files based on situations or use cases, the --filters option allows users to select which fields to compare.

As examples of when to compare fingerprint files:

• When bumping the Firecracker version up
• When bumping the kernel version up
• When applying a microcode update (or launching a new host (e.g. AWS EC2 metal instance))

Sample scenario

This section gives steps of creating and managing a custom CPU template in a sample scenario where the template is designed to provide a consistent set of CPU features to a heterogeneous fleet consisting of multiple CPU models.

Custom CPU template creation

1. Run the cpu-template-helper template dump command on each CPU model to retrieve guest CPU configuration.
2. Run the cpu-template-helper template strip command to remove identical entries across the dumped guest CPU configuration files.
3. Examine the differences of guest CPU configuration in details, determine which CPU features should be presented to guests and draft a custom CPU template.
4. Run the cpu-template-helper template verify command to check the created custom CPU template is applied correctly.
5. Conduct thorough testing of the template as needed to ensure that it does not contain any inconsistent entries and does not lead to guest crashes.

Custom CPU template management

1. Run the cpu-template-helper fingerprint dump command on each CPU model at the same time when creating a custom CPU template.
2. Store the dumped fingerprint files together with the custom CPU template.
3. Run the cpu-template-helper fingerprint dump command to ensure the template's validity whenever you expect changes to the underlying hardware and software stack.
4. Run the cpu-template-helper fingerprint compare command to identify changes of the underlying environment introduced after creating the template.
5. (if changes are detected) Review the identified changes, make necessary revisions to the CPU template, and replace the fingerprint file with the new one.

[!NOTE]

It is recommended to review the update process of the underlying stack on your infrastructure. This can help identify points that may require the above validation check.

Appendix

MSRs excluded from guest CPU configuration dump

ARM registers excluded from guest CPU configuration dump