======================= Virtio device migration

This work is licensed under the terms of the GNU GPL, version 2 or later. See the COPYING file in the top-level directory.

Saving and restoring the state of virtio devices is a bit of a twisty maze, for several reasons:

state is distributed between several parts:
- virtio core, for common fields like features, number of queues, ...
- virtio transport (pci, ccw, ...), for the different proxy devices and transport specific state (msix vectors, indicators, ...)
- virtio device (net, blk, ...), for the different device types and their state (mac address, request queue, ...)
most fields are saved via the stream interface; subsequently, subsections have been added to make cross-version migration possible

This file attempts to document the current procedure and point out some caveats.

Save state procedure

virtio core virtio transport virtio device

                                                  save() function registered
                                                  via VMState wrapper on
                                                  device class

virtio_save() <---------- ------> save_config() - save proxy device - save transport-specific device fields

save common device fields
save common virtqueue fields ------> save_queue() - save transport-specific virtqueue fields ------> save_device() - save device-specific fields
save subsections
- device endianness, if changed from default endianness
- 64 bit features, if any high feature bit is set
- virtio-1 virtqueue fields, if VERSION_1 is set

Load state procedure

virtio core virtio transport virtio device

                                                  load() function registered
                                                  via VMState wrapper on
                                                  device class

virtio_load() <---------- ------> load_config() - load proxy device - load transport-specific device fields

load common device fields
load common virtqueue fields ------> load_queue() - load transport-specific virtqueue fields
notify guest ------> load_device() - load device-specific fields
load subsections
- device endianness
- 64 bit features
- virtio-1 virtqueue fields
sanitize endianness
sanitize features
virtqueue index sanity check - feature-dependent setup

Implications of this setup

Devices need to be careful in their state processing during load: The load_device() procedure is invoked by the core before subsections have been loaded. Any code that depends on information transmitted in subsections therefore has to be invoked in the device's load() function after virtio_load() returned (like e.g. code depending on features).

Any extension of the state being migrated should be done in subsections added to the core for compatibility reasons. If transport or device specific state is added, core needs to invoke a callback from the new subsection.