Guide for setup of a guest Libvirt/QEMU Virtual Machine (VM). Includes general overview and references, and optimizations for Windows guests and hardware-passthrough (VFIO).

View develop branch...

Auto X.Org | Deploy VFIO | Generate Evdev | Libvirt Hooks | Power State Virtual Machine Manager

• Why?
• Host Optimizations
• Guest Optimizations
• Guest XML Layout
  • 1. Syntax
  • 2. First Lines in XML
  • 3. Memory
  • 4. CPU Topology (1 / 2)
  • 5. System Information Spoofing
  • 6. Features
  • 7. CPU Topology (2 / 2)
  • 8. Power Management
  • 9. Devices
  • 10. QEMU Command Line
  • 11. QEMU Overrides
• Benchmarking Guest Performance
• References
• Contact

The purpose of this document is to inform a new or returning user how to optimize a Guest machine, without demanding greater research and time.

This document does not serve to replace existing knowledge-bases. If you have any unexpected questions, wish to fact-check, or want to expand your knowledge, then please visit these places!

Copy and paste what you need from here and/or any example XML files, to your Guest XML file.

TODO: add here.

TODO: add here.

Below is an incomplete layout for building a Guest machine. The lines include additional features, of which are absent when creating a Guest XML (with the virsh CLI command or virt-manager GUI application).

<parent_tag_name attribute_name="attribute_value">
  <child_tag_name>child_tag_value</child_tag_name>
</parent_tag_name>

Note: The following formatting examples are a personal preference of the Author.

Format: purpose_vendor*_operating system_architecture_chipset_firmware_topology

* Optional, if Host machine contains two (2) or more video devices (GPU/VGA).

• Example systems and names:
  • Modern gaming machine:      game_nvidia_win10_x64_q35_uefi_6c12t
  • Older 2000s gaming machine: retro_amd_winxp_x86_i440fx_bios_2c4t
  • Retro 1990s gaming machine: retro_3dfx_win98_x86_i440fx_bios_1c1t
  • Intel MacOS workstation:      work_macos_amd_x64_q35_uefi_6c12t
• Purpose of the Guest (and suggested names):
  • Gaming PC:      game
  • Legacy/Retro PC: retro
  • Server:           server
  • Workstation PC:  work
  • etc.
• Vendor name of the Video device:
  • AMD:      amd
  • Intel:      intel
  • NVIDIA:    nvidia
  • emulated: virtgpu
  • non-mainstream or legacy:
    • 3DFX: 3dfx.
• Short name of the Operating System (OS):
  • Apple Macintosh:   macos
  • Linux:               arch, debian, redhat, ubuntu
  • Microsoft Windows: win98, winxp, win10
  • etc.
• Short name of the CPU architecture^ref:
  • AMD/Intel 32-bit: x86
  • AMD/Intel 64-bit: x64
  • ARM 32-bit:       aarch32
  • ARM 64-bit:       aarch64
  • etc.
• Virtualized chipset^ref:
  • I440FX: i440fx
    • Emulated, older chipset by Intel.
    • Does support legacy Guests (example: Windows NT 5 and before, XP/2000, 9x).
    • PCI bus only; expansion devices will exist on a PCI topology.
    • Will accept PCIe devices.
  • Q35:    q35
    • Virtual, newer platform.
    • Native PCIe bus; expansion devices will exist on a PCIe topology.
    • Does not support legacy Guests.
• Firmware^ref:
  • BIOS: bios
  • UEFI:  uefi
• Short-hand of Core topology: 4c8t
  • Given the amount of physical cores (example: 4).
  • Given the amount of logical threads per core (2 * 4 = 8).

To gather information of system memory, execute: free --kibi --total --wide

• Static allocation of Host memory pages into Guest memory pages.
• Huge: Memory page size greater than 4K bytes (2M or 1G bytes). The greater the size, the lower the Host overhead.
• Dynamic Host memory page allocation is more flexible, but will require defragmentation before use as Guest memory pages (before a Guest machine may start).
• Warning: If the specified Guest memory pages exceeds the allocated Host memory pages, then the Guest machine will fail to start.

To gather information about your CPU, execute: lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"

• Count does not exceed value as defined in <vcpu placement>.

• Threads should not overlap Host process threads.

• Emulator threads handle Interrupt Requests for Guest hardware emulation.
• Threads should not overlap Guest CPU threads as defined in vcpupin cpuset.

• IO threads handle IO processes for Guest virtual drives/disks.
• Threads should not overlap Guest CPU threads as defined in vcpupin cpuset.

  <!-- Given a 4-core, 8-thread CPU... -->
  <vcpu placement="static">4</vcpu>           <!-- Statically allocate four (4) cores to Guest. -->
  <iothreads>1</iothreads>                    <!-- Define one (1) thread to IO. -->
  <cputune>
    <vcpupin vcpu="0" cpuset="2"/>            <!-- Guest CPU: use the third core, first thread. -->
    <vcpupin vcpu="1" cpuset="6"/>            <!-- Guest CPU: use the third core, second thread. -->
    <vcpupin vcpu="2" cpuset="3"/>            <!-- Guest CPU: use the fourth core, first thread. -->
    <vcpupin vcpu="3" cpuset="7"/>            <!-- Guest CPU: use the fourth core, second thread. -->
    <emulatorpin cpuset="1,4"/>               <!-- Guest IRQ: use the second core, two threads. -->
    <iothreadpin iothread="1" cpuset="1,4"/>  <!-- Guest IO: use the second core, two threads. -->
  </cputune>

To gather information about your BIOS, execute:

 sudo dmidecode --type bios | grep --extended-regexp --ignore-case "vendor|version|release date"

To gather information about your system, execute:

 sudo dmidecode --type system | grep --extended-regexp --ignore-case "manufacturer|product name|version|serial number|sku number|family"

  <!-- BIOS and System spoofing (you may copy your actual info). -->
  <sysinfo type="smbios">                                       <!-- This line is necessary! -->
    <bios>
      <entry name="vendor">American Megatrends Inc.</entry>     <!-- AMI is the industry standard BIOS vendor. -->
      <entry name="version">version_of_bios_firmware</entry>
      <entry name="date">MM/DD/YYYY</entry>
    </bios>
    <system>
      <entry name="manufacturer">vendor_of_motherboard</entry>
      <entry name="product">product_name</entry>
      <entry name="version">Default string</entry>
      <entry name="serial">Default string</entry>
      <entry name="sku">Default string</entry>
      <entry name="family">Default string</entry>
    </system>
  </sysinfo>

TODO: make the following inline XML into chart, describe each feature.

    <hyperv mode="custom">
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vpindex state="on"/>
      <runtime state="on"/>
      <synic state="on"/>

      <stimer state="on">
        <direct state="on"/>
      </stimer>

      <reset state="on"/>
      <vendor_id state="on" value="1234567890ab"/>
      <frequencies state="on"/>
      <reenlightenment state="on"/>
      <tlbflush state="on"/>
      <ipi state="on"/>
      <evmcs state="on"/>
    </hyperv>
    <kvm>
      <hidden state="on"/>
    </kvm>
    <vmport state="off"/>
    <ioapic driver="kvm"/>
  </features>

To gather information about your CPU, execute: lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"

Thread(s) per core:                 2
Core(s) per socket:                 8
Socket(s):                          1

TODO: make the following inline XML into chart, describe each feature.

  <cpu mode="host-passthrough" check="none" migratable="on">  <!-- Spoof the CPU info, with the actual CPU info. -->
    <topology sockets="1" dies="1" cores="6" threads="2"/>
    <cache mode="passthrough"/>
    <feature policy="disable" name="hypervisor"/>
    <feature policy="disable" present="yes"/>
    <timer name="tsc" present="yes" mode="native"/>
  </clock>

  <!-- Power Management -->
  <pm>
    <suspend-to-mem enabled="yes"/>   <!-- Enable S3 Suspend (Sleep) -->
    <suspend-to-disk enabled="yes"/>  <!-- Enable S4 Suspend (Hibernation) -->
  </pm>

TODO: make the following inline XML into chart, describe each feature???

  <!-- Emulated, Paravirtual, Passed-through Real PCI/e, and Shared Memory devices -->
  <devices>
  ...
  </devices>

TODO: make this inline XML?

• QEMU
• RedHat
• Other legacy devices
  • SoftGPU https://github.com/JHRobotics/softgpu

lorem ipsum

TODO: make this inline XML?

• AMD
  • Integrated
  • Dedicated
  • Reset bug
• Intel
  • Integrated
  • Dedicated
    • Note: Not much research done regarding this topic. Please refer to any mentioned guides, the Reddit forum, or use an Internet search engine with the keywords intel gpu vfio.
• NVIDIA
  • Problems:
    • Boot bug (Solution: Use known-good copy of given GPU's video BIOS or VBIOS).
    • TODO: add references, programs used, instructions, and XML here.

TODO: make this inline XML?

• Looking Glass
• Scream
• ???

TODO: make the following inline XML into chart, describe each feature.

• Evdev

  <qemu:commandline>...</qemu:commandline>  <!-- Add Evdev here -->

TODO: make the following inline XML into chart, describe each feature.

  <qemu:override>...</qemu:override>

TODO: add here.

 _{I440FX | Wikipedia | QEMU documentation}

 _{PCI vs PCIe | RedHat documentation}

 _{Q35 | RedHat documentation}

 _{AArch32 | Wikipedia}

 _{AArch64 | Wikipedia}

 _{x86 | Wikipedia}

 _{x64 | Wikipedia}

 _{BIOS | Wikipedia}

 _{UEFI | Wikipedia}

 _{Huge memory pages | Arch Wiki | Debian Wiki}

 _{Memory Tuning on Virtual Machines | RedHat documentation}

Do you need help? Find any information to be inadequate? Notice any dead links? Please contact by raising an issue with the project itself.

• add About.
• add Documentation for:
  • guest XML layout
    • name syntax
    • introduction
    • memory
    • cpu topology (1/2)
    • system information spoofing
    • features
    • cpu topology (2/2)
    • power management
    • qemu command line
    • qemu overrides
  • host optimizations
  • guest optimizations
  • benchmarking performance
• add References.
• use collapsable dropdowns (https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab).
• add Guest XML file with all lines referenced in "Guest XML Layout".
• add pictures of virt-manager.
• flesh out "Guest XML Layout".
• how to setup virtio disks (virtio, virtio-scsi).