Git Product home page Git Product logo

vm-device's Introduction

vm-device

The vm-device crate provides:

  • device traits defining read and write operations on specialized buses
  • device manager (bus-specific traits and a concrete implementation) for operating devices and dispatching I/O
  • abstractions for defining resources and their constraints (e.g. a specific bus address range, IRQ number, etc)

Design

The virtual device model is built around four traits, DevicePio and MutDevicePio for Programmed I/O (PIO), and DeviceMmio and MutDeviceMmio for Memory-mapped I/O (MMIO). The traits define the same methods for handling read and write operations. The difference is that DevicePio and DeviceMmio only require immutable self borrows, whereas MutDevicePio and MutDeviceMmio require mutable borrows.

The device manager abstraction is implemented by the IoManager struct. It defines two buses, one for PIO and one for MMIO. For each bus, with the help of the PioManager and MmioManager traits, the manager provides methods for device registration, as well as for dispatching read and write requests. The manager will determine which device is responsible for handling the I/O request based on the accessed address range, and will route the request to that device.

PioManager and MmioManager traits are useful as interfaces for a couple of reasons. First, to allow for alternative implementations when the provided IoManager is not sufficient. Second, to allow other crates depend on the traits without depending on the actual implementation.

The interaction between a guest kernel, a host kernel and a VMM using IoManager is depicted at the diagram below. A driver in the guest kernel issues an I/O request. That request gets turned by the host kernel’s hypervisor (KVM) into a trigger for the VMM to handle. The trigger is either a vCPU exit or an eventfd notification. The VMM extracts address information and determines the target bus. Then it dispatches a new request to the IoManager, which checks that there’s a virtual device registered on the bus for the requested address, and finally sends the request to that device. vm-device

Usage

A device is usually attached to a particular bus and thus needs to implement a trait of only one type. For example, serial port on x86 is a PIO device, while VirtIO devices use MMIO. It’s also possible for a device to implement both. Once the type of I/O is determined, the next step is to choose between mutable and immutable trait variant. If read or write method needs to mutate the device’s internal state, then the mutable variant must be used.

Before dispatching any I/O requests to the new device, it needs to be registered with an IoManager instance within the specified address range on the bus. Creating a new IoManager is easy by calling IoManager::new() without any configuration. Internally the manager stores devices as trait objects wrapped in Arc’s, therefore if the device implements MutDevicePio or MutDeviceMmio, then it must be wrapped in a Mutex. The crate contains automatic implementation of DevicePio for Mutex<T> where T: MutDevicePio and DeviceMmio for Mutex<T> where T: MutDeviceMmio but only for the Mutex type in the standard library. For any other Mutex type from 3rd party crates the blanket implementation must be done by the user.

From now on the IoManager will be routing I/O requests for the registered address range to the device. The requests are dispatched by the client code, for example when handling VM exits, using IoManager's methods pio_read, pio_write, mmio_read and mmio_write.

Examples

Implementing a simple log PIO device

use std::sync::{Arc, Mutex};
use vm_device::bus::{PioAddress, PioAddressOffset, PioRange};
use vm_device::device_manager::{IoManager, PioManager};
use vm_device::MutDevicePio;

struct LogDevice {}

impl MutDevicePio for LogDevice {
    fn pio_read(&mut self, base: PioAddress, offset: PioAddressOffset, _data: &mut [u8]) {
        println!("mut pio_read: base {:?}, offset {}", base, offset);
    }

    fn pio_write(&mut self, base: PioAddress, offset: PioAddressOffset, data: &[u8]) {
        println!(
            "mut pio_write: base {:?}, offset {}, data {:?}",
            base, offset, data
        );
    }
}

Registering the device with IoManager and performing I/O

fn main() {
    let mut manager = IoManager::new();
    let device = LogDevice {};
    let bus_range = PioRange::new(PioAddress(0), 10).unwrap();
    manager
        .register_pio(bus_range, Arc::new(Mutex::new(device)))
        .unwrap();
    manager.pio_write(PioAddress(0), &vec![b'o', b'k']).unwrap();
}

Testing

The vm-device is tested using unit tests and integration tests. It leverages rust-vmm-ci for continuous testing. All tests are ran in the rustvmm/dev container.

License

This project is licensed under either of:

vm-device's People

Contributors

aghecenco avatar alexandruag avatar andreeaflorescu avatar dependabot-preview[bot] avatar dependabot[bot] avatar jiangliu avatar lauralt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vm-device's Issues

Benchmark performance of `struct Bus<A: BusAddress, D>`

It would be great to build some benchmark test cases for struct Bus<A: BusAddress, D>. It would be useful to test for

  1. there are a few devices/resource ranges registered on a bus.
  2. there are plenty devices/resource ranges registered on a bus.

Build dedicated interfaces for PIO on x86 platforms

X86 systems supports both MMIO and PIO, most other systems only support MMIO.
To use unify interfaces to support both MMIO and PIO on x86 systems, some enums are introduced:

/// IO Addresses.
#[derive(Debug, Copy, Clone, Deserialize, Serialize)]
pub enum IoAddress {
    /// Port I/O address.
    Pio(u16),

    /// Memory mapped I/O address.
    Mmio(u64),
}

This design causes much extra works to driver implementations and unit test cases. Even on the vCPU thread, we need to encapsulate PIO address and MMIO address into IoAddress. It works as:

  1. vCPU threads create IoAddress from PIO/MMIO address
  2. IoManager looks up device driver by IoAddress
  3. device driver decode IoAddress into PIO/MMIO address.

It also needs more storage/register to store IoAddress than Pio/MMIO address.

On the other hand, DeviceManager already has dedicated interfaces for PIO and MMIO, so suggest to build dedicated interfaces for PIO in DeviceIo, and make PIO related interfaces as conditional compilation for x86 only.

Add CODEOWNERS file

Please add a CODEOWNERS file with the details of the maintainers. Please use the following format as outlined in: https://help.github.com/en/articles/about-code-owners

Most projects can simply follow the wildcard syntax. e.g.

* @owner1 @owner2

Not only does this provide a way to see who is responsible or this repository they will also automatically be notified of incoming PR reviews.

Add Serial Console implementation

This is related to creating a minimal reference VMM using rust-vmm components. The serial console only needs the I/O Bus trait which is already merged into master.

Requirements for interrupt abstractions in vm-device

I'm opening this issue to start the discussion around adding interrupt abstractions to the vm-device crate.

Motivation

Firecracker is working on adding support for PCI devices which naturally require support for MSI/MSI-X interrupts. CloudHypervisor already has PCI support implemented and has added support for MSI interrupts. Firecracker currently does not support MSI interrupts and uses a very simple model based on irqfds to implement interrupt delivery.

There is a large amount of device code that can be reused and shared across both CloudHypervisor and Firecracker that is not quite interoperable between VMMs today, one of the reasons being that they use different ways to work with interrupts.

Requirements

The goal is to create some abstractions in vm-device that can model the interface for working with interrupts.
This would allow devices to have VMM-independent implementations that can be reused.

Devices should be able to:

  • generate events that signal and interrupt the VM vCPUs
  • configure interrupts based on a pre-defined specification
  • enable/disable interrupt generation
  • mask/unmask interrupts
  • request new interrupts

The VMM should be able to:

  • implement different interrupt mechanisms based on the machine specification
  • manage interrupt sources
  • deliver events generated by the devices to the vCPUs
  • not need device specific knowledge or implementation (apart from standard configuration templates)

General requirements:

  • interrupt events should not depend on the type of hypervisor used by the VM
  • while new interrupt mechanisms rarely get added, the interfaces should allow for future extension and addition of new interrupt types
  • vm-device can hold standard interfaces and templates for common interrupt types (e.g. ISA, INTx, MSI)

Even though the interface will allow VMMs to implement interrupt mechanisms independently, it may be a good idea to also have a reference implementation in vm-device for a common use-case (e.g. KVM in-kernel IRQChip).

Prior work

There have been several PRs to add Interrupt management support to vm-device (#21 and #23).
There is also a different vm-device fork implementing these types of abstractions in the Cloud Hypervisor Github org (cloud-hypervisor#5) and an implementation that Cloud Hypervisor uses (https://github.com/cloud-hypervisor/cloud-hypervisor/blob/main/vm-device/src/interrupt/mod.rs).
Crossvm is using a similar interface the Trigger proposal (called Event) and adds functionality around that primitive when needed by sending requests to the hypervisor (e.g. VmIrqRequest) through a Tube (https://chromium.googlesource.com/chromiumos/platform/crosvm/+/refs/heads/main/devices/src/pci/vfio_pci.rs#178).

Next steps

We're working on proposal and a POC in Firecracker to showcase how this interface would work. The proposal is based on the previous work mentioned above and satisfies the requirements listed.
It would be great if we can get more input on the requirements of this interface.

Thread safe device management

The current implementation of struct IoManager (and, generally, the device buses) can't be shared across threads (== sent to the vCPUs) because the Send and Sync constraints aren't enforced anywhere.

Bus interface: do we need to return both the range & the device on `device_mut`?

I just noticed that in the interface of Bus we are returning Option<(&BusRange<A>, &D)>, where A is the address type and D is the device type.

This is not really documented in the code, but why do we need to return both the range and the device? Given the name I was expecting the function to just return the device. I would guess this is some sort of optimization so that you don't need to retrieve them separately.

As a next step, I'd like to clarify this in the code as well since we're planning on publishing this to crates.io.

Pinging @alexandruag @jiangliu @sboeuf @rbradford @sameo

Road map of vm-device

Hi, everyone. Do we have a road-map/plan about what devices will be supported in this repo?
Maybe we can make a checklist here for future contribution.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.