From 19e5b83ddfad430109ce3e8d51f4a24796099127 Mon Sep 17 00:00:00 2001 From: Liu Jiang Date: Tue, 23 Nov 2021 23:54:35 +0800 Subject: [PATCH] ioctls: refine README.md and Cargo.toml Refine README.md and Cargo.toml, prepare for publishing. Signed-off-by: Liu Jiang --- README.md | 27 ++++++++++ crates/vfio-ioctls/Cargo.toml | 6 ++- crates/vfio-ioctls/README.md | 73 ++++++++++++++++++--------- crates/vfio-ioctls/src/vfio_device.rs | 6 +-- 4 files changed, 84 insertions(+), 28 deletions(-) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..f7b06fa --- /dev/null +++ b/README.md @@ -0,0 +1,27 @@ +# Safe wrappers for VFIO + +## Design + +This repository provides safe wrappers over the +[VFIO driver framework](https://www.kernel.org/doc/Documentation/vfio.txt). + +Many modern systems now provide DMA and interrupt remapping facilities to help ensure I/O devices behave within the boundaries they’ve been allotted. This includes x86 hardware with AMD-Vi and Intel VT-d, POWER systems with Partitionable Endpoints (PEs) and embedded PowerPC systems such as Freescale PAMU. The VFIO driver is an IOMMU/device agnostic framework for exposing direct device access to userspace, in a secure, IOMMU protected environment. In other words, the VFIO framework allows safe, non-privileged, userspace drivers. + +Why do we want that? Virtual machines often make use of direct device access (“device assignment”) when configured for the highest possible I/O performance. From a device and host perspective, this simply turns the VM into a userspace driver, with the benefits of significantly reduced latency, higher bandwidth, and direct use of bare-metal device drivers. + +Devices are the main target of any I/O driver. Devices typically create a programming interface made up of I/O accesses, interrupts, and DMA. Without going into the details of each of these, DMA is by far the most critical aspect for maintaining a secure environment as allowing a device read-write access to system memory imposes the greatest risk to the overall system integrity. + +To help mitigate this risk, many modern IOMMUs now incorporate isolation properties into what was, in many cases, an interface only meant for translation (ie. solving the addressing problems of devices with limited address spaces). With this, devices can now be isolated from each other and from arbitrary memory access, thus allowing things like secure direct assignment of devices into virtual machines. + +While for the most part an IOMMU may have device level granularity, any system is susceptible to expose a reduced granularity. The IOMMU API therefore supports a notion of IOMMU groups. A group is a set of devices which is isolated from all other devices in the system. Groups are therefore the unit of ownership used by VFIO. + +While the group is the minimum granularity that must be used to ensure secure user access, it’s not necessarily the preferred granularity. In IOMMUs which make use of page tables, it may be possible to share a set of page tables between different groups, reducing the overhead both to the platform (reduced TLB thrashing, reduced duplicate page tables), and to the user (programming only a single set of translations). For this reason, VFIO makes use of a container class, which may hold one or more groups. A container is created by simply opening the /dev/vfio/vfio character device. + +## Usage +This repository provides two crates to use the VFIO framework, please refer to crate documentations for detail information. +- [vfio-bindings](https://github.com/rust-vmm/vfio-ioctls/tree/ioctls/crates/vfio-bindings): a rust FFI bindings to VFIO generated using [bindgen](https://crates.io/crates/bindgen). +- [vfio-ioctls](https://github.com/rust-vmm/vfio-ioctls/tree/ioctls/crates/vfio-ioctls): a group of safe wrappers over the [VFIO APIs](https://github.com/torvalds/linux/blob/master/include/uapi/linux/vfio.h). + +## License + +This code is licensed under Apache-2.0 or BSD-3-Clause. diff --git a/crates/vfio-ioctls/Cargo.toml b/crates/vfio-ioctls/Cargo.toml index 604de08..ed99dda 100644 --- a/crates/vfio-ioctls/Cargo.toml +++ b/crates/vfio-ioctls/Cargo.toml @@ -2,10 +2,12 @@ name = "vfio-ioctls" version = "0.1.0" authors = ["The Cloud Hypervisor Authors", "Liu Jiang "] -description = "Safe wrappers over VFIO ioctls" -keywords = ["vfio"] license = "Apache-2.0 OR BSD-3-Clause" +description = "Safe wrappers over VFIO ioctls" +repository = "https://github.com/rust-vmm/vfio" +readme = "README.md" edition = "2018" +keywords = ["vfio"] [features] default = ["kvm"] diff --git a/crates/vfio-ioctls/README.md b/crates/vfio-ioctls/README.md index d2af0db..a75128e 100644 --- a/crates/vfio-ioctls/README.md +++ b/crates/vfio-ioctls/README.md @@ -1,41 +1,68 @@ -# Crate Name +# vfio-ioctls ## Design -TODO: This section should have a high-level design of the crate. +The [VFIO driver framework](https://www.kernel.org/doc/Documentation/vfio.txt) +provides unified APIs for direct device access. It is an IOMMU/device-agnostic framework for +exposing direct device access to user space in a secure, IOMMU-protected environment. +This framework is used for multiple devices, such as GPUs, network adapters, and compute +accelerators. With direct device access, virtual machines or user space applications have +direct access to the physical device. -Some questions that might help in writing this section: -- What is the purpose of this crate? -- What are the main components of the crate? How do they interact which each - other? +The VFIO framework is originally developed on Linux system, and later Microsoft HyperVisor +technology provides a compatible implementation. Therefore the VFIO framework is supported +by both Linux and Microsoft HyperVisor. + +The `vfio-ioctls` crate is a safe wrapper over the VFIO APIs. It provides three classes of structs: +- `VfioContainer`: a safe wrapper over a VFIO container object, and acts a container object + to associate `VfioDevice` objects with IOMMU domains. +- `VfioDevice`: a wrapper over a VFIO device object, provide methods to access the underlying + hardware device. +- `VfioIrq/VfioRegion`: describes capabilities/resources about a `VfioDevice` object. ## Usage -TODO: This section describes how the crate is used. +The `vfio-ioctls` crate may be used to support following usage scenarios: +- Direct device assignment to virtual machine based on Linux KVM, with default features. +- Direct device assignment to virtual machine based on Microsoft HyperVisor, with `--no-default-features --features=mshv`. +- User mode device drivers, with `--no-default-features`. + +First, add the following to your Cargo.toml: +```toml +vfio-ioctls = "0.1" +``` +Next, add this to your crate root: + +```rust +extern crate vfio_ioctls; +``` + +By default vfio-ioctls has the `kvm` feature enabled. You may turn off the default features by +`default-features = false`. To enable feature `mshv`, +```toml +vfio-ioctls = { version = "0.1", default-features = false, features = ["mshv"]} +``` -Some questions that might help in writing this section: -- What traits do users need to implement? -- Does the crate have any default/optional features? What is each feature - doing? -- Is this crate used by other rust-vmm components? If yes, how? ## Examples -TODO: Usage examples. +To create VFIO device object for user mode drivers, ```rust -use my_crate; +use std::sync::Arc; +use vfio_ioctls::{VfioContainer, VfioDevice}; + +fn create_vfio_device() { + // TODO: change to your device's path + let device_path = "/sys/bus/pci/devices/00:03.0"; + let vfio_container = Arc::new(VfioContainer::new(()).unwrap()); + let vfio_dev = VfioDevice::new(&Path::new(device_path), vfio_container.clone()).unwrap(); + let irqs = vfio_dev.max_interrupts(); -... + assert!(irqs > 0); +} ``` ## License -**!!!NOTICE**: The BSD-3-Clause license is not included in this template. -The license needs to be manually added because the text of the license file -also includes the copyright. The copyright can be different for different -crates. If the crate contains code from CrosVM, the crate must add the -CrosVM copyright which can be found -[here](https://chromium.googlesource.com/chromiumos/platform/crosvm/+/master/LICENSE). -For crates developed from scratch, the copyright is different and depends on -the contributors. +This code is licensed under Apache-2.0 or BSD-3-Clause. diff --git a/crates/vfio-ioctls/src/vfio_device.rs b/crates/vfio-ioctls/src/vfio_device.rs index fd14b61..d29fe2e 100644 --- a/crates/vfio-ioctls/src/vfio_device.rs +++ b/crates/vfio-ioctls/src/vfio_device.rs @@ -500,7 +500,7 @@ pub enum VfioRegionInfoCap { Nvlink2Lnkspd(VfioRegionInfoCapNvlink2Lnkspd), } -/// Information abour VFIO MMIO region. +/// Information about VFIO MMIO region. #[derive(Clone, Debug)] pub struct VfioRegion { pub(crate) flags: u32, @@ -509,7 +509,7 @@ pub struct VfioRegion { pub(crate) caps: Vec, } -/// Information abour VFIO interrupts. +/// Information about VFIO interrupts. #[derive(Copy, Clone, Debug, PartialEq)] pub struct VfioIrq { /// Flags for irq. @@ -716,7 +716,7 @@ impl AsRawFd for VfioDeviceInfo { } } -/// Vfio device to access underlying hardware devices. +/// A safe wrapper over a Vfio device to access underlying hardware device. /// /// The VFIO device API includes ioctls for describing the device, the I/O regions and their /// read/write/mmap offsets on the device descriptor, as well as mechanisms for describing and