abstract#

• latency在一些情况下很重要,但是过往研究主要集中关注如何提高虚拟机的吞吐量,而没有重点关注延迟
• LPNS关注的是本地存储,不考虑网络存储
  • self-feedback control
  • flexible IO queue
  • command scheduling
  • scalable polling design
  • deterministic network calculus-based formalization method(这个是一个性能评估的算法)
• 主要竞品: MDev-NVMe和SPDK

Introduction#

• 云服务器厂商提升他们的云服务器的IO吞吐量,并且还保证每个客户的带宽公平, 但是忽略了有的客户可能不需要很高的吞吐量,但是要保证他们的IO延迟不能过高
• 在现在的方案下,如果一块硬盘被多个客户机共享,那么可能延迟敏感的客户,在其它客户进行大量读写的时候,导致他自己的延迟陡增,不能满足他的需求
  • 原因: 现代SSD控制器在接收到大量IO请求的时候,是以轮转调度的方式来处理的,所以当其它客户大量发IO请求的时候,延迟敏感的客户就被拖下水了
• LPNS干了什么
  • 在传统的虚拟机和硬件之间夹进去了一个动态队列调度器
  • IO command调度器,保证延迟敏感客户不会被其它普通用户拖慢
  • 提出通过deterministic network calcules的方法的方法来从理论上计算细腻话存储中延迟的可预测性,给出确定的延迟上界
• 创新/贡献
  • 提出使用确定性网络演算,来为延迟上界提供数学模型
  • LPNS的架构
    • 自适应反馈机制:根据多租户负载动态调整资源分配
  • 基于中介式透传
    • 云服务器厂商不需要更新硬件/花钱
  • 实验结果确认有效果

background#

motivation#

• 过去虚拟化方向,都只注重吞吐量,不注重延迟
• K2针对延迟,但是吞吐量下降太多,而且不是针对虚拟化NVMe的

design and immplementation#

实验#

实验主要有两个部分,一方面是比较与其它主流SSD虚拟化的性能差异; 另一方面就是比较QoS控制系统的优劣

background#

• High-performance NVMe Storage
  • Efficient and scalable interface designed for high-performance SSDs
  • up to 65535 pairs of I/O queues, up to 65535 depth, enabling high-parallel I/o processing
  • High-parallel SQ/CQ interacation between the host and tghe SSD controller(with interrupt)
  • High throughtput and micro-second-level latency advantages over the traditional interfaces, such as sata
• two solutions for cloud instance storage resources
  • using remote storage pools or dedicated storage servers
    • disaggregated computing and Storgae resources
    • Efficient, scalable and portable
    • Related works: Reflex, LeapIO, Mellonox SNAP
  • using the local storage resources from the SSDs on the native servers
    • removing the additional bottleneck of latency performance incurred by the network
    • removing expensive costs on network devices of cloud infrastructures
    • more friendly to bare-metal cloud service tenents
    • related works: VFIO, VirtIO, SPDK, MDev-NVMe, VirtIO-fs, SR-IOV, FVM, BM-store

motivation#

• limitation of the current storage virtualization
  • only concentrating on high throughput, no efficient latency-predicatble QoS control
  • inaccurate perception of the imbalanced multi-tenant workloads with different QoS requirement
  • cannot bypass the device-sdie latency interference on the OS level
• device-side latency interference
  • multiple VMs with virtulized NVMe storage sharing the same underlying NVMe SSDs
  • throughtput-intensive VMs complete for NVMe I/O queue resources, congestion on SSD controller
  • latency degradation, unbounded latency, and the miss of latency QoS (or SLA)
• typical case
  • VM1 run light but latency-sensitive adata I/O workloads (e.g. data query) with a 30 us SLA
  • VM2 run thourhgput-intensive data I/O workloads without latency QoS requirement but arriving with burst
• further strudy in the device-side latency interference
  • the state-of-the-art NVMe virtualization (SPDK, SR-IOV, MDev-NVMe) cannot eliminate the latency innterference
  • the more intensive the competitor’s workload is, the more severe latency interference happens
  • average latency on the NVMe controller grows from 62.5% to 93.0% of the total latency with the increasing IOPS of othe VM2 workload: the controller of unpredicatble NVMe SSD is bottleneck
  • in HW/SW co-design virtualization, an accelerator card (smartNIC or FPGA) attacehs standard and unpredictable NVMe and use SR-IOV, cannot bypass the device-sidew latency interference
• Storage I/O QoS researches (with or without virtualized device designs)
  • High total throughput: SPDK, Spool, LeapIO, FVM, BM-Store2
  • Fair queueing control: FIOS, FLIN, MQFQ, D2FQ
  • latency control: differentiated storage services, autossd, K2

design#

• Scalable architecture
  • Hybrid depolyment of host processes, containeres, VMs
• full virtualization
  • each virtualized storage is a device with NVMe feature(/dev/nvmeXnY)
• self-feedback Qos control
  • SVM: VM with latency-predictable QoS
  • IVM: VM without latency-predictable QoS
• flexible and scalable polling
  • Accelerating VM I/O processing
  • flexibly configure the number of threads
  • one thread can utilize the high throughput
  • one dedicated polling thread for each SVM
• scalable I/O queue handling
  • design a hardware queue pool in the kernel module to manage queue resources
  • flexibly allocate any number of the HWQs(<the max number of NVMe SSD) into the Pool
  • An I/O queue scheduler to manage the Time Division Multiplexing(TDM) of I/O queues
  • 1-1 HWQs (bound to 1 VQ) and 1-N HWQs (shared by multiple VQ)
• I/O conmmand throttling with DNC
  • overcom device-level latency interference from OS-level storage virtualization
  • the polling thread can monitor the workload and control the command distribution speed
  • LPNS Module can control the throttling IVM throughput as $\theta$ times of IOPS of SVM with the slowest submission rate
  • a deterministic queuing system with deterministic network calculus
• using the deterministic network calculus to modle NVMe virtualization
  • Concentraing point: 4K block size random read and write, the device-level latency interference are caused by the congestion on the SSD controller
  • scenario: j iVMs co-running with i SVM, the IOPS of SVM is p and its command queue depth is d
  • arrive curve: $\alpha (t) = v \cdot t + b = (p \cdot (j \cdot \theta + i)) \cdot t + d \cdot(j \cdot \theta + i)$
  • service curve: $\beta (t) = R \cdot t + L_h$
  • latency upper bound $L_{max} <= \frac{b}{R} + L_h = \frac{d \cdot(j \cdot \theta + i)}{R} + L_h$
• for SVM
  • using 1-1 HWQ
  • dedicated polling thread for better latency performance
  • no I/O command throttling
  • no throughput sacrifice
• for IVM
  • using the idle 1-1 HWQs and sharing 1-N HWQs
  • one polling thread for all IVMs, reducing overhead
  • dynamic I/O command throttling according to DNC
  • smaller throughput sacrifice than previous works in K2

implementation#

• all the implementations of LPNS are included in a linux kernel source code
• LPNS implementation is based on mediated pass-through
  • cooperate by the vfio-0mdev support of linux kernel
  • a kernel module (nvme-mdev.ko) to provide virtualized storage with full NVMe
  • coexist and cooperate with the origianl nvme.ko module (the NVMe driver)
  • Admin and I/O queue resource management (resource emulation and pass-through)
  • polling thread is resposible for performance detection, optimization and I/O throttling
  • providing CONFIG_NVME_MDEV kconfig for cloud system maintainers
• LPNS works on the mainstream intel/AMD CPUs and all commercial NVMe SSD

evaluation#

• hardware comfiguration
• Micro benchmark (comparison with other storage virtualization)
  • Flexible I/O tester(FIO), 4K random read or write, using libaio and O_DIRECT
  • VirtIO, SPDK vhost-blk, MDev-NVMe on intel P5800X, MDev-NVMe and SR-IOV on samsung PM1735
• application benchmark
  • webuser
  • ycsb

conclusion#

• latency predictability trade-off
  • different NVMe SSD has different service curve
  • most NVMe SSDs has poorer random write performance thant read, wasting more total throughput than random read
• performance and resource overhead
  • flexible and scalable polling to reduce CPU overhead
  • periodical workload and performance detection, saving kernel memory resoures
• LPNS: Scalable and Latency-Predicatble virtualization
  • we analyze the device-level latency interference, and argu the signifigance of overcoming this interference from ths OS-level storage virtualization
  • we design LPNS, the first OS-level NVMe virtualization solution with latency-predictable QoS enhancement
  • we implement LPNS in the linux kernel without any further purchase of accelerator cards