Fairness Notions on Hardware Resource Configuration

Abstract

To meet performance and energy efficiency demand of modern workloads, specialized hardware accelerators implemented on FPGAs or ASICs have found adoption in modern servers and Systems-on-Chip (SoC). These hardware accelerators cater to a wide range of applications such as Machine Learning, databases, analytics and networking. Prior studies have shown excessive underutilization, utilization ranging from 60% to as low as 20%. Sharing hardware among multiple users can mitigate underutilization but makes achieving QoS requirements a challenge.

In this work, we explore hardware sharing on a GPU and employ game theory to present a formal model of sharing. We present notions of fairness, a performance model for heterogeneous hardware systems, and present a novel market-based mechanism to allocate and configure hardware resource. We implement the design on the open source Vortex GPU, and evaluate the system with 5 machine learning workloads – Resnet, AlexNet, YoloNet, K-Means clustering and multi-layer perceptron training. The evaluation showed that the market-based mechanism meets proportional QoS and we provide a theoretical guarantee. We observe up to 2x speedup and up to 10% higher utilization compared to traditional methods of providing equal resources. The system also chooses hardware configuration that maximizes the collective welfare of the users.