RoofSplit: An edge computing framework with heterogeneous nodes collaboration considering optimal CNN model splitting

Intelligent applications based on AI have put much challenge on the edge of wireless networks, considering the heterogeneous characteristics and insufficient resources of the edge nodes. In this paper, we propose an edge computing framework named ’RoofSplit’ with the integration of communication, caching and computing ability. With this framework, deep learning models such as convolutional neural network (CNN) can be executed at the network edge with high efficiency. We first utilize Roofline theory and cut the one special complex CNN to several optimal sub-models. And then deploy the sub models on different edge nodes and process them in parallel. We also build a hardware testbed with the GPU and caching optimization to adapt to the RoofSplit framework. Finally, we make the experiments on the testbed. The results show that, compared with non-split scheme, RoofSplit can increase throughput by 4%–15% and reduce memory usage by 40%–60%. By using Redis caching, database is deemed as message queue in internal communication. The delay of processing one learning model is 40%–70% lower than that of Cloud-Only scheme, and more than 10% lower than that of Edge–Cloud collaboration scheme, which also reflects good real-time performance of the RoofSplit framework.