Treffer: Overlay-ML: Unioning Memory and Storage Space for On-Device AI on Mobile Devices.

Recently, the technologies of on-device AI have been accelerated with the development of new hardware and software platforms. Therefore, many researchers and engineers focus on how to enable ML technologies on mobile devices with limited hardware resources. In this paper, we revisit on-device ML designed to support ML technologies on mobile devices and describe the three challenges when using on-device ML in detail. Then, we propose a new data management policy, called Overlay-ML, which efficiently solves two challenges that we discovered. Especially, we designed Overlay-ML to work in the application space with two key ideas. The first key idea is to extend the limited memory space using the usable space of the underlying storage device. The second key idea is to provide data transparency, which hides where the data is stored so that running ML models think the data is stored in the same place. For evaluation, we implemented an image detection application based on TensorFlow Lite which is a well-known on-device ML framework, and modified it to enable the features of Overlay-ML. All evaluation was performed on two state-of-the-art smartphones that are high-end embedded devices. Our evaluation results clearly show Overlay-ML can effectively prevent unexpected termination by Android OS and present a good loss value in real-world workload. [ABSTRACT FROM AUTHOR]

Copyright of Applied Sciences (2076-3417) is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)