面向移动终端智能的自治学习系统
作者:
作者单位:

作者简介:

徐梦炜(1992-),男,博士,主要研究领域为移动/边缘计算,操作系统.
刘渊强(1997-),男,硕士,主要研究领域为移动/边缘计算,操作系统.
黄康(1992-),男,博士,主要研究领域为机器学习,自然语言处理.
刘譞哲(1980-),男,博士,副教授,博士生导师,CCF专业会员,主要研究领域为服务计算,Web技术,软件工程.
黄罡(1975-),男,博士,教授,博士生导师,CCF高级会员,主要研究领域为软件中间件,软件体系结构,网构软件.

通讯作者:

刘譞哲,E-mail:xzl@pku.edu.cn

中图分类号:

基金项目:

国家杰出青年科学基金(61725201);广东省重点领域研发计划(2020B010164002)


Autonomous Learning System Towards Mobile Intelligence
Author:
Affiliation:

Fund Project:

Science Fund for Distinguished Young Scholars of China (61725201); R&D Projects in Key Areas of Guangdong Province of China (2020B010164002)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    在移动终端设备中部署机器学习模型已成为学术界和产业界的研究热点,其中重要的一环是利用用户数据训练生成模型.然而,由于数据隐私日益得到重视,特别是随着欧洲出台GDPR、我国出台《个人信息保护法》等相关法律法规,导致开发者不能任意从用户设备中获取训练数据(特别是隐私数据),从而无法保证模型训练的质量.国内外学者针对如何在隐私数据上训练神经网络模型展开了一系列研究,对其进行了总结并指出其相应的局限性.为此,提出了一种新型的面向移动终端隐私数据的机器学习模型训练模式,将所有与用户隐私数据相关的计算任务都部署在本地终端设备,无需用户以任何形式上传数据,从而保护用户隐私.这种训练模式被为自治式学习(autonomous learning).为了解决自治式学习面临的移动终端数据量不足与计算能力不足两大挑战,设计实现了自治学习系统AutLearn,通过云(公共数据,预训练)和端(隐私数据,迁移学习)协同的思想,以及终端数据增强技术,提高了终端设备上模型的训练效果.进一步地,通过模型压缩、神经网络编译器优化、运行时缓存等一系列技术,AutLearn可以极大地优化移动终端上的模型训练计算开销.基于AutLearn在两个经典的神经网络应用场景下实现了自治式学习,实验结果表明,AutLearn可以在保护隐私数据的前提下,训练模型达到甚至超过传统的集中式/联邦式模式,并且极大地减小了在移动终端上进行模型训练的计算和能耗开销.

    Abstract:

    How to efficiently deploy machine learning models on mobile devices has drawn a lot of attention in both academia and industry, among which the model training is a critical part. However, with increasingly public attention on data privacy and the recently adopted laws and regulations, it becomes harder for developers to collect training data from users and thus cannot train high-quality models. Researchers have been exploring approaches of training neural networks on decentralized data. Those efforts will be summarized and their limitations be pointed out. To this end, this work presents a novel neural network training paradigm on mobile devices, which distributes all training computations associated with private data on local devices and requires no data to be uploaded in any form. Such training paradigm autonomous learning is named. To deal with two main challenges of autonomous learning, i.e., limited data volume and insufficient computing power available on mobile devices, the first autonomous learning system AutLearn is designed and implemented. It incorporates the cloud (public data, pre-training)—client (private data, transfer learning) cooperation methodology and data augmentation techniques to ensure the model convergence on mobile devices. Furthermore, by utilizing a series of optimization techniques such as model compression, neural network compiler, and runtime cache reuse, AutLearn can significantly reduce the on-client training cost. Two classical scenarios of autonomous learning are implemented based on AutLearn and carried out a set of experiments. The results showed that AutLearn can train the neural networks with comparable or even higher accuracy compared to traditional centralized/federated training mode with privacy preserved. AutLearn can also significantly reduce the computational and energy cost of neural network training on mobile devices.

    参考文献
    相似文献
    引证文献
引用本文

徐梦炜,刘渊强,黄康,刘譞哲,黄罡.面向移动终端智能的自治学习系统.软件学报,2020,31(10):3004-3018

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2020-02-07
  • 最后修改日期:2020-04-04
  • 录用日期:
  • 在线发布日期: 2020-06-11
  • 出版日期: 2020-10-06
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号