基于相关性分析的工业时序数据异常检测
作者:
作者简介:

丁小欧(1993-),女,黑龙江哈尔滨人,博士生,CCF学生会员,主要研究领域为数据质量,数据清洗,时序数据挖掘,异常检测;王宏志(1978-),男,博士,教授,博士生导师,CCF杰出会员,主要研究领域为数据库,大数据,数据质量;于晟健(1997-),男,硕士生,CCF学生会员,主要研究领域为,数据清洗,时序数据挖掘,异常检测,大数据管理;高宏(1966-),女,博士,教授,博士生导师,CCF杰出会员,主要研究领域为复杂结构数据管理,无线传感器网络;王沐贤(1997-),男,主要研究领域为数据清洗,异常检测,时序数据库系统;杨东华(1976-),男,博士,副教授,博士生导师,主要研究领域为大数据管理与分析.

通讯作者:

王宏志,E-mail:wangzh@hit.edu.cn

基金项目:

国家重点研发计划(2016YFB100703);国家自然科学基金(U1509216,U1866602,61602129);CCF-华为数据库创新研究计划(CCF-Huawei DBIR2019005B)


Anomaly Detection on Industrial Time Series Based on Correlation Analysis
Author:
Fund Project:

National Key Research and Development Program of China (2016YFB100703); National Natural Science Foundation of China (U1509216, U1866602, 61602129); CCF-Huawei Database System Innovation Research Plan (CCF-Huawei DBIR2019005B)

  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [44]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    多维时间序列上的异常检测,是时态数据分析的重要研究问题之一.近年来,工业互联网中传感器设备采集并积累了大量工业时间序列数据,这些数据具有模式多样、工况多变的特性,给异常检测方法的效率、效果和可靠性均提出更高要求.序列间相互影响、关联,其隐藏的相关性信息可以用于识别、解释异常问题.基于此,提出一种基于序列相关性分析的多维时间序列异常检测方法.首先对多维时间序列进行分段、标准化计算,得到相关性矩阵,提取量化的相关关系;然后建立了时序相关图模型,通过在时序相关图上的相关性强度划分时间序列团,进行时间序列团内、团间以及单维的异常检测.在真实的工业设备传感器数据集上进行了大量实验,实验结果验证了该方法在高维时序数据的异常检测任务上的有效性.通过对比实验,验证了该方法从性能上优于基于统计和基于机器学习模型的基准算法.该研究通过对高维时序数据相关性知识的挖掘,既节约了计算成本,又实现了对复杂模式的异常数据的精准识别.

    Abstract:

    Anomaly detection on multi-dimensional time series is an important research problem in temporal data analysis. In recent years, large-scale industrial time series data have been collected and accumulated by equipment sensors from Industrial Internet of Things (IIoT). These data show the feature of diversity data patterns and workflows, which requires high performance of anomaly detection methods in efficiency, effectiveness, and reliability. Besides, there exists latent correlation between sequences from different dimensions. The correlation information can be used to identify and explain anomalies in data. Based on this, this study proposes a correlation analysis based anomaly detection on multi-dimensional time series data. It first computes correlation values among sequences after standardization steps, and a time series correlation graph model is constructed. Time series cliques are constructed according to correlation degree in the time series correlation graph. Anomaly detection is processed within and out of a clique. Experimental results on a real industrial sensor data set show that the proposed method is effective in anomaly detection tasks in high dimensional time series data. Through contrast experiments, the proposed method is verified to have a better performance than both the statistic-based and the machine learning-based baseline methods. Research in this study achieves reliable correlation knowledge mining between time series, which not only saves time costs, but also identifies abnormal patterns form complex conditions.

    参考文献
    [1] Zhang J, Qin W, Bao JS, et al. Big Data in Manufacturing Industry. Shanghai:Shanghai Science and Technology Press, 2016(in Chinese).
    [2] Industrial Big Data Special Unit of Industrial Internet Alliance. Industrial Big Data Technology and Application Practice. Beijing:Publishing House of Electronics Industry, 2017(in Chinese).
    [3] National Manufacturing Strategy Advisory Committee. "Made in China 2025" Technology Roadmap for Key Areas. 2015(in Chinese).
    [4] Wang JM. White Paper on Big Data Technology and Application in China's Industry. Beijing:Alliance of Industrial Internet, 2017(in Chinese).
    [5] Wang JM. Summary of industrial big data technology. Big Data Research, 2017,(6):3-14(in Chinese with English abstract).
    [6] Li J, Ni J, Wang AZ. From Big Data to Intelligent Manufacturing. Shanghai:Shanghai Jiaotong University Press, 2017(in Chinese).
    [7] Lee J, Wrote; Qiu BH, Trans. Industrial Big Data:The Revolutionary Transformation and Value Creation in INDUSTRY 4.0 Era.. Beijing:China Machine Press, 2015. (in Chinese).
    [8] Chandola V, Banerjee A, Kumar V. Anomaly detection:A survey. ACM Computing Surveys, 2009,41(3):15:1-15:58.
    [9] Toledano M, Cohen I, Ben Y, et al. Real-time anomaly detection system for time series at scale. In:Proc. of the SIGKDD Workshop. 2017. 56-65.
    [10] Gupta M, Gao J, Aggarwal C, Han JW. Outlier detection for temporal data. Morgan & Claypool Publishers, 2014,26(9):2250-2267.
    [11] Enders W, Wrote; Du J, Xie ZC, Trans. Applied Econometric Time Series. 2nd ed., Beijing:Higher Education Press, 2006(in Chinese).
    [12] Eamoon JK, Kaushik C, Sharad M, Michael JP. Locally adaptive dimensionality reduction for indexing large time series databases. ACM Trans. on Database Systems. 2002,27(2):188-228.
    [13] Ma J, Ma J, Perkins S, et al. Time-series novelty detection using one-class support vector machines. In:Proc. of the Int'l Joint Conf. on Neural Networks. IEEE, 2003.
    [14] Chandola V, Mithal V, Kumar V. Comparative evaluation of anomaly detection techniques for sequence data. In:Proc. of the 8th IEEE Int'l Conf. on Data Mining. IEEE Computer Society, 2008.
    [15] Li X, Han J, Kim S, et al. ROAM:Rule- and motif-based anomaly detection in massive moving object data sets. In:Proc. of the 7th SIAM Int'l Conf. on Data Mining. 2007. 273-284.
    [16] Golab L, Karloff H, Korn F, et al. Sequential dependencies. Proc. of the VLDB Endowment, 2009,2(1):574-585.
    [17] Song S, Zhang A, Wang J, et al. SCREEN:Stream data cleaning under speed constraints. In:Proc. of the ACM SIGMOD Int'l Conf. on Management of Data. ACM, 2015. 827-841.
    [18] Zhang J, Wang H. Detecting outlying subspaces for high-dimensional data:The new task, algorithms, and performance. Knowledge and Information Systems, 2006,10(3):333-355.
    [19] Papadimitriou S, Sun J, Faloutsos C. Streaming pattern discovery in multiple time-series. In:Proc. of the 31st Int'l Conf. on Very Large Data Bases. 2005. 697-708.
    [20] Ghoting A, Parthasarathy S, Otey ME. Fast mining of distance-based outliers in high-dimensional datasets. Data Mining and Knowledge Discovery, 2008,16(3):349-364.
    [21] Fujimaki R, Nakata T, Tsukahara H, et al. Mining abnormal patterns from heterogeneous time-series with irrelevant features for fault event detection. Statistical Analysis and Data Mining, 2009,2(1):1-17.
    [22] Ding J, Liu Y, Zhang L, et al. An anomaly detection approach for multiple monitoring data series based on latent correlation probabilistic model. Applied Intelligence, 2016,44(2):340-361.
    [23] Zhang A, Song S, Wang J, et al. Time series data cleaning:From anomaly detection to anomaly repairing. Proc. of the VLDB Endowment, 2017,10(10):1046-1057.
    [24] Ding XO, Wang HZ, Su JX, et al. Cleanits:A data cleaning system for industrial time series. Proc. of the VLDB Endowment, 2019,12(12):1786-1789.
    [25] Wang M, Zhang C, Yu J. Native API based windows anomaly intrusion detection method using SVM. In:Proc. of the IEEE Int'l Conf. on Sensor Networks. IEEE, 2006.
    [26] Gao B, Ma HY, Yang YH. HMMs (hidden Markov models) based on anomaly intrusion detection method. In:Proc. of the 2002 Int'l Conf. on Machine Learning and Cybernetics. 2002. 381-385.
    [27] Qiao Y, Xin XW, Bin Y, et al. Anomaly intrusion detection method based on HMM. Electronics Letters, 2002,38(13):663-664.
    [28] Zhang X, Fan P, Zhu Z. A new anomaly detection method based on hierarchical HMM. In:Proc. of the 4th Int'l Conf. on Parallel and Distributed Computing, Applications and Technologies. 2003. 249-252.
    [29] Zhou ZH. A brief introduction to weakly supervised learning. National Science Review, 2018,(1):48-57.
    [30] https://en.wikipedia.org/wiki/Pearson_correlation_coefficient
    [31] Wang C, Guo ZH, Wang JM. Industrial big data and its technical challenges. Telecommunications Network Technology, 2017,8(8):1-4(in Chinese with English abstract).
    [32] Cormen TH, Leiserson CE, Rivest RL, Stein C. Introduction to Algorithms. 3rd ed., MIT Press, 2009.
    [33] Romeo R. A short proof of Konig's matching theorem. Journal of Graph Theory, 2000,33(3):138-139.
    [34] https://en.wikipedia.org/wiki/Hungarian_algorithm
    附中文参考文献:
    [1] 张洁,秦威,鲍劲松,等.制造业大数据.上海:上海科学技术出版社,2016.
    [2] 工业互联网产业联盟工业大数据特设组.工业大数据技术与应用实践.北京:电子工业出版社,2017.
    [3] 国家制造强国建设战略咨询委员会.《中国制造2025》重点领域技术路线图.2015.
    [4] 王建民.中国工业大数据技术与应用白皮书.北京:工业互联网产业联盟,2017.
    [5] 王建民.工业大数据技术综述.大数据,2017,(6):3-14.
    [6] 李杰,倪军,王安正.从大数据到智能制造.上海:上海交通大学出版社,2017.
    [7] 李杰,著;邱伯华,译.工业大数据:工业4.0时代的工业转型与价值创造.北京:机械工业出版社,2015.
    [11] Enders W,著;杜江,袁景安,译.应用计量经济学:时间序列分析.第2版,北京:高等教育出版社,2006.
    [31] 王晨,郭朝晖,王建民.工业大数据及其技术挑战.电信网技术,2017,8(8):1-4.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

丁小欧,于晟健,王沐贤,王宏志,高宏,杨东华.基于相关性分析的工业时序数据异常检测.软件学报,2020,31(3):726-747

复制
分享
文章指标
  • 点击次数:5094
  • 下载次数: 13310
  • HTML阅读次数: 5421
  • 引用次数: 0
历史
  • 收稿日期:2019-07-20
  • 最后修改日期:2019-09-10
  • 在线发布日期: 2020-01-10
  • 出版日期: 2020-03-06
文章二维码
您是第20262868位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号