Anomaly Detection on Industrial Time Series Based on Correlation Analysis
Author:
Affiliation:

Fund Project:

National Key Research and Development Program of China (2016YFB100703); National Natural Science Foundation of China (U1509216, U1866602, 61602129); CCF-Huawei Database System Innovation Research Plan (CCF-Huawei DBIR2019005B)

  • Article
  • | |
  • Metrics
  • |
  • Reference [44]
  • |
  • Related
  • | | |
  • Comments
    Abstract:

    Anomaly detection on multi-dimensional time series is an important research problem in temporal data analysis. In recent years, large-scale industrial time series data have been collected and accumulated by equipment sensors from Industrial Internet of Things (IIoT). These data show the feature of diversity data patterns and workflows, which requires high performance of anomaly detection methods in efficiency, effectiveness, and reliability. Besides, there exists latent correlation between sequences from different dimensions. The correlation information can be used to identify and explain anomalies in data. Based on this, this study proposes a correlation analysis based anomaly detection on multi-dimensional time series data. It first computes correlation values among sequences after standardization steps, and a time series correlation graph model is constructed. Time series cliques are constructed according to correlation degree in the time series correlation graph. Anomaly detection is processed within and out of a clique. Experimental results on a real industrial sensor data set show that the proposed method is effective in anomaly detection tasks in high dimensional time series data. Through contrast experiments, the proposed method is verified to have a better performance than both the statistic-based and the machine learning-based baseline methods. Research in this study achieves reliable correlation knowledge mining between time series, which not only saves time costs, but also identifies abnormal patterns form complex conditions.

    Reference
    [1] Zhang J, Qin W, Bao JS, et al. Big Data in Manufacturing Industry. Shanghai:Shanghai Science and Technology Press, 2016(in Chinese).
    [2] Industrial Big Data Special Unit of Industrial Internet Alliance. Industrial Big Data Technology and Application Practice. Beijing:Publishing House of Electronics Industry, 2017(in Chinese).
    [3] National Manufacturing Strategy Advisory Committee. "Made in China 2025" Technology Roadmap for Key Areas. 2015(in Chinese).
    [4] Wang JM. White Paper on Big Data Technology and Application in China's Industry. Beijing:Alliance of Industrial Internet, 2017(in Chinese).
    [5] Wang JM. Summary of industrial big data technology. Big Data Research, 2017,(6):3-14(in Chinese with English abstract).
    [6] Li J, Ni J, Wang AZ. From Big Data to Intelligent Manufacturing. Shanghai:Shanghai Jiaotong University Press, 2017(in Chinese).
    [7] Lee J, Wrote; Qiu BH, Trans. Industrial Big Data:The Revolutionary Transformation and Value Creation in INDUSTRY 4.0 Era.. Beijing:China Machine Press, 2015. (in Chinese).
    [8] Chandola V, Banerjee A, Kumar V. Anomaly detection:A survey. ACM Computing Surveys, 2009,41(3):15:1-15:58.
    [9] Toledano M, Cohen I, Ben Y, et al. Real-time anomaly detection system for time series at scale. In:Proc. of the SIGKDD Workshop. 2017. 56-65.
    [10] Gupta M, Gao J, Aggarwal C, Han JW. Outlier detection for temporal data. Morgan & Claypool Publishers, 2014,26(9):2250-2267.
    [11] Enders W, Wrote; Du J, Xie ZC, Trans. Applied Econometric Time Series. 2nd ed., Beijing:Higher Education Press, 2006(in Chinese).
    [12] Eamoon JK, Kaushik C, Sharad M, Michael JP. Locally adaptive dimensionality reduction for indexing large time series databases. ACM Trans. on Database Systems. 2002,27(2):188-228.
    [13] Ma J, Ma J, Perkins S, et al. Time-series novelty detection using one-class support vector machines. In:Proc. of the Int'l Joint Conf. on Neural Networks. IEEE, 2003.
    [14] Chandola V, Mithal V, Kumar V. Comparative evaluation of anomaly detection techniques for sequence data. In:Proc. of the 8th IEEE Int'l Conf. on Data Mining. IEEE Computer Society, 2008.
    [15] Li X, Han J, Kim S, et al. ROAM:Rule- and motif-based anomaly detection in massive moving object data sets. In:Proc. of the 7th SIAM Int'l Conf. on Data Mining. 2007. 273-284.
    [16] Golab L, Karloff H, Korn F, et al. Sequential dependencies. Proc. of the VLDB Endowment, 2009,2(1):574-585.
    [17] Song S, Zhang A, Wang J, et al. SCREEN:Stream data cleaning under speed constraints. In:Proc. of the ACM SIGMOD Int'l Conf. on Management of Data. ACM, 2015. 827-841.
    [18] Zhang J, Wang H. Detecting outlying subspaces for high-dimensional data:The new task, algorithms, and performance. Knowledge and Information Systems, 2006,10(3):333-355.
    [19] Papadimitriou S, Sun J, Faloutsos C. Streaming pattern discovery in multiple time-series. In:Proc. of the 31st Int'l Conf. on Very Large Data Bases. 2005. 697-708.
    [20] Ghoting A, Parthasarathy S, Otey ME. Fast mining of distance-based outliers in high-dimensional datasets. Data Mining and Knowledge Discovery, 2008,16(3):349-364.
    [21] Fujimaki R, Nakata T, Tsukahara H, et al. Mining abnormal patterns from heterogeneous time-series with irrelevant features for fault event detection. Statistical Analysis and Data Mining, 2009,2(1):1-17.
    [22] Ding J, Liu Y, Zhang L, et al. An anomaly detection approach for multiple monitoring data series based on latent correlation probabilistic model. Applied Intelligence, 2016,44(2):340-361.
    [23] Zhang A, Song S, Wang J, et al. Time series data cleaning:From anomaly detection to anomaly repairing. Proc. of the VLDB Endowment, 2017,10(10):1046-1057.
    [24] Ding XO, Wang HZ, Su JX, et al. Cleanits:A data cleaning system for industrial time series. Proc. of the VLDB Endowment, 2019,12(12):1786-1789.
    [25] Wang M, Zhang C, Yu J. Native API based windows anomaly intrusion detection method using SVM. In:Proc. of the IEEE Int'l Conf. on Sensor Networks. IEEE, 2006.
    [26] Gao B, Ma HY, Yang YH. HMMs (hidden Markov models) based on anomaly intrusion detection method. In:Proc. of the 2002 Int'l Conf. on Machine Learning and Cybernetics. 2002. 381-385.
    [27] Qiao Y, Xin XW, Bin Y, et al. Anomaly intrusion detection method based on HMM. Electronics Letters, 2002,38(13):663-664.
    [28] Zhang X, Fan P, Zhu Z. A new anomaly detection method based on hierarchical HMM. In:Proc. of the 4th Int'l Conf. on Parallel and Distributed Computing, Applications and Technologies. 2003. 249-252.
    [29] Zhou ZH. A brief introduction to weakly supervised learning. National Science Review, 2018,(1):48-57.
    [30] https://en.wikipedia.org/wiki/Pearson_correlation_coefficient
    [31] Wang C, Guo ZH, Wang JM. Industrial big data and its technical challenges. Telecommunications Network Technology, 2017,8(8):1-4(in Chinese with English abstract).
    [32] Cormen TH, Leiserson CE, Rivest RL, Stein C. Introduction to Algorithms. 3rd ed., MIT Press, 2009.
    [33] Romeo R. A short proof of Konig's matching theorem. Journal of Graph Theory, 2000,33(3):138-139.
    [34] https://en.wikipedia.org/wiki/Hungarian_algorithm
    附中文参考文献:
    [1] 张洁,秦威,鲍劲松,等.制造业大数据.上海:上海科学技术出版社,2016.
    [2] 工业互联网产业联盟工业大数据特设组.工业大数据技术与应用实践.北京:电子工业出版社,2017.
    [3] 国家制造强国建设战略咨询委员会.《中国制造2025》重点领域技术路线图.2015.
    [4] 王建民.中国工业大数据技术与应用白皮书.北京:工业互联网产业联盟,2017.
    [5] 王建民.工业大数据技术综述.大数据,2017,(6):3-14.
    [6] 李杰,倪军,王安正.从大数据到智能制造.上海:上海交通大学出版社,2017.
    [7] 李杰,著;邱伯华,译.工业大数据:工业4.0时代的工业转型与价值创造.北京:机械工业出版社,2015.
    [11] Enders W,著;杜江,袁景安,译.应用计量经济学:时间序列分析.第2版,北京:高等教育出版社,2006.
    [31] 王晨,郭朝晖,王建民.工业大数据及其技术挑战.电信网技术,2017,8(8):1-4.
    Related
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

丁小欧,于晟健,王沐贤,王宏志,高宏,杨东华.基于相关性分析的工业时序数据异常检测.软件学报,2020,31(3):726-747

Copy
Share
Article Metrics
  • Abstract:4969
  • PDF: 12960
  • HTML: 5146
  • Cited by: 0
History
  • Received:July 20,2019
  • Revised:September 10,2019
  • Online: January 10,2020
  • Published: March 06,2020
You are the first2033332Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063