微信服务号

微信订阅号

2025-4-24- 13
Home > Archive>Volume 31, Issue 4, 2020 >1173-1188. DOI:10.13328/j.cnki.jos.005674
PDF HTML XML Export Cite reminder
Adaptive Online Kernel Density Estimation Method
Author:
Affiliation:

Clc Number:

TP181

Fund Project:

National Natural Science Foundation of China (61876076); Natural Science Foundation of Jiangsu Province of China (BK20171344)

  • Article
    • | |
  • Metrics
    • |
  • Reference [49]
    • |
  • Related [20]
    • | | |
  • Comments
    Abstract:

    Based on observed data, density estimation is the construction of an estimate of an unobservable underlying probability density function. With the development of data collection technology, real-time streaming data becomes the main subject of many related tasks. It has the properties of that high throughput, high generation speed, and the underlying distribution of data may change over time. However, for the traditional density estimation algorithms, parametric methods make unrealistic assumptions on the estimated density function while non-parametric ones suffer from the unacceptable time and space complexity. Therefore, neither parametric nor non-parametric ones could scale well to meet the requirements of streaming data environment. In this study, based on the analysis of the learning strategy in competitive learning, it is proposed a novel online density estimation algorithm to accomplish the task of density estimation for such streaming data. And it is also pointed out that it has pretty close relationship with the Gaussian mixture model. Finally, the proposed algorithm is compared with the existing density estimation algorithms. The experimental results show that the proposed algorithm could obtain better estimates compared with the existing online algorithm, and also get comparable estimation performance compared with state-of-the-art offline density estimation algorithms.

    Reference
    [1] Latecki LJ, Lazarevic A, Pokrajac D. Outlier detection with kernel density functions. In:Proc. of the Int'l Workshop on Machine Learning and Data Mining in Pattern Recognition. Berlin, Heidelberg:Springer-Verlag, 2007. 61-75.
    [2] Laxhammar R, Falkman G, Sviestins E. Anomaly detection in sea traffic-a comparison of the gaussian mixture model and the kernel density estimator. In:Proc. of the 12th Int'l Conf. on Information Fusion. 2009. 756-763.
    [3] Costa BSJ, Angelov PP, Guedes LA. Real-time fault detection using recursive density estimation. Journal of Control, Automation and Electrical Systems, 2014,25(4):428-437. https://doi.org/10.1007/s40313-014-0128-4
    [4] Chen H, Meer P. Robust computer vision through kernel density estimation. In:Proc. of the 2002 European Conf. on Computer Vision. Berlin, Heidelberg:Springer-Verlag, 2002. 236-250. https://doi.org/10.1007/3-540-47969-4_16
    [5] Yang C, Duraiswami R, Gumerov NA, Davis L. Improved fast Gauss transform and efficient kernel density estimation. In:Proc. of the IEEE Int'l Conf. on Computer Vision. 2003. 664-671. https://doi.org/10.1109/ICCV.2003.1238383
    [6] Elgammal A, Duraiswami R, Harwood D, et al. Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. In:Proc. of the IEEE. 2002. 1151-1163. https://doi.org/10.1109/JPROC.2002.801448
    [7] Mittal A, Paragios N. Motion-based background subtraction using adaptive kernel density estimation. In:Proc. of the 2004 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition. 2004. 302-309. https://doi.org/10.1109/CVPR.2004.1315179
    [8] Zivkovic Z, Ferdinand VDH. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognition Letters, 2006,27(7):773-780. https://doi.org/10.1016/j.patrec.2005.11.005
    [9] Nakaya T, Yano K. Visualising crime clusters in a space-time cube:An exploratory data-analysis approach using space-time kernel density estimation and scan statistics. Trans. in GIS, 2010,14(3):223-239. https://doi.org/10.1111/j.1467-9671.2010.01194.x
    [10] Lampe OD, Hauser H. Interactive visualization of streaming data with kernel density estimation. In:Proc. of the 2011 IEEE Pacific Visualization Symp. 2011. 171-178. https://doi.org/10.1109/PACIFICVIS.2011.5742387
    [11] McLachlan G, Peel D. Finite Mixture Models. New York:John Wiley & Sons, 2000. https://doi.org/10.1002/0471721182
    [12] Qiu TY. Research of online density estimation based on incremental Gaussian mixtures[MS. Thesis]. Nanjing:Nanjing University, 2016(in Chinese with English abstract).
    [13] Parzen E. On estimation of a probability density function and mode. The Annals of Mathematical Statistics, 1962,33(3):1065-1076.
    [14] Vapnik V, Mukherjee S. Support vector method for multivariate density estimation. In:Advances in Neural Information Processing Systems. 2000. 659-665.
    [15] Girolami M, He C. Probability density estimation from optimally condensed data samples. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2003,25(10):1253-1264.
    [16] Kim JS, Scott CD. Robust kernel density estimation. Journal of Machine Learning Research, 2012,13(Sep):2529-2565.
    [17] Botev ZI, Grotowski JF, Kroese DP. Kernel density estimation via diffusion. The Annals of Statistics, 2010,38(5):2916-2957.
    [18] Yin H, Allinson NM. Self-organizing mixture networks for probability density estimation. IEEE Trans. on Neural Networks, 2001, 12(2):405-411.
    [19] Vlassis N, Likas A. A greedy EM algorithm for Gaussian mixture learning. Neural Processing Letters, 2002,15(1):77-87. https://doi.org/10.1023/A:1013844811137
    [20] Han B, Comaniciu D, Davis L. Sequential kernel density approximation through mode propagation:Applications to background modeling. In:Proc. of the ACCV. 2004. 818-823.
    [21] Cheng Y. Mean shift, mode seeking, and clustering. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1995,17(8):790-799.
    [22] Song M, Wang H. Highly efficient incremental estimation of Gaussian mixture models for online data stream clustering. In:Proc. of the Intelligent Computing:Theory and Applications III. Int'l Society for Optics and Photonics, 2005. 174-184. https://doi.org/10.1117/12.601724
    [23] Engel PM, Heinen MR. Incremental learning of multivariate gaussian mixture models. In:Proc. of the Brazilian Symp. on Artificial Intelligence. Berlin, Heidelberg:Springer-Verlag, 2010. 82-91. https://doi.org/10.1007/978-3-642-16138-4_9
    [24] Kristan M, Leonardis A, Skočaj D. Multivariate online kernel density estimation with Gaussian kernels. Pattern Recognition, 2011,44(10-11):2630-2642. https://doi.org/10.1016/j.patcog.2011.03.019
    [25] Kristan M, Skočaj D, Leonardis A. Online kernel density estimation for interactive learning. Image and Vision Computing, 2010,28(7):1106-1116. https://doi.org/10.1016/j.imavis.2009.09.010
    [26] Kohonen T. Self-organized formation of topologically correct feature maps. Biological Cybernetics, 1982,43(1):59-69. https://doi.org/10.1007/BF00337288
    [27] Carpenter GA, Grossberg S. The ART of adaptive pattern recognition by a self-organizing neural network. Computer, 1988,21(3):77-88. https://doi.org/10.1109/2.33
    [28] Martinetz TM. A "neural-gas" network learns topologies. Artificial Neural Networks, 1991,1(1):397-402.
    [29] Martinetz TM, Berkovich SG, Schulten KJ. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks, 1993,4(4):558-569. https://doi.org/10.1109/72.238311
    [30] Martinetz T, Schulten K. Topology representing networks. Neural Networks, 1994,7(3):507-522.
    [31] Fritzke B. A growing neural gas network learns topologies. In:Advances in Neural Information Processing Systems. 1995. 625-632.
    [32] Fritzke B. Growing cell structures-a self-organizing network for unsupervised and supervised learning. Neural Networks, 1994, 7(9):1441-1460.
    [33] MacQueen J. Some methods for classification and analysis of multivariate observations. In:Proc. of the 5th Berkeley Symp. on Mathematical Statistics and Probability. 1967,1(14):281-297.
    [34] Gray R. Vector quantization. IEEE ASSP Magazine, 1984,1(2):4-29.
    [35] Robbins H, Monro S. A stochastic approximation method. In:Herbert Robbins Selected Papers. New York:Springer-Verlag, 1985. 102-109.
    [36] Zador P. Asymptotic quantization error of continuous signals and the quantization dimension. IEEE Trans. on Information Theory, 1982,28(2):139-149.
    [37] Terrell GR, Scott DW. Variable kernel density estimation. The Annals of Statistics, 1992,20(3):1236-1265.
    [38] Burges CJC. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 1998,2(2):121-167. https://doi.org/10.1023/A:1009715923555
    [39] Shen F, Hasegawa O. An incremental network for on-line unsupervised classification and topology learning. Neural Networks, 2006,19(19):90-106.
    [40] Shen F, Ogura T, Hasegawa O. An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Networks, 2007,20(8):893-903.
    [41] Shen F, Hasegawa O. Self-organizing incremental neural network and its application. In:Proc. of the Int'l Conf. on Artificial Neural Networks. Berlin, Heidelberg:Springer-Verlag, 2010. 535-540. https://doi.org/10.1007/978-3-642-15825-4_74
    [42] Qiu TY, Shen FR, Zhao JX. Review of self-organizing incremental neural network. Ruan Jian Xue Bao/Journal of Software,2016,27(9):2230-2247(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5068.htm[doi:10.13328/j. cnki.jos.005068]
    [43] Bache K, Lichman M. UCI machine learning repository, 2013. http://archive.ics.uci.edu/ml
    [44] Chang CC, Lin CJ. LIBSVM:A library for support vector machines. ACM Trans. on Intelligent Systems and Technology, 2011,2(3):27:1-27:27. https://doi.org/10.1145/1961189.1961199
    [45] Sun DW, Zhang GY, Zheng WM. Big data stream computing:Technologies and instances. Ruan Jian Xue Bao/Journal of Software, 2014,25(4):839-862(in Chinese). http://www.jos.org.cn/1000-9825/4558.htm[doi:10.13328/j.cnki.jos.004558]
    附中文参考文献:
    [12] 邱天宇.基于增量高斯混合模型的在线密度估计研究[硕士学位论文].南京:南京大学,2016.
    [42] 邱天宇,申富饶,赵金熙.自组织增量学习神经网络综述.软件学报,2016,27(9):2230-2247. http://www.jos.org.cn/1000-9825/5068.htm[doi:10.13328/j.cnki.jos.005068]
    [45] 孙大为,张广艳,郑纬民.大数据流式计算:关键技术及系统实例.软件学报,2014,25(4):839-862. http://www.jos.org.cn/1000-9825/4558.htm[doi:10.13328/j.cnki.jos.004558]
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

邓齐林,邱天宇,申富饶,赵金熙.一种自适应在线核密度估计方法.软件学报,2020,31(4):1173-1188

Copy
Share
Article Metrics
  • Abstract:2464
  • PDF: 7930
  • HTML: 3032
  • Cited by: 0
History
  • Received:March 03,2017
  • Revised:April 02,2018
  • Online: May 24,2019
  • Published: April 06,2020
Links:Institute of Software, CASChina Computer FederationCASNSFC
You are the first2038044Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063