微信服务号

微信订阅号

2025-4-9- 9
Home > Archive>Volume 23, Issue 9, 2012 >2336-2346. DOI:10.3724/SP.J.1001.2012.04150
PDF HTML XML Export Cite reminder
Support Vector Regression Optimization Problem without Bias
Author:
Affiliation:

  • Article
    • | |
  • Metrics
    • |
  • Reference [19]
    • |
  • Related
    • |
  • Cited by
    • | |
  • Comments
    Abstract:

    To study the role of bias in support vector regression (SVR), primal and dual optimization formulations of support vector regression optimization problem without bias (NBSVR) are proposed first, and the necessary condition of NBSVR optimization formulation’s global optima is presented and sub-optima solution of NBSVR dual problem has been proved for the dual problem of SVR then. An active set algorithm of dual optimization formulation without bias is proposed, and the linear convergence of the proposed algorithm has been proved. The experimental results on 21 benchmark datasets show that in the solution space of dual problem, SVR can only obtain the sub-optimal solution of NBSVR, the root mean square error (RMSE) of NBSVR tends to lower than SVR. The training time of NBSVR is not only less than SVR, but also less sensitive to kernel parameter.

    Reference
    [1] Cortes C, Vapnik V. Support vector networks. Machine Learning, 1995,20(3):273-297. [doi: 10.1023/A:1022627411411]
    [2] Deng WY, Zheng QH, Chen L, Xu XB. Research on extreme learning of neural networks. Chinese Journal of Computers, 2010,33(2):279-287 (in Chinese with English abstract).
    [3] Huang GB, Ding XJ, Zhou HM. Optimization method based extreme learning machine for classification. Neurocomputing, 2010,74(1-3):155-163. [doi: 10.1016/j.neucom.2010.02.019]
    [4] Lee YJ, Mangasarian OL. SSVM: A smooth support vector machine for classifiation. Computational Optimization and Applications,2001,20(1):5-22. [doi: 10.1023/A:1011215321374]
    [5] Mangasarian OL, Musicant DR. Lagrangian support vector machines. Journal of Machine Learning Research, 2001,1(3):161-177.[doi: 10.1162/15324430152748218]
    [6] Osuna E, Freund R, Girosit F. Training support vector machines: An application to face detection. In: Rougeaux S, Kuniyoshi Y,eds. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition: Puerto Rico: IEEE Press, 1997. 130-136. [doi:10.1109/CVPR.1997.609310]
    [7] Joachims T. Making large-scale SVM learning practical. In: Scholkopf B, Burges C, Smola A, eds. Proc. of the Advances in KernelMethods-Support Vector Learning. Cambridge: MIT Press, 1999. 169-184.
    [8] Smola AJ, Scholkopf B. A tutorial on support vector regression. Technical Report, TR 1998-030, London: Royal Holloway College,1998. [doi: 10.1023/B:STCO.0000035301.49549.88]
    [9] Scheinberg K. An efficient implementation of an active set method for SVMs. Journal of Machine Learning Research, 2006,7(10):2237-2257.
    [10] Vishwanathan SVN, Smola AJ, Murty MN. SimpleSVM. In: Fawcett T, Mishra N, eds. Proc. of the Int’l Conf. on Machine Leaning:Washington: AAAI Press, 2003. 760-767.
    [11] Mangasarian OL, Musicant DR. Active support vector machine classification. In: Leen TK, Dietterich TG, Tresp V, eds. Proc. ofthe Advances in Neural Information Processing Systems. Denver: MIT Press, 2001. 138-144.
    [12] Osuna E, Freund R, Girosi F. An improved training algorithm for support vector machines. In Principe J, Giles L, Morgan N,Wilson E, eds. Proc. of the Neural Networks for Signal Processing VII. Amelia Island: IEEE Press, 1997. 276-285. [doi: 10.1109/NNSP.1997.622408]
    [13] Zhang HR, Han ZZ. An improved sequential minimal optimization learning algorithm for regression support vector machine.Journal of Software, 2003,14(12):2006-2013 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/14/2006.htm
    [14] Zhu QD, Zhang Z, Xing ZY. Improved SMO learning method of support vector machine. Journal of Harbin Engineering University,2007,28(2):183-188 (in Chinese with English abstract).
    [15] Shevade SK, Keerthi SS, Bhattacharyya C, Murthy KRK. Improvements to the SMO algorithm for SVM regression. IEEE Trans. onNeural Networks, 2000,11(5):1188-1193. [doi: 10.1109/72.870050]
    [16] Fletcher R. Practical Methods of Optimization, Constrained Optimization Vol.2. Chichester-New York, Brisbane-Toronto: JohnWiley & Sons, 1981.
    [17] Blake CK, Merz CJ. UCI repository of machine learning databases. http://archive.ics.uci.edu/ml/
    [18] Mike M. Statistical datasets. http://lib.stat.cmu.edu/datasets/
    [19] Ghanty P, Paul S, Pal NR. NEUROSVM: An architecture to reduce the effect of the choice of kernel on the performance of SVM.Journal of Machine Learning Research, 2009,10(3):591-622. [doi: 10.1145/1577069.1577090]
    Related
    Cited by
Get Citation

丁晓剑,赵银亮.无偏置支持向量回归优化问题.软件学报,2012,23(9):2336-2346

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:August 17,2010
  • Revised:January 31,2011
  • Online: September 05,2012
Links:Institute of Software, CASChina Computer FederationCASNSFC
You are the first2034057Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063