RJXB软件学报Journal of Software1000-9825软件学报编辑部中国北京rjxb-30-3-79910.13328/j.cnki.jos.005698TP311智能数据管理与分析技术专刊SPECIAL ISSUE ON TECHNIQUES FOR INTELLIGENT DATA MANAGEMENT AND ANALYSIS因子分解机模型研究综述Survey on Factorization Machines Model赵衎衎ZHAOKan-Kan
12中国人民大学 信息学院, 北京 100872School of Information, Renmin University of China, Beijing 100872, China数据工程与知识工程教育部重点实验室(中国人民大学), 北京 100872Key Laboratory of Data Engineering and Knowledge Engineering of Ministry of Education(Renmin University of China), Beijing 100872, China李翠平, E-mail:licuiping@ruc.edu.cn
The traditional matrix factorization method has a wide range of applications in prediction and recommendation tasks because of its high scalability and good performance. In the big data era, more and more contextual features can be obtained easily, while the traditional matrix factorization approach lacks effective use of context information. In this context, Factorization Machines (FM) is proposed and popular. To better grasp the development process of FM model and adapt FM approach to the real application, this paper reviews existing FM models and their optimization algorithms. First, it introduces the evolution process from traditional Matrix Factorization (MF) to FM model. Second, the paper summarizes the existing researches on FM method from the perspective of model accuracy and efficiency; Third, the paper presents the studies of four representative optimization algorithms, which are suitable for various FM models. Finally, the paper analyzes the challenges in the current FM model, proposes possible solutions for these problems, and discusses the future work.
因子分解机高阶交互特征选择概率模型凸优化分布式框架优化方法factorization machinehigh-order interactionfeature selectionprobability modelconvex optimizationdistributed frameworkoptimization algorithm国家自然科学基金61772537国家自然科学基金6177253国家自然科学基金61702522国家自然科学基金61532021国家自然科学基金(61772537,61772536,61702522,61532021)National Natural Science Foundation of China61772537National Natural Science Foundation of China61772536National Natural Science Foundation of China61702522National Natural Science Foundation of China61532021National Natural Science Foundation of China (61772537, 61772536, 61702522, 61532021)
Comparison of the probabilistic interpretation of standard factorization machines (left) to Bayesian factorization machines (right) extended by hyperpriors
ReferencesGantzJReinselDThe digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east201220072012116
Gantz J, Reinsel D. The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east. IDC iView: IDC Analyze the Future, 2012, 2007(2012):1-16.
Huang LW, Jiang BT, Lv SY, Liu YB, Li DY. Survey on deep learning based recommender systems. Chinese Journal of Computers, 2018, 41(7): 1619-1647 (in Chinese with English abstract).
Rendle S. Factorization machines. In: Proc. of the 2010 IEEE 10th Int'l Conf. on Data Mining (ICDM). IEEE, 2010. 995-1000.
Koren Y. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In: Proc. of the ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining. ACM Press, 2008. 426-434.
Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing personalized Markov chains for next-basket recommendation. In: Proc. of the Int'l Conf. on World Wide Web. ACM Press, 2010. 811-820.
KorenYCollaborative filtering with temporal dynamics20105348997
Koren Y. Collaborative filtering with temporal dynamics. Communications of the ACM, 2010, 53(4):89-97.
Xiong L, Chen X, Huang TK, etal. Temporal collaborative filtering with bayesian probabilistic tensor factorization. In: Proc. of the 2010 SIAM Int'l Conf. on Data Mining. 2010. 211-222.
Rendle S, Schmidt-Thieme L. Pairwise interaction tensor factorization for personalized tag recommendation. In: Proc. of the ACM Int'l Conf. on Web Search and Data Mining. ACM Press, 2010. 81-90.
MengXWJiWYZhangYJA survey of recommendation systems in big data2015382115
Meng XW, Ji WY, Zhang YJ. A survey of recommendation systems in big data. Journal of Beijing University of Posts and Telecommunications, 2015, 38(2):1-15 (in Chinese with English abstract).
Dror G, Koenigstein N, Koren Y, etal. The Yahoo! music dataset and KDD-Cup'11. In: Proc. of the 17th ACM SIGKDD Conf. on Knowledge Discovery and Data Mining. San Diego: ACM Press, 2012. 8-18.
GolubGKahanKCalculating the singular values and pseudo-inverse of a matrix196522205224
Golub G, Kahan K. Calculating the singular values and pseudo-inverse of a matrix. Journal of the Society for Industrial and Applied Mathematics, 1965, 2(2):205-224.
Lee DD, Seung H. Algorithms for non-negative matrix factorization. In: Proc. of the 13th Advances in Neural Information Processing Systems (NIPS 2000). Denver: MIT Press, 2000. 556-562.
Salakhutdinov R, Mnih A. Probabilistic matrix factorization. In: Proc. of the 20th Advances in Neural Information Processing Systems. 2007, 20(3): 432-451.
DingWFZhengXLChenDRActive sampling based on PureSVD model for collaborative filtering20133642326
Ding WF, Zheng XL, Chen DR. Active sampling based on PureSVD model for collaborative filtering. Journal of Beijing University of Posts and Telecommunications, 2013, 36(4):23-26 (in Chinese with English abstract).
Adomavicius G, Sankaranarayanan R, Sen S, etal. Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. on Information Systems (TOIS), 2005, 23(1): 103-145.
Symeonidis P, Papadimitriou A, Manolopoulos Y, etal. Geo-social recommendations based on incremental tensor reduction and local path traversal. In: Proc. of the 3rd ACM SIGSPATIAL Int'l Workshop on Location-Based Social Networks. 2011. 89-96.
TuDDShuCCYuHYUsing unified probabilistic matrix factorization for contextual advertisement recommendation201324345446410.3724/SP.J.1001.2013.04238
Tu DD, Shu CC, Yu HY. Using unified probabilistic matrix factorization for contextual advertisement recommendation. Ruan Jian Xue Bao/Journal of Software, 2013, 24(3): 454-464 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4238.htm[doi:10.3724/SP.J.1001.2013.04238]
Koren Y. Factor in the neighbors: Scalable and accurate collaborative filtering. ACM Trans. on Knowledge Discovery from Data, 2010, 4(1): 2010.
Gantner Z, Drumond L, Freudenthaler C, etal. Learning attribute-to-feature mappings for cold-start recommendations. In: Proc. of the 2010 IEEE 10th Int'l Conf. on Data Mining (ICDM). IEEE, 2010. 176-185.
Agarwal D, Chen BC. Regression-based latent factor models. In: Proc. of the 15th ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining. ACM Press, 2009. 19-28.
Rendle S, Gantner Z, Freudenthaler C, etal. Fast context-aware recommendations with factorization machines. In: Proc. of the 34th Int'l ACM SIGIR Conf. on Research and Development in Information Retrieval. ACM Press, 2011. 635-644.
Freudenthaler C, Schmidt-Thieme L, Rendle S. Bayesian factorization machines. 2011.
Rendle S. Learning recommender systems with adaptive regularization. In: Proc. of the 5th ACM Int'l Conf. on Web Search and Data Mining. ACM Press, 2012. 133-142.
Knoll J. Recommending with higher-order factorization machines. In: Proc. of the Research and Development in Intelligent Systems XXXⅢ. Springer Int'l Publishing, 2016. 103-116.
Blondel M, Ishihata M, Fujino A, etal. Polynomial networks and factorization machines: New insights and efficient training algorithms. IEEE Trans. on Wireless Communications, 2016, 15(1): 131-145.
Blondel M, Fujino A, Ueda N, et al. Higher-order factorization machines. Advances in Neural Information Processing Systems. 2016. 3351-3359.
Lu CT, He L, Shao W, etal. Multilinear factorization machines for multi-task multi-view learning. In: Proc. of the 10th ACM Int'l Conf. on Web Search and Data Mining. ACM Press, 2017. 701-709.
Lu CT, He L, Ding H, etal. Learning from multi-view multi-way data via structural factorization machines. In: Proc. of the WWW. 2018.
Cao B, Zhou H, Li G, et al. Multi-view machines. In: Proc. of the Ninth ACM Int'l Conf. on Web Search and Data Mining. ACM, 2016. 427-436.
Zheng L, Noroozi V, Yu PS. Joint deep modeling of users and items using reviews for recommendation. In: Proc. of the Int'l Conf. on Web Search and Data Mining. ACM Press, 2017. 425-434.
Cai Y, Dong S, Hu J. Jointly modeling user and item reviews by CNN for multi-domain recommendation. In: Proc. of the China Conf. on Information Retrieval. Cham: Springer-Verlag, 2018. 237-248.
Zhang W, Du T, Wang J. Deep learning over multi-field categorical data. In: Proc. of the European Conf. on Information Retrieval. Cham: Springer-Verlag, 2016. 45-57.
Liu Y, Guo W, Zang D, etal. A hybrid neural network model with non-linear factorization machines for collaborative recommendation. In: Proc. of the China Conf. on Information Retrieval. Cham: Springer-Verlag, 2018. 213-224.
Guo H, Tang R, Ye Y, etal. DeepFM: A factorization-machine based neural network for CTR prediction. In: Proc. of the 26th Int'l Joint Conf. on Artificial Intelligence. AAAI Press, 2017. 1725-1731.
He X, Chua TS. Neural factorization machines for sparse predictive analytics. In: Proc. of the 40th Int'l ACM SIGIR Conf. on Research and Development in Information Retrieval. ACM Press, 2017. 355-364.
Xiao J, Ye H, He X, etal. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In: Proc. of the 26th Int'l Joint Conf. on Artificial Intelligence. AAAI Press, 2017. 3119-3125.
Cheng C, Xia F, Zhang T, etal. Gradient boosting factorization machines. In: Proc. of the 8th ACM Conf. on Recommender Systems. ACM Press, 2014. 265-272.
Xu J, Lin K, Tan PN, etal. Synergies that matter: Efficient interaction selection via sparse factorization machine. In: Proc. of the 2016 SIAM Int'l Conf. on Data Mining. Society for Industrial and Applied Mathematics, 2016. 108-116.
Yurochkin M, Nguyen XL. Multi-way interacting regression via factorization machines. In: Proc. of the Advances in Neural Information Processing Systems. 2017. 2598-2606.
Nguyen TV, Karatzoglou A, Baltrunas L. Gaussian process factorization machines for context-aware recommendations. In: Proc. of the 37th Int'l ACM SIGIR Conf. on Research & Development in Information Retrieval. ACM Press, 2014. 63-72.
Saha A, Acharya A, Ravindran B, etal. Nonparametric poisson factorization machine. In: Proc. of the IEEE Int'l Conf. on Data Mining (ICDM 2015). IEEE, 2015. 967-972.
Pan Z, Chen E, Liu Q, etal. Sparse factorization machines for click-through rate prediction. In: Proc. of the 2016 IEEE 16th Int'l Conf. on Data Mining (ICDM). IEEE, 2016. 400-409.
Blondel M, Fujino A, Ueda N. Convex factorization machines. In: Proc. of the Joint European Conf. on Machine Learning and Knowledge Discovery in Databases. Cham: Springer-Verlag, 2015. 19-35.
Chang Y, etal. Convex factorization machine for toxicogenomics prediction. In: Proc. of the Int'l Conf. on Knowledge Discovery and Data Mining. ACM Press, 2017. 1215-1224.
Lin X, Zhang W, Zhang M, etal. Online compact convexified factorization machine. In: Proc. of the Int'l Conf. on World Wide Web. ACM Press, 2018. 1633-1642.
Luo L, Zhang W, Zhang Z, etal. Sketched follow-the-regularized-leader for online factorization machine. In: Proc. of the 24th Int'l Conf. on Knowledge Discovery & Data Mining. ACM Press, 2018. 1900-1909.
Juan Y, Zhuang Y, Chin WS, etal. Field-aware factorization machines for CTR prediction. In: Proc. of the 10th ACM Conf. on Recommender Systems. ACM Press, 2016. 43-50.
Pan J, Xu J, Ruiz AL, etal. Field-weighted factorization machines for click-through rate prediction in display advertising. In: Proc. of the 2018 World Wide Web Conf. on World Wide Web. 2018. 1349-1357.
Oentaryo R, Lim E, Low J, Lo D, Finegold M. Predicting response in mobile advertising with hierarchical importance-aware factorization machine. In: Proc. of the 7th ACM Int'l Conf. on Web Search and Data Mining. New York: ACM Press, 2014. 123-132.
Hong L, Doumith A, Davison B. Co-factorization machines: Modeling user interests and predicting individual decisions in Twitter. In: Proc. of the 6th ACM Int'l Conf. on Web Search and Data Mining. ACM Press, 2013. 557-566.
Loni B, Shi Y, Larson M, etal. Cross-domain collaborative filtering with factorization machines. In: Proc. of the European Conf. on Information Retrieval. Cham: Springer-Verlag, 2014. 656-661.
Zhong E, Fan W, Yang Q. Contextual collaborative filtering via hierarchical matrix factorization. In: Proc. of the 2012 SIAM Int'l Conf. on Data Mining. Society for Industrial and Applied Mathematics, 2012. 744-755.
Wang S, Du C, Zhao K, etal. Random partition factorization machines for context-aware recommendations. In: Proc. of the Int'l Conf. on Web-Age Information Management. Cham: Springer-Verlag, 2016. 219-230.
Rendle S. Social network and click-through prediction with factorization machines. In: Proc. of the KDD-Cup Workshop. 2012. 113.
DingYWangDXinXSCFM: Social and crowdsourcing factorization machines for recommendation20186654855610.1016/j.asoc.2017.08.028
Ding Y, Wang D, Xin X, etal. SCFM: Social and crowdsourcing factorization machines for recommendation. Applied Soft Computing, 2018, 66:548-556.
Zhou J, Wang D, Ding Y, etal. SocialFM: A social recommender system with factorization machines. In: Proc. of the Int'l Conf. on Web-Age Information Management. Cham: Springer-Verlag, 2016. 286-297.
Qiang RW, Liang F, Yang JW. Exploiting ranking factorization machines for microblog retrieval. In: Proc. of the 22nd ACM Int'l Conf. on Information and Knowledge Management (CIKMn 2013). 2013. 1783-1788.
Rendle S, Freudenthaler C, Gantner Z, etal. BPR: Bayesian personalized ranking from implicit feedback. In: Proc. of the 25th Conf. on Uncertainty in Artificial Intelligence. AUAI Press, 2009. 452-461.
GuoWWuSWangLPersonalized ranking with pairwise factorization machines201621419120010.1016/j.neucom.2016.05.074
Guo W, Wu S, Wang L, etal. Personalized ranking with pairwise factorization machines. Neurocomputing, 2016, 214:191-200.
Yuan FJ, Guo GB, Jose JM, Chen L, Yu HT, Zhang WN. Lambdafm: Learning optimal ranking with factorization machines using lambda surrogates. In: Proc. of the 25th ACM Int'l on Conf. on Information and Knowledge Management (CIKM 2016). 2016. 227-236.
Yuan FJ, Guo GB, Jose JM, etal. Boostfm: Boosted factorization machines for top-n feature-based recommendation. In: Proc. of the 22nd Int'l Conf. on Intelligent User Interfaces. ACM Press, 2017. 45-54.
Rendle S. Scaling factorization machines to relational data. Proc. of the VLDB Endowment, 2013, 6(5): 337-348.
Liu H, He X, Feng F, etal. Discrete factorization machines for fast feature-based recommendation. In: Proc. of the Int'l Joint Conf. on Artificial Intelligence. 2018. 3449-3455.
Sun H, Wang W, Shi Z. Parallel factorization machine recommended algorithm based on MapReduce. In: Proc. of the Int'l Conf. on Semantics, Knowledge and Grids. IEEE, 2014. 120-123.
Li M, Liu Z, Smola AJ, etal. Difacto: Distributed factorization machines. In: Proc. of the 9th ACM Int'l Conf. on Web Search and Data Mining. ACM Press, 2016. 377-386.
Zhong E, Shi Y, Liu N, etal. Scaling factorization machines with parameter server. In: Proc. of the 25th ACM Int'l Conf. on Information and Knowledge Management. ACM Press, 2016. 1583-1592.
Zhao K, Zhang J, Zhang L, etal. CDSFM: A circular distributed SGLD-based factorization machines. In: Proc. of the Int'l Conf. on Database Systems for Advanced Applications. Cham: Springer-Verlag, 2018. 701-709.
TuDDShuCCYuHYUsing unified probabilistic matrix factorization for contextual advertisement recommendation201829361462610.13328/j.cnki.jos.005447
Zhao KK, Zhang J, Zhang LF, et al. Signed network prediction method based on the client-to-client distributed framework. Ruan Jian Xue Bao/Journal of Software, 2018, 29(3): 614-626 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5447.htm[doi:10.13328/j.cnki.jos.005447]