微信服务号

微信订阅号

2025年4月3日 15:38 星期四
首页 > 过刊浏览>2021年第32卷第5期 >1404-1426. DOI:10.13328/j.cnki.jos.006214
PDF HTML阅读 XML下载 导出引用 引用提醒
可信机器学习的公平性综述
作者:
作者简介:

刘文炎(1995-),女,博士生,主要研究领域为可信机器学习.
卢兴见(1986-),男,博士,副教授,CCF专业会员,主要研究领域为可信机器学习,云计算.
沈楚云(1997-),男,博士生,主要研究领域为可信机器学习,多智体强化学习,机器学习中的公平.
王晓玲(1975-),女,博士,教授,博士生导师,CCF专业会员,主要研究领域为可信机器学习,知识图谱,个性化推荐技术,机器学习及隐私保护技术.
王祥丰(1987-),男,博士,副教授,CCF专业会员,主要研究领域为分布式优化,多智能体强化学习,可信机器学习.
查宏远(1963-),男,博士,教授,博士生导师,主要研究领域为机器学习.
金博(1982-),男,博士,讲师,CCF专业会员,主要研究领域为可信机器学习,多智体强化学习,计算机视觉技术及应用.
何积丰(1943-),男,教授,博士生导师,CCF会士,主要研究领域为可信人工智能,可信软件.

通讯作者:

王祥丰,E-mail:xfwang@cs.ecnu.edu.cn;王晓玲,E-mail:xlwang@cs.ecnu.edu.cn

基金项目:

上海市科委创新行动计划人工智能科技支持专项（20DZ1100300，20511101100）；国家自然科学基金（61972155，61672231，12071145）；上海市自然科学基金（19ZR1414200）；国家重点研发计划（2020AAA0107400，2018YFB2101300）


Survey on Fairness in Trustworthy Machine Learning
Author:
Fund Project:

AI Project of Innovation Action Plan of Science and Technology Commission of Shanghai Municipality (STCSM) (20DZ1100300, 20511101100); National Natural Science Foundation of China (61972155, 61672231, 12071145); Natural Science Foundation of Shanghai Municipality (19ZR1414200); National Key Research and Development Program of China (2020AAA0107400, 2018YFB2101300)

  • 摘要
    • | |
  • 访问统计
    • |
  • 参考文献 [100]
    • |
  • 相似文献 [20]
    • |
  • 引证文献
    • | |
  • 文章评论
    摘要:

    人工智能在与人类生活息息相关的场景中自主决策时，正逐渐面临法律或伦理的问题或风险.可信机器学习是建立安全人工智能系统的核心技术，是人工智能领域的热门研究方向，而公平性是可信机器学习的重要考量.公平性旨在研究机器学习算法决策对个人或群体不存在因其固有或后天属性所引起的偏见或偏爱.从公平表征、公平建模和公平决策这3个角度出发，以典型案例中不公平问题及其危害为驱动，分析数据和算法中造成不公平的潜在原因，建立机器学习中的公平性抽象定义及其分类体系，进一步研究用于消除不公平的机制.可信机器学习中的公平性研究在人工智能多个领域中处于起步阶段，如计算机视觉、自然语言处理、推荐系统、多智能体系统和联邦学习等.建立具备公平决策能力的人工智能算法，是加速推广人工智能落地的必要条件，且极具理论意义和应用价值.

    Abstract:

    Artificial intelligence raises legal and ethical issues or risks when used to automated decision-making in areas closely related to daily life. Trustworthy machine learning is the core technology in artificial intelligence safety. It is a trending research direction, of which fairness is an essential aspect. Fairness is the absence of any prejudice or favoritism towards an individual or a group based on their inherent or acquired characteristics which are irrelevant in the particular context of decision-making. A comprehensive and structured overview of three research contents is provided, namely, fair representation, fair modeling, and fair decision-making algorithm. The potential causes and harmful consequences of unfairness are first identified in data and algorithm processing. Then, the abstract definition and primary mechanisms for eliminating unfairness are summarized. The research on fairness is at its early stage in fields such as computer vision, natural language processing, recommender systems, multi-agent systems, and federated learning. Fairness is a prerequisite for the application of machine learning, and constructing fair algorithms has theoretical significance and practical values.

    参考文献
    [1] Zhou ZH. Machine Learning. Beijing:Tsinghua University Press, 2016(in Chinese).
    [2] He JF. Safe and trustworthy artificial intelligence. Information Security and Communications Privacy, 2019(10):5-8(in Chinese).
    [3] Meng XF, Wang LX, Liu JX. Data privacy, monopoly and fairness for AI. Big Data Research, 2020,6(1):35-46(in Chinese with English abstract).
    [4] Liu RX, Chen H, Guo RY, Zhao D, Liang WJ, Li CP. Survey on privacy attacks and defenses in machine learning. Ruan Jian Xue Bao/Journal of Software, 2020,31(3):866-892(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5904.htm[doi:10.13328/j.cnki.jos.005904]
    [5] Tan ZW, Zhang LF. Survey on privacy preserving techniques for machine learning. Ruan Jian Xue Bao/Journal of Software, 2020, 31(7):2127-2156(in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6052.htm[doi:10.13328/j.cnki.jos.006052]
    [6] Cheng KY, Wang N, Shi WX, Zhan YZ. Research advances in the interpretability of deep learning. Journal of Computer Research and Development, 2020,57(6):1208-1217(in Chinese with English abstract).
    [7] Saxena NA, Huang K, DeFilippis E, Radanovic G, Parkes DC, Liu Y. How do fairness definitions fare? Examining public attitudes towards algorithmic definitions of fairness. In:Proc. of the AIES. 2019.99-106.
    [8] Kusner MJ, Loftus JR, Russell C, Silva R. Counterfactual fairness. In:Proc. of the NIPS. 2017.4066-4076.
    [9] Suresh H, Guttag JV. A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002v3, 2019.
    [10] Mehrabi N, Morstatter F, Saxena N, Lerman K, Aram G. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635v2, 2019.
    [11] Baeza-Yates R. Bias on the Web. Communications of the ACM, 2018,61(6):54-61.
    [12] Silva S, Kenney M. Algorithms, platforms, and ethnic bias. Communications of the ACM, 2019,62(11):37-39.
    [13] Alessandro BD, Neil CO, LaGatta T. Conscientious classification:A data scientist's guide to discrimination-aware classification. arXiv preprint arXiv:1907.09013v1, 2019.
    [14] Chouldechova A, Roth A. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810v1, 2018.
    [15] Hardt M, Price E, Srebro N. Equality of opportunity in supervised learning. In:Proc. of the NIPS. 2016.3315-3323.
    [16] Chen IY, Johansson FD, Sontag DA. Why is my classifier discriminatory? In:Proc. of the NeurIPS. 2018.3543-3554.
    [17] Buolamwini J, Gebru T. Gender shades:Intersectional accuracy disparities in commercial gender classification. In:Proc. of the FAT. 2018.77-91.
    [18] Dwork C, Hardt M, Pitassi T, Reingold O, Zemel RS. Fairness through awareness. In:Proc. of the ITCS. 2012.214-226.
    [19] Friedler SA, Scheidegger C, Venkatasubramanian S, Choudhary S, Hamilton EP, Roth D. A comparative study of fairness-enhancing interventions in machine learning. In:Proc. of the FAT. 2019.329-338.
    [20] Calders T, Verwer S. Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery, 2010,21(2):277-292.
    [21] Edwards H, Storkey AJ. Censoring representations with an adversary. In:Proc. of the ICLR (Poster). 2016.
    [22] Louizos C, Swersky K, Li YJ, Welling M, Zemel RS. The variational fair autoencoder. In:Proc. of the ICLR. 2016.
    [23] Zemel RS, Wu Y, Swersky K, Pitassi T, Dwork C. Learning fair representations. In:Proc. of the ICML. 2013.325-333.
    [24] Madras D, Creager E, Pitassi T, Zemel RS. Learning adversarially fair and transferable representations. In:Proc. of the ICML. 2018.3381-3390.
    [25] Adel T, Valera I, Ghahramani Z, Weller A. One-network adversarial fairness. In:Proc. of the AAAI. 2019.2412-2420.
    [26] Feldman M, Friedler SA, Moeller J, Scheidegger C, Venkatasubramanian S. Certifying and removing disparate impact. In:Proc. of the KDD. 2015.259-268.
    [27] Zafar MB, Valera I, Gomez-Rodriguez M, Gummadi KP. Fairness constraints:Mechanisms for fair classification. In:Proc. of the AISTATS. 2017.962-970.
    [28] Berk R, Heidari H, Jabbari S, Michael K, Roth A. Fairness in criminal justice risk assessments:The state of the art. In:Proc. of the Sociological Methods and Research. 2018.3-44.
    [29] Kleinberg JM, Mullainathan S, Raghavan M. Inherent trade-offs in the fair determination of risk scores. In:Proc. of the ITCS. 2017.1-23.
    [30] Chouldechova A. Fair prediction with disparate impact:A study of bias in recidivism prediction instruments. Big Data, 2017,5(2):153-163.
    [31] Pearl J. Causality. Cambridge University Press, 2009.
    [32] Chiappa S. Path-specific counterfactual fairness. In:Proc. of the AAAI. 2019.7801-7808.
    [33] Loftus JR, Russell C, Kusner MJ, Silva R. Causal reasoning for algorithmic fairness. arXiv preprint arXiv:1805.05859v1, 2018.
    [34] Beutel A, Chen JL, Zhao Z, Chi EH. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075v2, 2017.
    [35] Zhao H, Gordon GJ. Inherent tradeoffs in learning fair representations. In:Proc. of the NeurIPS. 2019.15649-15659.
    [36] Khosravifard M, Fooladivanda D, Gulliver TA. Confliction of the convexity and metric properties in f-divergences. IEICE Trans. on Fundamentals of Electronics, Communications and Computer Sciences, 2017,90(9):1848-1853.
    [37] Zhao H, Coston A, Adel T, Gordon GJ. Conditional learning of fair representations. In:Proc. of the ICLR. 2020.
    [38] Xu DP, Yuan SH, Zhang L, Wu XT. FairGAN:Fairness-aware generative adversarial networks. In:Proc. of the BigData. 2018.570-575.
    [39] Xu DP, Yuan SH, Zhang L, Wu XT. FairGAN+:Achieving fair data generation and classification through generative adversarial nets. In:Proc. of the BigData. 2019.1401-1406.
    [40] Xu DP, Wu YK, Yuan SH, Zhang L, Wu XT. Achieving causal fairness through generative adversarial networks. In:Proc. of the IJCAI. 2019.1452-1458.
    [41] Kocaoglu M, Snyder C, Dimakis AG, Vishwanath S. CausalGAN:Learning causal implicit generative models with adversarial training. In:Proc. of the ICLR (Poster). 2018.
    [42] Creager E, Madras D, Jacobsen JH, Weis MA, Swersky K, Pitassi T, Zemel RS. Flexibly fair representation learning by disentanglement. In:Proc. of the ICML. 2019.1436-1445.
    [43] Gordaliza P, Barrio E, Gamboa F, Loubes JM. Obtaining fairness using optimal transport theory. In:Proc. of the ICML. 2019.2357-2365.
    [44] Bechavod Y, Ligett K. Learning fair classifiers:A regularization-inspired approach. arXiv preprint arXiv:1707.00044v3, 2017.
    [45] Chris R, Kusner MJ, Loftus JR, Silva R. When worlds collide:integrating different counterfactual assumptions in fairness. In:Proc. of the NIPS. 2017.6414-6423.
    [46] Wu YK, Zhang L, Wu XT, Tong HH. PC-fairness:A unified framework for measuring causality-based fairness. In:Proc. of the NeurIPS. 2019.3399-3409.
    [47] Wu YK, Zhang L, Wu XT. Counterfactual fairness:Unidentification, bound and algorithm. In:Proc. of the IJCAI. 2019.1438-1444.
    [48] Agarwal A, Dudík M, Wu ZWS. Fair regression:Quantitative definitions and reduction-based algorithms. In:Proc. of the ICML. 2019.120-129.
    [49] Conitzer V, Freeman R, Shah N, Vaughan JW. Group fairness for the allocation of indivisible goods. In:Proc. of the AAAI. 2019.1853-1860.
    [50] Kusner MJ, Russell C, Loftus JR, Silva R. Making decisions that reduce discriminatory impacts. In:Proc. of the ICML. 2019.3591-3600.
    [51] Ustun B, Liu Y, Parkes DC. Fairness without harm:Decoupled classifiers with preference guarantees. In:Proc. of the ICML. 2019.6373-6382.
    [52] Tsang A, Wilder B, Rice E, Tambe M, Zick Y. Group-fairness in influence maximization. In:Proc. of the IJCAI. 2019.5997-6005.
    [53] Chen XY, Fain B, Lyu L, Munagala K. Proportionally fair clustering. In:Proc. of the ICML. 2019.1032-1041.
    [54] Jiang R, Pacchiano A, Stepleton T, Jiang H, Chiappa S. Wasserstein fair classification. In:Proc. of the UAI. 2019.862-872.
    [55] Jagielski M, Kearns MJ, Mao JM, Oprea A, Roth A, Sharifi-Malvajerdi S, Ullman J. Differentially private fair learning. In:Proc. of the ICML. 2019.3000-3008.
    [56] Wang TL, Zhao JY, Yatskar M, Chang KW, Ordonez V. Balanced datasets are not enough:Estimating and mitigating gender bias in deep image representations. In:Proc. of the ICCV. 2019.5309-5318.
    [57] Quadrianto N, Sharmanska V, Thomas O. Discovering fair representations in the data domain. In:Proc. of the CVPR. 2019.8227-8236.
    [58] DeVries T, Misra I, Wang CH, Maaten LVD. Does object recognition work for everyone? In:Proc. of the CVPR Workshops. 2019.52-59.
    [59] Wang ZY, Qinami K, Karakozis IC, Genova K, Nair P, Hata K, Russakovsky O. Towards fairness in visual recognition:Effective strategies for bias mitigation. In:Proc. of the CVPR. 2020.8916-8925.
    [60] Reid P, Martinez RD, Dass N, Kurohashi S, Jurafsky D, Yang DY. Automatically neutralizing subjective bias in text. In:Proc. of the AAAI. 2020.480-489.
    [61] Bolukbasi T, Chang KW, Zou JY, Saligrama V, Kalai AT. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In:Proc. of the NIPS. 2016.4349-4357.
    [62] Zhao JY, Wang TL, Yatskar M, Ordonez V, Chang KW. Men also like shopping:reducing gender bias amplification using corpus-level constraints. In:Proc. of the EMNLP. 2017.2979-2989.
    [63] Zhao JY, Wang TL, Yatskar M, Ordonez V, Chang KW. Gender bias in coreference resolution:Evaluation and debiasing methods. In:Proc. of the NAACL-HLT, Vol.2.2018.15-20.
    [64] Stanovsky G, Smith NA, Zettlemoyer L. Evaluating gender bias in machine translation. In:Proc. of the ACL. 2019.1679-1684.
    [65] Du YP, Wu YB, Lan M. Exploring human gender stereotypes with word association test. In:Proc. of the EMNLP/IJCNLP. 2019.6132-6142.
    [66] Papakyriakopoulos O, Hegelich S, Serrano JCM, Marco F. Bias in word embeddings. In:Proc. of the FAT*. 2020.446-457.
    [67] Ma PC, Wang S, Liu J. Metamorphic testing and certified mitigation of fairness violations in NLP models. In:Proc. of the IJCAI. 2020.458-465.
    [68] Badilla P, Marquez FB, Pérez J. WEFE:The word embeddings fairness evaluation framework. In:Proc. of the IJCAI. 2020.430-436.
    [69] Yadav H, Du ZX, Joachims T. Fair learning-to-rank from implicit feedback. arXiv preprint arXiv:1911.08054v1, 2019.
    [70] Beutel A, Chen JL, Doshi T, Qian H, Wei L, Wu Y, Heldt L, Zhao Z, Hong LC, Chi EH, Goodrow C. Fairness in recommendation ranking through pairwise comparisons. In:Proc. of the KDD. 2019.2212-2220.
    [71] Singh A, Joachims T. Policy learning for fairness in ranking. In:Proc. of the NeurIPS. 2019.5427-5437.
    [72] Patro GK, Biswas A, Ganguly N, Gummadi KP, Chakraborty A. FairRec:Two-sided fairness for personalized recommendations in two-sided platforms. In:Proc. of the WWW. 2020.1194-1204.
    [73] Patro GK, Chakraborty A, Ganguly N, Gummadi KP. Fair updates in two-sided market platforms:On incrementally updating recommendations. In:Proc. of the AAAI. 2020.181-188.
    [74] Chakraborty A, Patro GK, Ganguly N, Gummadi KP, Loiseau P. Equality of voice:Towards fair representation in crowdsourced top-K recommendations. In:Proc. of the FAT. 2019.129-138.
    [75] Fu ZH, Xian YK, Gao RY, Zhao JY, Huang QY, Ge YQ, Xu SY, Geng SJ, Shah C, Zhang YF, Melo GD. Fairness-aware explainable recommendation over knowledge graphs. In:Proc. of the SIGIR. 2020.69-78.
    [76] Geyik SC, Ambler S, Kenthapadi K. Fairness-aware ranking in search & recommendation systems with application to LinkedIn talent search. In:Proc. of the KDD. 2019.2221-2231.
    [77] Morik M, Singh A, Hong J, Joachims T. Controlling fairness and bias in dynamic learning-to-rank. In:Proc. of the SIGIR. 2020.429-438.
    [78] Hughes E, Leibo JZ, Phillips M, Tuyls K, Duéñez-Guzmán EA, Castañeda AG, Dunning I, Zhu T, McKee KR, Koster R, Roff H, Graepel T. Inequity aversion improves cooperation in intertemporal social dilemmas. In:Proc. of the NeurIPS. 2018.3330-3340.
    [79] Zhang CJ, Shah JA. Fairness in multi-agent sequential decision-making. In:Proc. of the NIPS. 2014.2636-2644.
    [80] Jiang JC, Lu ZQ. Learning fairness in multi-agent systems. In:Proc. of the NeurIPS. 2019.13854-13865.
    [81] Zhou J, Wang L, Wang L, Zheng XL. Shared learning:Ant financial's solution. Communications of the CCF, 2020,15(6):51-57(in Chinese).
    [82] Li T, Sanjabi M, Beirami A, Smith V. Fair resource allocation in federated learning. In:Proc. of the ICLR. 2020.
    [83] Mohri M, Sivek G, Suresh AT. Agnostic federated learning. In:Proc. of the ICML. 2019.4615-4625.
    [84] Kohavi R. Scaling up the accuracy of Naive-Bayes classifiers:A decision-tree hybrid. In:Proc. of the KDD. 1996.202-207.
    [85] Wightman LF. LSAC national longitudinal bar passage study. Research Report, Law School Admission Council, 1998.
    [86] Merler M, Ratha NK, Feris RS, Smith JR. Diversity in faces. arXiv preprint arXiv:1901.10436v6, 2019.
    [87] Horn GV, Aodha OM, Song Y, Cui Y, Sun C, Shepard A, Adam H, Perona P, Belongie SJ. The INaturalist species classification and detection dataset. In:Proc. of the CVPR. 2018.8769-8778.
    [88] Bagdasaryan E, Poursaeed O, Shmatikov V. Differential privacy has disparate impact on model accuracy. In:Proc. of the NeurIPS. 2019.15453-15462.
    [89] D'Amour A, Srinivasan H, Atwood J, Baljekar P, Sculley D, Halpern Y. Fairness is not static:Deeper understanding of long term fairness via simulation studies. In:Proc. of the FAT*. 2020.525-534.
    [90] Liu LT, Dean S, Rolf E, Simchowitz M, Hardt M. Delayed impact of fair machine learning. In:Proc. of the ICML. 2018.3156-3164.
    [91] Nori H, Jenkins S, Koch P, Caruana R. InterpretML:A unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223v1, 2019.
    [92] Bellamy RKE, Dey K, Hind M, Hoffman SC, Houde S, Kannan K, Lohia P, Martino J, Mehta S, Mojsilovic A, Nagar S, Ramamurthy KN, Richards JT, Saha D, Sattigeri P, Singh M, Varshney KR, Zhang TF. AI fairness 360:An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 2019,63(4):1-15.
    附中文参考文献:
    [1] 周志华.机器学习.北京:清华大学出版社,2016.
    [2] 何积丰.安全可信人工智能.信息安全与通讯保密,2019(10):5-8.
    [3] 孟小峰,王雷霞,刘俊旭.人工智能时代的数据隐私、垄断与公平.大数据,2020,6(1):35-46.
    [4] 刘睿瑄,陈红,郭若杨,赵丹,梁文娟,李翠平.机器学习中的隐私攻击与防御.软件学报,2020,31(3):866-892. http://www.jos.org.cn/1000-9825/5904.htm[doi:10.13328/j.cnki.jos.005904]
    [5] 谭作文,张连福.机器学习隐私保护研究综述.软件学报,2020,31(7):2127-2156. http://www.jos.org.cn/1000-9825/6052.htm[doi:10.13328/j.cnki.jos.006052]
    [6] 成科扬,王宁,师文喜,詹永照.深度学习可解释性研究进展.计算机研究与发展,2020,57(6):1208-1217.
    [81] 周俊,王力,王磊,郑小林.蚂蚁金服共享智能实践.中国计算机学会通讯,2020,15(6):51-57.
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

刘文炎,沈楚云,王祥丰,金博,卢兴见,王晓玲,查宏远,何积丰.可信机器学习的公平性综述.软件学报,2021,32(5):1404-1426

复制
分享
文章指标
  • 点击次数:4925
  • 下载次数: 15386
  • HTML阅读次数: 6391
  • 引用次数: 0
历史
  • 收稿日期:2020-07-19
  • 最后修改日期:2020-11-07
  • 在线发布日期: 2021-01-15
  • 出版日期: 2021-05-06
文章二维码
友情链接:中国科学院软件研究所中国计算机学会中国科学院国家自然科学基金委员会CCF数字图书馆Int'l J of Softw Info计算机系统应用
您是第19781238位访问者
版权所有：中国科学院软件研究所 京ICP备05046678号-3
地址：北京市海淀区中关村南四街4号,邮政编码：100190
电话：010-62562563 传真：010-62562533 Email：jos@iscas.ac.cn
技术支持：北京勤云科技发展有限公司

京公网安备 11040202500063号