微信服务号

微信订阅号

2025-4-21- 15
Home > Archive>Volume 34, Issue 9, 2023 >3941-3965. DOI:10.13328/j.cnki.jos.006875
PDF HTML XML Export Cite reminder
Quality Attributes and Practices of Trustworthy Artificial Intelligence Systems: A Tertiary Study
Author:
Affiliation:

  • Article
    • | |
  • Metrics
    • |
  • Reference [73]
    • |
  • Related [20]
    • | |
  • Materials
    • |
  • Comments
    Abstract:

    Artificial intelligence systems are widely used to solve various challenges in the real world in an unprecedented way, and they have become the core driving force for the development of human society. With the rapid popularization of artificial intelligence systems in all walks of life, the trustworthiness of artificial intelligence systems is becoming more and more worrying. The main reason is that the trustworthiness of traditional software systems is not enough to fully describe that of artificial intelligence systems. Therefore, research on the trustworthiness of artificial intelligence systems is urgently needed. At present, there have been a large number of relevant studies, which focus on different aspects. However, these studies lack a holistic and systematic understanding. This study is a tertiary study with the existing secondary study as the research object. It aims to reveal the research status of quality attributes and practices related to the trustworthiness of artificial intelligence systems and establish a more comprehensive quality attribute framework for trustworthy artificial intelligence systems. This study collects, sorts out, and analyzes 34 secondary studies published until March 2022. In addition, it identifies 21 quality attributes related to trustworthiness, as well as measurement methods and assurance practices of trustworthiness. The study finds that existing research mainly focuses on security and privacy, and extensive and in-depth research on other quality attributes is fewer. Furthermore, two research directions requiring interdisciplinary collaboration need more attention in future research. On the one hand, the artificial intelligence system is essentially a software system, and its trustworthiness as a software system is worthy of collaborative research by artificial intelligence and software engineering experts. On the other hand, artificial intelligence belongs to human’s exploration of machine anthropomorphism, and research on how to ensure the trustworthiness of machines in the social environment from the system level, such as how to satisfy humanism, is worthy of collaborative research by artificial intelligence and social science experts.

    Reference
    [1] Anthes G. Artificial intelligence poised to ride a new wave. Communications of the ACM, 2017, 60(7): 19-21. [doi: 10.1145/3088342]
    [2] McGraw G, Bonett R, Figueroa H, Shepardson V. Security engineering for machine learning. Computer, 2019, 52(8): 54-57. [doi: 10.1109/MC.2019.2909955]
    [3] Angwin J, Larson J, Mattu S, Kirchner L. Machine bias: Risk assessment in criminal sentencing. ProPublica, 2016.
    [4] Duan RJ, Mao XF, Qin AK, Chen YF, Ye SK, He Y, Yang Y. Adversarial laser beam: Effective physical-world attack to DNNs in a blink. In: Proc. of the 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 16057-16066.
    [5] Fredrikson M, Jha S, Ristenpart T. Model inversion attacks that exploit confidence information and basic countermeasures. In: Proc. of the 22nd ACM SIGSAC Conf. on Computer and Communications Security. Denver: ACM, 2015. 1322-1333.
    [6] Shneiderman B. Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy human-centered AI systems. ACM Transactions on Interactive Intelligent Systems, 2020, 10(4): 26. [doi: 10.1145/3419764]
    [7] Buruk B, Ekmekci PE, Arda B. A critical perspective on guidelines for responsible and trustworthy artificial intelligence. Medicine, Health Care and Philosophy, 2020, 23(3): 387-399. [doi: 10.1007/s11019-020-09948-1]
    [8] Tolmeijer S, Kneer M, Sarasua C, Christen M, Bernstein A. Implementations in machine ethics: A survey. ACM Computing Surveys, 2021, 53(6): 132. [doi: 10.1145/3419633]
    [9] ISO. ISO/IEC TR 24028: 2020 Information technology—Artificial intelligence—Overview of trustworthiness in artificial intelligence. Int’l Organization for Standardization, 2020.
    [10] European Commission. Ethics guidelines for trustworthy AI. Publications Office of the European Union, 2019. https://op.europa.eu/en/publication-detail/-/publication/d3988569-0434-11ea-8c1f-01aa75ed71a1
    [11] European Commission. Assessment list for trustworthy artificial intelligence. Publications Office of the European Union, 2019. https://op.europa.eu/en/publication-detail/-/publication/73552fcd-f7c2-11ea-991b-01aa75ed71a1
    [12] 中国信息通信研究院, 京东探索研究院. 可信人工智能白皮书. 2021. http://www.caict.ac.cn/kxyj/qwfb/bps/202107/t20210708_380126.htm
    China Academy of Information and Communications Technology, JD Explore Academy. White Paper on Trustworthy Artificial Intelligence. 2021 (in Chinese). http://www.caict.ac.cn/kxyj/qwfb/bps/202107/t20210708_380126.htm
    [13] Kitchenham B, Charters S. Guidelines for performing systematic literature reviews in software engineering. Keele: Keele University, 2007. https://www.elsevier.com/__data/promis_misc/525444systematicreviewsguide.pdf
    [14] Kaur D, Uslu S, Durresi A. Requirements for trustworthy artificial intelligence-a review. In: Proc. of the 23rd Int’l Conf. on Network-based Information Systems. Cham: Springer, 2021. 105-115.
    [15] Xiong PL, Buffett S, Iqbal S, Lamontagne P, Mamun M, Molyneaux H. Towards a robust and trustworthy machine learning system development: An engineering perspective. Journal of Information Security and Applications, 2022, 65: 103121. [doi: 10.1016/j.jisa.2022.103121]
    [16] Kaur D, Uslu S, Rittichier KJ, Durresi A. Trustworthy artificial intelligence: A review. ACM Computing Surveys, 2023, 55(2): 39. [doi: 10.1145/3491209]
    [17] Tariq MI, Memon NA, Ahmed S, Tayyaba S, Mushtaq MT, Mian NA, Imran M, Ashraf MW. A review of deep learning security and privacy defensive techniques. Mobile Information Systems, 2020, 2020: 6535834. [doi: 10.1155/2020/6535834]
    [18] Boulemtafes A, Derhab A, Challal Y. A review of privacy-preserving techniques for deep learning. Neurocomputing, 2020, 384: 21-45. [doi: 10.1016/j.neucom.2019.11.041]
    [19] Hanif A, Zhang XY, Wood S. A survey on explainable artificial intelligence techniques and challenges. In: Proc. of the 25th IEEE Int’l Enterprise Distributed Object Computing Workshop (EDOCW). Gold Coast: IEEE, 2021. 81-89.
    [20] Kitchenham BA, Pretorius R, Budgen D, Brereton OP, Turner M, Niazi M, Linkman S. Systematic literature reviews in software engineering - A tertiary study. Information and Software Technology, 2010, 52(8): 792-805. [doi: 10.1016/j.infsof.2010.03.006]
    [21] Kitchenham BA, Budgen D, Brereton P. Evidence-based Software Engineering and Systematic Reviews. New York: CRC Press, 2015.
    [22] Dyba T, Kitchenham BA, Jorgensen M. Evidence-based software engineering for practitioners. IEEE Software, 2005, 22(1): 58-65. [doi: 10.1109/MS.2005.6]
    [23] Kitchenham BA, Dyba T, Jorgensen M. Evidence-based software engineering. In: Proc. of the 26th Int’l Conf. on Software Engineering. Edinburgh: IEEE, 2004. 273-281.
    [24] Serban A, van der Blom K, Hoos H, Visser J. Practices for engineering trustworthy machine learning applications. In: Proc. of the 1st IEEE/ACM Workshop on AI Engineering-software Engineering for AI (WAIN). Madrid: IEEE, 2021. 97-100.
    [25] Dilmaghani S, Brust MR, Danoy G, Cassagnes N, Pecero J, Bouvry P. Privacy and security of big data in AI systems: A research and standards perspective. In: Proc. of the 2019 IEEE Int’l Conf. on Big Data (Big Data). Los Angeles: IEEE, 2019. 5737-5743.
    [26] Fagbola TM, Thakur SC. Towards the development of artificial intelligence-based systems: Human-centered functional requirements and open problems. In: Proc. of the 2019 Int’l Conf. on Intelligent Informatics and Biomedical Sciences (ICIIBMS). Shanghai: IEEE, 2019. 200-204.
    [27] Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A. A survey on bias and fairness in machine learning. ACM Computing Surveys, 2022, 54(6): 115. [doi: 10.1145/3457607]
    [28] Meenakshi K, Maragatham G. A review on security attacks and protective strategies of machine learning. In: Hemanth DJ, Kumar VDA, Malathi S, Castillo O, Patrut B, eds. Emerging Trends in Computing and Expert Technology. Cham: Springer, 2020. 1076-1087.
    [29] Zhang Y, Cai Y, Zhang M, Li X, Fan YF. A survey on privacy-preserving deep learning with differential privacy. In: Proc. of the 3rd Int’l Conf. on Big Data and Security. Shenzhen: Springer, 2022. 18-30.
    [30] Tiwari K, Shukla S, George JP. A systematic review of challenges and techniques of privacy-preserving machine learning. In: Shukla S, Unal A, Kureethara JV, Mishra DK, Han DS, eds. Data Science and Security. Singapore: Springer, 2021. 19-41.
    [31] Madhusudhanan S, Nair RR. Converging security threats and attacks insinuation in multidisciplinary machine learning applications: A survey. In: Proc. of the 2019 Int’l Seminar on Research of Information Technology and Intelligent Systems (ISRITI). Yogyakarta: IEEE, 2019. 217-222.
    [32] Chai CL, Wang JY, Luo YY, Niu ZP, Li GL. Data management for machine learning: A survey. IEEE Trans. on Knowledge and Data Engineering, 2023, 35(5): 4646-4667.
    [33] Došilović FK, Brčić M, Hlupić N. Explainable artificial intelligence: A survey. In: Proc. of the 41st Int’l Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). Opatija: IEEE, 2018. 210-215.
    [34] Chakraborty S, Tomsett R, Raghavendra R, Harborne D, Alzantot M, Cerutti F, Srivastava M, Preece A, Julier S, Rao RM, Kelley TD, Braines D, Sensoy M, Willis CJ, Gurram P. Interpretability of deep learning models: A survey of results. In: Proc. of the 2017 IEEE Smartworld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (smartworld/SCALCOM/UIC/ATC/CBDcom/IOP/SCI). San Francisco: IEEE, 2017. 1-6.
    [35] Vollert S, Atzmueller M, Theissler A. Interpretable machine learning: A brief survey from the predictive maintenance perspective. In: Proc. of the 26th IEEE Int’l Conf. on Emerging Technologies and Factory Automation (ETFA). Vasteras: IEEE, 2021. 1-8.
    [36] Ha T, Dang TK, Le H, Truong TA. Security and privacy issues in deep learning: A brief review. SN Computer Science, 2020, 1(5): 253. [doi: 10.1007/s42979-020-00254-4]
    [37] Gezici B, Tarhan AK. Systematic literature review on software quality for AI-based software. Empirical Software Engineering, 2022, 27(3): 66. [doi: 10.1007/s10664-021-10105-2]
    [38] França HL, Teixeira C, Laranjeiro N. Techniques for evaluating the robustness of deep learning systems: A preliminary review. In: Proc. of the 10th Latin-American Symp. on Dependable Computing (LADC). Florianópolis: IEEE, 2021. 1-5.
    [39] Liu B, Ding M, Shaham S, Rahayu W, Farokhi F, Lin ZH. When machine learning meets privacy: A survey and outlook. ACM Computing Surveys, 2022, 54(2): 31. [doi: 10.1145/3436755]
    [40] Guan ZY, Bian LX, Shang T, Liu JW. When machine learning meets security issues: A survey. In: Proc. of the 2018 IEEE Int’l Conf. on Intelligence and Safety for Robotics (ISR). Shenyang: IEEE, 2018. 158-165.
    [41] Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable AI: A review of machine learning interpretability methods. Entropy, 2021, 23(1): 18. [doi: 10.3390/e23010018]
    [42] 刘俊旭, 孟小峰. 机器学习的隐私保护研究综述. 计算机研究与发展, 2020, 57(2): 346-362. [doi: 10.7544/issn1000-1239.2020.20190455]
    Liu JX, Meng XF. Survey on privacy-preserving machine learning. Journal of Computer Research and Development, 2020, 57(2): 346-362 (in Chinese with English abstract). [doi: 10.7544/issn1000-1239.2020.20190455]
    [43] 纪守领, 杜天宇, 李进锋, 沈超, 李博. 机器学习模型安全与隐私研究综述. 软件学报, 2021, 32(1): 41-67. http://www.jos.org.cn/1000-9825/6131.htm
    Ji SL, Du TY, Li JF, Shen C, Li B. Security and privacy of machine learning models: A survey. Ruan Jian Xue Bao/Journal of Software, 2021, 32(1): 41-67 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6131.htm
    [44] 纪守领, 李进锋, 杜天宇, 李博. 机器学习模型可解释性方法、应用与安全研究综述. 计算机研究与发展, 2019, 56(10): 2071-2096. [doi: 10.7544/issn1000-1239.2019.20190540]
    Ji SL, Li JF, Du TY, Li B. Survey on techniques, applications and security of machine learning interpretability. Journal of Computer Research and Development, 2019, 56(10): 2071-2096 (in Chinese with English abstract). [doi: 10.7544/issn1000-1239.2019.20190540]
    [45] 何英哲, 胡兴波, 何锦雯, 孟国柱, 陈恺. 机器学习系统的隐私和安全问题综述. 计算机研究与发展, 2019, 56(10): 2049-2070. [doi: 10.7544/issn1000-1239.2019.20190437]
    He YZ, Hu XB, He JW, Meng GZ, Chen K. Privacy and security issues in machine learning systems: A survey. Journal of Computer Research and Development, 2019, 56(10): 2049-2070 (in Chinese with English abstract). [doi: 10.7544/issn1000-1239.2019.20190437]
    [46] 谭作文, 张连福. 机器学习隐私保护研究综述. 软件学报, 2020, 31(7): 2127-2156. http://www.jos.org.cn/1000-9825/6052.htm
    Tan ZW, Zhang LF. Survey on privacy preserving techniques for machine learning. Ruan Jian Xue Bao/Journal of Software, 2020, 31(7): 2127-2156 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6052.htm
    [47] 刘睿瑄, 陈红, 郭若杨, 赵丹, 梁文娟, 李翠平. 机器学习中的隐私攻击与防御. 软件学报, 2020, 31(3): 866-892. http://www.jos.org.cn/1000-9825/5904.htm
    Liu RX, Chen H, Guo RY, Zhao D, Liang WJ, Li CP. Survey on privacy attacks and defenses in machine learning. Ruan Jian Xue Bao/Journal of Software, 2020, 31(3): 866-892 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5904.htm
    [48] 刘文炎, 沈楚云, 王祥丰, 金博, 卢兴见, 王晓玲, 查宏远, 何积丰. 可信机器学习的公平性综述. 软件学报, 2021, 32(5): 1404-1426. http://www.jos.org.cn/1000-9825/6214.htm
    Liu WY, Shen CY, Wang XF, Jin B, Lu XJ, Wang XL, Zha HY, He JF. Survey on fairness in trustworthy machine learning. Ruan Jian Xue Bao/Journal of Software, 2021, 32(5): 1404-1426 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6214.htm
    [49] Pessach D, Shmueli E. A review on fairness in machine learning. ACM Computing Surveys, 2023, 55(3): 51. [doi: 10.1145/3494672]
    [50] Hu YP, Kuang WX, Qin Z, Li KL, Zhang JL, Gao YS, Li WJ, Li KQ. Artificial intelligence security: Threats and countermeasures. ACM Computing Surveys, 2023, 55(1): 20. [doi: 10.1145/3487890]
    [51] Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D. A survey of methods for explaining black box models. ACM computing surveys, 2019, 51(5): 93. [doi: 10.1145/3236009]
    [52] Zhang H, Babar MA, Tell P. Identifying relevant studies in software engineering. Information and Software Technology, 2011, 53(6): 625-637. [doi: 10.1016/j.infsof.2010.12.010]
    [53] ISO. ISO/IEC TR 24027:2021 Information technology—Artificial intelligence (AI)—Bias in AI systems and AI aided decision making. Int’l Organization for Standardization, 2021.
    [54] ISO. ISO/IEC TR 24029-1:2021 Artificial intelligence (AI)—Assessment of the robustness of neural network—Part 1: Overview. Int’l Organization for Standardization, 2021.
    [55] IEEE. IEEE 3652.1-2020 IEEE guide for architectural framework and application of federated machine learning. IEEE, 2021.
    [56] IEEE. IEEE P7001/D4-2021 IEEE approved draft standard for transparency of autonomous systems. IEEE. 2021. https://ieeexplore.ieee.org/document/9574622
    [57] Wohlin C. Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: Proc. of the 18th Int’l Conf. on Evaluation and Assessment in Software Engineering. London: ACM, 2014. 38.
    [58] Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology, 2006, 3(2): 77-101. [doi: 10.1191/1478088706qp063oa]
    [59] Charmaz K. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. Thousand Oaks: Sage, 2006.
    [60] Jia CJ, Cai Y, Yu YT, Tse TH. 5W+1H pattern: A perspective of systematic mapping studies and a case study on cloud software testing. Journal of Systems and Software, 2016, 116: 206-219. [doi: 10.1016/j.jss.2015.01.058]
    [61] Hart G. The five W's: An old tool for the new task of audience analysis. Technical Communication, 1996, 43(2): 139-145.
    [62] Pan ZD, Kosicki GM. Framing analysis: An approach to news discourse. Political Communication, 1993, 10(1): 55-75. [doi: 10.1080/10584609.1993.9962963]
    [63] ISO. ISO/IEC 25010:2011 Systems and software engineering—Systems and software quality requirements and evaluation (SQuaRE)—System and software quality models. Int’l Organization for Standardization, 2011.
    [64] Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K. An empirical comparison of model validation techniques for defect prediction models. IEEE Transactions on Software Engineering, 2017, 43(1): 1-18. [doi: 10.1109/TSE.2016.2584050]
    [65] Wu SH, Zhang C, Wang FT. Extracting software security concerns of problem frames based on a mapping study. In: Proc. of the 24th Asia-Pacific Software Engineering Conf. Workshops. Nanjing: IEEE, 2017. 121-125.
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

李功源,刘博涵,杨雨豪,邵栋.可信人工智能系统的质量属性与实现: 三级研究.软件学报,2023,34(9):3941-3965

Copy
Share
Article Metrics
  • Abstract:1849
  • PDF: 4032
  • HTML: 2742
  • Cited by: 0
History
  • Received:September 04,2022
  • Revised:October 13,2022
  • Online: January 13,2023
  • Published: September 06,2023
Links:Institute of Software, CASChina Computer FederationCASNSFC
You are the first2036676Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address：4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code：100190
Phone：010-62562563 Fax：010-62562533 Email：jos@iscas.ac.cn
Technical Support：Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063