智能运维的实践: 现状与标准化
作者:
作者简介:

包航宇(1981-),男,工程师,主要研究领域为运维管理;殷康璘(1992-),男,博士,助理研究员,CCF学生会员,主要研究领域为基于机器学习的智能运维,软件工程,软件系统韧性;曹立(1981-),女,硕士,主要研究领域为智能运维,风险管理,混沌工程;李世宁(1977-),男,硕士,主要研究领域为运维管理;孙永敬(1978-),男,工程师,主要研究领域为运维管理;尹汇锋(1998-),男,硕士生,主要研究领域为智能运维;汤汝鸣(1991-),男,博士,主要研究领域为基于机器学习的智能运维;侯岳(1982-),女,高级工程师,主要研究领域为运维管理;王士强(1982-),男,高级工程师,主要研究领域为运维管理;裴丹(1973-),男,博士,副教授,博士生导师,CCF专业会员,主要研究领域为基于机器学习的智能运维;杨晓勤(1970-),女,高级经济师,主要研究领域为运维管理;王立新(1968-),男,高级工程师,主要研究领域为运维管理.

通讯作者:

殷康璘,E-mail:yinkanglin@tsinghua.edu.cn

基金项目:

国家重点研发计划(2019YFB1802504, 2019YFE0105500); 国家自然科学基金面上项目(62072264)


AIOps in Practice: Status Quo and Standardization
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [139]
  • |
  • 相似文献
  • |
  • 引证文献
  • | |
  • 文章评论
    摘要:

    IT系统运维目前正面临着IT规模快速膨胀、系统架构日趋复杂、自主可控要求日益突出等众多挑战. 智能运维技术作为一种利用大数据和机器学习对海量运维数据分析的手段, 能够辅助运维人员更为高效地运行和维护IT系统. 然而, 在企业进行智能运维工程化实践的过程中, 往往会遇到各种困难, 需要智能运维技术的标准规范以指导企业开展智能运维的能力建设工作. 为推动智能运维的标准化工作, 对多个行业的智能运维实施单位开展了问卷调研, 分析总结国内智能运维的实践现状; 对国内外现行的运维标准、人工智能标准和智能运维标准进行梳理, 研究智能运维的标准化工作当前进展; 根据对实践现状和现有标准的调研分析结果, 提出智能运维的能力建设标准框架AIOps-OSA. 该框架从企业建设智能运维能力的角度列举出在组织、场景和能力上的关键要点. 在实际标准的编制过程中通过对AIOps-OSA内各项要点提出具体的规范要求, 可形成对企业具有指导作用的智能运维标准规范.

    Abstract:

    The operation of IT systems faces many challenges of rapid IT scale expansion, increasingly complex system architecture, and growing demand for autonomy. By employing big data and machine learning technologies to analyze massive operation data, artificial intelligence for IT operations (AIOps) can assist IT operators in operating and maintaining IT systems more efficiently. However, enterprises often encounter various difficulties when practicing AIOps. Thus standards of AIOps are required to guide enterprises in building AIOps capability. To promote the standardization of AIOps, this study surveys the AIOps-in-practice enterprises in various industries to analyze the practice status of AIOps. Existing standards on operation, artificial intelligence, and AIOps are studied to figure out the current progress of AIOps standardization. According to the conclusions above, the study proposes an AIOps capability standard framework AIOps-OSA. The framework lists the critical points of organization, scenarios, and abilities from the perspective of building AIOps capabilities of enterprises. During actual standard preparation, aguiding AIOps standard for enterprises can be formed by applying detailed requirements to AIOps-OSA.

    参考文献
    [1] The 20. The Cost of IT Downtime. 2021. https://www.the20.com/blog/the-cost-of-it-downtime/
    [2] Fletcher C. Innovation insight for algorithmic IT operations platforms. Gartner. 2016. https://www.gartner.com/en/newsroom/press-releases/2017-04-11-gartner-says-algorithmic-it-operations-drives-digital-business#:~:text=What%20is%20algorithmic%20IT%20operations%20%28AIOps%29%3F%20A.%20AIOps,%28APIs%29%20allowing%20for%20a%20vendor-agnostic%20data%20ingestion%20capability
    [3] Prasad P, Rich C. Market guide for AIOps platforms. Gartner. 2016. https://www.gartner.com/en/documents/4000217
    [4] Wan ZY, Xia X, Lo D, Murphy GC. How does machine learning change software development practices?. IEEE Transactions on Software Engineering, 2021, 47(9): 1857-1871. [doi: 10.1109/TSE.2019.2937083]
    [5] Sabharwal N, Bhardwaj G. AIOps challenges. In: Hands-on AIOps. Berkeley: Apress, 2022. 61-72.
    [6] Navin Singh, AIOps Challenges, Expectations vs. Reality. 2021. https://www.lntinfotech.com/wp-content/uploads/2021/06/AIOps-Challenges-Expectations-Vs-Reality.pdf
    [7] Dang YN, Lin QW, Huang P. AIOps: Real-world challenges and research innovations. In: Proc. of the 41st IEEE/ACM Int’l Conf. on Software Engineering: Companion Proceedings (ICSE-Companion). Montreal: IEEE, 2019. 4-5.
    [8] Xie ZJ, Hall J, McCarthy IP, Skitmore M, Shen LY. Standardization efforts: The relationship between knowledge dimensions, search processes and innovation outcomes. Technovation, 2016, 48-49: 69-78.
    [9] Tassey G. Standardization in technology-based markets. Research Policy, 2000, 29(4-5): 587-602.
    [10] Zielke T. Is artificial intelligence ready for standardization?. In: Proc. of the 27th European Conf. on Systems, Software and Services Process Improvement. Düsseldorf: Springer, 2000. 259-274.
    [11] Golenkov V, Guliakina N, Golovko V, Krasnoproshin V. Artificial intelligence standardization is a key challenge for the technologies of the future. In: Proc. of the 10th Int’l Conf. on Open Semantic Technologies for Intelligent System. Minsk: Springer, 2020. 1-21.
    [12] Saunderson C. Hype cycle for I&O automation. Gartner. 2021. https://www.gartner.com/en/documents/4006837
    [13] Ji K, Xiang M, Chen O. Hype cycle for ICT in China (2021). Gartner. 2021. https://www.gartner.com/en/documents/4003537
    [14] 云计算开源产业联盟. 中国AIOps现状调查报告(2022). 中国信息通信研究院, 2022. http://www.caict.ac.cn/kxyj/qwfb/ztbg/202301/P020230109534000377704.pdf
    OSCAR. Investigation Report on Current Situation of AIOps in China (2022). China Academy of Information and Communications Technology, 2022 (in Chinese). http://www.caict.ac.cn/kxyj/qwfb/ztbg/202301/P020230109534000377704.pdf
    [15] Grieser T. Worldwide IT operations analytics software forecast, 2021-2025: Data volumes and AIOps capabilities drive growth. Int’l Data Corporation. 2021. https://www.idc.com/getdoc.jsp?containerId=US48927721&pageType=PRINTFRIENDLY
    [16] Notaro P, Cardoso J, Gerndt M. A systematic mapping study in AIOps. In: Proc. of the 2020 Int’l Conf. on Service-oriented Computing. Springer, 2020. 110-123.
    [17] Reiter L, Wedel FH. AIOps—A systematic literature review. 2021. https://www.fh-wedel.de/fileadmin/Mitarbeiter/Records/Reiter_2021_-_AIOps_-_A_Systematic_Literature_Review.pdf
    [18] Notaro P, Cardoso J, Gerndt M. A survey of AIOps methods for failure management. ACM Transactions on Intelligent Systems and Technology, 2021, 12(6): 81. [doi: 10.1145/3483424]
    [19] Rijal L, Colomo-Palacios R, Sánchez-Gordón M. AIOps: A multivocal literature review. In: Misra S, Tyagi AK, Piuri V, Garg L, eds. Artificial Intelligence for Cloud and Edge Computing. Springer, 2022. 31-50.
    [20] Jiang ZX, Li T, Zhang ZG, Ge JG, You JL, Li LX. A survey on log research of AIOps: Methods and trends. Mobile Networks and Applications, 2021, 26(6): 2353-2364. [doi: 10.1007/s11036-021-01832-3]
    [21] Bogatinovski J, Nedelkoski S, Acker A, Schmidt F, Wittkopp T, Becker S, Cardoso J, Kao O. Artificial intelligence for IT operations (AIOPS) workshop white paper. arXiv:2101.06054, 2021.
    [22] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 28827.1-2012 信息技术服务 运行维护 第1部分: 通用要求. 北京: 中国标准出版社, 2013.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of the People’s Republic of China. GB/T 28827.1-2012 Information technology service—Operations and maintenance—Part 1: General requirements. Beijing: Standards Press of China, 2013 (in Chinese).
    [23] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 28827.2-2012 信息技术服务 运行维护 第2部分: 交付规范. 北京: 中国标准出版社, 2013.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of the People’s Republic of China. GB/T 28827.2-2012 Information technology service—Operations and maintenance—Part 2: Delivery specification. Beijing: Standards Press of China, 2013 (in Chinese).
    [24] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 28827.3-2012 信息技术服务 运行维护 第3部分: 应急响应规范. 北京: 中国标准出版社, 2013.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of the People’s Republic of China. GB/T 28827.3-2012 Information technology service—Operations and maintenance—Part 3: Emergency response specification. Beijing: Standards Press of China, 2013 (in Chinese).
    [25] 国家市场监督管理总局, 国家标准化管理委员会. GB/T 28827.4-2019. 信息技术服务 运行维护 第4部分: 数据中心服务要求. 北京: 中国标准出版社, 2019.
    State Administration for Market Regulation, Standardization Administration of the People’s Republic of China. GB/T 28827.4-2019 Information technology service—Operations and maintenance—Part 4: Service requirements for data center. Beijing: Standards Press of China, 2019 (in Chinese).
    [26] 国家市场监督管理总局, 国家标准化管理委员会. GB/T 28827.6-2019 信息技术服务 运行维护 第6部分: 应用系统服务要求. 北京: 中国标准出版社, 2019.
    State Administration for Market Regulation, Standardization Administration of the People’s Republic of China. GB/T 28827.6-2019 Information technology service—Operations and maintenance—Part 6: Application system service requirements. Beijing: Standards Press of China, 2019 (in Chinese).
    [27] 中国电子工业标准化技术协会. ITSS. 1-2015 信息技术服务 运行维护服务能力成熟度模型. 2015. https://www.itss.cn/file/cms/article/2021-03/1615517038723/%E4%BF%A1%E6%81%AF%E6%8A%80%E6%9C%AF%E6%9C%8D%E5%8A%A1%20%E8%BF%90%E8%A1%8C%E7%BB%B4%E6%8A%A4%E6%9C%8D%E5%8A%A1%E8%83%BD%E5%8A%9B%E6%88%90%E7%86%9F%E5%BA%A6%E6%A8%A1%E5%9E%8B.pdf
    CESA. ITSS. 1-2015 Information technology service—Maintenance service capability maturity model. 2015 (in Chinese). https://www.itss.cn/file/cms/article/2021-03/1615517038723/%E4%BF%A1%E6%81%AF%E6%8A%80%E6%9C%AF%E6%9C%8D%E5%8A%A1%20%E8%BF%90%E8%A1%8C%E7%BB%B4%E6%8A%A4%E6%9C%8D%E5%8A%A1%E8%83%BD%E5%8A%9B%E6%88%90%E7%86%9F%E5%BA%A6%E6%A8%A1%E5%9E%8B.pdf
    [28] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 33136-2016 信息技术服务 数据中心服务能力成熟度模型. 北京: 中国标准出版社, 2017. https://std.samr.gov.cn/gb/search/gbDetailed?id=71F772D815CBD3A7E05397BE0A0AB82A
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of the People’s Republic of China. GB/T 33136-2016 Information technology service—Service capability maturity model of data center. Beijing: Standards Press of China, 2017 (in Chinese). https://std.samr.gov.cn/gb/search/gbDetailed?id=71F772D815CBD3A7E05397BE0A0AB82A
    [29] 国家市场监督管理总局, 国家标准化管理委员会. GB/T 38633-2020. 信息技术服务 大数据 系统运维和管理功能要求. 北京: 中国标准出版社, 2020. https://std.samr.gov.cn/gb/search/gbDetailed?id=A47A713B763314ABE05397BE0A0ABB25
    State Administration for Market Regulation, Standardization Administration of the People’s Republic of China. GB/T 38633-2020 Information technology—Big data—Functional requirements for system operation and management. Beijing: Standards Press of China, 2020 (in Chinese). https://std.samr.gov.cn/gb/search/gbDetailed?id=A47A713B763314ABE05397BE0A0ABB25
    [30] ISO. ISO/IEC 20000-1:2018 Information technology—Service management—Part 1: Service management system requirements. 2018. https://www.iso.org/standard/70636.html
    [31] ISO. ISO/IEC 20000-2:2019 Information technology—Service management—Part 2: Guidance on the application of service management systems. 2019. https://www.iso.org/standard/72120.html
    [32] ISO. ISO/IEC 20000-3:2019 Information technology — Service management — Part 3: Guidance on scope definition and applicability of ISO/IEC 20000-1. 2019. https://www.iso.org/standard/72121.html
    [33] ISO. ISO/IEC TS 20000-5:2022 Information technology—Service management—Part 5: Implementation guidance for ISO/IEC 20000-1. 2022. https://www.iso.org/standard/81164.html
    [34] ISO. ISO/IEC TS 20000-11:2021 Information technology—Service management—Part 11: Guidance on the relationship between ISO/IEC 20000-1 and service management frameworks: ITIL®. 2021. https://www.iso.org/standard/81165.html
    [35] ISO. ISO/IEC TR 20000-12:2016 Information technology—Service management—Part 12: Guidance on the relationship between ISO/IEC 20000-1: 2011 and service management frameworks: CMMI-SVC. 2016. https://www.iso.org/standard/69203.html
    [36] ISO. ISO/IEC 20000-10:2018 Information technology—Service management—Part 10: Concepts and vocabulary. 2018. https://www.iso.org/standard/74316.html
    [37] ISO. ISO/IEC TS 33054:2020 Information technology—Process assessment—Process reference model for service management. 2020. https://www.iso.org/standard/55145.html
    [38] ISO. ISO/IEC TS 33074:2020 Information technology—Process assessment—Process capability assessment model for service management. 2020. https://www.iso.org/standard/55165.html
    [39] ISO. ISO/IEC 33004:2015 Information technology—Process assessment—Requirements for process reference, process assessment and maturity models. 2015. https://www.iso.org/standard/54178.html
    [40] ETSI. Network Functions Virtualisation (NFV); Management and Orchestration, ETSI GS NFV-MAN 001, 2014. https://www.etsi.org/deliver/etsi_gs/NFV-MAN/001_099/001/01.01.01_60/gs_NFV-MAN001v010101p.pdf
    [41] ETSI. ETSI TS 128 530 V16.3.0 (2020-10)—5G; Management and orchestration; Concepts, use cases and requirements. 2019. https://www.etsi.org/deliver/etsi_ts/128500_128599/128530/15.03.00_60/ts_128530v150300p.pdf
    [42] ETSI. ETSI GS ZSM 001 V1.1.1 (2019-10)—Zero-touch network and Service Management (ZSM); Requirements based on documented scenarios. 2019. https://www.etsi.org/deliver/etsi_gs/ZSM/001_099/001/01.01.01_60/gs_ZSM001v010101p.pdf
    [43] ETSI. ETSI GS ZSM 002 V1.1.1 (2019-08)—Zero-touch network and Service Management (ZSM); Reference Architecture. 2019. https://www.etsi.org/deliver/etsi_gs/ZSM/001_099/002/01.01.01_60/gs_ZSM002v010101p.pdf
    [44] ETSI. ETSI GS ZSM 007 V1.1.1 (2019-08)—Zero-touch network and Service Management (ZSM); Terminology for concepts in ZSM. 2019. https://www.etsi.org/deliver/etsi_gs/ZSM/001_099/007/01.01.01_60/gs_ZSM007v010101p.pdf
    [45] ITU-T.M.3010 Principles for a telecommunications management network. 2000. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=4869&lang=en
    [46] ITU-T.M.3400 TMN management functions. 2001. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=4876&lang=en
    [47] ITU-T.M.3070 Overview of end-to-end cloud computing management. 2016. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=12714
    [48] ITU-T.M.3040 Principles for on-site telecommunication smart maintenance. 2019. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=13877&lang=en
    [49] ITU-T.M.3041 Framework of smart operation, management and maintenance. 2020. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14181&lang=en
    [50] 裴丹, 张圣林, 裴昶华. 基于机器学习的智能运维. 中国计算机学会通讯, 2017, 13(12): 68-72.
    Pei D, Zhang SL, Pei CH. Intelligent operation based on machine learning. Communications of CCF, 2017, 13(12): 68-72 (in Chinese). (查阅所有网上资料, 未找到对应的英文翻译, 请确认)
    [51] 中华人民共和国国家质量监督检验检疫总局. GB/T 5271.28-2001 信息技术 词汇 第28部分: 人工智能 基本概念与专家系统. 北京: 中国标准出版社, 2004.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China. GB/T 5271.28-2001 Information technology—Vocabulary—Part 28: Artificial intelligence—Basic concepts and expert system. Beijing: Standards Press of China, 2004 (in Chinese).
    [52] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 5271.31-2006 信息技术 词汇 第31部分: 人工智能 机器学习. 北京: 中国标准出版社, 2006.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of the People’s Republic of China. GB/T 5271.31-2006 Information technolgy—Vocabulary—Part 31: Artificial intelligence—Machine learning. Beijing: Standards Press of China, 2006 (in Chinese).
    [53] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 5271.34-2006 信息技术 词汇 第34部分: 人工智能 神经网络. 北京: 中国标准出版社, 2006.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of the People’s Republic of China. GB/T 5271.29-2006 Information technolgy—Vocabulary—Part 34: Artificial intelligence—Neural networks. Beijing: Standards Press of China, 2006 (in Chinese).
    [54] 中华人民共和国国家质量监督检验检疫总局, 中国国家标准化管理委员会. GB/T 5271.29-2006 信息技术 词汇 第29部分: 人工智能 语音识别与合成. 北京: 中国标准出版社, 2006.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China, Standardization Administration of the People’s Republic of China. GB/T 5271.29-2006 Information technology—Vocabulary—Part 29: Artificial intelligence-Speech recognition and synthesis. Beijing: Standards Press of China, 2006 (in Chinese).
    [55] ISO. ISO/IEC 2382:2015 Information technology—Vocabulary. 2015. https://www.iso.org/standard/63598.html
    [56] ISO. ISO/IEC DIS 22989:2021 Information technology—Artificial intelligence—Artificial intelligence concepts and terminology. 2021. https://www.iso.org/standard/74296.html
    [57] 中国电子工业标准化技术协会. T/CESA 1037-2019 信息技术 人工智能 面向机器学习的系统框架和功能要求. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28146/
    CESA. T/CESA 1037-2019 Information technology—Artificial intelligence—Framework and functional requirements of system for machine learning. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28146/
    [58] ISO. ISO/IEC DIS 23053:2022 Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML). 2022. https://www.iso.org/standard/74438.html
    [59] ISO. ISO/IEC 20546:2019 Information technology—Big data—Overview and vocabulary. 2019. https://www.iso.org/standard/68305.html
    [60] ISO. ISO/IEC TR 20547:2020 Information technology—Big data reference architecture. 2020. https://www.iso.org/standard/71275.html
    [61] ISO. Standards by ISO/IEC JTC 1/SC 42. 2022. https://www.iso.org/committee/6794475/x/catalogue/p/0/u/1/w/0/d/0
    [62] ISO. ISO/IEC TR 24732:2021 Information technology—Artificial intelligence (AI)—Overview of computational approaches for AI systems. 2021. https://www.iso.org/standard/78508.html
    [63] 中国电子工业标准化技术协会. T/CESA 1040-2019 信息技术 人工智能 面向机器学习的数据标注规程. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28148/
    CESA. T/CESA 1040-2019 Information technology—Artificial intelligence-code of practice for data annotation of machine learning. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28148/
    [64] 中国电子工业标准化技术协会. T/CESA 1034-2019 信息技术 人工智能 小样本机器学习样本量和算法要求. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28143/
    CESA. T/CESA 1034-2019 Information technology—Artificial intelligence—Sample size and algorithm requirements for few-shot learning. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28143/
    [65] 中国电子工业标准化技术协会. T/CESA 1197-2022 人工智能 深度合成图像系统技术规范. 2022. http://www.ttbz.org.cn/StandardManage/Detail/64686/
    CESA. T/CESA 1197-2022 Artificial intelligence—Technical specification for deep synthetic image system. 2022 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/64686/
    [66] 中国电子工业标准化技术协会. T/CESA 1198-2022 人工智能 视频图像审核系统技术规范. 2022. http://www.ttbz.org.cn/StandardManage/Detail/64687/
    CESA. T/CESA 1198-2022 Artificial intelligence—Technical specifications for video and image auditing system. 2022 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/64687/
    [67] 中国电子工业标准化技术协会. T/CESA 1199-2022 人工智能 智能字符识别技术规范. 2022. http://www.ttbz.org.cn/StandardManage/Detail/64689/
    CESA. T/CESA 1199-2022 Artificial intelligence—Technical specification for intelligent character recognition. 2022 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/64689/
    [68] 中国电子工业标准化技术协会. T/CESA 1035-2019 信息技术 人工智能 音视频及图像分析算法接口. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28164/
    CESA. T/CESA 1035-2019 Information technology—Artificial intelligence—Audio, video and image analysis algotithm interface. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28164/
    [69] 中华人民共和国国家质量监督检验检疫总局. GB/T 40691-2021 人工智能 情感计算用户界面 模型. 2021. https://std.samr.gov.cn/gb/search/gbDetailed?id=CE1E6A1DD4AF58F6E05397BE0A0A68DF
    ASQIQ. GB/T 40691-2021 Artificial intelligence—Affective computing user interface—Model. 2021 (in Chinese). https://std.samr.gov.cn/gb/search/gbDetailed?id=CE1E6A1DD4AF58F6E05397BE0A0A68DF
    [70] ISO. ISO/IEC TR 24028:2020 Information Technology—Artificial Intelligence—Overview of trustworthiiness in artificial intelligence. 2020. https://www.iso.org/standard/77608.html
    [71] ISO. ISO/IEC TR 24027:2021 Information Technology—Artificial Intelligence (AI)—bias in AI systems and AI aided decision making. 2021. https://www.iso.org/standard/77607.html
    [72] ISO. ISO/IEC TR 24029-1:2021 Artificial Intelligence (AI)—assessment of the robustness of neural networks—Part 1: Overview. 2021. https://www.iso.org/standard/77609.html
    [73] 中国电子工业标准化技术协会. T/CESA 1026-2018 人工智能 深度学习算法评估规范. 2018. http://www.ttbz.org.cn/StandardManage/Detail/24193/
    CESA. T/CESA 1026-2018 Artificial intelligence—Assessment specification for deep learning algorithms. 2018 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/24193/
    [74] 中国电子工业标准化技术协会. T/CESA 1036-2019 信息技术 人工智能 机器学习模型及系统的质量要素和测试方法. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28145/
    CESA. T/CESA 1036-2019 Information technology - Aritficial intelligence - Quality elements and testing methods of machine learning model and system. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28145/
    [75] 中国电子工业标准化技术协会. T/CESA 1041-2019 信息技术 人工智能 服务能力成熟度评价参考模型. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28166/
    CESA. T/CESA 1041-2019 Information technology—Artificial intelligence—Reference model of service capability maturity evaluation. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28166/
    [76] 中国电子工业标准化技术协会. T/CESA 1069-2021 信息技术 人工智能 服务器系统性能测试规范. 2021. http://www.ttbz.org.cn/StandardManage/Detail/49918/
    CESA. T/CESA 1069-2021 Information technology—Artificial intelligence—Specification for performance benchmarking for server systems. 2021 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/49918/
    [77] 中国人民银行. JR/T 0221-2021 人工智能算法金融应用评价规范. 2021. https://www.qybz.org.cn/user/detail/ae6ba173412c0dd171a112d4448fb91cd9a3aca23f18f50c0713c4807cf1f7831667e8270e92630f8ddeed1ee4fdb693
    PBC People’s Bank of China. JR/T 0221-2021 Evaluation specification of artificial intelligence algorithm in financial application. 2021 (in Chinese). https://www.qybz.org.cn/user/detail/ae6ba173412c0dd171a112d4448fb91cd9a3aca23f18f50c0713c4807cf1f7831667e8270e92630f8ddeed1ee4fdb693
    [78] ISO. ISO/IEC TR 24030:2021 Information technology—Artificial Intelligence (AI)—Use cases. 2021. https://www.iso.org/standard/77610.html
    [79] ISO. ISO/IEC TR 38507:2022 Information Technology—Governance of AI—Governance implications of the use of artificial intelligence by organizations. 2022. https://www.iso.org/standard/56641.html
    [80] 中国电子工业标准化技术协会. T/CESA 1038-2019 信息技术 人工智能 智能助理能力等级评估. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28147/
    CESA. T/CESA 1038-2019 Artificial intelligence—Artificial Intelligence—Classified assessment on intelligent assistant’s capabilities. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28147/
    [81] 中国电子工业标准化技术协会. T/CESA 1039-2019 信息技术 人工智能 机器翻译能力等级评估. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28153/
    CESA. T/CESA 1039-2019 Information technology—Artificial intelligence—Classified assessment for machine translation capabilities. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28153/
    [82] 中国电子工业标准化技术协会. T/CESA 1044-2019 信息技术 人工智能 机动车驾驶员驾驶状态视频采集系统规范. 2019. http://www.ttbz.org.cn/StandardManage/Detail/28152/
    CESA. T/CESA 1044-2019 Information technology—Artificial intelligence—Specification of driving status video capture system for vehicle drivers. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/28152/
    [83] 中国电子工业标准化技术协会. T/CESA 1127-2020 基于人工智能语音交互的养老服务平台要求. 2020. http://www.ttbz.org.cn/StandardManage/Detail/41495/
    CESA. T/CESA 1127-2020 Requirements of elderly care service platform based on artificial intelligence speech interaction. 2020 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/41495/
    [84] 广东省质量技术监督信息协会. T/GZBC 8-2019 肺结节人工智能辅助诊断系统技术要求. 2019. http://www.ttbz.org.cn/StandardManage/Detail/26124
    GZBC. T/GZBC 8-2019 Technical requirements of the artificial intelligence diagnostic system for pulmonary nodules. 2019 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/26124
    [85] 贵州省市场监督管理局. T/JPMA 011-2021 基于人工智能的糖尿病视网膜病变筛查的技术规范. 2021. http://www.ttbz.org.cn/StandardManage/Detail/56098/
    JPMA. T/JPMA 011-2021 Specification for screening of diabetic retinopathy by artificial intelligence technology. 2021 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/56098/
    [86] IEEE. IEEE 7000TM projects. 2022. https://ethicsinaction.ieee.org/p7000/
    [87] ITU-T.Y.3172 Architectural framework for machine learning in future networks including IMT-2020. 2019. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=13894
    [88] ITU-T.Y.3174 Framework for data handling to enable machine learning in future networks including IMT-2020. 2020. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14134
    [89] ITU-T.Y.3176 Machine learning marketplace integration in future networks including IMT-2020. 2020. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14402
    [90] ITU-T.Y.3179 Architectural framework for machine learning model serving in future networks including IMT-2020. 2021. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14614
    [91] ITU-T.Y.3170 Requirements for machine learning-based quality of service assurance for the IMT-2020 network. 2018. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=13691
    [92] ITU-T.Y.3175 Functional architecture of machine learning-based quality of service assurance for the IMT-2020 network. 2021. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14255
    [93] ITU-T.Y.3177 Architectural framework for artificial intelligence-based network automation for resource and fault management in future networks including IMT-2020. 2021. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14598
    [94] ITU-T.Y.3178 Functional framework of artificial intelligence-based network service provisioning in future networks including IMT-2020. 2021. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14613
    [95] ITU-T.Y.3180 Mechanism of traffic awareness for application-descriptor-agnostic traffic based on machine learning. 2022. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14856
    [96] ITU-T.Y.3173 Framework for evaluating intelligence levels of future networks including IMT-2020. 2020. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14133
    [97] ITU-T.M.3080 Framework of artificial intelligence enhanced telecom operation and management (AITOM). 2021. https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14590
    [98] ETSI. Experiential Networked Intelligence (ENI); System Architecture, ETSI GS ENI 005, 2021. https://www.etsi.org/deliver/etsi_gs/ENI/001_099/005/02.01.01_60/gs_ENI005v020101p.pdf
    [99] ETSI. Experiential Networked Intelligence (ENI); ENI use cases, ETSI GR ENI 001, 2020. https://www.etsi.org/deliver/etsi_gr/ENI/001_099/001/01.01.01_60/gr_ENI001v010101p.pdf
    [100] 中国电子节能技术协会. T/DZJN 24-2020 数据中心基础设施智能运维技术通则. 2020. http://www.ttbz.org.cn/StandardManage/Detail/42109
    CEESTA. T/DZJN 24-2020 General technical principles for Artificial Intelligence Operations of Data Center infrastructure. 2020 (in Chinese). http://www.ttbz.org.cn/StandardManage/Detail/42109
    [101] 中国通信标准化协会. T/CCSA 382.1-2022 云计算智能化运维(AIOps)能力成熟度模型 第1部分: 通用能力要求. 2022. https://www.ccsa.org.cn/standardDetail/?standardNum=T%2FCCSA%20382.1-2022
    CCSA. T/CCSA 382.1-2022 The capability maturity model of cloud artificial intelligence for IT operations Part1: General competency requirement (in Chinese). 2022. https://www.ccsa.org.cn/standardDetail/?standardNum=T%2FCCSA%20382.1-2022
    [102] 中国电子工业标准化技术协会. 信息技术服务标准体系建设报告5.0版. 全国信息技术标准化技术委员会, 2021. https://www.itss.cn/file/cms/article/2021-11/%E4%BF%A1%E6%81%AF%E6%8A%80%E6%9C%AF%E6%9C%8D%E5%8A%A1%E6%A0%87%E5%87%86%E4%BD%93%E7%B3%BB%E5%BB%BA%E8%AE%BE%E6%8A%A5%E5%91%8A5.0.pdf
    CESA. Construction Report on Information Technology Service Standard 5.0, China National Information Technology Standardization Technical Committee, 2021 (in Chinese). https://www.itss.cn/file/cms/article/2021-11/%E4%BF%A1%E6%81%AF%E6%8A%80%E6%9C%AF%E6%9C%8D%E5%8A%A1%E6%A0%87%E5%87%86%E4%BD%93%E7%B3%BB%E5%BB%BA%E8%AE%BE%E6%8A%A5%E5%91%8A5.0.pdf
    [103] Chang W. ISO/IEC JTC 1/SC 42(AI)/WG 2(Data) data quality for analytics and machine learning (ML). 2022. https://jtc1info.org/wp-content/uploads/2022/06/01_06_Wo_2022_05_24_ISO-IEC-JTC1-SC42-WG2-Data-Quality-for-Analytics-and-Machine-Learning-Wo-Chang-NIST-final.pdf
    相似文献
    引证文献
引用本文

包航宇,殷康璘,曹立,李世宁,孙永敬,尹汇锋,汤汝鸣,侯岳,王士强,裴丹,杨晓勤,王立新.智能运维的实践: 现状与标准化.软件学报,2023,34(9):4069-4095

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2022-09-05
  • 最后修改日期:2022-10-13
  • 在线发布日期: 2023-01-13
  • 出版日期: 2023-09-06
文章二维码
您是第19700975位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号