Metric System and Its Completeness of Adversarial Robustness Evaluation
Author:
Affiliation:

Clc Number:

TP309

Fund Project:

  • Article
  • |
  • Figures
  • |
  • Metrics
  • |
  • Reference
  • |
  • Related
  • |
  • Cited by
  • |
  • Materials
  • |
  • Comments
    Abstract:

    The assessment of adversarial robustness requires a complete and accurate evaluation of deep learning models’ noise resistance by combining the attack ability and noise magnitude of adversarial samples. However, the lack of completeness in the adversarial robustness evaluation metric system is a key problem with the existing adversarial attack and defense methods. The existing work on adversarial robustness evaluation lacks analysis and comparison of the evaluation metric system. The impact of attack success rate and different norms on the completeness of the robustness evaluation metric system and the restrictions on designing attack and defense methods are neglected. In this study, the adversarial robustness evaluation metric system is discussed in two dimensions: norm selection and metric indicators. The theoretical analysis of robustness evaluation completeness is carried out from three aspects: the inclusion relation of the evaluation metric domain, robustness description granularity, and the order relationship of the robustness evaluation metric system. The following conclusions are drawn: using noise statistical quantities such as the mean results in a larger and more comprehensive definition domain of evaluation indicators compared to using attack success rates, while also ensuring that any two adversarial sample sets can be compared. Using the $L_2 $ norm is more complete in the description of adversarial robustness evaluation compared to using other norms. Extensive experiments on 23 models and 20 adversarial attacks across 6 datasets validate these conclusions.

    Reference
    Related
    Cited by
Get Citation

石育澄,韩亚洪.对抗鲁棒性评估的指标体系及其完备性.软件学报,,():1-23

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:September 09,2023
  • Revised:November 08,2023
  • Adopted:
  • Online: June 14,2024
  • Published:
You are the firstVisitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063