基于大模型语义匹配的跨平台移动应用测试脚本录制回放
CSTR:
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

TP311

基金项目:

国家自然科学基金(62272220, 62372228); 中央高校基本科研业务费专项资金(14380029)


Semantic Matching-based Cross-platform Mobile App Test Script Record and Replay via Large Language Models
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    GUI测试是移动应用质量保障的重要手段之一. 随着移动生态的不断发展, 尤其是国产移动应用(如鸿蒙等)生态的强势崛起, GUI测试脚本跨平台录制回放成为了当前GUI测试的主要挑战之一. 开发者需将传统平台中GUI测试脚本迁移至新兴环境中, 以保证应用质量可靠性与多平台用户体验一致性. 然而, 不同平台间的底层实现差异导致了移动应用测试跨平台迁移的重大障碍, 这一挑战在面向新兴国产移动生态平台的测试迁移方面尤为突出. 移动应用的跨平台测试脚本录制回放是确保应用在不同操作系统和设备上保持一致性和高质量用户体验的关键. 现有技术仅解决了“一对一”事件匹配的情况, 而由于平台间GUI开发实践的不一致性, 测试事件的回放并非完全一对一映射, 而存在普遍的“多对多”映射情况, 即若干测试事件所对应的业务流程在不同平台上对应数量不等的测试事件. 为解决上述问题与挑战, 提出了一种基于大模型语义匹配的跨平台移动应用测试脚本录制回放方法(LLMRR). LLMRR方法结合图像匹配、文本匹配和大语言模型语义匹配技术, 在录制阶段通过图像分割算法记录用户操作信息, 并保存为录制测试脚本; 在回放阶段, 通过图像匹配和文本匹配模块在回放页面上找到对应的控件, 执行操作, 当无法匹配时, 调用大模型语义匹配模块进行语义匹配, 确保在不同平台上的高效运行. 对国产鸿蒙应用的测试进行了探索, 选择了20个应用共100个测试脚本, 在iOS、安卓和鸿蒙平台之间进行迁移测试, 并与当前最先进跨平台测试脚本录制回放方法LIRAT和MAPIT进行有效性对比. 结果表明, LLMRR方法在测试脚本录制回放中均表现出显著优势.

    Abstract:

    GUI testing is one of the most important measures to ensure mobile application (App) quality. With the continuous development of the mobile ecosystem, especially the strong rise of the domestic mobile ecosystem, e.g., HarmonyOS, GUI test script recording and replay has become one of the prominent challenges in GUI testing. GUI test scripts must be migrated from traditional mobile platforms to emerging mobile platforms to ensure the reliability of App quality and consistency in user experience across diverse platforms. However, differences in underlying implementations across platforms have created substantial obstacles to the cross-platform migration of mobile App test scripts. This challenge is particularly pronounced in the testing migration for emerging domestic mobile ecosystem platforms. Cross-platform test script recording and replay is essential for maintaining consistency and a high-quality user experience across different platforms and devices. Current state-of-the-art approaches only address the “one-to-one” test event matching situations. However, due to inconsistencies in development practices across platforms, the replay of test events does not always map “one-to-one”; instead, “multiple-to-multiple” mapping situations are common. This means that some test events need to be mapped to a different number of test events to fulfill the same business logic. To address these issues and challenges, this study proposes a cross-platform mobile App test script recording and replay method based on large language model semantic matching (LLMRR). The LLMRR method integrates image matching, text matching, and large language model semantic matching technologies. During the recording phase, user operation information is captured using image segmentation algorithms and saved as recorded test scripts. During the replay phase, corresponding widgets on the replay App page are located using image matching and text matching modules to execute operations. When matching fails, the large language model semantic matching module is invoked for semantic matching, ensuring efficient operation across different platforms. This study presents the first exploration of testing for domestic HarmonyOS Apps, using 20 Apps and a total of 100 test scripts for migration testing across iOS, Android, and HarmonyOS platforms. The effectiveness of the LLMRR method is compared with the current state-of-the-art cross-platform test script recording and replay approaches, LIRAT and MAPIT. The results demonstrate that the LLMRR method exhibits significant advantages in test script recording and replay.

    参考文献
    相似文献
    引证文献
引用本文

虞圣呈,房春荣,钟葉,张犬俊,刘钦,刘嘉,郑滔,陈振宇.基于大模型语义匹配的跨平台移动应用测试脚本录制回放.软件学报,,():1-24

复制
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-08-30
  • 最后修改日期:2024-11-19
  • 录用日期:
  • 在线发布日期: 2025-08-27
  • 出版日期:
文章二维码
您是第位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号