Scenario Model Based Testing of Integrated DDS-based Naval Mission Systems
Author:
Affiliation:

Clc Number:

TP311

  • Article
  • | |
  • Metrics
  • |
  • Reference [41]
  • |
  • Related [20]
  • | | |
  • Comments
    Abstract:

    Modern integrated naval mission systems (NMS) built on data-distribution service (DDS) have special characteristics in development, structure, and application which, in combination, make their testing challenging. Model-based testing (MBT) is considered a promising technique for testing such systems. However, for NMS-like systems under test, due to their high complexity and cooperative ways of development, traditional MBT techniques requiring a complete model of the system internals are difficult to be used. This paper presents a scenario-based MBT approach for NMS-like systems. The approach builds scenario models to express the interaction scenarios in a DDS-based system from the external perspective. A scenario model uses an extended form of regular expression to model interaction sequences and uses basic data element restrictions (e.g., ranges and enumerations), constraints, and calculation functions to model interaction data. It can express the interaction processes in an abstract, convenient, and relatively comprehensive way. On the models, algorithms are proposed to generate directly executable test cases for testing. Experiments on a real NMS show that the approach can be used to test 21 kinds of common risky scenarios identified from historical failures reported during the development of a family of NMS. This indicates that the approach might be helpful for testing NMS-like DDS-based industrial systems.

    Reference
    [1] 王达, 左艳军, 郭俊. 美国海军新一代水面舰艇作战系统体系架构. 指挥控制与仿真, 2018, 40(1): 132–140.
    Wang D, Zuo YJ, Guo J. New generation surface warship combat system of the US navy system architecture. Command Control & Simulation, 2018, 40(1): 132–140.
    [2] 宋敏, 韦正现, 印桂生. 面向数据流的网构软件服务动态演化分析. 软件学报, 2013, 24(12): 2797–2813. http://www.jos.org.cn/1000-9825/4396.htm
    Song M, Wei ZX, Yin GS. Evolution analysis of data flow oriented internetware service. Ruan Jian Xue Bao/Journal of Software, 2013,24(12):2797–2813 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4396.htm
    [3] 陶传奇, 李必信, Gao J. 构件软件的回归测试复杂性度量. 软件学报, 2015, 26(12): 3043–3061. http://www.jos.org.cn/1000-9825/4876.htm
    Tao CQ, Li BX, Gao J. Complexity metrics for regression testing of component-based software. Ruan Jian Xue Bao/Journal of Software, 2015, 26(12): 3043–3061 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4876.htm
    [4] Piel EAB, González A, Gross HG. Automating integration testing of large scale publish/subscribe systems. Principles and Applications of Distributed Event-based Systems. 2010. 140–163.
    [5] Köksal Ö, Tekinerdogan B. Obstacles in data distribution service middleware: A systematic review. Future Generation Computer Systems, 2017, 68: 191–210.
    [6] OMG. A data distribution service for real-time systems. 2015. https://www.omg.org/spec/DDS/1.4
    [7] Anand S, Burke EK, Chen TY, Clark J, Cohen MB, Grieskamp W, Harman M, Harrold MJ, McMinn P. An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software, 2013, 86(8): 1978–2001. [doi: 10.1016/j.jss.2013.02.061]
    [8] Utting M, Pretschner A, Legeard B. A taxonomy of model-based testing approaches. Software Testing, Verification and Reliability, 2012, 22(5): 297–312. [doi: 10.1002/stvr.456]
    [9] Gurbuz HG, Tekinerdogan B. Model-based testing for software safety: A systematic mapping study. Software Quality Journal, 2018, 26(4): 1327–1372. [doi: 10.1007/s11219-017-9386-2]
    [10] Li YS, Pierce BC, Zdancewic S. Model-based testing of networked applications. In: Proc. of the 30th ACM SIGSOFT Int’l Symp. on Software Testing and Analysis (ISSTA). Virtual: ACM, 2021. 529–539.
    [11] Nebut C, Fleurey F, Le Traon Y, Jezequel JM. Automatic test generation: A use case driven approach. IEEE Transactions on Software Engineering, 2006, 32(3): 140–155. [doi: 10.1109/TSE.2006.22]
    [12] Dadeau F, Castillos KC, Tissot R. Scenario-based testing using symbolic animation of B models. Software Testing, Verification and Reliability, 2012, 22(6): 407–434. [doi: 10.1002/stvr.1467]
    [13] Kesserwan N, Dssouli R, Bentahar J, Stepien B, Labrèche P. From use case maps to executable test procedures: A scenario-based approach. Software & Systems Modeling, 2019, 18(2): 1543–1570. [doi: 10.1007/s10270-017-0620-y]
    [14] Bandyopadhyay A, Ghosh S. Test input generation using UML sequence and state machines models. In: Proc. of the 2009 Int’l Conf. on Software Testing Verification and Validation (ICST). Denver: IEEE, 2009. 121–130.
    [15] Rocha M, Simão A, Sousa T. Model-based test case generation from UML sequence diagrams using extended finite state machines. Software Quality Journal, 2021, 29(3): 597–627. [doi: 10.1007/s11219-020-09531-0]
    [16] Dan HT, Hierons RM. Conformance testing from message sequence charts. In: Proc. of the 4th IEEE Int’l Conf. on Software Testing, Verification and Validation (ICST). Berlin: IEEE, 2011. 279–288.
    [17] Minhas NM, Masood S, Petersen K, Nadeem A. A systematic mapping of test case generation techniques using UML interaction diagrams. Journal of Software: Evolution and Process, 2020, 32(6): e2235. [doi: 10.1002/smr.2235]
    [18] Wang LZ, Yuan JS, Yu XF, Hu J, Li XD, Zheng GL. Generating test cases from UML activity diagram based on gray-box method. In: Proc. of the 11th Asia-Pacific Software Engineering Conf. Busan: IEEE, 2004. 284–291.
    [19] Grieskamp W. Multi-paradigmatic model-based testing. In: Proc. of the 1st Combined Int’l Workshops on Formal Approaches to Software Testing and Runtime Verification. Seattle: Springer, 2006. 1–19.
    [20] Masson PA, Julliand J, Plessis JC, Jaffuel E, Debois G. Automatic generation of model based tests for a class of security properties. In: Proc. of the 3rd Int’l Workshop on Advances in Model-based Testing. London: ACM, 2007. 12–22.
    [21] Larsen PG, Lausdahl K, Battle N. Combinatorial testing for VDM. In: Proc. of the 8th Int’l Conf. on Software Engineering and Formal Methods (SEFM). Pisa: IEEE, 2010. 278–285.
    [22] Ledru Y, du Bousquet L, Maury O, Bontron P. Filtering TOBIAS combinatorial test suites. In: Proc. of the 7th Int’l Conf. on Fundamental Approaches to Software Engineering (FASE). Barcelona: Springer, 2004. 281–294.
    [23] Polo M, Pedreira O, Places ÁS, de Guzmán IGR. Automated generation of oracled test cases with regular expressions and combinatorial techniques. Journal of Software: Evolution and Process, 2020, 32(12): e2273. [doi: 10.1002/smr.2273]
    [24] Belli F, Budnik CJ, Hollmann A. A holistic approach to testing of interactive systems using statecharts. In: Proc. of the 2nd South-east European Workshop on Formal Methods.2005. 59–73.
    [25] Liu P, Miao HK. Theory of test modeling based on regular expressions. In: Proc. of the 3rd Int’l Workshop on Structured Object-oriented Formal Language and Method. Queenstown: Springer, 2013. 17–31.
    [26] RTI in aerospace and defense. https://www.rti.com/docs/RTI_for_Defense.pdf
    [27] Hoffman DM, Ly-Gagnon D, Strooper P, Wang HY. Grammar-based test generation with YouGen. Software: Practice and Experience, 2011, 41(4): 427–447. [doi: 10.1002/spe.1017]
    [28] Ali S, Iqbal MZ, Khalid M, Arcuri A. Improving the performance of OCL constraint solving with novel heuristics for logical operations: A search-based approach. Empirical Software Engineering, 2016, 21(6): 2459–2502. [doi: 10.1007/s10664-015-9392-6]
    [29] Sartaj H, Iqbal MZ, Khan MU. Testing cockpit display systems of aircraft using a model-based approach. Software and Systems Modeling, 2021, 20(6): 1977–2002. [doi: 10.1007/s10270-020-00844-z]
    [30] Dadeau F, Fourneret E, Bouchelaghem A. Temporal property patterns for model-based testing from UML/OCL. Software & Systems Modeling, 2019, 18(2): 865–888. [doi: 10.1007/s10270-017-0635-4]
    [31] Kanso B, Taha S. Specification of temporal properties with OCL. Science of Computer Programming, 2014, 96: 527–551. [doi: 10.1016/j.scico.2014.02.029]
    [32] Michlmayr A, Fenkam P, Dustdar S. Architecting a testing framework for publish/subscribe applications. In: Proc. of the 30th Annual Int’l Computer Software and Applications Conference (COMPSAC’06). Chicaco: IEEE, 2006. 467–474.
    [33] Cotroneo D, Natella R, Russo S, Scippacercola F. State-driven testing of distributed systems. In: Proc. of the 17th Int’l Conf. on Principles of Distributed Systems. Nice: Springer, 2013. 114–128.
    [34] Grace P, Barbosa J, Pickering B, Surridge M. Taming the interoperability challenges of complex IoT systems. In: Proc. of the 1st ACM Workshop on Middleware for Context-aware Applications in the IoT. Bordeaux: ACM, 2014. 1–6.
    [35] Bozkurt M, Harman M, Hassoun Y. Testing and verification in service-oriented architecture: A survey. Software Testing, Verification and Reliability, 2013, 23(4): 261–313. [doi: 10.1002/stvr.1470]
    [36] de Moura L, Bjørner N. Z3: An efficient SMT solver. In: Proc. of the 14th Int’l Conf. on Tools and Algorithms for the Construction and Analysis of Systems. Budapest: Springer, 2008. 337–340.
    [37] DDSTest. https://github.com/juqian/ddstest
    [38] automaton. http://www.brics.dk/automaton/
    Cited by
Get Citation

钱巨,王寅,程浩,韦正现.基于场景模型的DDS架构一体化舰船任务系统测试.软件学报,2022,33(5):1711-1735

Copy
Share
Article Metrics
  • Abstract:900
  • PDF: 4273
  • HTML: 2818
  • Cited by: 0
History
  • Received:August 10,2021
  • Revised:October 09,2021
  • Online: January 28,2022
  • Published: May 06,2022
You are the first2036651Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-4
Address:4# South Fourth Street, Zhong Guan Cun, Beijing 100190,Postal Code:100190
Phone:010-62562563 Fax:010-62562533 Email:jos@iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063