Abstract:Distributed systems are the pillars of the current computing ecosystem, which make modern computing more powerful, reliable, and flexible, covering several key fields from cloud computing and big data processing to the Internet of Things. However, due to the complexity of the system, some code defects are inevitably introduced during the code implementation of distributed systems, thus posing a huge threat to the availability, robustness, and security of the system. Therefore, the testing and defect detection work of distributed systems is very important. Dynamic testing technology conducts real-time analysis during the system operation to detect its defects and evaluate its behavior and functions, and is widely used in the defect detection of various system applications and has successfully found many code defects. A four-layer defect threat model of distributed systems is proposed in this study. Based on it, the testing requirements and main challenges of distributed systems are analyzed, and a general framework for dynamic testing of distributed systems is proposed. Then, typical dynamic testing tools for distributed systems are introduced from the perspective of detecting different types of system defects. Next, the study highlights critical techniques such as multidimensional test input generation, system-critical state awareness, and defect judgment criteria. Additionally, the paper reviews popular dynamic testing tools and evaluates their effectiveness in defect discovery and test coverage. The coverage and defect discovery capabilities of the current mainstream dynamic testing tools for distributed systems are evaluated. The findings show that multidimensional input generation significantly enhances testing efficiency. Finally, the study discusses emerging trends and future directions in dynamic testing of distributed systems, aiming to address their inherent challenges and improve testing outcomes.

Abstract:The critical reliability and availability of distributed systems are threatened by crash recovery bugs caused by incorrect crash recovery mechanisms and their implementations. The detection of crash recovery bugs, however, can be extremely challenging since these bugs only manifest themselves when a node crashes under special timing conditions. This study presents a novel approach Deminer to automatically detect crash recovery bugs in distributed systems. Observations in the large-scale distributed systems show that node crashes that interrupt the execution of related I/O write operations, which store a piece of data (i.e., common data) in different places, e.g., different storage paths or nodes, are more likely to trigger crash recovery bugs. Therefore, Deminer detects crash recovery bugs by automatically identifying and injecting such error-prone node crashes under the usage guidance of common data. Deminer first tracks the usage of critical data in a correct run. Then, it identifies I/O write operation pairs that use the common data and predicts error-prone injection points of a node crash on the basis of the execution trace. Finally, Deminer tests the predicted injection points of the node crash and checks failure symptoms to expose and confirm crash recovery bugs. A prototype of Deminer is implemented and evaluated on the latest versions of four widely used distributed systems, i.e., ZooKeeper, HBase, YARN, and HDFS. The experimental results show that Deminer is effective in finding crash recovery bugs. Deminer has detected six crash recovery bugs.

Abstract:The software fault injection testing (SFIT) technique has been developed for thirty years. It is one of the most active parts in software testing research. As a non-traditional testing technique, it plays a very important role in enhancing software quality, eliminating software failures and improving the process of software development. A detailed review of the research on SFIT is presented based on the survey and classification of the current SFIT techniques. Then, some important testing frameworks and tools that are effective at present are also discussed. Meanwhile, a brief description of the testing system CSTS (Component Security Testing System) is provided as well. Based on the precise investigation on SFIT, the issues and challenges of SFIT are pointed out and the future development trend for SFIT is proposed.

Abstract:Fault-injection, as a perfected technique for software testing, now has been put into practical use. However, only a very few of the documents have expounded on how to use fault-injection technique in soft ware black-box testing. A new approach to the difficulty, i.e., peripherally in jecting faults in the software is presented in this paper. By means of a fault- imbedded environmental simulation, faults are injected in the input level of the software under test. These faults may induce inherent failure mode, thus bringi ng about unexpected output--consequently, the anticipated goal of the test is attained. This method has been used for testing railway signaling control system software and desirable results have been achieved.