Abstract:Interactive systems with dynamic tasks have been widely used recently. In this study, a computational interaction model ICOM^DT for dynamic tasks is proposed based on the existing research, which is used to describe interactive processes and predict user's intention. More specifically, ICOM^DT is applied to moving target selection, and two experiments are designed to verify the validity of the model. In the first experiment, user data is collected to fit the model and predict error rates. Results show that the proposed model fits the empirical data well and the predicted value is also close to the true value. In the second experiment, an assistant moving target selection technique ICOMPointer is achieved. By comparing with the basic selection technique and the other two state-of-the-art selection techniques in a game, it is found that ICOMPointer performs well. The ICOM^DT model is of great significance for computer to understand user's intentions and improve the efficiency of interaction between users and dynamic tasks.

Abstract:Mid-air gestures have been largely overlooked for transferring contents among multiple screens. For verification, a cross-screen interfaces sharing system is designed, which enables users to move interface content across screens by gestures in real time. According to the user-designed gesture survey, two mid-air gestures, "grab-drag" and "grab-pull-drop", are implemented. User experiment shows that they are novel, useful, and easy to learn. In general, "grab-drag" provides better user experience, while "grab-pull-drop" is more versatile in complex scenarios. The former gesture is more suitable when the two screens are adjacent and the operation distance is short too; while the latter suits are better in away-from-screen scenarios, when there is a third screen among target screens or screens are widely spaced. Therefore, a combination of the two gestures will be a rational solution.

Abstract:Fatigue driving is one of the main causes of traffic accidents. It is important social significance to accurately and effectively detect and prevent the drivers' fatigue driving. Based on the research and comparison of previous work, this study designs a driver fatigue detection mechanism based on machine vision and image processing. First, the continuous frame image (video) is used to perform face detection using AdaBoost algorithm, and the approximate human eye area is segmented according to the distribution features of the human face "three courts and five holes". In the process of human eye positioning, the OSTU threshold segmentation, nonlinear point operations, and integral projections are used to eliminate eyebrows, and three influence factors, namely the fuzzy comprehensive evaluation algorithm for the ratio of the length to the width of the rectangular area of the eye, fitting the area of the ellipse, and the proportion of pupil melanin are analyzed to determine the open or closed state of the eye. Finally, according to the PERCLOS principle, the fatigue state of the driver is detected. The experimental results show that the proposed algorithm can accurately distinguish the open or closed state of the eyes, thus detect the driver's fatigue state with higher accuracy and practicability.

Abstract:Quanzhou puppet is one of the intangible cultural heritages of China. It is the physical embodiment of traditional Chinese culture. However, the large size of the puppet and inconvenience to carry and manipulate directly makes it hard to reach a wider audience. In order to realize the effective inheritance and protection of Quanzhou puppet, this study designs a virtual real-line puppet animation scheme based on gesture recognition, builds a prototype system which uses MYO Armband EMG signal to control the generation of animation, and applies it in user experiment to verify the high accuracy and easy manipulation of the algorithm. Firstly, low-pass filtering and smoothing is used to process the original multi-channel EMG data. Secondly, after eight-channel EMG signal time-domain feature and time-frequency-domain feature extraction, the dimension of the feature vector is reduced to six by linear discriminator to eliminate the correlation between features and enhance the robustness of the algorithm. Thirdly, a multi-class support vector machine is constructed which uses feature vector to determine the result of gesture recognition. Experiments show that the average recognition accuracy of offline action is 95.59%, the average recognition accuracy of real-time action is 90.75%, and the gesture recognition is completed within 1.1 s. For the puppet task, two users task is designed:the common users and the expert users. In the common user study, the gestures recognition accuracy is high. In the aspects of user's willingness to use and easiness to learn, the performance of this system is significantly higher than real puppets manipulation. In the expert user study, user's acceptance and usability of the system are also highly evaluated. These two user tasks indicate the system meets the requirements of real-time and accuracy, and has good interactivity and interesting. Relevant research can be widely applied to similar systems, such as computer animation. It has practical significance for experiencing and protecting the puppet.

Abstract:Sterile and non-contact environment is the basic requirement of medical operating room, which makes the computer room and operating room need to be physically isolated. At the same time, if the attending doctor needs to look at the image of the lesion during the operation, he usually instructs the nurse or the assistant to operate the image of the lesion in the computer operating room, because of the isolation between the operating room and the computer room, and because the intention between the attending doctor and the assistant may not be understood accurately, it is easy to lead the nurse or surgical assistant in the operating room and computer room to and fro many times, which increases the risk of prolonged operation time, increased blood loss, and organ exposure time, minimizing the time to locate the lesion image in the operation is important for doctors and patients. To meet the above requirements, a non-contact multi-channel natural interactive surgical environment under aseptic conditions is constructed by means of human skeleton extraction, gesture tracking and understanding, far-field speech recognition in operating room environment, multi-modal information processing, and fusion technology. This environment allows the attending physician to quickly locate the lesion to be observed during surgery by combining voice commands, gestures, and the above interaction. In the experimental environment close to the real environment, the non-contact multi-channel natural interactive surgical environment established in this study can significantly reduce the localization time of the lesion image under the condition of ensuring the accuracy. Intelligent interactive operating room in aseptic environment provides technical and methodological validation for the next generation of efficient surgery.

Abstract:The electroencephalograph (EEG) driven brain-computer interaction can promote daily life and rehabilitation training for physically disabled people, however, EEG has several problems such as low signal-noise ratio, significant individual difference, and these problems result in the low accuracy and efficiency for EEG feature extraction and classification. In the context of reducing numbers of electrodes and increasing identified classes, this study proposed an approach to classify motor imagery (MI) EEG signal based on convolutional neural network (CNN). Firstly, based on existed approaches, experiments were conducted and the CNN was constructed with three convolution layers, three pooling layers, and two full-connection layers. Secondly, MI experiment was conducted with the imagination of left hand movement, right hand movement, foot movement, and resting state, and the MI EEG data were collected at the same time. Thirdly, the MI EEG data set were used to build the classification model based on CNN, and the experiment results indicate that the average accuracy of classification is 82.81%, which is higher than the related classification algorithms. Finally, the classification model was applied in the online classification of MI EEG, and a BCI prototype system was designed and implemented to drive the real-time human-robot interaction. The prototype system can help users to control motion states of the humanoid robot, such as raising hands, moving forward. Furthermore, the experimental results show that the average accuracy of robot controlling reaches to 80.31%, and it verifies the proposed approach not only can classify MI EEG data with high accuracy in real time, but also promote applications of human-robot interaction with BCI.

Abstract:Bitcoin transaction network analysis helps people understand the trading patterns in bitcoin transactions. Bitcoin trading network is anonymous and large-scale making it difficult for users to gain insight from the entire trading network. A visual exploration approach for bitcoin trading network is proposed based on topological structure recommendation to support the interactive exploration in Bitcoin trading network. The key idea of the proposed approach is to generate a vectorized representation for each node, and consequently a set of similar structures can be easily found upon user intention. The case study demonstrates the effectiveness of the approach in supporting exploration and analysis of trading patterns of Bitcoin transactions.

Abstract:This paper presents a markerless synchronized motion capture method for body and face. A long-focus color camera and a standard Kinect camera are used which are synthesized and calibrated. Flush is added into the capture space to make the cameras synthesized. Then, the cameras are calibrated to get the relative transformation of the cameras by Zhang's calibration, In this way, the hybrid camera is configured. The user's body is scanned by Kinect fusion, and the skeleton is embedded for the body model. During capture, the translation reference is firstly obtained from the depth camera. After that, the facial pose and expression and identity are reconstructed by 2D feature points. Non-rigid ICP is applied to reconstruct the body pose. Finally, the head pose from the face capture camera is transformed to body capture camera to get the merged results. The comparison and user study show that the result of proposed synchronized motion capture by two cameras is better than that of single camera.

Abstract:With the development of digital image processing technology and computer supported cooperative work, eye tracking has been applied in the process of multiuser collaborative interaction. However, existed eye tracking technique can only track single user's gaze, and the computing framework for multiple user's gaze data tracking is not mature; besides, the calibration process is much complex, and the eye tracking data recording, transition, and visualization mechanisms need to be further explored. Hence, this study proposed a new collaborative calibration method based on gradient optimization algorithms, so as to simplify the calibration process; and then in order to optimize the eye tracking data transition and management, the computing framework oriented to multiple user's eye tracking is proposed. Furthermore, to explore the influence of visual attention caused by visualization of eye tracking data sharing among multiple users, visualizations such as dots, clusters and trajectories are designed, and it is validated that the dots could improve the efficiency for collaborative visual search tasks. Finally, the code collaborative review systems are designed and built based on eye tracking, and this system could record, deliver, and visualize the eye tracking data in the forms of dots, code borders, code background, lines connected codes, among the code reviewing process. The user experiment result shows that, compared to the no eye tracking data sharing condition, sharing eye tracking data among multiple users can reduce the bug searching time with 20.1%, significantly improves the efficiency of collaborative work, and it validates the effectiveness of the proposed approach.

Abstract:As a revolution in software engineering, there are many reasons for the rapid development of DevOps in the past ten years. This study focuses on the practical impact of automation tools in the Chinese DevOps practice and a series of problems arising from automation tools. Systematic Literature Review (SLR) is used to identify the most popular tools, and finally 69 automation tools are identified from 50 researches, including Docker, Chef, Jenkins, and Puppet. Three levels of problems of automation tools in DevOps are summarized from some Chinese blogs using Gray Literature Review (GRL). Finally, ethnographic interview is used to analyze the opinions and suggestions from three aspects of DevOps practice in China, obtaining two effects of the automation tools:1) the role of automation tools in the DevOps practice is obvious at the beginning, and DevOps practice is considered using automation tools; 2) software organizations need to reduce the dependence on automation tools and form their own culture of DevOps. To solve the problems of automation tools in Chinese DevOps practice, this paper summarizes three suggestions from the interview and gives a paradigm.

Abstract:Software testing is a common way to guarantee software quality. How to achieve high coverage is a very important and challenging goal in testing. Fuzz testing and symbolic execution, as two mainstream testing techniques, have been widely studied and applied to academia and industry, both technologies have certain advantages and limitations. Fuzz testing can execute and cover deeper branches by randomly mutating test cases and dynamically executing programs. However, it is difficult to generate test cases that can cover complex conditional branches by random mutation. Symbolic execution can cover complex conditional branches with SMT solvers, but it is difficult to cover deeper branches due to state explosion during symbolic execution. Current works have shown that hybrid testing involving fuzzing and symbolic execution can archive better performance than fuzzing or symbolic execution. By analyzing the advantages and disadvantages in fuzzing and symbolic execution, this study proposes a branch coverage-based hybrid testing approach that combines the two methods with each other to achieve better test cases with high branch coverage. Specifically, fuzz testing (e.g., AFL) quickly generates a large number of test cases that can cover deeper branches, and symbolic execution (e.g., KLEE) performs a search based on the coverage of fuzz testing, and generating test cases for uncovered branches. To evaluate the effectiveness of Afleer, the study selects the standard benchmark LAVA-M and one real project oSIP as the evaluation object, and uses bug detection and coverage as the evaluation measures. The experimental results show that:1) For bug discovery, Afleer found 755 bugs while AFL only found 1; 2) For coverage, Afleer achieved some improvement on benchmarks and real project. In the project oSIP, Afleer increases the branch coverage by 2.4 times and the path coverage by 6.1 times. In addition, Afleer found a new bug in oSIP.

Abstract:With the improvement of the scale and complexity of software, software quality problems become the focus of attention. Software defect is the opposite of software quality, threatening the software quality. How to dig up defect modules in the early stages of software development has become a urgent problem that needs to be solved. Software defect prediction (SDP) designs the internal metrics related defects by mining software history repositories, and then in advance finds and locks the defect modules with the aid of machine learning methods, so as to allocate the limited resources reasonably. Therefore, SDP is one of the important ways of software quality assurance (SQA), which has become a very important research subject in software engineering in recent years. Based on the form of defect perfection, this research offers a systematic analysis of the existing research achievements of the domestic and foreign researchers in recent eight years (2010~2017). First, the research framework of SDP is given.Then the existing research achievements are classified and compared from three aspects, including datasets of SDP, the methods models and the evaluation indicators. Finally, the possible research directions are pointed out.

Abstract:Non-functional requirements state quality related expectations to a system, is used as important criteria for software design decision making and design evaluation. Comparing to functional requirements descriptions, non-functional requirements statements are often scattered and implicit, which may require much effort to group and interpret manually. This study proposes an automated non-functional requirements recognition and classification approach, which identifies the sentences that are likely to include information about non-functional requirements, to save manual efforts. Then, given non-functional requirements statement is classified into five possible non-functional requirements types, including:performance, reliability, usability, security, and maintainability. The training of classification model and experiment are based on a requirements dataset from the past projects, and then the proposed approach is validated with an industrial project requirements data.

Abstract:With the extensive implementation of DevOps in major software enterprises, the version delivery and deployment of system software class products has been accelerated. In the process of implementation, it is found that the important part of the product, the product document, is still developed by traditional R&D method and lack of supporting processes and tools, which leads to the delay in the product document delivery comparing with the delivery of the software version, and which seriously affected the complete and just-in-time delivery of the product. The "Document DevOps" introduced in this paper is a continuous product document delivery mode. By analyzing the document delivery solutions of open source and DITA, and combining with the R&D characteristics of system software, a set of comprehensive document delivery solutions that is fit for system software has been summarized. With the concept of DevOps, and based on the DevOps tool chain of the industry, it has contracted a "Document DevOps Platform" (hereinafter referred to as the platform), to achieve a user-oriented document delivery solution. Its typical characteristics are:integration with software iteration process, information unit homologous citation, management of multi-format content sources, and protection of continuous integration quality. Zhongxing has achieved the simultaneous delivery of product documents and software versions by the "Document DevOps", which greatly improves the accuracy, integrity, and delivery efficiency of documents. The iDoc platform has been successfully applied in more than 50 software products. The problem solved by the "Document DevOps" is universal, helping the other system software to achieve agile value delivery in a wider range. Moreover, the "Document DevOps" is a supplement to DevOps, for it extends the application scope of DevOps:the delivery object covers product documentation, and the process extends to market planning, covering a broader range of collaborative personnel.

Abstract:Under the widespread of smartphones and tablets, Android devices have gradually become one of the most important elements in our daily life. Along with it, Android applications are now flourishing and their complexity increases geometrically. Meanwhile, Android fragmentation is aggravating, which forces developers to design and develop the same Android apps for different Android versions and devices. In this case, employing models are proposed for requirements and designs in Android app development. With models, dividing and conquering these requirements and designs are possible which reduces the general complexity. At the same time, models of high expression help developers to better understand the purpose and finally guide the development work. However, the traditional models are no longer suitable, since Android apps are even-driven and GUI centric. Therefore, Interaction Flow Modeling Language (IFML) is adopted, the new OMG standard for front-end design and event interaction, in Android app modeling to describe apps' GUI structures and workflows and guide the development. Furthermore, an extension of IFML is proposed for Android to improve its usability and compatibility for Android apps. A formal definition of the IFML model is also given in this paper. The rich semantics of IFML models can elaborate the designs for Android apps, which will further systematically guide the development of these apps during their evolution. Moreover, these IFML models are used to check the consistency between design and implementation in the form of testing. In this way, the effort of writing test cases is reduced and productivity is enhanced as the apps evolve. A tool for modeling and testing for Android apps with IFML is presented, called ADAMANT. To verify the proposed approach's feasibility, ADAMANT is applied on five real-world apps. The results show that the use of the extended IFML in Android app development is effective, and the IFML models can directly be used for testing, ensuring the design is in consistent with the implementation. In this manner, it ensures the quality of development and benefits the sustainable development of apps.

Abstract:With the popularity of App software applications, the number of user's comments for App software has increased dramatically. Mining valuable software usage feedback based on user's comments can help developers to maintain and improve App software pertinently. Aimed at different types of usage feedback for App software, this study proposes the extracting rules of evaluation object and evaluation opinion. Moreover, the comment modes and comment seeds are defined. User's comments that are same or similar to comment seeds reflecting usage feedback are mined. Based on the initial comment seeds labeled manually, a candidate comment mode library is built continuously. A semi-supervised learning method is used to dynamically expand the comment seed library based on the candidate comment mode library. The scope of mining user's comments reflecting usage feedback is expanded by interactive mining process. Finally, the experimental results show that the proposed method can effectively mine App software user's comments reflecting usage feedback with an average mining rate of 77.82%.

Abstract:User stories are widely used in agile development projects. Every user story tells what the user/customer wants the system to do. However, a user story can only contain a small piece of the requirements but not the whole business logic. That means that when the customers submit user stories, the developers need to combine them together according to the relationships among them for producing or updating the system requirements. That is very tedious, time-consuming, and error-prone. This study proposes a human-machine collaborative approach to support the user story understanding and system functional requirements generation. This approach proposes to include the scenarios as the fine-grained representation of user stories and presents a feature-scenario model to capture the elements of user stories in three dimensions, i.e., the story description, the function attributes, and the scenarios. It designs a three-step algorithm to accept the submission of user stories, extract the features of each user story, and construct its functional scenario. As there are relationships among different user stories, it defines three types of correlative relations among them based on the functional scenarios. With the help of the customers, it supports the measurement and the identification of these relations and then constructs the system's view of the functional requirements. It is also applicable when obtaining new user stories for tolerating the requirements evolution. A case study shows the feasibility of this approach.

Abstract:DevOps has been proposed for nearly a decade. As an extension of the agile approach from development to the entire software lifecycle, DevOps aims to break the barrier between development and operation and reshape the software process from the aspects of culture, automation, standardization, architecture, tool support, etc. The process is to achieve the shortest possible cycle from code submission to product release, while guaranteeing high quality. In an increasingly competitive market environment, users are increasingly demanding the stability of product services and the frequency and efficiency of updates. Therefore, DevOps has attracted a large number of researchers and practitioners. Puppet Labs began a global survey of the state of DevOps in 2013 and has published five reports so far. The development of DevOps in China is relatively lagging behind. The research on the state of DevOps in China is in a blank in both industry and academia. Two questionnaire surveys are conducted on the state of DevOps in China in 2016 and 2018 to fill this gap. The numbers of responses received in the two surveys were 74 and 66 respectively. Based on the results of the two surveys, this study performed a comprehensively analysis from eight aspects, including IT performance, organizational culture and related practices, development and operation and maintenance practices, tool support, leadership, work ratio, employee engagement, and satisfaction. The development status and trends of DevOps in China with the global status reported by Puppet Labs are compared. Overall, DevOps has shown a steady development trend in China, but there is still a significant gap compared with the international level. At present, only 6% of the respondents could achieve the international high-level IT performance. 17 findings from the comparison are summarized. Then, three main conclusions are obtained through the synthesis of these findings:(1) The quality of employees and the shortage of talents are the crux of lagging behind and the process maturities of most of organizations are not high; (2) The more mature DevOps leads to the higher degree of the dedication and satisfaction of employees; (3) Scrum agile development and trunk-based development are the most commonly applied practices. Based on the analysis results, several recommendations are provided for practice and research.

Abstract:During recent years, DevOps gains its popularity. As the support of DevOps, microservices architecture has become a hot spot of software industry for its agility, flexibility, and scalability. However, finding the adequate granularity of microservices is a big challenge. Microservices architecture still lacks the criteria for evaluating its granularity. To cope with this problem, this paper designs four evaluation metrics based on several principles of best microservices practice to quantitatively measure the rationality of microservice decomposition. Based on that, a granularity evaluation approach of microservices based on bounded context is proposed. By implementing a tool prototype, evaluation results can be automatically calculated. A case study is used to evaluate the decomposition of a microservices-oriented system and compare the results with the architects' expectations. The results prove that the proposed approach can be applied to evaluate the granularity of microservices-oriented system.