Identifying Defects Related to the Order in which Messages are Received in Message-Passing Systems

Abstract

Improving the quality of software artifacts and products is an essential activity for everyone working on the development of software. Testing is one approach to reveal defects and faults in software. In recent years, message-passing systems have grown to a significant degree due to the rise of distributed systems, embedded systems, and so forth. In message-passing systems, components communicate with each other through sending and receiving messages. This message-passing mechanism introduces new opportunities for testing programs due to the fact that the time a message is delivered is not guaranteed, so the order in which messages are delivered is also not guaranteed. This non-determinism introduces interleaving and parallelization and subsequently a new source of software defects like race conditions. In this thesis, we have explained a new approach to testing a given component for identifying software faults related to the order in which messages are received by that component. We reorder messages coming to a certain component and deliver them in a different distinct ordering each time. We have three different methods for achieving message reordering: Blocking, Buffering, and Adaptive Buffering. We evaluate the effectiveness of our new testing methods using four metrics: Ordering Coverage, Coverage Rate, Slowdown Overhead, and Memory Overhead. We have implemented our Reordering Framework on QNX Neutrino 6.5.0 and compared our reordering methods with each other and with the naive random case using our experiments. We have also showed that our testing approach applies to real programs and can reveal real bugs in software.