Results of Manual Inspection of the Ground Truth in the Predictive Accuracy Study of Dynamic Impact Analysis
change location (statement no.) | 3582 |
query (method) | net.n3.nanoxml.XMLElement: boolean isLeaf() |
tests | all covering tests: 74, 75, 76 randomly selected: 74, 75, 76 |
PI/EAS results | <net.n3.nanoxml.XMLElement: boolean isLeaf()> <CheckLeaf1_wy_v1: void main(java.lang.String[])> <AllDriversMain: void main(java.lang.String[])> <CheckLeaf2_wy_v1: void main(java.lang.String[])> <CheckLeaf3_wy_v1: void main(java.lang.String[])> |
mDEA results | <net.n3.nanoxml.XMLElement: boolean isLeaf()> <CheckLeaf1_wy_v1: void main(java.lang.String[])> <CheckLeaf2_wy_v1: void main(java.lang.String[])> <CheckLeaf3_wy_v1: void main(java.lang.String[])> |
manual ground-truth | <net.n3.nanoxml.XMLElement: boolean isLeaf()> <CheckLeaf1_wy_v1: void main(java.lang.String[])> <CheckLeaf2_wy_v1: void main(java.lang.String[])> <CheckLeaf3_wy_v1: void main(java.lang.String[])> |
ground-truth conformance | Fully conformed: The changed method, although transitively returned into <AllDriversMain: void main(java.lang.String[])>, which is thus identified as impacted by PI/EAS, did not affect (any execution states in) this test driver entry method The change is for a boolean value returned by the changed method; when this value is changed, only its direct caller and itself are affected because such change did not propagate further along |
change location (statement no.) | 4068 |
query (method) | method: net.n3.nanoxml.XMLUtil: void errorWrongClosingTag(net.n3.nanoxml.IXMLReader,java.lang.String,java.lang.String) |
tests | all covering test: 128, 129, 145, 149, 153, 157, 161, 165, 169, 173, 185, 189 randomly selected: 129, 153, 185 |
PI/EAS results | <net.n3.nanoxml.StdXMLParser: java.lang.Object parse()> <Parser1_vw_v1: void main(java.lang.String[])> <net.n3.nanoxml.StdXMLParser: void processElement()> <net.n3.nanoxml.XMLUtil: void errorWrongClosingTag(net.n3.nanoxml.IXMLReader,java.lang.String,java.lang.String)> <net.n3.nanoxml.StdXMLParser: void scanData()> <AllDriversMain: void main(java.lang.String[])> <net.n3.nanoxml.StdXMLReader: int getLineNr()> <net.n3.nanoxml.StdXMLParser: void scanSomeTag(boolean)> <net.n3.nanoxml.XMLParseException: void <init>(int,java.lang.String)> |
mDEA results as ground-truth | <net.n3.nanoxml.XMLParseException: void <init>(int,java.lang.String)> <net.n3.nanoxml.XMLUtil: void errorWrongClosingTag(net.n3.nanoxml.IXMLReader,java.lang.String,java.lang.String)> |
manual ground-truth | <net.n3.nanoxml.XMLParseException: void <init>(int,java.lang.String)> <net.n3.nanoxml.XMLUtil: void errorWrongClosingTag(net.n3.nanoxml.IXMLReader,java.lang.String,java.lang.String)> |
ground-truth conformance | Fully conformed: The change involves the construction of an exception message; when this change is made, only the enclosing method is affected, plus the constructor of the exception class which takes the message as the single parameter. |
change locations (statement no.s) | 4031 1007 |
queries (methods) | <net.n3.nanoxml.XMLUtil: void errorExpectedInput(net.n3.nanoxml.IXMLReader,java.lang.String)> <Parser6_vw_v0: void main(java.lang.String[])> |
tests | all covering tests: 133,134, 136, 141 randomly selected: 133,136,141 |
PI/EAS results | <net.n3.nanoxml.StdXMLParser: java.lang.Object parse()> <net.n3.nanoxml.XMLParseException: void <init>(int,java.lang.String)> <net.n3.nanoxml.XMLUtil: void errorExpectedInput(net.n3.nanoxml.IXMLReader,java.lang.String)> <Parser1_vw_v1: void main(java.lang.String[])> <net.n3.nanoxml.StdXMLParser: void processElement()> <net.n3.nanoxml.XMLUtil: java.lang.String scanString(net.n3.nanoxml.IXMLReader,char,boolean,net.n3.nanoxml.XMLEntityResolver)> <net.n3.nanoxml.StdXMLParser: void scanData()> <AllDriversMain: void main(java.lang.String[])> <net.n3.nanoxml.StdXMLReader: int getLineNr()> <net.n3.nanoxml.StdXMLParser: void processAttribute()> <net.n3.nanoxml.StdXMLParser: void scanSomeTag(boolean)> <net.n3.nanoxml.StdXMLParser: void processSpecialTag(boolean)> <net.n3.nanoxml.StdXMLParser: void processDocType()> |
mDEA results as ground-truth | <net.n3.nanoxml.XMLParseException: void <init>(int,java.lang.String)> <net.n3.nanoxml.XMLUtil: void errorExpectedInput(net.n3.nanoxml.IXMLReader,java.lang.String)> <Parser6_vw_v0: void main(java.lang.String[])> |
manual ground-truth | <net.n3.nanoxml.XMLParseException: void <init>(int,java.lang.String)> <net.n3.nanoxml.XMLUtil: void errorExpectedInput(net.n3.nanoxml.IXMLReader,java.lang.String)> <Parser6_vw_v0: void main(java.lang.String[])> |
ground-truth conformance | Fully conformed: Only the two queried methods themselves are trivially impacted because of the changes made in them; the only other truly affected one is the exception class constructor, which takes the changed message as its sole input parameter. Because the value change in the second method did not change the predicate anyway, and the second change only affects the exception message, which was not further used by method executed after, only these three methods identified manually above were truly impacted, which is consistent with the mDEA-based ground truth used in our study. |
change location (statement no.) | 317 |
query (method) | <ScheduleClass: void upgrade_process_prio(int,float)> |
tests | all covering tests: 2647 of the total 2650 tests all covered the change randomly selected: 101, 895, 1793 |
PI/EAS results | <ScheduleClass: void finish_process()> <ScheduleClass: void block_process()> <ScheduleClass: ScheduleClass$Ele new_process(int)> <ScheduleClass: ScheduleClass$List append_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void schedule()> <ScheduleClass: ScheduleClass$Ele new_ele(int)> <ScheduleClass: void upgrade_process_prio(int,float)> <ScheduleClass: void add_process(int)> <ScheduleClass: void quantum_expire()> <ScheduleClass: void _main(java.lang.String[])> <ScheduleClass: void finish_all_processes()> <ScheduleClass: void main(java.lang.String[])> <ScheduleClass$Ele: void <init>()> <ScheduleClass: ScheduleClass$List del_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void unblock_process(float)> <ScheduleClass$List: void <init>()> <ScheduleClass: ScheduleClass$List new_list()> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> |
mDEA results as ground-truth | <ScheduleClass: void upgrade_process_prio(int,float)> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> <ScheduleClass: ScheduleClass$List del_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void finish_process()> <ScheduleClass: ScheduleClass$List append_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void schedule()> <ScheduleClass: void unblock_process(float)> <ScheduleClass: void quantum_expire()> |
manual ground-truth | <ScheduleClass: void upgrade_process_prio(int,float)> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> <ScheduleClass: ScheduleClass$List del_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: ScheduleClass$List append_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void schedule()> <ScheduleClass: void unblock_process(float)> <ScheduleClass: void finish_process()> <ScheduleClass: void quantum_expire()> |
ground-truth conformance | Fully conformed: All the methods identified by PI/EAS were executed after the queried one, among which only those listed in the manual ground truth were affected by the change because others did not use variables or predicates directly or transitively dependent on the changed variable. |
change location (statement no.) | 210 |
query (method) | <ScheduleClass: void finish_all_processes()> |
tests | all covering tests: 2433 out of the total 2650 tests covered the change randomly selected: 33, 126, 2223 |
PI/EAS results | <ScheduleClass: void finish_process()> <ScheduleClass: void unblock_process(float)> <ScheduleClass: void block_process()> <ScheduleClass: ScheduleClass$Ele new_process(int)> <ScheduleClass: ScheduleClass$List append_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void schedule()> <ScheduleClass: ScheduleClass$Ele new_ele(int)> <ScheduleClass: void upgrade_process_prio(int,float)> <ScheduleClass: void add_process(int)> <ScheduleClass: void quantum_expire()> <ScheduleClass: void _main(java.lang.String[])> <ScheduleClass: void finish_all_processes()> <ScheduleClass: void main(java.lang.String[])> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> <ScheduleClass$Ele: void <init>()> <ScheduleClass: ScheduleClass$List del_ele(ScheduleClass$List,ScheduleClass$Ele)> |
mDEA results as ground-truth | <ScheduleClass: void finish_process()> <ScheduleClass: void finish_all_processes()> <ScheduleClass: void schedule()> |
manual ground-truth | <ScheduleClass: void finish_process()> <ScheduleClass: void finish_all_processes()> <ScheduleClass: void schedule()> |
ground-truth conformance | Fully conformed: Change at this location affects the number of iterations of a loop; however, only the three methods are affected through variables changed because of the change in the number of iterations (via a global variable that these methods used). |
[multi-method change]
change locations (statement no.s) | 291 13 |
queries (methods) | <ScheduleClass: void unblock_process(float)> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> |
tests | all covering tests: 2109 out of the total 2650 tests covered the change randomly selected: 579, 1501, 2645 |
PI/EAS results | <ScheduleClass: void unblock_process(float)> <ScheduleClass: void _main(java.lang.String[])> <ScheduleClass: void main(java.lang.String[])> <ScheduleClass: void finish_process()> <ScheduleClass: void block_process()> <ScheduleClass: ScheduleClass$Ele new_process(int)> <ScheduleClass: ScheduleClass$List append_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass$List: void <init>()> <ScheduleClass: ScheduleClass$List new_list()> <ScheduleClass: void schedule()> <ScheduleClass: ScheduleClass$Ele new_ele(int)> <ScheduleClass: void add_process(int)> <ScheduleClass$Ele: void <init>()> <ScheduleClass: ScheduleClass$List del_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void upgrade_process_prio(int,float)> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> <ScheduleClass: void finish_all_processes()> |
mDEA results as ground-truth | <ScheduleClass$Ele: void <init>()> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> <ScheduleClass: ScheduleClass$Ele new_ele(int)> <ScheduleClass: ScheduleClass$Ele new_process(int)> <ScheduleClass: ScheduleClass$List append_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: ScheduleClass$List del_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void _main(java.lang.String[])> <ScheduleClass: void upgrade_process_prio(int,float)> <ScheduleClass: void unblock_process(float)> <ScheduleClass: void finish_all_processes()> <ScheduleClass: void block_process()> <ScheduleClass: void add_process(int)> <ScheduleClass: void finish_process()> <ScheduleClass: void schedule()> |
manual ground-truth | <ScheduleClass: void unblock_process(float)> <ScheduleClass: ScheduleClass$Ele find_nth(ScheduleClass$List,int)> <ScheduleClass: ScheduleClass$List del_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: ScheduleClass$List append_ele(ScheduleClass$List,ScheduleClass$Ele)> <ScheduleClass: void upgrade_process_prio(int,float)> <ScheduleClass: void finish_process()> <ScheduleClass: void schedule()> <ScheduleClass: void add_process(int)> <ScheduleClass: ScheduleClass$Ele new_process(int)> <ScheduleClass: ScheduleClass$Ele new_ele(int)> <ScheduleClass: void finish_all_processes()> <ScheduleClass: void block_process()> <ScheduleClass: void unblock_process(float)> <ScheduleClass: void _main(java.lang.String[])> |
ground-truth conformance | Fully conformed: The three false-positive impacts found by PI/EAS include two initializers that used constants to initialize some global variables, thus they were not affected by the two changes; the other one is the entry method, which was finally returned into, yet it was not really affected by the changes because it does not use any of the two changes directly or transitively. |