|
1 evalid_test_notes.txt |
|
2 |
|
3 testfiles.zip contains two matching trees, each divided into |
|
4 three sub-trees: |
|
5 |
|
6 left\ok\... |
|
7 left\missing\... |
|
8 left\fail\... |
|
9 |
|
10 right\ok\... |
|
11 right\missing\... |
|
12 right\fail\... |
|
13 |
|
14 Running "evalid left\ok right\ok" should produce 100% OK. |
|
15 The "ok" trees contain at least one example of each type of file, |
|
16 and should grow to illustrate any new form of non-significant |
|
17 difference. |
|
18 |
|
19 Running "evalid left\missing right\missing" should report |
|
20 only failures in the shape of the tree. |
|
21 |
|
22 Running "evalid left\fail right\fail" should report differences |
|
23 in all pairs of files. |
|
24 The "fail" trees should grow whenever a new form of significant |
|
25 difference is found, so that we can avoid making EVALID too |
|
26 tolerant. |
|
27 |
|
28 |
|
29 ------------- |
|
30 Significant Difference test cases |
|
31 |
|
32 Most examples are synthetic test cases obtained by copying one or more pairs from |
|
33 the ok tree into the fail tree, then modifying the right\fail file to introduce |
|
34 a specific type of difference. The files are usually renamed to indicate the |
|
35 nature of the difference. Using the pair of files from the ok tree means that the |
|
36 files have a collection of non-significant differences in addition to the extra |
|
37 difference introduced. |
|
38 |
|
39 Other cases are copies of real examples which showed a signficant difference. |
|
40 |
|
41 unknown_format\data\BuildInfo.txt |
|
42 |
|
43 - simple example of a text file with different contents |
|
44 |
|
45 unknown_format\ksa\layout.bin |
|
46 |
|
47 - simple example of a binary file with different contents |
|
48 |
|
49 SIS_file\release\armi\urel\gdbstub.sis |
|
50 |
|
51 - stub SIS file, currently not correctly recognised? |
|
52 |
|
53 Intel_DLL\wins\udeb\C32.dll |
|
54 |
|
55 - ??? not recorded why this is different yet |
|
56 |
|
57 Intel_DLL\wins\urel\BAFL.dll |
|
58 |
|
59 - Compiler generated thunks are laid out in a different order. |
|
60 This is a known problem with Developer Studio which is triggered |
|
61 by building the source from a different location. |
|
62 |
|
63 |