|
600
|
1 |
evalid_test_notes.txt
|
|
|
2 |
|
|
|
3 |
testfiles.zip contains two matching trees, each divided into
|
|
|
4 |
three sub-trees:
|
|
|
5 |
|
|
|
6 |
left\ok\...
|
|
|
7 |
left\missing\...
|
|
|
8 |
left\fail\...
|
|
|
9 |
|
|
|
10 |
right\ok\...
|
|
|
11 |
right\missing\...
|
|
|
12 |
right\fail\...
|
|
|
13 |
|
|
|
14 |
Running "evalid left\ok right\ok" should produce 100% OK.
|
|
|
15 |
The "ok" trees contain at least one example of each type of file,
|
|
|
16 |
and should grow to illustrate any new form of non-significant
|
|
|
17 |
difference.
|
|
|
18 |
|
|
|
19 |
Running "evalid left\missing right\missing" should report
|
|
|
20 |
only failures in the shape of the tree.
|
|
|
21 |
|
|
|
22 |
Running "evalid left\fail right\fail" should report differences
|
|
|
23 |
in all pairs of files.
|
|
|
24 |
The "fail" trees should grow whenever a new form of significant
|
|
|
25 |
difference is found, so that we can avoid making EVALID too
|
|
|
26 |
tolerant.
|
|
|
27 |
|
|
|
28 |
|
|
|
29 |
-------------
|
|
|
30 |
Significant Difference test cases
|
|
|
31 |
|
|
|
32 |
Most examples are synthetic test cases obtained by copying one or more pairs from
|
|
|
33 |
the ok tree into the fail tree, then modifying the right\fail file to introduce
|
|
|
34 |
a specific type of difference. The files are usually renamed to indicate the
|
|
|
35 |
nature of the difference. Using the pair of files from the ok tree means that the
|
|
|
36 |
files have a collection of non-significant differences in addition to the extra
|
|
|
37 |
difference introduced.
|
|
|
38 |
|
|
|
39 |
Other cases are copies of real examples which showed a signficant difference.
|
|
|
40 |
|
|
|
41 |
unknown_format\data\BuildInfo.txt
|
|
|
42 |
|
|
|
43 |
- simple example of a text file with different contents
|
|
|
44 |
|
|
|
45 |
unknown_format\ksa\layout.bin
|
|
|
46 |
|
|
|
47 |
- simple example of a binary file with different contents
|
|
|
48 |
|
|
|
49 |
SIS_file\release\armi\urel\gdbstub.sis
|
|
|
50 |
|
|
|
51 |
- stub SIS file, currently not correctly recognised?
|
|
|
52 |
|
|
|
53 |
Intel_DLL\wins\udeb\C32.dll
|
|
|
54 |
|
|
|
55 |
- ??? not recorded why this is different yet
|
|
|
56 |
|
|
|
57 |
Intel_DLL\wins\urel\BAFL.dll
|
|
|
58 |
|
|
|
59 |
- Compiler generated thunks are laid out in a different order.
|
|
|
60 |
This is a known problem with Developer Studio which is triggered
|
|
|
61 |
by building the source from a different location.
|
|
|
62 |
|
|
|
63 |
|