bintools/evalid/evalid_test_notes.txt
author marvin shi <marvin.shi@nokia.com>
Wed, 17 Nov 2010 16:47:32 +0800
changeset 695 46ca13b54f56
parent 600 6d08f4a05d93
permissions -rw-r--r--
features: bsym for rofsbuild and log input support for rombuild
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
600
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     1
evalid_test_notes.txt
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     2
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     3
testfiles.zip contains two matching trees, each divided into 
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     4
three sub-trees:
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     5
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     6
left\ok\...
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     7
left\missing\...
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     8
left\fail\...
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
     9
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    10
right\ok\...
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    11
right\missing\...
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    12
right\fail\...
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    13
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    14
Running "evalid left\ok right\ok" should produce 100% OK.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    15
The "ok" trees contain at least one example of each type of file,
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    16
and should grow to illustrate any new form of non-significant
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    17
difference. 
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    18
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    19
Running "evalid left\missing right\missing" should report
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    20
only failures in the shape of the tree.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    21
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    22
Running "evalid left\fail right\fail" should report differences
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    23
in all pairs of files.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    24
The "fail" trees should grow whenever a new form of significant
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    25
difference is found, so that we can avoid making EVALID too
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    26
tolerant.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    27
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    28
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    29
-------------
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    30
Significant Difference test cases
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    31
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    32
Most examples are synthetic test cases obtained by copying one or more pairs from
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    33
the ok tree into the fail tree, then modifying the right\fail file to introduce
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    34
a specific type of difference. The files are usually renamed to indicate the
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    35
nature of the difference. Using the pair of files from the ok tree means that the
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    36
files have a collection of non-significant differences in addition to the extra
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    37
difference introduced.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    38
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    39
Other cases are copies of real examples which showed a signficant difference.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    40
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    41
unknown_format\data\BuildInfo.txt
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    42
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    43
- simple example of a text file with different contents
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    44
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    45
unknown_format\ksa\layout.bin
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    46
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    47
- simple example of a binary file with different contents
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    48
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    49
SIS_file\release\armi\urel\gdbstub.sis
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    50
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    51
- stub SIS file, currently not correctly recognised?
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    52
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    53
Intel_DLL\wins\udeb\C32.dll
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    54
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    55
- ??? not recorded why this is different yet
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    56
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    57
Intel_DLL\wins\urel\BAFL.dll
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    58
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    59
- Compiler generated thunks are laid out in a different order.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    60
  This is a known problem with Developer Studio which is triggered
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    61
  by building the source from a different location.
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    62
6d08f4a05d93 add deprecated files
jjkang
parents:
diff changeset
    63