|
1 # ciftest.cif |
|
2 # |
|
3 # Copyright (c) 2010 Accenture. All rights reserved. |
|
4 # This component and the accompanying materials are made available |
|
5 # under the terms of the "Eclipse Public License v1.0" |
|
6 # which accompanies this distribution, and is available |
|
7 # at the URL "http://www.eclipse.org/legal/epl-v10.html". |
|
8 # |
|
9 # Initial Contributors: |
|
10 # Accenture - Initial contribution |
|
11 # |
|
12 |
|
13 ==name ciftest |
|
14 |
|
15 ==short-description |
|
16 |
|
17 Run fshell command smoke tests. |
|
18 |
|
19 ==long-description |
|
20 |
|
21 This command runs smoke-tests for any or all commands that define a C<==smoke-test> section in their CIF file. A C<==smoke-test> section defines a short snippet of fshell script which tests the basic functionality offered by the command. It can be as simple as running the command with no arguments to make sure nothing catastrophic is wrong, or it can be a more in-depth test of all the command's functionality, or anything in between. |
|
22 |
|
23 Example CIF file that supports ciftest: |
|
24 |
|
25 ==name mycmd |
|
26 |
|
27 [...] |
|
28 |
|
29 ==smoke-test |
|
30 |
|
31 mycmd | export -s RESULT |
|
32 var RESULT == "Expected results of running mycmd" || $Error |
|
33 |
|
34 The following environment variables are defined for convenience when ciftest runs a smoke-test section: |
|
35 |
|
36 =over 5 |
|
37 |
|
38 =item * Error |
|
39 |
|
40 Expands to a string that will cause a test to fail. Additionally it prints the current environment, hence is useful to use when C<var> commands fail, as in the above example. Equivalent to something like C<env && error>. |
|
41 |
|
42 =item * SCRIPT_NAME |
|
43 |
|
44 The script name is appended with ":smoke-test", eg "cifname.cif:smoke-test". |
|
45 |
|
46 =item * SCRIPT_PATH, 0 |
|
47 |
|
48 Set as in any other script. |
|
49 |
|
50 =item * SCRIPT_LINE |
|
51 |
|
52 Set as in any other script. Line numbers are relative to the start of the CIF file, not the first line of the smoke-test section. |
|
53 |
|
54 =item * Quiet |
|
55 |
|
56 Used to supress stdout from a command, for when you don't want it to appear in the smoketest results. Usage: |
|
57 |
|
58 mynoisycommand $Quiet |
|
59 |
|
60 Equivalent to putting C<E<gt>/dev/null> on the end of the command. |
|
61 |
|
62 =item * Silent |
|
63 |
|
64 Supresses both stdout and stderr. Useful when an operation is expected to fail. Usage: |
|
65 |
|
66 mycommand expectfailure $Silent && $Error |
|
67 |
|
68 Note how $Silent is combined with C<&& $Error> such that if the command actually succeeded where it was expected to fail, the $Error case would cause the script to abort. |
|
69 |
|
70 =item * Verbose |
|
71 |
|
72 Defined if the C<--verbose> option was given to ciftest. Example usage: |
|
73 |
|
74 var Verbose defined && echo "About to test something-or-other" |
|
75 |
|
76 =back |
|
77 |
|
78 The environment used for running the smoke-test snippets is not shared between commands, so do not set things in one smoketest script and expect to be able to see them in another. (Ie the snippets are run as if with "fshell" not "source"). |
|
79 |
|
80 ==argument string command optional |
|
81 |
|
82 If specified, run the tests associated with the specified command. If not specified, run tests for all commands. |
|
83 |
|
84 ==option bool v verbose |
|
85 |
|
86 Print information about every test even when they succeed. By default only failures are printed. Also causes a summary to be printed at the end. Scripts can also print extra information themselves if this flag is set, by checking for the C<Verbose> environment variable. |
|
87 |
|
88 ==option bool k keep-going |
|
89 |
|
90 Rather than stop on the first failure, attempt to run all tests even if some of them fail. Only relevant if no command argument is given. |
|
91 |
|
92 ==copyright |
|
93 |
|
94 Copyright (c) 2010 Accenture. All rights reserved. |
|
95 |
|
96 ==smoke-test |
|
97 |
|
98 # Ciftest itself doesn't have a smoketest! |