|
1 iExploder 1.3.2 |
|
2 =============== |
|
3 |
|
4 Welcome to iExploder. a highly inefficient, but fairly effective web |
|
5 browser tester. The code still has a lot of work to be done, but it's |
|
6 definitely usable. Here are some notable features: |
|
7 |
|
8 * Tests all common HTML and CSS tags and attributes, as parsed from |
|
9 the KHTML, WebKit and Mozilla source trees, as well as tags for |
|
10 Internet Explorer from MSDN. This also includes a few Javascript hooks. |
|
11 * Numeric, and String overflow and formatting tests |
|
12 * Sequential and Randomized Test Case Generation |
|
13 * Test Case Lookups |
|
14 * Subtest generation |
|
15 |
|
16 |
|
17 Installation (Standalone) |
|
18 ------------------------- |
|
19 Make sure you have Ruby installed (comes with Mac OS X, most Linux |
|
20 distributions). See http://www.ruby-lang.org/ if you do not. |
|
21 |
|
22 If you do not already have a webserver setup, you can use the server |
|
23 built into iexploder. Simply go into the htdocs directory and type: |
|
24 |
|
25 % ruby webserver.rb |
|
26 |
|
27 A webserver will then start on port 2000 with the iexploder form. If |
|
28 port 2000 is not preferable, you can pass it another port on the command |
|
29 line: |
|
30 |
|
31 % ruby webserver.rb 2001 |
|
32 |
|
33 Please note that lasthit.rb does not currently work with the logs output |
|
34 from this method. I recommend using a seperate instance/port number |
|
35 for each browser you test simultaneous using this method. |
|
36 |
|
37 |
|
38 Installation (External Webserver) |
|
39 --------------------------------- |
|
40 If you wish to use an external webserver (required for lasthit.rb use), |
|
41 you may do so. IExploder has been tested with apache. |
|
42 |
|
43 Copy the contents of the htdocs/ folder to any directory served |
|
44 by your webserver. Make sure that directory can execute CGI scripts. If |
|
45 performance seems to be low, please try using mod_ruby. |
|
46 |
|
47 |
|
48 FAQ: |
|
49 ---- |
|
50 1) Are the tests always the same? |
|
51 |
|
52 The test cases should always be the same on a single installation, but not |
|
53 necessarily on different installations of iExploder. Random generator seeds |
|
54 may differ between operating systems and platforms. If you alter the tag and |
|
55 property counts in config.rb, it will change the test cases as well. |
|
56 |
|
57 |
|
58 2) How do I look up the last successful test for a client? |
|
59 |
|
60 Use tools/lasthit.rb. When I get a crash, I usually do something like: |
|
61 |
|
62 % tail -15000 /var/log/apache2/access_log | ./lasthit.rb |
|
63 |
|
64 Letting you know how many tests and what the last test id was for each |
|
65 client tested. You can then try to repeat the test, or go through the |
|
66 subtests to see if you can repeat the crash. |
|
67 |
|
68 |
|
69 3) How do subtests work? |
|
70 |
|
71 If you see a crash on a particular test, and would like to determine the exact |
|
72 line that is crashing it, you can use subtests. To do so, go back to the test |
|
73 submission form, and start the test from the number that a crash was indicated |
|
74 on. Instead of leaving the "subtest" field blank, set it to 1. This will rotate |
|
75 through each subtest for a particular test. |
|
76 |
|
77 Each subtest will rotate through a tag offset and a number of tags to |
|
78 garble, which should help you isolate the instance. The number of tags |
|
79 used doubles each cycle. Here is an idea of how many subtests to expect |
|
80 based on your $HTML_MAX_TAGS settings: |
|
81 |
|
82 tags subtests |
|
83 ---------------- |
|
84 32 138 |
|
85 48 236 |
|
86 64 332 |
|
87 96 558 |
|
88 128 782 |
|
89 |
|
90 Most of the time you will be able to replicate a crash within the first |
|
91 $HTML_MAX_TAGS subtests, but sometimes crashes are due to a combination |
|
92 of corrupted tags. |
|
93 |
|
94 |
|
95 4) How come I can't seem to repeat the crash? |
|
96 |
|
97 Many browser crashes are race conditions that are not easy to repeat. Some |
|
98 crashes only happen when going from test 4 -> test 5 -> test 6. If you can't |
|
99 repeat the crash through subtests or a lookup of the failing test, try going |
|
100 back a few tests. |
|
101 |
|
102 That said, some crashes are due to race conditions that are very difficult |
|
103 to replicate. |
|
104 |
|
105 |
|
106 5) Why did you write this? |
|
107 |
|
108 I wanted to make sure that FireFox had as many bugs fixed in it as possible |
|
109 before the 1.0 release. After 1.0 came out, I kept improving it. |
|
110 |
|
111 |
|
112 6) Why does Internet Explorer run the tests so slowly? |
|
113 |
|
114 <META> refresh tags are very fragile in Internet Explorer, and can be easily |
|
115 be rendered useless by other tags on the page. If this happens, a javascript |
|
116 refresh will execute after a 1 second delay. |
|
117 |
|
118 |
|
119 |
|
120 7) How do I change the number of tags iExploder tests per page? |
|
121 |
|
122 See config.rb. I personally recommend 32-128 HTML tags per page. While this |
|
123 seems to be a lot to go through when designing a test case, that's why the |
|
124 subtest engine was made. Different web browsers will have different |
|
125 performance characteristics when it comes to the number of tags per page. |
|
126 |
|
127 Here are the results with Firefox 2.0b1 (Bon Echo) and the iExploder |
|
128 built-in webserver running tests 1-250. |
|
129 |
|
130 tags seconds pages/second tags/second |
|
131 ----------------------------------------- |
|
132 32 60 4.0 131 |
|
133 48 85 2.9 141 |
|
134 64 95 2.6 168 |
|
135 96 120 2.1 200 *DEFAULT* |
|
136 128 140 1.8 228 |
|
137 196 228 1.1 210 |
|
138 256 308 0.8 207 |
|
139 |
|
140 If you find pages/second to be more important than tags/second, I would |
|
141 change $HTML_MAX_TAGS to 32. Do keep in mind that large tag counts mean |
|
142 longer subtest generation periods. |
|
143 |
|
144 |
|
145 8) What other performance enhancements can I make? |
|
146 |
|
147 * Before using iExploder, reset your browser history |
|
148 * Minimize your browser while iExploder is running |
|
149 * If using Apache, make use of mod_ruby |