| author | timothy.murphy@nokia.com |
| Tue, 11 May 2010 14:36:11 +0100 | |
| branch | fix |
| changeset 547 | 9fe7d0ab0f8f |
| parent 533 | 408bfff46ad7 |
| permissions | -rw-r--r-- |
|
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
1 |
Resources were copied using make rules. This is sensible in local |
|
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
2 |
machine builds but non-optimal in cluster builds. It is entirely IO |
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
3 |
bound so that instead of benefitting from running on the cluster it |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
4 |
simply creates more IO as files need to be transferred over the network, |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
5 |
possibly multiple times. |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
6 |
|
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
7 |
This change introduces the <finalcopy> tag to the log which the frontend |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
8 |
reads in a new "default" filter called FilterCopyFile. Thus the python |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
9 |
frontend does the copying rather than the cluster build engine. |
|
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
10 |
|
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
11 |
This happens at the end of each invocation of the build engine or "stage". |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
12 |
Since resources are built in their own stage, the copying is completed |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
13 |
before build tasks in the other stages require them. The copied files |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
14 |
are not needed in the resource stage itself. |
|
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
15 |
|
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
16 |
The filter uses <progress:end ...> tags to determine when a stage |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
17 |
is finished and this requires that the timing feature be switched on |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
18 |
permanently. |
|
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
19 |
|
|
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
20 |
The format of the tag is: |
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
21 |
<finalcopy source="sourcefilename">dest_filename1 dest_filename2 ...</finalcopy> |
|
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
22 |
|
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
23 |
Spaces may not be used in filenames. The sequence "%20" may be used |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
24 |
instead. |