author | Iain Williamson <iain.williamson@nokia.com> |
Fri, 28 May 2010 15:15:09 +0100 (2010-05-28) | |
changeset 579 | e26b09d36de0 |
parent 547 | 9fe7d0ab0f8f |
permissions | -rw-r--r-- |
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
1 |
Resources were copied using make rules. This is sensible in local |
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
2 |
machine builds but non-optimal in cluster builds. It is entirely IO |
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
3 |
bound so that instead of benefitting from running on the cluster it |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
4 |
simply creates more IO as files need to be transferred over the network, |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
5 |
possibly multiple times. |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
6 |
|
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
7 |
This change introduces the <finalcopy> tag to the log which the frontend |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
8 |
reads in a new "default" filter called FilterCopyFile. Thus the python |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
9 |
frontend does the copying rather than the cluster build engine. |
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
10 |
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
11 |
This happens at the end of each invocation of the build engine or "stage". |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
12 |
Since resources are built in their own stage, the copying is completed |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
13 |
before build tasks in the other stages require them. The copied files |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
14 |
are not needed in the resource stage itself. |
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
15 |
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
16 |
The filter uses <progress:end ...> tags to determine when a stage |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
17 |
is finished and this requires that the timing feature be switched on |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
18 |
permanently. |
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
19 |
|
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
20 |
The format of the tag is: |
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
21 |
<finalcopy source="sourcefilename">dest_filename1 dest_filename2 ...</finalcopy> |
533
408bfff46ad7
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.
timothy.murphy@nokia.com
parents:
diff
changeset
|
22 |
|
547
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
23 |
Spaces may not be used in filenames. The sequence "%20" may be used |
9fe7d0ab0f8f
fixes for review comments. better docs. copyrights. make copy filter more modular, change <copy> tag to <filtercopy>
timothy.murphy@nokia.com
parents:
533
diff
changeset
|
24 |
instead. |