equal
deleted
inserted
replaced
|
1 Resources were copied using make rules. This is sensible in local |
|
2 machine builds but non-optimal in cluster builds. It is entirely IO |
|
3 bound so that instead of benefitting from running on the cluster it |
|
4 simply creates more IO as files need to be transferred over the network, |
|
5 possibly multiple times. |
|
6 |
|
7 This change introduces the <finalcopy> tag to the log which the frontend |
|
8 reads in a new "default" filter called FilterCopyFile. Thus the python |
|
9 frontend does the copying rather than the cluster build engine. |
|
10 |
|
11 This happens at the end of each invocation of the build engine or "stage". |
|
12 Since resources are built in their own stage, the copying is completed |
|
13 before build tasks in the other stages require them. The copied files |
|
14 are not needed in the resource stage itself. |
|
15 |
|
16 The filter uses <progress:end ...> tags to determine when a stage |
|
17 is finished and this requires that the timing feature be switched on |
|
18 permanently. |
|
19 |
|
20 The format of the tag is: |
|
21 <finalcopy source="sourcefilename">dest_filename1 dest_filename2 ...</finalcopy> |
|
22 |
|
23 Spaces may not be used in filenames. The sequence "%20" may be used |
|
24 instead. |