diff -r 9e83d2225bfb -r 33be37141e47 sbsv2/raptor/notes/localresourcecopying.txt --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/sbsv2/raptor/notes/localresourcecopying.txt Mon May 10 13:58:23 2010 +0100 @@ -0,0 +1,18 @@ +Resources were copied using make rules. This is sensible in local +machine builds but non-optimal in cluster builds. It is entirely IO +bound and the cluster aspect simply creates more IO as files need to +be transferred over the network, possibly multiple times. This change +introduces the tag to the log which the frontend reads in a new +"default" filter called FilterCopyFile. Thus the python frontend does +the copying rather than the cluster build engine. + +This happens at the end of the build and since resources are built in +their own "stage" it's is all completed before any other build tasks +are invoked. The copied files are not needed in the resource stage +itself so it is ok that it happens at the end of that. + + +The format of the tag is: +dest_filename1 dest_filename2 ... + +Spaces may not be used in filenames. The sequence "%20" may be used instead.