sbsv2/raptor/notes/localresourcecopying.txt
author timothy.murphy@nokia.com
Mon, 26 Apr 2010 17:33:17 +0100
branchfix
changeset 533 408bfff46ad7
child 547 9fe7d0ab0f8f
permissions -rw-r--r--
fix performance: copy resources in the frontend. Helps cluster builds since remote copying is inefficient.

Resources were copied using make rules.  This is sensible in local
machine builds but non-optimal in cluster builds.  It is entirely IO
bound and the cluster aspect simply creates more IO as files need to
be transferred over the network, possibly multiple times.  This change
introduces the <copy> tag to the log which the frontend reads in a new
"default" filter called FilterCopyFile.  Thus the python frontend does
the copying rather than the cluster build engine.

This happens at the end of the build and since resources are built in
their own "stage" it's is all completed before any other build tasks
are invoked.  The copied files are not needed in the resource stage 
itself so it is ok that it happens at the end of that.


The format of the tag is:
<copy source="sourcefilename">dest_filename1 dest_filename2 ...</copy>

Spaces may not be used in filenames. The sequence "%20" may be used instead.