Wrote a script to extract data for e32tests & f32tests based on input from: richardcoles / Symbian Foundation Community Member and Krishna Achugatla from Symbian.

Include -j option for sf-do-full-export target

1) Add support for parallel makefile generation by spliting compile into two-step operation. export first, followed by compile. 2) Remove smoketest from sources.csv and into sources_test.csv. This enables a fully resolved csv file to be fed directly back into a platform build.

Merge

Include armv5 bootloader binaries into lib zipfile.

Updated version checker to permit all versions of hg from v1.2 to v1.5

Moved bc_filter tool to utilities repo.

Changes from v800008

Adjusted invocation of ConE so it doesn't completely fail if /epoc32/rom/config/ does not exist. Should get the CI build running again.

Regularised indentation

Fix java version is not available

Update baseline for CompilerCompatibility builds

Update baseline for symbian3 builds

Added ATS property files for v800004

Turn off zipContentMacro failure for empty zip targets in Helium 7

Relocated Helium 7 *.log.dir property settings from root build.xml to common\build.xml so that 'sf.spec.job.drive' works again.

build_md5.zip now in release_metadata.xml

findphysicaldrive.pl - Fixed so it works with empty (freshly formatted) drives.

Catchup merge

Catchup merge

catchup merge.

Packaging tools with the binaries.

Catchup of Shabe's Helium7 work.

Added tag PDK_3.0.h for changeset ebd555518364

Move copying of System Model to after compile-main

Export system model to BOM directory. This used to be done by generate-layers, however this isnt called by compile-main anymore.

Merge

Fix scan_antlogs.pl execution plus fix ConE calls to use PDT_HOME env variable

Add initial support for Helium7 for the platform. * Use sf.spec.sbs.options to pass args direct to Raptor. * Backwards compatibility with Helium5 maintained. * Listdir function to be provided by sbs check filter, which will be added later.

quick fix for zipping winscw lib files. Added property 'sf.spec.package.zip.format' for the post-build zip format (default is '7z').

Changes to zipping during platform build; binary files are divided into 4 more zips, reducing individual sizes.

Added tool for filtering BC header reports

sourcesync template - Updated generation of 'changes.txt' so that it does a 'follow' rather than being restricted to a 'branch'. This enables us to see the history if the previous tag was an ancestor but on a different branch.

Corrected property name for activating the hg cache, so it should take effect!

Remove Raptor log preprocessing as no longer needed

Take the UH parser from the utilities repo

Add Helium and java to the enironment info

Update baseline for symbian3 and CC. Remove symbian2 property file as we are no loger building for symbian2

s/bishare/v800020/

Reviewed rules defining the BRAG status

Merge

Merge

Add Helium 7 support for package builds. - During operation, if last.major.helium.version is detected as 7.0 then it will use new sbsinput mechanism. - Restore sf-postbuild dependency on sf-build-noprep target

Added error correction, so if a cache repository is corrupted in some way, then it will by-pass it, and try to recover it, or delete it.

Moved definition of dollar, so it's available for use when needed.

Stopped cache being located/created in a stupid place, relative to the CWD on the cache drive. Work around limitation of ant-contrib regular expressions, so they're not thwarted by \s in the cache location property.

Add missing / Avoids potential problem of it not seeing an existing cache, depending on the CWD on the cache drive at the point the target is running (which could be anything really).

Turned on use of cache for all platform builds (unless over-ridden by the particular job properties of course).

Bug 1661: Simplified the sequence of pulls/clones via the cache, so the changes simply go from source -> cache -> destination. Made use of --uncompressed flag to speed things up. Including setting the flag when we create a cache entry.

Bug 1661: Improve build speed with better caching strategy Changed the way that the cache is located, so the same code can run on any build machine without any configuration. Made work around for Bug 419 the only option, as we're not using the alternative anyway, and I don't want to commit completely untested code! Changed caching algorithm, to improve build times.

Updated capabilities of findPhysicalDrive to support imminent work on locating the hg cache.

Indentation and other minor changes to make the file easier to read and understand.

Added profiling timer to sf-get-source target.

Fix version of new mwccsym2 is reported as N.A.

Added tag PDK_3.0.g for changeset fc600bbaa15c

Fix BRAG status is different as viewed from Diamonds and from the _BRAG.xml page

Update baseline for symbian3 and CompilerCompatibility builds. Removed unused property file for NewGraphicsArchitecture

Dump environment info to Diamonds

Fix when accessing uh index from brag summary, pages linked from uh index are not found

Make BRAG point to the results of UH parser instead of intermediate artifacts

Remove redundant computation of some BRAG intermediate xml files

Commented out log-to-brag call on build environment checker as its results shouldn't be brag affecting

Make index.html report on all packages, including the ones built successfully. Make the list of packages be alphabetically sorted

Enabled filtering results after Static BC analysis

Added tag PDK_3.0.f for changeset ef25bf39d67c

Creation of TAGS branch. The TAGS branch should be used for *all* tags, including PDK release tags and package release tags. Do not merge this branch with any "code" branches.

Merge

Remove check-tool-dependencies to allow > Perl 5.6.1

Disabled filtering results after Static BC analysis

adding EPL headers to smoke test

sf-run-analysis-missing-files - changed to depend on a new undefined parameter 'sf.spec.checkbuild' so that it can be turned on and off

sf-s60-cone-cenrep - S60 config with epoc32/data/z combintation was missing.

Add ConE or ConfigurationTool support. ConfigurationTool is used for Symbian2 only.

Updated to ignore messages of the form "Nothing to be done for ..." as these do not represent a problem of any kind at all.

Removed copying smoketest source to EPOCROOT from sf-build-smoketestpkg target

Changed the way that the sf.spec.sourcesync.enable property is used to control the syncing, so that it can sync test repos even if it's not syncing any source repos.

Move targets round so that those related to syncing the source are all together.

Intelligent way of getting timings for ALL uses of sf-zip-content

Removed changes in build.test.xml to copy smoketest source to EPOCROOT

Removed copying smoketest source to EPOCROOT

Added stopwatch tasks through most of the code hit during a platform build.

Added sf-scrub-build, for completely freeing up the resources used by a build (ie deleting it from the HD). Removed import of generated ant script as it had no effect, and was causing strange failures with sf-scrub-build.

Removed check for value of PDT_HOME, as it doesn't matter where it's installed, and it's turning the CI builds red.

Code to automatically find a physical drive for running a build, triggered as necessary.

Added new logic to deal with drive letter allocation when builds are launched from Hudson.

Added framework for specifying a "sources.csv" for test repos (separate from both the FBF and the sources repos).

Catchup merge

Fixed websrv bc test suite. Increased time-outs for selected bc tests.

updated version of sysdefdowngrade.xsl from BobR

add xsl for merging / downgrading system model

Added smoke test for Media management

Fixed ats_generate_index.pl to not include the link to Smoke Test Report

Disabled BC check for Platform builds

Fixed creation of the ats_reports folder

Fixed copying temp index

Implemented publishing Dynamic BC Test Reports

Added new property for publishing Smoke Test Report

Added tool for generating the top-level index for BC test reports

Add escape left angle bracket to preprocessing of raptor log (see bug #1519)

PDK baseline for symbian3 builds back to PDK_3.0.d as PDK_3.0.e is not stable yet

ArnaudL: corrected last entry that had "????" left behind.

Arnaudl: Added scripts and files related to the development of "task 286 - Package test status prep"

Added tag PDK_3.0.e for changeset a1ebf7f8e66f

Move gcc into tools_epoc.zip so that syborg/beagle rom building works during build from source only.

Added new param for calling ats_specialise_test_drop.pl from build.test.xml

Added smoke test for making a connection to a web server through ethernet

Update the PDK baseline for symbian3 builds

Fixed smoke test for Localisation

Added smoke test for Graphics

Added tag PDK_3.0.d for changeset b9c988316957

Added tag PDK_2.0.2 for changeset 2a9e6403efaf

Platform Build - Added 'sf-postbuild' onto deps for 'sf-build-noprep' as although it isn't the best place to have it, it's the same as the deps for the version in 'common' and is needed for the external Hudson jobs.

Ensured build environment is always compared with expected environment, whether or not it would abort the build because of it.

Changed building of tools targets so that it builds the targets listed in ${sf.spec.sbs.tools.config}, in sequence, rather than just buildng tools2_rel.whatlog followed by tools_rel.whatlog, regardless of the property.

Moved hlm:argSet definitions in sf-os-compile to be adjacent to the call that uses them. (And deleted three unused ones in the process.)

Update the PDK baseline for symbian2 builds

Updates to make the build environment check more reasonable: Mercurial v1.3 permitted The Java compiler is not a showstopping issue 7-zip can be installed in any location Update to Helium 5 Helium can be installed in PDT 1.*, not necessarily 1.0 Raptor installation path not significant Update to Raptor 2.9.* The Raptor patch to update the bundled version of python is no longer relevant BRAG calculations updated to ignore items not being in the system path, as this just doesn't matter. Overall effect is that the build environment check should pass on a machine that is able to do a build!

Regularised indentation

Fixed postAction in ats_specialise_test_drop.pl

Implemented FileStoreAction for BC tests

Implemented exporting Smoke Test Report to Diamonds

Updated postbuild to publish smoketest results

Updated ats_specialise_test_drop.pl to insert FileStoreAction param

Added new postAction to smoketest.xml

Restored signifcance of the MS tools not being available, as this only occurs when running a configuration that would try to use them.

Re-indented target "sf-os-compile" (following best guess as to the standard) to make it easier to read.

Re-assessed some of the raptor error messages to reflect the inconsequential nature of missing the MS dev tools when not trying to build the tools target.

Changed mergeXML.pl to use a better version of the XML printing algorithm, so it can generate well-formed content if the input includes some rogue &s or <>s.

If any files are in both the list of SDK headers, and the list of RnD headers, then they're only zipped in the binaries_epoc_additional zip, and not the binaries_epoc_sdk zip.

Fixed format checking of summary.csv to deal with new field.

Test: Add MediaMgmt smoke tests (not being built or run ATM)

Test: Remove the media management smoke test for now

Re-instated call to process raptor logs for BRAG purposes.

Remove s60 postfix from sf-s60-create-cenrep for Symbian^3

Fixed format checking of summary file so that it actually checks something!

Updated framework for packaging SDK headers so the contents to zip can be set by a project property (or it can zip everything together if it's not set).

add media mgmt to smoketest build

Fix cenrep export location

SDK file list - Fixed duplicates

Fixed and enabled smoke test for Set Language

Defined new devices for v800017

Disabled smoke test for Set language

Small changes to make the output file more regular, and therefor easier to read and understand.

Added processing of error tags.

Fixed perl scripts for building BC test drops

Fixed Set language smoke test

Changed to package the SDK header files in a different zip to the rest of the epoc32 tree.

Fixed smoketest.xml

Added new smoke test for phone languages

Removed code for building ROMs (now done by Raptor) but added code for publishing ROM building logs.

Fix releaseables are missing. Fix stray call to summarize.pl

Catchup merge

Disabled BC check from running as default on builds. Should be enabled specifically at the command line.

Added new smoke test for Setting wallpaper

Disabled bccheck for normal builds

Catchup merge

Platform Build cenrep generatation - modified 'sf-s60-create-cenrep' to hopefully cope with the different CRML export paths coming from the different platform_paths.hrh. This isn't pretty.

Fix Diamonds links are no longer appearing

Added conditions for BC check results to be published

Changed delays between key presses

Added smoke test to close UI setup dialogs

Increased delay before presskey

Enabled bccheck for normal builds

Update send_xml_to_diamonds.py and update invoking code to match new i/f

Updated IF statements for BC reports

Add new recipe-error categories armcc_modifier_not_allowed, mwccsym2_file_cannot_be_opened

Changed current baseline version for BC check.

Fix freemarker.core.ParseException: Encountered { at line 25, column 15 in linksForDiamonds.xml.ftl

Added Static BC reports for Platform Builds

Add NewGraphicsArchitecture_props.ant.xml

Update PDK baselines for package builds

Merge Tags

Add Raptor Build Summary to the links for Diamonds

Publish HTML files summary of Raptor build

Now creates src-by-layer entries on the fly, so it doesn't create release zips with no content.

Added interpretation of recipe "win32def2lib" failing.

Fix XML-SAX bug #43174 (see http://rt.cpan.org/Public/Bug/Display.html?id=43174)

Improved Raptor parser, now "uh"

Changed versions of the baselines for Static BC test

Disabled Dynamic BC tests

Merge

Added tag PDK_2.0.1 for changeset e13acd883fbe

merged heads - non trivial merge in smoketest. Manual merge of those changes.

Test: Input methods smoke tests added.

Test: Fixed up the app install smoke tests so that both now pass

Added new smoketest

Add call to parse (preprocessed) raptor logs directly for BRAGG.

Updated XSL to include counts of failures at each severity, and also the new effect/causes distinction, for failures that are not package specific.

New script which parses raptor logs directly to produce a BRAG summary. New version of XML structure accommodates causes of failure as well as the effect of the failure, and deals with excessive error messages. New structure also avoids a potential problem (bug?, questionable feature?) that both IE and firefox exhibit. Some simplifications of brag utilities module.

Moved common code to shared module.

Updated the options to pass to getenv, to include -k. This avoids aborts if things are not exactly as the tool expects.

Added tag PDK_3.0.c for changeset 12815724a053

Added new smoketest

Added new smoketest

Bug 881: Now also provides the locations of the two split-build system definitions to the release note generator, so it will work with both single and split builds.

source sync template - enabled getting the mercurial changes if you have sf.spec.sourcesync.archive set

Transplant fix in ats BC Test - 'bld_sis' change that was knocked out with the rest of the transplant (bcdrivers/utilities)

Added call to munge license headers at the top of the postbuild phase, so it should be done within the platorm build.

Switched to using revised Package/FCL analysis script.

Remove meta data from package definition and replace with helium script

Test: Forgot to execute Profile smoke tests

Test: Changes to test drop files to run AppInstall

Test: Tweaked Profile tests and added AppInstall

Add new raptor-analysis recipe-failure category: elf2e32_symbol_missing_from_elf_file

Override Helium defaults for RVCT checks to allow winscw builds without RVCT

Update PDK baselines for package builds

Add CompilerCompatibility properties file for package build

Changed device name for lon-engbuild89

Changed device name for lon-engbuild87

Changed device name for C100725

Changed device name

host name is added for any number of devices

Added smoketest for Notepad

Add recipe-failure category armcc_internal_fault

fix bug with multiple harnesses

export to 'smoketest' instead of 'testframework'

capitals

updated smoketest plan to support both harnesses

Add recipe-failure categories armcc_controlling_expression_is_constant and armcc_generic_warnings_errors

Adding profiles smoke tests

Removed unnecessary dependencies, to untangle the dependency graph a little.

In raptor-analysis create summary.csv file anyway, otherwise some targets following may crash the build

Added smoketest for email

Update to add all components and create zip file for PDT

Add all components to bc test suite build

Fix some build configurations are left as 'noconfig'

Add raptor recipe error category armar_file_does_not_exist

Add raptor error categories unknown_build_config and no_build_configs_given

Merging tag

Added tag PDK_3.0.b for changeset cdfb88270be0

Let sbs targets be taken from sf.spec.sbs.config

Added 3 new smoke tests

Changed dst path for evptests.txt

Add missing components to the BC build script

Added new device for LON-ENGBUILD89

Added new device for LON-ENGBUILD87

catchup merge.

catchup merge

source sync template - Changed the BOM changes targets to limit the listed changes to the branch if the source was synced up using a branch, to make it easier to read.

Added Core Telephony BC tests

Added all of the tests for ASP

If makefile_template copy fails then ignore, as its no longer needed for S^3.

Merge

Add sf-rombuild target to build ROMs. Backout ARM license check suppression.

Change baseline for symbian3 to \\bishare\releases\PDK_3.0.a

Fix only one raptor log gets processed by Dario's Raptor parser

Send to Diamonds only 3 links: brag, build bom, build logs

Change default location of build results/reports to \\v800020\Publish\SF_builds

Make script stop execution after help is displayed if --help

Remove default "test" tag

Catchup merge of tags

Added tag PDK_2.0.0 for changeset 7a343c038e7d

Added tag PDK_3.0.a for changeset 4709730d5226

Added new BC test

Added new BC test

catchup merge.

catchup merge.

sf-run-evalid - minor change to make sure that evalid is run at the root of the build drive, which can stop the MD5s from being generated. (evalid requires dirs to be relative and not absolute)

added a missing testdata file

Added new BC test

Updated throughout to use the "temp_build_files" dir on the build drive rather than the "generated" directory in the fbf source tree.

Moved the generation of the release metadata entry for the MD5 zip outside of the parallel section, so it can't co-incide with the zipping of the binaries. Ensured that any errors generated when merging log files are not just hidden by putting them in the output file (which will render it not well-formed XML).

Replaced <available> check with an <uptodate> check, to greatly help people working on the FBF who want the file automatically regenerated according to the changes they have made.

Restored call to summarise final BRAG status for regular platform builds.

Major packaging change to work around parallel calls - stopped zipping targets from having direct dependency on 'sf-preprocess-package-config', and instead have 'sf-zip-content' call 'sf-preprocess-package-config' if needed. Changed 'sf-prep' to call a target that wipes out the 'generated' directory. It's not perfect but it gets us going again.

Moved invocation of sf-brag-to-diamonds, so it's right at the end of the postbuild (apart from publishing zipped logs). This avoids the possibility of a build abort when parallel threaqds are trying to a) convert a log file into a brag file, and b) merge all brag files ready to determine the brag status. It also makes more sense - don't calculate or send the BRAGG status until the build is on the cusp of completion.

Merge

Add standard C++ tests to the build

Changed timeout

Added new BC test

Added new BC test

Bug 479: RnD package zips contain content not releaseable to public Updated packaging tools to separate binaries and source in RnD repos.

tried changing the directory for pbkaddressselect.rsc

Added new BC test

Merge

Add missing components to build

Add use lib statement to look for Text-CSV in local lib folder

Bug 478: build_*.zip missing from generated release_metadata.xml Added call to zip system to add entries for these zips to the release_metadata.xml

Updated comments in perl scripts

Added new BC test

fixed up the phonebook tests to run faster

Replaced section "Package failures by phase/step/severity" with "Package failures by package/severity" Added internal links from the package/severity breakdown to the peices of the new section.

Merge

Make csvToSysDef.pl work with codelines different than 3k

Updated comments

Move QEMU into adaptation layer when packaged

ATS fix for getting the smoketest to run.

Added lib dir to pick up Text::CSV when it's not included with the installed perl distribution.

Add new BC test

Added lists of error messages themselves, categorised by phase, step, ...

Added tables for eack package with failures, broken down by severity. Added anchor links to new tables from table of packages.

Added understanding of the "ar" failure.

Added post-processing of YARP analysis to feed into BRAG status.

Added copy to copy the XSL so it's available if ever an XML is generated there.

Created a dedicated sf-log-to-brag target so it can take care of the commonality of the task, including the creation of the directory for XML output.

catchup merge.

sf-gen-buildinfo-txt target in platform build - De-CamelCasing buildinfo.txt deletion and creation to fix zip inclusion/exclusion problems.

Fixed generation of rnd excludes file, so there are the necessary new-lines.

Add new BC test

New BC test added

Device name removed

sf-brag-to-diamonds - Only copy over the Brag XML if '${sf.spec.publish.enable}' is true.

ATS target 'sf-build-smoketestpkg' - fix for argument name change in 'ats_specialise_test_drop.pl'. Update command-line help for 'ats_specialise_test_drop.pl'

catchup merge.

Modified sf-check-missing-os-files to deal with 'sf.os.compile.iterate' property. Added new target 'sf-compile-options' for setting the gmake sbs compile option, and added it as a dep for the compile targets.

Allow to copy all the files in the summary directory to the bishare to allow link for BRAG to work in Diamonds.

remove 'test.xml'

added basesvs.xml

sorted out more stuff with the dynamic bc, fewer failures now

some more updates to bc stuff

graphics test plan now only runs fbserv tests

New BC test

Set PDK baseline to PDK_3.0.a_alpha_FCL.120

Merge

Skip sf-compare-to-baseline if sf.spec.publish.enable is false. Update FBF documentation.

Make sure the BRAG xml and xsl files are sent to bishare in an unzipped way to allow access from link in Diamonds.

Make sure the BRAG xml and xsl files are sent to bishare in an unzipped way to allow access from link in Diamonds.

Make sf.spec.job.codeline the publishing category

Merge

Fix layer names for production and test builds

sf-run-analysis - Making 'sf-brag-to-diamonds' depend on 'sf.spec.publish.enable' property

catchup merge

Fixed md5 zipping in case no publish is requested. Added overwriting of variant.cfg if it already exists

catchup merge

sf-check-env - Fixed typo causing build to always halt if 'sf.suppress.buildenv.check' isn't set.

catchup merge

source-spec.ant.xml.ftl - Added change info for the BOM that lists submitted changesets between the baseline ('sf.previous.pdk.tag') and the synced changeset and writes to a changes.txt file in the BOM dir.

Added work-around for the QEMU package whilst it's still in an interim location, but we want to release it in a PDK. (The proper fix would be to move the repository into the right place and update the sources.csv, but time is of the essence!)

merge

fixed up multimedia to match fts

fixed up most of the issues with the dynamic bc test plans

sf-preprocess-package-config - fix for another BRAG step that presumes that "${build.log.dir}/summary" exists.

sf-check-env - Adding mkdir for BRAG output file.

sf-prep - Partial Revert of changeset aa36fa431e3d - 'Added processing of the BuildEnvironmentCheck for BRAG purposes.' as we need 'sf-check-env' to be optional with 'sf.suppress.buildenv.check' variable.

Keepgoing in the event of testpackage submit error.

Adjustsed BRAG to BRAGG, to accommodate the possibility of there being Green builds and Gold builds.

Adjusted capitalisation to make the final output more aesthetically pleasing.

Added processing of the BuildEnvironmentCheck for BRAG purposes.

Parse log generated during proprocessing of zip config, so it gets included in BRAG analysis.

Merge

Upload codeline and sf.spec.diamonds.tag as Diamonds tags

Add link in Diamonds to read the BAG status from _brag.xml.

Sorted out misalignment between capture of diamonds build id and its re-use.

Copy Symbian_OS.HRH to required location

Add codeline specific property files

Merge

Merge

Reviewed build category and tags

Allow codeline-specific property file to be read (if present)

Added target to push a collated and calculated BRAG status to diamonds.

Save the diamonds build ID in a file as soon as it is generated.

catchup merge.

Creation of 'sf-create-public-PDK' for munging licences in text files. Not tested yet as I've got another build running on this machine, so submitting and kicking via Hudson.

Added support for being passed wildcards in an XML file argument.

Added schema setting, so Diamonds should accept the data.

Bug 419 source sync workaround - Added property sf.spec.sourcesync.bug419 to common_props.ant.xml and set to true.

Catchup merge

Bug 419 source sync workaround - Added new variable 'sf.spec.sourcesync.bug419' that can be used during source sync that just performs the sync manually without using Maven.

Merge

Update commands for clone and copy and use of makesis

New XSL rules to generate a diamonds compatible BRAG status, which can be posted into the database.

Added XML escaping, so that the output should be well-formed XML even if the input includes '<'s, '>'s, or '&'s.

Added code to handle the script being given a log file argument which includes wild cards.

Fixed handling or regular expression strings, so the regular expression functiuonality works, not just substring detection! Added special handling if the severity marker is "ignore". Unsuprisingly, it ignores that line.

Adjusted sf-run-analysis-raptor so that when preprocessing the files, STDERR is appended, but STDOUT is not.

Pre-emptive performance fix - avoid having to read the complete log file into memory at the same time before processing it.

Static BC tests enabled

Catchup merge.

Added helium target sf-run-sfl-licence-munging for Public PDK creation.

Added framework script for turning log files into XML BRAG summaries.

Time-outs reduced

Script to merge multiple MXL files together, according to a merge specification string

Adfjusted XML format used, so there's a document-level tag - <buildStatus> which can wrap multiple <phase> tags.

Device NAME removed

Device NAME removed

rmtree check removed

Time-out changed

Device NAME removed

Device NAME removed

Device NAME removed

Device NAME removed

Files names updated

Merge

Skip armcc licence checking