tempbottle / autobuild Goto Github PK

/**
 @page Autobuild and scoreboard tools

 @section Autobuild_Introduction Introduction

The autobuild tools are a set of perl scripts, perl modules and
XML configuration files to automatically build our projects (ACE+TAO)
on multiple platforms.  The tools run on Unix (several flavors) and
Windows NT/2k/XP.

SOFTWARE REQUIREMENTS
=====================

(1) perl 5.6.1 or higher

(2) Libwww-perl module.  You can get this from http://www.cpan.org.
    Red Hat Linux distributions may have an optional package perl-libwww-perl.


QUICK GUIDE GETTING STARTED WITH AUTOBUILD
==========================================

(1) Check out the latest build scripts from subversion using:

       git clone https://github.com/DOCGroup/autobuild

(2) Look at the existing XML configuration files in
    autobuild/configs/autobuild/*.  Create a new XML configuration
    for your build and customize it according
    to your local site.

    In the <configuration></configuration> section, you can set environment
    variables like ACE_ROOT, PATH, CVSROOT, LD_LIBRARY_PATH, etc.
    For example:

    <environment name="ACE_ROOT"        value="/export/bugzilla/Minimum_newbuild/ACE_wrappers" />

    The "root" variable is important. It is the root of all future
    "file_manipulation" commands.

    The "configs" variable should contain a space-separated list of -Config
    options that are passed to auto_run_tests.pl. For Windows, it should
    include Win32.

    The "make_program" variable enabled you to specify a different executable
    to call for make.

(3) In the <configuration></configuration> section, specify the directory
    where to place the log files from the build.
    Ideally this directory should be accessible by a web server, so
    that you can see the log files via http.
    For example:
  <variable name="log_root" value="/home/bugzilla/.www-docs/auto_compile_logs/curufin_SingleThreaded" />


(4) Edit one of the files in autobuild/configs/scoreboard/*.xml. The
    logic for choosing the file is as follows. If you choose to donate
    an ACE build which you think is important for the beta/release
    please edit ace.xml and add the following information. If you
    choose to donate a build which compiles ACE on a new
    platform/compiler or different sets of options or your own
    development branch please edit the file ace_future.xml. The same
    logic applies for TAO too.  An example on how to edit the xml
    file is given below:

<scoreboard>
    <preamble>
      Put some <b>HTML</b> introductory text between these tags.
    </preamble>
    <group>
        <name>Linux</name>
        <build>
            <name>Debian_Full</name>
            <url>http://doc.ece.uci.edu/~bugzilla/auto_compile_logs/sirion_Full/</url>
            <build_sponsor>UC Irvine</build_sponsor>
            <build_sponsor_url>http://doc.ece.uci.edu</build_sponsor_url>
        </build>
    </group>
</scoreboard


(5) To run the build, you run autobuild.pl with the XML config file as the
    argument:

  autobuild.pl Debian_Minimum_Static.xml

(6) To update the scoreboard, you run scoreboard.pl with acetao.xml as the
    argument:

  scoreboard.pl -d [Output directory where the html files are placed]
                -f [name and path of the XML file that needs
                   to be used as a meta-file for HTML
                   generation]
                -o [name of the output file. The html file will
                   be saved by this name and placed in the
                   directory pointed by -d].


OTHER NOTES
===========
(1)  In the <configuration></configuration> section, you set shell environment
     variables with the following tag:

     <environment name="ENV VARIABLE NAME"  value="ENV VARIABLE VALUE" />

(2)  In the <configuration></configuration> section, you set Perl global
     variables with the following tag:

     <variable name="PERL GLOBAL VARIABLE NAME"  value="VARIABLE VALUE" />

(3)  Outside the configuration section, commands are specified with the
     following tag:

     <command name="COMMAND NAME" options="COMMAND OPTIONS" />

     Each command is located in Perl modules in the autobuild/command
     directory.  If you look in each Perl module, you will see a line
     such as:

     main::RegisterCommand ("COMMAND NAME", new COMMAND ());

     This line registers "COMMAND NAME" in a global table used by
     autobuild.pl to locate a command specified in the XML file.

(4)  To run an arbitrary command for which a Perl "command module" has
     not been written, you can use the "shell" command.  For example, to run
     the ls command on a Unix system:

     <command name="shell" options="ls" />

(5)  When setting up a Windows build system make sure you disable the firewall
     and the automatic windows updates.

SCOREBOARD NOTES
===================

(1) PDF - to display a PDF file use the <pdf> build option in the
    configuration file for the scoreboard.pl script. For example,
    <pdf>test.pdf</pdf> will cause a hyperlink to appear called
    'pdf' to appear on that row in the scoreboard. Clicking on the
    link will download/open the file. A column header will appear
    called PDF. If this tag is the config file then the column will
    not appear in the screen. This is useful if you use doxygen to
    create postscript output and convert it to PDF format for
    greater reader usability.

    The <url> contents plus the <pdf> contents are used to
    refer to the document on the webserver. So if the <url> is
    http://deuce.doc.wustl.edu/test/machine_one_wustl_edu and the
    <pdf> is test.pdf, the webserver will look for the file at
    http://deuce.doc.wustl.edu/test/machine_one_wustl_edu/test.pdf

(2) PS - to display a PS file use the <ps> build option in the
    configuration file for the scoreboard.pl script. For example,
    <ps>test.ps</ps> will cause a hyperlink to appear called
    'ps' to appear on that row in the scoreboard. Clicking on the
    link will download/open the file. A column header will appear
    called PS. If this tag is the config file then the column will
    not appear on the screen. This is useful if you use doxygen to
    create postscript output.

    The <url> contents plus the <ps> contents are used to
    refer to the document on the webserver. So if the <url> is
    http://deuce.doc.wustl.edu/test/machine_one_wustl_edu and the
    <ps> is test.ps, the webserver will look for the file at
    http://deuce.doc.wustl.edu/test/machine_one_wustl_edu/test.ps

(3) HTML - to display a HTML file use the <html> build option in the
    configuration file for the scoreboard.pl script. For example,
    <html>test.html</html> will cause a hyperlink to appear called
    'html' to appear on that row in the scoreboard. Clicking on the
    link will download/open the file. A column header will appear
    called HTML. If this tag is not in the config file then the
    column will not appear on the screen. This is useful if you
    use doxygen to create html output.

(4) SNAPSHOT - for users of automake and autoconf, to display a
    created snapshot of the present code, use the
    <snapshot>test.tar.gz</snapshot>. This will cause a hyperlink
    to appear called 'snapshot' to appear on that row in the
    scoreboard. Clicking on the link will download/open the file.
    A column header will appear called SNAPSHOT. If this tag is
    not in the config file then the column will not appear on the
    screen for that row. This is useful if you want to release
    a snapshot of the CVS repository ready for a user to download
    and run "./configure".

(5) The time of last build on the scoreboard is colored to show
    when a build is late. Non-late builds are white, late builds
    are orange, and very late builds are red.  The defaults for
    orange and red are 24 and 48 hours respectively.  These are
    based on the assumption that builds will run daily or less,
    perhaps continuously.  The values for orange and red can be
    changed for individual builds with the parameters <orange>
    hours </orange> and <red> hours </red>.

(6) A schedule file is available for less than daily builds.  This
    file has an .ini format.  Build names are specified in square
    brackets and parameters follow.  The one the scoreboard uses
    is "runon" with a list of days (date abbrevs).  For example,
      [build_xyz]
      runon Mon Thu
    thus, build_xyz will not be considered late until the current
    time passes the next build by the amount specified in orange and
    red plus the number of days between the last and next builds.
    This feature is activated by the presence of the -s <file>
    switch.

(7) There are situations where you may have a number of system that
    share a single data store, via NFS and/or SMB.  In this case you
    may wish to have the machine running scoreboard.pl do some of the
    function normally done by autobuild.pl, speciffically the
    maintenance of the output logs.  This can be accomplished with
    the -c option.  When this option is specified the directory
    specified in the required -d option is maintained by the
    scoreboard program.  This allows, for example, the autobuild to
    specify only "process_logs move" and the remainder, "prettify,
    clean, and index" will be done by the scoreboard.  It also allows
    the faster relative refs to be used as url or skipped entirely,
    if the name of the build matches the name of the sub-directory.

(8) To retain the full capabilities of autobuild.pl in the situation
    above(7).  The KEEP parameter was added to the scoreboard.  The
    value for keep can be specified globally with the -k switch to
    scoreboard.pl or for individual builds with the <keep> number
    </keep>.

    This parameter also works with the existing (no -c) mode and
    applies to the number of items kept in the cache.

    The default for keep is 5.

(9) To help identify builds on the scoreboard there is a "Build
    Sponsor" column.  This consists of two parts the name of the
    sponsor defined by <build_sponsor> name </build_sponsor> and
    a url defined by <build_sponsor_url> url </build_sponsor_url>
    that would be displayed when the name is clicked.

(10) To optimize the speed of scoreboard generation and (greatly)
     decrease diskspace requirements one could use the -b option.
     When specified the scoreboard script will not create local
     cache copies of build logfiles (raw and prettified) but
     will use build URL based links throughout *unless* a build
     configuration explicitly specifies that the local cache must
     be used through the definition of a '<cache/>' tag in the
     '<build>' section for that build.

USING Cygwin on Windows
=======================

On Windows we need to have access to several unix like tools, the
easiest is to use Cygwin. The following steps describe how to obtain
Cygwin and install it.

* Go to www.cygwin.com and download the setup.exe from the page and
  install it on the desktop

* Start setup.exe from the desktop, press next, another time next, root
  directory is ok by default, change local package directory to
  c:\cygwin\download, select how to connect to the internet, select a
  download site in your area. Now the package selection is shown, press
  the view button once, besides the default packages also select cvs,
  openssh, rsync and svn and then press next to install everything.
  Let the installer create the desktop icon.

* Start cygwin using the desktop icon and do a ssh to another server and
  close the cygwin shell again. Then copy the ssh keys from another system
  to c:\cygwin\home\<user>\.ssh.

* For perl we prefer active state perl from www.activestate.com

* Then have a look at other build files how to setup a config file.

MISCELLANEOUS NOTES
===================

(1) check_compiler  ->  print out the version of the compiler being used

    For example:

<command name="check_compiler" options="gcc" />

    The values which can be specified in the options field are in
    the Run() method of autobuild/command/check_compiler.pm

(2) configs  ->  Specify a special configuration for auto_run_tests.pl

    For example:

<variable name="configs"  value="Linux ST" />

   Will invoke: auto_run_tests.pl -Config Linux -Config ST

(3) print_os_version -> print out some information about the operating system
                        being used for the build

<command name="print_os_version" />

   If you run on Windows with Cygwin or MingW and you have uname installed
   you can printed os version and uname results with

<command name="print_os_version" options="useuname" />

(3) print_perl_version -> print out some information about the perl version
                          being used for the build

<command name="print_perl_version" />

(4) print_make_version -> print out some information about the make version
                          being used for the build

<command name="print_make_version" />

E-MAIL NOTIFICATION
===================

Preliminary e-mail notification support has been added.  It is still a work in
progress.  Unix supported only for now, NT support on the way.

(1)  Add the following lines to the configuration file:

     <variable name="MAIL_ADMIN"  value="[email protected]" />
     <variable name="SCOREBOARD_URL"  value="http://mydomain.com/my_scoreboard_url" />

     When build errors are detected, an e-mail will be sent to [email protected],
     showing an abbreviated list of errors, and referring the recipient to
     look at SCOREBOARD_URL for a full list of the errors.


     Alternatively, if you would like the build errors to be sent to a group of
     people, use the following line instead of MAIL_ADMIN above:

     <variable name="MAIL_ADMIN_FILE"  value="C:/foo/mail_map.txt" />

     The MAIL_ADMIN_FILE variables point to a file that contains the email addresses
     of people who should notified by email when an error occurs. Note that file
     should have one email address per line.

(2)  For Windows NT only, you also need to add the domain name of the SMTP
     server used to send outgoing mail.  You do not need to add this line on
     UNIX platforms:

     <variable name="MAIL_ADMIN_SMTP_HOST"  value="smtphostame.mydomain.com" />

NOTE:
If you'd like to change the email address of autobuild displayed to users who
recieve email notification, add and modifiy the line below:

<variable name="MAIL_SENDER_ADDRESS"  value="[email protected]" />
*/