icrar / boinc-magphys Goto Github PK

As shown in the log below, the Boinc client (on debian) makes two scheduler requests in quick succession getting the same task twice. It then attempts to run the same task twice. Likely solution is to configure the dispatching of scheduler requests so that they are not sent until the first task is completed ? See <next_rpc_delay/> in http://boinc.berkeley.edu/trac/wiki/ProjectOptions.

18-Jan-2012 13:35:57 [---] Fetching configuration file from http://www.boinc-magphys.org/magphys/get_project_config.php
18-Jan-2012 13:36:13 [magphys] Master file download succeeded
18-Jan-2012 13:36:18 [magphys] Sending scheduler request: Project initialization.
18-Jan-2012 13:36:18 [magphys] Requesting new tasks
18-Jan-2012 13:36:19 [magphys] Scheduler request completed: got 1 new tasks
18-Jan-2012 13:36:19 [magphys] [task_debug] result state=NEW for test_0 from handle_scheduler_reply
18-Jan-2012 13:36:20 [magphys] [task_debug] result state=FILES_DOWNLOADING for test_0 from CS::update_results
18-Jan-2012 13:36:21 [magphys] Started download of wrapper
18-Jan-2012 13:36:21 [magphys] Started download of fit_sed
18-Jan-2012 13:36:22 [magphys] Finished download of wrapper
18-Jan-2012 13:36:22 [magphys] Finished download of fit_sed
18-Jan-2012 13:36:22 [magphys] Started download of job.xml
18-Jan-2012 13:36:22 [magphys] Started download of zlibs.dat
18-Jan-2012 13:36:23 [magphys] Finished download of job.xml
18-Jan-2012 13:36:23 [magphys] Finished download of zlibs.dat
18-Jan-2012 13:36:23 [magphys] Started download of filters.dat
18-Jan-2012 13:36:23 [magphys] Started download of observations.dat
18-Jan-2012 13:36:24 [magphys] Finished download of filters.dat
18-Jan-2012 13:36:24 [magphys] Finished download of observations.dat
18-Jan-2012 13:36:24 [magphys] Started download of infrared_dce08_z0.4600.lbr
18-Jan-2012 13:36:24 [magphys] Started download of starformhist_cb07_z0.4600.lbr
18-Jan-2012 13:36:26 [magphys] Finished download of infrared_dce08_z0.4600.lbr
18-Jan-2012 13:36:26 [magphys] Finished download of starformhist_cb07_z0.4600.lbr
18-Jan-2012 13:36:26 [magphys] Started download of OptiLIB_cb07.bin
18-Jan-2012 13:36:26 [magphys] Started download of OptiLIBis_cb07.bin
18-Jan-2012 13:36:29 [magphys] Finished download of OptiLIB_cb07.bin
18-Jan-2012 13:36:29 [magphys] Started download of InfraredLIB.bin
18-Jan-2012 13:36:29 [magphys] Sending scheduler request: To fetch work.
18-Jan-2012 13:36:29 [magphys] Requesting new tasks
18-Jan-2012 13:36:30 [magphys] Scheduler request completed: got 1 new tasks
18-Jan-2012 13:36:30 [magphys] [task_debug] result state=NEW for test_1 from handle_scheduler_reply

Automated scripts for creating the Magphys Boinc project, building binaries for various platforms, update versions and create work units.

Create a website with information about Magphys and BOINC.
BOINC project comes with a web-site template in PHP which will need to be modified/rewritten.

Create a work generator that will make jobs available for BOINC clients that attach to the Magphys project.
Jobs could be broken down by sets of galaxy observations. For example, if you have observations from 100 galaxies, there could be 10 jobs created each dealing with a different set of 10 galaxies. Also, for validation purposes we will want multiple clients to work on the same job - so, you could have 100 jobs - each of the 10 galaxy observations repeated 10 times. This will depend a lot on the final form that the JHU Magphys code takes.

There are currently no Unit Tests around wrapper.cpp.

Add a comprehensive set of Unit Tests around wrapper.cpp.

If this is done before turning the build into a one-step build, add instructions to README.markdown for a linux build.

If this is done after turning the build into a one-step build, add Linux option for the one-step build.

Once the result files have passed through the validator they need to be stored in the database.

ICRAR wants to be able to run Magphys fortran code using BOINC infrastructure. BOINC provides a wrapper that allows running external programs without modification in their infrastructure. Setup a BOINC server and test this approach with current version of Magphys code.

The current Magphys Fortran code is being re-work by Institut D'Astrophysique de Paris to include some options required by JHU (Johns Hopkins - see Kevin Vinsen for details). Depending on the final form that it takes, changes have to be made to run with the wrapper:

• inputs and outputs
• dealing with large data files
  For the spike, I have modified the current version of the code to read inputs from command line params. Also, for the spike I restricted it to working with one set of galaxy observations.

Setup a BOINC server - including a puppet script to automate this process. A set of Rake tasks exist to create the project and build publish versions of the app for linux platform (see #2).

Create a validator program that can look at the results submitted by BOINC Magphys clients and compare for correctness, throw out incorrect/incomplete results etc.

Use DocTest if possible in the Python code (http://docs.python.org/library/doctest.html)

The build instructions seem fairly straight-forward, but it would be nice for the build instructions to be a single command.

Rake is being used elsewhere in this project, so it could be used for the main build as well. Or the standard CPP configure/make could be used, though this would require a separate approach for Windows.