scisweeper - scientific parameter sweeper

scisweeper is a utility for parameter sweeps in a scientific environment. By defining the write_input and collect_output function, as well as a link to the executable the users can link scisweeper to their own code. Afterwards scisweeper can execute the parameter sweep either locally using a given number of parallel threads or submit the individual calculations to a queuing system like LFS, MOAB, SGE, SLURM, or TORQUE using the built-in interface to pysqa. After the calculations are finished the results are summarized in a pandas.DataFrame, which makes them easily accessible for machine learning tools like tensorflow or pytorch. Coming from a scientific environment scisweeper was develop with a focus on debugging. Broken jobs can be identified, executed again manually or deleted and in case you need to update the collect_output function there is no need to execute the calculations again, the output can be parsed separatly to update the results in the pandas.DataFrame.

The scisweeper can either be installed via pip using:

pip install scisweeper

Or via anaconda from the conda-forge channel

conda install -c conda-forge scisweeper

For a simple executable like:

#!/bin/bash
awk '{ print $1+$2*$3 }1{print 1}' input_file > output.log

Which is located in your home directory as ~/test.sh a parameter sweep can be conducted using the following steps. Start with importing scisweeper :

from scisweeper import SciSweeper
ssw = SciSweeper()

Define your write_input and collect_output function in a derived class. The important part about these write_input and collect_output function is that all the necessary import statements have to be nested within the function. This might be counterintuitive in the beginning, but is essential as it allows scisweeper to transfer the function to the compute node where the code is going to be executed:

class BashSciSweeper(ssw.job):
    @property
    def executable(self): 
        return 'bash ~/test.sh'
    
    @staticmethod
    def write_input(input_dict, working_directory='.'):
        import os 
        from jinja2 import Template
        template = Template("{{value_1}} {{value_2}} {{value_3}}")
        template_str = template.render(value_1=input_dict['value_1'],
                                       value_2=input_dict['value_2'],
                                       value_3=input_dict['value_3'])
        with open(os.path.join(working_directory, 'input_file'), 'w') as f:
            f.writelines(template_str)

    @staticmethod
    def collect_output(working_directory='.'):
        import os 
        with open(os.path.join(working_directory, 'output.log'), 'r') as f:
            output = f.readlines()
        return {'result': int(output[1])}

And apply this function to our scisweeper instance:

ssw.job_class = BashSciSweeper 

Afterwards we build a list of dictionaries with the input values we want to iterate over:

input_lst = []
for i in range(10):
    for j in range(10):
        for k in range(10):
            input_lst.append({'value_1': i, 'value_2': j, 'value_3': k}) 

Optionally we can define a function to generate custom job names for the individual calculations. This can help identifying broken calculation:

def job_name(input_dict, counter):
    return 'job_' + str(counter) + '_' + str(input_dict['value_1'])

ssw.job_name_function = job_name

Then we execute the caluclation in parallel using twenty cores:

ssw.run_jobs_in_parallel(input_dict_lst=input_lst, cores=20)

And collect the results:

ssw.collect()
ssw.results

If we want to update our output parser, because we want to parse more quantities, we can do so by replacing the interface:

class BashSciSweeper2(ssw.job):
    @staticmethod
    def collect_output(working_directory='.'):
        with open(os.path.join(working_directory, 'output.log'), 'r') as f:
            output = f.readlines()
        return {'result': int(output[0])}

And repeat the steps from above:

ssw.job_class = BashSciSweeper2
ssw.run_collect_output()
ssw.collect()
ssw.results

Or we can identify broken calculations using:

ssw.broken_jobs

For more information feel free to look at the unit tests and the example notebooks. https://github.com/scisweeper/scisweeper/blob/master/notebooks/demo.ipynb

To interface with the queuing system we use pysqa - https://github.com/pyiron/pysqa - which is constructed around the idea that even though modern queuing systems allow for an wide range of different configuration, most users submit the majority of their jobs with very similar parameters. Sample configurations for the specific queuing systems are availabe in the pysqa tests:

• lsf - https://github.com/pyiron/pysqa/tree/master/tests/config/lsf
• moab - https://github.com/pyiron/pysqa/tree/master/tests/config/moab
• SGE - https://github.com/pyiron/pysqa/tree/master/tests/config/sge
• slurm - https://github.com/pyiron/pysqa/tree/master/tests/config/slurm
• torque - https://github.com/pyiron/pysqa/tree/master/tests/config/torque

The scisweeper is released under the BSD license https://github.com/scisweeper/scisweeper/blob/master/LICENSE . It is a spin-off of the pyiron project https://github.com/pyiron/pyiron therefore if you use the scisweeper for your publication, please cite:

@article{pyiron-paper,
  title = {pyiron: An integrated development environment for computational materials science},
  journal = {Computational Materials Science},
  volume = {163},
  pages = {24 - 36},
  year = {2019},
  issn = {0927-0256},
  doi = {https://doi.org/10.1016/j.commatsci.2018.07.043},
  url = {http://www.sciencedirect.com/science/article/pii/S0927025618304786},
  author = {Jan Janssen and Sudarsan Surendralal and Yury Lysogorskiy and Mira Todorova and Tilmann Hickel and Ralf Drautz and Jörg Neugebauer},
  keywords = {Modelling workflow, Integrated development environment, Complex simulation protocols},
}