Git Product home page Git Product logo

scoop's Introduction

SCOOP logo

SCOOP (Scalable COncurrent Operations in Python) is a distributed task module allowing concurrent parallel programming on various environments, from heterogeneous grids to supercomputers. Its documentation is available on http://scoop.readthedocs.org/ .

Philosophy

SCOOP was designed from the following ideas:

  • The future is parallel;
  • Simple is beautiful;
  • Parallelism should be simpler.

These tenets are translated concretely in a minimum number of functions allowing maximum parallel efficiency while keeping at minimum the inner knowledge required to use them. It is implemented with Python 3 in mind while being compatible with Python 2.6+ to allow fast prototyping without sacrificing efficiency and speed.

Some comments we received on SCOOP:

Features

SCOOP features and advantages over futures, multiprocessing and similar modules are as follows:

  • Harness the power of multiple computers over network;
  • Ability to spawn multiple tasks inside a task;
  • API compatible with PEP-3148;
  • Parallelizing serial code with only minor modifications;
  • Efficient load-balancing.

Anatomy of a SCOOPed program

SCOOP can handle multiple diversified multi-layered tasks. With it, you can submit your different functions and data simultaneously and effortlessly while the framework executes them locally or remotely. Contrarily to most multiprocessing frameworks, it allows to launch subtasks within tasks.

http://scoop.readthedocs.org/en/latest/_images/introductory_tree.png

Through SCOOP, you can execute simultaneously tasks that are different by nature, shown by the task color, or different by complexity, shown by the task radius. The module will handle the physical considerations of parallelization, such as task distribution over your resources (load balancing), communications, etc.

Applications

The common applications of SCOOP consist but is not limited to:

  • Evolutionary Algorithms
  • Monte Carlo simulations
  • Data mining
  • Data processing
  • Graph traversal

Citing SCOOP

Authors of scientific papers including results generated using SCOOP are encouraged to cite the following paper.

{{{ @inproceedings{SCOOP_XSEDE2014, title={Once you SCOOP, no need to fork}, author={Hold-Geoffroy, Yannick and Gagnon, Olivier and Parizeau, Marc}, booktitle={Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment}, pages={60}, year={2014}, organization={ACM} } }}}

Useful links

You can download the latest stable version, check the project documentation, post to the mailing list or submit an issue if you've found one.

scoop's People

Contributors

cmd-ntrf avatar fzesch avatar joernhees avatar leconteur avatar maharjun avatar mznkcyr avatar ntosme avatar soravux avatar timgates42 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.