Git Product home page Git Product logo

blaster's Introduction

Hello!

👋 My name is Ryan and I am a Principal Software Quality Engineer at Red Hat working on Ansible Automation Platform.

  • I have a bachelors degree in Computer Science
  • I have over 11 years of industry experience
  • I am highly experienced in
    • Programming / automation
    • Debugging / troubleshooting
    • Problem solving
    • Designing and building applications / frameworks
    • Containers / linux
    • Testing / writing tests
    • Continuous integration / continuous development (CI/CD)
  • I am always looking to learn new skills to advance my technical toolbox

⚡ When I am not working with technology, I enjoy

  • 🏠 Spending time with my family
  • 😺 Hanging out with my cat
  • 🚙 Auto (car) detailing
  • ⛳️ Golf
  • 💪 Biking, hiking, running
  • 🎮 Video games

blaster's People

Contributors

ryankwilliams avatar

Stargazers

avatar avatar

Watchers

avatar avatar

Forkers

dannyb48

blaster's Issues

RFE: Be able to specify task timeout so that long running tasks to don't run indefinitely

Hi Ryan,

We've had a couple different teams request the capability to have Carbon tasks provide an optional parameter for timeouts so that long running tasks don't just run indefinitely.

My thinking is that this belongs in blaster since carbon is using it as a task scheduler and queue. My initial thinking is that timeout could be a part of the task definition like so

 {
            'name': 'House 1',
            'task': House,
            'methods': [
                'foundation',
                'frame',
                'roof',
                'furnish',
                'enjoy'
            ],
            'timeout': 60
            'style': 'contemporary'
        }

Wanted to see what your thoughts were and if we could collaborate on this?

Limit the number of processes per blast off

With the current release of blaster, if you give 50 tasks to be run. It will create 50 processes and then begin to run them all. This is good since you will get your results much quicker. The down side is the system blaster is running on may see a performance impact.

The feature to be added here is to allow blaster to only have X amount of processes to process a list of tasks. Blaster will be smart enough to decide how many processes it can start.

Support only Python >= 3.6

Python 2.7 will be ending support starting in 2020. This issue is to update blaster to drop Python 2.7 support. A future release of blaster (TBD) will be the final supported one with Python 2.7 support. All other releases will be only Python >= 3.6 support.

Updated README

The main purpose of this issue is to add to the README.md. Currently the README.md has minimal information. The content to be added should contain the following: a detailed description of blaster, the purpose for blaster and how blaster can help running tasks concurrently to save time.

Task failed but does not return traceback

With the latest release of blaster v0.1.3. If a task method call fails, it captures the traceback and sets it. Just that data is not being returned back to the user at the end of blastoff method.

2017-08-17 10:30:34,187 INFO [blaster.blast.blastoff:47] ..............................
2017-08-17 10:30:34,188 INFO [blaster.blast.blastoff:48] START: BLASTER PREPARATION
2017-08-17 10:30:34,188 INFO [blaster.blast.blastoff:49] ..............................
2017-08-17 10:30:34,189 INFO [blaster.blast.blastoff:68] 1. TASK ~ House, METHODS: ['foundation', 'frame', 'roof', 'furnish', 'enjoy']
2017-08-17 10:30:34,190 INFO [blaster.blast.blastoff:68] 2. TASK ~ House, METHODS: ['foundation', 'frame', 'roof', 'furnish', 'post_build_tasks', 'enjoy']
2017-08-17 10:30:34,190 INFO [blaster.blast.blastoff:68] 3. TASK ~ House, METHODS: ['foundation', 'frame', 'roof', 'furnish', 'post_build_tasks', 'enjoy']
2017-08-17 10:30:34,191 INFO [blaster.blast.blastoff:68] 4. TASK ~ House, METHODS: ['foundation', 'frame', 'roof', 'furnish', 'post_build_tasks', 'enjoy']
2017-08-17 10:30:34,191 INFO [blaster.blast.blastoff:68] 5. TASK ~ House, METHODS: ['foundation', 'frame', 'roof', 'furnish', 'post_build_tasks', 'enjoy']
2017-08-17 10:30:34,191 INFO [blaster.blast.blastoff:73] ..............................
2017-08-17 10:30:34,191 INFO [blaster.blast.blastoff:74] END: BLASTER PREPARATION
2017-08-17 10:30:34,192 INFO [blaster.blast.blastoff:75] ..............................
2017-08-17 10:30:34,193 INFO [blaster.blast.blastoff:81] ..............................
2017-08-17 10:30:34,193 INFO [blaster.blast.blastoff:82] START: BLASTER BLAST OFF
2017-08-17 10:30:34,193 INFO [blaster.blast.blastoff:83] ..............................
2017-08-17 10:30:34,193 INFO [blaster.blast.blastoff:92] Total processes used by blaster is 5.
2017-08-17 10:30:34,195 INFO [blaster.foundation:21] Building the foundation for contemporary house..
2017-08-17 10:30:34,198 INFO [blaster.foundation:21] Building the foundation for colonial house..
2017-08-17 10:30:34,198 INFO [blaster.foundation:21] Building the foundation for cape house..
2017-08-17 10:30:34,198 INFO [blaster.foundation:21] Building the foundation for ranch house..
Traceback (most recent call last):
  File "/home/rywillia/github/blaster/blaster/engine/processor.py", line 49, in run
    getattr(task_obj, method)()
  File "/home/rywillia/github/blaster/examples/build_houses.py", line 23, in foundation
    raise Exception('Cannot build!')
Exception: Cannot build!
2017-08-17 10:30:34,199 INFO [blaster.foundation:21] Building the foundation for split house..
2017-08-17 10:30:35,197 INFO [blaster.frame:27] Frame the house for contemporary house..
2017-08-17 10:30:35,200 INFO [blaster.frame:27] Frame the house for colonial house..
2017-08-17 10:30:35,200 INFO [blaster.frame:27] Frame the house for cape house..
2017-08-17 10:30:35,200 INFO [blaster.frame:27] Frame the house for split house..
2017-08-17 10:30:36,198 INFO [blaster.roof:31] Roof the house for contemporary house..
2017-08-17 10:30:36,201 INFO [blaster.roof:31] Roof the house for cape house..
2017-08-17 10:30:36,201 INFO [blaster.roof:31] Roof the house for colonial house..
2017-08-17 10:30:36,201 INFO [blaster.roof:31] Roof the house for split house..
2017-08-17 10:30:37,200 INFO [blaster.furnish:35] Furnish the house for contemporary house..
2017-08-17 10:30:37,202 INFO [blaster.furnish:35] Furnish the house for cape house..
2017-08-17 10:30:37,202 INFO [blaster.furnish:35] Furnish the house for split house..
2017-08-17 10:30:37,203 INFO [blaster.furnish:35] Furnish the house for colonial house..
2017-08-17 10:30:38,201 INFO [blaster.enjoy:39] Enjoy your new contemporary house :)
2017-08-17 10:30:38,203 INFO [blaster.post_build_tasks:43] Perform post building tasks for split house..
2017-08-17 10:30:38,203 INFO [blaster.post_build_tasks:43] Perform post building tasks for cape house..
2017-08-17 10:30:38,204 INFO [blaster.post_build_tasks:43] Perform post building tasks for colonial house..
2017-08-17 10:30:39,204 INFO [blaster.enjoy:39] Enjoy your new cape house :)
2017-08-17 10:30:39,204 INFO [blaster.enjoy:39] Enjoy your new split house :)
2017-08-17 10:30:39,205 INFO [blaster.enjoy:39] Enjoy your new colonial house :)
2017-08-17 10:30:40,208 INFO [blaster.blast.blastoff:148] Total blast off duration: 0h:0m:6s
2017-08-17 10:30:40,208 INFO [blaster.blast.blastoff:150] ..............................
2017-08-17 10:30:40,209 INFO [blaster.blast.blastoff:151] END: BLASTER BLAST OFF
2017-08-17 10:30:40,209 INFO [blaster.blast.blastoff:152] ..............................
[   {   '_id': '2ba7f0dc-6cf7-4db8-8b1f-762ec351709c',
        'methods': [   'foundation',
                       'frame',
                       'roof',
                       'furnish',
                       'post_build_tasks',
                       'enjoy'],
        'name': 'House 4',
        'status': 1,
        'style': 'ranch',
        'task': <class '__main__.House'>},
    {   'methods': ['foundation', 'frame', 'roof', 'furnish', 'enjoy'],
        'name': 'House 1',
        'status': 0,
        'style': 'contemporary',
        'task': <class '__main__.House'>},
    {   '_id': '60eaa333-6577-4cbc-aac5-1bc339b159e8',
        'methods': [   'foundation',
                       'frame',
                       'roof',
                       'furnish',
                       'post_build_tasks',
                       'enjoy'],
        'name': 'House 5',
        'status': 0,
        'style': 'split',
        'task': <class '__main__.House'>},
    {   '_id': 'df061d03-0080-486e-8379-42359b2134b2',
        'methods': [   'foundation',
                       'frame',
                       'roof',
                       'furnish',
                       'post_build_tasks',
                       'enjoy'],
        'name': 'House 2',
        'status': 0,
        'style': 'cape',
        'task': <class '__main__.House'>},
    {   '_id': '0641fb02-4d22-454d-abd4-63232dca4801',
        'methods': [   'foundation',
                       'frame',
                       'roof',
                       'furnish',
                       'post_build_tasks',
                       'enjoy'],
        'name': 'House 3',
        'status': 0,
        'style': 'colonial',
        'task': <class '__main__.House'>}]

Need to determine why this data is not being returned back properly.

Expand documentation

This issue is to expand the existing documentation (README.md). Maybe look into creating documentation with github pages for better viewing.

Reduce duplicated code in blastoff method

The current blaster blastoff method has duplicate code for handling actions such as: start, wait and process results when handling keyboard interrupts. This issue is to further enhance this method to call another method or function to handle these actions. This way then depending on the exception being handled. It can call the one location of code to run through. It will make the blastoff method more robust.

Reference: https://github.com/rywillia/blaster/blob/master/blaster/blast.py

Save return value from task methods

This issue is to allow blaster to capture the return data from each method called. Currently if a method returns data it is not being capture. This gives users the ability to have the return data of the method in their blaster results.

Expand unit tests

This issue is to expand the unit tests. Need to create more tests to gain better code coverage.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.