EOExecutor doesn't return WorkflowResults after execution of workflows about eo-learn HOT 11 CLOSED

Hi @divyanipatil

Are you running the Slovenia land use land cover example?

from eo-learn.

from eo-learn.

Dear @DanielMoraite

I'm working at Sinergise with the EO Research team, where we are also developing and maintaining eo-learn.

If you want to include Planet data that you already have at hand, the best way would be to look at some EOTask examples and create your own EOTask that appropriately loads the data that you have.

Cheers!

from eo-learn.

Hi @mlubej
I checked the Slovenia land use land cover example, but as I've now added a description above, saving and loading from a disk is not feasible in my case as the number of EOPatches is huge. I'd prefer getting final results from the EOExecutor itself.

Thanks,
Divyani

from eo-learn.

Hi @mlubej
I checked the Slovenia land use land cover example, but as I've now added a description above, saving and loading from a disk is not feasible in my case as the number of EOPatches is huge. I'd prefer getting final results from the EOExecutor itself.

Thanks,
Divyani

Dear @divyanipatil,

the EOExecutor is moody sometimes, or at least this is what we noticed. You could try executing the last processing chain in the standard workflow and not with the executor, in the same way as the workflows before, in the same notebook.

See also this comment in #73

from eo-learn.

Hi @mlubej
I think there's some miscommunication. I was wondering why the return type of EOExecutor.run() is None. This specific use case of mine requires run() to return the WorkflowResult object obtained from execution of the Workflow.

Currently, this is how the method looks:

 def _execute_workflow(cls, process_args):
    workflow, input_args, log_path = process_args

    if log_path:
        logger = logging.getLogger()
        logger.setLevel(logging.DEBUG)
        handler = cls._get_log_handler(log_path)
        logger.addHandler(handler)

    stats = {'start_time': dt.datetime.now()}
    try:
        _ = workflow.execute(input_args, monitor=True)
    except BaseException:
       stats['error'] = traceback.format_exc()
    stats['end_time'] = dt.datetime.now()

    if log_path:
        handler.close()
        logger.removeHandler(handler)
    return stats

And this is how I require it to be:

 def _execute_workflow(cls, process_args):
    workflow, input_args, log_path = process_args

    if log_path:
        logger = logging.getLogger()
        logger.setLevel(logging.DEBUG)
        handler = cls._get_log_handler(log_path)
        logger.addHandler(handler)

    stats = {'start_time': dt.datetime.now()}
    try:
        result = workflow.execute(input_args, monitor=True)
    except BaseException:
       stats['error'] = traceback.format_exc()
    stats['end_time'] = dt.datetime.now()

    if log_path:
        handler.close()
        logger.removeHandler(handler)
    return result, stats

from eo-learn.

@AleksMat was there a specific reason for the above implementation of EOExecutor? I also think that getting the resulting WorkflowResults would be good :)

@divyanipatil It seems that you already exactly know what you want, so it would be great if you could do a pull request and contribute yourself!

Thanks!

from eo-learn.

@AleksMat was there a specific reason for the above implementation of EOExecutor? I also think that getting the resulting WorkflowResults would be good :)

This is exactly what i need to know, any specific reason the current implementation of EOExecutor. If not, I could go ahead and raise a PR for same.
Thank you so much!

from eo-learn.

Please, go ahead and create a PR! If you need it, then there is value in it. We can discuss the details later.

Thank you so much for contributing!

from eo-learn.

@divyanipatil @mlubej An alternative perhaps is to add at the end of the EOWorkflow a task that will either:

• delete everything except results in the EOPatch and save it to disk
• move results from an EOPatch to a new one and save it to disk

In both cases you'll end up with a light EOPatch that will contain only the feature that you're interested in (i.e. results of a classification) and will not require large amounts of storage.

from eo-learn.

In both cases you'll end up with a light EOPatch that will contain only the feature that you're interested in (i.e. results of a classification) and will not require large amounts of storage.

Hi @azupanc
As my use-case is semantic segmentation, the result would be an array of values, not a single classification category, hence storing and loading from disk is not feasible.

from eo-learn.