Enable downloading assets within a pytest environment about radiant-mlhub HOT 5 CLOSED

Thanks @guidorice it looks like your fix is working, just tested it

from radiant-mlhub.

I thought of a simple fix for this, which is to add a new environment variable to our GitHub action for CI runner. Call it MLHUB_CI. The hilighted code in the description would then become:

        if 'PYTEST_CURRENT_TEST' in os.environ and 'MLHUB_CI' in os.environ:
            ...

This should free up the pytest context to work normally for users of radiant_mlhub. Does this sound OK @kbgg @mirob9363 ?

from radiant-mlhub.

Re: the underlying issue with vcr.py and multithreading, there are a bunch of related github issues, but these comments seem pretty relevant, from a former maintainer:

but this is the one area of code I would say I am probably still the resident expert in, and I really don't think that vcrpy is EVER going to be multi threaded (at least with its current design). kevin1024/vcrpy#212 (comment)

from radiant-mlhub.

@guidorice thanks for taking a look at this! Does the above comment indicate that it is just not a good idea to run catalog_downloader.py within a pytest environment?

from radiant-mlhub.

@rbavery I believe your use case should be fine (unless you are using vcr.py in your own code). Here is a feature branch, would you be able to test it out by installing it manually?

https://github.com/radiantearth/radiant-mlhub/tree/Fix/148-pytest-compatibility

I put together a little test and it seems to work for me by running pytest:

from radiant_mlhub import Dataset
from pathlib import Path


def test_mlhub_and_pytest():
    ds = Dataset.fetch('su_sar_moisture_content_main')
    ds.download()
    path = Path.cwd() / 'su_sar_moisture_content_main'
    csv_files = list(path.glob('**/*.csv'))
    assert len(csv_files) == 2616

from radiant-mlhub.