biglocalnews / bln-etl Goto Github PK

Basic wrapper class that cribs most frequently used bits from the BLN sdk, and provides a simple Project and file-oriented interface for managing files on the platform.

• Client.user_projects - returns Project instances
• Client.open_projects - returns Project instances
• Project.get(id='foo')
• Project.create(name='bar')
• Project.upload_files (or just File.upload?)
• Project.files
• Project.slug (first 15 of UUID and first 20 of project name)
• Test coverage for
  • Project.files
  • File.delete

Project.update should enable updating of optional metadata such as project description.

Should create proper docstrings and update API usage docs to list all options.

• Add Apache 2.0 license
• Set GH repo to public
• Update README pip install instructions

File.download needs to be implemented.

Using @pytest.mark.vcr() to record web interactions for testing produces the below error related to creating file upload URIs. We've sidestepped by marking such tests as a "webtest" and allowing them to hit the live platform, but it's worth investigating more deeply for a potential fix that avoids hitting the live platform.

    def test_project_upload_files():
        # NOTE: This is a live webtest to sidestep headaches
        # related to handling errors raised during the process
        # of generating file upload URIs (KeyError: 'createFileUploadUri')
        kwargs = {
            'is_open': False ,
            'description': 'This is a test project.'
        }
        uuid = '<FAKE_UUID>'
        project = Project.get(uuid, TOKEN)
        to_upload = [
            fixture_path('test.csv'),
            fixture_path('test2.csv')
        ]
        project.upload_files(to_upload)
        expected = ['test.csv', 'test2.csv']
        actual = [f.name for f in project.files]
>       assert project.files == expected

Basic wrapper class that shells out to git cli utilities. At outset, just need the basics to clone and update:

• Repository('/local/path/to/repo')
• Repository.clone
• Repository.pull

It seems like a good thing to do long term, and might be required by tickets like this.

The current webtests assume the existence of (and access to) a Test project created by me. This is gross, since other folks can't use it without being added to the project as admins and the tests will break if the project gets deleted (its UUID is hard-coded in several tests).

Update the test setup with a module-scope fixture that bootstrap and tear down a new platform project on each run of webtests.