doneholmes / nuget-package-scanner Goto Github PK

This call occasionally fails with

Traceback (most recent call last): File "C:\Python38\lib\runpy.py", line 193, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Python38\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "E:\repositories\nuget-package-scanner\nuget_package_scanner\__main__.py", line 15, in <module> loop.run_until_complete(app.run(org, token, output)) File "C:\Python38\lib\asyncio\base_events.py", line 616, in run_until_complete return future.result() File "E:\repositories\nuget-package-scanner\nuget_package_scanner\app.py", line 151, in run package_containers: List[PackageContainer] = await build_org_report(org, token) File "E:\repositories\nuget-package-scanner\nuget_package_scanner\app.py", line 95, in build_org_report await wait_or_raise([net_core_task, net_framework_task]) File "E:\repositories\nuget-package-scanner\nuget_package_scanner\async_utils.py", line 14, in wait_or_raise raise d.exception() File "E:\repositories\nuget-package-scanner\nuget_package_scanner\github_search.py", line 115, in search_package_configs return await self.search_github_code(f'package+org:{org}+filename:packages.config', limit) File "E:\repositories\nuget-package-scanner\nuget_package_scanner\github_search.py", line 103, in search_github_code await wait_or_raise(tasks) File "E:\repositories\nuget-package-scanner\nuget_package_scanner\async_utils.py", line 14, in wait_or_raise raise d.exception() File "E:\repositories\nuget-package-scanner\nuget_package_scanner\github_search.py", line 74, in __process_search_page details = await self.get_request_as_json(details_url) File "E:\repositories\nuget-package-scanner\nuget_package_scanner\github_search.py", line 48, in get_request_as_json return await response.json() File "E:\repositories\nuget-package-scanner\env\lib\site-packages\aiohttp\client_reqrep.py", line 1021, in json await self.read() File "E:\repositories\nuget-package-scanner\env\lib\site-packages\aiohttp\client_reqrep.py", line 973, in read self._body = await self.content.read() File "E:\repositories\nuget-package-scanner\env\lib\site-packages\aiohttp\streams.py", line 358, in read block = await self.readany() File "E:\repositories\nuget-package-scanner\env\lib\site-packages\aiohttp\streams.py", line 380, in readany await self._wait('readany') File "E:\repositories\nuget-package-scanner\env\lib\site-packages\aiohttp\streams.py", line 296, in _wait await waiter aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed

There's likely something missing or off in how i'm handling errors/timeouts since updating to async calls.

I occasionally get errors in the following places when hitting the Github API..

Found an issue that seems related (curiously also w/ calling the Github API)

Since these are intermittent errors where it's often the same problem files (but not always), My assumption is that i'm tripping on Possibility 2 listed in the aiohttp docs.