Comments (2)
I changed the code to this:
async def _url_open( # pylint: disable=too-many-branches
self,
method,
region,
bucket_name=None,
object_name=None,
body=None,
headers=None,
query_params=None,
session=None,
):
"""Execute HTTP request."""
credentials = self._provider.retrieve() if self._provider else None
url = self._base_url.build(
method,
region,
bucket_name=bucket_name,
object_name=object_name,
query_params=query_params,
)
headers, date = await self._build_headers(
url.netloc,
headers,
body,
credentials
)
if credentials:
headers = sign_v4_s3(
method,
url,
region,
headers,
credentials,
headers.get("x-amz-content-sha256"),
date,
)
if session is None:
async with aiohttp.ClientSession() as session:
return await self._do_req_in_session(session, method, url, body, headers)
else:
return await self._do_req_in_session(session, method, url, body, headers)
async def _do_req_in_session(self, session, method, url, body, headers):
response = await session.request(
method,
urlunsplit(url),
data=body,
headers=headers)
# if should_attach_finalizer:
# _attach_finalizer(response, session, asyncio.get_running_loop())
if response.status in [200, 204, 206]:
return response
response_data = await response.content.read()
content_types = response.headers.get("content-type", "").split(";")
if method != "HEAD" and "application/xml" not in content_types:
raise InvalidResponseError(
response.status,
response.headers.get("content-type"),
response_data if response_data else None,
)
if not response_data and method != "HEAD":
raise InvalidResponseError(
response.status,
response.headers.get("content-type"),
None,
)
response_error = S3Error.fromxml(
response, response_data
) if response_data else None
error_map = {
301: lambda: self._handle_redirect_response(
method, bucket_name, response, True,
),
307: lambda: self._handle_redirect_response(
method, bucket_name, response, True,
),
400: lambda: self._handle_redirect_response(
method, bucket_name, response, True,
),
403: lambda: ("AccessDenied", "Access denied"),
404: lambda: (
("NoSuchKey", "Object does not exist")
if object_name
else ("NoSuchBucket", "Bucket does not exist")
if bucket_name
else ("ResourceNotFound", "Request resource not found")
),
405: lambda: (
"MethodNotAllowed",
"The specified method is not allowed against this resource",
),
409: lambda: (
("NoSuchBucket", "Bucket does not exist")
if bucket_name
else ("ResourceConflict", "Request resource conflicts"),
),
501: lambda: (
"MethodNotAllowed",
"The specified method is not allowed against this resource",
),
}
if not response_error:
func = error_map.get(response.status)
code, message = func() if func else (None, None)
if not code:
raise ServerError(
f"server failed with HTTP status code {response.status}",
response.status
)
response_error = S3Error(
code,
message,
url.path,
response.headers.get("x-amz-request-id"),
response.headers.get("x-amz-id-2"),
response,
bucket_name=bucket_name,
object_name=object_name,
)
if response_error.code in ["NoSuchBucket", "RetryHead"]:
self._region_map.pop(bucket_name, None)
raise response_error
then, it worked well.
from miniopy-async.
I've tried your modified code.
When using fget_object
, your code cannot download anything.
Unclosed connector
is a warning, not critical, but annoy.
However, it's hard to remove this warning while keeping every function workable.
from miniopy-async.
Related Issues (14)
- Is there any more detailed Useage? HOT 2
- Custom metadata
- async def _get_region blocks forever HOT 2
- session argument ignored in method _execute HOT 3
- Can't import aiostream HOT 2
- Checksum in object stat/get HOT 13
- Add snowball object HOT 3
- `get_object` has required session parameter but other methods hasn't session parameter. Why for? HOT 3
- [SSLCertVerificationError]: How do I ignore checking a self-signed certificate HOT 4
- Reorganize and format the code HOT 1
- update to the latest version of official minio python driver (minio-py) HOT 3
- support async method read() in read_part_data HOT 1
- fget_object downloads zero size files HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from miniopy-async.