Host files in multiple S3 regions about gluon-nlp HOT 13 CLOSED

That works. I had some speed issues but in the end a VPN is at fault. Sorry about the noise.

from gluon-nlp.

I always keep the global s3 in sync with the China s3 mirror

from gluon-nlp.

the global s3 is backed with s3-accelerate, which already utilizes all cloudfront edge locations.

from gluon-nlp.

Try export MXNET_GLUON_REPO=https://apache-mxnet.s3.cn-north-1.amazonaws.com.cn/

from gluon-nlp.

Thanks, it's great you already thought about this issue and that there is already a solution!
What do you think about putting that behind Cloudfront. If I understand correctly we may use a single link then and the correct region will be selected automatically based on the source of the request.

from gluon-nlp.

Due to regulation in China, it requires a business entity and registration with the government to set up CDN in China.

from gluon-nlp.

Also, China AWS accounts are separate from global AWS accounts, so we would at least need to cloudfront end points. Then we would need DNS magic to point to the right endpoint, so I haven't gone through that route yet.

from gluon-nlp.

Btw, the China S3 mirror also slow in Asia-Pacific region outside of mainland (eg. HK).

from gluon-nlp.

@leezu how's the speed with the global repo? Tokyo/Singapore edges might be fast.

from gluon-nlp.

Is this issue resolved?

from gluon-nlp.

I think the current S3 Accelerate approach is sufficient. AWS recently added CloudFront endpoints in Hong Kong and mainland China.
S3 Accelerate doesn't make use of the endpoints in mainland China though, but maybe they will in the future.

Do we have any reports of trouble downloading in China? I'll close this for now, but let's reopen once we have more evidence of the issue.

from gluon-nlp.

@leezu they won't. China is a separate region entirely and there's no data exchange with the global region except on the public network.

from gluon-nlp.

Cloudflare does, so I think Cloudfront will need to do eventually as well. Cloudflare advertises this feature heavily https://www.cloudflare.com/network/china/

from gluon-nlp.