Consider adding a second IPv6 registry about globalping-hwprobe HOT 36 OPEN

I believe the fix is to use this command instead. dir: is not supported.

skopeo copy docker://globalping/globalping-probe:latest docker-archive:image-contents
//skip tar creation step
docker load < image-contents

from globalping-hwprobe.

lol https://isgithubipv6.live/

from globalping-hwprobe.

Not that I expected anything else.

from globalping-hwprobe.

So what do you think? Docker Hub or something else?

from globalping-hwprobe.

If we had 2 or more mirrors, how would the system decide which one to use? Do we modify the docker config to use different mirrors? Does it use round robin in case of failure? Does lack of ipv6 count?

from globalping-hwprobe.

If you add Docker hub to workflow and it works with multi-arch, it automatically gets mirrored to ECR too. So we could have 3 mirrors in total. But only if it makes sense.
Otherwise I could setup a proxy cache for ghcr

from globalping-hwprobe.

I don't think that's necessary at all. We would just switch everything to the new one and keep the old one working for existing devices.

from globalping-hwprobe.

Your proposal still means pushing to and maintaining 2 registries forever. We might as well include both of them in the config and allow the devices to failover in case one is down or gets blocked by the gov.

from globalping-hwprobe.

Well, it is possible, just doesn't seem too useful. Either way we need to decide on a "primary" that's documented everywhere and that must support both protocols, and should be the first one we try here as well. Then we can possibly try others on failure.

from globalping-hwprobe.

Regardless of the new registry, ghcr.io, is already documented and used everywhere and will need to continue to work forever. So that doesn't change.

The question is how do we add ipv6 support and ideally fix China jsdelivr/globalping-probe#198 in a single change.

Adding a list of mirrors to the docker config sounds like the best solution. Now if the mirrors are real registries or just proxy caches thats a separate issue.

from globalping-hwprobe.

As I see it, we need:

1. An address that supports both IPv4 and IPv6 that will be the new primary source and will be used in all updated docs. ghcr.io, of course, remains and can even be used here as a fallback, but shouldn't be the one we advertise on the website because of IPv6.
2. An address that works better in China.

Now, if 1) and 2) are the same, excellent. If not, 2 can be used after some geo detection automatically in HW probes, and be documented as an additional option for people using docker probes.

Also, if 1) ends up being a proxy for ghcr.io that just adds IPv6, I suppose that's fine. What matters is that all our docs show something that works in all environments.

from globalping-hwprobe.

We could create a custom domain like registry.globalping.io and proxy cache ghcr via Cloudflare workers https://github.com/ciiiii/cloudflare-docker-proxy

This way no scripts change, everything is still pushed to ghcr and all we have to do is change the docs. And we still control the domain so in the future we could change the underlying system if needed.

from globalping-hwprobe.

If that works reliably, sure, but would need some research. I don't know enough about the registry APIs to say if it's a good idea to use a simple proxy like this, plus we'd need to add some restrictions to make only our projects available.

from globalping-hwprobe.

If we want a custom domain with CF workers we need to host the domain with CF. globalping.io is not with CF so cant use it.

So we either decide to use a different domain with CF or consider duplicating all scripts and logic to also push to docker hub.
By pushing to Hub we also get the option to use Amazon's free proxy https://gallery.ecr.aws/. We then make primary the one that performs best in all regions

from globalping-hwprobe.

Overall duplicating the scripts and pushing to docker hub seems easier to me.

from globalping-hwprobe.

It was done by Artem so I'm not sure how it works exactly, can you add Docker hub support? Need to make sure it remains multi arch too.

from globalping-hwprobe.

https://hub.docker.com/repository/docker/globalping/globalping-probe/general

@jimaek do you want to do any perf tests before we move with the changes here or should we go ahead and change the image source?

from globalping-hwprobe.

Yep I'll run some tests first

from globalping-hwprobe.

Google is faster for me and many others complain about Hub's speed, but since it's the fastest one in China we should just use it for everyone.
Maybe document somewhere the other mirrors too for anyone technical enough to use them

from globalping-hwprobe.

All right, so @kernelgurumeditation please replace all uses of the current image with https://hub.docker.com/repository/docker/globalping/globalping-probe/general and verify the builds there work.

from globalping-hwprobe.

@MartinKolarik ,
ok, I will do it later today, and run tests for one or two days

from globalping-hwprobe.

@jimaek , @MartinKolarik

I don't know what you guys did, but the download tool is failing to grab the container, so no builds are possible at this moment..
Did you guys change the type of the image? it seems that now is OCI...

from globalping-hwprobe.

@kernelgurumeditation the only thing I changed directly is the registry where the image is pushed, but I also upgraded the dependencies (actions that build the image) in the process. It is possible that resulted in some more changes. I'm not sure about the image format - what format was it before, and what problem does the new (if it's new) format cause?

from globalping-hwprobe.

@MartinKolarik

the previous version was: vnd.docker.distribution.manifest.v2+json
now is: vnd.oci.image.index.v1+json

The problem is that the tool we use to download the image, can't download this format.

So right now, we can't build new firmware versions, nor any HW probes in the field can be updated by the users.

from globalping-hwprobe.

No way to update the tool being used to support the more modern format?

from globalping-hwprobe.

@kernelgurumeditation just to make sure I understand the impact - new builds of the FW are failing but this doesn't affect existing live probes in any way, right?

As a short-term solution, I can try downgrading our build process back to the previous version, which will likely bring back the old manifest format. Long-term, we should add support for the new format here.

from globalping-hwprobe.

@jimaek

No way to update the tool being used to support the more modern format?

It will be a big amount of work.

Yesterday, I tried Scopeo, and it failed to generate a usable image.

from globalping-hwprobe.

@kernelgurumeditation just to make sure I understand the impact - new builds of the FW are failing but this doesn't affect existing live probes in any way, right?

As a short-term solution, I can try downgrading our build process back to the previous version, which will likely bring back the old manifest format. Long-term, we should add support for the new format here.

@MartinKolarik, right now if a user decides to use the USB update method, it will fail

from globalping-hwprobe.

But their docs say "skopeo can work with OCI images as well as the original Docker v2 images."

from globalping-hwprobe.

skopeo can download the image but "docker load" fails .....

from globalping-hwprobe.

I'll revert, and let's move this to a separate task.

from globalping-hwprobe.

Wait, no need to revert anything. Its not impacting the production so we have time to look into it

from globalping-hwprobe.

Order and priorities... I want to test and close this one first.

from globalping-hwprobe.

If you want to test just build it locally and manually push to your own docker hub repo for tests. No need to make changes to prod back and forth

from globalping-hwprobe.

I want @kernelgurumeditation to be able to finish this task. Reverting the change is not an issue. Having a broken build process is.

from globalping-hwprobe.

@kernelgurumeditation can use any container from any registry and org space for tests, prod is not related to this. Once we know a fix we can push it to prod. Prod is working, only a local build process is broken, so local testing and fix is also the solution

from globalping-hwprobe.