ipfs / public-gateway-checker Goto Github PK
View Code? Open in Web Editor NEWChecks which public gateways are online or not
Home Page: https://ipfs.github.io/public-gateway-checker/
License: Other
Checks which public gateways are online or not
Home Page: https://ipfs.github.io/public-gateway-checker/
License: Other
Minor design change:
We currently mark online gateways with ✅ and offline with ❌.
✅ visually connotes "ok" which is different than "online", and maybe stronger than we want to suggest.
Could use 🌍 or something instead?
We should add tests for trustless (verifiable) response types: application/vnd.ipld.raw and application/vnd.ipld.car
Should have tests for requesting both ?format=raw|car
and Accept: application/vnd.ipld.raw|car
and passing only when both work.
Ref.
public-gateway-checker/index.html
Line 62 in 7e65373
fix to address PR comment: #194 (comment)
The cookie button introduced in #309 is placed in a very unfortunate, prominent place.
It looks like an important item related to gateway checks:
An easy fix would be to move it to top bar, we already have github icon there:
It seems that gateways get ✅ for Origin isolation and 💚 EVEN when path gateway does not redirect to a subdomain.
This broken setup is not possible with go-ipfs, but someone could do this type of menace with a custom Nginx config, defeating the origin isolation provided by subdomains.
Figuring out if /ipfs/{cid}
returns HTTP 200 or 301 will be tricky because JS fetch
follows redirects.
Workarounds to investigate (did not check, just an idea):
window.location.origin
from iframe via postMessage and fail if origin is not https://{cid}.ipfs.{gateway}
I want to contribute here, because ipfs represents me in many ways ... this is a small project in which i can contribute, kind of grateful to ipfs ... i write this because it takes boringly long for PRs to be merged
It seems to be no active admins for this project, and believe me, everybody who start to know about ipfs, they search for 'public gateways' because it is the first concept you understand, specially if you come from web development ...
You know, you think things like:
bam ... you reach here ... and you find that in order to add a gateway you have to create a PR, thing only known among programmers and one or two sysadmin :P , think about how many people you know who could share a gateway but would be completely stuck at the moment of creating a PR ... what ? pull req... what ? what does it mean ?
This project seems to be kind of the 'official' list .. (at least the one you reach from google) ... and let me say, it s*cks
Then you want to contribute, only to find that nobody cares about it, and you cannot do it because somebody else have the control ... which goes pretty much against ipfs itself ... it is supposed to be a decentralized system (also philosophically speaking) ...
Maybe i am the one who is missing something ... can somebody clarify it ?
Is this project kind of a ghost one ? ... what's the point ?
My girlfriend told me
Then she stroke my face and said
So, for short ... i want to contribute but i can't ... i don't want to give up, but you know, you have to do it at some point ...
I don't know who to tag ... sorry if i am tagging people who don't care about it ...
@daviddias @Stebalien @bonedaddy @lidel @olizilla @vasco-santos @vmx @koalalorenzo @whyrusleeping @jamiew @victorb
The codebase is currently in javascript, and multiple "check" type objects exist that do not follow a consistent interface. Migrating to typescript will make catching these problems easier.
The separate Classes/objects should also be abstracted out into their own files
Edit: Seeing #171, and then being unable to easily debug the code was a catalyst for this(#190) issue.
While working on #116 I noticed that some gateways aeem to be actively blocking requests for hotlinking images and/or loading .js
files, which breaks our existing "Online" check (eg. Cloudflare).
Had no bandwidth to fix that, and additional analysis is required to see if we are able to tell the difference between gateway that is down and one that does not like third party origins fetching data from it.
Sometimes people submit gateway that resolves /ipfs/
just fine, but fails with /ipns/en.wikipedia-on-ipfs.org
Implementation-wise, IPNS test could be done by trying to load Img from /ipns/en.wikipedia-on-ipfs.org/favicon.ico
(we already have that for /ipfs
, so just run the same test but for wikipedia favicon)
I think it's because it's redirecting to the www version. The proper host name is https://www.eternum.io.
It might make sense to add a feature to IPFS gateways to opt-in to gateway announcement. This gateway checker could then pull the list of gateways from that announcement. I'm not sure the best way to announce your gateway availability to the network, but two things come to mind (I'm new to all of this, so these might be completely unfeasible):
One concern is that this would result in a lot of noise, so I wonder if a static list is also maintained (like the one here), but it's used to signify that a gateway is verified/trusted.
Country flags no longer show on https://ipfs.github.io/public-gateway-checker/
Right now it's wrapping index.html
, app.json
and gateways.json
The ipfg
tool seems to be using that hash to cache version of gateway.json; is that how it's intended to be used? Hash of the latest list of gateways basically? (that makes sense to me)
Could also wire this up to that new GitHub app for automatically publishing
I will no longer be operating this gateway, as it has been abused by someone hosting a fake tech support scam resulting in a complaint to my hosting provider.
I may re-launch it another day, maybe on AWS.
e.g. don't edit the very last line, since trailing comma drama complicates merges
Maybe describe your server if you're willing? Location, connection speed, available space etc
The cookie prompt introduced in #309 is pushing page contents down, causing reflow when user submits the choice.
Would it be ok to move it down, and have it fixed to the bottom of viewport? This way we dont have reflow:
I was thinking about an easier way to add gateways. maybe through a web form ...
PRs are fine, but they put a little bit of friction to the process ... are they the only way, because there is something i am missing ?
Any thoughts on this ?
links
https://github.com/orbitdb/ipfs-log
https://github.com/orbitdb/orbit-db
Some time ago @fsdiogo made this:
Creating PR with a port of above design could be a good "first issue"
Sources are at: https://github.com/fsdiogo/ipfs-infra-status
Gateways should guard against the risk described in ipfs/kubo#4025
(additional notes in gateway spec draft here)
Not guarding should be punished with a big red error.
Thank you for submitting this PR!
A maintainer will be here shortly to review it.
We are super grateful, but we are also overloaded! Help us by making sure that:
The context for this PR is clear, with relevant discussion, decisions
and stakeholders linked/mentioned.
Your contribution itself is clear (code comments, self-review for the
rest) and in its best form. Follow the code contribution
guidelines
if they apply.
Getting other community members to do a review would be great help too on complex PRs (you can ask in the chats/forums). If you are unsure about something, just leave us a comment.
Next steps:
A maintainer will triage and assign priority to this PR, commenting on
any missing things and potentially assigning a reviewer for high
priority items.
The PR gets reviews, discussed and approvals as needed.
The PR is merged by maintainers when it has been approved and comments addressed.
We currently aim to provide initial feedback/triaging within two business days. Please keep an eye on any labelling actions, as these will indicate priorities and status of your contribution.
We are very grateful for your contribution!
Originally posted by @WELCOME[bot] in ipfs/js-ipfs#4165 (comment)
Some gateways, for example ipfs.mttk.net, are reported as offline but instead they are online right now
We've made the Origin check more prominent in #148:
I assume this is because of Subdomain gateway changes? https://docs.ipfs.io/how-to/address-ipfs-on-web/#subdomain-gateway
There should be a check for if a gateway automatically redirects HTTP to HTTPS.
example
https://ipfs.github.io/public-gateway-checker/Qmaisz6NMhDB51cCvNWa1GMS7LU1pAxdF4Ld6Ft9kZEP2a<-custom-hash-here
This may be a bit offtopic here, but do you have a list of IPFS gateways by location or is gateways.json somehow easily put into geoiplookup or similar?
I am currently trying to find European gateway and only @swedneck's name gives me a hint where the gateway might be located, but their gateway is currently down according to the checker and my attempt to connect.
Please consider keeping master of this repo on IPNS, much like we see with https://dist.ipfs.io/
Discussion: #310 (comment)
Since all of the checks implement the same interface, we can simplify the checks to be run the same way.
AC:
e.g.:
//...,
this.origin.check().then(() => log.debug(this.gateway, 'Origin success')).then(this.onSuccessfulCheck.bind(this)),
...
can be refactored as:
const allChecks = [..., OriginCheck, TrustlessCheck, ...];
await Promise.all(async allChecks.map(check => {
try {
await check.check();
log.debug(this.gateway, `${check.name} success`);
this.onSuccessfulCheck();
} catch (err) {
log.error(err);
throw err;
}
}));
Hey!
A discussion has been going on at https://github.com/ipfs/ipfs-gui/issues/80, to upgrade this site to a more ipfs-gui flavour.
I've taken a first stab at it and ported it to React. I was going to open a PR here, but as it was a complete rewrite I created a new repo for it.
We are hoping to add more features to it, would you like to keep this as a separate tool or are you ok with merging both or swapping one for the other?
We have a site to check a list of known IPFS gateways
https://ipfs.github.io/public-gateway-checker/
Let's upgrade it
Currently, we can easily check if the public GW support CORS
, if it is Online
.
To my understanding there is still a huge demand on calling those GW's API(like pinata/infura's IPFS GW services), like /api/v0/add
, Do we have any plan to detect and provide a way to show whether those public GWs allow us to call APIs?
We should add public-gateway-checker metrics to county
base level metrics are covered by ipfs/ipfs-gui#125
and eventually more metrics:
This gateway is responding "143" for some sampled ipfs hash, which is not matching the original content.
Example:
https://via0.com/ipfs/QmX4QTB4k6t6gWZ9MuireyM3pa1wrWhkorVAuCntu4K4BL
https://via0.com/ipfs/QmX4QTBSk6t6gWZ9MuireyM3pa1wrWhkorVAuCntu4K4BL
https://via0.com/ipfs/QmX4QTBX4QTBX4QTBX4QTBX4QTBX4QTBX4QTBX4QTBX4QT
hi
should we just list all gateways that are known?
https://ipfs.infura.io/ipfs/QmZbj5ruYneZb8FuR9wnLqJCpCXMQudhSdWhdhp5U1oPWJ
In the readme, we state:
A CLI version ipfg is available here: https://github.com/JayBrown/Tools/tree/master/ipfg
I propose we encapsulate the logic in this repo into its own encapsulated logic so that one could run the check in both the browser and via CLI, but also via other packages.
It would also make sense to support the often requested "gateway race" logic within this package.
I can't find a public relay list ... It is more important than to have a public gateway list ... by having a relay list available, web pages can actively participate in the ipfs network ... with gateways they can only participate passively by only asking for data without sharing it, furthermore, you cannot use any other kind of distributed communication system like pubsub ...
Should i create a git project to list public relays ?
Would i be the only one sharing my relay node ?
:P
Automate removal of no-longer registered domain names, like in #280
If gateway is a subdomain one, we should also test if it is on the Public Suffix List used by browsers.
Why? Without PSL the owner of the suffix can access cookies of subdomains,
so lack of PSL requires putting more trust into gateway operator.
Ref.
The current UI is purely functional and gets the job done, but:
AC:
Error
Failed to load https://www.eternum.io/: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://ipfs.github.io' is therefore not allowed access. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
I'm not 100% sure but the simplest solution could be just loading an img from the gateway (with plain html) with a green checkbox, and listen onerror in case the gateway is down (replace with red offline label).
Thanks @jamiew, for all the Pull requests approved today.
Sadly this added a lot of dead gateways.
Should somebody consider removing them?
@schomatis raised a good point in #156 (comment)
My only request would be that CID we're using to test subdomain redirection not be static so a malicious/lazy gateway couldn't just bypass it with a basic redirect that doesn't even understand the protocol.
Good idea, but implementation needs more analysis. The main challenge here is content routing speed – if we generate random text for each page load, it will take some time for gateways to find it and implies the page needs to run js-ipfs with some relay/preload setup.
Hi everyone,
we run a new advanced public IPFS gateway here:
https://www.ipfsgate.com
https://search.ipfsgate.com/ipfs/hash
We would like to add the gateway to your 'public gateway checker' (https://ipfs.github.io/public-gateway-checker/), but I am not sure that I fully understand how to "upload" updated json file? What exactly means "submit a pull request"?
Thank you
Best Regards.
Adam.
I've just added my own Gateway ipfs.1-2.dev to the list. Besides the fact, that the server is located in Germany and not USA as reported, it also falsely reports missing origin isolation. However, I've just tested it in my fork by removing everything, except for this specific gateway and there, this isn't an issue at all: https://softcreatr.github.io/public-gateway-checker/
Maybe a problem with the amount of Gateways to be checked synchronously?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.