Comments (5)
rgaudin
commented on May 30, 2024
1
OK, I've looked into this. It is also related to image's srcset but it's not fixed by #63 (which added support for them).
What we're seeing here is that the crawler is not making requests for all of the images in the srcset (or those fail to complete) ; so there are missing images. Depending on the one your browser picks (kinda hard to predict but you can inspect what its trying to display) you may got one that was crawled or not.
I've also found that it's sort of random in selecting which image gets crawled…
I've opened a ticket upstream: webrecorder/browsertrix-crawler#3
@kelson42 should we keep that open until it gets solved upstream?
from zimit.
kelson42
commented on May 30, 2024
I believe I can confirm the problem, with latest wiki.openzim.org scrape, I had myself a few images missing.
from zimit.
rgaudin
commented on May 30, 2024
Please share the ZIM and indications of where to find such images so we can look at what's special about them. Task ID or link as well so we can check the logs.
from zimit.
Popolechien
commented on May 30, 2024
https://farm.youzim.it/pipeline/d1c2f201514f3da67f887df5 for the task - images are missing on every second page or so.
from zimit.
kelson42
commented on May 30, 2024
@rgaudin Yes, please keep this ticket open please.
from zimit.
Related Issues (20)
- Error when building docker image HOT 8
- Support Linux/ARM64 architecture HOT 1
- Full scrape fails while limited one succeeds HOT 2
- External links should not be open in a new tab HOT 1
- Handling of external links seems incoherent HOT 1
- Non-clickable link HOT 1
- NDLA recipe failed HOT 1
- Frequent initial connect timeouts
- New zim should not be defaulted to English HOT 7
- Better auto-detection of multilanguage content HOT 1
- Zim files on Library Education on Wikipedia HOT 1
- dp.la recipe succeeded but file is not complete HOT 1
- Issues with task 672c1 HOT 35
- Conflicting lang param
- MIT OpenCources Recipe issue HOT 1
- ZIMit 2.0 HOT 4
- Use browser-generated text as IndexData
- zim do not open in the Firefox ESR browser using the Kiwix extension: offline wikipedia reader HOT 5
- onclick link not rewritten
- Adding new --config argument to zimit HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zimit.