Comments (6)
ikreymer
commented on May 30, 2024
1
We are still working on the release image, please try openzim/zimit:dev instead of openzim/zimit for now.
from zimit.
rgaudin
commented on May 30, 2024
Can you share the exact command you used and its output? You may have to assign enough ram to the container as well. And I think shm-size value units is single char (so --shm-size=2g).
from zimit.
chjohn22
commented on May 30, 2024
Command used: docker run -v /volume1/downloads/zimfiles:/output --cap-add=SYS_ADMIN --cap-add=NET_ADMIN \ --shm-size=2g openzim/zimit --url http://example.com/ --name samplezim --workers 1 --wait-until domcontentloaded
Output:
/output/tmp.bXVWcU7BQX /app
2020-11-09 16:41:54,792: [INFO]: Created Directory: /output/tmp.bXVWcU7BQX/collections/capture
2020-11-09 16:41:54,792: [INFO]: Created Directory: /output/tmp.bXVWcU7BQX/collections/capture/archive
2020-11-09 16:41:54,792: [INFO]: Created Directory: /output/tmp.bXVWcU7BQX/collections/capture/indexes
2020-11-09 16:41:54,792: [INFO]: Created Directory: /output/tmp.bXVWcU7BQX/collections/capture/static
2020-11-09 16:41:54,792: [INFO]: Created Directory: /output/tmp.bXVWcU7BQX/collections/capture/templates
2020-11-09 16:41:54,793: [INFO]: Created Directory: /output/tmp.bXVWcU7BQX/static
2020-11-09 16:41:54,793: [INFO]: Created Directory: /output/tmp.bXVWcU7BQX/templates
Running: warc2zim --url http://example.com/ --u http://example.com/ --name samplezim
[INFO] Arguments valid, no inputs to process. Exiting with error code 100
Crawl failed, ZIM creation skipped
Error: Unable to launch browser, error message: Failed to launch the browser process!
[1109/164158.552327:FATAL:zygote_host_impl_linux.cc(117)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. pages/second)
#0 0x55858e3182a9 base::debug::CollectStackTrace()
#1 0x55858e290a83 base::debug::StackTrace::StackTrace()
#2 0x55858e2a21a0 logging::LogMessage::~LogMessage()
#3 0x55858c8e105e content::ZygoteHostImpl::Init()
#4 0x55858de3abbb content::ContentMainRunnerImpl::Initialize()
#5 0x55858de8a96a service_manager::Main()
#6 0x55858de39191 content::ContentMain()
#7 0x55858de8952d headless::(anonymous namespace)::RunContentMain()
#8 0x55858de8922c headless::HeadlessShellMain()
#9 0x55858b631ca7 ChromeMain
#10 0x7f755428c09b __libc_start_main
#11 0x55858b631aea _start
Received signal 6
#0 0x55858e3182a9 base::debug::CollectStackTrace()
#1 0x55858e290a83 base::debug::StackTrace::StackTrace()
#2 0x55858e317e4b base::debug::(anonymous namespace)::StackDumpSignalHandler()
#3 0x7f75594c0730 (/lib/x86_64-linux-gnu/libpthread-2.28.so+0x1272f)
#4 0x7f755429f7bb gsignal
#5 0x7f755428a535 abort
#6 0x55858e316da5 base::debug::BreakDebugger()
#7 0x55858e2a261b logging::LogMessage::~LogMessage()
#8 0x55858c8e105e content::ZygoteHostImpl::Init()
#9 0x55858de3abbb content::ContentMainRunnerImpl::Initialize()
#10 0x55858de8a96a service_manager::Main()
#11 0x55858de39191 content::ContentMain()
#12 0x55858de8952d headless::(anonymous namespace)::RunContentMain()
#13 0x55858de8922c headless::HeadlessShellMain()
#14 0x55858b631ca7 ChromeMain
#15 0x7f755428c09b __libc_start_main
#16 0x55858b631aea _start
r8: 0000000000000000 r9: 00007ffee3aac090 r10: 0000000000000008 r11: 0000000000000246
r12: 00007ffee3aad360 r13: 00007ffee3aacb20 r14: 00007ffee3aad370 r15: 00007ffee3aac2f0
di: 0000000000000002 si: 00007ffee3aac090 bp: 00007ffee3aac2e0 bx: 0000000000000006
dx: 0000000000000000 ax: 0000000000000000 cx: 00007f755429f7bb sp: 00007ffee3aac090
ip: 00007f755429f7bb efl: 0000000000000246 cgf: 0000000000000033 erf: 0000000000000000
trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
at Cluster.<anonymous> (/app/node_modules/puppeteer-cluster/dist/Cluster.js:119:23)
at Generator.throw (<anonymous>)
at rejected (/app/node_modules/puppeteer-cluster/dist/Cluster.js:6:65)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
/app
Removing temp dir /output/tmp.bXVWcU7BQX
from zimit.
chjohn22
commented on May 30, 2024
We are still working on the release image, please try
openzim/zimit:devinstead ofopenzim/zimitfor now.
docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"--url\": executable file not found in $PATH": unknown. ERRO[0002] error waiting for container: context canceled
from zimit.
ikreymer
commented on May 30, 2024
The command should be:
docker run -v /volume1/downloads/zimfiles:/output --cap-add=SYS_ADMIN --cap-add=NET_ADMIN \ --shm-size=2g openzim/zimit:dev zimit --url http://example.com/ --name samplezim --workers 1 --wait-until domcontentloaded
from zimit.
chjohn22
commented on May 30, 2024
@ikreymer Thanks, I forgot zimit, but it does only work for me without --wait-until domcontentloaded.
The created zimfiles do need the service worker, which does not work on Kiwix MacOS or in the firefox addon. I will try them on Arch Linux tomorrow.
from zimit.
Related Issues (20)
- Add support for `--logging` parameter of browsertrix crawler
- Pass scraper parameter to warc2zim HOT 1
- Remove cookie banners HOT 1
- Add parameter to exclude certain resources
- tvtropes is failing HOT 3
- TV Tropes 403 errors HOT 3
- Invalid leading whitespace in header HOT 1
- URL is different in error message HOT 2
- solar.lowtechmagazine.com is very unstable HOT 4
- Upgrade to browsertrix crawler 1.0.0 beta HOT 7
- Enhance integration test to assert final content of the ZIM
- Add support for downloading the browser profile from a URL
- networkidle is no longer a valid waitUntil HOT 7
- Browsertrix Crawler is stopping on disk full while it is not full HOT 2
- Zimit2: Youtube videos are not working everywhere HOT 8
- --exclude question HOT 4
- No output after quitting early HOT 3
- [zimit1] scraper never exits
- Crawler error: Cannot convert argument to a ByteString HOT 3
- Add option to only crawl website and not run warc2zim conversion HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from zimit.