Unable to use semicolon in arg about zimit HOT 2 CLOSED

Unfortunately, there's not a good fix for this, other than double quotes, eg. --tag "'some;param'" as the params are first handled by bash, then passed to JS, then passed to warc2zim. This would also affect other characters that are shell commands, like &.

I can see about removing the bash script, but it does many useful things now, as far as changing permissions, creating dirs, trapping signal, etc.. which would otherwise have to be implemented in JS.

Because crawling often involves lots of complex options, the general practice is to use config files instead of trying to send all parameters on the command line.

To simplify creating specifying parameters, my recommendation would be to support a yaml based config file, which could be passed in as a volume or even be part of the output volume, eg. docker run -v /my/config.yaml:/config.yaml -it openzim/zimit zimit --url https://example.com/. The config file could then specify the warc2zim command-line nicely:

warc2zim_opts: "--tag=a;b --publisher Test Publisher"

as well as other options which may be common to different types of crawls.

from zimit.

The obvious solution is to replace that shell script with a python one. Will be a lot clearer. You could even import warc2zim to check its part of the paeans...

Or am I missing something ?

from zimit.