This solution is super fast, handles errors and malformated entries without a problem and ignores duplicates.
#! /usr/bin/env bash
file="$1"
while read p; do
wget -nv -N "$p"
done <$file
Usage: $ ./download.sh samplefile.txt
Usage: $ ./image_downloader.py samplefile.txt
Testing: $ pytest
For manual testing a testserver can be used like this:
$ python3 -m http.server 60451 --bind 127.0.0.1
This type of program can easily be parallelized by splitting the input file into multiple smaller files and iterating over each separately.