exceen / 4chan-downloader Goto Github PK
View Code? Open in Web Editor NEWPython3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation
License: MIT License
Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation
License: MIT License
Hi! Just downloaded your script but every file downloaded is corrupt. I picked a random thread https://boards.4chan.org/b/thread/573855146/pink-ids-will-have-pink-eye-for-the-rest-of-their, but also tried with some more with same result.
Is there any kind of logging i can enable to help you sort this out, if you're not experiencing the same problem?
Cheers!
Luca
i want all the images of a certian tag to go into just one folder instead of each individual one, is there a way to do that?
also is there a feature to autodownload threads in the queue text file when one appears
Cheers. it's all in the title.
I poked around in the source. it's all regexps.
extending that means ripping all that out and replacing it with beautifulsoup or lxml.
At the moment, the script monopolises the bandwith. It would be nice to have an optional input argument to limit how much connection is used.
I've run into a few issues with filenames. One of them isn't entirely Windows specific, and I have an idea of what needs to be done to fix it, but no idea how to. Duplicate filenames within a thread simply overwrite any preceding files. Usually Spoiler_Image
or file.png
.
The other issue is Windows specific and I've managed to solve it at least locally by:
from django.utils.text import get_valid_filename
...snip...
img_path = ntpath.join(directory, get_valid_filename(img))
...snip...
The issue in question, filenames like this used to make the script halt on Windows:
C:\vtai>inb4404.py -c -d -l -t https://boards.4channel.org/vt/thread/34806758
[2022-10-11 07:50:46 PM] [ 9/278] vt/34806758/[sound=files.catbox.moe%2F7a2f1v.m4a]{{takanashi kiara}}, {{{1girl}}}, {begging},[[[lipstick]]],[[[lip gloss]]],kusogaki,closed eyes,{{pov}}, {{incoming kiss}}, close-up, {{horny}}, blushing, solo, puffy sleeves, orange skirt, aqua choker, orange hair, ba.png
Traceback (most recent call last):
File "C:\vtai\inb4404.py", line 171, in <module>
main()
File "C:\vtai\inb4404.py", line 31, in main
download_thread(thread, args)
File "C:\vtai\inb4404.py", line 112, in download_thread
with open(img_path, 'wb') as f:
OSError: [Errno 22] Invalid argument: 'C:\\vtai\\downloads\\vt\\34806758\\[sound=files.catbox.moe%2F7a2f1v.m4a]{{takanashi kiara}}, {{{1girl}}}, {begging},[[[lipstick]]],[[[lip gloss]]],kusogaki,closed eyes,{{pov}}, {{incoming kiss}}, close-up, {{horny}}, blushing, solo, puffy sleeves, orange skirt, aqua choker, orange hair, ba.png'
After importing it appears to work.
The 2to3
tool should automate this as far as possible. I think the script should be converted to python3 because this is the new standard, support for python2 will be dropped in near future.
File "Python\Python38-32\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
Process Process-1:
File "Python\Python38-32\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "4chdl\inb4404.py", line 44, in download_thread
if args.use_names or os.path.exists(os.path.join(workpath, 'downloads', board, thread_tmp)):
AttributeError: 'NoneType' object has no attribute 'use_names'
Traceback (most recent call last):
File "Python\Python38-32\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "Programs\Python\Python38-32\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "4chdl\inb4404.py", line 44, in download_thread
if args.use_names or os.path.exists(os.path.join(workpath, 'downloads', board, thread_tmp)):
AttributeError: 'NoneType' object has no attribute 'use_names'
Regardless of thread, board, script location, drive letter, attribute, or host machine the script returns "Something went wrong" and times out after several tries.
Above picture showing different threads and boards and different attribute flags, all three failing almost immediately.
Interestingly however it works fine on an arch install. I've tried it on two machines running Windows 10, reinstalled python 3 as well
Not entirely sure why it's dropping almost immediately on Windows. Assume this is a Windows problem, but nothing changed since I used it yesterday. will close if I find a solution to using it on windows.
sorry there isn't much information to go off of.
The script keeps "checking" the first post of the thread specified as thread_link
I'm using python 2.7.13, script doesn't work with python 3 because of an error at line 73.
Single thread and multiple thread (using a file) works just fine, but the script is unable to reload the file and update the queue properly as of now.
when i run the shitty python file it immediately closes, and the instructions are ass and dont tell me how to run it
I'm not versed in Python. how can I change the directory from workpath\downloads\(board)\(thread)
to something like this C:\Users\Me\Downloads\Video\(board)\(thread)
Thanks.
with running a command:
python inb4404.py -t [Thread URL]
Terminal returns with:
Could not import the required modules! Disabling --title option...
Program continues & runs fine, but files are downloaded with non original names.
NSFW
How to reproduce:
1° Try to download from this link: https://boards.4chan.org/gif/thread/13521437
This error appears
File TTT Part 2 Top Tier Titties \1527053877481.webm from https://i.4cdn.org/gif/1536882732639.webm UNKNOWN ERROR OCCURRED [WinError 183] cannot create an existing file: 'C:\\Users\\mario\\4channer\\TTT Part 2 Top Tier Titties '
But the "TTT Part 2 Top Tier Tittiles" and the two .webm files doens't exist. Using a custom folder location doens't work.
I guess I miss something very simple, but cannot figure it out, so here goes nothing
I followed the instructions to in README.md and typed
$ python inb4404.py http://boards.4chan.org/w/thread/2096591/lain-thread-anyone-have-any-arisu
The only result was:
File "inb4404.py", line 129
print(line.replace(link, '-' + link), end='')
^
SyntaxError: invalid syntax
And I'm too dumb to know if this is a typo to fix or I should use a different URL (direct one to the first pic in thread maybe?).
(After hundreds of shameful edits): also the output points at end=''
equation, but for love of god I have no idea how to paste it here so markdown does not cut out all spaces.
a batch file perhaps. --batch moreThanOneThread.txt
It downloads, but I can't see nothing in the downloaded videos.
A picture of how the videos are rendered: http://imgur.com/a/vJkiR
I mean it seems so. The download.py is gone from the main repo and the one I have has started to 404
Make a requirements file so that it's easier to download the necessary python libraries. Suggested example attached to this comment. Please add anything I might have missed.
requirements.txt
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.