non stop evaluate about tumblthree HOT 6 CLOSED

Hello,

what's the url of the tumblr blog you're trying to download? Maybe I can fix the issue with your help.

If you need to login to tumblr.com in order to view the site (e.g. a non-public blog), the crawler won't work yet as there is no authentification mechanism implemented yet.

from tumblthree.

Ive once use tumblOne 1.02 and upgrade to tumblOne 1.04 and active since then. Now seeing others like tumblTwo and tumblThree ive really happy. So i do know how the program works. 

Every blog that i use before is PUBLIC and seems ok in tumblOne. But TumblOne limitation seems i like TumblThree more. But the EVALUATE is really make it pain more.

This always happen if tumblr blog have more than 1500 more images in its blog. I also need to collect more image from a blog that have more than 50k, 100k, 150k images in its blog. So constantly evaluate a blog is not make it download ever. It evaluate like hell.

Its maybe related to my internet connection but seems not always happen. TumblOne mange to download some but not TumblThree.

I will send you the blog if still insist to help me.

Thanks for reply.

Sent from myMail for iOS

Sunday, 7 August 2016, 17:32 +0800 from [email protected] [email protected]:

Hello,
what's the url of the tumblr blog you're trying to download? Maybe I can fix the issue with your help.
If you need to login to tumblr.com in order to view the site (e.g. a non-public blog), the crawler won't work yet as there is no authentification mechanism implemented yet.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub , or mute the thread .

from tumblthree.

It sounds like the application has opened too many connections to the Tumblr network which were timed out and got closed by the servers and the application now sits there waiting for data doing nothing. That might happen if your connection speed isn't satisfying enough to processes so many parallel connections.

You might try to decrease the Parallel Images number in the Settings to maybe the half. Make sure it is always higher than the Parallel Blogs value as both values get divided and thats the number of connections the application opens per blog to download simultaneous data. So, 25 for Parallel Images / 2 Parallel Blogs would be 12 open connections to the Tumblr servers per blog.

Thats probably why TumblOne is working. It doesn't have to evalute the image urls beforehand as it simply downloads one after another. But at least here in my tests the maximum download speeds where around 0.5 MB/s (per connection). TumblThree should be much faster ..

Simply change the values, play a bit around and restart the crawl process again. If that doesn't fix the issue, I am certainly interested in a blog url to see if I can reproduce the error.

from tumblthree.

Wrong item.... Sorry...

from tumblthree.

Please, read what I wrote and answer to my explanations.

Are you actually sure it's not downloading at all? I've just checked a blog and downloaded only the videos. The "evaluating xxx urls .." stands there as long as a whole video file is completely downloaded as the database only records successfully downloaded files on purpose (so we can easily continue were we've left). So if you haven't changed the defaults you'll end up in downloading 25 video files in parallel maybe each a few hundred megabytes in size. Depending on your connection speed, that will probably take a while until you've downloaded at least one file entirely.

Open the Task Manager/Resource Monitor and check if there is network traffic or disk I/O to the corresponding .mp4 files in the Blogs folder.

All the other things I've already explained on my homepage or do work as intended. And obviously, as I've explained above, there will never be a enable/disable evaluate function. The whole purpose of doing this is the gather all the possible urls of images and/or videos to enhance the download speed as it allows parallel downloading instead of step wise downloading one file after another.

from tumblthree.

Sorry man. Its already ok. Sorry for make u read wrong mail. Next i will resend more suggestion to u for new version.

Sent from myMail for iOS

Thursday, 18 August 2016, 04:27 +0800 from [email protected] [email protected]:

Please, read what I wrote and answer to my explanations.
Are you actually sure it's not downloading at all? I've just checked a blog and downloaded only the videos. The "evaluating xxx urls .." stands there as long as a whole video file is completely downloaded as the database only records successfully downloaded files on purpose (so we can easily continue were we've left). So if you haven't changed the defaults you'll end up in downloading 25 video files in parallel maybe each a few hundred megabytes in size. Depending on your connection speed, that will probably take a while until you've downloaded at least one file entirely.
Open the Task Manager/Resource Monitor and check if there is network traffic or disk I/O to the corresponding .mp4 files in the Blogs folder.
All the other things I've already explained on my homepage or do work as intended. And obviously, as I've explained above, there will never be a enable/disable evaluate function. The whole purpose of doing this is the gather all the possible urls of images and/or videos to enhance the download speed as it allows parallel downloading instead of step wise downloading one file after another.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub , or mute the thread .

from tumblthree.