got that one again

[7256] #yoda Traceback (most recent call last):
File "c:\python\scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.0', 'console_scripts', 'instaloader')()
File "c:\python\scripts\instaloader.py", line 1050, in main
args.fast_update)
File "c:\python\scripts\instaloader.py", line 897, in download_profiles
download_comments=download_comments)
File "c:\python\scripts\instaloader.py", line 744, in download_hashtag
download_comments=download_comments)
File "c:\python\scripts\instaloader.py", line 592, in download_node
metadata = self.get_node_metadata(shortcode)
File "c:\python\scripts\instaloader.py", line 559, in get_node_metadata
if "graphql" in pic_json["entry_data"]["PostPage"][0]
KeyError: 'PostPage'

from instaloader.

[1081] #aaylasecura #aaylasecura/funkycook_2017-02-02_09-59-34.jpg Traceback (most recent call last):
File "c:\python\lib\site-packages\urllib3\response.py", line 302, in _error_catcher
yield
File "c:\python\lib\site-packages\urllib3\response.py", line 384, in read
data = self._fp.read(amt)
File "c:\python\lib\http\client.py", line 449, in read
n = self.readinto(b)
File "c:\python\lib\http\client.py", line 493, in readinto
n = self.fp.readinto(b)
File "c:\python\lib\socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "c:\python\lib\ssl.py", line 1002, in recv_into
return self.read(nbytes, buffer)
File "c:\python\lib\ssl.py", line 865, in read
return self._sslobj.read(len, buffer)
File "c:\python\lib\ssl.py", line 625, in read
v = self._sslobj.read(len, buffer)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.0', 'console_scripts', 'instaloader')()
File "c:\python\scripts\instaloader.py", line 1050, in main
args.fast_update)
File "c:\python\scripts\instaloader.py", line 897, in download_profiles
download_comments=download_comments)
File "c:\python\scripts\instaloader.py", line 744, in download_hashtag
download_comments=download_comments)
File "c:\python\scripts\instaloader.py", line 632, in download_node
downloaded = self.download_pic(filename=filename, url=node["display_src"], date_epoch=date)
File "c:\python\scripts\instaloader.py", line 337, in download_pic
shutil.copyfileobj(resp.raw, file)
File "c:\python\lib\shutil.py", line 79, in copyfileobj
buf = fsrc.read(length)
File "c:\python\lib\site-packages\urllib3\response.py", line 401, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "c:\python\lib\contextlib.py", line 100, in exit
self.gen.throw(type, value, traceback)
File "c:\python\lib\site-packages\urllib3\response.py", line 320, in _error_catcher
raise ProtocolError('Connection broken: %r' % e, e)
urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

from instaloader.

We cannot reproduce this issue, however I suspect this could be caused by commit 151ccfd. Could you check out bf5e3b9 and try whether this exception occurs there?

You could install this version with

pip3 install --upgrade git+https://github.com/Thammus/instaloader@bf5e3b9

or

git checkout bf5e3b9

in your local repository clone.

from instaloader.

In commit 9fbe9b0 we tried to address this issue.

from instaloader.

ran all day with bf5e3b9 and did not see the issue, most have been resolved

thanks

from instaloader.

Thanks for testing! Then I think commit 9fbe9b0 fixes this issue, so the most recent version should work, also.

from instaloader.

It happened again with bf5e3b9, will try 9fbe9b0 next

[6682] #snoke #snoke/darth_gravid_2017-01-20_14-54-44.jpg [😂😜😥 Credit to photographer #…]
[6683] #snoke Traceback (most recent call last):
File "c:\python\lib\site-packages\urllib3\connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "c:\python\lib\site-packages\urllib3\util\connection.py", line 83, in create_connection
raise err
File "c:\python\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "c:\python\lib\site-packages\urllib3\connection.py", line 284, in connect
conn = self._new_conn()
File "c:\python\lib\site-packages\urllib3\connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x0474A1D0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\lib\site-packages\requests\adapters.py", line 440, in send
timeout=timeout
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "c:\python\lib\site-packages\urllib3\util\retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='instagram.fymy1-1.fna.fbcdn.net', port=443): Max retries exceeded with url: /t51.2885-15/e35/15877025_1355266104524370_2103191518587125760_n.jpg (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x0474A1D0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.0', 'console_scripts', 'instaloader')()
File "c:\python\lib\site-packages\instaloader.py", line 1025, in main
args.fast_update)
File "c:\python\lib\site-packages\instaloader.py", line 872, in download_profiles
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 719, in download_hashtag
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 604, in download_node
downloaded = self.download_pic(filename=filename, url=node["display_src"], date_epoch=date)
File "c:\python\lib\site-packages\instaloader.py", line 332, in download_pic
resp = self.get_anonymous_session().get(url, stream=True)
File "c:\python\lib\site-packages\requests\sessions.py", line 515, in get
return self.request('GET', url, **kwargs)
File "c:\python\lib\site-packages\requests\sessions.py", line 502, in request
resp = self.send(prep, **send_kwargs)
File "c:\python\lib\site-packages\requests\sessions.py", line 612, in send
r = adapter.send(request, **kwargs)
File "c:\python\lib\site-packages\requests\adapters.py", line 504, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='instagram.fymy1-1.fna.fbcdn.net', port=443): Max retries exceeded with url: /t51.2885-15/e35/15877025_1355266104524370_2103191518587125760_n.jpg (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x0474A1D0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))

====================================

====================================

[5917] #thelastjedi Traceback (most recent call last):
File "c:\python\lib\site-packages\urllib3\connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "c:\python\lib\site-packages\urllib3\util\connection.py", line 83, in create_connection
raise err
File "c:\python\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "c:\python\lib\site-packages\urllib3\connection.py", line 284, in connect
conn = self._new_conn()
File "c:\python\lib\site-packages\urllib3\connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x03B6C6F0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\lib\site-packages\requests\adapters.py", line 440, in send
timeout=timeout
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "c:\python\lib\site-packages\urllib3\util\retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='instagram.fymy1-1.fna.fbcdn.net', port=443): Max retries exceeded with url: /t51.2885-15/e15/20479195_1747731618860544_7817214553191612416_n.jpg (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x03B6C6F0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.0', 'console_scripts', 'instaloader')()
File "c:\python\lib\site-packages\instaloader.py", line 1025, in main
args.fast_update)
File "c:\python\lib\site-packages\instaloader.py", line 872, in download_profiles
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 719, in download_hashtag
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 604, in download_node
downloaded = self.download_pic(filename=filename, url=node["display_src"], date_epoch=date)
File "c:\python\lib\site-packages\instaloader.py", line 332, in download_pic
resp = self.get_anonymous_session().get(url, stream=True)
File "c:\python\lib\site-packages\requests\sessions.py", line 515, in get
return self.request('GET', url, **kwargs)
File "c:\python\lib\site-packages\requests\sessions.py", line 502, in request
resp = self.send(prep, **send_kwargs)
File "c:\python\lib\site-packages\requests\sessions.py", line 612, in send
r = adapter.send(request, **kwargs)
File "c:\python\lib\site-packages\requests\adapters.py", line 504, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='instagram.fymy1-1.fna.fbcdn.net', port=443): Max retries exceeded with url: /t51.2885-15/e15/20479195_1747731618860544_7817214553191612416_n.jpg (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x03B6C6F0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))

=================================================================

It happened again with bf5e3b9, will try 9fbe9b0 next

from instaloader.

I installed 9fbe9b0 and it happened again during the night. I will run it again throughout the day.

[2012] #starwarsfan Traceback (most recent call last):
File "c:\python\lib\site-packages\urllib3\connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "c:\python\lib\site-packages\urllib3\util\connection.py", line 83, in create_connection
raise err
File "c:\python\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "c:\python\lib\site-packages\urllib3\connection.py", line 284, in connect
conn = self._new_conn()
File "c:\python\lib\site-packages\urllib3\connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x03F7C8D0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\lib\site-packages\requests\adapters.py", line 440, in send
timeout=timeout
File "c:\python\lib\site-packages\urllib3\connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "c:\python\lib\site-packages\urllib3\util\retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='scontent-ort2-2.cdninstagram.com', port=443): Max retries exceeded with url: /t51.2885-15/e35/20478527_164643877416588_2518162252180226048_n.jpg (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x03F7C8D0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.1', 'console_scripts', 'instaloader')()
File "c:\python\lib\site-packages\instaloader.py", line 1161, in main
args.fast_update, args.stories, args.stories_only)
File "c:\python\lib\site-packages\instaloader.py", line 993, in download_profiles
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 834, in download_hashtag
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 619, in download_node
downloaded = self.download_pic(filename=filename, url=node["display_src"], mtime=date)
File "c:\python\lib\site-packages\instaloader.py", line 355, in download_pic
self._get_and_write_raw(url, filename)
File "c:\python\lib\site-packages\instaloader.py", line 159, in _get_and_write_raw
resp = self.get_anonymous_session().get(url, stream=True)
File "c:\python\lib\site-packages\requests\sessions.py", line 515, in get
return self.request('GET', url, **kwargs)
File "c:\python\lib\site-packages\requests\sessions.py", line 502, in request
resp = self.send(prep, **send_kwargs)
File "c:\python\lib\site-packages\requests\sessions.py", line 612, in send
r = adapter.send(request, **kwargs)
File "c:\python\lib\site-packages\requests\adapters.py", line 504, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='scontent-ort2-2.cdninstagram.com', port=443): Max retries exceeded with url: /t51.2885-15/e35/20478527_164643877416588_2518162252180226048_n.jpg (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x03F7C8D0>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))

from instaloader.

Thanks for testing. When you run it again, could you please try the latest release, version 2.2?

from instaloader.

Not sure what I did wrong. I installed 2.2 but now I get this error when trying "instaloader #test". Something is wrong with 2.2 mesa think.

Traceback (most recent call last):
File "C:\python\Scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.2', 'console_scripts', 'instaloader')()
File "c:\python\lib\site-packages\instaloader.py", line 1160, in main
loader.download_profiles(args.profile, args.login.lower(), args.password, args.sessionfile,
AttributeError: 'NoneType' object has no attribute 'lower'

from instaloader.

Sorry for that. Version 2.2.1 should have this fixed.

from instaloader.

Error again with 2.2.1 :(

[6344] #starwars Traceback (most recent call last):
File "c:\python\lib\site-packages\instaloader.py", line 167, in _get_and_write_raw
raise ConnectionException("Request returned HTTP error code {}.".format(resp.status_code))
instaloader.ConnectionException: Request returned HTTP error code 404.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\python\scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.2.1', 'console_scripts', 'instaloader')()
File "c:\python\lib\site-packages\instaloader.py", line 1164, in main
args.fast_update, args.stories, args.stories_only)
File "c:\python\lib\site-packages\instaloader.py", line 995, in download_profiles
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 835, in download_hashtag
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 620, in download_node
downloaded = self.download_pic(filename=filename, url=node["display_src"], mtime=date)
File "c:\python\lib\site-packages\instaloader.py", line 356, in download_pic
self._get_and_write_raw(url, filename)
File "c:\python\lib\site-packages\instaloader.py", line 169, in _get_and_write_raw
print("URL: " + url + "\n" + err, file=sys.stderr)
TypeError: must be str, not ConnectionException

from instaloader.

Thanks, 2.2.2 didn't crash this time and downloaded some 21,000 files. Probably fixed 👍

from instaloader.

That is nice to hear. Thank you very much for testing and for providing the stack tracebacks that made these adjustments possible.

Happy instaloading!

from instaloader.

@MatthieuBonenfant I watching so I get emails about all issues. Just curious if you've experienced any "limits" imposed by Instagram when downloading. I am a bit scared to download 100 images at a time, let alone 21,000 for fear of being banned.

from instaloader.

I have been downloading 5-10 hashtags at the time with the default timeout in between images. Haven't had any problem. I can usually do a few thousands before hitting these errors.

It happened again :(

[2806] #r2d2 Traceback (most recent call last):
File "c:\python\scripts\instaloader-script.py", line 11, in
load_entry_point('instaloader==2.2.2', 'console_scripts', 'instaloader')()
File "c:\python\lib\site-packages\instaloader.py", line 1201, in main
args.fast_update, args.stories, args.stories_only)
File "c:\python\lib\site-packages\instaloader.py", line 1032, in download_profiles
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 860, in download_hashtag
download_comments=download_comments)
File "c:\python\lib\site-packages\instaloader.py", line 593, in download_node
metadata = self.get_node_metadata(shortcode)
File "c:\python\lib\site-packages\instaloader.py", line 548, in get_node_metadata
if "graphql" in pic_json["entry_data"]["PostPage"][0]
TypeError: 'NoneType' object is not subscriptable

from instaloader.

That's good to know. Thank you @MatthieuBonenfant! And just wanted to say thanks @Thammus for the great work as well!

from instaloader.

I don't think Instagram can do much beside killing the service which allows that script to work, not like you need to be logged in to make the script work anyways. I have downloaded about 500,000 images in 1 month and still going!

from instaloader.