Its showing me too many request when I run the script.
File "gsmarena-scraper.py", line 32 logger.info(f"Too many requests.") ^ SyntaxError: invalid syntax

How can I solve this?

hello again. thanks for quick fix. but there is another problem. parser do not saves each data under required column.

you can see an example https://imgur.com/a/Pna0RkW

hi. it would be good to save each brand into csv file when finishes and starts scraping another brand. otherwise remote host blocks requests:

Traceback (most recent call last):
File "C:\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 603, in urlopen
chunked=chunked)
File "C:\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 344, in _make_request
self._validate_conn(conn)
File "C:\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 843, in validate_conn
conn.connect()
File "C:\Python37-32\lib\site-packages\urllib3\connection.py", line 370, in connect
ssl_context=context)
File "C:\Python37-32\lib\site-packages\urllib3\util\ssl.py", line 355, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Python37-32\lib\ssl.py", line 423, in wrap_socket
session=session
File "C:\Python37-32\lib\ssl.py", line 870, in _create
self.do_handshake()
File "C:\Python37-32\lib\ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Python37-32\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "C:\Python37-32\lib\site-packages\urllib3\util\retry.py", line 368, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Python37-32\lib\site-packages\urllib3\packages\six.py", line 685, in reraise
raise value.with_traceback(tb)
File "C:\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 603, in urlopen
chunked=chunked)
File "C:\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 344, in _make_request
self._validate_conn(conn)
File "C:\Python37-32\lib\site-packages\urllib3\connectionpool.py", line 843, in validate_conn
conn.connect()
File "C:\Python37-32\lib\site-packages\urllib3\connection.py", line 370, in connect
ssl_context=context)
File "C:\Python37-32\lib\site-packages\urllib3\util\ssl.py", line 355, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Python37-32\lib\ssl.py", line 423, in wrap_socket
session=session
File "C:\Python37-32\lib\ssl.py", line 870, in _create
self.do_handshake()
File "C:\Python37-32\lib\ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\Users\XXX.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\ptvsd_launcher.py", line 43, in
main(ptvsdArgs)
File "c:\Users\XXX.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd_main_.py", line 432, in main
run()
File "c:\Users\XXX.vscode\extensions\ms-python.python-2019.9.34911\pythonFiles\lib\python\ptvsd_main_.py", line 316, in run_file
runpy.run_path(target, run_name='main')
File "C:\Python37-32\lib\runpy.py", line 263, in run_path
pkg_name=pkg_name, script_name=fname)
File "C:\Python37-32\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "C:\Python37-32\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "c:\Users\XXX\Desktop\gsmarena-scraper-master\gsmarena-scraper.py", line 192, in
main()
File "c:\Users\XXX\Desktop\gsmarena-scraper-master\gsmarena-scraper.py", line 59, in main
File "C:\Python37-32\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Python37-32\lib\site-packages\requests\api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python37-32\lib\site-packages\requests\sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python37-32\lib\site-packages\requests\sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "C:\Python37-32\lib\site-packages\requests\adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

Hey dbeley,

What do you suggest to do to avoid being banned while scrapping?

Hi,

great tool.

It is so sad, that gsmarena does not offer a good way to retrieve the information that people really need... .

My approach to find a new phone is presently:

1. look at https://www.gsmarena.com/battery-test.php3 for the phones with the longest runtime.
2. find manually those that are not too big, have fm-radio etc.

It would be very cool if this scraper would also retrieve the (available) data from https://www.gsmarena.com/battery-test.php3 and add it to the referring phones as extra column(s).

Thanks for taking this into consideration.