amgedr / webchk Goto Github PK

I would like to be able to specify an authentication header for all of the URLs in my list. Could this feature be added? Thanks!

I keep getting this error in webchk, not sure why. It happens sometimes. When I check the output file, I still see good progress, plenty of 200s and 404s, but also a lot of Operation timed outs.

Exception in thread Thread-350 (_process_url):
Traceback (most recent call last):
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner
self.run()
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\threading.py", line 946, in run
self._target(*self._args, **self.kwargs)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\site-packages\webchk_main.py", line 9, in _process_url
resp = http_response(url, parse=parse_xml)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\site-packages\webchk\http.py", line 114, in http_response
result = _http_request(loc, get_request=get_request)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\site-packages\webchk\http.py", line 78, in _http_request
conn.request(method, loc.path)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1276, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1322, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1271, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1031, in _send_output
self.send(msg)
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 969, in send
self.connect()
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1448, in connect
self.sock = self._context.wrap_socket(self.sock,
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1070, in _create
self.do_handshake()
File "C:\Users\Menna\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

Windows 10 Enterprise 18.03
Steps: When I use -o argument, I am not getting the file created and ouput saved there.

webchk -i list.txt -o test.txt

Result: I am getting everything printed in console, but no file is created and no content is inserted there.
Please advise.

Thank you,
Paul

Dear Amged,

I hope your health is well.

I would love to know how I can pass user agent using webchk. Different user agents may be necessary to visit some sites or get the appropriate status code.

Thank you so much.

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create separate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

Hello,

I love the project but webchk fails when I provide a link which results in connection timed out. Example URL: 2gactivation.apple.com Webchk crashes with an error rather than just printing could resolve and checking the next link in my input file. This prevents all the URLs from being checked.