Git Product home page Git Product logo

deadlinks's People

Contributors

kdeldycke avatar kernc avatar silentlamb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

deadlinks's Issues

Packaging pelican-deadlinks for PyPi distribution.

Since Pelican 4.5, plugins can be distributed as independent python package and published to PyPi.

The community is working hard splitting out the https://github.com/getpelican/pelican-plugins mono-repo to the new https://github.com/getpelican/pelican-plugins umbrella organization. I already did some work to migrate the neighbors to its own repo: pelican-plugins/neighbors#2

Can I assist in any capacity to package up the pelican-deadlinks plugin? ๐Ÿ™‚

[Feature] Introduce automated tests if possible

Investigate and introduce automated tests for deadlinks plugin.

  • How to mock pelican internals
  • How to mock requests.get(...) calls

Implement several tests cases:

  • Destination not available (timeout, etc)
  • Communication issues (invalid SSL, etc)
  • Several different response codes (200, 403, 404, 500, etc)

[Bug] UnicodeEncodeError: 'ascii' codec can't encode characters in position 10-17: ordinal not in range(128)

1. Summary

I get stack trace, if I use deadlinks plugin.

2. Settings

My project โ€” https://github.com/Kristinita/KristinitaPelican,
Nas-Izu.md file โ€” https://github.com/Kristinita/KristinitaPelican/blob/master/content/Giologica/Nas-Izu.md with Cyrillic symbols.

Part of my pelicanconf.py file:

PLUGIN_PATHS = ['pelican-plugins']
PLUGINS = [
    'pagefixer',
    'pelican_javascript',
    'section_number', 'interlinks', 'deadlinks'
]

DEADLINK_VALIDATION = True

DEADLINK_OPTS = {
    'archive': True,
    'classes': ['custom-class1', 'disabled'],
    'labels': True
}

3. Steps to reproduce

I run command in terminal:

pelican content --debug > DeadlinkDebug.txt 2>&1

See full output on Gist โ€” https://gist.github.com/Kristinita/63c81829c196afd7dc68cbe5e3dba12a.

4. Expected behavior

Not stack trace.

5. Actual behavior

ERROR: Could not process Giologica\Nas-Izu.md
  | UnicodeEncodeError: 'ascii' codec can't encode characters in position 10-17: ordinal not in range(128)
  |___
  | Traceback (most recent call last):
  |   File "c:\python36\lib\site-packages\pelican\generators.py", line 629, in generate_context
  |     context_sender=self)
  |   File "c:\python36\lib\site-packages\pelican\readers.py", line 572, in read_file
  |     context=context)
  |   File "c:\python36\lib\site-packages\pelican\contents.py", line 153, in __init__
  |     signals.content_object_init.send(self)
  |   File "c:\python36\lib\site-packages\blinker\base.py", line 267, in send
  |     for receiver in self.receivers_for(sender)]
  |   File "c:\python36\lib\site-packages\blinker\base.py", line 267, in <listcomp>
  |     for receiver in self.receivers_for(sender)]
  |   File "D:\Kristinita\pelican-plugins\deadlinks\deadlinks.py", line 163, in content_object_init
  |     avail, success, code = get_status_code(url)
  |   File "D:\Kristinita\pelican-plugins\deadlinks\deadlinks.py", line 32, in get_status_code
  |     urlopen(url)
  |   File "c:\python36\lib\urllib\request.py", line 223, in urlopen
  |     return opener.open(url, data, timeout)
  |   File "c:\python36\lib\urllib\request.py", line 526, in open
  |     response = self._open(req, data)
  |   File "c:\python36\lib\urllib\request.py", line 544, in _open
  |     '_open', req)
  |   File "c:\python36\lib\urllib\request.py", line 504, in _call_chain
  |     result = func(*args)
  |   File "c:\python36\lib\urllib\request.py", line 1346, in http_open
  |     return self.do_open(http.client.HTTPConnection, req)
  |   File "c:\python36\lib\urllib\request.py", line 1318, in do_open
  |     encode_chunked=req.has_header('Transfer-encoding'))
  |   File "c:\python36\lib\http\client.py", line 1239, in request
  |     self._send_request(method, url, body, headers, encode_chunked)
  |   File "c:\python36\lib\http\client.py", line 1250, in _send_request
  |     self.putrequest(method, url, **skips)
  |   File "c:\python36\lib\http\client.py", line 1117, in putrequest
  |     self._output(request.encode('ascii'))
  | UnicodeEncodeError: 'ascii' codec can't encode characters in position 10-17: ordinal not in range(128)

6. Environment

Operating system and version:
Windows 10 Enterprise LTSB 64-bit EN
Python:
3.6.1
Pelican:
3.7.1
BeautifulSoup4:
4.5.3

Thanks.

[Bug] Warnings for good links

1. Summary

  1. deadlinks mark as dead good links.
  2. deadlinks mark as dead good links, but blocked from my IP (see Internet censorship in Russia).

2. Settings

My project โ€” https://github.com/Kristinita/KristinitaPelican,

Part of my pelicanconf.py file:

PLUGIN_PATHS = ['pelican-plugins']
PLUGINS = [
    'pagefixer',
    'pelican_javascript',
    'section_number', 'interlinks', 'deadlinks'
]

DEADLINK_VALIDATION = True

DEADLINK_OPTS = {
    'archive': True,
    'classes': ['custom-class1', 'disabled'],
    'labels': True
}

3. Steps to reproduce

I run command in terminal:

pelican content --debug > DeadlinkDebug.txt 2>&1

See full output on Gist โ€” https://gist.github.com/Kristinita/63c81829c196afd7dc68cbe5e3dba12a.

4. Expected behavior

Discover and replace real 403/404 links, not links from 1.1 and 1.2 items of my issue.

5. Actual behavior

List of links, mark as dead.

https://rsdn.ru/article/patterns/framework.xml#EKB
http://vaden-pro.ru/blog/laravel/laravel-chto-eto
http://web.archive.org/web/20150615162941/http://www.xpomo.com/ruskolan/tolpa/piramida.htm
http://www.spy-soft.net/chto-takoe-rat/
http://loveread.ec/read_book.php?id=45782&p=12
http://archive.is/20160611162905/http://liwihelp.ru/sistema/avtomaticheskoe_vklyuchenie_kompyutera.html
https://learn.javascript.ru/window-methods
http://javascript.ru/window-location
https://colocat.ru/texts/realip.html
http://dizems.ru/v-chem-otlichie-staticheskix-sajtov-ot-dinamicheskix
http://www.Is
http://optimakomp.ru/virustotal-totalnoe-skanirovanie-fajjlov-i-sajjtov-desyatkami-antivirusov/
http://wolandblog.com/3-pochemu-ya-ne-ispolzuyu-dnsbl-v-pomoshh-nachinayushhemu-postmasteru/
https://www.projecthoneypot.org/
https://urlquery.net/
http://www.dnsbl.info/dnsbl-database-check.php
http://wikireality.ru/wiki/MDK
http://archive.is/20160518165040/https://www.youtube.com/watch?v=qet1ypk3qDM&lc=z13owrebxvn2vt3e422du3wowrmzz5xxz04
http://archive.is/20160522103717/https://www.youtube.com/watch?v=8Lsrvn7oa60&lc=z12bz5axmkngxx10i22ucr15rtvnsjpyy04
http://archive.is/
http://web.archive.org/web/20150615162941/http://www.xpomo.com/ruskolan/tolpa/piramida.htm
http://archive.is/20160518125518/https://www.facebook.com/permalink.php?story_fbid=517539018447713&id=100005748574402%23
http://archive.is/20160601035438/http://www.sports.ru/profile/1021517009/comments/?p=30
http://archive.is/20160601041255/http://www.sports.ru/profile/70045047/comments/?p=38
http://web.archive.org/web/20150615162941/http://www.xpomo.com/ruskolan/tolpa/piramida.htm
http://alternativeto.net/software/resource-hacker/
https://www.google.ru/search?q=status+bar&newwindow=1&source=lnms&tbm=isch&sa=X&ved=0ahUKEwi-j9WygojTAhVGiSwKHfRhATYQ_AUIBigB&biw=1173&bih=729

I can successful visit this links without proxy and other anonymisation tools:

Some links working, but blocked by government of my country (Russia), example:

6. Environment

Operating system and version:
Windows 10 Enterprise LTSB 64-bit EN
Python:
3.6.1
Pelican:
3.7.1
BeautifulSoup4:
4.5.3

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.