nicolaslm / atoma Goto Github PK

View Code? Open in Web Editor NEW

108.0 7.0 15.0 85 KB

Atom, RSS and JSON feed parser for Python 3

License: MIT License

Python 99.69% Makefile 0.31%

atom feed parser syndication rfc-4287 python3 rss rss-feed-parser json-feed

atoma's People

Contributors

Stargazers

Watchers

Forkers

paregorios rtw87 bendikro f1n63r sgml bigapartmentsin lxn5321 kaestnja codingequalsfreedom labi0 jthodge hugovk samhoice vydd grcarmenaty

atoma's Issues

ModuleNotFoundError: No module named 'atoma' on heroku

even though I think I correctly declared the module in the requirements file.

No docs

It'd be really useful if there was some documentation about how to use this package. The examples in the readme are great, thanks for that, but a link to some full docs would go a long way.

unable to import module

Python 3.5.5 (default, Jul 24 2018, 10:23:26)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux

import requests
import atoma
Traceback (most recent call last):
File "", line 1, in
File "/misc/home/mname/git/venv3/lib/python3.5/site-packages/atoma/init.py", line 1, in
from .atom import parse_atom_file, parse_atom_bytes
File "/misc/home/mname/git/venv3/lib/python3.5/site-packages/atoma/atom.py", line 23
text_type: str = attr.ib()
^
SyntaxError: invalid syntax

Hi,
I think the validation on the source is a little too strict, for example if a description tag is present but empty in the channel section of an rss feed, atoma should not exit with an error in my opinion.

An example https://www.giornalettismo.com/feed

Thanks

Project status inquiry

Is the project dead or is it still alive? I filed an issue over a month ago, and it received no attention. I just want to know if it's worth using this project.

FeedParseError on some common public RSS feeds

RSS feeds from the New York Times and the Washington Post fail. Python 3.6.4. atoma 0.0.3.

Steps to reproduce:

>>> import atoma
>>> import requests
>>> response = requests.get('http://feeds.washingtonpost.com/rss/national')
>>> feed = atoma.parse_atom_bytes(response.content)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/atom.py", line 267, in parse_atom_bytes
    return _parse_atom(root)
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/atom.py", line 214, in _parse_atom
    title = _get_text_construct(root, 'feed:title')
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/atom.py", line 112, in _get_text_construct
    child = get_child(element, name, optional)
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/utils.py", line 25, in get_child
    .format(name, element.tag)
atoma.utils.FeedParseError: Could not parse RSS channel: "feed:title" required in "rss"
>>> response = requests.get('http://rss.nytimes.com/services/xml/rss/nyt/Science.xml')
>>> feed = atoma.parse_atom_bytes(response.content)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/atom.py", line 267, in parse_atom_bytes
    return _parse_atom(root)
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/atom.py", line 214, in _parse_atom
    title = _get_text_construct(root, 'feed:title')
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/atom.py", line 112, in _get_text_construct
    child = get_child(element, name, optional)
  File "/Envs/feedpod/lib/python3.6/site-packages/atoma/utils.py", line 25, in get_child
    .format(name, element.tag)
atoma.utils.FeedParseError: Could not parse RSS channel: "feed:title" required in "rss"

Since recently been unable to parse Reddit's web feeds (RSS)

import requests
import atoma
response = requests.get("https://www.reddit.com/r/<insert subreddit here>.rss")
decoded = response.content
parsed = atoma.parse_atom_bytes(decoded)

will yield
raise FeedXMLError('Not a valid XML document')
It used to work flawlessly. I'll look into the details I can get when debugging and update this Issue accordingly.

atoma/atom.py", line 22

Hi
when i run my script i got this error
Traceback (most recent call last): File "import_shopify.py", line 15, in <module> import atoma File "/usr/lib/python3.4/site-packages/atoma/__init__.py", line 1, in <module> from .atom import parse_atom_file, parse_atom_bytes File "/usr/lib/python3.4/site-packages/atoma/atom.py", line 22 text_type: str = attr.ib() ^ SyntaxError: invalid syntax
THIS MY SCRIPT
`
import atoma, requests

def shopify(url):
response = requests.get(url)
feed = atoma.parse_atom_bytes(response.content)
return feed

url = "https://www.mydomain.com/all.atom"
feed = shopify(url)
print(feed)
`

Thanks in advance for your help

JSON Feed version 1.0 author field cause AttributeError

feed url: https://hnrss.org/newest.jsonfeed
feed content: newest.txt

  File "/Users/kk/.pyenv/versions/rssant377/lib/python3.7/site-packages/atoma/json_feed.py", line 201, in parse_json_feed
    items=_get_items(root)
  File "/Users/kk/.pyenv/versions/rssant377/lib/python3.7/site-packages/atoma/json_feed.py", line 74, in _get_items
    rv.append(_get_item(item))
  File "/Users/kk/.pyenv/versions/rssant377/lib/python3.7/site-packages/atoma/json_feed.py", line 92, in _get_item
    author=_get_author(item_dict),
  File "/Users/kk/.pyenv/versions/rssant377/lib/python3.7/site-packages/atoma/json_feed.py", line 137, in _get_author
    name=_get_text(author_dict, 'name'),
  File "/Users/kk/.pyenv/versions/rssant377/lib/python3.7/site-packages/atoma/json_feed.py", line 173, in _get_text
    rv = root.get(name)
AttributeError: 'str' object has no attribute 'get'

Expect to compatible with 1.0 author field, although it's deprecated.

https://jsonfeed.org/version/1.1

Deprecated items remain valid forever, but you should move to the new fields when you can. A feed using fields from JSON Feed 1.0 is still a valid feed for version 1.1 and future versions of JSON Feed.

And thank you for the handy library! I'm also glad to create PR to fix it if you don't have time, just tell me go ahead.

Atom feed reader wrong error message

Try to parse a RSS feed with the ATOM reader leads to a dubious message:

Python 3.6.5 (default, Apr  1 2018, 05:46:30) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import atoma

In [2]: import requests

In [3]: url = 'https://hyperdev.fr/feed.xml'

In [4]: response = requests.get(url)

In [5]: atoma.parse_atom_bytes(response.content)
---------------------------------------------------------------------------
FeedParseError                            Traceback (most recent call last)
<ipython-input-5-14e8e040a115> in <module>()
----> 1 atoma.parse_atom_bytes(response.content)

~/.local/share/virtualenvs/socialite-xGgT8vff/lib/python3.6/site-packages/atoma/atom.py in parse_atom_bytes(data)
    275     """Parse an Atom feed from a byte-string containing XML data."""
    276     root = parse_xml(BytesIO(data)).getroot()
--> 277     return _parse_atom(root)

~/.local/share/virtualenvs/socialite-xGgT8vff/lib/python3.6/site-packages/atoma/atom.py in _parse_atom(root, parse_entries)
    222 def _parse_atom(root: Element, parse_entries: bool=True) -> AtomFeed:
    223     # Mandatory
--> 224     id_ = get_text(root, 'feed:id', optional=False)
    225 
    226     # Optional

~/.local/share/virtualenvs/socialite-xGgT8vff/lib/python3.6/site-packages/atoma/utils.py in get_text(element, name, optional)
     49 
     50 def get_text(element: Element, name, optional: bool=True) -> Optional[str]:
---> 51     child = get_child(element, name, optional)
     52     if child is None:
     53         return None

~/.local/share/virtualenvs/socialite-xGgT8vff/lib/python3.6/site-packages/atoma/utils.py in get_child(element, name, optional)
     39         raise FeedParseError(
     40             'Could not parse RSS channel: "{}" required in "{}"'
---> 41             .format(name, element.tag)
     42         )
     43 

FeedParseError: Could not parse RSS channel: "feed:id" required in "rss"

In [6]: atoma.__version__
Out[6]: '0.0.9'

That is misleading:

FeedParseError: Could not parse RSS channel: "feed:id" required in "rss"

No simple method for parsing strings

This is sometimes useful, especially if the bytes downloaded use some strange encoding, which needs to be handled manually before.

Atom reader issue

Hi there, I'm having an issue using the atom bytes parse tool. Unfortunately the following code yields an error;

import requests
import atoma

response = requests.get('https://api.icis.com/v1/entities/ref-data/currency', auth=('[email protected]', 'password'))

a = atoma.parse_atom_bytes(response.content)

This yields the following error;

ValueError: 'application/vnd.icis.iddn.entity+xml' is not a valid AtomTextType

If you're able to provide any assistance that would be brilliant!

The shortened redacted file output (including the section that is causing the issues of the above is as follows;

<atom:title>http://iddn.icis.com/ref-data/currency/0</atom:title>
        <atom:id>http://iddn.icis.com/ref-data/currency/0</atom:id>
        <atom:updated>2018-05-20T08:12:00.701348Z</atom:updated>
        <atom:relevance-score>70656</atom:relevance-score>
        <atom:content type="application/vnd.icis.iddn.entity+xml">

long_description_content_type in setup.py

At this time, the readme is not properly formatted on pypi. The setup() function in setup.py may benefit from long_description_content_type='text/x-rst'. That's according to the packaging docs. After this change, please release to pypi to confirm it works.

atoma parse failed with fc2blog rss

For example:

http://fc2information.blog.fc2.com/?xml

In [9]: rss = atoma.parse_rss_bytes(r.content)
---------------------------------------------------------------------------
FeedParseError                            Traceback (most recent call last)
<ipython-input-9-6170d3c57b9f> in <module>
----> 1 rss = atoma.parse_rss_bytes(r.content)

~/.pyenv/versions/3.7.1/lib/python3.7/site-packages/atoma/rss.py in parse_rss_bytes(data)
    217     """Parse an RSS feed from a byte-string containing XML data."""
    218     root = parse_xml(BytesIO(data)).getroot()
--> 219     return _parse_rss(root)

~/.pyenv/versions/3.7.1/lib/python3.7/site-packages/atoma/rss.py in _parse_rss(root)
    165     if rss_version != '2.0':
    166         raise FeedParseError('Cannot process RSS feed version "{}"'
--> 167                              .format(rss_version))
    168
    169     root = root.find('channel')