mmw125 / mudima Goto Github PK

File "/Users/mshuler/Documents/MuDimA/server/news_fetcher_test.py", line 42, in test_get_top_headlines
self.assertEqual(self.top_headline.get_source().get_id(), "usa-today")
AttributeError: 'dict' object has no attribute 'get_id'

anyone else getting this error when building the database?

Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/newspaper/parsers.py", line 54, in fromstring
cls.doc = lxml.html.fromstring(html)
File "/Library/Python/2.7/site-packages/lxml/html/init.py", line 706, in fromstring
doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
File "/Library/Python/2.7/site-packages/lxml/html/init.py", line 600, in document_fromstring
value = etree.fromstring(html, parser, **kw)
File "lxml.etree.pyx", line 3032, in lxml.etree.fromstring (src/lxml/lxml.etree.c:68121)
File "parser.pxi", line 1786, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:102470)
File "parser.pxi", line 1667, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:101229)
File "parser.pxi", line 1035, in lxml.etree._BaseParser._parseUnicodeDoc (src/lxml/lxml.etree.c:96139)
File "parser.pxi", line 582, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:91290)
File "parser.pxi", line 683, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:92476)
File "parser.pxi", line 631, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:91904)
XMLSyntaxError: line 396: Element script embeds close tag
You must download and parse an article before parsing it!

It seems like there is an issue with the article URLs. Not all URLs will be valid and they throw this specific error. I will look more into it before assigning a fix.

grab favicons from each source for front end display

www.url.com/favicon.ico

Communicate with us on what to ask users. We want to know what is missing in the news sources today

requirement: keywords cannot be boring

When calling localhost/updateStories table is not updated because:

OperationalError: table article has no column named date

[2017-10-23 00:38:21,809] ERROR in app: Exception on /updateStories [GET]
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Library/Python/2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/Library/Python/2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Library/Python/2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/Library/Python/2.7/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "app.py", line 17, in update_stories
news_fetcher.update_database()
File "/Users/mshuler/Documents/MuDimA/server/news_fetcher.py", line 67, in update_database
database_writer.write_topics_to_database(grouped)
File "/Users/mshuler/Documents/MuDimA/server/database_writer.py", line 21, in write_topics_to_database
grouping.get_uuid()))
OperationalError: table article has no column named date
127.0.0.1 - - [23/Oct/2017 00:38:23] "GET /updateStories HTTP/1.0" 500 -

There was an article included in the database which had 5 words and was attached to a cluster. This is not ideal because it looks like an outlier

Pagination is currently only enabled server-side. We need this since we currently only show a limited number on the home screen. I tried in the pagination branch, but was unable to get it to work. We could also use an infinite scroller instead such as https://github.com/CassetteRocks/react-infinite-scroller/blob/master/docs/src/index.js.

• All of the articles are graphed.
• Figure out a calculation to enclose all of the articles within a cluster.
• Consider overlap of topics.
• Pan feature to see all of the articles.

Script runs for over 9 seconds for 9 articles. Need further evidence that it's the algorithm and not the http requests. I will test mocking the http requests so they do not need to be downloaded before assigning a fix.

news_fetcher.get_top_headlines() is reading from the following json

{
"source": {
"id": null,
"name": null
},
"author": null,
"title": null,
"description": null,
"url": null,
"urlToImage": null,
"publishedAt": null
},

  self.assertEqual(self.top_headline.get_source().get_id(), None)

E AssertionError: u'the-next-web' != None

_______________ RigerousClassifierTest.test_classifier_grouping ________________

self = <server.classifier_test.RigerousClassifierTest testMethod=test_classifier_grouping>

def test_classifier_grouping(self):
    self.concat_articles = self.hurricane_harvy_urls + self.tom_petty_urls + self.las_vegas_urls

  self.assertEqual(3, len(classifier.group_articles(self.concat_articles)))

E AssertionError: 3 != 418

server/classifier_test.py:49: AssertionError
__________________ FindURLErrorTest.test_keywordless_articles __________________

self = <server.classifier_test.FindURLErrorTest testMethod=test_keywordless_articles>

def test_keywordless_articles(self):
    """Tests that keywordless articles are put into a separate grouping."""
    articles = [models.Article(url="example.com", keywords=[]), models.Article(url="test.com", keywords=[])]
    articles.extend(test_utils.SIMILAR_ARTICLES)
    groups = classifier.group_articles(articles)

  self.assertEqual(2, len(groups))

E AssertionError: 2 != 418

server/classifier_test.py:67: AssertionError
_____________________ FindURLErrorTest.test_parsing_issue ______________________

self = <server.classifier_test.FindURLErrorTest testMethod=test_parsing_issue>

def test_parsing_issue(self):

  self.assertEqual(1, len(classifier.group_articles(self.url)))

E AssertionError: 1 != 417

server/classifier_test.py:58: AssertionError

codelucas/newspaper#471

Don't know of any solutions. My AWS deployment is on halt until this solution is fixed. Is anyone else getting this issue?

• graph all articles on front end

Sort based on:

• newest
• popular?
• search bar
• type of article

Find a way to track users on click of an article and enable them to have a reputation to be able to vote on the article. Maybe time spent on the article?

A bubble should pop up with appropriate information when hovering over an article

The server should contain another webserver, outside of localhost, for the users.

make it nice and use printer paper.

This will help us get on the same page of what the front end will look like