Working example of the entire script can be viewed here: https://tuinwestor.pl/newsy
News aggregation repository to parse newspapers with a given list of RSS FEED. Articels are analyzed using NLP techniques to detect similar topics, estimate text readability and reading time. Then, articles are processed to detect word variations of given companies in MySQL DB table. Based on the text analysis, the algorithm selects the most important topics of the day and groups similar articles concerning the same company.
Used libraries:
- mysql.connector
- dotenv
- nltk
- pandas
- numpy
- sqlalchemy
- sklearn
- spacy
- re
- time
- os
- random
- scrappy framework
Prior to running the algorithm, the tables must be imported to the MYSQL database.