Summary

Write unit tests to add support coverage to exporter / importer functionality

Background

Unit tests ensure code remains functional and healthy across changes. It lowers the barrier for contributions and boosts the confidence of potential contributors to make a new change. It also enables different tools such as continuous integration based on unit tests.

Details

Explore pytest again. See test_demo.py locally for an example of how to do this.

Outcome

Project functionality is ensured through unit tests and clear indicators provided when project breaks; increased confidence of new contributors to make changes

exporting from lastfm is running into parsing issues it seems and importing into librefm is stopping after a few thousands scrobbles, too.

Try the user "widardd" on lastfm, page 463 and 464 (of 1673) for example. If I skip both it works again until 470 only to stop again:

Got page 470 of 1673.. Wrote page 465-470 of 1673 to file export3.json Traceback (most recent call last): File "./lastexport.py", line 204, in <module> main(server, username, startpage, outfile, infotype) File "./lastexport.py", line 182, in main for page, totalpages, tracks in get_tracks(server, username, startpage, tracktype=infotype): File "./lastexport.py", line 163, in get_tracks tracklist = get_tracklist(response) File "./lastexport.py", line 110, in get_tracklist xmlpage = ET.fromstring(response) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1311, in XML parser.feed(text) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1659, in feed self._raiseerror(v) File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1523, in _raiseerror raise err xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 45, column 127

Summary

Support JSON format for exporting history from a platform

Background

Currently, last2libre writes data out to a text file separated by tabs and new lines. It would be cleaner to offer support for JSON export. This puts the data into a more easily programmable format for other developers to use the data for their wishes.

Additionally, the importer module can be updated to read in the JSON instead of doing that step manually like it is now.

Details

Write all entries to a dict where each key represents a unique listen. All the values in the dict represent different metadata pieces. At the end of the import process, write the dict out as JSON instead of a text file.

Ideally, if all text file functionality can be deprecated, I consider this a major win.

Outcome

Support data export format to improve ease of use for external developers / users

Summary

Use Python logging to log various error messages as configured

Background

Currently, the application emits various print statements to notify of successful and failed behavior. It makes more sense to bring these print statements into the logger module to more effectively enable logging of errors and notices by the application. This makes the application friendlier to use for someone automating the application.

Details

Exploring Python logging is a first step. Later, integrating the debug flag as a global argument would be helpful to enable verbose logging across the application.

Outcome

Logged errors and notices are more visible and effectively captured than what print statements can offer