Verify that all occuring elements are handled by the parser. One possible solution might be the output of xml-tags in the default part of the parsers switch construct.

The modelling of the release identifier data is bad. It is not clear if the attributes value, type and description are enough to unambiguously identify every record. May be directly add the release id as a foreign key and add the release to the primary key together with the value or type.

Most likely there will be parts that could be shared between the model files of all parser. Analyze the classes to find these parts and put them in a common super class to remove or reduce duplication.

Locate the reason for failing inserts due to duplicate keys. Should the primary composited of more attributes than the id? May the primary keys should be entirely dropped on import?

Start a separate thread for each file; this way the files can be parsed and written to the database parallel and fast then sequential.

Gathering stats seems to block the execution -> Singleton is bottleneck.

Create a gatherer for each writer, collect statistics parallel and merge them at the end.

Reorganize the scripts to simplify the creation and deletion of the schema. Create an install and uninstall script that hides the single scripts but leave the single scripts to allow the creation and deletion of the work or import schema for development purposes.

Image objects have no attribute or no attribute collection that uniquely identifies each tuple. Add a surrogate to to uniquely identify every single tuple.

Determine where the videos of masters are overlooked:
-during parsing?
-during writing to the database?

Both available schemas have different table names and as a consequence the statistics need to adapt to this.

After preprocessing drop the import tables, as they are not required anymore.

Finish the backend to write the parsed data to a PostgreSQL database

A releases-file possibly contains tracks, enclosed by , at at least two places. A superficial examination did not show any instances of existing tracks. Continue examination of the releases file and find track entities to complete the parser.

The tracks, or tracklist, of a release is not written after parsing.

The following things have to be implemented for this:

• db Table ReleaseTrack (id [surrogate key],position, title, duration)
• db Table track_of_release(track_id, release_id)
• PreparedStatement to write read entity to database
• SQL statement to write/transform data from import schema to work schema

The diagram modelling the data for releases is missing the entity for release formats.

The er-diagram for releases misses the entity for release videos and the relationship to connect releases and release videos.

1. Let the main-thread wait for completion of the workers
2. Manage the workers in a thread pool, in case there are less than three cores available

Rename relations by using a prefix to allow a group attribution by looking at the first letter of a relation.

src is an unfitting primary key. Add the release add as a foreign key and drop the (useless) connection table.

Add tests to verify correctness by examples instead of checking against a huge dump file.

A masters-file possibly contains tracks, enclosed by . A superficial examination did not show any instances of existing tracks. Continue examination of the masters file and find track entities to complete the parser.

Rethink if a surrogate primary key is neccessary. Maybe a primary key, referencing the release is enough in combination with the position attribute.

In the transformation step for some entities the rows are aggregated based on the primary key, where column values are aggregated in arrays. These often contain null values or duplicate values. Examine if deduplication is required or if deduplication possibly destroys a relation between array rows of the same relation.

Operations on import tables are very slow. Add (matching) indexes to speed up queries on the import tables.

Use sql queries to preprocess the imported data and to import them in to the final tables, obsoleting the import tables

Test the backend with unit tests before hand. In this case the parsing has not to be executed before possible trouble in the db stuff.

Reevaluate the usage of arrays of a way to aggregate informations about the same object (primary key) to avoid key constraint issues. These informations are related to each other and should be stored together, not separately in separate arrays where association by the same array field number can only be assumed, but not enforced and ensured.

This affects all relations that aggregate informations in arrays during the transformation process.

Currently the parsers directly use the PostgreSQL-Connector to write the entities to the database. Abstract from the concrete implementation and use a generic DB-Connector, that hides the concrete implementation.

• the 'master'-tag has an id-attribut
  -the 'main_release'-tag is missing

Create artificial relationships to connect the disparate parts of the database using the artist id's.

Let the parser only write strings to intermediate data structures (*Entity). The conversion is applied, if neccessary, in the db backend.

Add incrementing statements to importers to fill statistics

Some LabelEntities have null elements, e.g. id or data quality. Fix the parser to allow the import into the database.

• connect to db on program startup
• probe for existence of the required schema/tables before parsing
  -> saves a lot of time and anger

Release formats seem to be missing as the getters are not used in ReleaseFormat.java.

Locate the place where the formats are overlooked.

Several tables are empty after importing the dumps in the database. Analyze whether there is an error between parsing and storing in the database or if these data are not existing in the source dumps.