kmeisthax / pd-diffusion Goto Github PK

Wikimedia Commons is ostensibly CC-BY-SA; but there are specific legal rationales, exclusions, and other rights assertions in the wikitext. These need to be parsed to determine the copyright status of the image. At a minimum:

• The date the image was originally drawn, painted, or taken must be known.
• Any copyright claim to label data independent of the underlying image must be known.
• Image creation dates must precede the worldwide copyright cut-off date.
• Copyright claims to label data must not be incompatible with CC-BY-SA.

Any data that breaches these rules should be dropped. Specifically, images with active copyright must be excluded from the dataset at export time, and label data not compatible with CC-BY-SA must be ignored.

Structured wikidata is claimed to be public domain. In practice this is either because the data itself is not copyrightable or because of explicit permissive licensing or dedications. While this may have odd copyright implications in countries that do not recognize public domain dedication, such as Germany, it does not affect the licensing status of generated imagery, so it will be allowed to remain for the time being.

Currently we use Dataset.from_generator to pull data from SQL into a Dataset. This has several problems; most notably, we can't use dataset preloading or other features that require pickle-able datasets.

MediaWiki's API for scraping categories will only include either hidden or non-hidden categories, but not both. We need both in order to enforce category checks elsewhere in the code.

We also need a scraping pass for getting parent category data, since this only returns categories that are directly attached to the page.

Currently, CLIP calculation takes over an hour on a dataset of 90k images. This has to be done every time the training process restarts, which is a pain in the ass.

Intersects with #1 - if we move to static datasets then we need to also store CLIP vectors in that dataset. If we do this before static datasets then we need SQL tables to store CLIP data per trained model.

Wikimedia Commons has a LOT of maps that are uploaded in multiple formats. For example:

https://commons.wikimedia.org/wiki/File:5th_plan,_from_7th_east_to_13th_Street_and_G_Street_south_to_East_Capitol_Street_-_(S.E._Washington_D.C.)._LOC_88694121.jpg

https://commons.wikimedia.org/wiki/File:5th_plan,_from_7th_east_to_13th_Street_and_G_Street_south_to_East_Capitol_Street_-_(S.E._Washington_D.C.)._LOC_88694121.tif

This is inflating the number of maps in the training set and causing the model to overfit on drawing maps rather than the condition specified by CLIP.