Regarding the Wikidata mongo dump about neuraldb HOT 12 CLOSED

Also, which version of wikidata did you use in wikidata_file (for wikidata_index.py)?

from neuraldb.

Was based off January 2021. The DB can be re-created from the dump. But it takes a few days to decompress the file and index it. The Wikidata file will be released with the other data when the infrastructure's been set up.

from neuraldb.

Looking at the wikimedia website it looks like the oldest dump they have is from march.
https://dumps.wikimedia.org/wikidatawiki/
Am I missing something?

from neuraldb.

I think Wikimedia only keep the last 2-3 releases.

from neuraldb.

I think I found a mirror.
https://archive.org/details/wikidatawiki-20210120
Can you confirm?
Also, if this is indeed the correct dump, are all the 250GB necessary for the preprocessing?

from neuraldb.

Or is it this one:
https://archive.org/download/wikibase-wikidatawiki-20210125/wikidata-20210125-all.json.bz2

from neuraldb.

I seem to remember it being about 50Gb

from neuraldb.

I see.
Also, do you think it is possible to use this to create the mongo dump?
Seems like it is the same format, and will take around 8 hours to process.

from neuraldb.

@j6mes When would the mongo_wikidata_dump.gz be available?

from neuraldb.

same issue, can't find the mongo_wikidata_dump.gz

from neuraldb.

Could the mongo_wikidata_dump.gz be shared? Thanks.

from neuraldb.

It has recently been uploaded to the page. Apologies for the delay.

from neuraldb.