After exploring and analysing the yelp dataset, I was interested to play around with other big datasets. After some googling, I found the Stack Overflow dataset from the Internet archive. I downloaded the following dump files using the torrent link:

• stackoverflow.com-Posts.7z - 16.58 GB
• stackoverflow.com-Users.7z - 857.7 KB
• stackoverflow.com-Tags.7z - 733.0 MB

1. Unzip the .7z Files

   for i in *.7z; do 7za -y -oextracted x $i; done

2. 3 xml files were extracted:

   • Posts.xml - 90.21GB
   • Tags.xml - 5.5 MB
   • Users.xml - 5.02 GB

3. Initial graph model visualised using arrows.app
   

4. Convert to csv format for importing using the script provided [python3 to_csv.py extracted]:

   • 53086330 posts records processed in 67mins
   • 14839629 users records processed in 10mins
   • 61061 tag records processed

./bin/neo4j-admin import --multiline-fields=true  --skip-bad-relationships \
--nodes=Post=./import/sof/posts.csv  \
--nodes=User=./import/sof/users.csv  \
--nodes=Tag=./import/sof/tags.csv  \
--nodes=PostType=./import/sof/posttypes.csv \
--relationships=PARENT_OF=./import/sof/posts_rel.csv \
--relationships=HAS_TAG=./import/sof/tags_posts_rel.csv \
--relationships=POSTED=./import/sof/users_posts_rel.csv \
--relationships=IS_TYPE=./import/sof/posts_type_rel.csv \
--database=sof

5. The import took 5mins on my desktop

IMPORT DONE in 5m 2s 17ms. 
Imported:
  67987014 nodes
  147194601 relationships
  403463278 properties
Peak memory usage: 1.756GiB

What is the breakdown of all the posts?

MATCH(p:Post)-[:IS_TYPE]->(t:PostType)
RETURN t.type, count(*) as numPosts

Both the Cypher commands can find the Top 10 Users, but one is faster than the other. Use EXPLAIN to see why.

But I digress, the results of both commands:

Seems like Gordon Linoff is quite the expert at databases and SQL. He has only answered questions and have not started a question thread himself!

MATCH (u:User {userId: '1144035'})-[:POSTED]-(answers:Post)<-[:PARENT_OF]-(q:Post)-[:HAS_TAG]->(t:Tag)
RETURN t.tagId, count(distinct answers) as numAnswers
ORDER BY numAnswers DESC LIMIT 10

MATCH (t:Tag {tagId:'neo4j'})<-[:HAS_TAG]-()
       -[:PARENT_OF]->()<-[:POSTED]-(u:User) 
WITH u, count(*) AS freq ORDER BY freq DESC LIMIT 10
RETURN u.displayname,freq

And other categories that they are active in

MATCH (neo:Tag {tagId:"neo4j"})<-[:HAS_TAG]-()
      -[:PARENT_OF]->()<-[:POSTED]-(u:User) 
WITH neo,u, count(*) as freq order by freq desc limit 10
MATCH (u)-[:POSTED]->()<-[:PARENT_OF]-(p)-[:HAS_TAG]->(other:Tag)
WHERE NOT (p)-[:HAS_TAG]->(neo)
WITH u,other,count(*) AS freq2 ORDER BY freq2 DESC 
RETURN u.displayname, collect(distinct other.tagId)[1..5] AS tags