Git Product home page Git Product logo

domargan / awesome-dynamic-graphs Goto Github PK

View Code? Open in Web Editor NEW
119.0 8.0 15.0 66 KB

A collection of resources on dynamic/streaming/temporal/evolving graph processing systems, databases, data structures, datasets, and related academic and industrial work

License: Creative Commons Zero v1.0 Universal

graph graphs dynamic-graphs dynamic-graph-processing streaming-graphs streaming-graph-processing temporal-graphs temporal-graph-processing streaming-graph-data graph-databases

awesome-dynamic-graphs's Introduction

Awesome Dynamic Graphs

A collection of resources on dynamic/streaming/temporal/evolving graph processing systems, databases, data structures, datasets, and related academic and industrial work.

Contributing

To add, remove or change things on the list: please submit a pull request to the GitHub repository.

Open Source Projects

Data Structures, Systems and Frameworks

  • Aspen [C++]
    • Streaming graph processing system based on compressed purely-functional trees
    • Paper
  • Differential Dataflow [Rust]
    • General data-parallel programming framework suitable for processing large dynamic graphs
    • Paper
  • GraphBolt [C++]
    • Streaming graph processing system that provides BSP guarantees
    • Paper
  • GraphJet [Java]
    • Streaming graph processing system for generating content recommendations at Twitter
    • Paper
  • GraphOne [C++]
    • Storage engine for streaming graph data
    • Paper
  • GreyCat [Java]
    • Many-world temporal graph data store
    • Paper
  • GoFFish[Java]
    • Multi-source streaming graph processing system
  • KickStarter[C++]
    • Streaming graph processing system optimising for path-based/monotonic graph algorithms
    • Paper
  • LLAMA [C++]
    • CSR-based storage engine and streaming graph processing system
    • Paper
  • PCSR [C++]
    • CSR-inspired, packed memory array based data structure for streaming graphs
    • Paper
  • PPCSR [C++]
    • Parallel and NUMA-aware implementation of PCSR data structure for streaming graphs
  • Raphtory [Scala]
    • Snapshot-based distributed streaming graph processing system
    • Paper
  • STINGER [C/C++]
    • Data structure for streaming graphs based on linked lists of blocks
    • Paper

Benchmarking

  • GraphTides [Java/TypeScript]
    • Framework for evaluating streaming graph processing platforms
    • Paper

Papers

Check Julian Shun's list for a further reference and an extensive collection of related papers.

Data Structures and Storage

STINGER: Spatio-Temporal Interaction Networks and Graphs (STING) Extensible Representation (2009)

STINGER: High Performance Data Structure for Streaming Graphs (HPEC 2012)

Packed Compressed Sparse Row: A Dynamic Graph Representation (HPEC 2018)

GraphOne: A Data Store for Real-time Analytics on Evolving Graphs (FAST 2019)

GraphTinker: A High Performance Data Structure for Dynamic Graph Processing (IPDPS 2019)

CSR++: A Fast, Scalable, Update-Friendly Graph DataStructure (OPODIS 2020)

LiveGraph: A Transactional Graph Storage System With Purely Sequential Adjacency List Scans (VLDB 2020)

A Parallel Packed Memory Array to Store Dynamic Graphs (ALENEX 2021)

Systems and Frameworks

STINGER: Spatio-Temporal Interaction Networks and Graphs (STING) Extensible Representation (2009)

On Querying Historical Evolving Graph Sequences (VLDB 2011)

STINGER: High Performance Data Structure for Streaming Graphs (HPEC 2012)

Facilitating Real-Time Graph Mining (CloudDB 2012)

Kineograph: Taking the Pulse of a Fast-Changing and Connected World (EuroSys 2012)

Differential Dataflow (CIDR 2013)

Efficient Snapshot Retrieval over Historical Graph Data (ICDE 2013)

Naiad: A Timely Dataflow System (SOSP 2013)

Chronos: A Graph Engine for Temporal Graph Analysis (EuroSys 2014)

Towards Large-Scale Graph Stream Processing Platform (WWW 2014)

CellIQ : Real-Time Cellular Network Analytics at Scale (NSDI 2015)

DISTINGER: A Distributed Graph Data Structure for Massive Dynamic Graph Processing (Big Data 2015)

ImmortalGraph: A System for Storage and Analysis of Temporal Graphs (Transactions on Storage 2015)

LLAMA: Efficient Graph Analytics Using Large Multiversioned Arrays (ICDE 2015)

Real-time Analytics for Fast Evolving Social Graphs (CCGrid 2015)

GraphIn: An Online High Performance Incremental Graph Processing Framework (Euro-Par 2016)

GraphJet: Real-Time Content Recommendations at Twitter (VLDB 2016)

Storing and Analyzing Historical Graph Data at Scale (EDBT 2016)

Synergistic Analysis of Evolving Graphs (TACO 2016)

Time-Evolving Graph Processing at Scale (GRADES 2016)

Tornado: A System For Real-Time Iterative Analysis Over Evolving Data (SIGMOD 2016)

Towards a Distributed Large-Scale Dynamic Graph Data Store (IPDPSW 2016)

Automatic Algorithm Transformation for Efficient Multi-Snapshot Analytics on Temporal Graphs (VLDB 2017)

KickStarter: Fast and Accurate Computations on Streaming Graphs via Trimmed Approximations (ASPLOS 2017)

Auxo: A Temporal Graph Management System (BDMA 2018)

Distributed Evaluation of Subgraph Queries Using Worst-case Optimal Low-Memory Dataflows (VLDB 2018)

GraPU: Accelerate Streaming Graph Analysis through Preprocessing Buffered Updates (SOCC 2018)

GraphBolt: Dependency-Driven Synchronous Processing of Streaming Graphs (EuroSys 2019)

Low-Latency Graph Streaming Using Compressed Purely-Functional Trees (PLDI 2019)

GraphOne: A Data Store for Real-time Analytics on Evolving Graphs (FAST 2019)

LiveGraph: A Transactional Graph Storage System With Purely Sequential Adjacency List Scans (VLDB 2020)

Raphtory: Streaming analysis of distributed temporal graphs (FGCS 2020)

RisGraph: A Real-Time Streaming System for Evolving Graphs (2020)

Tegra: Efficient Ad-Hoc Analytics on Time-Evolving Graphs (NSDI 2021)

Surveys

The Ubiquity of Large Graphs and Surprising Challenges of Graph Processing (VLDB 2018)

Practice of Streaming and Dynamic Graphs: Concepts, Models, Systems, and Parallelism (2020)

Datasets

Note: Very small datasets are not included in the list.

Category Data Source Name Type Vertex Meaning Edge Meaning Size (|V|) Volume (|E|) Edge Weight Edge Timestamps Multiple Edges Source/Link
Authorship network Wikipedia edit-enwiki Bipartite, undirected User, article Edit 50,757,442 572,591,272 N Y Y http://konect.cc/networks/edit-enwiki/
Citation network DBLP dblp-cite Unipartite, directed Publication Citation 12,590 49,759 N Y N http://konect.cc/networks/dblp-cite/
Co-citation network arXiv ca-cit-HepPh Unipartite, undirected Author Co-citation 28,093 4,596,803 N Y Y http://konect.cc/networks/ca-cit-HepPh/
Co-citation network arXiv ca-cit-HepTh Unipartite, undirected Author Co-citation 22,908 2,673,133 N Y Y http://konect.cc/networks/ca-cit-HepTh/
Communication network Facebook facebook-wosn-wall Unipartite, directed User Wall post 46,952 876,993 N Y Y http://konect.cc/networks/facebook-wosn-wall/
Communication network Linux kernel mailing list lkml-reply Unipartite, directed Person Reply 63,399 1,096,440 N Y Y http://konect.cc/networks/lkml-reply/
Computer network Internet topology topology Unipartite, undirected Autonomous system Connection 34,761 171,403 N Y Y http://konect.cc/networks/topology/
Feature network BibSonomy bibsonomy-2ti Bipartite, undirected Tag, publication Assignment 972,120 2,555,080 N Y Y http://konect.cc/networks/bibsonomy-2ti/
Feature network CiteULike citeulike-ti Bipartite, undirected Tag, publication Assignment 885,046 2,411,819 N Y Y http://konect.cc/networks/citeulike-ti/
Feature network Delicious delicious-ti Bipartite, undirected Tag, URL Tag assignment 38,289,740 301,183,605 N Y Y http://konect.cc/networks/delicious-ti/
Feature network Google+ stevgong_attribute Bipartite, undirected User, attribute Attribute-link 10,442,421 20,592,962 N Y N http://konect.cc/networks/stevgong_attribute/
Feature network Twitter munmun_twitterex_ti Bipartite, undirected Hashtag, URL Co-occurrence 1,502,611 2,635,885 N Y Y http://konect.cc/networks/munmun_twitterex_ti/
Human contact network MIT mit Unipartite, undirected Person Contact 96 1,086,404 N Y Y http://konect.cc/networks/mit/
Hyperlink network Wikipedia link-dynamic-dewiki Unipartite, directed Article Reference 2,166,669 86,337,879 N Y N http://konect.cc/networks/link-dynamic-dewiki/
Hyperlink network Wikipedia link-dynamic-frwiki Unipartite, directed Article Reference 2,212,682 59,008,831 N Y N http://konect.cc/networks/link-dynamic-frwiki/
Hyperlink network Wikipedia link-dynamic-itwiki Unipartite, directed Article Reference 1,204,009 34,826,283 N Y N http://konect.cc/networks/link-dynamic-itwiki/
Hyperlink network Wikipedia link-dynamic-nlwiki Unipartite, directed Article Reference 1,039,252 20,070,561 N Y N http://konect.cc/networks/link-dynamic-nlwiki/
Hyperlink network Wikipedia link-dynamic-plwiki Unipartite, directed Article Reference 1,033,050 25,026,208 N Y N http://konect.cc/networks/link-dynamic-plwiki/
Hyperlink network Wikipedia link-dynamic-simplewiki Unipartite, directed Article Reference 100,312 1,627,472 N Y N http://konect.cc/networks/link-dynamic-simplewiki/
Interaction network LastFm lastfm_band Bipartite, undirected User, band Listening 175,069 19,150,868 N Y Y http://konect.cc/networks/lastfm_band/
Interaction network LastFm lastfm_song Bipartite, undirected User, song Listening 1,085,612 19,150,868 N Y Y http://konect.cc/networks/lastfm_song/
Online contact network Stack Exchange sx-askubuntu Unipartite, directed User Answer/comment 159,316 964,437 N Y Y http://konect.cc/networks/sx-askubuntu/
Online contact network Stack Exchange sx-mathoverflow Unipartite, directed User Answer/comment 24,818 506,550 N Y Y http://konect.cc/networks/sx-mathoverflow/
Online contact network Stack Exchange sx-stackoverflow Unipartite, directed User Answer/comment 2,601,977 63,497,050 N Y Y http://konect.cc/networks/sx-stackoverflow/
Online contact network Stack Exchange sx-superuser Unipartite, directed User Answer/comment 194,085 1,443,339 N Y Y http://konect.cc/networks/sx-superuser/
Online contact network Twitter higgs-twitter Unipartite, directed User Retweet/mention/reply ? 563,069 N Y Y (?) https://snap.stanford.edu/data/higgs-twitter.html
Online contact network Twitter munmun_twitterex_at Unipartite, directed User Mention 2,919,613 12,887,063 N Y Y http://konect.cc/networks/munmun_twitterex_at/
Online contact network Wikipedia elec Unipartite, directed User Vote 7,118 103,675 N Y N http://konect.cc/networks/elec/
Online contact network Wikipedia wikiconflict Unipartite, undirected User Edit conflict 118,100 2,917,785 Y Y Y http://konect.cc/networks/wikiconflict/
Online social network Bitcoin soc-bitcoin Unipartite, directed Member Trust/distrust 24,575,382 122,948,162 ? Y N https://networkrepository.com/soc-bitcoin.php
Online social network Digg digg-friends Unipartite, directed User Vote 279,630 1,731,653 N Y N http://konect.cc/networks/digg-friends/
Online social network Facebook facebook-wosn-links Unipartite, undirected User Friendship 63,731 817,035 N Y N http://konect.cc/networks/facebook-wosn-links/
Online social network Flickr flickr-growth Unipartite, directed User Friendship 2,302,925 33,140,017 N Y N http://konect.cc/networks/flickr-growth/
Online social network Google+ stevgong_social Unipartite, directed User Interaction 28,943,739 462,994,069 N Y N http://konect.cc/networks/stevgong_social/
Online social network YouTube youtube-u-growth Unipartite, undirected User Friendship 3,223,589 9,375,374 N Y N http://konect.cc/networks/youtube-u-growth/
Rating network Amazon rec-amz-Books Bipartite, undirected User, product Rating 10,356,390 22,507,155 Y Y N https://networkrepository.com/rec-amz-Books.php
Rating network Epinions epinions-rating Bipartite, undirected User, product Rating 876,252 13,668,320 N Y Y http://konect.cc/networks/epinions-rating/
Rating network Netflix netflix Bipartite, undirected User, movie Rating 497,959 100,480,507 Y Y N http://konect.cc/networks/netflix/
Rating network Yahoo yahoo-song Bipartite, undirected Person, song Rating 1,625,951 256,804,235 Y Y N http://konect.cc/networks/yahoo-song/

About

This list was compiled by Domagoj Margan with help and resources from the systems community.

awesome-dynamic-graphs's People

Contributors

domargan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.