Git Product home page Git Product logo

twitter-engine's Introduction

Twitter Engine for Highly Concurrent Systems

Objective

The goal of this first part of a two part project is to implement the user/engine interaction and test it using a simulator built over it. It is supposed to be a twitter clone with functionalities that mimic the real social networking service. This is implemented using the concurrent programming model of Elixir[1], The Actor Model.

Architecture

Architecture Diagram The Engine has three main parts:

  • The Client:
    • This module is a GenServer that mimics the functionality of logged in application provided to the user by the service. For example Twitter for Android.
  • The User:
    • This module is a GenServer that identifies as a part of the engine itself and demark the presence of a signed up user in the whole network. The customer/client can interact with it, yet it runs as a part of the engine.
  • The Relay:
    • This is the component that forms the communication bridge between various signed up users. All the tweets, follow notifications etc. crossover from one user to the another via relay as the mediator.

Functionalities

We Provide the following functionalities:

  • Signup for a new user
  • Login/Logout for an existing user
  • Follow for existing users from A to B
  • Tweet by an existing user
  • Live delivery of tweets if a user is logged in via a registered client
  • Support for mentioning another user in tweets and retweets, which generates a notification for the person mentioned
  • Support for creating new, following and using hastags in tweets and retweets
  • Retweet by an existing user of an already existing tweet by another existing user
  • Query tweets by an existing user on the basis of
    • Tweets that have been mentioned in
    • Tweets by users they follow
    • Tweets by hashtags they follow
    • Tweets they made
  • Fetch one users followers as well as all the users they follow
  • Population of an users timeline by leveraging the above functionality
  • Deletion of a user if requested along with its followed-by table and its entry in all the other users’ followed-by tables

Modules

The above functionalities are provided through an intricate interplay of the modules, which are: Module Diagram

Observations

Observation Table Observation Graph The above table and graph show how much time was taken for each user to receive a copy of a tweet on its logged in client. Note that each user other than the one tweeting was subscribed to receive tweet. We would also like to mention that, after the 5000 users mark, the CPU of the machine started to throttle due to heat and hence the results after that are not on the same processor clock speed. This was tested on an Intel i7-8750H hex core CPU.

Zipf Distribution of followers

In this project we have simulated a Zipf distribution of followers. The simulation has been done based on the number of followers/subscribers for all active users. We take the maximum subscribers a user can have as an input parameter and the user with most number of followers are simulated to have those many number of subscribers, and the user with the second most number of followers had maximum subscribers/2 and the user which is third in rank having maximum subscribers/3 and so on. This has been implemented using a reduce algorithm and accumulators. Zipf first graph Zipf second graph Zipf third graph

Results

Even though we were able to host a maximum of 100,000 users on the test machine, we are certain that this engine can easily do more due to its highly distributive nature, even within the engine. This Engine provides a layered architecture which finally exposes a top layer API to host a twitter like engine along with compatible clients.

Dependencies

  • elixir~>1.9.2
  • mix~>1.9.2

Preferred choice of OS

  • GNU/Linux

References

[1] Elixir

twitter-engine's People

Contributors

dronabanerjee avatar r0ck3r008 avatar

Stargazers

 avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.