p-society / gc-server Goto Github PK
View Code? Open in Web Editor NEWStay updated in real-time and engage with the thrill of the game like never before.[WIP]
License: Apache License 2.0
Stay updated in real-time and engage with the thrill of the game like never before.[WIP]
License: Apache License 2.0
Please list down other necessary features and suggest API endpoints to implement each feature in this thread.
In light of recent discussions and insights, I'm reconsidering our decision to utilize SSE over Websockets for emitting scores to clients.
During a conversation with a senior colleague, they shared their experience building a product (Hubilo Studio) using Websockets, WebRTC,and a LOT more things, - capable of scaling and handling up to 1L~ concurrent users. This has prompted me to reassess our choice.
He emphasized that Websockets currently offer the most robust solution, particularly considering the ecosystem and industry trends. He highlighted that many companies are shifting towards Websockets for real-time communication due to the comprehensive support and scalability options available and OSS tools around it
Our initial concern (as we , @majorbruteforce and @me discussed ) was scalability for the GC server if we choosed ws. If viable solutions exist for Websockets, it seems prudent to explore them further. The broader support and resources available for it compared to SSE may offer advantages in terms of implementation and future maintenance.
I propose revisiting our technology stack decision and considering the potential benefits of adopting Websockets for our real-time communication needs,as most of the work in the core sections (majorly the crud apis,are completed).
Stuffs like auto-healing, handling backpressure, message sharing between distributed servers using redis, connection retry mechanisms, and ack/nack support might align better with our requirements and facilitate smoother integration and maintenance.
Feedback and opinions on this matter are welcome for further discussion.
CC: @majorbruteforce @punitkr03 @soumil-kumar17
To ensure consistency across environments, we aim to containerize our Auth
, Mail
, and Core
APIs using Docker. This involves creating Dockerfiles for each service and orchestrating them with a Docker Compose configuration.
Auth API: Develop a Dockerfile for our Auth API service, ensuring it includes all necessary dependencies and configurations.
Mail API: Construct a Dockerfile for our Mail API service, considering any specific requirements it may have.
Core API: Build a Dockerfile for our Core API service, including any libraries or tools required for its functionality.
Develop a docker-compose.yml file to orchestrate the deployment and interaction of the Auth, Mail, and Core APIs.
Define service configurations within the Docker Compose file, specifying dependencies, ports, environment variables, and any other necessary settings
Ensure proper networking and communication between the services within the Docker Compose environment.
Document the Dockerfile structure and usage guidelines for each service to facilitate future development and maintenance.
Provide instructions on how to use the Docker Compose configuration for local development, testing, and production deployment.
Have separate collections for Admin/Player and PSA
PSA can have same mail which was used to register as player.
After a successful upgrade:
Admin is responsible for moderating a match and all required API endpoints for that will be described in a separate thread.
The PSA creates a match with the following details:
A squad is an independent participant of the tournament (e.g. CSE-2 , IT-1 etc) and is a super set of teams that will play matches.
PSA will select a set of players from all verified players who have registered to the platform for a squad.
Include details:
Event, Team, Player, Match, Stats and Squad are separate entities.
After PSA assigns a match to an Admin, the admin is responsible to create the team that plays in that match.
Match is a collection that has admin id, squad 1 and 2 id, team 1 and 2 id.
Admin can create teams from 'squad-player' collection
A match generates events
Events has team players + event data + sport + user id+ match id + tbd
A single player will have a single stat collection, stat document for the user will be updated after each event
Events will be trailed to the client in real-time + stored in db + tbd
Events will be the core
To be further discussed: Events implementation, Statistics Calculation, Substitutions.
For instances of the application running on the browser, display the score in the 'title' of the page and update it as changes occur.
The goal of the GCSB project is to develop a robust system with multiple independent and decoupled APIs for sports. Currently, Server-Sent Events (SSE) have been identified as a suitable choice for achieving real-time comm. from server-->client due to their lightweight nature and ease of setup,but there are certain concerns that needs to be solved.
Need for Kafka/Similar Queue or Pub-Sub Messaging:
Question: Considering a maximum of 1000 concurrent users at worst case, do we really need a queuing or pub-sub architecture like Kafka/RabbitMQ? What are the pros and cons? Can SSE alone can handle the expected load or if a more scalable solution is necessary?
Please share your thoughts, concerns, and suggestions regarding the API design uniformity and the need for a queuing/pub-sub architecture in this issue thread.
Consider this as a High Priority Issue.
As our team works on the project, we should recognize the importance of establishing a testing environment to ensure the reliability and maintainability of our codebase. This issue proposes the implementation of unit testing practices and adopting Test-Driven Development (TDD) as part of our development process.
I know it will slow down our Development speed and maybe is a redundant thing,but incorporating TDD from the beginning instills good development practices.
Open for suggestions
So the idea is that we have seen the win probability of google during the IPL and international matches. How about we create a same system but its for our own college and GC!
A functionality can be built to prevent the app from being opened in multiple tabs like WhatsApp web does. This is to cater the limitation that SSE can have only 6 concurrent tabs open at a time and also reduce connections per user with the server.
Currently,The server code is in active development alongside testing phases. Fundamental API endpoints are being developed, setting the groundwork we envision.We are also currently testing the many options we have in the server side development,and it would take some time to reach it's fulfillment.
Concurrently, we shall gear up for the design and development phase of client-side interfaces, for both mobile apps and web interfaces. Integration between front-end and back-end components is crucial for the success of our project and needs collective efforts.๐ท๐บ
This thread will serve as our central hub for discussions concerning the synchronization of front-end and back-end development. Kindly utilize this thread as a PARENT THREAD for all related discussions, keeping the communication streamlined and accessible.
Your insights, thoughts, and recommendations on the design and development process are highly valued.
The goal is to implement Continuous Integration (CI) for the project by using TravisCI / Github Actions (preferred) , which which currently has some Go and Node.js servers (planned). The project structure includes some Go servers and Node.js servers in the src/api
directory.
The current CI is flaky and unreliable.
As the project is now in it's initial stages,lets keep the CI simple - keeping it limited to build and lint checks for the above mentioned languages only.
Project requires a good guide on several topics so that new developers can contribute to the project
We shall be using MongoDB GridFS
for handling multimedia file uploads in our project instead of s3 or blob storage , as the multimedia part is limited (eg:Player Image,etc..) .GridFS is easy to work with (+Cheaper$$).
Open to discussions and feedback regarding the proposed solution.
We have developed a player API to support player dashboards. However, there are several functionalities and enhancements that need to be implemented to improve the overall functionality.
Implement an API endpoint to retrieve all player data.
Ensure proper pagination to handle large datasets.
Include flags or filters to allow for customization of the query results.
Develop an API endpoint to update information for multiple players simultaneously.
Implement validation checks to ensure data integrity and security.
Provide options for bulk updates with proper error handling.
Soft Deletion of Players API:
Create an API endpoint to perform soft deletion of player records.
A flag or status field to indicate deleted players without removing them from the database to be implemented.
Ensure proper handling of soft deletion.
Proper pagination: Implement pagination for large datasets to improve performance and user experience.
Flags and filters: Allow users to customize queries with flags and filters to retrieve specific subsets of data.
Validation checks: Implement validation rules to ensure data integrity and prevent invalid updates.
Error handling: Provide comprehensive error handling to handle various scenarios gracefully and provide meaningful feedback to users.
Security considerations: Implement authentication and authorization mechanisms to restrict access to sensitive endpoints and data.
Documentation: Document APIs and endpoints thoroughly to facilitate integration and usage by developers.
Events are any real world match instances which contain a snapshot of the match. For instance, in cricket: a ball, a wicket, a substitution, an interruption by nature, a new batsman entering the crease etc are all events.
This particular service will be responsible for accepting the data from client (admin), processing it (if required) and pushing it to other layers of services managing SSE connections to the clients (viewers).
Need for Kafka:
Extras to be implemented:
All implementations are to be done using the go library sarama
Refer past discussions regarding usage of Kafka and Sarama: #4
All queries and suggestions related to the event streaming service(s) can be done here.
A Trello work board has been created to assign, track and review the work being done. Contributors are advised to join the board to ensure an organised flow of work.
Click here to view the board.
We have checked compatibility + working of SSE Servers in client side for the web.
We need to develop a rough application for trial purposes to check the ability to capture EventSource (Server-Sent Events or SSE) in Mobile Apps in order to enable real-time updates in client side. This will serve as a crucial step in exploring and implementing real-time communication within our application.
It's just for trial,so no need to have a design,just a simple flutter app that consumes data from a simple SSE server.
One can check out Eventsource
and http
package to implement the same.
Anyone from backend team can assist you in creating a sample SSE Server ,providing you with a SSE Endpoint.
Articles you might find useful :
What is SSE : https://ably.com/topic/server-sent-events
Official eventsource
package at pub.dev : https://pub.dev/packages/eventsource
SSE + Flutter : https://medium.com/flutter-community/server-sent-events-sse-with-flutter-cf331f978b4f
Please Keep in mind,this is NOT Websockets ,This is SSE.
Steps for Development :
app:sse-trial
from main
branch:main
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.