Git Product home page Git Product logo

dnd-statistics's Introduction

DnD-statistics

A webscraping program reading the die throws from DnD Beyond, categorizes them and calculate statistics on them.

Essentially, DnD Beyond is a powerful website for DnD sessions, but they have no APIs to make use of. So, in order to obtain data behind an account, it is necessary to use webscraping. This brings a plethora of new problems for me, but which i'll try my darndest to solve.

My initial thoughts are to make the program an extension to be downloaded with Chrome Web Store, allowing the extension to be online at the same time as the GM is on and thereby collecting regularly the new dice rolls.

//Update 31-07-2022 I've learned how to web scrape with cheerio, however, my journey has experienced an abrupt stop as to web scraping behind a login page. DnDbeyond requires login with verification from a second party (twitch for instance). I've tried to automate this process with Selenium webdriver, but since their software insists that i use the newest version of Chrome, 104.x.xxx, and the newest my computer apparently wants to cooperate with is 103.x.xxx, I've decided to put this to rest for a bit. Cypress looks interesting and made me realize the behemoth I am against. Through automate testing I figured out that when I automate the process of logging in, DnDBeyond captures it and redirects it to a human verification challenge of which they've blocked the button behind a shadowroot(closed). For now, this means that I'm stranded. My next attempt is to see if I can web scrape from and already logged in session from the web page. Wish me luck!

dnd-statistics's People

Contributors

silverbrother avatar

Watchers

 avatar

dnd-statistics's Issues

Draw statistics out of the rawData

Create statistics based on the arrays of dice rolls.
These statistics will be specified by the customer, but these mathematical equations will be the second last part of the project.

Parse data behind an account

Allow the program access to an account (by their own volition) to web scrape whatever data is available from this site. Shortly. Console.log anything from within an account.

Parse campaign, players, types of dice rolls and the dice rolls

The essential data collecting of the program.

  • First keep it simple, as to obtain the campaign and obtain one players dice rolls and store those in an array.
  • Afterwards, move backwards and store the specific types of dice rolls from the player in an array.
  • Repeat with number of players and numbers of campaigns.

Update data every 5 minutes

Maintain a connection with the website and web scrape for anything new among the dice rolls and store it in the arrays. This update can be based on either time or on activity on the website (such as every new dice roll appearing in the code).

How do I do this if SetInterval doesn't work?

Convert data and statistics into CSV

As described in the title. But can I manipulate with the CSV file to place the data in specific slots?

Once this has been completed, the project will be completed as well

Webscrape from homepage

Using Cheerio to parse data from the public homepage of DnDBeyond. Simply put: Make console.log print out the title of the website.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.