aemmadi / coursebook-api Goto Github PK

The Problem (with steps to reproduce)

Not all classes return the same amount of details on coursebook.utdallas.edu.

For example:
CS 4337 has a row called Cross Listed With at the bottom of class details, whereas a course like MATH 2418 does not.

When the API is called for MATH 2418 users are presented with a Error Code 500: Internal Server Error, and an index out of bounds error on the server logs.

Potential Solution (optional)

API can no longer assume everything is where its supposed to be. Have to dynamically scrape information returned by coursebook.utdallas.edu.

• Add backend for caching course data into a database
• POST /update to refresh contents of the database

• GET /v1/<course> : Returns class info for all sections for the course
• GET /v1/prof/<term>/<lastname>/<firstname> : Returns all classes taught by professor
• POST /v1/prof/<term>/update : Refreshes database with professors freshly fetched from coursebook

• Reduce the time taken to return a response
• Scrape GET /v1/<course> using multi-threading

Add proper documentation using docsify or swagger

• Render the project's README.md file at / endpoint.
• Render API v1 Documentation at /v1/

The Problem (with steps to reproduce)

API returns a Error Code 500: Internal Server Error for requests that are valid. Occurs randomly, can be fixed if request is re-sent by client.

Potential Solution (optional)

The webscraper is not accounting for any delays caused by coursebook.utdallas.edu.

Currently API only returns Error Code 500: Internal Server Error. Need to send proper error codes depending on the error.

A lot of redundancy calls to O(N) functions. Can make it more efficient by processing data directly during the scraping process.