samgozman / finra-short-api Goto Github PK

View Code? Open in Web Editor NEW

9.0 2.0 1.0 3.54 MB

API for TightShorts. FINRA short sales parser and screener. Backend.

Home Page: https://tightshorts.extr.app/

License: MIT License

JavaScript 0.61% Shell 0.28% TypeScript 98.30% Dockerfile 0.81%

stocks stock-market nestjs mongodb finra finance

finra-short-api's Introduction

Hi there 👋

My name is Sam Gozman, 27yo 🇮🇱, speaking in en & ru. I started programming at the age of 12 and have been working full-time as a backend developer since 2016, mainly in the fintech sector. My main programming languages are Golang, TypeScript and Rust. I have been a member of the GitHub open-source community for a long time :)

If you are interested in learning more about me, my OSS projects or reading a dev blog that I've rebuilt recently - I encourage you to visit my website: 👨‍💻 https://gozman.space

📫 How to reach me: Telegram (preferably), Email

📇 LinkedIn: in/samgozman

finra-short-api's People

Contributors

Stargazers

Watchers

Forkers

oskarhulter

finra-short-api's Issues

Make the transition to GraphQL

https://docs.nestjs.com/graphql/quick-start

Restart last update process

Restar the last filter update process if it was interrupted before its completion. (for example, when PM2 restarts the application without my request and does not write logs about it)

[bug] Error on each request

After latest dep updates

finra-api_1  | This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
finra-api_1  | Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
finra-api_1  |     at new NodeError (node:internal/errors:371:5)
finra-api_1  |     at ServerResponse.setHeader (node:_http_outgoing:576:11)
finra-api_1  |     at ServerResponse.header (/app/node_modules/express/lib/response.js:776:10)
finra-api_1  |     at ServerResponse.send (/app/node_modules/express/lib/response.js:170:12)
finra-api_1  |     at ServerResponse.json (/app/node_modules/express/lib/response.js:267:15)
finra-api_1  |     at ExpressAdapter.reply (/app/node_modules/@nestjs/platform-express/adapters/express-adapter.js:42:62)
finra-api_1  |     at ExceptionsHandler.handleUnknownError (/app/node_modules/@nestjs/core/exceptions/base-exception-filter.js:38:24)
finra-api_1  |     at ExceptionsHandler.catch (/app/node_modules/@nestjs/core/exceptions/base-exception-filter.js:17:25)
finra-api_1  |     at ExceptionsHandler.next (/app/node_modules/@nestjs/core/exceptions/exceptions-handler.js:16:20)
finra-api_1  |     at /app/node_modules/@nestjs/core/router/router-proxy.js:24:35
finra-api_1  |     at Layer.handle_error (/app/node_modules/express/lib/router/layer.js:71:5)
finra-api_1  |     at trim_prefix (/app/node_modules/express/lib/router/index.js:321:13)
finra-api_1  |     at /app/node_modules/express/lib/router/index.js:284:7
finra-api_1  |     at Function.process_params (/app/node_modules/express/lib/router/index.js:341:12)
finra-api_1  |     at next (/app/node_modules/express/lib/router/index.js:275:10)
finra-api_1  |     at Layer.handle_error (/app/node_modules/express/lib/router/layer.js:67:12)

Add Docker config

Filter where 5-day averages are greater / less than 20-day averages.

Log progress

Modify nestjs logger to be able to track updater progress

Prevent data manipulation during update

Block any data manipulation during the update.

Add rate limiter protection

https://docs.nestjs.com/security/rate-limiting

Refactor filters factory (Go?)

To utilize minimum parameters. Maybe start using Go for filters calculation process

Health checks

Add HTTP checks for FINRA and tinkoff api.
Add MongooseHealthIndicator

https://docs.nestjs.com/recipes/terminus

{"log":"\u001b[33m[Nest] 25  - \u001b[39m05/05/2022, 1:30:00 AM \u001b[33m   WARN\u001b[39m \u001b[38;5;3m[CollectionController] \u001b[39m\u001b[33m(¬_¬) CRON updater task has started\u001b[39m\n","stream":"stdout","time":"2022-05-04T22:30:00.030188805Z"}
{"log":"\u001b[33m[Nest] 25  - \u001b[39m05/05/2022, 1:30:00 AM \u001b[33m   WARN\u001b[39m \u001b[38;5;3m[CollectionService] \u001b[39m\u001b[33mFetching last day data from FINRA has started\u001b[39m\n","stream":"stdout","time":"2022-05-04T22:30:00.030579406Z"}
{"log":"\u001b[32m[Nest] 25  - \u001b[39m05/05/2022, 1:30:17 AM \u001b[32m    LOG\u001b[39m \u001b[38;5;3m[CollectionService] \u001b[39m\u001b[32mfillDataBase: first duplication error. Data recording is stopped\u001b[39m\n","stream":"stdout","time":"2022-05-04T22:30:17.379451092Z"}
{"log":"\u001b[32m[Nest] 25  - \u001b[39m05/05/2022, 1:30:17 AM \u001b[32m    LOG\u001b[39m \u001b[38;5;3m[CollectionService] \u001b[39m\u001b[32mFetching last day data from FINRA has finished\u001b[39m\n","stream":"stdout","time":"2022-05-04T22:30:17.380016141Z"}
{"log":"\u001b[32m[Nest] 25  - \u001b[39m05/05/2022, 1:30:17 AM \u001b[32m    LOG\u001b[39m \u001b[38;5;3m[CollectionController] \u001b[39m\u001b[32m(¬_¬) CRON updater task has finished\u001b[39m\n","stream":"stdout","time":"2022-05-04T22:30:17.38044083Z"}

0|finra-sh | [Nest] 133516  - 09/27/2021, 8:30:46 AM   ERROR [FilterUnitService] Error in abnormalVolumeFilter
0|finra-sh | [Nest] 133516  - 09/27/2021, 8:30:46 AM   ERROR [FilterUnitService] MongoNetworkTimeoutError: connection timed out
0|finra-sh | [Nest] 133516  - 09/27/2021, 8:30:46 AM   ERROR [FiltersService] Error in updateAll
0|finra-sh | [Nest] 133516  - 09/27/2021, 8:30:46 AM   ERROR [FiltersService] InternalServerErrorException: Internal Server Error
0|finra-sh | [Nest] 133516  - 09/27/2021, 8:30:46 AM   ERROR [FilterUnitService] Error in abnormalVolumeFilter
0|finra-sh | [Nest] 133516  - 09/27/2021, 8:30:46 AM   ERROR [FilterUnitService] MongoNetworkTimeoutError: connection timed out