Product Search REST Service - http://localhost:8080
Product Search REST Service is designed to provide following endpoints:
- Public
- /allProducts - GET
- /product/{item} - GET
- /search/{text} - GET
- Private
- /stop - GET
- /cache/use/{enabled} - GET
- /cache/all - GET
- /cache/clear - GET
- /documentation - GET Auto generated by Hapi-Swagger for Public endpoints
Endpoints
Public
/allProducts
This is an endpoint that provides the list of all the product codes available on https://gist.githubusercontent.com/daniyalzade/8e32cd266aebd6d2ce35/raw/f8c6b89f9cfafdbb3db7b653a6354c9f5b7b106d/data_file.csv
/product/{item}
This endpoint provides product details for a give productCode (item).
/search/{text}
This is the endpoint, which is the primary focus of this exercise. It returns an array of Product Codes for a given search text.
Private
/stop
As the endpoint alludes, this stops the server.
/cache/use/{enabled}
We can enable/disable the caching to see performance difference.
{enabled} can be either true or false;
/cache/all
Returns everything stored in cache.
/cache/clear
Clears the cache.
Technology/Framework used
- Nodejs
- NPM
- Chai
- Mocha
- Nock
- Nodemon
- Istanbul
- Async
- Hapi
- Hapi-Swagger
- Inert
- Joi
- Request
- Vision
Code Structure
Application has been laid out in a simple folder structure mimicing the use.
<root>
|
|- src
| |- app.js // Starting point of this application.
| |- server
| | |- server.js // Main module that manages the server, routes and basic handlers.
| |- api
| | |- search.js // Main focus of this exercise, thus in apis folder. All other endpoints are optional.
| |- helper
| | |- allProducts.js // Static file that maintains the list of all the product codes available on data_file.csv
| | |- config.js // Configuraton file for providing correct URLs.
| | |- product.js // Critical module that fetches details about a product code. Used both in /search/{text} and /product/{item}
| |- routes
| | |- routes.js // Started to be a possible container for routing information, but abandoned for triviality of the use case. Still leaving here to highlight the possible use-case.
| |
|- test
| | ---- All the files in this folder are created to test the corresponding file.
Performance
Response Times Analysis
With a random delay of var timeout = 400 + Math.floor(Math.random() * 200) + 1; milliseconds.
I revisited this delay today, looks like this was only helpful that one day when I ran the tests for the first time.
Response time:
Request Count: 1
Bytes Sent: 448 (headers:448; body:0)
Bytes Received: 223 (headers:195; body:28)
ACTUAL PERFORMANCE
--------------
ClientConnected: 11:18:13.923
ClientBeginRequest: 11:24:20.748
GotRequestHeaders: 11:24:20.748
ClientDoneRequest: 11:24:20.748
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 0ms
HTTPS Handshake: 0ms
ServerConnected: 11:24:20.749
FiddlerBeginRequest: 11:24:20.749
ServerGotRequest: 11:24:20.749
ServerBeginResponse: 11:24:32.904
GotResponseHeaders: 11:24:32.904
ServerDoneResponse: 11:24:32.904
ClientBeginResponse: 11:24:32.904
ClientDoneResponse: 11:24:32.904
Overall Elapsed: 0:00:12.155
RESPONSE BYTES (by Content-Type)
--------------
~headers~: 195
application/json: 28
With a static delay of var timeout = 100; milliseconds. This was the best delay I was able to identify.
Running on both my Mac host and Windows VM, 100 ms gave the best results with no 403s. I am running both the machines from behind a proxy.
Response time:
Request Count: 1
Bytes Sent: 448 (headers:448; body:0)
Bytes Received: 291 (headers:227; body:64)
ACTUAL PERFORMANCE
--------------
ClientConnected: 11:32:49.223
ClientBeginRequest: 11:32:49.225
GotRequestHeaders: 11:32:49.225
ClientDoneRequest: 11:32:49.225
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 0ms
HTTPS Handshake: 0ms
ServerConnected: 11:32:49.226
FiddlerBeginRequest: 11:32:49.226
ServerGotRequest: 11:32:49.226
ServerBeginResponse: 11:32:53.823
GotResponseHeaders: 11:32:53.823
ServerDoneResponse: 11:32:53.823
ClientBeginResponse: 11:32:53.823
ClientDoneResponse: 11:32:53.823
Overall Elapsed: 0:00:04.597
RESPONSE BYTES (by Content-Type)
--------------
~headers~: 227
application/json: 64
With a static delay of var timeout = 80; milliseconds.
Response time:
Request Count: 1
Bytes Sent: 448 (headers:448; body:0)
Bytes Received: 291 (headers:227; body:64)
ACTUAL PERFORMANCE
--------------
NOTE: This request was retried after a Receive operation failed.
ClientConnected: 11:32:49.223
ClientBeginRequest: 11:41:57.340
GotRequestHeaders: 11:41:57.340
ClientDoneRequest: 11:41:57.340
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 0ms
HTTPS Handshake: 0ms
ServerConnected: 11:41:57.341
FiddlerBeginRequest: 11:41:57.341
ServerGotRequest: 11:41:57.341
ServerBeginResponse: 11:41:57.340
GotResponseHeaders: 11:42:01.167
ServerDoneResponse: 11:42:01.167
ClientBeginResponse: 11:42:01.167
ClientDoneResponse: 11:42:01.167
Overall Elapsed: 0:00:03.827
RESPONSE BYTES (by Content-Type)
--------------
~headers~: 227
application/json: 64
With a static delay of var timeout = 70; milliseconds.
Response time:
Request Count: 1
Bytes Sent: 448 (headers:448; body:0)
Bytes Received: 291 (headers:227; body:64)
ACTUAL PERFORMANCE
--------------
ClientConnected: 11:32:49.223
ClientBeginRequest: 11:48:30.215
GotRequestHeaders: 11:48:30.215
ClientDoneRequest: 11:48:30.215
Determine Gateway: 0ms
DNS Lookup: 0ms
TCP/IP Connect: 0ms
HTTPS Handshake: 0ms
ServerConnected: 11:48:30.216
FiddlerBeginRequest: 11:48:30.216
ServerGotRequest: 11:48:30.216
ServerBeginResponse: 11:48:33.991
GotResponseHeaders: 11:48:33.991
ServerDoneResponse: 11:48:33.991
ClientBeginResponse: 11:48:33.991
ClientDoneResponse: 11:48:33.991
Overall Elapsed: 0:00:03.776
RESPONSE BYTES (by Content-Type)
--------------
~headers~: 227
application/json: 64
With a static delay of less than 100 milliseconds threw:
HTTP/1.1 403 Forbidden
Content-Type: application/json
Date: Wed, 06 Jun 2018 18:40:18 GMT
Retry-After: 1
Server: Mashery Proxy
X-Error-Detail-Header: Account Over Queries Per Second Limit
X-Mashery-Error-Code: ERR_403_DEVELOPER_OVER_QPS
X-Mashery-Responder: prod-j-worker-us-west-1b-07.mashery.com
Content-Length: 75
Connection: Close
{"errors":[{"code":403,"message":"Account Over Queries Per Second Limit"}]}
after every 12 requests or so.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent