expressgateway / express-gateway Goto Github PK
View Code? Open in Web Editor NEWA microservices API Gateway built on top of Express.js
Home Page: https://www.express-gateway.io
License: Apache License 2.0
A microservices API Gateway built on top of Express.js
Home Page: https://www.express-gateway.io
License: Apache License 2.0
For the authentication that is NOT handled by the EG, needs to be re-directed an external authorization URL so that the URL can be redirected to when a user accesses the EG.
This should be specified in authorization_url
Example description
publicEndpoints: # specify pipeline per domain and path
cats:
host: ".cats.com" # regexp support
path: / # optional defaults to /
pipeline: cats_pipeline
mycats:
host: ".mycats.com"
path: /cats
pipeline: cats_pipeline
big-dogs:
host: big-dogs.com
pipeline: dogs_pipeline
animals-cats:
host: "animals.com"
path: /cats
pipeline: cats_pipeline
animals-dogs:
host: "animals.com"
path: /dogs
pipeline: dogs_pipeline
Add API Consumer to configure the EG for finer granularity of control. at Quality of service, ACLs, and etc.
Use Redis connector from loopback. Support for both standalone and distributed environment.
Extend express-gateway-admin to deploy EG Admin API.
Create policy to handle different OAuth2 strategy
The EG runtime configuration needs associate API consumers with the APIs (and their endpoints) through the configured pipelines. Although it's not necesary to explicity do so, granularity could be specified to link APIs with specific instances of pipelines where a processor may have specific parameter values for a given API consumer.
Example:
Customer API is linked to a pipeline where a rate limiter is specified and Consumer Bob and Consumer Bill exist
Consumer Bob may have rate limiting parameters: 1000 req/minute
Consumer Bill may have rate limiting parameters: 2000 req/minute
The SAME API backed by the SAME pipeline have two different parameter values for Bob and Bill.
The authorization system would need to provide a set of scopes that the user is authorizing to be provided to the client application. We need API endpoints with OAuth2 scopes
Allow TLS and SNI support for https to be configurable externally so that multiple certs can be used for multiple domains.
Create consumer via CLI
As an EG admin, I would want to be able to configure multiple EG instances that are all driven off central configuration, so that I have scale and high availability.
An EG admin should be able to create mutiple gateway instances that are assigned different configurations but also share centralized information.
Example:
Gateway 1: has 2 pipelines and fronts /catalog and /cart public endpoints
Gateway 2: has 1 pipeline and fronts /user endpoint
Both Gateway 1 and 2 should share the same set of API consumer data for access and assignment
EG App data would be more key-value and or rows and tables and not have hierarchical nor document like structure like the config data. Because this can live in distributed and single data stores, the lowest common denominator would be something like keys and values that can be transposed into rows/columnar based store and grouped into tables or namespaces.
Add environment variable override based on the 12-factor app config for the EG config.
Extend the express-gateway-admin to dynamically manipulate the EG config
Create application via CLI
Allow a user to specify the OAuth2 tag within a pipeline's path to match all endpoints URLs
Example:
condition: ["always"]
or by a particular path
condition: ["pathExact", "/protected/resourceA"]
this would trigger the policy.
EG has APIs as containers for public endpoints. EG has private endpoints as well. We should think about the parallel that we have already within LunchBadger. A microservice is a unit of deployment of functionality that has dependencies and externalization in the form of private enpoints. In LB - a model has a built in private endpoint. If you proxy in front of a microservice, the microservice can have N number of private endpoints. Therefore a microservice is a container of private endpoints.
For reference, review and discussion
Extend LB workspace to deploy the new EG created in LB workspace.
Need to discuss to breakdown to smaller tasks
Have consistent entities between EG UX and LB UX.
-change EG Readme processors to policies
-check and update the reference name in codes accordingly.
Store default config file in /etc/expressgateway/gateway.conf
To ensure EG will be capable of playing both roles as the authorization and resource server in the OAuth2 use case
Create EG Admin API to manipulate the JSON config parameters.
Investigating options with and without LoopBack.
There is EG app data (such as consumer records) that is kept separate from the config data so that I can separate configuration from master/app data
EG should support the ability to separate app data from config data. Config data would be data that is used to configure Express Gateway and be considered metadata. App data would be data that a USER of EG would generate using EG, for example - creating API consumers in the consumer management system
Ability to passing in EG custom headers downstream so that use cases like path tracing, audit, request/response time introspection and metadata can be realized.
EG should be able to inject, modify and preserve upstream and downstream headers as needed for its own instrumentation as well as preserving system metadata for consumers and backend producers.
Add unique ID for each entity to refer the entity through the EG API
Need to have CLI for key-gen that accepts keys as user inputs or makes new ones
The LB architecture has an API called a workspace API. The workspace API manipulates LoopBack projects. The same workspace API needs to be extended to manipulate Express Gateways. The big question here is - how do we surface an API for Express Gateway? Do we do this without LoopBack?
As an LB Developer, I want to be able to deploy the changed EG config without reloading (hot deploy).
Investigate:
expressjs/express#2596
https://docs.google.com/document/d/1o7aXdcb8TskjjNFtDi29x10wnjF0TwSPHNcN-OkVy6s/edit
Everyone feel free to add and edit
For OAuth2, that can potentially be extended for others authentication later
Allow a user to specify OAuth2 scope as OAuth2 scope tags within a pipeline to match public endpoint URLs
Example:
{
"condition": ["pathExact", "/protected/resourceA"],
"action": "tag",
"params": {
"tags": ["oauth2", "scope:resourceA"]
}
}
Ability to create a pipeline with key based processor to provide authentication through a set of EG generated keys for each API Consumer
An API Consumer should be able to have a set of key credentials associated with their identity.
A developer familiar with Express middleware should be able to take any middleware that extracts the parameters that are necessary to construct a configured instance of it to be passed dynamically when instantiating that middleware.
Those parameters should go into EG config files in a standardized format.
Create a Design/Architectural Document that describes how and what the EG admin API be built, so that all dependencies from the EG OSS project AND LB commercial product are identified with the right decisions.
Breakdown config file to more linear structure
####Public Endpoints
host, path, pipeline_name
####Private Endpoints
url, port, path
http server configs
port
https server configs
port and certificates
pipelines
APIs - group of public endpoint (Does not include consumer management)
YAML example:
publicEndpoints: # specify pipeline per domain and path
cats:
host: ".cats.com" # regexp support
path: / # optional defaults to /
pipeline: cats_pipeline
mycats:
host: ".mycats.com"
path: /cats
pipeline: cats_pipeline
big-dogs:
host: big-dogs.com
pipeline: dogs_pipeline
animals-cats:
host: "animals.com"
path: /cats
pipeline: cats_pipeline
animals-dogs:
host: "animals.com"
path: /dogs
pipeline: dogs_pipeline
http: # if not specified EG will not start http server
port: 9080
https: # if not specified EG will not start https server
port: 9443
certificates: # will use SNI callback to provide proper certificate for domain
"*.lunchbadger.io":
key: "example/keys/lunchbadger.io.key.pem"
cert: "example/keys/lunchbadger.io.cert.pem"
"api.lunchbadger.com":
key: "example/keys/lunchbadger.com.key.pem"
cert: "example/keys/lunchbadger.com.cert.pem"
"default":
key: "example/keys/lunchbadger.io.key.pem"
cert: "example/keys/lunchbadger.io.cert.pem"
privateEndpoints: # urls to downstream services
cats_service:
url: "http://localhost"
port: 3000
path: / # optional defaults to /
dogs_service:
url: http://localhost
port: 4000
pipelines:
cats_pipeline:
title: Cats Pipeline
policies:
- action:
name: proxy # proxy to cats_service downstream service
privateEndpoint: cats_service
dogs_pipeline:
title: dogs Pipeline
policies:
- condition:
name: always
# action:
# name: oauth2-basic
# params:
# - action:
# name: oauth2-bearer
# - action:
# name: ensure-autenticated # ensure user has logged in 403 otherwise
# - action:
# name: oauth2-scope
# allow-scopes:
# - '/dogs'
action:
name: proxy # proxy to dogs_service downstream service
privateEndpoint: dogs_service
apis:
animals:
name: animals
plan: free
public_endpoints:
- cats
- mycats
- dogs
Investigate the OAuth2 design per Roman to see if it can do both authorization and resource server in OAuth2 use case.
https://docs.google.com/document/d/1CKQEAvnAgG5E7twGIUZdCZqqdhoM7Zeri9CN9uTLFgA/edit
There are public endpoints, and then there are APIs, we need to validate the logical model that an API is a set of public endpoints.
Roman's design within the Express gateway has the most granular level listed within the configuration - public endpoint URLs (non-grouped) and private endpoints that the EG proxies to. Need to think about introducing API entity as a set of public endpoints.
https://docs.google.com/document/d/1AxrFidpb92Y_pnMdPrCVLiWmIMnQnMX60S9J_kWblzI/edit
Everyone feel free to add and edit
Example:
Consumer Bob uses OAuth2 credentials to authenticate and authorize externally into the EG, a set of internal credentials can be used further downstream into internal systems and central IAM systems like LDAP through SAML.
Ability to create a pipeline with a Basic Auth processor to provide basic authentication for API Consumers.
An API Consumer should be able to have basic auth credentials associated with their identity.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.