wso2 / apk Goto Github PK

This issue is created to track the progress of Admin Domain Service implementation.

Functionality of Admin DS

The Admin DS is capable of handling all the admin related flows and use cases in APK control plane. This includes,

• Rate limiting policy related flows
• Key manager related flows
• Role/Scope mappings

Proposed Solution

The service layer will be implemented using Ballerina. Ballerina interoperability feature will be used to bring the existing (java) business logic in carbon-apimgt component to the ballerina service.

High level view of a domain service will be as follows.

Sub tasks

• #34
• #35
• #36
• #37
• #38
• #39
• #40
• #93
• #121

Description:
This is to extract data for Enforcer XDS.

Need to debug and check how enforcer gets consumer keys currently.
Update application.proto to send consumer keys and do the related changes in enforcer side.

• ApplicationData CRD (which contains a list of apps) has changed to single Application CRD. Need to consider this design change.

Description:
This is to detect inserts, updates and deletions of CRDs.

Description:
This to handle Management Server to Agent XDS communication.

A new configuration model will be introduced for the domain service since the previous carbon dependencies will no longer be used with new services.

Proposed Approach

• Single toml file to contain all the configurations. According to the call with the ballerina team, we can use the Config.toml for this purpose.
• Record mappings on the ballerina side (similar to api-manager.xml).
• Startup service to start all internal components related to apimgt (similar to APIMStartupComponent). Need to use ballerina interops for this too to call as a function from the ballerina code.
• Use an object mapper from the Java side to map the configurations in runtime.

This issue is created to track the progress of removing all kernel, axis2, osgi, etc. related dependencies from the admin related flows.

The new domain services will not rely on the carbon framework. Hence we will be removing all those from the existing carbon-apimgt code

Description:
This is to track the PostgreSQL transactions related to Management Server.

Description:
This is to define CP and DP CRDs.

Description:
This is to finalize the data related to the user flows defined in https://docs.google.com/document/d/1F15zHzHi8ercomq5J3yR8P6TjkVWhhUQ1lnOF4_JCcU/edit?usp=sharing

This is created to track the initial evaluations done on creating services using Balleina.

• Evaluate options for CMS

Description

• Feature: Dynamically configuring (xDS) rate limit policies on Envoy Rate limit Server
  https://github.com/envoyproxy/ratelimit/issues/ 201
• Move xDS ADS client to Go Control Plane

This issue is created to track the progress of fixes/improvements that should come from ballerina

• ballerina-platform/ballerina-library#5152
• https://github.com/wso2-enterprise/internal-support-ballerina/issues/198
• ballerina-platform/ballerina-library#3393

Description:
This is to define user flows related to the events in CP and DP.

Description:
This is to handle partitioning-like logic for CRDs for applications, and subscriptions.

Implement the runtime API-related rest API resources in the back office DS

This is the documentation created - https://docs.google.com/document/d/1CL5ug6neMn-caLbjzLOJms8TNuMSw9evSaEHjQRFudQ/edit#

This is created to track the progress of error handling in admin service flows. The previous error handling mechanism was reliant on CXF. The new approach will remove this coupling and make the error handling generic.

Implement the backoffice API-related rest API resources in backoffice DS

This issue is created to track the progress of Runtime API Implementation.

Functionality of Runtime API

The Runtime API interacts with the Kubernetes API Server in the Data plane and serves as a backend for the Runtime UI and APK CLI.

Sub tasks

Develop Interfaces

• #42
• #76

Implement the API Interfaces in Go

• #77
• #78

Improvements to Runtime API Server

• #79
• #70
• Merge k8s client implementation with server
• Implement Authorization for APIs with STS(oauth2)

This is created to track the progress of moving the business logic in existing REST API layer to a lower level so that it can be reused through Ballerina interoperability with the new DS.

Description

This issue is to track the Runtime-UI Implementation of APK

Subtasks

• #64
• #65
• #72
• #73
• #74
• #75

Description

Support API Level (API, Resources, and HTTP method level), Application Level, and Subscription Level rate limitings using Envoy Rate-limit service.

Sub Tasks

• #68

• API Rate Limiting Redis Analysis
  https://docs.google.com/document/d/1-wZlBx4LwPOV5Jm0HzZyfo_HJky2558i3xdC43-UNW0/edit#

• #69

• #3

• Ratelimit service issues

  • Stop increasing Redis keys when the request is over limited
    https://github.com/envoyproxy/ratelimit/issues/ 348
  • Support for month, year rate limit units
    https://github.com/envoyproxy/envoy/issues/ 22626

• Rate limit CRD
  https://docs.google.com/document/d/1CL5ug6neMn-caLbjzLOJms8TNuMSw9evSaEHjQRFudQ/edit#heading=h.scodf9jh4tp4

• #71

• Control Plane - Rate limit Configuration

  • User Interfaces for Inline API Level Rate limits.
  • User Interfaces for Application/Subscription Rate limits.
  • Rest APIs and DB Schemas for API/Application/Subscription level Rate limits.

Finalize all the resources which need for the backoffice DS.

Re-design the DB schema for the Admin/Backoffice/Devportal domain services

Purpose

This issue is created to track the progress of Developer Portal Domain Service.

Functionality of DevPortal DS

The DevPortal DS consists of all the developer portal related flows and functionalities related in APK control plane. This includes,

• API related flows
• Application related flows
• Subscription related flows
• Throttling policy related flows

Sub tasks

• #47
• #48
• #49
• #50
• #51
• #52
• #53
• #55
• #54
• #56
• #57
• #58
• #59
• #60
• #122

Create mock services for Admin, Devportal, and Publisher services for demo purposes

Purpose

This issue is created to track the progress of POC to support multiple language builds and CI/CD use cases using Gradle build tool.

Sub tasks

• Use Gradle for building, testing, and running custom tasks for each language module (Java, Ballerina, Go, Node.js).
• Run PR checks, release components, and create release version tags per component upon PR open/merge via GitHub actions and Gradle tasks.
• Manage dependencies among projects (outer modules) and sub-projects (inner modules).
• Push docker images to docker hub.
• Publish build artifacts to remote maven repository with POM.

This issue is created to track the progress of Runtime API OAS development. The following API will be defined in the OAS specification.

• APIs (List, Create, Update, Delete)
• Services (List)
• Gateways (List, Create, Update, Delete)
• Validations
• Import Export
• Policies (List, Create, Delete)

Description:
This is to handle retries between Agent to Management Server using a worker pool

• POC: Registry persistence with postgres and Indexing capabilities
• DAO Restructure
• DAO Integration Tests

This is to track changes related to operator related boilerplate code and following tasks related to application and subscription data CRDs.

• #30
• #31
• #32
• #96

Description:
This is to check the possibility to store the last status locally to handle restarts/get the diff from the last update.

This issue is to track BackOffice UI Implementation of APK.

• #41
• #61
• #62
• #80
• #81
• #82

Functionality of APK Agent Client

APK Agent is the responsible party for managing the data-plane cluster by managing its components as well as keeping the control plane sync-up with data-plane.

Mainly it has the following functionality

• Talks to the K8s API server and manages (Create, Update, Delete) resources in the K8s cluster(DP) -> Create APIs (in UI driven approach), Applying policies (from UI)
• Update rate limit service with rate limit policies -> Will be an xDS communication
• Update enforcer with subscription details, etc. -> Will be an xDS communication
• Should have a K8s operator to listen on the resources created on the DP cluster.
• Communicate with CP - xDS

Functionality of APK Management Server

Management server is the responsible party for keeping the CP & DP sync by communicating with APK clients in each data-planes (or in the same control-plane when we run in a single cluster).

• Send API create/delete/update, Subscription, Policy events to the APK client - xDS
• Service registering.
• Update control plane with the resources created on dataplane(using cli) such as APIs, Policies, etc..

APK Management Server to APK Agent Client Communication

API Agent Client should be associated with a k8s operator, which is watching for resource updates and sending the updates to the control plane’s Management Server and the Gateway.

Since we have only one control plane available, we decided to make this communication as a gRPC client to server streaming communication.

Aug 29th - Sept 12th

Oct 3rd -

• #5

Aug 29th - Sept 5th

• PKCE support for web portals

Sept 19th

• Verify the security / dev experience

Sept 26th -

• Removing unwanted UI features from portal apps

Sept 26th - Oct 10th

• OIDC flow implementation

• Make React apps independent

Description:
This is to extract data for the Management Server (CRDs -> XDS coversion)

Description: BackOffice UI Designing using Figma.

This issue is created to track the progress of Backoffice Domain Service implementation.

Functionality of Back Office DS

The Backoffice DS is capable of handling all the backoffice related flows and use cases in APK control plane. This includes,

• Creating backoffice APIs from runtime APIs
• Add metadata to the backoffice APIs
• Manage the backoffice APIs
• Publish the backoffice APIs

Sub tasks

• #43
• #218
• #44
• #91
• #105
• #106
• #108
• #45
• #46

Description:
This is to design and implement the gRPC unary call from the APK Agent Client to Management Server.