The executionandtracelog from openbackhaul

minLength:6 at operation-name

Decision on the Application Implementation call on 13th or May:
minLength:6 to be added to all operation-name attributes in the ingress bodies.

Basic services

Some OperationClients have to be changed from "basic management" to "basic services":

ExecutionAndTraceLog://v1/record-service-request
OamLog://v1/record-oam-request
AdministratorAdministration://v1/approve-oam-request

Required changes in ServiceList:

move three OperationClients from own-oam::basic to service::basic
change uuids from e.g. tar-1-0-0-op-c-0040 to tar-1-0-0-op-c-bs-eatl-1-0-0-000 (make off-line table of changes!)

Required changes in ForwardingList:

update uuids (by CTRL+h)

Required changes in CONFIGfile:

move OperationClients
update uuids of OperationClient (by CTRL+h) according to table
update uuids of HttpClient and TcpClient

Required changes in OAS:

update uuids of OperationClients, HttpClients and TcpClients (by CTRL+h) according to table

Add implementation for the individual service /v1/bequeath-your-data-and-die

Describe your feature request :

Support functionality and forwarding for the service /v1/bequeath-your-data-and-die to perform version upgrade

Implementation tasks and next steps :

Alternate proposals if any :

Additional context :

server: service-record-profile uuid to be updated

As per issue #30, the uuid pattern is decided to be updated from ^eatl-0-0-1-op-s-\d{4}$ to ^eatl-0-0-1-service-record-p-\d{6}$.

Problem statement:
As per the decision, the server-side code is updated to create service-record-profile entries having uuid(s) with 6 digits.
This tends to be problem when there is a difference in server-code and /api/openApi.yaml file in server.
Additionally, due to presence of "op-s" in uuid pattern instead of "service-record-p", it gives 400 error on trying to access data from OAM layer

proposal 1: Update server-logic to create 4-digit uuid suffix
proposal 2: Update api/openapi.yaml file for uuid pattern of service-record-profile to have ^eatl-0-0-1-service-record-p-\d{6}$

Update Profiles

This update covers:

changes in the *generic-representation services
encapsulation of application data
update of UUIDs

ProfileList:

Copy ProfileList from ApplicationPattern
Adapt ProfileList (keep IntegerProfile)
- keep at least ActionProfile, GenericResponseProfile
- keep FileProfile, if data shall be stored into external LOADfile
- Check old CONFIGfile for existing profile instances (ignore application data) and create corresponding Profiles
- delete obsolete Profiles

ServiceList:

Add docs path

        - service-name: /docs
          uuid: ro-2-0-0-op-s-bs-005

CTRL+h from "ro-2-0-0-" to "tar-2-0-0-" (replace tar by application name abbreviation)

ProfileInstanceList:

Copy ProfileInstanceList from RegistryOffice
Add FileProfile from ApplicationPattern-ProfileInstanceList, if required
CTRL+h from "ro-2-0-0-" to "tar-2-0-0-" (replace tar by application name abbreviation)
Adapt ProfileInstanceList
- compare with ApplicationPattern-ProfileInstanceList and jump over basic profile instances
- open OAS and translate descriptions of existing *-in-generic-representation requests into ActionProfile and GenericResponseProfile instances
- Add FileProfile instance, if data shall be stored into external LOADfile

CONFIGfile:

copy profile instances from RegistryOffice
adapt UUIDs
compare with ApplicationPattern-ProfileInstanceList and jump over basic profile instances
delete obsolete profile instances
Add individual ActionProfile instances
Add individual ResponseProfile instances
delete application data profile instances
Create application data storage reference (instance of FileProfile), if required
Add individual Profile instances, if required

Update OAS:

Add implementation for the individual services /v1/start-application-in-generic-representation

Describe your feature request :

Support functionality and forwarding for the service /v1/start-application-in-generic-representation

Implementation tasks and next steps :

Complement the startApplicationInGenericRepresentation operation in controller/IndividualServices.js
Complement the startApplicationInGenericRepresentation operation in services/IndividualServices.js
Support forwarding ServiceRequestCausesLoggingRequest to Execution and Trace Log

Alternate proposals if any :

Additional context :

Refactor : the security validation machanism for apiKeyAuth and basicAuth validations

Currently , the apiKeyAuth and basicAuth validations are handled in the controllers/ for each services in their corresponding methods.
This validation in each methods can be replaced by integrating the security check mechanism as a part of express framework instantiation.

Reference : OperationKeyManagement

ActionProfile

Add ActionProfile to /core-model-1-4:control-construct:
Add - 'action-profile-1-0:PROFILE_NAME_TYPE_GENERIC_DISPLAY_PROFILE' to all Profiles in /core-model-1-4:control-construct:
Update all consequent-action-lists with Descriptions referencing the ActionProfile
Add ActionProfiles to LOADfile
Add ActionProfiles to ProfileList

Add OAM services implementation

Describe your feature request :

Include support for OAM Services

Implementation tasks and next steps :

Include basic services
Test the integrated fucntionality
commit and merge the changes

Alternate proposals if any :

Additional context :

OAM layer : Get request responseBody attribute needs to be corrected to view the ipv-4-address of tcp-server and tcp-client

Issue#1 :
For the OAM service /core-model-1-4:control-construct/logical-termination-point={uuid}/layer-protocol=0/tcp-server-interface-1-0:tcp-server-interface-pac/tcp-server-interface-configuration/local-address/ipv-4-address , the response-body should have the attribute "tcp-server-interface-1-0:ipv-4-address".
But in the implementation , it is "tcp-server-interface-1-0:local-address"

Proposal : "tcp-server-interface-1-0:local-address" needs to be corrected to "tcp-server-interface-1-0:ipv-4-address"

Issue#2 :
Similarly , for the service /core-model-1-4:control-construct/logical-termination-point={uuid}/layer-protocol=0/tcp-client-interface-1-0:tcp-client-interface-pac/tcp-client-interface-configuration/remote-address/ip-address/ipv-4-address , the response-body should have the attribute "tcp-client-interface-1-0:ipv-4-address".
But in the implementation , it is "tcp-client-interface-1-0:remote-address"

Proposal : "tcp-client-interface-1-0:remote-address" needs to be corrected to "tcp-client-interface-1-0:ipv-4-address"

OAS : Request to include description for the callback “PromptingNewReleaseForUpdatingServerCausesRequestForBroadcastingInfoAboutBackwardCompatibleUpdateOfOperation”

As per the decision made in the issue openBackhaul/RegistryOffice#19 , we require to include a description to the callback “PromptingNewReleaseForUpdatingServerCausesRequestForBroadcastingInfoAboutBackwardCompatibleUpdateOfOperation” of /v1/bequeath-your-data-and-die

description: >
              'This callback belongs to the sequence of actions that have to be done during the bequeath-your-data-and-die process, despite the forwarding gets neither managed nor directly initiated by the /v1/bequeath-your-data-and-die request.
              After consuming applications have been redirected to the new release, the new release is triggered (this callback) to request the RegistryOffice for broadcasting information about backward compatible replacements of services.'

server : unexpected termination of server during /v1/bequeath-your-data-and-die

While triggering /v1/bequeath-your-data-and-die, a series of callbacks will be called to other applications for scenarios like transferring application specific data, creating/ending subscriptions.
While the callbacks are being triggered from the application, we could observe an unexpected behavior.

Problem:
The server is abruptly stopped with an error message thrown after the callback request fails.
Error message :

${forwarding} is not success for the input[object Object] node:internal/process/promises:246
triggerUncaughtException(err, true /* fromPromise */);

Expected behavior:
When any callback request fails, /v1/bequeath-your-data-and-die should not proceed further for next callback. The application should not stop abruptly and it runs normally.

Add docker file to create docker image

Describe your feature request :

Include docker file to create docker image

Implementation tasks and next steps :

Create docker file including the following :

Create app directory

Install and app dependencies
2.1. use a wildcard to ensure both package.json AND package-lock.json are copied
2.2. RUN npm install

Bundle app source

Command to start the application

Test whether this docker file is creating a docker image

Alternate proposals if any :

Additional context :

/v1/list-applications : OldRelease and NewRelease data to be removed

As per conclusion made on issue openBackhaul/ApplicationPattern#276, the service /v1/list-applications need not contain data of OldRelease and NewRelease applications in the response-body.

Current implementation of /v1/list-applications responds with the data of OldRelease and NewRelease applications which needs to be removed.

Add basic services implementation

Describe your feature request :

Include support for basic services

Implementation tasks and next steps :

Include basic services
Test the integrated fucntionality
commit and merge the changes

Alternate proposals if any :

Additional context :

OAS : callback uri definition : /v1/regard-application

Problem:

Operation-name is statically defined in callback request "ApprovedApplicationCausesRequestForServiceRequestInformation"

Proposal:

Line 1056 to be changed from "https://{$request.body#application-address}:{$request.body#application-port}/v1/redirect-service-request-information" to "https://{$request.body#application-address}:{$request.body#application-port}[/core-model-1-4:control-construct/logical-termination-point=eatl-0-0-1-op-s-0002/layer-protocol=0/operation-client-interface-1-0:operation-client-interface-pac/operation-client-interface-configuration/operation-name]"

Related issue : openBackhaul/OamLog#6

Update format of addresses in services

Add protocol and update address at:

/v1/regard-application:
/v1/list-applications:

domain name to be added

Data structure of the TcpClient and TcpServer shall be complemented by a domainName attribute for preparing potential future appliance of a DNS function.

Modelling has to assure that there is either IP address or domain name communicated and stored.

The following elements of the OAS have to be changed:

Coding examples:

ip-address:
  type: object
  minProperties: 1
  additionalProperties: false
  properties:
    ipv-4-address:
      type: string
      pattern: '^((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$'
      description: >
        '..'
domain-name:
  type: string
  pattern: '^([a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,}$'
  description: >
    '..'

example:
  new-application-name: 'NewApplicationName'
  new-application-release: '0.0.2'
  new-application-address:
    ip-address:
      ipv-4-address: '10.118.125.157'
  new-application-port: 7000

Further details can be found in the callbacks inside the ApplicationPattern OAS.

/v1/bequeath-your-data-and-die service is getting failed during the execution of callbacks

Issue Description :

While triggering /v1/bequeath-your-data-and-die, a series of callbacks will be invoked to other applications for scenarios like transferring application specific data, creating/ending subscriptions.
While the callbacks are being triggered from the application, we could observe it is getting aborted with the following error message :

PromptForBequeathingDataCausesNewApplicationBeingRequestedToInquireForApplicationTypeApprovalsforwarding is not success
upgradeSoftwareVersion failed with error: operation is not success

Root cause :

In latest Application Pattern code implementation the the parameters being sent to dispatchEvent() method have been modified due to which the issue is being occurred.

servers statement to be removed

The servers statement at the beginning of the OAS, does not provide any benefit.
It should be removed.

This applies to the ApplicationPattern and all applications.

Add individual services implementation

Include implementation for the Individual services of Execution And Trace Log

Implementation tasks and next steps :

Alternate proposals if any :

Additional context :

Use ExecutionAndTraceService from onf-core-model-ap

ExecutionAndTraceService has been moved from package onf-core-model-ap-bs to onf-core-model-ap in PR openBackhaul/ApplicationPattern#141

Please, change code to use ExecutionAndTraceService from onf-core-model-ap.

Fix : required to send relay-server-replacement service only once as a part of bequeath-your-data-and-die service

As a part of bequeath-your-data-and-die service , relay-server-replacement will be called by the old application to replace the old release with the new release. This relay-server-replacement service is getting called twice from the old application which is unnecessary.
This needs to be fixed in the server side code.

Note : There is no problem in calling the service twice as it is idempotent, but it is not required.

After updating all TAC applications: Update Callbacks

This issue is recommended to be executed after all path: statements of all OperationsServers have been updated at all the TAC applications.

Double check, whether the callbacks in the OAS are in line with the entries of the ForwardingList
Double check, whether the data structures in the callbacks are still in line with the OperationServer definitions in the targeted applications

Update UUIDs of Services

Add implementation for the individual service to record service request

Describe your feature request :

Support functionality and forwarding for the service /v1/record-service-request

Implementation tasks and next steps :

Complement the recordServiceRequest operation in controller/IndividualServices.js
Complement the recordServiceRequest operation in services/IndividualServices.js
Support forwarding ServiceRequestCausesLoggingRequest to Execution and Trace Log

Alternate proposals if any :

Additional context :

Add test cases

Create export of test case collection from Postman (regard proper placement of jump marks)
Upload file ApplicationName+testcases.json to Code folder of develop branch
Update the link inside the readme.md

Updating the UUIDs of FCs

Process for updating the UUIDs of FCs

Katharina to update fc-uuid in CONFIGfile and OAS wherever applicable referencing from ForwardingList on feature branch
Katharina to create pull request from feature branch
Iswaryaa to review the pull request
If no errors found
Iswaryaa to Approve the pull request
Iswaryaa to merge feature branch to develop
Iswaryaa to delete feature branch and close this issue

Retrieving data should support pagination

Observation regarding EaTL server unavailability

When EaTL application goes down ,
Impact :

Service records from other applications will not be captured
Even after coming live , the lost records will not be recovered.

Note :
When EaTL is down , it’s very difficult to identify this. Until we manually try to use this application.

Proposal for High availability :

Proposal#1 : If we are planning to continue with docker engine , a simple shell script that

monitors the pid of the application’s port
starts the container if the pid is not available
(already we have such scripts for other applications like traffic checker. Monitoring dashboard)

Proposal#2 : If we are planning to move to an orchestrator(docker swarm, Kubernetes) then the orchestrator will take care of it.

Proposal to prevent data loss :

Including a message broker(like rabbitmq) that buffers the message until the EaTL application is available again. (But of course the message broker should also be highly available)
Reference : https://microservices.io/patterns/communication-style/messaging.html

Priority: low

Add implementation for the individual service to include, exclude and list applications

Describe your feature request :

Support functionality and forwarding for the service /v1/regard-application , /v1/disregard-application, /v1/list-applications.

Implementation tasks and next steps :

Alternate proposals if any :

Additional context :

Explanation of Operation Modes required

The operation modes
- 'string-profile-1-0:STRING_VALUE_TYPE_REACTIVE'
- 'string-profile-1-0:STRING_VALUE_TYPE_SANITATION'
- 'string-profile-1-0:STRING_VALUE_TYPE_PROTECTION'
- 'string-profile-1-0:STRING_VALUE_TYPE_OFF'
need explanation.

IntegerProfile for configuring the UpdatePeriod for operation-keys

An additional IntegerProfile shall be defined, made available for configuration on the OAM layer and be put into the LOADfile, for managing the period for updating the operation-keys on the connected applications.

userName attribute add to data file

"userName" is an attribute required in ExecutionAndTraceLog+data.json.

Usage:
user-name is an integral parameter in request-headers of each service in Service Layer. This parameter enables identification of the user who is accessing a service.
"userName" attribute present at this location of data file, is used by Integration and Acceptance test-suites for testing for forming the header parameters. This helps to understand the the set of requests are triggered from test-suites if in future analysis.

Problem:
Absence of this attribute makes the value an empty string.

Upgrade ApplicationPattern and BasicService npm package to version 1.0.1

The following new releases of npm package for both ApplicationPattern and BasicServices are available,
• onf-core-model-ap+1.0.1
• onf-core-model-ap-bs+1.0.1

Upgrade ApplicationPattern and BasicService npm package to version 1.0.1.
Changes ,

In the file https://github.com/openBackhaul/ExecutionAndTraceLog/blob/develop/server/package.json , change the versions of the following two pacakges to 1.0.1
Perform basic test and validate the changes and push the code.

Reference : openBackhaul/ApplicationPattern#338

Update ElasticsearchClient during application update process

Context :
In EaTL , we are storing the “execution trace” records in the profile-collection as “service-record-profile” instance.
Based on the decision from the ApplicationPattern#issues#105 , we have to separate the application date from the configuration data.
Also in EaTL , continuous parallel updation in the profile-collection sometimes makes the application down.
So , we are in a plan to include a database (or) opensource logging framework (or) flat file to store the application data.

Discussion on July 15th SDN application call :

To store the application data , dev team(Martin) is currently analysing about ElasticSearch.
Using the REST interface of ElasticSearch,

data can be transferred and stored from the EaTL application.
json data can be stored in the index specifically created for EaTL application.
data can be retrieved from ElasticSearch for list-records, list-records-of-flow APIs using REST interface.

Open points :

Where to store the connection string, username and password for access the ElasticSearch ?
proposals :
- Maintaining an application names ElasticSearchAuthProvider(ESAP) to holds the connection details of elastic search. On application approval by RegistryOffice , this ESAP application will communicate the connection details to EaTL.
- Maintaining the connection details in configuration data. During upgradation , old application will transfer the connection details to new application.
During bequeath-your-data-and-die , how we are going to transfer the application data ? (From recent mail chain)
Proposal :
- EATL-v2 should just use the same index "eatl-records" as EATL-v1. It should just connect to the same ES and not doing any data transfer from EATL-v1 to EATL-v2 via REST API.

Implementation to be updated in controllers/BasicService for the service redirect-topology-change-information

As per the changes in the issue openBackhaul/ApplicationPattern#305 , there is a modification in the response of the service /v1/redirect-topology-change-information. This service requires to send the details of the LTPs and FCs to the calling application.

Require to change the responseCode to “responseCodeEnum.code.OK” in the controllers/BasicServices.js redirectTopologyChangeInformation function.

Note : Implementation will be taken care as a part of the issue openBackhaul/ApplicationPattern#352

detailed-logging-is-on to be added to CC

Decision on the Application Implementation call on 13th or May:
The detailed-logging-is-on attribute shall be added to the response body of the /core-model-1-4:control-construct path in the OAS.
It shall be added to the properties section, but not the required section.

OAS : /v1/bequeath-your-data-and-die : callback : reference path needs update

Error description:
In callback "PromptForBequeathingDataCausesRObeingRequestedToNotifyApprovalsOfNewApplicationsToNewRelease" and "PromptForBequeathingDataCausesRObeingRequestedToNotifyWithdrawnApprovalsToNewRelease", we have request-body attribute "subscriber-operation", the reference path under description of the attribute needs an update. The path is intended to refer an operation-server ltp (eatl-0-0-1-op-s-3001 and eatl-0-0-1-op-s-3002), but the proceeding path points to operation-client.

Possible solution:
In the description, path needs an update to point to operation-server-interface-capability/operation-name as below.

For PromptForBequeathingDataCausesRObeingRequestedToNotifyApprovalsOfNewApplicationsToNewRelease, the current description contains /core-model-1-4:control-construct/logical-termination-point=eatl-0-0-1-op-s-3001/layer-protocol=0/operation-client-interface-1-0:operation-client-interface-pac/operation-client-interface-configuration/operation-name, which actually would be
/core-model-1-4:control-construct/logical-termination-point=eatl-0-0-1-op-s-3001/layer-protocol=0/operation-server-interface-1-0:operation-server-interface-pac/operation-server-interface-capability/operation-name

Similarly, For PromptForBequeathingDataCausesRObeingRequestedToNotifyWithdrawnApprovalsToNewRelease, the description would be updated as
/core-model-1-4:control-construct/logical-termination-point=eatl-0-0-1-op-s-3002/layer-protocol=0/operation-server-interface-1-0:operation-server-interface-pac/operation-server-interface-capability/operation-name

Note: eatl-0-0-1-op-s-3001 refers to /v1/regard-application and eatl-0-0-1-op-s-3002 refers to /v1/disregard-application

Datatype enumeration value "String" -> "string"

'String' shall be replaced by 'string' in the OAS of ApplicationPattern and all other applications.

Load file : detailed-logging-is-on attribute to be removed from logical-termination-point instances

As per understanding, detailed-logging-is-on attribute is applicable only for the operation "/v1/record-service-request" of ExecutionAndTraceLog

Problem Description:
In load-file, ExecutionAndTraceLog_0.0.2_tsi.211217.1630+data.2.json, for ltp instance eatl-0-0-1-op-c-2040 contains an additional attribute detailed-logging-is-on, which needs to be removed.

Note :

eatl-0-0-1-op-c-2040 refers to /v1/redirect-service-request-information

consequent-action-list to be complemented with displayInNewBrowserWindow attribute

In

/v1/start-application-in-generic-representation :: responses :: consequent-action-list

displayInNewBrowserWindow attribute to be added.

APIgateway performance testing for container's east-west communication

Creating this issue to make changes to the api/openapi.yaml and database/load.json file to include the prefix /eatl to the path to facilitate the east-west communication via APIgateway.

api/openapi.yaml :
Included the prefix /eatl to each path of the basic and individual service
Removed the ipv-4 pattern string to support domain name temporarily

database/load.json :
Included the prefix /eatl for all operation-name in operation servers
Included the prefix /ro for all RegistryOffice operation-name
Included the prefix /tar for all TypeApproval operation-name
Modified the ipv-4-address, port of ro,tar,eatl to "mwkibana.testmob.de" ,"80"

index.js :
Including the following code to the swagger router will help in hosting the swagger GUI and api file to /eatl/docs /eatl/api-docs

swaggerUI : {
        swaggerUIPath: '/eatl/docs',
        apiDocsPath: '/eatl/api-docs'
    }

OAS : OAM layer : incorrect uuid pattern for service-record-profile

In OAS specification :: OAM layer, there is a API request for retrieving service-record-profile-capability
"/core-model-1-4:control-construct/profile-collection/profile={uuid}/service-record-profile-1-0:service-record-profile-pac/service-record-profile-capability".
Here, the pattern of uuid is defined as pattern: '^eatl-0-0-1-op-s-\d{4}$' , which actually could be pattern: '^eatl-0-0-1-service-record-p-\d{6}$'

/core-model-1-4:control-construct/uuid is missing

In /core-model-1-4:control-construct :: responses
core-model-1-4:control-construct should be complemented with

required:
  - uuid

and

properties:
  uuid:
    type: string

Add server side stub for the specification

Describe your feature request :

Generate server side code for the specification

Implementation tasks and next steps :

Generate server side stub for the latest specification https://github.com/openBackhaul/ExecutionAndTraceLog/blob/tsi/ExecutionAndTraceLog_0.0.2_tsi.211217.1630.yaml
Add the required dependencies in the package.json

Alternate proposals if any :

Additional context :

Updating UUIDs version to 2.0.0

In Current files, the release number of applications points to version 1.0.0. This needs to be updated to 2.0.0

Version update to be made in

CONFIG file
OAS
Forwarding List
Service List

PUT operations in OAM layer should return 204 (Implementation)

Problem description
For PUT request in OAM layer , after successful updation , server is returning 200 OK as the ResponseCode.
As per the specification and generic concepts, this is not correct.

Proposed solution
Since the PUT operation doesn't return any value in the ResponseBody , the ResponseCode should be 204 NO_CONTENT.

Add implementation for the individual service to list the records

Describe your feature request :

Support functionality and forwarding for the service /v1/list-records, /v1/list-records-of-flow, /v1/list-records-of-unsuccessful

Implementation tasks and next steps :

Complement the listRecords operation in controller/IndividualServices.js
Complement the listRecords operation in services/IndividualServices.js
Support forwarding ServiceRequestCausesLoggingRequest to Execution and Trace Log
Complement the listRecordsOfFlow operation in controller/IndividualServices.js
Complement the listRecordsOfFlow operation in services/IndividualServices.js
Support forwarding ServiceRequestCausesLoggingRequest to Execution and Trace Log
Complement the listRecordsOfUnsuccessful operation in controller/IndividualServices.js
Complement the listRecordsOfUnsuccessful operation in services/IndividualServices.js
Support forwarding ServiceRequestCausesLoggingRequest to Execution and Trace Log

Alternate proposals if any :

Additional context :

Add implementation for Execution And Trace Log V0.0.2

Describe your feature request :

Include implementation for the specification of Execution And Trace Log V0.0.2

Implementation tasks and next steps :

complete #9
complete #10
complete #11
complete #12

Alternate proposals if any :

Additional context :

openbackhaul / executionandtracelog Goto Github PK

executionandtracelog's People

Contributors

Stargazers

Watchers

Forkers

executionandtracelog's Issues

Proposal for High availability :

Proposal to prevent data loss :

Open points :

Recommend Projects

Recommend Topics

Recommend Org