kyma-incubator / github-slack-connectors Goto Github PK

NOTE
Merged to issue #49

Description
During tests some bugs and shortcomings have came up. They need to be fixed

AC

• error handling in SendRegisterRequest (92 line)
• enhance comments
• dynamic URL address based on Kyma application.yaml name (Release.Name-app)

Description
Lambdas need subscribtion to certain events in Event-Bus so they can be triggered by them - in order to do that, we need to get familiar with Subscription Custom Resource and Event Activation CR.

AC

• Short document describing the idea and structure of Subscription CR and Event Activation CR

Description
To allow GitHub webhooks to send us events we have to expose an address outside of Kyma.

AC

• Create yaml as described here
• Add configuration to values.yaml if needed

Description
Task was moved to #99

Add the required milestones to our zenhub board

Description
This task seems obsolete so we're closing it.

Description
During Code Review it turned out that our component application registry can be improved. We should refactor it to be more readable, useful (for example in mocking) and golang style using interfaces.

AC

• Prepare an interface containing all necessary functions
• Move the json to separate file
• Put retrying to send the request into separate function (generic, example in link)
• Check if tests need to be edited

DESCRIPTION
In order to understand the concept of existence of Github Connector in Kyma ecosystem, we need to do research about core components and connections between them.
AC
Understanding of core components and connections between them.

Description
To ensure that our component is well-documented, we need to validate existing docs and update them if necessary, plus add the parts of documentation which are missing.

AC

• Hack showcase docs are being compatible with the kyma templates
• All the docs are updated to the newest stable versions of the components

Description
In order to keep information about future plans for the product in one place there is /docs/roadmap.md file waiting to be filled. It should provide some sort of summary for future development plans of the project in a semi-technical language so it doesn't repeat things that are included in concept.md.

AC

• Document describing future development plans and milestones located in /docs/roadmap.md

DESCRIPTION
It is necessary to handle errors that can occur during runtime of existing structure of the connector.

AC

• Think about the possible error cases
• Design of error handling and messaging structure
• Application of error handling in all reasonable places

Description
To protect the project from regression-related errors and to automate part of our DoD, we need to integrate basic Continuous Integration into our project. For the start, it would be enough to block merging unless all unit tests pass. The recommended tool for that is prow and it is used in Kyma. There is no need to set up our own prow, we can use the existing one and add new job configuration to it.

AC

• Do research about Prow CI - contact someone who is responsible for this topic
• Prepare all files necessary to connect our repository to Prow and create Prow webhook in GitHub
• Create PR to kyma/test-infra which allow our repository to use Prow
• Create Makefile and before-commit.sh files which are responsible for executing tests
• All unit tests should run on a merge request to master branch
• Fail of any unit test should block merging a branch to master

DESCRIPTION
In order to use application in Kyma ecosystem we need to register our app in Application Registry Component. To achieve this we need to create Application CR.

AC

• Create Application CR
• Add Application to helm chart
• Create json to register service
• Send json with POST to Application Registry
• Ask PB how to create Application CR (go, helm chart, client?)

DESCRIPTION

After creating basic structure of description of GitHub events in AsyncAPI standard we need to add another layer which uses references to obtain complete format.

AC

• Create full structure of GitHub event description in AsyncAPI standard

Description
Our Slack Connector needs to be deployed in Kyma environment. To achieve this goal we could create Kubernetes resources and deploy them one after the other, but we can automate this process by creating Helm chart.

AC

• Prepare all necessary resources in YAML
• Add all labels from Helm best practices
• Take those comments into consideration
• Template values (if needed check GtiHub Connector chart for example)

In order to document our progress there is a need to describe the content of release. We have decided to list added features in form of short descriptions starting with infinitive verbs

AC

• List of features

Description
Main GitHub Connector's responsibility is to handle an endpoint, on which GitHub should send events. User has to configure webhook on GitHub, pointing to that endpoint, so GitHub knows where to send events. We should automate this creation process from our GItHub Connector.

AC

• Automate the creation of GitHub webhook process
• Prepare a document explaining to user how he can configure the webhook

Description
For smoother planning this week we need to define scope of the next sprint.

Description
To register event in Kyma it has to meet the requirements of OpenAPI specification.

AC

• Check if GitHub events are described in OpenApi somewhere
• If not - describe it

DESCRIPTION
In order to start dealing with hack-showcase review documentation it is necessary to determine the exact purpose of this document.

AC

• Determine the exact purpose of /hack-showcase/README.md

Description
In order to implement the project concept's functionalities in our lambda functions we should experiment with the Azure Service. The expected outcome would be the idea for implementing mechanism of handling the Azure response in lambda. In other words, we should test a lot of possible comments (we can use those which are already on Kyma repository) and try to figure out what can we do with the response payload. Some of the ideas could be:

• Notifying about issues which body's sentiment level is <0.3
• If issue body contains key-words like ["bug", "error"], add label bug to the issue

The other thing to research is the procedure for communicating with Azure Services - endpoints, payload formatting, auth etc.

AC

• A document describing steps needed to connect to Azure Services (authentication, payload formatting etc.)
• A document describing possible Azure's response handling scenarios
• Presentation meeting

Description
At the moment our project's architecture is not clearly declared. We should create diagrams explaining and visualising our project architecture. My proposal is to use UML Sequence Diagram for the detailed flow inside the system, and any type of flow diagram (could look like those in Kyma docs) for the bigger picture (how components interact with each other on a higher level)

AC

• Diagrams visualising project's architecture published on our repository, which are understandable for any person:
  • High-level software architecture diagram (high level communication)
  • Sequence diagram of the system (more details)

Description
OpenAPI is an international standard for describing APIs. Kyma Application Registry requires it for service registration, so we need to find or prepare one for Slack API.

AC

• Search for Slack API description in OpenAPI 3.0
• If not found, search for Slack API description in other standard and try to parse it to OpenAPI 3.0
• If not found, create Slack API description in OpenAPI 3.0

Description
A new application boilerplate should be created. It should contain basic project structure for Slack Connector based on existing Github Connector. The project structure should cover domain directories for app's packages, so we can start working on functionalities separately. It should present shapes of structs and interfaces.

AC

• Slack Connector architecture model presented to the team
• Slack Connector feature branch with proper project structure (directories)
• Description of further tasks

Description
In order to create the Proof of Concept app we need a Github Connector application boilerplate. We are going to use Golang here, as it will make the process of integrating the connector into Kyma easier in the future. For the start, it should have:

• possibility to communicate with Github API (app -> github communication)
• example functionallity (like notifying about star on given repo in console log or something)

AC

• Functional demo covering bulletpoints above
• Document describing the structure of the app and steps needed to run it
• Unit tests for core functionalities of each module

Description
We have created a script, which installs demo scenario of GitHub Connector usage in Kyma, but it's really simple and doesn't include any parameters validation nor explains how to use it. It should be developed.

AC

• Check parameters provided by user

Description
Adding lambda from Kyma UI does the CR creation automatically, so it suits out test scenario - that way we can easily check if lambda is triggering correctly on given event. We should create a lambda from UI, add a trigger to it and check in logs if the event is handled correctly.

AC

• Example lambda should trigger on 'issue opened' event and console log a message
• Short document describing the steps to reproduce it

DESCRIPTION
To write tests of good quality we need to prepare the list of questions/problems that we want to discuss during the workshop organized for us.

AC

• Create a list where every team member can write down his/her questions
• Send this list to Michał who will organize the meeting

Description
Lambda function which is triggered on incoming github event should be extended with functions for building a slack message from github payload and sending it to slack channel.

AC

• Slack message template
• Lambda in Kyma triggered on incoming Github event should send a message to a given channel

Description
In order to document our progress, we want every sprint review demo to be available as release on our repository. This requires creating a description with links to PRs covered in this particular release and publishing the release it on repository

AC

• Complete release description
• Release v0.2.0 available on repository

Description
To communicate with Slack from within Kyma we need to register it in Application Registry. We should check if it can be registered once or every channel has to be registered separately.

AC

• Research about Slack API to find out which way of registration is proper
• Create JSON required by Kyma Application Registry
• Post JSON to http://application-registry-external-api.kyma-integration.svc.cluster.local:8081/{APPNAME}/v1/metadata/services

Description
Defining an Event Activation Custom Resource for events that are currently implemented in Github Connector (which for now are 'issue opened' and 'pull request review comment') is necessary for Lambda triggering. Those should be automatically deployed with our Helm Chart as part of Github Connector deployment.

AC

• Event Activation CR for 'issue opened' event
• Event Activation CR for 'pull request review comment' event
• Adding CRs above to existing Helm Chart structure so they're included in Github Connector deployment

Description
Defining a Subcription Custom Resource for events that are currently implemented in Github Connector (which for now are 'issue opened' and 'pull request review comment') is necessary for Lambda triggering. Those should be automatically deployed with our Helm Chart as part of Github Connector deployment.

AC

• Subscription CR for 'issue opened' event
• Subscription CR for 'pull request review comment' event
• Adding CRs above to existing Helm Chart structure so they're included in Github Connector deployment

Description
In order to maintain the kyma level of documentation, we need to check if all the links in our documentation are alive. We can set up a job in Prow to achieve that

AC

• Research how the documentation is checked in kyma
• Set up our job

DESCRIPTION
In order to allow events from GitHub enter the Kyma ecosystem we need to create service which will parse JSON files to format accepted by Event Service component.

Note

• This is basically done in Wordpress Connector (check send_event() method)

$data = '{"event-type": "'.$this->event_type.'", "event-type-version": "'.$this->event_version.'", "event-id": "'. $this->gen_uuid() .'","event-time": "'.date("c",time()).'","data": {'.$data.'}}';

• Github's webhook payload JSON can be pushed into "data" field of the parsed event
• Also check /components/event-service/docs/api.yaml for details

AC

• Unit test

Description

Create the definition of done for our team workflow.

AC

• Organize team meeting about DoD
• Evaluate outcomes of the meeting and prepare document with our DoDs

Description
At some moment of development we may need Redis serving as a test-database. We should get to know how to manage it, so we can use it in the future.
Example scenario would be adding some of the events incoming from Github webhooks to Redis through a lambda (e.g. instead of console logging a message when lambda is triggered by an event we could add some log to redis)

AC

• Document describing process of deploying a redis pod in a cluster with brief summary of the possible use cases (redis functionallities)
• Working redis pod on a cluster

Description
For now Github Connector catches the webhook's payload and console log the event in handler method. The event handler needs to be modified so instead of console logging the incoming event it should use event-parser to form proper payload and send it to Kyma. It is supposed to act this way:

1. Incoming Github payload is catched by Github Connector
2. Github Connector handles the payload based on "action" field in incoming JSON payload (e.g. the event is recognized as "new issue opened" and at the moment it console logs the message about the creator of the issue (ksputo create a new issue: issue title))
3. Github Connector calls helper method from event-parser to create payload to send it to event-service (recognized event-type should be the same as event type registered in app registry and the github webhook payload should be inserted into 'data' field)
4. The created payload is send to event-service (POST request to '/v1/events' endpoint)
5. Now the lambda which is binded to this event type should be triggered

AC

• Events that are currently registered in Application Registry should be recognized properly by Github Connector resulting in adding the same event-type name as the one that is registered (e.g. for Github webhook payload for issue event with 'action': 'opened' the given event-type in final payload should be 'issuesevent.opened')
• Creating a example lambda #42 should works as intended - Github Connector needs to send event to Kyma and Lambda needs to react to incoming event

Description
AsyncAPI is an international standard for describing events from APIs. Kyma Application Registry requires it for service registration, so we need to find or prepare one for Slack API.

AC

• Search for Slack API events description in AsyncAPI
• If not found, search for Slack API events description in other standard and try to parse it to AsyncAPI
• If not found, create Slack API events description in AsyncAPI

Description
Using one of Azure Services is a part of the project concept. To use it, we need to call given endpoint from lambda with proper credentials provided by Azure Service Broker. The idea is to send some information from Github webhook payload (most likely a PR comment content and title) to Azure Text Analytics and handle the response - based on sentiment. The lambda should perform a proper action when the comment is breaking Code of Conduct - it can be private Slack message to the person responsible for community.

AC

• An example lambda calling one of Azure Services (handling authorization and payload building)
• Azure Services OpenAPI Broker utilization
• Description of scenario covering handling the response from service and use case

DESCRIPTION
We have to connect all the elements, that we were working on, together into one complete component with demo functionalities.
Moreover we need to prepare the presentation that we're going to show on the sprint review.

AC

• Prepare the presentation
• Push all the elements on the master branch
• Provide the kyma teams with an agenda and invitations
• Test if the component is working properly
• Dry run the presentation without using Zoom
• Dry run the presentation with Zoom

Description

AC

Description
We need a simple and fast way to install our GitHub Connector inside Kyma. To do so we will use helm. We have to:

• understand how helm charts and templates work
• deploy a chart deploying boilerplate which can be easily edited with parameters
• run chart inside Kyma cluster

AC

• Create a boilerplate chart which creates all required Kubernetes objects
• Run chart on Kyma and check if all required Kubernetes objects created and are working

Tidy up our zenhub board and create a task template

DESCRIPTION
In order to present our work to wider audience, there is a need to update the review of hack-showcase.

AC

• Describe the user scenario
• Include GitHub connector setup
• Include Slack connector seutp
• Describe Lambda usage, information and setup

In some points it may be reasonable to utilise the contents of GitHub Connector's README.md

Description
Before lambda function can subscribe an event with Subscription CR, a Service Instance CR must be created. It depends on Service Class CR external name, which is generated randomly, so it's necessary to get it automatically. We can do it using a Kubernetes Job

AC

• Create a Role and Role Binding to allow Job to get Service Class external name
• Create a Job getting Service Class external name
• Create Service Instance CR
• Ask Szymon about using scripts in Kubernetes Jobs
• Prepare helm chart installation flow to be sure about order in which files will be deployed