Git Product home page Git Product logo

grokify / chathooks Goto Github PK

View Code? Open in Web Editor NEW
40.0 5.0 10.0 6.56 MB

Service to convert webhook messages to your favorite chat / team messaging format. Run transparently as a formatting webhook proxy.

Home Page: https://grokify.github.io/chathooks/

License: MIT License

Go 99.51% Makefile 0.14% Shell 0.09% Dockerfile 0.25% Procfile 0.01%
glip webhook go golang glip-webhook slack slack-webhook

chathooks's Introduction

Chathooks - A webhook formatter for chat

Build Status Go Report Card Docs LOC License Heroku

Overview

Chathooks is a webhook proxy service that converts generic outbound webhook messages to a canonical CommonChat message format which is then sent to any chat / team messaging platform supported by the CommonChat chat post abstraction library.

This is useful because:

  • many services with outbound webhooks need to be formatted before they can be consumed by an inbound webhook. This proxy service does the conversion so you don't have to. The code is all open source so customizations can be done easily.
  • the conversion can be done one time for all chat / team messaging solutions supported by CommonChat.
  • one service can proxy an arbitrary number of webhook sources and event types so you don't have to configure multiple inbound webhooks going to the same group / channel.

See the following video to get started quickly:

Supported HTTP Engines

Chathooks supports multiple HTTP engines including the following:

Supported Webhook Formats

Multiple input webhook formats are supported via handlers. New ones can be easily created by using the handlers.Handler interface.

  1. Aha!
  2. AppSignal
  3. Apteligent/Crittercism
  4. Bugsnag
  5. Circle CI
  6. Codeship
  7. Confluence
  8. Datadog
  9. Desk.com
  10. Enchant
  11. GoSquared
  12. Heroku
  13. Librato
  14. Magnum CI
  15. Marketo
  16. Nixstats (Slack webhook proxy)
  17. OpsGenie
  18. Papertrail
  19. Pingdom
  20. Raygun
  21. Runscope
  22. Semaphore CI, Deploy
  23. StatusPage
  24. Travis CI
  25. Userlike
  26. VictorOps

Here is an exmaple Webhook Message from Travis CI formatted for Glip.

The above are "outbound" webhook formats from the perspective of the service providing the events.

A special webhook format supported is the Slack "inbound" webhook format. This format message isn't sent by Slack but is accepted by Slack. It is useful to allow modification of messages already formatted for Slack.

Supported Chat Services

Chathooks can post messages to any service supported by CommonChat. New services can be added by creating an adapter using the commonchat.Adapter interface.

Note: The emoji to URL is designed to take a icon_emoji value and convert it to a URL. EmojiURLFormat is a fmt format string with one %s verb to represent the emoji string without :. You can use any emoji image service. The example shows the emoji set from github.com/wpeterson/emoji forked and hosted at grokify.github.io/emoji/.

Installation

Local

$ go get github.com/grokify/chathooks

Heroku

Deploy

or

$ heroku create
$ git push heroku master
$ heroku open

AWS Lambda

The following will generate a main.zip file to upload to AWS Lambda.

$ go get github.com/grokify/chathooks
$ cd chathooks
$ sh aws-package.sh
$ ls main.zip

Configuration

Environment Variables

Chathooks uses two environment variables:

Variable Name Value
CHATHOOKS_ENGINE The engine to be used: awslambda for aws/aws-lambda-go, nethttp for net/http and fasthttp for valyala/fasthttp. Leave empty for eawsy/aws-lambda-go-shim as it does not require a server to be started.
CHATHOOKS_TOKENS Comma-delimited list of verification tokens. No extra leading or trailing spaces.

Using the net/http and fasthttp Engines

  1. To adjust supported handlers, edit server.go to add and remove handlers.
  2. Set the CHATHOOKS_ENGINE environment variable to nethttp or fasthttp.
  3. Start the service in main.go.

For testing purposes, use:

$ go run main.go

For production services, compile and run the binary:

$ go build main.go
$ ./main

Using the AWS Engine

To use the AWS Lambda engine, you need an AWS account. If you don't hae one, the free trial account includes 1 million free Lambda requests per month forever and 1 million free API Gateway requests per month for the first year.

Installing using the AWS Lambda UI

Installation via AWS Lambda

See the AWS docs for deployment:

https://docs.aws.amazon.com/lambda/latest/dg/lambda-go-how-to-create-deployment-package.html

Using the aws-cli you can use the following approach:

$ GOOS=linux go build main.go
$ zip main.zip ./main
# --handler is the path to the executable inside the .zip
$ aws lambda create-function \
  --region region \
  --function-name Chathook \
  --memory 128 \
  --role arn:aws:iam::account-id:role/execution_role \
  --runtime go1.x \
  --zip-file fileb://main.zip \
  --handler main

You can use the aws-package.sh shell script to package your handler.

Note: you can install the aws-cli using the appropriate command below:

$ pip install awscli
$ pip3 install awscli

After installation, you may need to add your Python bin directory to your path. The following are examples for OS X.

$ export PATH=$PATH:/Users/~/Library/Python/2.7/bin
$ export PATH=$PATH:/Library/Frameworks/Python.framework/Versions/3.6/bin

Update Lambda Code:

You can update the Lambda function code using the following:

https://docs.aws.amazon.com/cli/latest/reference/lambda/update-function-code.html

$ aws lambda update-function-code --function-name='MyFunction' --zip-file='fileb://main.zip' --publish --region='us-east-1'

The aws-update.sh file has this command with default settings.

Make sure to set your AWS credentials file.

Manual Setup

  • Configure AWS Lambda Function
    1. Create AWS Lambda function
    2. Set CHATHOOKS_ENGINE to awslambda
    3. Set CHATHOOKS_TOKEN as desired
    4. Set Handler to main
  • Configure API Gateway
    1. Create "REST" API
    2. Select "New API"
    3. Enter "API name"
    4. Click "Create API"
    5. Select "Actions" > "Create Resource" on "/"
    6. Click "Configure as proxy resource"
    7. Use Resource Path {proxy+}
    8. Click "Enable API Gateway CORS"
    9. Click "Create Resource"
    10. Leave "Integration Type" as "Lambda Function Proxy"
    11. In "Lambda Function", paste in your Lamda ARN
    12. Click "Save"
    13. Click "Actions" > "Deploy API" and create stage if necessary.
    14. Copy "Invoke URL" which should end in {stageName}

Troubleshooting

  • 401 Unauthorized Error: If you can run a test on API Gateway and have deployed your API but are running into a HTTP Status 401 Unauthorized error, check to see if you have configured the CHATHOOKS_TOKEN environment variable but not entered a token query string parameter.

Usage

Creating the Glip Webhook

  1. create a Glip webhook
  2. use webhook URL's GUID to create the proxy URL as shown below
  3. use the proxy URL in your outbound webhook service
Query Parameter Required? URL
inputType required An handler service like marketo
outputType required An adapter service like glip
outputURL required A webhook URL or UID, e.g. 11112222-3333-4444-5555-666677778888
outputFormat optional one of [nocard,card]
token optional Must be included if service is configured to use auth tokens

The webhook proxy URLs support both inbound and outbound formats. When available, these should be represented in the handler key.

To create the Glip webhook and receive a webhook URL do the following:

Add the Webhook Integration

At the top of any conversation page, click the Settings gear icon and then click Add Integration.

Select the Glip Webhooks integration.

Get the Webhook URL

Once you get the URL, the proxy URL is created by appending the GUID (e.g. 1112222-3333-4444-5555-666677778888) to the proxy URL base, hooks?inputType=slack&outputType=glip (e.g. https://glip-proxy.example.com/hooks?inputType=slack&outputType=glip&url=1112222-3333-4444-5555-666677778888). Use the proxy URL in the app that is posting the Slack webhook and the payload will be sent to Glip.

Development

Testing new handlers

It's easy to test handlers by sending messages using the example messages per service.

Code is provided to both send the message locally (without HTTP) and over HTTP.

cURL

The following is an example curl command that can be used with any host, local or remote.

curl -XPOST 'https://example.com/webhook?inputType=datadog&outputType=glip&url=https://hooks.glip.com/webhook/11111111-2222-3333-4444-555566667777' --data "@docs/handlers/datadog/event-example_formatted1.json" -H 'Content-Type: application/json' --verbose

Adding Handlers

The easiest way to add a handler is to inspect the code of an existing handler and build something similar. It needs satisfy the handlers.Handler interface.

Notes

Maintenance

Use godep to rebuild the vendor directory with:

$ rm -rf Godeps
$ export GO15VENDOREXPERIMENT=1
$ godep save ./...

More information on deploying Go on Heroku here:

Heroku

Set up your own Go app on Heroku easily using goheroku.

Credits

Chathooks is built using:

chathooks's People

Contributors

dependabot[bot] avatar drewligman avatar grokify avatar ylecuyer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

chathooks's Issues

[AWS] Token not parsed from URL request

When using the awslambda engine, the request query string parameter token is not processed when CHATHOOKS_TOKENS is required and a 401 error is returned.

The following appears in the CloudWatch logs:

time="2019-04-04T23:53:21Z" level=warning msg=E_NO_TOKEN

Slack → Glip conversion slightly broken when using newer JWT glip tokens.

We (the engage digital team) recently updated some of our Glip tokens in our updown.io integration (which is using the Slack to Glip translation provided by chathooks) and noticed the following differences:

Before (old token):
image

After (JWT token):
image

  • The most important missing part here (the website name/url/link) comes from the author_name & author_link attributes in the Slack webhook payload.
  • I also noticed the pretext is missing (it's the "ALERT: DOWN" text).
  • The author_image (which was broken previously in this example because it's a 404) is now missing, it's not a big deal but if it can be fixed it would be nice
  • And finally the integration image is no longer showing the updown.io logo (which is in the icon_url field of the Slack webhook).

cc @ylecuyer @spuyet @CedricBm @nel

Support stripping Stack inbound webhook code block auto URL links

Support removing auto URL link in code block because Glip [code] block cannot interpret links with in the block.

Example:

"curl -gkvL -H 'User-Agent: updown.io daemon 2.2' -H 'Accept: */*' -H 'Connection: Close' -H 'Accept-Language: en' -m 30 --connect-timeout 5 \u003chttp://example.com|http://example.com\u003e"

https://github.com/grokify/chathooks/blob/master/docs/handlers/slack/event-example_link-emoji.json

{
  "username":"updown.io",
  "attachments":[
    {
      "text":"Down since *20:17:29 (CEST)*, reason: *Couldn't find the string*",
      "fallback":"DOWN ALERT: http://example.com Down since 20:17:29 (CEST), reason: Couldn't find the string",
      "mrkdwn_in":[
        "text",
        "pretext",
        "fields"
      ],
      "pretext":"DOWN ALERT: :frowning:",
      "author_name":"http://example.com",
      "author_icon":"https://staging.updown.io/checks/mycheck/favicon",
      "author_link":"http://example.com",
      "color":"#e43",
      "fields":[
        {
          "title":"String match:",
          "value":"_“12345”_",
          "short":true
        },
        {
          "title":"🔇 Mute alerts:",
          "value":"\u003chttps://staging.updown.io/checks/mycheck/mute#1h|For 1 hour\u003e, \u003chttps://staging.updown.io/checks/mycheck/mute#1d|for 1 day\u003e, \u003chttps://staging.updown.io/checks/mycheck/mute#1w|for 1 week\u003e, \u003chttps://staging.updown.io/checks/mycheck/mute#recovery|until recovery\u003e or \u003chttps://staging.updown.io/checks/mycheck/mute#forever|forever\u003e"
        },
        {
          "title":"↻ Reproduce request:",
          "value":"```curl -gkvL -H 'User-Agent: updown.io daemon 2.2' -H 'Accept: */*' -H 'Connection: Close' -H 'Accept-Language: en' -m 30 --connect-timeout 5 \u003chttp://example.com|http://example.com\u003e```"
        }
      ]
    }
  ],
  "icon_url":"https://updown.io/square-logo.png"
}

[Heroku] Install v0.5.0 issue

Migrating to Go modules from godep results in an issue with Heroku deployment.

Heroku cannot find go 1.15 in go.mod file and attempts to load Go 1.12.17 which fails as shown below.

-----> Go app detected
-----> Fetching jq... done
-----> Fetching stdlib.sh.v8... done
-----> 
       Detected go modules via go.mod
-----> 
       Detected Module Name: github.com/grokify/chathooks
-----> 
 !!    The go.mod file for this project does not specify a Go version
 !!    
 !!    Defaulting to go1.12.17
 !!    
 !!    For more details see: https://devcenter.heroku.com/articles/go-apps-with-modules#build-configuration
 !!    
-----> New Go Version, clearing old cache
-----> Installing go1.12.17
-----> Fetching go1.12.17.linux-amd64.tar.gz... done
-----> Determining packages to install
 !     Push rejected, failed to compile Go app.
 !     Push failed
Build finished
Run scripts & scale dynos
Deploy to Heroku

[Godep] Update fails

Currently Heroku deployment is broken as godep save ./... fails with the following error:

$ godep save ./...
godep: cannot save github.com/aws/aws-lambda-go/lambda/handlertrace at revision af0b813d5803d9754b920ed666b1cf8c16becfb3: already have github.com/aws/aws-lambda-go/events at revision b2a9167f3412a1daa5a3051a10839631063bd3d8.
Run `godep update github.com/aws/aws-lambda-go/events' first.

[Slack-to-Glip] Emoji in text issue

When receiving the following message from Slack inbound to Glip inbound, the emoji :bangbang: appears as the literal text :bangbang: in Glip, not as an emoji.

payload={
	"channel":"#mychannel",
	"username":"My Alert Service",
	"text":":bangbang: Test message from My Alert Service",
}

Change RC App "activity" line for Slack proxy

When using the Slack to RC proxy, the RC App activity line just says "shared items". Can it be changed to something more meaningful?

For bespoke integrations the commonchat.Message.Activity property is set which customizes the activity line field in RC App, but it's not set for the Slack proxy.

Allow handlers to return status code

Allow handers to provide richer information to:

  • return status code
  • return instructions to skip sending (other than errors) for desired filtering

[Slack] Support Attachments

Support inbound Slack attachments:

func Normalize(config config.Configuration, bytes []byte) (cc.Message, error) {
slack, err := SlackWebhookMessageFromBytes(bytes)
if err != nil {
return cc.Message{}, err
}
message := cc.Message{
Activity: slack.Username,
Text: slack.Text,
IconEmoji: slack.IconEmoji,
IconURL: slack.IconURL}
return message, nil
}

This is needed for UpDown.io:

https://github.com/grokify/chathooks/blob/5b55995b1d4b155670aac3d42121fe72a1cdad3e/docs/handlers/slack/event-example_attachment.json

Add UI to auto-create / manage tokens

The goal is to have people use a deployed version of Chathooks without manual request or provisioning

The requirement is to be able to manage tokens by individuals / organizations, including CRUD, though CRD may be good enough here.

To meet this requirement, the following should be implemented:

  • UI with OAuth-based login
  • database to store tokens

Dependabot

Vulnerability

remote: GitHub found 1 vulnerability on grokify/chathooks's default branch (1 moderate). To find out more, visit:
remote:      https://github.com/grokify/chathooks/security/dependabot/go.sum/github.com%2Fmicrocosm-cc%2Fbluemonday/open

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.