Git Product home page Git Product logo

go-zeek-broker-ws's Introduction

Zeek Broker websocket interface library for Golang

This library implements the Zeek Broker websocket interface.

Helper functions are provided with reasonable mappings between Zeek and Go types.

Usage

The library contains two packages:

encoding

encoding models the recursive JSON-based data structure for representing Zeek types (the Data struct), as well as the Broker encoding of Zeek events and error messages (the DataMessage struct). These structures can (and in some cases must) be used directly (see the subscription example in the next section), but some helper functions make building messages more convenient. To construct a Zeek string manually:

zeekString := encoding.Data{
    DataType:  encoding.TypeString,
    DataValue: "foo",
}

...or via the helper:

zeekString := encoding.String("foo")

The vector helper in particular is a variadic function that accepts encoding.Data:

zeekVector := encoding.Vector(encoding.Count(1), encoding.Count(2), encoding.Count(3))

Finally, events can be created directly:

zeekEvent := encoding.NewEvent("some_event_name", zeekVector, zeekString)

client

client provides the websocket glue to speak to the broker WS API, wrapping github.com/gorilla/websocket:

To publish an event:

broker, err := client.NewClient(...)

err := broker.PublishEvent("/the/topic", zeekEvent)

Topic subscriptions are passed as a slice of strings to client.Newclient(). The ReadEvent() method of the client returns a single event from Broker (on any of the subscribed topics), or an error that could occur in the library itself ir errors received from Broker):

broker, err := client.NewClient(..., []string{"/the/topic"})

topic, zeekEvent, err := broker.ReadEvent()

If the broker connection is closed gracefully, the client.IsNormalWebsocketClose() function can be used to check the returned error.

Client code must access event argument values via type assertions:

if len(evt.Arguments) < someConstantGreaterOrEqualToOne {
	// handle too few arguments case
}

if evt.Arguments[0].DataType != encoding.TypeString {
	// handle unexpected data type case
}

stringArgument0, ok := evt.Arguments[0].DataValue.(string)
if !ok {
	// handle type assertion error - this would indicate a bug (we trust+verify).
}

// now we use stringArgument0

Asynchronous handling and dispatching of events received via subscriptions would best implemented as a Client.ReadEvent() wrapper. A simple implementation is provided in client.AsyncSubscription().

More advanced handling of the websocket connection (e.g., setting timeouts, handling re-connection, etc.) is best implemented as a wrapper of client.Client, or a new/replacement implementation that uses the encoding package (contributions/PRs are welcome!).

Broker TLS details

Broker network connections (both native, and the websocket interface) enable TLS by default with an odd configuration that disables host verification and selects a set of cipher that allow encryption without certificates. To use this mode requires passing weirdtls.BrokerDefaultTLSDialer as the dailer function argument to encoding.NewClient. Note that this pulls in OpenSSL as a dependency.

Alternatively the standard library crypto/tls implementation can be used if both sides (the client and zeek/broker) is configured to use TLS with certificates. This library provides a convenient helper function (securetls.MakeSecureDialer()) that returns a dialer function given PEM files for the CA and client certificate/key. See this btest case for an example of this configuration.

Finally, TLS can be turned off for broker connections using redef Broker::disable_ssl = T;. See this btest case for an example where encoding.NewClient is called with arguments for insecure operation.

Ping/pong example

Running a zeek-side broker script:

$ cd example/
$ zeek listen.zeek

Now run the example:

$ cd example/
$ go build && ./example 
2023/05/05 12:56:55 connected to remote endpoint with UUID=c9b0bfd6-3b8d-5de2-a51c-9af7b81aaad3 version=2.5.0-dev
2023/05/05 12:56:55 > topic=/topic/test | event ping("my-message": string, "1": count)
2023/05/05 12:56:55 < topic=/topic/test | event pong("my-message": string, "2": count)
2023/05/05 12:56:56 > topic=/topic/test | event ping("my-message": string, "2": count)
2023/05/05 12:56:56 < topic=/topic/test | event pong("my-message": string, "3": count)

...meanwhile the zeek script:

peer added, [id=e537f8b4-de32-52ea-9587-4e6e15bdfe20, network=[address=127.0.0.1, bound_port=50690/tcp]]
receiver got ping: my-message, 1
receiver got ping: my-message, 2

Dev workflow

Most helpers and basic data structures in the encoding package have unit tests (run go test ./...).

End-to-end tests are implemented as two btest cases (run cd tests/; btest).

go-zeek-broker-ws's People

Stargazers

Jared Gore avatar

Watchers

Christian Kreibich avatar Simeon Miteff avatar Robin Sommer avatar Arne Welzel avatar Carl A Lewis avatar  avatar

go-zeek-broker-ws's Issues

Btests will fail after 9 June 2025

The server cert used in btests is generated by minica which sets the expiry date too soon for comfort:

$ openssl x509 -in tests/Files/cert.pem -enddate | head -n1
notAfter=Jun  9 01:50:28 2025 GMT

Support encoding structs with the undocumented vector-based broker encoding for zeek records

From zeek/zeek/discussions/2879:

Mapping Zeek's records to Broker's data model expects records as a tuple of values, with each member in its own Broker-model rendering. The vector must cover the full list of fields, including Nones for optional ones.

This actually works. Assuming

type Foo: record {
  bar: string;
  baz: count;
};

...we could construct a value like this:

encoding.Vector(
  encoding.String("value of bar"),
  encoding.Count(42),
)

We could add support to go-zeek-broker-ws to encode Go structs in this manner, but given the discussion referenced above: should we?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.