Git Product home page Git Product logo

webwire-go's Introduction


WebWire

WebWire for Go
An asynchronous duplex messaging library

Travis CI: build status Coveralls: Test Coverage GoReportCard CodeBeat: Status CodeClimate: Maintainability
Licence: MIT GoDoc

OpenCollective


WebWire is a high-performance transport independent asynchronous duplex messaging library and an open source binary message protocol with builtin authentication and support for UTF8 and UTF16 encoding. The webwire-go library provides a server implementation for the Go programming language.

Table of Contents

Installation

Choose any stable release from the available release tags and copy the source code into your project's vendor directory: $YOURPROJECT/vendor/github.com/qbeon/webwire-go. All necessary transitive dependencies are already embedded into the webwire-go repository.

Dep

If you're using dep, just use dep ensure to add a specific version of webwire-go including all its transitive dependencies to your project: dep ensure -add github.com/qbeon/[email protected]. This will remove all embedded transitive dependencies and move them to your projects vendor directory.

Go Get

You can also use go get: go get github.com/qbeon/webwire-go but beware that this will fetch the latest commit of the master branch which is currently not yet considered a stable release branch. It's therefore recommended to use dep instead.

Contribution

Contribution of any kind is always welcome and appreciated, check out our Contribution Guidelines for more information!

Maintainers

Maintainer Role Specialization
Roman Sharkov Core Maintainer Dev (Go, JavaScript)
Daniil Trishkin CI Maintainer DevOps

WebWire Binary Protocol

WebWire is built for speed and portability implementing an open source binary protocol. Protocol Subset Diagram

The first byte defines the type of the message. Requests and replies contain an incremental 8-byte identifier that must be unique in the context of the senders' session. A 0 to 255 bytes long 7-bit ASCII encoded name is contained in the header of a signal or request message. A header-padding byte is applied in case of UTF16 payload encoding to properly align the payload sequence. Fraudulent messages are recognized by analyzing the message length, out-of-range memory access attacks are therefore prevented.

Examples

Features

Request-Reply

Clients can initiate multiple simultaneous requests and receive replies asynchronously. Requests are multiplexed through the connection similar to HTTP2 pipelining. The below examples are using the webwire Go client.

// Send a request to the server,
// this will block the goroutine until either a reply is received
// or the default timeout triggers (if there is one)
reply, err := client.Request(
	context.Background(), // No cancelation, default timeout
	nil,                  // No name
	wwr.Payload{
		Data: []byte("sudo rm -rf /"), // Binary request payload
	},
)
defer reply.Close() // Close the reply
if err != nil {
	// Oh oh, the request failed for some reason!
}
reply.PayloadUtf8() // Here we go!

Requests will respect cancelable contexts and deadlines

cancelableCtx, cancel := context.WithCancel(context.Background())
defer cancel()
timedCtx, cancelTimed := context.WithTimeout(cancelableCtx, 1*time.Second)
defer cancelTimed()

// Send a cancelable request to the server with a 1 second deadline
// will block the goroutine for 1 second at max
reply, err := client.Request(timedCtx, nil, wwr.Payload{
	Encoding: wwr.EncodingUtf8,
	Data:     []byte("hurry up!"),
})
defer reply.Close()

// Investigate errors manually...
switch err.(type) {
case wwr.ErrCanceled:
	// Request was prematurely canceled by the sender
case wwr.ErrDeadlineExceeded:
	// Request timed out, server didn't manage to reply
	// within the user-specified context deadline
case wwr.TimeoutErr:
	// Request timed out, server didn't manage to reply
	// within the specified default request timeout duration
case nil:
	// Replied successfully
}

// ... or check for a timeout error the easier way:
if err != nil {
	if wwr.IsErrTimeout(err) {
		// Timed out due to deadline excess or default timeout
	} else {
		// Unexpected error
	}
}

reply // Just in time!

Client-side Signals

Individual clients can send signals to the server. Signals are one-way messages guaranteed to arrive, though they're not guaranteed to be processed like requests are. In cases such as when the server is being shut down, incoming signals are ignored by the server and dropped while requests will acknowledge the failure. The below examples are using the webwire Go client.

// Send signal to server
err := client.Signal(
	[]byte("eventA"),
	wwr.Payload{
		Encoding: wwr.EncodingUtf8,
		Data:     []byte("something"),
	},
)

Server-side Signals

The server also can send signals to individual connected clients.

func OnRequest(
  _ context.Context,
  conn wwr.Connection,
  _ wwr.Message,
) (wwr.Payload, error) {
	// Send a signal to the client before replying to the request
	conn.Signal(
		nil, // No message name
		wwr.Payload{
			Encoding: wwr.EncodingUtf8,
			Data:     []byte("example")),
		},
	)

	// Reply nothing
	return wwr.Payload{}, nil
}

Namespaces

Different kinds of requests and signals can be differentiated using the builtin namespacing feature.

func OnRequest(
	_ context.Context,
	_ wwr.Connection,
	msg wwr.Message,
) (wwr.Payload, error) {
	switch msg.Name() {
	case "auth":
		// Authentication request
		return wwr.Payload{
			Encoding: wwr.EncodingUtf8,
      			Data:     []byte("this is an auth request"),
		}
	case "query":
		// Query request
		return wwr.Payload{
			Encoding: wwr.EncodingUtf8,
			Data:     []byte("this is a query request"),
		}
	}

	// Otherwise return nothing
	return wwr.Payload{}, nil
}
func OnSignal(
	_ context.Context,
	_ wwr.Connection,
	msg wwr.Message,
) {
	switch string(msg.Name()) {
	case "event A":
		// handle event A
	case "event B":
		// handle event B
	}
}

Sessions

Individual connections can get sessions assigned to identify them. The state of the session is automagically synchronized between the client and the server. WebWire doesn't enforce any kind of authentication technique though, it just provides a way to authenticate a connection. WebWire also doesn't enforce any kind of session storage, the user could implement a custom session manager implementing the WebWire SessionManager interface to use any kind of volatile or persistent session storage, be it a database or a simple in-memory map.

func OnRequest(
	_ context.Context,
	conn wwr.Connection,
	msg wwr.Message,
) (wwr.Payload, error) {
	// Verify credentials
	if string(msg.Payload()) != "secret:pass" {
		return wwr.Payload{}, wwr.ReqErr {
			Code:    "WRONG_CREDENTIALS",
			Message: "Incorrect username or password, try again",
		}
	}
	// Create session (will automatically synchronize to the client)
	err := conn.CreateSession(/*something that implements wwr.SessionInfo*/)
	if err != nil {
		return nil, fmt.Errorf("Couldn't create session for some reason")
	}

	// Complete request, reply nothing
	return wwr.Payload{}, nil
}

WebWire provides a basic file-based session manager implementation out of the box used by default when no custom session manager is defined. The default session manager creates a file with a .wwrsess extension for each opened session in the configured directory (which, by default, is the directory of the executable). During the restoration of a session the file is looked up by name using the session key, read and unmarshalled recreating the session object.

Concurrency

Messages are parsed and handled concurrently in a separate goroutine by default. The total number of concurrently executed handlers can be independently throttled down for each individual connection, which is unlimited by default.

All exported interfaces provided by both the server and the client are thread safe and can thus safely be used concurrently from within multiple goroutines, the library automatically synchronizes all concurrent operations.

Hooks

Various hooks provide the ability to asynchronously react to different kinds of events and control the behavior of both the client and the server.

Server-side Hooks

  • OnClientConnected
  • OnClientDisconnected
  • OnSignal
  • OnRequest

SessionManager Hooks

  • OnSessionCreated
  • OnSessionLookup
  • OnSessionClosed

SessionKeyGenerator Hooks

  • Generate

Graceful Shutdown

The server will finish processing all ongoing signals and requests before closing when asked to shut down.

// Will block until all handlers have finished
server.Shutdown()

While the server is shutting down new connections are refused with 503 Service Unavailable and incoming new requests from connected clients will be rejected with a special error: RegErrSrvShutdown. Any incoming signals from connected clients will be ignored during the shutdown.

Server-side client connections also support graceful shutdown, a connection will be closed when all work on it is done, while incoming requests and signals are handled similarly to shutting down the server.

// Will block until all work on this connection is done
connection.Close()

Multi-Language Support

The following libraries provide seamless support for various development environments providing fully compliant protocol implementations supporting the latest features.

Security

A webwire server can be hosted by a TLS protected server transport implementation to prevent man-in-the-middle attacks as well as to verify the identity of the server during connection establishment. Setting up a TLS protected websocket server for example is easy:

// Setup a secure webwire server instance
server, err := wwr.NewServer(
	serverImplementation,
	wwr.ServerOptions{
		Host: "localhost:443",
	},
	// Use a TLS protected transport layer
	&wwrgorilla.Transport{
		TLS: &wwrgorilla.TLS{
			// Provide key and certificate
			CertFilePath:       "path/to/certificate.crt",
			PrivateKeyFilePath: "path/to/private.key",
			// Specify TLS configs
			Config: &tls.Config{
				MinVersion:               tls.VersionTLS12,
				CurvePreferences:         []tls.CurveID{tls.X25519, tls.CurveP256},
				PreferServerCipherSuites: true,
				CipherSuites: []uint16{
					tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
					tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
					tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
				},
			},
		},
	},
)
if err != nil {
	panic(fmt.Errorf("failed setting up wwr server: %s", err))
}
// Launch
if err := server.Run(); err != nil {
	panic(fmt.Errorf("wwr server failed: %s", err))
}

The above code example is using the webwire-go-gorilla transport implementation.


ยฉ 2018 Roman Sharkov [email protected]

webwire-go's People

Contributors

chebyrash avatar kernelpryanic avatar romshark avatar rzarifov avatar znarf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

webwire-go's Issues

Support for JSON message encoding

Currently only UTF8 and UTF16 text as well as plain binary messages are supported, so theoretically native support for JSON could prove handy.

Support for concurrent message processing

Currently the server does handle multiple connections concurrently but it processes the messages sequentially. Requests and signals of a single client are always processed in the goroutine handling that particular client, this could turn out critical, because even if we spawn goroutines manually in the OnRequest handler for example - we'd still block other requests when waiting for the blocking one to return a response. It'd therefore make sense to move the processing of messages into separate goroutines and ideally make this feature optional by providing a server option for disabling concurrent message processing.

TLS handshake error doesn't stop autoconnect

Bug Report

Expected Behavior

In the case of TLS handshake errors (such as a certificate verification error due to untrusted authority on a self-signed certificate), the client must immediately return an error aborting the automatic connection establishment process.

Actual Behavior

The automatic connection establishment process enters an infinite loop trying to connect indefinitely.

Reproduction Steps

  • Launch the chatroom example server with the provided self-signed certificate.
  • Launch the chatroom example client setting InsecureSkipVerify to false in the TLS configuration and watch it trying to connect indefinitely.

Environment

Listing all connections of a session

There's currently no way to list all connections (aka client agents) belonging to a certain session. The only function that comes close to solving this issue is SessionRegistry.SessionConnections though it returns only the number of connections of the given session. Wouldn't it make more sense to return []*Client instead?

Autoconnect in the background

Issue #11 already covers the autoconnect feature but misses one important case:

When the client connects to the server just to listen to server-side signals - the connection could get lost at some point (due to a server reboot for example) and the user-code will assume, that there's just nothing happening on the server, but in reality the server just won't send the signals due to the client being disconnected whole the time.
In such a case it's very important for the client to try to reconnect in the background as soon as the connection is lost to ensure maximum uptime.

Update session relevance

Sessions must get a new field "lastUse" of date type to indicate when a particular session was lastly updated. This is important for the session manager to know whether a session is still relevant during garbage collection.

Allow custom session key generators

Currently, there's no way to implement your own session key generator which is rather restrictive. It'd be nice to have the possibility to provide your own generator or use the built-in generator by default.

Default session storage

Currently sessions are silently disabled if one of the sessions related hooks is undefined which is rather unintuitive. Consider adding a default built-in session storage implementation to be used by default if all sessions related hooks are undefined. This would improve usability and help newcomers getting started faster.

Limit concurrency per client

As of #23 the maximum number of concurrently executed handlers can be limited globally affecting all connected clients, though it might make sense to also allow limiting the number of concurrently executed handlers per client as well.

Use standard error interface in OnRequest hook

Currently a custom error type webwire.Error is used in the OnRequest hook, which is fine but consider using the default Go error interface as this would ease the use of the library. When a non-webwire.Error is returned the transmitted reply error could probably have some default code value.

Deferred client-side session destruction

When the session is closed by the client offline due a temporarily interrupted connection, then the client will just remove and forget it while it will remain open on the server even when the connection is reestablished.

The client should be made responsible for cleaning up when a session was closed offline. It must remember the session key and request the destruction of the closed sessions as soon as the connection is reestablished, to reduce server-load.

Unsynchronized session getter

The client agent method client.Session() should return a copy of the session object though there is a problem with it returning a shallow copy of the session info map. This could lead to unsychronized access to the session info map as both session objects reference the same info object but can be owned by different goroutines.

Subprotocol Support

Generic Issue

Add support for custom subprotocols. Currently, sub-protocols have to be implemented using the namespacing feature, where different types of requests/signals have different names, which is fine so far but webwire doesn't distinguish between different subprotocols at all, it should at least avoid connecting to a webwire server that doesn't speak it's language.

Deferred client agent closure

Calling client.Close() on a client agent while processing it's request causes an internal server error, because the response cannot be sent due to the client being disconnected immediately.

client.Close() must defer closing the client agent until all work on that client agent is finished.

Concurrent access to client session causes data race

Problem

Concurrent access to the session object in the client by both the reading goroutine created during the connection setup and the writing goroutine calling the API (when calling the client.RestoreSession method for example) causes a data race!

Proposed Solution

Synchronize read-write access to the internal client session object.

Duplex Request-Reply

Currently the request-reply topology works in a single direction: client -> server which is fine for now, but shouldn't we consider adding support for duplex request-reply to also be able to send requests from the server to the client?

Multi-Connection Session

Problem

Currently a session can only be assigned to a single connection, though sessions should accept multiple simultaneous connections to support authentication across multiple browser tabs/windows of the same domain.

Proposed Solution

Ease this restriction by adding an option for the maximum number of simultaneous connections associatable with a single session.

Request message requires a payload

Request message requires a payload even though a payload shouldn't be obligatory. In some cases a request name is sufficient. For example if we just want to notify the server about an event like client.Request("event_a_happend", wwr.Payload{}) then we won't need a payload, but currently we have to provide a payload otherwise an error is returned indicating that the request message couldn't be parsed.

WEBWIRE_ERR: 2018/05/15 18:26:36 serverHttp.go:72: Failed parsing message: Invalid request message, too short

Request delivery order

Currently, requests performed one after the other will arrive at the server in serial order, though the order is broken when in the meanwhile the server is unavailable and the client is trying to reconnect before sending out accumulated requests. I suggest making the client retain the order the requests were initially made in to avoid unsynchronized delivery.

Graceful Shutdown

The webwire server should keep track of currently processed requests and signals and wait for them to finish before closing to avoid data loss and errors. During a requested shutdown new connections, requests and signals must not be accepted.

Auto-connect and auto-reconnect

Make the webwire client automatically connect to the server right after initialization and make it optionally automatically reconnect on connection loss

Add new constructor function with TLS support

Server feature request

Description

For use webwire on https we need to do a lot of action, as in example. It will be more convenient to have a constructor that will accept, in addition to the ServerImplementation and ServerOptions, a certificate with a key and listen https connections, like:

func NewSecureServer(
	implementation ServerImplementation,
	opts ServerOptions,
	certFile,
	keyFile string,
)

Support multiple underlying websocket connections

Client Feature Request

Proposed Behavior

The client should provide an option for multiple underlying connections to be used for potentially increased throughput and lower latencies in network environments with high packet loss. Messages should be distributed fan-out round-robin on the available connections.
Ideally, the number of underlying connections should be dynamically adjustable.

Actual Behavior

The webwire client currently utilizes just one single websocket connection. In network environments where packet loss is small having an HTTP/2-style multiplexed single TCP/IP connection (which the underlying WebSocket basically is) is ideal, though in network environments with high packet loss HTTP/1.1-style multiple individual TCP/IP connections should, in theory, provide better throughput and latency due to the TCP congestion-avoidance algorithm, which could potentially cause artificial slowdowns resulting in unsaturated network usage, at the cost of memory usage and higher connection establishment latency.

Proposed API

To be discussed

Proposed Implementation

To be discussed

Additional information

The server must also support this feature by providing an option of a maximum number of acceptable connections for a single client, though I'm not quite sure whether this is the case.

Use *log.Logger instead of io.Writer for logging

Use *log.Logger instead of io.Writer for logging, it allows the library user to set custom logging flags and prefixes. Currently this is not possible and the default prefixes are used like: WARNING: and ERROR: , though we'd like to differentiate between error logs produced by webwire and our own code.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.