sensu / sensu-go Goto Github PK

• Create UI directory structure
• POC a page with a "working" API endpoint (i.e. checks)

The Sensu agent needs to accept Sensu events and telemetry data.

Initial ideas include:

• HTTP /events (with basic authentication or full RBAC?)
• TCP line protocol (utilize the websocket protocol? Do we need this?)
• Statsd TCP/UDP listeners (the only protocol that doesn't use the Sensu event structure)

depends on:

#8

• determine if an event indicates an incident or metric
• fork/exec a pipe handler command and write the JSON event data to child process STDIN
• capture and log pipe handler command STDOUT

• Conform to Daemon spec (agentd??)
• Move Agent Server to its own package

This issue serves the purpose of collecting/creating a list of our favourite UI technologies.

To add to the list, please comment on this issue.

We are fast approaching the time we MUST introduce some level of authentication and authorization (RBAC). RBAC applies to agent -> backend connections and every Sensu API query (e.g. CLI or dashboard).

We need to look at how others have implemented such capabilities, i.e. Hosted Chef platform.

Chef uses node environment and API resources as the basis for RBAC, we could do something similar with the addition of agent/check subscriptions. This would allow a Sensu backend cluster to be shared by several groups within an organization, allowing certain agents and users access to one or more environments and API resources.

• update unit tests that are Linux platform specific (e.g. write binary replacement for cat)

• fork/exec a process (posix spawn?)
• capture STDOUT/ERR and exit status code
• execution timeout
• write to STDIN

• Measure and track coverage
• Improve general coverage until we are comfortable
• Store "fake" (mix of fixtures and mock)
• Apply benchmark utility to another component

Using log.println all over the place is pretty crumby. Also, etcd has its own logger. We should standardize on structured logging and get everything logging in the same way.

• isIncident()
• containsMetrics()

• discus
• update https://github.com/sensu/engineering/wiki/Sensu-Events

depends on:

#6
#13

• store agent data from agent keepalives as sensu entities in etcd

The current MessageBus interface should be replaced as such:

// MessageBus is the interface to the internal messaging system. It must be
// safe to be used by multiple goroutines.
type MessageBus interface {
	daemon.Daemon

	// Subscribe allows a consumer to subscribe to a topic,
	// binding a read-only channel to the topic. Topic messages
	// are delivered to the channel as simple byte arrays. Consumers should
	// use type assertions to convert the incoming messages to a pointer to an
	// appropriate value.
	Subscribe(topic string, consumer string, channel chan<- interface{}) error

	// Unsubscribe allows a consumer to unsubscribe from a topic,
	// removing its read-only channel from the topic's bindings.
	// The channel is not closed, as it may still having other
	// topic bindings.
	Unsubscribe(topic string, consumer string) error

	// Publish sends a message to a topic. Publishers should be sure to pass by
	// reference as this is intended to be used by multiple goroutines.
	Publish(topic string, message interface{}) error
}```

• Run an embedded NSQ process within sensu-backend

• create basic cluster (not manage)

CLI/ENV argument/variable for peers.

depends on:

#6
#8

• store event data in etcd

• run etcd within backend service

The Sensu Go agent and backend services will support configuration via several means, by command arguments, environment variables, and a SINGLE configuration file. We have yet to declare what the configuration file format is.

Some possible formats:

• TOML
• JSON
• YAML

Me must keep in mind "custom" nested configuration options (attributes).

One we settle on a configuration file format, the Sensu Go services must be updated to support it (if necessary).

The Sensu CLI needs to "modes" of operation:

• Invoked as a single command with several arguments
• Interactive - prompting for user input
• Accept JSON via STDIN (with support for legacy configuration translation when needed)

Raw examples (ideas):

$sensu-cli ACTION THING

$sensu-cli create check -n foobar -i 60 -s webserver,api -c 'check-http.rb -u https://localhost/health'

$sensu-cli create check 
What is the name of the check?
foobar
...

$cat some.json | sensu-cli import

The CLI tool will eventually need to handle authentication and authorization. For now we can no-op.

The Sensu agent needs to discover some facts about its system and publish them in keepalives (tags).

Facts to send:

• hostname
• network interfaces (more than one address)
• os
• platform
• platform family
• platform version
• uptime

Could use a library like:

https://github.com/shirou/gopsutil

This work will lay the foundations for the traditional Sensu pipeline configuration.

• store.go mutators
• http_api.go mutators
• store.go handlers
• http_api.go handlers

See #40 for example implementation.

We may need to refactor http_api.go, splitting up/out the endpoints.

I want us to move to an Interface + Mock methodology for unit testing. So instead of having an in-memory fixture store that's incapable of easily simulating all interactions with a Store (i.e. simulating error cases is brittle and/or non-trivial), I'd like us to move back to the mock store that was there before + testify.mock behavior simulation in faked methods per test. (We can develop some reusable faked methods if this ends up being super copypasta).

The key being that the mock should simulate behavio

• create agent keepalive data
• agent keepalive interval timer
• agent keepalive publishing
• backend keepalive consumer

This issue serves the purpose of collecting/creating a list of our favourite user interfaces.

To add to the list, please comment on this issue.

• api post /checks
• backend check request interval timers
• backend check request publishing
• agent check request consumer
• agent check execution
• agent event publishing
• backend event consumer
• api get /events/:entity/:check ?

Right now we have several backend processes, and they all sort of conform to the same interface that looks a little like:

type Daemon interface {
    Start() error
    Stop() error
    Err() <-chan error
    Status() bool
}

Make things actually adhere to this interface.

depends on:

#10
#12
#25

• /checks/*
• /entities/*
• /events/*
• /incidents/*

We currently do not actually run a command beyond sh -c '' - https://github.com/sensu/sensu-go/blob/master/agent/check_handler.go#L28

Implement a Sensu logger package, e.g. logger/logger.go.

The logger package is a proxy/gate for Logrus (https://github.com/sirupsen/logrus).

backend/checker.go -> backend/schedulerd/schedulerd.go

pipelined and apid can be used for reference.

• Add test coverage for every router endpoint
• Conform to Daemon spec (apid)
• Split router endpoints up into separate files

• sensu-backend
• sensu-agent

Expose cli arguments and ENV variables.

backend.go is getting out of control.

• Refactor transport.Server
  • websocket listener moved into it
  • configuration for listener port, etc.
  • Start()/Err() pattern applied to it
  • Monitor Err() in backend Run()
• Error handling in backend.Run()
  • Fix not-often hit race on errChan
  • Separate channel for receiving errors from monitored processes / returning an error via Err()
  • Figure out how to prevent send to closed errChan that isn't tedious

The Sensu Type Validate() functions need to be improved, for example, they should use regular expressions to validate certain field string values (e.g. name).

The current Sensu Settings validators can be used as a reference. Sensu Go doesn't currently implement many features of Ruby Sensu and therefor doesn't have certain fields, they can be left out for the time being.

Sensu Settings check validator: https://github.com/sensu/sensu-settings/blob/master/lib/sensu/settings/validators/check.rb#L179

TODO

• Update types/* Validate() functions