scribery / meta Goto Github PK

PostgreSQL has recently added support for JSON storage and indexing.
@plaes asked if something like this could be used for the backend.

Investigate what's possible with this solution and what it could take, roughly, to use it.

See https://fosdem.org/2017/schedule/event/postgresql_infrastructure_monitoring/
and http://www.slideshare.net/StevenSimpson30/infrastructure-monitoring-with-postgres

Implement Fluentd normalization for tlog and aushape, as well as Elasticsearch mappings for the ViaQ project.

Latest instructions for setting up a test environment and hacking from @richm:

mkdir ~/viaq
cd ~/viaq
git clone https://github.com/ViaQ/docker-elasticsearch
git clone https://github.com/ViaQ/docker-fluentd
git clone https://github.com/ViaQ/integration-tests
cd integration-tests
./openshift-test.sh

You can add ES index templates to docker-elasticsearch/index_templates - the openshift-test will rebuild and redeploy ES with the new index templates

You can add fluentd inputs/filters either by hacking integration-tests/openshift-fluentd/fluent.conf, or by dropping in files matching openshift-fluentd/configs.d/openshift/input-pre-.conf, openshift-fluentd/configs.d/openshift/input-post-.conf, configs.d/openshift/filter-pre-.conf, configs.d/openshift/filter-post-.conf

The mappings should be contributed to https://github.com/ViaQ/elasticsearch-templates

Consider implementing playback of recorded sessions with OpenShift JavaScript terminal communicating with tlog-play in a container. See if the code from Cockpit integration attempt (#4) could be reused.

OpenShift seems to have a service for invoking a terminal to a container, we could hijack that, but also pass some parameters to tlog-play running there, specifying which session to play.

Kubernetes, which is the basis of OpenShift, already has audit logging for API calls, and an extension proposal is up.

We need to make sure we can correlate the tlog and aushape logs with Kubernetes API calls, so that it is possible to see the effect the recorded terminal session had on Kubernetes/OpenShift.

I sent a question to kubernetes-dev maillist and got a suggestion to pass any necessary identifying information in the bearer token, joining any authenticating information there. So far I presume that would require providing a wrapper to kubectl, and making sure that modified wrapper is used, instead of the basic kubectl, or just an HTTP client, such as cURL. I don't think the latter part is practical, but perhaps if we make our route easy, and users prefer it, then that will be good enough.

Contribute and agree on audit record/field description structure with the audit team. The file(s) should be usable for aushape code, schema, and Elasticsearch and ViaQ mapping generation. They should define which field for which record has which type of the raw value, "interpreted" value, and whether raw and "interpreted" value have different data.

This is needed for Scribery/aushape#4 and Scribery/aushape#58.

Tlog cannot be used to record superuser sessions securely. For those sessions
another approach should be used, such as jump server. In a jump-server setup
the user first logs in to a machine which records all the terminal I/O, and
then from that machine he/she logs in to the target machine.

This allows recording terminal I/O securely. Still, audit logs need to be
collected on the target machine, and will remain authentic only until the
point when the superuser decides to circumvent them. Even though it seems like
a partial solution, it is still useful.

The jump-server setup can be done by putting a sort of "captive portal" on the
jump server, only allowing the user to interact with a program selecting which
server to log in to. This approach is used by e.g.
https://github.com/aker-gateway/Aker.

Another approach can be similar to "virtual hosting" and NAT, where a single
jump server can have a number of IP addresses on the external interface,
mapped one-to-one to addresses of the protected servers, perhaps aided by
split DNS views. When users would log in to one of those external addresses,
the jump server would know which server the user wanted to reach and can start
recording and initiate further SSH connection to the protected server
automatically.

The challenge here is how to then link such session recordings to the session
on the target server, for both the terminal I/O and audit logs.

For specific user requirements and typical jump-server setups we can contact
@anazmy, the author of Anker, linked above.

Implement support for session recording and playback in Cockpit, for the
controlled machine.

For the start don't implement configuring session recording in Cockpit.
Leave that to users to do manually, then add that to SSSD configuration when
it is added to Cockpit.

Implement local session recording in tlog, so that actual recorded sessions
could be listed.

For example, consider recording to journal with extra meta-data, so that it is
then possible to list all recorded sessions and play them back. See
Scribery/tlog#88.

If that doesn't work, consider recording to per-session files in some
directory in the filesystem.

Implement an interface in Cockpit listing available recorded sessions, where
you could click on and play back a specific session.

Implement playback by running tlog-play on the local machine and showing its
output in a JavaScript terminal emulator, similarly to how it's done with
local console in Cockpit. Try to reach @dperpeet for that.

Also, support forwarding recordings somewhere else using the regular logging
channel. If not recording to journal, implement support for multiple writers
(Scribery/tlog#76).

This might also need support for rewind/fast-forward in tlog-play (see Scribery/tlog#90).

See if we can use a terminal emulator library in tlog, which would present a single terminal type to the user, while translating its protocol to the protocol of the terminal we run under. Then we'll be able to record the I/O in that single protocol the library uses and thus normalize the recorded data.

That in turn will greatly simplify playback, as we won't need to implement support for playback of all the terminal protocols users can use (although there aren't that many of them nowadays), or will use. That will also help us normalize searching and make it more predictable, plus allow for easier analysis of the recorded data.

On tlog-play side, we will also need to use that library to translate the recording to the protocol of the actually used terminal.

At the moment, at least on Fedora, when logging in via login on console or with su, tlog is supplied with ASCII as the character encoding to use and aborts (because it only supports UTF-8). At the same time ASCII is not the actual character encoding to use, and the configured encoding is in fact UTF-8. This is due to Fedora (at least) setting it in /etc/locale.conf, to be sourced by shells when they read startup files.

This ad-hoc solution is leaving other programs, which are not prepared to interpret shell language (or even just the limited language of /etc/locale.conf) in the cold. Instead something like the Debian approach should be used. Debian sets locale using pam_env, which works for any program.

Find a way to read Fedora (and RHEL) locale settings in tlog, and/or change the way the distributions do it.

Setup a blog with public updates on the project, so anyone could subscribe to it and see news.
Several people outside Red Hat had already asked for updates.

An important UI target is to have a view of related logs synchronized with terminal playback position.
That should be possible to do with native JavaScript playback.

However, with playback running on the host and displayed in a JavaScript terminal we will need an additional channel through which to communicate the playback location, so that we can query and display related logs.

Explore the possibility of implementing this with low effort, and only low-key design changes to tlog-play.

Idea is to create some small website with JS-player and ability to playback sessions recorded by tlog.
It might use Google Drive as a storage of user files, but we may just play uploaded file without storing it.
Should be done after implementation of random-positioning.

Talk to OpenShift and/or CloudForms teams regarding the possibility of integrating Session Recording and playback.

Find a way to build a Kibana dashboard listing currently active sessions, or just sessions, grouped by day, host, user. Find out if it's possible to add a session playback link to each session, pointing to some other service (e.g. #7). Also show session ID explicitly.

Consider implementing setup of rsyslog, tlog, and aushape using Ansible.