cossacklabs / acra Goto Github PK

This is for the very distant future - a ticket to accumulate all suggestions for Web UI to control AcraServer. Each suggestion will turn into separate issue.

[ ] Enable WebUI via port knocking
[ ] MFA Authenticate WebUI via side channel (OTP delivery integration)

Port Acra request filtering via pre-configured rules file to open-source version.

Think about this closer to 1.0, talk to users.

If we do:

When triggered poison record rotation:

• acraserver generates new poison records
• user feeds select that triggers poison records
• acra reads output, replaces poison records with old keys with poison records with new keys, generates a dump of inserts.
• user receives sql commands and is able to run it / server runs it for user?

By default create keys for acraproxy and acraserver but add option for generating one of.

Some service queries like "ROLLBACK" (used in transactions) and regular expressions in PostgreSQL can't be correctly handled by sql parser that we use in censor.

We use https://github.com/xwb1989/sqlparser that is not a best choice for PostgreSQL protocol.

The list of supported databases is impressive. Is Microsoft SQL Server support planned? As well as the Azure SQL Server service?

• Move all CLI arguments to conf file
• Separate conf file for every piece of Acra
• Scan source code for CLI arguments that were not documented

--wholecell: consider acrastruct to occupy exactly one cell (default case, higher peformance)
--injectedcell: consider acrastruct to occupy any sub-sequence of byte stream (non-default case, use case: hiding the acrastruct in other data, including steganography and other weird stuff)

move log writers from cmd/acraserver to cmd/utils and use this general writers in all cmd utils for filtering logs output

[ ] Check every mark "Check/verify"
[ ] Check and revise key paths everywhere
[ ] Check zone stuff for consistency, duplicate into separate section
[ ] Check naming consistency everywhere
[ ] Throwaway keys > temp_writer / Temporary Writer keys
[ ] Zone ID is a context in SCell encryption, change

Note in docs that acraserver work only without ssl on postgresql side. Now we have just one note in https://github.com/cossacklabs/acra/wiki/AcraProxy-and-AcraWriter , but it's important for acraserver too.

Draft a table w/ Go version / OS / Architecture compatibility

Working without zones is actually working with a default zone. We might want to unify the terminology at some point, and streamline the process on AcraServer side.

acraproxy authentication via cached token

https://www.postgresql.org/docs/9.1/static/auth-methods.html

On all default debian machines apt get install golang 1.3

Hello, I'm trying to build Acra from sources but I'm failing at the first step of the guide which is to install acra-keymaker.

This is the error I get when I run go get github.com/cossacklabs/acra/cmd/acra-keymaker

ubuntu@ip-10-1-1-1:~/work/src/go.opencensus.io$ go get github.com/cossacklabs/acra/cmd/acra-keymaker
package go.opencensus.io/exporter/jaeger: cannot find package "go.opencensus.io/exporter/jaeger" in any of:
        /usr/lib/go-1.10/src/go.opencensus.io/exporter/jaeger (from $GOROOT)
        /home/ubuntu/work/src/go.opencensus.io/exporter/jaeger (from $GOPATH)

This got resolved when I downgraded go.opencensus.io/ from latest to v0.19.x because the package jaeger is moved to contrib.go.opencensus.io

For the record in the wrappers - ruby has been updated to use the new begin tag but both nodejs and python are like php … they use the old one.

-s --shutdown
-p --poison

etc.

// Copyright 2016, Cossack Labs Limited
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

... and commit Apache 2 license to root folder.

acraserver help, show -- for long, - for short.

pieces off #51

because it's required param, we can check for it ourselves and automatically add if user didn't pass him

Explain about key naming for acraproxy and acraserver (keys should be with "_server" prefix) in docs

• management web console
• simplified connection mode
• integrated connection mode
• move acrastruct handling to Themis
• mongodb? (based on Hermes-base experience)

Closer to ending existing line of problems, allow users to use Zones as source of Random keys.

• generateZonePool of size N for application XYZ
• getRandomZoneFromPool for application XYZ
• getZoneFromPool for application XYZ where zone_id is “ABC”

• prior to release, run through goreporto and add goreport card to readme

Interface, that allows to configure (generate configuration, attach storage) and fire up a new instance.

Deleting .acrakeys on server side does not affect server's behavior (keys are cached in memory), so we have to provide a way to flush caches.

For initial release documentation, I'd like us to have a final performance benchmark:

Measure reads:

• write very simple client app
• deploy Acra, first with one key, for 10000 random-length records:
  • measure speed of 100000 requests with decryption through Acra.
  • measure speed of 100000 requests passing through acra without decryption (no acrastruct)
• implement zone keys in test app, have 10000 records with 100 zone keys:
  • measure speed of 100000 requests with decryption through Acra
  • measure speed of 100000 requests without acrastruct
• point client to database directly, bypass acra/acraproxy, measure clean reads.

Measure writes:

• in client app, use generate 10000 acrastructs with random payload and single acra key, write to db, measure speed
• in client app, generate 10000 acrastructs with 100 different zone keys, write to db, measure speed
• in client app, write 10000 plain writes with random payload

For test purposes, use full environment:

• 1 machine w/ client code, AcraProxy, AcraWriter
• 1 machine with AcraServer
• 1 machine w/ PgSQL

replace all names to new
now we have warning of usage old names

now key pair for poison records has filename poison_key<.pub>. Now possible intersection of client's key names and poison record.
simple way is placing poison keypair into subdirectory

Light: SSL (instead of SS) and lighter integration.
Hardcore: Deeper integration into app runtime and secret protection.

As we will need this is Hermes's ecosystem at some point, it's wise to study process authentication for platforms which enable it well.

$subj :)

Hello,

I have got acra-connector and acra-server working and they're working as intended and encrypt and new entry for the columns that I choose.

I now want to convert the existing records of the table to ciphertext.

What's the simplest way I can convert my database with plaintext records to have ciphertext columns

acraserver@acra-playground-02:~$ bin/acraserver --db_host=37.139.9.167 --db_port=55432 -v
\2017/02/15 11:28:32 Info: start listening 0.0.0.0:9393

2017/02/15 11:28:37 Info: new connection: 176.37.32.190:38026
fatal error: runtime: out of memory

runtime stack:
runtime.throw(0x964120, 0x16)
/usr/local/go/src/runtime/panic.go:530 +0x90
runtime.sysMap(0xc820200000, 0xfe000000, 0x0, 0xd84798)
/usr/local/go/src/runtime/mem_linux.go:206 +0x9b
runtime.(*mheap).sysAlloc(0xd6aae0, 0xfe000000, 0x0)
/usr/local/go/src/runtime/malloc.go:429 +0x191
runtime.(*mheap).grow(0xd6aae0, 0x7f000, 0x0)
/usr/local/go/src/runtime/mheap.go:651 +0x63
runtime.(*mheap).allocSpanLocked(0xd6aae0, 0x7f000, 0x419d01)
/usr/local/go/src/runtime/mheap.go:553 +0x4f6
runtime.(*mheap).alloc_m(0xd6aae0, 0x7f000, 0x100000000, 0xc820122000)
/usr/local/go/src/runtime/mheap.go:437 +0x119
runtime.(*mheap).alloc.func1()
/usr/local/go/src/runtime/mheap.go:502 +0x41
runtime.systemstack(0x7f3d863f3de8)
/usr/local/go/src/runtime/asm_amd64.s:307 +0xab
runtime.(*mheap).alloc(0xd6aae0, 0x7f000, 0x10100000000, 0xc820016000)
/usr/local/go/src/runtime/mheap.go:503 +0x63
runtime.largeAlloc(0xfdfff4ff, 0x1, 0x45e980)
/usr/local/go/src/runtime/malloc.go:766 +0xb3
runtime.mallocgc.func3()
/usr/local/go/src/runtime/malloc.go:664 +0x33
runtime.systemstack(0xc820016000)
/usr/local/go/src/runtime/asm_amd64.s:291 +0x79
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1048

goroutine 18 [running]:
runtime.systemstack_switch()
/usr/local/go/src/runtime/asm_amd64.s:245 fp=0xc82002eb18 sp=0xc82002eb10
runtime.mallocgc(0xfdfff4ff, 0x7ba2c0, 0x1, 0x0)
/usr/local/go/src/runtime/malloc.go:665 +0x9eb fp=0xc82002ebf0 sp=0xc82002eb18
runtime.newarray(0x7ba2c0, 0xfdfff4ff, 0xc820120030)
/usr/local/go/src/runtime/malloc.go:798 +0xc9 fp=0xc82002ec30 sp=0xc82002ebf0
runtime.makeslice(0x7a9120, 0xfdfff4ff, 0xfdfff4ff, 0x0, 0x0, 0x0)
/usr/local/go/src/runtime/slice.go:32 +0x165 fp=0xc82002ec80 sp=0xc82002ec30
github.com/cossacklabs/acra/utils.ReadData(0x7f3d885310d8, 0xc820118000, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/acraserver/src/github.com/cossacklabs/acra/utils/utils.go:73 +0x1b1 fp=0xc82002ed28 sp=0xc82002ec80
main.(*SServer).initSSession(0xc8200c6960, 0x7f3d88531000, 0xc820118000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/acraserver/src/github.com/cossacklabs/acra/cmd/acraserver/listener.go:52 +0x92 fp=0xc82002ee78 sp=0xc82002ed28
main.(*SServer).handleConnection(0xc8200c6960, 0x7f3d88531000, 0xc820118000)
/home/acraserver/src/github.com/cossacklabs/acra/cmd/acraserver/listener.go:129 +0x139 fp=0xc82002ef98 sp=0xc82002ee78
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82002efa0 sp=0xc82002ef98
created by main.(*SServer).Start
/home/acraserver/src/github.com/cossacklabs/acra/cmd/acraserver/listener.go:159 +0x8da

goroutine 1 [IO wait]:
net.runtime_pollWait(0x7f3d88576b70, 0x72, 0x0)
/usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc82004f2c0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82004f2c0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).accept(0xc82004f260, 0x0, 0x7f3d88576c30, 0xc82011c040)
/usr/local/go/src/net/fd_unix.go:426 +0x27c
net.(*TCPListener).AcceptTCP(0xc820022068, 0xc820043b10, 0x0, 0x0)
/usr/local/go/src/net/tcpsock_posix.go:254 +0x4d
net.(*TCPListener).Accept(0xc820022068, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/tcpsock_posix.go:264 +0x3d
main.(*SServer).Start(0xc8200c6960)
/home/acraserver/src/github.com/cossacklabs/acra/cmd/acraserver/listener.go:153 +0x523
main.main()
/home/acraserver/src/github.com/cossacklabs/acra/cmd/acraserver/acraserver.go:132 +0xc8d

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1
acraserver@acra-playground-02:~$ \

describe acra_rollback script

Storage keys: acra_genkeys parameter: --storage; key name "someid_storage"
Client, server - leave intact

Don't check for poison records if no reaction is specified

issues:

In the documentation, this is mentioned :

Transparent encryption proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code.

However, There is not possibility for my not editable application to work with this proxy as :

Encrypted data is binary data. As AcraServer doesn't know the nature of data, it returns the decrypted binary data to the web app. You’ll need to change the source code of your web app for the app to expect the decrypted data as binary, then to encode it into the original format (strings, numbers, bytes, etc.).

Expected behaviour :

As a transparent proxy user , I expect to have to define the database field type either in the acra server/connector configuration and in the database (change encrypted field into binary field) but not in the application code.

Regards,

• Make install-build-run and quick-start one document
• Re-explain poison records, based on new changes
• Describe acra_rollback process
• Explain zone/default zone concept
• Outline Acra affecting PostgreSQL security measures
• Decscribe new keypair scheme
• Explain AcraWriter as a part of client code, elaborate on examples (integrating into ORM)
• Guidance regading app<>proxy, server <> db interfaces
• Enforce Client encrypts - server decrypts formulation everywhere
• Explain threat model and attack vectors (key stores are crown jewels), explain key db
• Trying Docker - clarify point on logging
• Integration tutorials - add at least some text of what's going on
• Installing AcraWriter
• Key management into separate document and describe paths for key storage

Cosmetic changes:

• Rewrite Wiki home
• Add availbility table
• Add table with links to all examples
• When you don't need Acra section in Wiki