caiyeon / goldfish Goto Github PK

Goldfish strictly uses approle to launch, but there are some unsafe defaults that are currently used.
Ideally, http should not be allowed, and cookies should have an https only flag turned on.
For this to work, an override (perhaps a dev flag) should be in place as well.

Also, currently, goldfish relies on having a system with swap disabled, but does not check for this. Instead, it would be ideal if it used memlock the same way vault does (and refuse to launch when it is unable to achieve mlock)

In terms of enabling https, the option of retrieving a certificate from vault should be explored.

If solving this problem involves too many new command line arguments, a settings file (e.g. HCL) should be also made possible.

ISSUE TYPE

• Bug Report

COMPONENT NAME

Secrets Backend

GOLDFISH VERSION

v0.3.3

OS / ENVIRONMENT

CentOS 7.3

SUMMARY

When removing the last Key from a Secret goldfish won't let me save it.

STEPS TO REPRODUCE

1. in the secrets path click Add Secret Name it 'Blah'
2. add a key: 'foo' value: 'bar'
3. Save it
4. Now try to remove 'foo' from the Secret 'Blah'

EXPECTED RESULTS

This should succeed and the Secret should be removed since no keys remain

ACTUAL RESULTS

Error: 400
Vault: missing data fields

A docker image is in construction https://github.com/Caiyeon/goldfish/blob/master/docker/Dockerfile
In order for this to work, it must be stacked with a vault version 0.7.0 (0.6.x will fail)
Once HashiCorp releases their official docker image for the new version, a stack with goldfish and vault will be released.

If a user searches for policies that match foo/bar, policies that allow access to foo/* should match. This should be implemented entirely in javascript on the client side

rbastic@marvin:~/go/src/github.com/caiyeon/goldfish$ goldfish -dev -vault_token $(vault write -f -wrap-ttl=20m \

-format=json auth/approle/role/goldfish/secret-id
| jq -r .wrap_info.token)
flag provided but not defined: -dev
Usage of goldfish:
-addr string
Vault address (default "http://127.0.0.1:8200")
-approle_path string
The login path of the mount e.g. 'auth/approle/login' (default "auth/approle/login")
-config_path string
The vault path containing goldfish config data e.g. 'secret/goldfish' (default "data/goldfish")
-role_id string
The role_id the secret_id was generated from (default "goldfish")
-token string
Wrapping token that should contain a secret_id

I have followed the instructions completely. Looking forward to getting to try this out. I'm a Vault newbie, so help is appreciated!

Vault offers a convenient API for finding if a client token has capabilities on a given path. This should be implemented in the wrapper, and the handlers can use it to determine what kind of response to give back to the frontend.

One particularly useful instance is when the user opens up Token Creator, and the page tries to load a list of roles. If the user does not have capabilities to show roles, there should be a message saying so. If there are no roles, nothing should show up. At the moment, no roles vs no access are treated the same way: they both get a 403 error.

In token creator, if I create a token with 5h ttl and 5s max_ttl, the token that is created will be only 5s ttl, because ttl can never exceed max_ttl.

This is working as intended, but if we're going to make the token creation process smooth, we might as well take advantage of displaying warning signs when settings are in conflict.

There should be a warning message displayed on the right half side when ttl > max_ttl

I'm trying to start goldfish and pass the cert and key for TLS on the CLI, but it fails with the error flag provided but not defined: -cert_file. I can clearly see the cert_file flag defined in server.go. These appear to be the flags it has defined:

Usage of goldfish:
  -approle_path string
        The approle mount's login path (default "auth/approle/login")
  -config_path string
        A generic backend endpoint to store run-time settings. E.g. 'secret/goldfish'
  -dev
        Set to true to save time in development. DO NOT SET TO TRUE IN PRODUCTION!!
  -role_id string
        The approle role_id (default "goldfish")
  -vault_addr string
        Vault address (default "http://127.0.0.1:8200")
  -vault_token string
        The approle secret_id (must be in the form of a wrapping token)

Secrets should be delete-able in the UI

In server.go there's a csrf.Secure(false) parameter that states whether gorilla csrf package should accept http/https or https only.

For development purposes, there's no reason to use csrf.Secure(true)
However, going by gorilla csrf package's philosophy of sane defaults, it would be nice to have a dev mode that sets a flag determining whether secure is true or false (amongst other secure default variables that I may be missing)

The current way to launch the server is to use server -addr {addr} -token {token}
The ideal way to launch the server (in development) would be server -dev -addr {addr} -token {token}

This should probably be put in an init() function in server.go

The script portion of this secrets/index.vue page is fairly messy, and could use some refactoring before it gets out of hand.

Config options are becoming quite verbose and they'll only grow to be more so.
It would be nice to follow vault's convention of allowing both cmd args and also a file to describe the settings.

At the moment, TLS config for vault client is set to default, and therefore vault's certificates must be imported into the system's certificate bundle in order for goldfish to communicate with vault.

Allowing for a tls_skip_verify option would allow for self-signed vaults to be goldfish'd more easily

After running npm run build, the fonts are generated under public/assets/fonts
However, when the webpage is served, the server tries to serve fonts from public/assets/css/assets/fonts

Changing frontend/build/webpack.base.conf.js query.name for font files will result in a change in both.
i.e. changing query.name from utils.assetsPath('fonts/[name].[hash:7].[ext]') to utils.assetsPath('css/[name].[hash:7].[ext]') will result in fonts generated at public/assets/css/fonts and referenced at public/assets/css/assets/css, and does not solve the problem.

It would be nice if it is possible to implement scanning of policy changes given GitHub access. Significant planning would be needed for this feature.

I want to put goldfish behind a load balancer (haproxy). HAProxy already has the necessary TLS certs. Can you please support running goldfish in HTTP in prod mode?

Is this dependant on the SAML-LDAP integration?

If not, it's sorely missed :D

Moved up to 0.4.0 and I'm now getting this everytime I launch goldfish be it via systemd or straight command line.

panic: Failed to unwrap provided token, revoke it if possible

goroutine 1 [running]:
main.main()
	/Users/tony/work/src/github.com/caiyeon/goldfish/server.go:77 +0x1dd

config.hcl contents:

listener "tcp" {
  address       = "url"
  tls_cert_file = "/etc/letsencrypt/live/cert_path/cert.pem"
  tls_key_file  = "/etc/letsencrypt/live/key_path/privkey.pem"
}
vault {
  address       = "vault_url:8200"
}

At the moment, policy change requests live indefinitely in the cubbyhole of goldfish's vault token. Policy changes are considered invalid if the 'snapshot' policy at the time of request creation does not match the current living policy according to vault, and invalid policy changes cannot be viewed or approved. While this is secure, it's not exactly ideal to have a bunch of artifacts lying around in the cubbyhole storage (even if they take minimal space).

There are two possible ways to solve this. One being a garbage-collector go routine that periodically marks cubbyhole secrets to be purged, or change the storage location from cubbyhole to time-limited wrapped responses.

The backend go files should have a corresponding test file.
Vault's test core package can be used to instantiate vault in dev mode for testing purposes.

Goldfish's state is entirely stored and dependent on vault, and so a renew for the token was never required (a redeploy worked just fine). But for long term usage, it should periodically renew its own token.

Deployment process (including the one in vagrant) includes some unnecessary fuzz. Getting rid of these details will make it easier to deploy and easier for new users to understand how to use the software. In particular, remounting secret to data was probably an idea that my past-midnight self decided was wonderful, but is in fact useless.

Hi,

We've got a bunch of policy paths namespaced by slash

These don't work, all returning 404 but any other path seems to work fine

(Vault version is 0.6.4 if that helps?)

Transit tool page's age is showing. It's simplistic, and doesn't allow as much freedom as it should. In particular, it's only able to use one transit key, which is predefined in the run-time settings.

Preferably, the run-time settings should provide a default key, while manually overriding the key should be also possible.

Leaving the requester accessor hash as is (sha256) instead of hmac: 6c39ba8

Your demo looks amazing! This is the tool I've decided to use for an interface to vault for anyone afraid of the CLI. However I'm having some issues with it giving me 404 or {"message": "Not Found"}

So I'm running it out of a user I created for this goldfish...I've got it as a systemd service and it starts up, but everytime I try to hit goldfish from the outside I get

404 file not found

if I try to curl localhost from the host of course I get
{"message":"Not Found"}

Any ideas here?

I'm running this in AWS on ubuntu 16.04 with go 1.7

Some npm packages are no longer needed. They should be cleaned up.

Interestingly, the select element is only squished in npm run build but not in npm run dev

Worth looking into later. For now, it does absolutely no harm, especially since it actually seems to size according to the text. It just ignores the angle icon.

It would be nice to be able to provide a page to users who may have to occasionally use vault, but not often enough to have to install vault cli or remember the specific rest API calls.

This page should be able to abstract away some of the vault technicalities. For operators that want to describe a single task to users who rarely use vault, screenshots of this page should be much easier to understand than rest calls.

A tools page should contain (at least) 3 basic operations:

Reading
Writing
Listing

For this tool to be useful, it should be easier to use than vault CLI.
In particular, reactive computed properties of vuejs should provide an instant feedback as to what the payload is (see Token Creator page).

My vault addr is HTTPS but with a self-signed certificate. When I start goldfish pointing to this vault, I see:

goldfish.default(O): panic: runtime error: invalid memory address or nil pointer dereference
goldfish.default(O): [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x756367]
goldfish.default(O):
goldfish.default(O): goroutine 1 [running]:
goldfish.default(O): github.com/caiyeon/goldfish/vault.StartGoldfishWrapper(0x7fff781dd9bb, 0x24, 0x7fff781dda0a, 0x8, 0x7fff781dd9ee, 0x12, 0x3f, 0x922a40)
goldfish.default(O): 	/hab/cache/src/go/src/github.com/caiyeon/goldfish/vault/vault.go:70 +0x237
goldfish.default(O): main.init.1()
goldfish.default(O): 	/hab/cache/src/go/src/goldfish/server.go:52 +0x43e
goldfish.default(O): main.init()
goldfish.default(O): 	/hab/cache/src/go/src/goldfish/server.go:146 +0x76

When I run vault CLI, I pass in the VAULT_SKIP_VERIFY environment variable. Can goldfish support this as well so it doesn't panic?

We have a Cluster consisting of multiple Vault instances for fault-tolerance.
It seems like goldfish can only use one Vault (I cannot see any documentation saying otherwise).
It would be useful if a user could configure multiple Vault addresses so there is either a dropdown to select the instance, or Goldfish could automatically connect to the first available Vault.

It would be nice to have a wrap and unwrap tool page, where arbitrary key:value maps can be stored, and ensured that they can be only read once.

Similar to how policy changes are based on request and approvals, mount tuning should behave the same way.

Also, whether or not new mounts, remounts, or deleting mounts should also have a similar process.

It would be awesome if goldfish supported an LDAP login method.

Generally, production deployments should be as stable as possible. Therefore, they should be done using the executable and zip file from the release section, rather than a go get and compile command.

For the users that want cutting edge features, at the cost of potential unstability, compiling instructions should be added at the end of the wiki page with a big warning sign.

Note that unstable does not imply insecure. Goldfish should be secure at all times. Backend wrapping functions should be altered very carefully. Unstable may just mean that a particular frontend component is not working as expected.

First, thank you for Goldfish! The live demo looks awesome and the project seems very well run.

Primary issue is that I ran through the deployment instructions and everything appeared to succeed until I got to the last line:

./goldfish \
-goldfish_addr ":443" -cert_file /path/to/certificate.crt -key_file /path/to/privatekey.pem \
-vault_addr $VAULT_ADDR -vault_token $VAULT_TOKEN \
-approle_path auth/approle/login -role_id goldfish \
-config_path secret/goldfish

I get the error: Failed to read config secret from vault

I'm able to read/write secrets from the vault CLI. The vault is unsealed, etc.

Some suggestions for the deployment docs (and possibly things that I messed up because I deviated from the docs)

1. In preparing the vault, the instructions assume that the goldfish source is local. This line: vault policy-write goldfish $GOPATH/src/github.com/caiyeon/goldfish/vagrant/policies/goldfish.hcl. In my case, I just copied that single file from github to a local folder on my machine.

2. It wasn't clear from the instructions (and to a goldfish newbie) that the key that's output from step 2 is the input for step 3 ($VAULT_TOKEN). Clarifying that, if it's correct, would be awesome.

3. Goldfish version in step 3 is v0.3.0. When I got the error, I tried bumping this to v0.3.3. I still got the same error, but I'm guessing you want to bump the version number in the deployment instructions.

Thanks for any direction you can provide. Apologies this is an issue instead of a PR :)

The next major feature on the roadmap is to provide a way to check the dependency of a particular vault resource.

This functionality needs to be able to answer at least the following questions:

1. If I remove this policy, will any current tokens/users/approle/etc. be affected?
2. If I remove this secret path, will anyone be affected?
3. Is this backend still being used by any policies?

The likely scenario is that the user will choose from a list of types of resources, which will expand over time, and provide an instance of that resource to check.

There are now enough features that documenting how they can be done with screenshots is a necessity.

• Adding a policy change request
• Slack integration
• Github integration with mass policy changes from a github commit

Self-explanatory. Major and suitable minor version releases should come with a packed up static folder and pre-compiled binaries of goldfish

Following vault and many other softwares' standard, having an option to display just the version and exit would be good to keep track of things.

It's undecided whether certificates should remain in the Administration -> Users page (for obvious reasons: it's not a user).

Some things that need to be done:
Map out the details that a certificate management should have, in the eyes of a user
Design the page
Design backend wrapper & handlers

With the database plugin system, one backend can handle multiple connections, and the mapping of allowed roles to connections can get a little spaghetti. A dozen databases with a couple of dozen roles can seem very cryptic, even with vault's API.

It would be nice to have a page that is able to fetch this information and display it in an intuitive fashion.

A couple of problems that would have be considered are:

How to handle multiple database backends

How to fetch this data at scale. Loading every connection config upon visiting the page would be overkill, if it can be avoided.

When a policy change request is placed, a hash (which is also the change id) is returned. This hash is safe to present to anyone, because read privileges on the policy is required in order for the hash to return anything in vault.

Therefore, to speed up the process of delivering the hash to administrators, a slack webhook setting (hot-reloadable) should be considered. When a new change is requested, the hash can be sent to the webhook (and a specified channel). Administrators can then open up goldfish, login, and pull up the change details via providing the hash.

The error logging format needs a revamp.

Firstly, if the goldfish system needs to be locked down and no one can access the system, then there's no point of a stdout error logging mechanism. A switch to syslog or some other kind would be required.

Secondly, goldfish treats a lot of errors from vault (most of which are 403) as 500, and although they don't conflict with usability, it may be confusing to the layman as to why a 500 response is returned, for example, when the user isn't logged in.

A proper runbook and architecture layout would be helpful for skeptical operations personnel trying to use goldfish.

Each major feature should have proper documentation on what is going on behind the scenes. In particular, the policy change request process may be unnerving because it contains the usage of unseal tokens, and justifications for why the process is secure needs to be formally presented.

Short term:

• Add bulletin secrets
• Add users to userpass auth
• Add pki & certificates

Long term, it would be nice to have an interpreter that takes a bash script with vault commands, and execute them programatically. That way, development environments can be transferred via a simple script.

There should be a metadata field to token creator's JSON payload.

For some reason, a plain json object is unable to be bound by Echo as a map property in Vault API's TokenCreateRequest struct.

Worst case, it would have to be handled manually. But there should be an elegant solution.

I run this to get my wrapping token:

vault write -f -wrap-ttl=20m -format=json auth/approle/role/goldfish/secret-id

I take the value and plug it in here as the -vault_token arg:

goldfish -goldfish_addr=https://0.0.0.0:443 -cert_file=/hab/svc/goldfish/data/cert.pem -key_file=/hab/svc/goldfish/data/key.pem -vault_addr=http://192.168.56.100:8200 -vault_token=fa9e53ae-63f2-6ebc-aac5-9603fc970f4e -approle_path=auth/approle/login -role_id=goldfish -config_path=secret/goldfish

What results is this 400 error:

* wrapping token is not valid or does not exist

How should I troubleshoot this? All of the preliminary vault setup for the approle has already been set up.