collegevine / confcrypt Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 3.0 128 KB

A sane tool for managing configuration in source control

License: BSD 3-Clause "New" or "Revised" License

Haskell 98.39% Shell 1.31% Dockerfile 0.30%

confcrypt's People

Contributors

Stargazers

Watchers

Forkers

jhenligne unisay alyssa-siegmann

confcrypt's Issues

Track test-coverage percentage

I'd like to keep track of confcrypt's test coverage percentage and fail tests if it drops below 80%.

Validate `--type Boolean`

I ran this command:

 confcrypt aws add config/Local/Grape.econf --key … --name ROLLBAR_MUTE --value Blah --type Boolean

Note: I wrote --value Blah and expected it to throw an error.

Use explicit imports in all files

Remove WriterT from the transformer stack

It isn't really necessary and obfuscates the API

Convert MonadError usages in Encryption into MonadThrow

Confcrypt.Encryption makes use of throwError from Control.Monad.Except, which can have unpredictable behavior when parallelizing calls. Instead, it makes sense to use the MonadThrow class and associated Control.Exception infrastructure because these natively support asynchronous exceptions.

Address All HLint errors

Improve error text

It appears that confcrypt currently creates text of errors by simply showing the error, resulting in messages like this:

FormatParseError "1:3:\n  |\n1 | %x\n  |   ^\nUnrecognized variable x\n"

This is what you get if you pass a bad template to read --format, I suspect others are worse since the text in the error is created via show rather than errorBundlePretty.

Add support for external key management systems

The current design of confcrypt assumes that users have a local RSA key. This is fine when working on small projects, but as @maxrothman pointed out in a separate conversation, a shared RSA key makes things like key rotation or usage auditing much more difficult.

To that end, confcrypt needs to be extended to support an alternative KMS-based encryption and decryption mechanism. My initial thoughts are that the encrypt and decrypt functions should move into separate typeclasses. The next step is introducing the KMS providers we'd like to support. Initially this will probably just be AWS, although I can see Azure or GCloud making an appearance at some point.

Run HLint before merging PRs

I'd like to gate new commits on a successful HLint run. Look into integrating this via Danger, as described in this blog article.

Remove --key requirement if --aws flag is set

KMS embeds a keyid into the ciphertext, so there's no reason to ask users to provide a key at the command line when using any decrypt functions.

Improve generality of command handling

The command handlers are split with separate instances (algorithms) for remote & local keys. The only real reason this happens is to unwrap local keys into their raw representation. This can be improved & the instances should be merged.

Parallelize Remote decryption calls

Right now confcrypt makes a string of synchronous calls to the remote encryption provider. Instead of iterating serially, we should iterate in parallel to reduce wait time for users.

Unable to store empty strings as values

This happens because AWS refuses to encrypt empty payloads:

 λ confcrypt aws add -n FOO -v "" -t String -p -k <key> <file>.econf                                                                                                      

confcrypt: ServiceError (ServiceError' {
  _serviceAbbrev = Abbrev "KMS",
  _serviceStatus = Status {statusCode = 400, statusMessage = "Bad Request"}, 
  _serviceHeaders = [("x-amzn-RequestId","e68c2e4a-791c-4920-97e2-f4778afa963a"),("Cache-Control","no-cache, no-store, must-revalidate, private"),("Expires","0"),("Pragma","no-cache"),("Date","Tue, 19 Jan 2021 17:11:04 GMT"),("Content-Type","application/x-amz-json-1.1"),("Content-Length","175")], 
  _serviceCode = ErrorCode "Validation", 
  _serviceMessage = Just (ErrorMessage "1 validation error detected: Value at 'plaintext' failed to satisfy constraint: Member must have length greater than or equal to 1"), 
  _serviceRequestId = Just (RequestId "e68c2e4a-791c-4920-97e2-f4778afa963a")
})

MacOS Big Sur (11.2.1) won't open app from unidentified developer

When trying to execute rake cv:config:load_development, Big Sur will pop up the following error message:

You need to navigate to System Preferences > Security & Privacy > General and manually whitelist the app.

Config file with duplicated keys passes validation

Setup:

config file with duplicate key/value pairs.
latest confcrypt version

Actual behavior:

confcrypt aws validate file.econf outputs no validation erros.
confcrypt aws edit ... removes duplicated keys.

Expected behavior:

confcrypt aws validate file.econf reports duplicated keys.
confcrypt aws edit ... validates file prior to editing.

Document options for `--type`, e.g. `String`, `Boolean`, `Int`

See title. Had to look it up in README.md.

Support encrypting an entire file.

This should follow the algorithm here https://docs.aws.amazon.com/kms/latest/APIReference/API_GenerateDataKey.html, but with some confcrypt conventions. The gist of it is:

Introduce a new command group large that supports read and encrypt
read should create an unencrypted file with the final .econf extension stripped
encrypt generates a new confcrypt file with an extra .econf extension added on
In an encrypted file, the encrypted key is stored in the KEY parameter, while the data is stored in FILE. This is a convention expected and enforced by confcrypt.

And there we go, that should provide whole-file encryption.

Add ability to update config in-place

Problem

When adding/updating a key in a confcrypt config, one always has to output the result into a separate file and then replace the original with the new output. This is because confcrypt doesn’t have the ability to update configuration in-place

Proposal

Add ability to update configuration in-place, e.g. using a --in-place option (see stylish-haskell).

Workaround

Another workaround using sponge from moreutils:

# Install `sponge`:
brew install moreutils

# Add config:
confcrypt aws add \
    --key $CONFCRYPT_KMS_KEY \
    --name API_BASE_URL \
    --type String \
    --value http://localhost:3000 \
    config/Local/Api.econf | sponge config/Local/Api.econf

Could not find module ‘Text.Megaparsec’

yura@yura:~/p/c/confcrypt|master✓
➤ stack clean
yura@yura:~/p/c/confcrypt|master✓
➤ stack build
Building all executables for `confcrypt' once. After a successful build of all of them, only specified executables will be rebuilt.
confcrypt-0.1.0.4: configure (lib + exe)
Configuring confcrypt-0.1.0.4...
confcrypt-0.1.0.4: build (lib + exe)
Preprocessing library for confcrypt-0.1.0.4..
Building library for confcrypt-0.1.0.4..
[1 of 8] Compiling ConfCrypt.Types  ( src/ConfCrypt/Types.hs, .stack-work/dist/x86_64-linux-tinfo6/Cabal-2.2.0.1/build/ConfCrypt/Types.o )
[2 of 8] Compiling ConfCrypt.Providers.AWS ( src/ConfCrypt/Providers/AWS.hs, .stack-work/dist/x86_64-linux-tinfo6/Cabal-2.2.0.1/build/ConfCrypt/Providers/AWS.o )
[3 of 8] Compiling ConfCrypt.Parser ( src/ConfCrypt/Parser.hs, .stack-work/dist/x86_64-linux-tinfo6/Cabal-2.2.0.1/build/ConfCrypt/Parser.o )

/home/yura/projects/collegevine/confcrypt/src/ConfCrypt/Parser.hs:10:1: error:
    Could not find module ‘Text.Megaparsec’
    There are files missing in the ‘megaparsec-7.0.3’ package,
    try running 'ghc-pkg check'.
    Use -v to see a list of the files searched for.
   |
10 | import Text.Megaparsec (Parsec, parse, getSourcePos, SourcePos(..), unPos, (<?>), try, failure, oneOf, anySingle)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

/home/yura/projects/collegevine/confcrypt/src/ConfCrypt/Parser.hs:11:1: error:
    Could not find module ‘Text.Megaparsec.Char’
    There are files missing in the ‘megaparsec-7.0.3’ package,
    try running 'ghc-pkg check'.
    Use -v to see a list of the files searched for.
   |
11 | import Text.Megaparsec.Char (char, space, eol, string', digitChar, alphaNumChar,
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...


--  While building package confcrypt-0.1.0.4 using:
      /home/yura/.stack/setup-exe-cache/x86_64-linux-tinfo6/Cabal-simple_mPHDZzAJ_2.2.0.1_ghc-8.4.3 --builddir=.stack-work/dist/x86_64-linux-tinfo6/Cabal-2.2.0.1 build lib:confcrypt exe:confcrypt --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1
yura@yura:~/p/c/confcrypt|master✓
➤ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
yura@yura:~/p/c/confcrypt|master✓
➤ uname -a
Linux yura 4.19.4-1-MANJARO #1 SMP PREEMPT Fri Nov 23 18:23:26 UTC 2018 x86_64 GNU/Linux

Run tests before merging PRs

Either Travis or CircleCI should work fine. We'll need to gate merging until tests pass.

`--type string` causes `Prelude.read: no parse`

Repro:

> confcrypt aws add --key "…" --type string --name FOO --value BAR config.econf

Solution:

> confcrypt aws add --key "$CONFCRYPT_KMS_KEY" --type String --name FOO --value bar app_config/dev.econf

-confcrypt aws add --key "$CONFCRYPT_KMS_KEY" --type string --name FOO --value bar app_config/dev.econf
+confcrypt aws add --key "$CONFCRYPT_KMS_KEY" --type String --name FOO --value bar app_config/dev.econf

Get Test coverage of expressions to 80%

From a local run of hpc:

confcrypt-0.1.0.0: Test suite confcrypt-test passed
Generating coverage report for confcrypt's test-suite "confcrypt-test"
 56% expressions used (460/811)
 25% boolean coverage (3/12)
       0% guards (0/8), 8 unevaluated
      66% 'if' conditions (2/3), 1 unevaluated
     100% qualifiers (1/1)
 53% alternatives used (39/73)
 71% local declarations used (27/38)
 30% top-level declarations used (38/126)

Write Brew package for confcrypt

Show help when calling `confcrypt` without commands

Calling confcrypt without arguments just gives me:

> confcrypt
Missing: COMMAND

Usage: confcrypt COMMAND

It’d be nice if it would output the full help:

> confcrypt --help
confcrypt: a tool for sane configuration management

Usage: confcrypt COMMAND

Available options:
  -h,--help                Show this help text

Available commands:
  add                      Add a new parameter to the configuration file. New
                           parameters are added to the end of the file.
  edit                     Modify an existing parameter in-place. This should
                           preserve a clean diff.
  delete                   Removes an existing parameter from the configuration.
  read                     Read in the provided config and decrypt it with the
                           key. Results are printed to StdOut.
  get                      Get a single parameter value from the configuration
                           file.
  validate                 Check that the configuration is self-consistent and
                           obeys the confcrypt rules.
  new                      Produce a new boilerplate confcrypt file. This should
                           be piped into your desired config.

confcrypt's documentation and source is avaiable at <TODO fill me in>

Add a version command

Support a default Key

While Confcrypt supports multiple keys per-file, often times we only use a single key for the entire file. In that situation, the steps of tracking down a key and including it on the command line are unnecessary friction.

I'd like to extend the existing file format and CLI with the concept of a default key. That should look something like this:

confcrypt aws new --name Test --type String --value foobar --use-default-key test.econf

Inside the file, you'd find:

file_default_key=<key path/id here>
... config values below

That new key, file_default_key should be optional, meaning previous concrypt files should all continue to decrypt.

Restore `Add` tests

When adding KMS support I commented out two of the tests for Add since they no longer work due to using true randomness rather than a DRG. Its still possible to run these within a DRG, so please make the necessary changes.