runtheops / ssm-diff Goto Github PK

In parameter store if you have a numerical only value for a parameter and you ssm-diff init that parameter the following error occurs.

'int' object has no attribute 'splitlines'

key: /some/key
value: 123456

It will work if it is wrapped in quotes in parameter store or changed from a non numerical only value but that limits the values we can use if we want to us this tool.

A final suggestion is to change the top-level package name to something that's more meaningful (e.g. ssm_diff). A programmatic user currently imports from states which isn't necessarily obvious. The diff on a PR would be useless so don't forget to update setup.py if you go this route. Perhaps something like ssm_to_yaml could be clearer (and more Google-friendly).

There should be a flag for apply command (called --merge), or a new command (called 'merge'), to allow you to merge local changes with the existing remote configuration, without deleting remote parameters which don't exist locally.
So if you have a parameter that has changed value locally, it will change to remote as well.
But if there's a remote parameter which you don't have locally, it should not delete from remote.

I think the commands would be more intuitive if they mirrored git/docker e.g. (edit: add/tweaked mappings):

• init = clone (or fetch)
• pull = pull
• plan = status (diff only shows unstaged changes so it's not the same)
• apply = push

Certainly alias them for backwards compatibility, but consider deprecating the old names.

I've been digging through the implementation and it looks like the force flag in pull only supports two modes:

• Keep all local value and add any new values from SSM (don't force)
• Overwrite all local values with all remote values, implicitly preserving added values (force)

It would be nice to be able to provide more granular concurrency options. For example, consider the following three cases:

1. Key1 changed locally but not on SSM
2. Key1 changed locally and was changed in SSM
3. Key1 only changed on SSM

In git, the pull behavior is more like:

1. Preserves this value
2. Identifies the conflict and throws an exception (unless you prefer-local or prefer-remote)
3. Update this value

It'd be nice to support something similar.

It seems to me that the following will result in the deletion of many keys:

• ssm-diff init -p "a/b/c"
• ssm-diff plan -p "a/b/c"
• ssm-diff apply

It would be easy to forget to append the -p flag. If this would result in massive deletions, it seems like a very dangerous design. I suggest moving the -p behavior into an ENV variable. This ensures that it must be deliberately altered, not merely forgotten.

Expected Output:

• Something like 'ssm repo initialized'

Actual Output:

• 'int' object has no attribute 'splitlines'

Python Version: Python 2.7.15rc1
OS: Ubuntu Linux 18.04 upgraded from 16.04

If you need any further information, please let me know.
I'm not into python development, so i don't know what and how to provide.

Unless absolutely necessary, I'd rather not put decrypted secrets on a local machine. SSM also logs access to secrets so I'd rather not leave an unnecessary trail of secrets logs. I can think of two ways to handle this:

• Give an option to not decrypt secrets
• Give an option to skip encrypted parameters entirely

Both could make sense, but the second option solves both problems (on-disk and audit logs). I think it should be an ENV variable (vs. a flag) so you don't accidentally delete the encrypted params if you forget to include the flag when you apply.

Our legacy YAML files include inline documentation, e.g.

client:
  # The client name is:
  #  - exposed to the algorithm/process engine through the ENV directive for selecting configurations
  #  - used to set Django's sites value
  #  - used to configure the Mirth server name
  name: demo
  # Release determines which "tags" may be uploaded to the machine.  Accepted values are `dev`, `alpha`, `beta`, and
  # `production`. For example, a production system will not accept a tag like `1.x.x-dev`.
  release: beta

When extracting configurations from SSM, it'd be great to be able to preserve/leverage similar documentation. I'm not sure if any of the YAML parsers can preserve comments, but it would be nice if I could select a YAML file as a "template" and have it overwrite (or append) key-value pairs, preserving whatever documentation already exist.

Obviously, this would need to be combined with path-specific exports or adding new clients/services would be a mess. There would definitely be some edge cases (e.g. whether to repeat templates in lists), but it'd be useful to address the simplest/most general case and refine as-needed.

It appears that when you define an integer in SSM like

DEBUG_PORT: 80

This breaks ssm-diffs ability to initialize. Example of error output (I also added some print statements in to see where exactly the code was breaking).

NODE_ENV': production, u'TESTS_RUNNING': false, u'DEBUG_PORT': Traceback (most recent call last):
  File "/usr/local/bin/ssm-diff", line 80, in <module>
    args.func(args)
  File "/usr/local/bin/ssm-diff", line 9, in init
    print(r.get(flat=False, paths=args.path))
TypeError: __repr__ returned non-string (type int)

This essentially means that it is not possible to put any integer values into SSM. I can take a stab at fixing this but wanted to raise it as an issue first in the event you guys had a quick/easy fix.

**It fails if user has access to specific path because ssm-diff tries to DescribeParameters on ***

Handling parameters description would be a nice touch, maybe via YAML comments.
What do you think ?

I pip installed the client to a python 3.8 virtual environment, and ran the initial ssm-diff init command, but encountered a type error:

Traceback (most recent call last):
  File "/Users/roberttownley/.pyenv/versions/demo/bin/ssm-diff", line 82, in <module>
    args.func(args)
  File "/Users/roberttownley/.pyenv/versions/demo/bin/ssm-diff", line 11, in init
    l.save(r.get(flat=False, paths=args.path))
  File "/Users/roberttownley/.pyenv/versions/3.8.2/envs/demo/lib/python3.8/site-packages/states/states.py", line 108, in get
    add(obj=output,
  File "/Users/roberttownley/.pyenv/versions/3.8.2/envs/demo/lib/python3.8/site-packages/states/helpers.py", line 61, in add
    obj[part] = value
TypeError: 'str' object does not support item assignment

I added a print statement to the add funciton within states/helpers.py to see the state of obj, and the forloop iteration just before it fails has converted the object into a string, which causes the next item placement to fail.

In the iteration prior to that, the object contains two keys, with one of them containing a nested object. I added the following debug print statements, and have the last few iterations of output pasted below it:

# Modified add function

def add(obj, path, value):
    parts = path.strip("/").split("/")
    last = len(parts) - 1

    for index, part in enumerate(parts):
        print("Current obj: ", type(obj))
        print(obj)
        if index == last:
            obj[part] = value
        else:
            obj = obj.setdefault(part, {})

# Output
{'Networking': {'VPC': {'Managment': {'AandBblock': 'XXXXX'}, 'Research': 'YYYYY'}}}
Current obj:  <class 'dict'>
{'VPC': {'Managment': {'AandBblock': 'XXXXX}, 'Research': 'YYYYY'}}
Current obj:  <class 'dict'>
{'Managment': {'AandBblock': 'XXXXX'}, 'Research': 'YYYYY'}
Current obj:  <class 'str'>
YYYYY

I have been using ssm-diff for some time, but after upgrading to Debian Bookworm (with Python 3.11) I'm getting this error on a ssm-diff plan operation:

Traceback (most recent call last):
  File "/usr/local/bin/ssm-diff", line 82, in <module>
    args.func(args)
  File "/usr/local/bin/ssm-diff", line 42, in plan
    diff = helpers.FlatDictDiffer(r.get(paths=args.path), l.get(paths=args.path))
                                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/states/states.py", line 114, in get
    return flatten(output) if flat else output
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/states/helpers.py", line 44, in flatten
    if isinstance(d[k], collections.MutableMapping):
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'collections' has no attribute 'MutableMapping'

Any thoughts on adding a root path configuration? For example, I may only want to extract /Prod/Service/<service>/ to a YAML for revision and then push it back up.

We've been storing non-secret client params (we dedicated instances for security reasons) in separate YAML files. I've been pulling my hair out trying to figure out how to get the client-level bulk edit experience on SSM, but this project is perfect -- i.e. keep doing it the same way.

For import (and ongoing management and not pulling unnecessary secrets to disk), I really prefer to work with a subset of the tree at once (a single client or service). It'd be nice if I could pull just that branch. FWIW it could also make sense to allow me to filter that branch on a particular tag.

Hi,

Thanks for this great tool. We adopted entirely in our team as soon as we found a way to point ssm-diff to a specific region. As you may know, you have to run:

AWS_DEFAULT_REGION=<your region> ssm-diff ...

I think this should be in the README, do you agree?