Git Product home page Git Product logo

ansible-jsonpatch's Introduction

ansible-jsonpatch

An Ansible module for patching JSON files

Description

This Ansible module follows RFC 6901 for JSON notation and RFC 6902 for JSON patching.

In addition to the standard, there is an extra notation feature not outlined in RFC 6901: you can specify every member of an array with a * character in the path.

Installation

In order to make use of this module, download the json_patch.py file to the library directory in the root of your Ansible project:

myproject/
├── ansible.cfg
├── inv/
├── library/
│   ├── json_patch.py
├── playbooks/
├── roles/

Usage

The module json_patch takes the following options:

Name Description Choices
src The path to a file containing JSON
dest The path to an optional output file to write the patched JSON (default is to overwrite src file)
operations A list of valid operations to perform against the given JSON file See RFC 6902
backup Backup the JSON file (the one that will be overwritten) before editing (default: no) yes/no
unsafe_writes Allow Ansible to fall back to unsafe (i.e. non-atomic) methods of writing files (default: no) yes/no
pretty Write JSON output in human-readable format yes/no
create Create file if it does not exist yes/no
create_type Type of JSON structure to create if empty (default: object) object/array

Operations

Operations may consist of two or more of the following properties:

Name Required Value
op yes one of add, remove, replace, move, copy, or test
path yes RFC 6901 JSON notation
from no RFC 6901 JSON notation (only applies to move and copy)
value no any valid JSON value

Special Notations

There are two special notations:

  • The - character refers to the end of an array (for appending a non-existent value)
  • (non-RFC 6901 compliant) The * character may refer to every member of an array (for testing each member)

Examples

ADD

Test JSON
{
    "foo": "bar"
}
Patch
- name: Add or replace 'baz' member
  json_patch:
    src: "test.json"
    operations:
      - op: add
        path: "/baz"
        value: "quz"
Patched JSON
{
    "foo": "bar",
    "baz": "quz"
}

REMOVE

Test JSON
[
    {"foo": "bar", "baz": "quz"},
    {"foo": "qux", "hello": "world"}
]
Patch
- name: Remove the 'hello' property
  json_patch:
    src: "test.json"
    operations:
      - op: remove
        path: "/1/hello"
Patched JSON
[
    {"foo": "bar", "baz": "quz"},
    {"foo": "qux"}
]

REPLACE

Test JSON
[
    {
        "foo": [
            {"vegetable": "corn"},
            {"vegetable": "broccoli"},
            {"vegetable": "tomato"}
        ]
    }
]
Patch
- name: Ensure 'tomato' isn't listed as a vegetable
  json_patch:
    src: "test.json"
    operations:
      - op: replace
        path: "/0/foo/2/vegetable"
        value: "cauliflower"
Patched JSON
[
    {
        "foo": [
            {"vegetable": "corn"},
            {"vegetable": "broccoli"},
            {"vegetable": "cauliflower"}
        ]
    }
]

MOVE

Test JSON
[
    {"one": 1, "two": 2, "six": 6},
    {"four": 4, "five": 5, "three": 3}
]
Patch
- name: Move the numbers into their appropriate objects
  json_patch:
    src: "test.json"
    operations:
      - op: move
        from: "/0/six"
        path: "/1/six"
      - op: move
        from: "/1/three"
        path: "/0/three"
Patched JSON
[
    {"one": 1, "two": 2, "three": 3},
    {"four": 4, "five": 5, "six": 6}
]

COPY

Test JSON
[
    {"name": "The Shawshank Redemption", "rating": "good"},
    {"name": "The Dark Knight", "rating": "bad"}
]
Patch
- name: Correct the rating for The Dark Knight
  json_patch:
    src: "test.json"
    backup: yes
    operations:
      - op: copy
        from: "/0/rating"
        path: "/1/rating"
Patched JSON
[
    {"name": "The Shawshank Redemption", "rating": "good"},
    {"name": "The Dark Knight", "rating": "good"}
]

TEST

Test JSON
{
    "name": "Ansible",
    "gitPath": "/home/ubuntu/projects/ansible",
    "enabled": true
}
Patch
- name: Ensure the Ansible project is enabled
  json_patch:
    src: "test.json"
    operations:
      - op: test
        path: "/enabled"
        value: true

SPECIAL NOTATIONS

End of array character (-)

Test JSON
{
    "media": "/mnt/media",
    "scan": [
      {"movie": "Independence Day"},
      {"movie": "The Sandlot"},
      {"movie": "Bird Box"}
    ]
}
Patch
- name: Add all of our movies to the list
  json_patch:
    src: "test.json"
    operations:
      - op: add
        path: "/scan/-"
        value:
          movie: "{{ item }}"
  with_items:
    - "Se7en"
    - "Firestarter"
    - "The Last Samurai"
    - "Fast and the Furious 17"
Patched JSON
{
    "media": "/mnt/media",
    "scan": [
      {"movie": "Independence Day"},
      {"movie": "The Sandlot"},
      {"movie": "Bird Box"},
      {"movie": "Se7en"},
      {"movie": "Firestarter"},
      {"movie": "The Last Samurai"},
      {"movie": "Fast and the Furious 17"}
    ]
}

Each array member character (*)

Test JSON
{
    "media": "/mnt/media",
    "scan": [
      {"movie": "Independence Day"},
      {"movie": "The Sandlot"},
      {"movie": "Bird Box"},
      {"movie": "Se7en"},
      {"movie": "Firestarter"},
      {"movie": "The Last Samurai"},
      {"movie": "Fast and the Furious 17"}
    ]
}
Patch
- name: Ensure The Last Samurai is definitely in our list
  json_patch:
    src: "test.json"
    operations:
      - op: test
        path: "/scan/*/movie"
        value: "The Last Samurai"

Testing

Testing is simple if you have pytest installed:

pytest

Note you may have to copy the json_patch.py file into your Ansible installation's lib/ansible/modules/files/ dir.

Contributing

PRs welcome! Just make sure you update tests as well.

Current needs:

  • Special character handling (RFC 6901 - Syntax)
  • Applying * support to other operations:
    • add should affect every member of an array
    • remove should remove the path from each member of an array
    • replace should replace for each member of an array

ansible-jsonpatch's People

Contributors

lvrfrc87 avatar mastervolkov avatar particledecay avatar tpo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

ansible-jsonpatch's Issues

Where to copy the file in ansible

Hi

I'm new with Ansible, may i know where to copy the file so i can call the jsonpatch in playbook,
ansible is installed in CentOS

[Feature Request] Support for file creation

Many other modules support a create: parameter, that indicates wether a nonexistent target file should be created at runtime.

This saves a touch operation earlier in the playbook, so I would ask for this to be added.

pass type value is wrong

I have file json
{
"foo" : 1
},
When I pass value using variable ansible, replace is wrong

operations:
- op: replace
path: "/foo"
value: "{{a}}"
Type of a variable is int a = 2 but output file json is
{
"foo" : "2"
},
expected output is:
{
"foo" : 2
},

exceptions.Exception does not take keyword arguments

My playbook:

    - copy:
        content: ''
        dest: /etc/docker/daemon.json
        force: no
        group: docker
        owner: root
    - jsonpatch:
        src: /etc/docker/daemon.json
        operations:
          - op: add
            path: "/insecure-registries"
            value: "[]"
          - op: add
            path: "/insecure-registries/0"
            value: "reg:5000"

wish: add ansible-jsonpatch to galaxy

Hi,

currently ansible-jsonpatch is living on my system in my ansible repo as a submodule. Submodule handling in git sucks in various ways. I could manage it outside of my ansible repo, but then, why not do the next step too and have ansible-jsonpatch on galaxy. That way I could manage it just like any other galaxy role/module...

?

Would be nice!

In case you loathe learning new galaxy processes then I can assist you through it having done it recently. -> DM.

Thanks for ansible-jsonpatch!
*t

quotation marks are force at integers

I've a json with like this:
{
"bla": 3,
"devices": [
{
"dummy": "/var"
},
{
"tempdir": "/var/temp",
"memlimit": 100,
"units": 50
}
]
}

and try to change the unit value from 50 to 250 with following ansible code:

  • name: "manage json file test"
    json_patch:
    src: "{{ src }}"
    dest: "{{ dest }}"
    pretty: yes
    operations:
    - op: replace
    path: devices/1/units
    value: 250

but regardless of how I write the value, the result is always "units": "250" (with quotation marks around the integer). If I replace the whole inner curly bracket (tempdir, memlimit, units) like this:

path: devices/1
value: '{ "tempdir": "/var/temp", "memlimit": 100, "units": 250 }'

it works. But I think there must be a way to output a number without quotation marks, right?
(sorry, I haven't figured out how to make indentations here)

Changed always true for some reason

I'm trying to write task to change docker config, and restart docker service only if it was changed - and json_patch task for some reason always return changed: True

Return:

{
  "changed": false,
  "dest": "/etc/docker/daemon.json",
  "diff": {
    "after": "{\n    \"log-driver\": \"json-file\",\n    \"log-opts\": {\n        \"max-size\": \"10m\",\n        \"max-file\": \"10\"\n    }\n}",
    "after_header": "/etc/docker/daemon.json (content)",
    "before": "{\n    \"log-driver\": \"json-file\",\n    \"log-opts\": {\n        \"max-size\": \"10m\",\n        \"max-file\": \"10\"\n    }\n}",
    "before_header": "/etc/docker/daemon.json (content)"
  },
  "failed": false,
  "gid": 0,
  "group": "root",
  "mode": "0644",
  "owner": "root",
  "secontext": "system_u:object_r:container_config_t:s0",
  "size": 110,
  "state": "file",
  "uid": 0
}

Task example:

    - name: Configure docker
      become: true
      block:
        - name: Setup docker logrotation by default
          json_patch:
            src: "/etc/docker/daemon.json"
            operations:
              - op: add
                path: "/log-driver"
                value: "json-file"
              - op: add
                path: "/log-opts"
                value: {}
              - op: add
                path: "/log-opts/max-size"
                value: "10m"
              - op: add
                path: "/log-opts/max-file"
                value: "10"
            pretty: true
            create: true
          register: json_patch_output
          changed_when: json_patch_output.diff.before != json_patch_output.diff.after

        - name: Set is_docker_config_changed
          ansible.builtin.set_fact:
            is_docker_config_changed: "{{ json_patch_output.diff.before != json_patch_output.diff.after }}"

        - name: Restart docker
          when: is_docker_config_changed
          ansible.builtin.systemd:
            name: docker
            state: restarted

How to add object with multiple values into array

Hello, i have this json

{
  "repositories": [
    { 
      "name": "some-name", 
      "type": "vcs", 
      "url": "[email protected]" 
    }
  ]
}

and i am trying to create this

{
  "repositories": [
    { 
      "name": "some-name", 
      "type": "vcs", 
      "url": "[email protected]" 
    },
    { 
      "name": "new-value", 
      "type": "vcs", 
      "url": "[email protected]" 
    }
  ]
}

but i dont know how. Is there possibility add multiple values into object?

like this:

- name: append key/values
  json_patch:
    src: /file.json
    operations:
      - op: add
        path: '/repositories/-'
        value: 
          name: "new-value"
          type: "vcs"
          url: "[email protected]"
    pretty: yes
    create: yes

Adding a structure to the top level of a structure

If I do this:

- name: "copying log file management data to '{{ docker_daemon_file }}'"
  json_patch:
    src: "{{ docker_daemon_file }}"
    operations:
      - op: add
        path: '/'
        value: "{{ docker_logging_options }}"
    pretty: true
    create: true

I'm expecting this:

        {
          "log-driver": "json-file",
          "log-opts": {
          "max-size": "50m",    
          "max-file": "1"    
                      }
        } 

But I'm getting this:

{
    "": {
        "log-driver": "json-file",
        "log_opts": {
            "max-size": "50m",
            "max-file": 1
        }
    }
}

How do I add a structure to the root node of a json structure? I tried setting path to an empty string.

BTW, great module, thank you so much!

converting to pretty format only

Hi, I'm using this module only to convert the format to "pretty" however I get an error message:

my code:

 - name: Ensure the Ansible project is enabled
    json_patch:
      src: "test8.json"
      pretty: yes
      create: yes
      operations:
        - op: copy
          path: "/etc/ansible/test9_jolehet.json"

here's my output:

MSG:

MODULE FAILURE
See stdout/stderr for the exact error


MODULE_STDERR:

Traceback (most recent call last):
  File "/Users/asd/.ansible/tmp/ansible-local-12529opgh61t/ansible-tmp-1602859283.986748-1320-41886083824797/AnsiballZ_json_patch.py", line 102, in <module>
    _ansiballz_main()
  File "/Users/asd/.ansible/tmp/ansible-local-12529opgh61t/ansible-tmp-1602859283.986748-1320-41886083824797/AnsiballZ_json_patch.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/Users/asd/.ansible/tmp/ansible-local-12529opgh61t/ansible-tmp-1602859283.986748-1320-41886083824797/AnsiballZ_json_patch.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.json_patch', init_globals=None, run_name='__main__', alter_sys=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 188, in run_module
    fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/var/folders/sz/g96y86bd1ndfx_r38rlq2ds00000gn/T/ansible_json_patch_payload_5banL0/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 567, in <module>
  File "/var/folders/sz/g96y86bd1ndfx_r38rlq2ds00000gn/T/ansible_json_patch_payload_5banL0/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 561, in main
  File "/var/folders/sz/g96y86bd1ndfx_r38rlq2ds00000gn/T/ansible_json_patch_payload_5banL0/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 246, in run
  File "/var/folders/sz/g96y86bd1ndfx_r38rlq2ds00000gn/T/ansible_json_patch_payload_5banL0/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 341, in patch
TypeError: copy() takes exactly 4 arguments (3 given)

:ansible asd$ ansible --version | grep "python version"
python version = 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27) [Clang 6.0 (clang-600.0.57)]
:ansible asd$ ansible --version
ansible 2.9.9

Any idea what could it be?

write pretty formatted json file

Hi,
I dont' know if this is an issue with the module or generally with my bad Ansible knowledge:

The manipulated json file is being removed all (human readable) formatting (\n etc).

 - name: "edit config.json"
    json_patch:
      src: "{{riot_path}}/config.sample.json"
      dest: "{{riot_path}}/config.json"
      backup: yes
      operations:
        - op: replace
          path: "/default_server_config/m.homeserver/server_name"
          value: "my.server.com"

I also tried with first copying config.sample.json to config.json and then just using src: with json_patch so it overwrites the original filename - it's the same - all formatting gets lost and config.json is a proper json but in an unpretty "oneline format".

Is this me using it wrong or is it supposed to be like that?

Thanks in advance
Jojo

Multiple operations only writes changes when last op has changes

When multiple op are done and the last op does not change the file, the previous changes are not written.

Steps to reproduce

  1. Run a task against a file with multiple operations where the last operation does not change the value of the path.
  • json_patch:
    src: /tims/etc/PaiConfiguration.json
    operations:
    - op: replace
    path: /fgaActive
    value: "{{ PAI_fgaActive }}"
    - op: replace
    path: /fgaContactType
    value: "{{ PAI_fgaContactType }}"
    - op: replace
    path: /uwcDoorSignalActive
    value: "{{ PAI_uwcDoorSignalActive }}"
    - op: replace
    path: /uwcDoorSignalContactType
    value: "{{ PAI_uwcDoorSignalContactType }}"
    pretty: yes
  1. The ansible debug output shows the changes that should me done, but the when checking the file, it is still the same.

This also happens when no Ansible variables, but strings are used.

Workaround:
Add two dummy operations at the end which guarantees a change:
- op: add
path: /dummy
value: "{{ ansible_date_time.iso8601_basic_short }}"
- op: remove
path: /dummy

Adding a new object to JSON Array

Is there any option to push a new object to an array? something like this:

Existing array:

[
    {foo: bar},
    {bar: biz}
]

Desired array:

[
    {foo: bar},
    {bar: biz},
    {biz: foo},
]

Note: I don't know how many objects this array contains!

wish: support --diff

json_patch works as advertized:

$ ansible-playbook rojava.yml --check --diff

PLAY [rojava] **********************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************************************************************************
ok: [rojava]

TASK [service : enable alerts] ****************************************************************************************************************************************************************************************
changed: [rojava]

PLAY RECAP *************************************************************************************************************************************************************************************************************************
rojava                     : ok=2    changed=1    unreachable=0    failed=0   

However it'd be very nice, if it could display the diff of what it would have done.

Thanks,
*t

UTF-8 not supported?

I have UTF-8 encoded JSON file with string:
März_Lebensmittel
but module changes file encoding to ASCII and string becomes:
M\u00e4rz_Lebensmittel

Is UTF-8 supported?

Missing key is not created when trying to add an element to a key that does not exist yet.

I want to add an item to an array in the JSON file, if the key already exists it should be added, if the key doesn't exist yet it should be created.

Ansible task:

- name: configure docker daemon for denyusernshost
  json_patch:
    src: "/etc/docker/daemon.json"
    create: yes
    pretty: yes
    operations:
      - op: add
        path: "/authorization-plugins/-"
        value: "denyusernshost"

Expected result:

{
    "authorization-plugins": ["denyusernshost"]
}

Traceback:

Traceback (most recent call last):
File "/tmp/ansible_json_patch_payload_pk9_yaou/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 404, in add
KeyError: 'authorization-plugins'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/username/.ansible/tmp/ansible-tmp-1630314042.2142181-13278-33899836330021/AnsiballZ_json_patch.py", line 102, in <module>
    _ansiballz_main()
File "/home/username/.ansible/tmp/ansible-tmp-1630314042.2142181-13278-33899836330021/AnsiballZ_json_patch.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
File "/home/username/.ansible/tmp/ansible-tmp-1630314042.2142181-13278-33899836330021/AnsiballZ_json_patch.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.json_patch', init_globals=None, run_name='__main__', alter_sys=True)
File "/usr/lib/python3.8/runpy.py", line 207, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
File "/tmp/ansible_json_patch_payload_pk9_yaou/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 567, in <module>
File "/tmp/ansible_json_patch_payload_pk9_yaou/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 561, in main
File "/tmp/ansible_json_patch_payload_pk9_yaou/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 246, in run
File "/tmp/ansible_json_patch_payload_pk9_yaou/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 341, in patch
File "/tmp/ansible_json_patch_payload_pk9_yaou/ansible_json_patch_payload.zip/ansible/modules/json_patch.py", line 406, in add
__main__.PathError: could not find 'authorization-plugins' member in JSON object

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.