ignore-match import mode ignored when importing json using appconfig/kvset profile about appconfiguration HOT 9 CLOSED

@carlin-q-scott If I understand you right, since changes to the key-values are made by the operations engineer through the GUI, the next deployment using CD would contain configuration settings that may not reflect the current state of the resource, and this is why those settings are written (or "overwritten" in your case). Another way of looking at this is, assuming you had updated a value property of an existing kvset configuration in your json file, then based on your expectation of this functionality, that change would not be reflected in your store after importing the file because a key with the same label already exists in the store. This means only additions are registered and updates would be ignored.

The idea is that the import will ignore matching key-labels and only add new ones requested by our development team by adding entries to the kvset.

The aim of import is to ensure that the resource always reflects the key-values specified in an imported file. We do not ignore match based on the key and label alone, else, that could result in confusions assuming configurations are being updated from different sources. Currently, the latest operation should always be reflected in the store.

In order to avoid overwriting key-values in your use case via CD imports, I would suggest the development team's imported JSON file contains only configuration settings that you wish to update or add.

I also still think the implementation of ignore-match import mode contradicts the help text. The key-values do already exist and are being overwritten.

I agree with this, and thanks for bringing this to our attention. I believe the messaging could be potentially misleading and we will work on updating the messaging for clarity. This will highlight that we only write key-values that do not already exist, or whose key, label, value, content-type or tags are different from what is already in the store.

Kindly let me know if this sounds good to you.

from appconfiguration.

Your proposed resolution is acceptable to me. The dev team can produce update files instead of full configuration files.

from appconfiguration.

Hi @carlin-q-scott , could you please provide information on the version of the cli that you are currently using?

Also, to clarify, we only avoid overwriting existing key-values in ignore-match mode that have not changed. With ignore-match, a key-value with a given key and label will only be ignored if the value and content-type in your configuration file are identical to what is already in the server.

from appconfiguration.

@carlin-q-scott

Are you able to help with the version information @albertofori mentioned?

from appconfiguration.

Closing as Albert has provided an explanation on the expected behavior and there is no response.

from appconfiguration.

@albertofori

we only avoid overwriting existing key-values in ignore-match mode that have not changed. With ignore-match, a key-value with a given key and label will only be ignored if the value and content-type in your configuration file are identical to what is already in the server.

I don't understand why that would be a useful feature. Only ignore the match if nothing has changed at all? Shouldn't it always do that regardless of CLI flags? I don't care if you overwrite an identical value, but that is less efficient in the backend, so that seems like an internal implementation detail that shouldn't be exposed through the public CLI.

The CLI help text also contradicts your statement:

If import mode is "ignore-match", source key-values that already exist at the destination will not be overwritten. Import mode "all" writes all key-values to the destination regardless of whether they exist or not. Allowed values: all, ignore-match. Default: ignore-match.

It's clearly stating that it won't overwrite existing keys.

I'm using version 2.61.0 of the az CLI. All my extensions are up to date.

from appconfiguration.

@carlin-q-scott May I know how you are identifying that an existing key-value that has not changed in your json file is being overwritten? The only way I see you could tell would be using the key-value's etag or last_modified property. Perhaps, some examples of this behavior with dummy data would be of help here.

Shouldn't it always do that regardless of CLI flags? I don't care if you overwrite an identical value, but that is less efficient in the backend, so that seems like an internal implementation detail that shouldn't be exposed through the public CLI.

I agree with this statement and that is why we default to ignore-match in order to avoid unnecessary writes and consumption of write request quota. However, this has not always been the behavior of the CLI. Prior to the introduction of this behavior, all key-values in a configuration file used to be written during import regardless of whether they have changed or not. On every write, some internally set properties always change, i.e., etag and last_modified, and some users might be relying on these properties to tell whether a recent import has been made to their resource. This made it necessary to keep the original behavior as an option, i.e., import-mode --all.

Only ignore the match if nothing has changed at all?

If none of the properties of a given key-value has changed. (i.e., key, label, content-type, value, tags), we do not write said key-value. Just to clarify, I am not talking about the whole configuration file here; comparison is done for each key-value in the file. Also note that a key-value is uniquely identified by the key and label properties. So if you update the label or key of a configuration in your kvset file, this is considered a new configuration setting altogether and is written as such.

from appconfiguration.

Thank you @albertofori for clarifying why the default of overwriting existing key-values was preserved. I appreciate the complexity in supporting potential use cases of a public API.

In regard to your question, I am not modifying the label of the keys that I am importing. In fact, I'm not modifying the values either; Our operations engineer is doing that in through App Config GUI. On the next deployment through our CD system, we are re-importing the same key-label-values as we did when the keys were created initially. The idea is that the import will ignore matching key-labels and only add new ones requested by our development team by adding entries to the kvset.

The reason I chose Azure App Config over a different kv-store, is that it has a pretty good GUI integrated with Azure Portal that can be used to modify values after creation. I think it loses value as a solution if there isn't a create-only kv-import option through the CLI or API.

I also still think the implementation of ignore-match import mode contradicts the help text. The key-values do already exist and are being overwritten.

from appconfiguration.

The ignore-match help message has been updated in this PR Azure/azure-cli#29200

from appconfiguration.