keeper-security / secrets-manager Goto Github PK

This issue was automatically created by the GitHub Action workflow Publish to Azure Pipeline Extension.

Due to failure in run: 1391425806.

If error is related to access denied (expiration of Personal Access Token, aka PAT), here are the steps to generate a new one:

-Steps to generate one documented HERE

• Update "Password" field in record UID DJz3ilHBHIbIZqkTClDV5Q (located in shared folder named "Secrets Manager")

The loadJsonConfig function is not documented in the documentation for the KSM JavaScript SDK. This function is exported and works as expected, though the only way one would know about what it does is to dive in to the JavaScript SDK codebase.

At the current release, 16.5.4, there is a cyclical dependency. keeper-secrets-manager-core depends on keeper-secrets-manager-helper, which in turn depends on keeper-secrets-manager-core, making this package impossible to build in some environments.

Hello,

Is it plan to integrate keeper secret manager as a provider in external secrets ?

https://external-secrets.io/v0.7.1/provider/aws-secrets-manager/

if yes have you some visibility for the release of the feature ?

Like System.Text.Json - would be really better to have that option set to "False" - common libraries need to be updated without any issues on other components

The latest version of ksm (1.1.1) throws an exception when running in the redis:alpine container (and possibly others). We're using ksm in our docker-compose file to start redis via a shell script, using the "exec" command to replace an environment variable with the secret. We run the keeper/keeper-secrets-manager-cli:latest image first and the redis container executes /cli/musl/ksm and we see the following exception:

Traceback (most recent call last):
  File "ksm.py", line 1, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "keeper_secrets_manager_cli/__init__.py", line 13, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "keeper_secrets_manager_core/__init__.py", line 1, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "keeper_secrets_manager_core/core.py", line 24, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "keeper_secrets_manager_core/utils.py", line 31, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "keeper_secrets_manager_core/keeper_globals.py", line 12, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "importlib_metadata/__init__.py", line 6, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 540, in exec_module
  File "zipp/__init__.py", line 9, in <module>
ModuleNotFoundError: No module named 'zipp.py310compat'

This error does not occur in the 1.1.0 version.

The ManifestLoader is expecting the classPath to contain exclaimation mark but Wildfly always return "vfs:" instead of "file:", and vfs does not contain the !. And because neither the build/classes nor out/production/classes directory exists, it throws IndexOutOfBoundException: : String index out of range: -1 during initialization.

Here're the 2 paths when running in Eclipse versus when running in Wildfly:

jar:file:/path/to/.m2/repository/com/keepersecurity/secrets-manager/core/16.3.3/core-16.3.3.jar!/com/keepersecurity/secretsManager/core/ManifestLoader.class vfs:/path/to/wildfly/standalone/deployments/app.war/WEB-INF/lib/core-16.3.3.jar/com/keepersecurity/secretsManager/core/ManifestLoader.class

This is the code I'm talking about. It is from SecretsManager.class:

internal object ManifestLoader {
internal val version: String

init {
    val clazz = javaClass
    val classPath: String = clazz.getResource(clazz.simpleName.toString() + ".class")!!.toString()
    val libPathEnd = classPath.lastIndexOf("!")
    val filePath = if (libPathEnd > 0) {
        val libPath = classPath.substring(0, libPathEnd)
        "$libPath!/META-INF/MANIFEST.MF"
    } else { // we might be testing
        var buildPathCoreIdx = classPath.lastIndexOf("build/classes")
        if (buildPathCoreIdx < 0) {
            buildPathCoreIdx = classPath.lastIndexOf("out/production/classes")
        }
        val buildPath = classPath.substring(0, buildPathCoreIdx)
        "${buildPath}build/tmp/jar/MANIFEST.MF"
    }
    val manifest = Manifest(URL(filePath).openStream())
    version = manifest.mainAttributes.getValue("Implementation-Version")
}

}

On the ansible-galaxy for keeper, it states:

keepersecrity.keeper_secrets_manager

This is incorrect, it should be:

keepersecurity.keeper_secrets_manager

keeper_create in the ansible plugin does not use custom record types.
This is due to test is done BEFORE the load of the custom record types

The latest linux release is only 20 Bytes large...

When running ksm profile export PROFILE_NAME, I receive the following error when running on Windows 10 Pro 21H2 with python 3.10.4:

ksm had a problem: [Errno 13] Permission denied: 'C:\\Users\\<Username>\\AppData\\Local\\Temp\\tmpykw4w_ym'

Upon further inspection, the file has proper NTFS file permissions such that my user has full access to the file and the temporary directory as well. Further digging reveals that the issue is being caused due to this temporary file being opened twice, which is not permitted by Windows. I was able to work around the issue by adding a call to tf.close() immediately before config.save() is called on line 58 of export.py.

tempfile.NamedTemporaryFile() creates and opens a new temporary file. The problem comes when config.save() is called as it calls open() on this same file again for writing on line 114 of config.py when the file is already open in the first place. Since the file is already open, Windows fails to open the file with an error, crashing the application. This behavior does not occur on UNIX since this type of usage is permitted on UNIX-based platforms. This behavior is explicitly called-out in the python tempfile package documentation:

Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows).

See: https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile

The open file object is likely what should be passed around through the application to prevent a need for double-opening, ensuring proper cross-platform support, or a different technique should have been used to create the temporary file without opening it first.

There is an issue with parsing. The given uid is converted to lower case and hence can not get a file.

[ TEST4 2022-04-07 14:35 ]
username@host:~/git/eval-ksm (main*$%=) $ ksm secret download --uid=pluIUSAmO3r7oyAPYKMDOg --name=java11_db-client.jks --file-output=client.jks
Error: Cannot find a record for UID pluiusamo3r7oyapykmdog. Cannot download java11_db-client.jks

[ TEST4 2022-04-07 14:39 ]
username@host:~/git/eval-ksm (main*$%=) ksm secret get pluiusamo3r7oyapykmdog && echo success
success

[ TEST4 2022-04-07 14:40 ]
username@host:~/git/eval-ksm (main*$%=) $ ksm secret get pluIUSAmO3r7oyAPYKMDOg

 Record: pluIUSAmO3r7oyAPYKMDOg
 Title: test-mysql-credentials
 Record Type: databaseClusterCredentials

 Field                   Value
 ----------------------- ------------------------------------------------------------------------------
 Type                    mysql
 Hostname or IP Address  [{"port": "32100", "hostName": "db11"}]
 Hostname or IP Address  [{"port": "32100", "hostName": "db12"}]
 Hostname or IP Address  [{"port": "32100", "hostName": "db13"}]
 Login                   dbuser
 Password                ****
 trustStore              ["4JjLTEq6KO6Qowvwf8ZyNA", "xrE4oKfWqBd6JzTGpxGN_A", "q-q9jRPvshsHfNo2e6Ib4Q"]


 File Name              Type Size
 ---------------------- ---- ----
 client.jks                  3551
 java11_db-trusted.jks       1143
 java11_db-client.jks        3551

As you can see the lower case query also returns success ( rc=0 ), which should be another bug.

You should also note that it is possible to upload multiple files with the same name and even the same content. Does this make sense? How should I download a specific file with same filename but different content? For what are the uid-like id's in the array of the tustStore field?

The documentation does not seem to show I can create folders to group the secrets?

Hi,

I have downloaded Javascript SDK and wanted to test it. Followed initialisation guide I found here https://docs.keeper.io/secrets-manager/secrets-manager/developer-sdk-library/javascript-sdk but only the first request where I do initializeStorage with one time token is successful. After I remove one time token and initializeStorage line I am getting Unable to load the key appKey error:

Error: Unable to load the key appKey
    at /path/keeper-poc/node_modules/@keeper-security/secrets-manager-core/dist/index.cjs.js:273:15
    at Generator.next (<anonymous>)
    at /path/keeper-poc/node_modules/@keeper-security/secrets-manager-core/dist/index.cjs.js:54:71
    at new Promise (<anonymous>)
    at __awaiter (/path/keeper-poc/node_modules/@keeper-security/secrets-manager-core/dist/index.cjs.js:50:12)
    at loadKey (/path/keeper-poc/node_modules/@keeper-security/secrets-manager-core/dist/index.cjs.js:260:37)
    at /path/keeper-poc/node_modules/@keeper-security/secrets-manager-core/dist/index.cjs.js:367:23
    at Generator.next (<anonymous>)
    at /path/keeper-poc/node_modules/@keeper-security/secrets-manager-core/dist/index.cjs.js:54:71
    at new Promise (<anonymous>)

I tried with both localConfigStorage and loadJsonConfig without any success. Config looks fine and it has appKey value.

Thanks
Ivan

Hello,

I would like to know if it's possible to create a new folder and share it with a specific user using python SDK.

Thanks

A small script is asking for the one-time-token (which is generated with the Desktop Application) and establish the connection to the Keeper Secrets vault:

from keeper_secrets_manager_core import SecretsManager
from keeper_secrets_manager_core.storage import FileKeyValueStorage

oneTimeToken = input("OneTimeToken: ")
secrets_manager = SecretsManager(
    token=oneTimeToken,
    config=FileKeyValueStorage('ksm-config.json')
)

The resulting ksm-config.json is generated and includes the following keys:

{
    "hostname": "...",
    "clientId": "...",
    "privateKey": "...",
    "serverPublicKeyId": "..."
}

When trying to use this connection, the script is not able to get any secret. I think, this cannot work because some documented JSON keys are missing:

{
    "hostname": "...",
    "clientId": "...",
    "privateKey": "...",
    "serverPublicKeyId": "...",
    "appKey": "..missing!!..",
    "appOwnerPublicKey": "..missing!!.."
}

When generating the config.json with the Desktop Applikation (by adding a device), these fields are included - and the script connection is working.

Do I miss something?

Following the examples in the documentation isn't working for me. What am I missing?

I've dug into the issue a little. But, hit a roadblock. It may be an issue in your SDK?

The Code:

from keeper_secrets_manager_core import SecretsManager
from keeper_secrets_manager_core.storage import FileKeyValueStorage

if __name__ == '__main__':
    secrets_manager = SecretsManager(
        hostname="keepersecurity.com",
        token='US:************************************',
        config=FileKeyValueStorage('ksm-config.json')
    )

    secrets = secrets_manager.get_secrets()

    if secrets is not None:
        for secret in secrets:
            print("\tRecord details: %s" % secret.dict)
    else:
        print('None')

The Error

Traceback (most recent call last):
  File "/Users/*********/PycharmProjects/SentinelOne/keeperConfig.py", line 18, in <module>
    secrets = secrets_manager.get_secrets(['herfegDiPfWJnVEVpIjJsg'])[0]
  File "/Users/*********/PycharmProjects/SentinelOne/venv/lib/python3.9/site-packages/keeper_secrets_manager_core/core.py", line 639, in get_secrets
    records_resp = self.fetch_and_decrypt_secrets(uids)
  File "/Users/*********/PycharmProjects/SentinelOne/venv/lib/python3.9/site-packages/keeper_secrets_manager_core/core.py", line 573, in fetch_and_decrypt_secrets
    client_key = url_safe_str_to_bytes(self.config.get(ConfigKeys.KEY_CLIENT_KEY))
  File "/Users/*********/PycharmProjects/SentinelOne/venv/lib/python3.9/site-packages/keeper_secrets_manager_core/utils.py", line 71, in url_safe_str_to_bytes
    b = base64.urlsafe_b64decode(s + '==')
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Any help is appreciated! :)

Tried to download the latest client (vault-plugin-secrets-ksm_1.0.2_windows_amd64.zip) for windows but get this instead:

Sorry for the Swedish but it says there is a trojan called Win32/Wacatac.B!ml in the zip.

// Michael Medin

The ksm init k8s command output is missing a new line between type and immutable keys when called with --immutable option.

...
kind: Secret
metadata:
  name: ksm-config
type: Opaqueimmutable: True

It should be easy to resolve with an additional \n after type: Opaque or before immutable: True\n.

Hi guys, i'm trying to test the ansible integration but i'm getting an error.

Code:

- name: test
  hosts: localhost
  collections:
   - keepersecurity.keeper_secrets_manager
  tasks: 
  - name: Get value
    debug:
      msg: "{{ lookup('keepersecurity.keeper_secrets_manager.keeper', uid='myId', field='login') }}"

Error:

ImportError: cannot import name 'KeeperAnsible' from 'ansible_collections.keepersecurity.keeper_secrets_manager.plugins.common.keeper_secrets_manager_ansible' (/Users/user/.ansible/collections/ansible_collections/keepersecurity/keeper_secrets_manager/plugins/common/keeper_secrets_manager_ansible/__synthetic__)

Ansible config:

❯ ansible --version
ansible 2.9.18
  config file = None
  configured module search path = ['/Users/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/user/.asdf/installs/python/3.8.16/lib/python3.8/site-packages/ansible
  executable location = /Users/user/.asdf/installs/python/3.8.16/bin/ansible
  python version = 3.8.16 (default, Mar 13 2023, 22:19:52) [Clang 14.0.0 (clang-1400.0.29.202)]

Am i missing anything?

Hello, please excuse the bonehead question but is there a nice way to get a list of Keeper records based on the name of a shared folder?

e.g. if in my keeper vault I have a shared folder called "myfolder" containing two records, "secret1" and "secret2" is there a way that I can sensibly invoke https://github.com/Keeper-Security/secrets-manager/blob/master/sdk/python/core/keeper_secrets_manager_core/dto/dtos.py#L269-L290 to get it to spit out the folder representation and the secrets so I can go:

myfolder = SecretsManager.folder_by_name("myfolder")
for secret in myfolder.records:
  print(secret.uid)

Currently today there is no option to rename (change the title of) a record Or duplicate a record with the cli. This is however an option via the UI. But as you can imagine going through and doing this through the gui can be extremely tedious for those of us that have hundreds of records.

This issue was automatically created by the GitHub Action workflow Publish to Azure Pipeline Extension.

due to failure in run: 1387792761.

ksm exec -- source .my.secrests.file

Error: Cannot execute command: [Errno 2] No such file or directory: 'source'

However, when I run it directly, it does as I expect and does not throw an error.

Hi,

I noticed that, even following session time-out or record deletion, keeper stores records as plaintext JSON in memory. So long as a user initiated signout or app restart is not performed one can extract all credentials from memory in plain text. This includes the browser extensions.

When calling get_secrets(), records of type general are not retrieved while records of other types are correctly fetched.

Here is a minimal example showcase the issue:

from keeper_secrets_manager_core import SecretsManager
from keeper_secrets_manager_core.storage import FileKeyValueStorage

client = SecretsManager(
    token="...",
    config=FileKeyValueStorage('ksm-config.json')
)

records = client.get_secrets()

for record in records:
    print(record)

with the following shared folder

returns:

[Record: uid=..., type: login, title: Login, files count: 0]
[Record: uid=..., type: encryptedNotes, title: TEST, files count: 0]
[Record: uid=..., type: databaseCredentials, title: Database, files count: 0]

I stumbled across this issue while working on Kubernetes integration for applications that utilize the JavaScript/TypeScript KSM SDKs. The documentation for Kubernetes integration indicates that the config will be loaded from the KSM_CONFIG environment variable. However, this only works on the Python SDK, which is the only SDK which currently attempts to read the KSM_CONFIG environment variable. This seems to conflict with the Kubernetes integration documentation where it seems to indicate that all SDKs should read KSM_CONFIG.

I can confirm that KSM_CONFIG is not read from the JavaScript SDK as a search through all the files yields no results for process.env.KSM_CONFIG. Searching for "KSM_CONFIG" through the entire KSM repository yields only the references added in #148 , which only affected the Python SDK.

I considered attempting to create a solution for the JavaScript SDK in a new fork, followed by submitting a PR, though this issue impacts all SDKs other than the Python SDK and I imagine Keeper would like to keep some consistency in their APIs function calling and naming conventions across languages.

In the meantime, I have worked around this issue in the JavaScript SDK by adding code in the application that is consuming the SDK to manually check for a value in the KSM_CONFIG environment variable, constructing the KeyValueStorage object using the loadJsonConfig function exported by the KSM package. When that env variable is not found, it will then try to create a KeyValueStorage object using localConfigStorage and a local file.

If all this is expected behavior, the Kubernetes documentation should probably be updated to indicate that it exposes the config as the KSM_CONFIG environment variable and that the Python SDK will automatically detect this, noting that other SDKs will require manually passing in the value from this environment variable for it to be used. Either way, having all SDKs behave similar to how the Python SDK functions with the KSM_CONFIG environment variable would be ideal since this would likely be a common use case.

KSM and ansible plugin creates records only on the root of shared folder.
Is it possible to allow it in any subfolder of a shared folder ?

We initialize KSM cli using an environment KSM_CONFIG.

When we use any ksm command, the first time it will generate a keeper.ini file in the current directory.

I've noticed we can set KSM_INI_DIR to define where the file is stored. Even with this var defined, KSM will create a keeper.ini file in the current directory.

Would it be possible to prevent this file creation if the KSM_INI_DIR already contains a valid config?

Running a simple python script using SDK library to get secrets from secrets manager (based on the as code presented in Usage example of the documentation) returns a TypeError.

In keeper_secrets_manager_core/core.py, line 573: the value returned by the method self.config.get(ConfigKeys.KEY_CLIENT_KEY) is None, so it throws a TypeError exception when given to url_safe_str_to_bytes function.

See python tracebacks below for more details:

Traceback (most recent call last):
File "tests/test_ksm.py", line 5, in
all_secrets = secrets_manager.get_secrets()
File "/home/sam/autopylot/venv/lib/python3.8/site-packages/keeper_secrets_manager_core/core.py", line 639, in get_secrets
records_resp = self.fetch_and_decrypt_secrets(uids)
File "/home/sam/autopylot/venv/lib/python3.8/site-packages/keeper_secrets_manager_core/core.py", line 573, in fetch_and_decrypt_secrets
client_key = url_safe_str_to_bytes(self.config.get(ConfigKeys.KEY_CLIENT_KEY))
File "/home/sam/autopylot/venv/lib/python3.8/site-packages/keeper_secrets_manager_core/utils.py", line 71, in url_safe_str_to_bytes
b = base64.urlsafe_b64decode(s + '==')
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Environment information:

• package keeper-secrets-manager-core in version 16.3.5
• python version: 3.8.10
• OS: Linux Ubuntu 20.04 LTS (codename: focal)

When creating new secret with values containing accents, they are replaced by question mark.

The issue comes from the options in JsonUtils class.

JsonSerializerOptions Options = new()
        {
            DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingDefault,
            Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
        };

var test = JsonSerializer.Deserialize<string>(CryptoUtils.StringToBytes(JsonSerializer.Serialize("éeèeàa°o", Options)));
test
"?e?e?a?o"

var test2 = JsonSerializer.Deserialize<string>(CryptoUtils.StringToBytes(JsonSerializer.Serialize("éèà°")));
test2
"éèà°"

getting an error trying to use ksm cli tool with a record who's UUID starts with -

 ksm secret notation '-iT4kWE2ZAkdHLqVlCfoWQ/field/Test'
Usage: ksm secret notation [OPTIONS] TEXT
Try 'ksm secret notation --help' for help.

Error: no such option: -i

I've tried single and double quotes around the UUID, as well as escaping the - with \

Anyone run into this issue before?

I've noticed the same with ksm secret get UUID, but this can be avoided by using the -u option, but there is not an equivalent for ksm sync, or ksm secret notation.

When called SecretsManagerClient.GetSecrets () method, the following error: System.AggregateException: 'One or more errors occurred. (Valid values are between -62135596800 and 253402300799, inclusive. (Parameter 'seconds'))'

The POST https://keepersecurity.com/api/rest/sm/v1/get_secret route is returning an expiredate of 1695767681000, which is outside the bounds. Converting that number from milliseconds to seconds returns a correct date/time.

i'm trying to create a new command in my keeper orb, which executes the following command : ksm exec -- $SHELL

in order to attribute secrets to my environment vairables in the job, this is how the command is like :

`
description: >
Run a command with secret environment variables loaded from SecretHub

steps:
  - install
  - run:
      name: fetching secrets
      command: ksm exec -- $SHELL

`

and this is the job to test it :

integration-test-exec-command:
   
   docker:
     - image: cimg/base:stable
   environment:
       MY_USER: keeper://vNrItsdDxL3M9uTuxvzY_g/field/login
   steps:
     - checkout           
     - keeper/exec

i'm having this error on circleCI :

ksm exec -- $SHELL
Imported config saved to profile App1 at keeper.ini.
ksm had a problem: Keeper url missing information about the uid, field type, and field key

it is strange because i have tested the command locally on my machine, and it is working, and i have tried it with all type of executors on CircleCI but it's not working,

Can you please guys take a look on this issue, as we need this command in our workflow, and get back to us
this is quite urgent as we will be fully migrating to keeper by next week ?
thanks

Within our organization, we are experimenting with using Keeper in our CI/CD pipeline, and we were reasoning about a solution for storing and accessing secrets for different deployments of the same product (e.g. production, staging, integration). We initially thought about using nested directories as shown below:

our-app               (shared folder)
`-- production        (folder)
|   `-- REDIS_AUTH    (secret)
|   `-- DATABASE_URL  (secret)
`-- staging           (folder)
|   `-- REDIS_AUTH    (secret)
|   `-- DATABASE_URL  (secret)

And since the deployment environment is known by the time we need to the vault from our CI/CD platform (i.e. GitLab sets the CI_ENVIRONMENT_NAME environment variable with information about the target environemnt), we thought we could easily access secrets with a notation like: file:///$CI_ENVIRONMENT_NAME/REDIS_AUTH; however, Keeper Notation does not seem to support accessing records by their names / paths relative to the application directory, so we are kind of stuck, wondering if there wasn't a better way to deal with this.

Of course we could have each environment use different Keeper record IDs:

$ cat .env.production
REDIS_AUTH=keeper://KEEPER_ID_1/field/password

$ cat .env.staging
REDIS_AUTH=keeper://KEEPER_ID_2/field/password

And then source these files before before calling ksm exec --:

$ dotenv -f .env.$CI_ENVIRONMENT_NAME export
# This loads/exports from the Right .env file

$ ksm exec -- terraform ...

This works, but it requires us to copy/paste dozens of Keeper IDs, and we were hoping for something a bit less error prone than that. What other alternatives do we have? Would it make sense to store these .env files, with the actual secrets and not the Keeper notation, directly inside the vault? This way one would simply need to deal with two IDs (or three, or as many as they have environments to deploy to) and at deploy time, they would only have to read the current environment file from the vault, source/re-export it, and move on with the deployment steps.

I am sure our team is not the first one having to implement this, so please let us know if you thought there were better ways to deal with this.

Thanks,
M.

These exist in the commander CLI but not in the SDK.

  hosts: localhost
  connection: local
  gather_facts: Yes
  become: No
  tasks:
    - block:
        - keepersecurity.keeper_secrets_manager.keeper_copy:
            notation: "some_record_uid/field/keyPair[privateKey]"
            dest: "/tmp/id_rsa"
            mode: "0600"
            owner: "{{ lookup('env','USER') }}"
            group: "{{ lookup('pipe', 'id -g -n') }}"

error message:

ERROR! Unexpected Exception, this is probably a bug: cannot import name 'KeeperAnsible'
the full traceback was:

Traceback (most recent call last):
  File "/usr/bin/ansible-playbook", line 123, in <module>
    exit_code = cli.run()
  File "/usr/lib/python3.6/site-packages/ansible/cli/playbook.py", line 128, in run
    results = pbex.run()
  File "/usr/lib/python3.6/site-packages/ansible/executor/playbook_executor.py", line 169, in run
    result = self._tqm.run(play=play)
  File "/usr/lib/python3.6/site-packages/ansible/executor/task_queue_manager.py", line 282, in run
    play_return = strategy.run(iterator, play_context)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/strategy/linear.py", line 249, in run
    action = action_loader.get(task.action, class_only=True)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/loader.py", line 552, in get
    self._module_cache[path] = self._load_module_source(name, path)
  File "/usr/lib/python3.6/site-packages/ansible/plugins/loader.py", line 525, in _load_module_source
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/username/git/myrepo/collections/ansible_collections/keepersecurity/keeper_secrets_manager/plugins/action/keeper_copy.py", line 14, in <module>
    from ansible_collections.keepersecurity.keeper_secrets_manager.plugins.common.keeper_secrets_manager_ansible import KeeperAnsible
ImportError: cannot import name 'KeeperAnsible'

Hi!

A space present in the one's username will cause an error at line 292 of /sdk/python/core/keeper_secrets_manager_core/utils.py when instantiating the SecretsManager as get_windows_user_sid_and_name() will return more than two values to unpack.

I suggest splitting on the last occurrence of the space character at line 213 instead:

return reversed(user_sid.split('\\')[-1].rsplit(' ',1))

Thanks!
Arnaud

I tried running the code to fetch some records, which runs fine on Windows machine but takes 30 minutes on Linux machine. Originally, I blamed it on a network issue but the network people have assure me it's not a network issue, so I'm at a block here. Could you please verify if it is indeed working properly on Linux?

The test code:

LocalConfigStorage storage = new LocalConfigStorage(configPath);
SecretsManagerOptions options = new SecretsManagerOptions(storage);

System.out.print("Initializing KeeperSecrets: ");
System.out.println(new java.util.Date());
KeeperSecrets keeperSecrets = SecretsManager.getSecrets(options);

System.out.print("Before making API call: ");
System.out.println(new java.util.Date());

keeperSecrets.getRecords().stream().forEach(Keeper::print);
System.out.print("After at: ");
System.out.println(new java.util.Date());

Here's the output on an Oracle Linux Server machine:

$ java -cp ./lib/*:. Keeper ./ksm-config.json
Initializing KeeperSecrets: Thu Jun 30 11:06:49 CDT 2022
Before making API call: Thu Jun 30 11:36:09 CDT 2022
Test Title
TEST_BI
After at: Thu Jun 30 11:36:09 CDT 2022

Below is the thread dump during the run. This is the first time I've ever looked at a thread dump, but it looks like both "Finalizer" #3 daemon and "Reference Handler" #2 daemon are waiting on the same objects they themselves have the locks on?

]$ jstack -l 26856
2022-06-30 11:12:28
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.202-b08 mixed mode):

"Attach Listener" #8 daemon prio=9 os_prio=0 tid=0x00007f792c001000 nid=0x6a6e runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007f79600c2800 nid=0x68f2 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f79600b7800 nid=0x68f1 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f79600b5000 nid=0x68f0 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f79600b3000 nid=0x68ef runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f796007f800 nid=0x68ee in Object.wait() [0x00007f7950369000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000d6f88ed0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
        - locked <0x00000000d6f88ed0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)

   Locked ownable synchronizers:
        - None

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f796007d000 nid=0x68ed in Object.wait() [0x00007f7964126000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000d6f86bf8> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        - locked <0x00000000d6f86bf8> (a java.lang.ref.Reference$Lock)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

   Locked ownable synchronizers:
        - None

"main" #1 prio=5 os_prio=0 tid=0x00007f7960009800 nid=0x68e9 runnable [0x00007f79663bd000]
   java.lang.Thread.State: RUNNABLE
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:255)
        at sun.security.provider.NativePRNG$RandomIO.readFully(NativePRNG.java:424)
        at sun.security.provider.NativePRNG$RandomIO.ensureBufferValid(NativePRNG.java:526)
        at sun.security.provider.NativePRNG$RandomIO.implNextBytes(NativePRNG.java:545)
        - locked <0x00000000d77b2598> (a java.lang.Object)
        at sun.security.provider.NativePRNG$RandomIO.access$400(NativePRNG.java:331)
        at sun.security.provider.NativePRNG$Blocking.engineNextBytes(NativePRNG.java:268)
        at java.security.SecureRandom.nextBytes(SecureRandom.java:468)
        at com.keepersecurity.secretsManager.core.CryptoUtils.getRandomBytes(CryptoUtils.kt:80)
        at com.keepersecurity.secretsManager.core.SecretsManager.generateTransmissionKey(SecretsManager.kt:634)
        at com.keepersecurity.secretsManager.core.SecretsManager.fetchAndDecryptSecrets(SecretsManager.kt:842)
        at com.keepersecurity.secretsManager.core.SecretsManager.getSecrets(SecretsManager.kt:283)
        at com.keepersecurity.secretsManager.core.SecretsManager.getSecrets$default(SecretsManager.kt:282)
        at com.keepersecurity.secretsManager.core.SecretsManager.getSecrets(SecretsManager.kt)
        at Keeper.main(Keeper.java:18)

   Locked ownable synchronizers:
        - None

"VM Thread" os_prio=0 tid=0x00007f7960073800 nid=0x68ec runnable

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007f796001e800 nid=0x68ea runnable

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007f7960020800 nid=0x68eb runnable

"VM Periodic Task Thread" os_prio=0 tid=0x00007f79600c5800 nid=0x68f3 waiting on condition

JNI global references: 5

Additional stracktraces:

]$ jstack -F 26856
Attaching to process ID 26856, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.202-b08
Deadlock Detection:

No deadlocks found.

Thread 27246: (state = BLOCKED)


Thread 26863: (state = BLOCKED)


Thread 26862: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove(long) @bci=59, line=144 (Interpreted frame)
 - java.lang.ref.ReferenceQueue.remove() @bci=2, line=165 (Interpreted frame)
 - java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=216 (Interpreted frame)


Thread 26861: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=502 (Interpreted frame)
 - java.lang.ref.Reference.tryHandlePending(boolean) @bci=54, line=191 (Interpreted frame)
 - java.lang.ref.Reference$ReferenceHandler.run() @bci=1, line=153 (Interpreted frame)


Thread 26857: (state = IN_NATIVE)
 - java.io.FileInputStream.readBytes(byte[], int, int) @bci=0 (Interpreted frame)
 - java.io.FileInputStream.read(byte[], int, int) @bci=4, line=255 (Interpreted frame)
 - sun.security.provider.NativePRNG$RandomIO.readFully(java.io.InputStream, byte[]) @bci=13, line=424 (Interpreted frame)
 - sun.security.provider.NativePRNG$RandomIO.ensureBufferValid() @bci=198, line=526 (Interpreted frame)
 - sun.security.provider.NativePRNG$RandomIO.implNextBytes(byte[]) @bci=26, line=545 (Interpreted frame)
 - sun.security.provider.NativePRNG$RandomIO.access$400(sun.security.provider.NativePRNG$RandomIO, byte[]) @bci=2, line=331 (Interpreted frame)
 - sun.security.provider.NativePRNG$Blocking.engineNextBytes(byte[]) @bci=4, line=268 (Interpreted frame)
 - java.security.SecureRandom.nextBytes(byte[]) @bci=5, line=468 (Interpreted frame)
 - com.keepersecurity.secretsManager.core.CryptoUtils.getRandomBytes(int) @bci=10, line=80 (Interpreted frame)
 - com.keepersecurity.secretsManager.core.SecretsManager.generateTransmissionKey(com.keepersecurity.secretsManager.core.KeyValueStorage) @bci=28, line=634 (Interpreted frame)
 - com.keepersecurity.secretsManager.core.SecretsManager.fetchAndDecryptSecrets(com.keepersecurity.secretsManager.core.SecretsManagerOptions, java.util.List) @bci=87, line=842 (Interpreted frame)
 - com.keepersecurity.secretsManager.core.SecretsManager.getSecrets(com.keepersecurity.secretsManager.core.SecretsManagerOptions, java.util.List) @bci=14, line=283 (Interpreted frame)
 - com.keepersecurity.secretsManager.core.SecretsManager.getSecrets$default(com.keepersecurity.secretsManager.core.SecretsManagerOptions, java.util.List, int, java.lang.Object) @bci=12, line=282 (Interpreted frame)
 - com.keepersecurity.secretsManager.core.SecretsManager.getSecrets(com.keepersecurity.secretsManager.core.SecretsManagerOptions) @bci=10 (Interpreted frame)
 - Keeper.main(java.lang.String[]) @bci=104, line=18 (Interpreted frame)

Thanks.

Running ksm profile init --token ... produces a world readable keeper.ini in the current directory.

As stated in your own documentation (https://docs.keeper.io/secrets-manager/secrets-manager/about/security-encryption-model#client-device-configuration-file)

This file should be protected on your local filesystem. It contains keys can authenticate with the Keeper API and decrypt secrets that have been explicitly associated with the Application and Client Device.

Tested with:

$ lsb_release  -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04 LTS
Release:        22.04
Codename:       jammy

$ ksm version
Python Version: 3.10.4
Python Install: /usr/bin/python3
CLI Version: 1.0.14
CLI Install: /home/odormond/.local/lib/python3.10/site-packages/keeper_secrets_manager_cli
SDK Version: 16.3.4
SDK Install: /home/odormond/.local/lib/python3.10/site-packages/keeper_secrets_manager_core
Config file: /home/odormond/keeper.ini

$ ls -l keeper.ini
-rw-rw-r--  1 odormond odormond       645 Jul 15 09:32  keeper.ini

The keeper.ini should be created with an explicit mode of 0600 instead.

I believe there is an issue with the windows powershell version (as opposed to PowerShell Core) of this module. I am working with a fresh install of server 2019 and going through setup of the module. This is what I am seeing:

Windows PowerShell
Copyright (C) 2016 Microsoft Corporation. All rights reserved.

PS C:\Windows\system32> Set-PSRepository psgallery -InstallationPolicy Trusted
PS C:\Windows\system32> Install-Module -Name Microsoft.PowerShell.SecretManagement
PS C:\Windows\system32> Install-Module -Name SecretManagement.Keeper -RequiredVersion 16.0.4
PS C:\Windows\system32> Install-Module -Name Microsoft.Powershell.SecretStore
PS C:\Windows\system32> Register-SecretVault -Name LocalStore -ModuleName Microsoft.Powershell.SecretStore
PS C:\Windows\system32> register-KeeperVault -Name Keeper -LocalVaultName LocalStore -OneTimeToken *******
Storing Keeper Vault config KeeperVault.Keeper in Microsoft.Powershell.SecretStore Vault named LocalStore
Unable to find type [SecretManagement.Keeper.Client].
At C:\Program Files\WindowsPowerShell\Modules\SecretManagement.Keeper\16.0.4\SecretManagement.Keeper.psm1:29 char:13

• $result = [SecretManagement.Keeper.Client]::GetVaultConfig($OneTime ...

•         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  

  • CategoryInfo : InvalidOperation: (SecretManagement.Keeper.Client:TypeName) [], RuntimeException
  • FullyQualifiedErrorId : TypeNotFound

PS C:\Windows\system32> get-host
Name : ConsoleHost
Version : 5.1.14393.4583
InstanceId : ****
UI : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture : en-US
CurrentUICulture : en-US
PrivateData : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled : True
IsRunspacePushed : False
Runspace : System.Management.Automation.Runspaces.LocalRunspace

Hello,

according this documentation https://docs.keeper.io/secrets-manager/secrets-manager/integrations/hashicorp-vault i'm trying to deploy ksm plugin.

I'm unable to create the ksm path with this error :

/etc/vault/vault_plugins $ vault secrets enable -path=ksm  vault-plugin-secrets-
ksm
Error enabling: Error making API request.

URL: POST http://127.0.0.1:8200/v1/sys/mounts/ksm
Code: 400. Errors:

* invalid backend version: 2 errors occurred:
        * fork/exec /etc/vault/vault_plugins/vault-plugin-secrets-ksm: no such file or directory
        * fork/exec /etc/vault/vault_plugins/vault-plugin-secrets-ksm: no such file or directory

Hey there,

I am getting the error message in the title when trying to initialize a profile with: ksm profile init XX:...; I thought maybe I did get the secret token wrong in the first place, but when I tried to remove one random character from it, ksm bailed out with a different error message instead (which leads me to believe that the token might indeed be correct):

Error: Could not init the profile: Error: access_denied, message=Unable to validate application access

So what am I doing it wrong? Do you know what could be going on here? I even did a little bit of printf-debugging, but unfortunately it did not lead me anywhere interesting:

• Request URL: https://keepersecurity.com/api/rest/sm/v1/get_secret
• Request Headers: {'Content-Type': 'application/octet-stream', 'Content-Length': '310', 'PublicKeyId': '10', 'TransmissionKey': 'BK7fpjoKwHtRQO L15TwAcIhHqPmyq0cPJIRCXJubgeI3Ld+K4yHgCun8RI3TgPa+rEgqTl4MYDGd7ZK+KHtZCdhKeZqfmnGvDBRLFRkHEUHWrzNkdD+Lv3n5n4MqcKps3WwXyGtZzzYEkpuR1t+10Lpya7aAxCtpxvQoe90=', 'Authorization': 'Sig nature MEUCIFLGmtaF64ZtanwMnjTeI3v/Sq1K+7Ovz7DYFmk2APOSAiEAqafS7ll9ioK5eIK6CfA2zcvdO8i0ezPHuqKwSl2EuYo='}
• Response content: {"path":"https://keepersecurity.com/api/rest/sm/v1/get_secret, POST, p ython-requests/2.27.1","additional_info":"","location":"default exception manager - api validation exception","error":"access_denied","message":"Signature is invalid"}

Also, I tried running ksm profile init ... from different OSes (Windows, Ubuntu, and MacOS), but the error message is always the same.

Thanks in advance,
M.

Copied from keeper-sdk-dotnet repository
Keeper-Security/keeper-sdk-dotnet#103

PowerCommander doesn't seem to have the option to return a base64 configuration string when creating a Keeper Secret Manager Client. Only the one time access token is provided as a return value. Example:
Add-KeeperSecretManagerClient -App “someApp”

Whereas, the regular Commader CLI can do the following:
secrets-manager client add --app someApp --config-init b64

Could you please add the ability to create base64 configs in the PowerCommander Add-KeeperSecretManagerClient function?

Alternatively, is there a current method in PowerCommander to convert a one time access token into a base64 config?

Extra Context
I am evaluating the Secret Manager product as a possible purchase add-on for my organization (we are a current Keeper customer).

Here's what I am trying to accomplish:

During an automated Infrastructure Deployment...

Create Shared Folders, Applications and Device Clients with PowerCommander
Invoke Keeper Terraform to create secrets for the Shared Folders (your terraform module requires a configuration, which is why I need the b64 value)
Use SecretsManager PowerShell plugin to pull secrets as needed (also using b64 config value).

We're currently testing keeper software. we ran into this issue, wondering what it means exactly. Did some google search but nothing came up. At the moment, we have to keep cryptography dependency version at 3.4.7, hence latest version of keeper manager we can use is 16.4.0. Wondering if this might be the issue, and if there is a workaround:

Traceback (most recent call last):
  File "./test_keeper.py", line 4, in <module>
    secret = manager.get_secrets(['XXXXXXXXXX'])[0]
  File "/disk/software/lib/python-venvs/keeper/lib/python3.8/site-packages/keeper_secrets_manager_core/core.py", line 650, in get_secrets
    records_resp = self.fetch_and_decrypt_secrets(uids)
  File "/disk/software/lib/python-venvs/keeper/lib/python3.8/site-packages/keeper_secrets_manager_core/core.py", line 567, in fetch_and_decrypt_secrets
    decrypted_response_bytes = self._post_query(
  File "/disk/software/lib/python-venvs/keeper/lib/python3.8/site-packages/keeper_secrets_manager_core/core.py", line 459, in _post_query
    self.handler_http_error(ksm_rs.http_response)
  File "/disk/software/lib/python-venvs/keeper/lib/python3.8/site-packages/keeper_secrets_manager_core/core.py", line 552, in handler_http_error
    raise err
  File "/disk/software/lib/python-venvs/keeper/lib/python3.8/site-packages/keeper_secrets_manager_core/core.py", line 545, in handler_http_error
    raise KeeperError(msg)
keeper_secrets_manager_core.exceptions.KeeperError: invalid client version id: 16, mp16.2.0

winget is a powerful tool which can be used to automate the setup of developer machines. Having the KSM available on day one of imaging / laptop setup will be a big win for encouraging developers to use appropriate secrets storage and management.

The Keeper windows client is on winget already so adding this developer tool makes sense.

We have a library of .NET Framework 4.7.2 assemblies that require referenced assemblies be strong named, thus including the NuGet package Keeper.SecretsManager Version 16.4.0 results in the following error:

System.IO.FileLoadException: 'Could not load file or assembly 'SecretsManager, Version=16.4.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. A strongly-named assembly is required.

Based on this Microsoft documentation from Sept 2021, it recommends You should strong name your open-source .NET libraries. Strong naming an assembly ensures the most people can use it, and strict assembly loading only affects .NET Framework..
https://learn.microsoft.com/en-us/dotnet/standard/library-guidance/strong-naming

Please consider strong naming your .NET SDK assemblies.

It seems that the Linux binary is not compatible with Alpine Linux. We tried to run it in an Alpine-based container and we have

/bin/sh: ksm: not found

You can use the following Dockerfile as example:

FROM alpine:3.15

ENV OS=linux
ENV VERSION=1.0.8
ENV ARCHIVE_NAME=keeper-secrets-manager-cli-$OS-$VERSION
ENV LINK_TAR=https://github.com/Keeper-Security/secrets-manager/releases/download/ksm-cli-$VERSION/$ARCHIVE_NAME.tar.gz

WORKDIR /tmp

RUN apk add --no-cache curl
RUN curl -fsSL "${LINK_TAR}" | $SUDO tar -xz;

RUN ls -al
RUN ksm

The only way we found to run KSM cli on Alpine is to install it using PIP.

To give you some context, we wrote a small CircleCI orb to use KSM in our pipelines to ease our migration from SecretHub.
We have a command env-export which is used to create an environment variable with a secret. This command requires ksm to be installed. On any other Linux distribution, the installation from the binary is quick. However, on Alpine-based, this installation step takes roughly 1min which could make our pipeline longer.

Hi!

When using the Ansible 'lookup' plugin in playbooks that target a large number of hosts I seem to hit rate limits very quickly. For example, I have a playbook that targets about 50 hosts and it is causing the playbook to fail with the following error:

fatal: [example-host-1 -> localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ lookup(keeper_vault, uid='xxxxxxxxxxxxx', field='password') }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while running the lookup plugin 'keepersecurity.keeper_secrets_manager.keeper'. Error was a <class 'Exception'>, original message: Cannot get record: Error: throttled, message=get_secret. Cannot get record: Error: throttled, message=get_secret"}

How can this be avoided? It seems like an entirely new API call is made anytime a variable with the lookup is referenced per host, regardless if the record has already been looked up by a previous task. Are there any options to cache secrets that have already been obtained?

Thanks!

This issue was automatically created by the GitHub Action workflow Publish to Azure Pipeline Extension.

due to failure in run: 1387856119.

If error is related to access denied (expiration of Personal Access Token, aka PAT), here are the steps to generate a new one:
-Steps to generate are documented HERE

• Update "Password" field in record UID DJz3ilHBHIbIZqkTClDV5Q (located in shared folder named "Secrets Manager"