Would be nice to have a --print option
Could take the following values:

• verbose (print every kourou request response) util for dev/debugging
• default (current behavior)
• json (only the resut of the action)

For example: kourou sdk:execute --print json < ./snippet.js could return only the return of the snippet.js

Expected Behavior

Allow to kill running Kuzzle instances with Kourou.
Kourou should interactively ask which instance.

Possible Solution

kourou instance:stop

Expected Behavior

Using powershell the npm install should be successful

Current Behavior

npm install fails :

Context (Environment)

Windows > powershell

Expected Behavior

After an app:scaffold, the github actions added should run fine

Current Behavior

The functional-test action is errored

kuzzle_1         | npm ERR! Missing script: "prod"

Possible Solution

Add the appropriate "prod" script in the backend/package.json
Change "prod" to "dev" of line 12 in .github/actions/backend-functional-test/action.yml

           DOCKER_COMMAND="npm run dev" docker-compose up -d

To use dev in github runner, must add too

      - run: sudo sysctl -w fs.inotify.max_user_watches=524288

Steps to Reproduce

1. Create Github repository
2. run kourou app:scaffold backend
3. move the .github to the root of the repository
4. Push to trigger pipeline and check the action error

Context (Environment)

Kuzzle version: 2.18.1
Node.js version: 14.17.0
SDK version: 7.10.0
Kourou version: 0.20.6

https://github.com/kuzzleio/kourou/blob/98f2cbbbb9ab00032c596567b036b686a9937305/src/commands/app/start-services.ts#L76C66-L76C68

https://docs.docker.com/compose/migrate/

The command is now docker compose and not docker-compose anymore

• es:snapshot:dump <es-host> <repo_name> <snapshot_name>

• Dump the ES data into a snapshot

• es:snapshot:restore <es-host> <repo_name> <snapshot_name>

• Restore a specific snapshot from a specific repository to an ES

• es:snapshot:create-repository <es-host> <repo_name> <path_of_the_repo> --compress

• Create a repository into an ES cluster

• es:snapshot:list-repository <es-host>

• List repository known by the current ES

• es:snapshot:list-snapshot <es-host> <repo_name>

• List snapshot from a repository known by the current ES

Expected Behavior

When importing index/collection aliases, Kourou should create ES aliases instead of duplicating the aliased content.

Current Behavior

When exporting indexes with Kourou, if these indexes are aliases, or if they contain aliased collections, then the entire content is duplicated when re-importing the dumps, instead of having these aliases recreated.

Possible Solution

Detect if an index/collection is an alias or not. In case of an alias, some kind of dedicated json / file content should be written instead of dumping data.

When importing aliases, Kourou shouldn't care if their aliased sources exist: if they don't, the resulting ES error should simply be printed, and the import should be stopped.

In the same way Flutter provide the super convenient flutter doctor command to perform checks before development begin (Android SDK version, Android emulator version, Dart version...), we decided that we needed our own command.

Checks and information retrieved need to be discuss but IMO we need at least:

• Node.js:
  • Installed and configured (binary file in $PATH for example)
  • Version
• Kuzzle version check
• Elasticsearch:
  • Running
  • Version
  • Kernel settings (vm.max_map_count=262144)
• Redis:
  • Running
  • Version

Let's talk about it below 😄

This cause an error because we are separating the file from the extension with a split on character .

kourou vault:encrypt ./file/secrets.json

Current feature only allow fs snapshot, this should be improved to allow other ways of storing snapshots

Move current:

• kourou es:get INDEX ID-> es:indices:get
• kourou es:insert INDEX -> es:indices:insert
• kourou es:list-index -> es:indices:cat

Adds command to recover admin account by directly sending requests to Elasticsearch:

• reset user password (either with policies or set a new password)
• creates a new user with admin profile

Kourou v1.0

Here is the list of needed changes before releasing the v1.0

• api-key:create : The description flag should be an argument
• api-key:search : The filter flag should be named query
• app:start-services : Should be renamed in services:start and a flag should be added to specify Docker Compose project name
• config:diff : Bug when displaying differences between object values (with the --values flag)
• document:search : The aggregations flag should be added (maybe just name it aggs)
• *:export : Export commands should all be in the export: namespace (e.g. index:export => export:index)
• *:import : Import commands should be deleted and only the global import command should be kept
• * : All commands using Kuzzle API should accept the --arg flag to pass arbitrary arguments in the request
• instance:* : Instance commands should be removed since they are obsolete with kaaf
• redis:list-keys : There is a bug in the command when no matching key is found
• kourou sdk:execute : Should be renamed to kourou execute
• export:user : A --mappings flag should be added to export mappings (and the former mappings related commands should be removed)
• sdk:query : Should be renamed to api and the automatic inference of commands as API action should be removed

Why?

Elasticsearch recommends now to use Point in Time API instead of scrolls https://www.elastic.co/guide/en/elasticsearch/reference/7.17/point-in-time-api.html#search-slicing. Scrolls are huge memory consumers but Point in Time API only consume disk space and file descriptors (which are cheaper than memory)

Expected Behavior

existing param to pass onExistingUsers when loading fixtures thru cli

Current Behavior

no way to specify the flag

Possible Solution

Steps to Reproduce

Context (Environment)

Kuzzle version:
Node.js version:
SDK version:

Expected Behavior

this should work as same before

Current Behavior

throwing error about Help package

Possible Solution

Steps to Reproduce

1. install any node version. (I used Homebrew to install v14 and v16)
2. npm install -g kourou (I tested some previous version of package also, but same error)
3. kourou --help (or any command)

/usr/local/lib/node_modules/kourou/bin/run:5
const Help_showHelp = Help.prototype.showHelp;
                           ^

TypeError: Cannot read properties of undefined (reading 'prototype')
    at Object.<anonymous> (/usr/local/lib/node_modules/kourou/bin/run:5:28)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47

Context (Environment)

[email protected]
node v16.13.1
npm 8.1.2
macOS BigSur 11.6.1

If you need anything else please let me know.

paas:logs command is actually undocumented.

The command:
https://github.com/kuzzleio/kourou/blob/master/src/commands/paas/logs.ts

List of Kourou commands:
https://github.com/kuzzleio/kourou#commands

Expected Behavior

kourou sdk:execute < script.ts should work.

Current Behavior

It doesn't work.

Possible Solution

import node-ts/register before the code eval.

Description

See kuzzleio/sdk-javascript#596

Expected Behavior

I want to install Kuzzle locally on my windows 10 computer. However, it doesn't give a proper error description, why I can't install it. I followed the official instructions to run Kuzzle.

Current Behavior

I got this error:

node:events:491
      throw er; // Unhandled 'error' event
      ^

Error: spawn rm ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:285:19)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
    at ChildProcess._handle.onexit (node:internal/child_process:291:12)
    at onErrorNT (node:internal/child_process:483:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'spawn rm',
  path: 'rm',
  spawnargs: [ '-rf', '/tmp/project-templates' ]
}

after the kourou app:scaffold playground command.
No files/folders are created.

Possible Solution

Steps to Reproduce

The steps are in the official website: https://docs.kuzzle.io/core/2/guides/getting-started/run-kuzzle/

Context (Environment)

My versions are: kourou/0.25.0 win32-x64 node-v19.0.0

Adds a new command to spawn only services needed by Kuzzle

Since this PR, Kuzzle uses aliases instead of directly using indices when dealing with ES. Even though both are very similar for now, they may evolve independently in the future. That is why kourou should provide a command to list aliases similar to the one currently listing indices.
Once this new command has been implemented, Kuzzle could suggest it in the doc section Internal Representation

We have too many formats: Kourou, Kuzzle and the bulk format.

We should use Kuzzle format for mappings, users, profiles and roles. Kourou can try to guess which file correspond to which kind of data (frequency analysis, pattern recognition)

For fixtures, if there is less than 2000 documents (for example), then we can use the standard fixtures format, otherwise Kourou will use JSONL format.

This command should use security:getUserMapping and security:updateUserMappings.

The user mappings dump file should be auto loaded by the kourou import command

app:scaffold command use nodemon to watch for changes.

But nodemon is not part of devDeps inside the package.json

Description

kourou vault:generate <prefix>

Using kourou to dump my collection stored on elastic search the kourou cli fails on big collections.
The command that are failing are:
kourou index:export
and
kourou collection:export

Expected Behavior

I am supposed to be able to dump the whole collection directly into a JSONL format file

Current Behavior

The command kourou collection:export fails while exporting the data to a JSONL format file.
It seems the scroll API closes itself before the whole collection has been retrieved.

Possible Solution

I suspect that the sharding starts to fails on elastic search on big collections.

Steps to Reproduce

Try to dump a collection that has more than 17000 documents insides, and export it to a JSONL file using the kourou collection:export command

Context (Environment)

Trying to extract my data stored using kuzzle framework into jsonL files.

Kuzzle version: 2.18.1
Node.js version: 16.15.1
SDK version: 7.10.1

Overview

Since we implemented Elasticsearch getDocument inside the paas, we wanted to add a way to retrieve all files via kourou

• The command will be named paas:elasticsearch:dump and will use available API call from the paas backend to retrieve all documents.

Expected Behavior

kourou index:export tenant-[group]-[name]
Should be able to dump collections with more than 2000 documents.
Same thing with kourou collection:export tenant-[group]-[name] [my-collection].

Current Behavior

I get the following error while dumping a collection with 13398 documents:

[X] Error stack: 
KuzzleError: Non-existing or expired scroll identifier.
       at WebSocketProtocol.query (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/kuzzle-sdk/src/protocols/abstract/Base.js:109:23)
       at Kuzzle._timeoutRequest (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/kuzzle-sdk/src/Kuzzle.js:742:34)
       at Kuzzle.query (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/kuzzle-sdk/src/Kuzzle.js:575:21)
       at DocumentSearchResult.next (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/kuzzle-sdk/src/core/searchResult/SearchResultBase.js:41:33)
       at JSONLDumper.dump (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/lib/support/dump-collection.js:105:43)
       at processTicksAndRejections (node:internal/process/task_queues:96:5)
       at async CollectionExport.runSafe (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/lib/commands/collection/export.js:30:9)
       at async CollectionExport.run (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/lib/common.js:98:17)
       at async CollectionExport._run (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/@oclif/command/lib/command.js:43:20)
       at async Config.runCommand (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/@oclif/config/lib/config.js:173:24)
       at async Main.run (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/@oclif/command/lib/main.js:27:9)
       at async Main._run (/home/skymer/.nvm/versions/node/v16.16.0/lib/node_modules/kourou/node_modules/@oclif/command/lib/command.js:43:20)
Error status: 404
Error id: services.storage.unknown_scroll_id (https://docs.kuzzle.io/core/2/api/errors/error-codes/services)
 [X] undefined

(I've tried with node 14 & updating kourou & using batch-size, but it has a limit)

Steps to Reproduce

1. Import a collection with more than 2000 documents (see documents.jsonl.txt and mappings.json.txt) into your kuzzle: kourou collection:import [../../]
2. Export the collection kourou collection:export tenant-[group]-[name] [collection]

Context (Environment)

Trying to export a collection in order to backup data
NodeJS 14 / 16
Kuzzle: 2.19.5
Kourou : kourou/0.24.2 linux-x64 node-v14.20.1

Could be nice to let kourou generate a vault-key
kourou vault:generate-key --prefix
=> ku_<prefix>_<key>

Description

The kourou import command walk files and directories On OSX there is an error because of the .DS_Store files, Kourou try to open and import them.

We should only open .json files

With the arrival of KaaF and Kourou's app module, it would be cool to provide framework mode users with a simple way to package their application via CLI. The generated artifact could be a Docker image or an archive such as ZIP or TAR.GZ.

Targeted UI:

$ kourou app:package --<docker|archive>  <OUTPUT: image name|archive filename> 

On Mac, app:scaffold is stuck on Installing dependencies (this can take some time) forever...

... Or maybe it's just soooooooo long we need some feedback in the stdout to be reassured it's not stuck.

The -v option of kourou instance:spawn only allow to choose between both 1 and 2 Kuzzle major version .
It should be named like kuzzleVersion or majorVersion or allow user to specify a full version like 2.3.3

Make es:migrate test all indexes from the source don't exist on the destination before processing

Description

When the batch size is too big, then the time to download a page of documents may excess the scroll TTL and thus the export fail.

The scroll TTL should be configurable