Currently the database initialization is executed by an ansible scripts run by a job. This is very slow to run and consumes memory.

We want a python module (nuvolaris/couchdb_init.py) in the nuvolaris-operator that performs the same task in python.

In order to implement follow those instructions

• checkout nuvolaris with --recurse-submodules
• poopen the container then the workspace of the nuvolaris-operator,
• open the terminal and type task start-couchdb

Now you can execute task init-couchdb that is currenly an example only showhing how to reuse templates and connect to couchdb

You have to re-implement in python (using requests and jinja2) the logic in the ansible script orig/initdb.yml

Note that all the templates used by those scripts are available in nuvolaris/templates.

All the secrets should come from an YAML (that can be modified) so put them in tests/operator-obj.yaml

So far we found the following pain point that should be documented properly in the developer guide

• Under Linux Ubuntu you should NOT use docker.io but download a docker-ce of the right version
• Under Linux because of docker limitations you do better to develop with an UID of 1000 and never as root
• Explaning how to use properly gitmodules and how to contribute (i.e. create a new remote, NOT modifying .gitmodules)
• Note that you should NOT use clone with vscode for the top level project becuase it does not recurse subfolders
• In general distinguish working with a single subproject (that is the easy way) and how to work with the toplevel project that puts everthing together with submodules (the hard way)

┆Issue is synchronized with this Jira Task by Unito

Change hello in ciao in nuvolaris-hello

The script should include a simple command to launch and destroy, and a configuration file to create and destroy an EKS cluster in Kubernetes to test nuvolaris.

┆Issue is synchronized with this Jira Task by Unito

Contribute to the nuvolaris-controller a demo using MongoDB

Modify this demo.

In detail, copy the demo-couchdb folder to demo-mongodb, add to deploy the scripts to launch also mongodb replace the configuration file in the image in /home/etc/standalone-kcf.conf with a configmap so, to task deploy create a cluster using couchdb.

It may be necessary to fix OpenWhisk as it could not be supported! But simply open the bugs in this case, do not fix them.

┆Issue is synchronized with this Jira Task by Unito

nuv wsk is supposed to use ~/.nuvolaris/.wskprops by setting the environment variable WSK_CONFIG_FILE before executing the body of wsk.
It does not look it works.
Furthermore in the ~/.nuvolaris/.wskprops there is APIHOST=https:// and should be http

Add the devcontainer support, build a kubernetes cluster and run a basic operator in it.
Document development procedure.

The cli setup currently only checks that the operator is up and running.
Howevr the operator starts the controller and in turns initializes some action.

We need to add a check that waits until openwhisk is ready to answer and a check that deploying functions works.

The check that waits until openwhisk is ready can be done executing wsk namespace until it does not return an error.

Then it should deploy a simple action and execute it.

A simple action can be function main(args) { return { "body":"hello"} } and the operation to execute are

• repeat wsk action create hello hello.js until it succeeds
• test that wsk action invoke hello returns hello

The operator should receive CRDs of type Static and deploy them.

Deployment should happen creating a configmap from the crd, then an http server that will expose the content to the ingress controller.

Deployment should happen writing the content to s3-compatible buckets, since Kubernetes objects (hence CRDs) are limited to 1MB so they are not fit for deploying static content.

There will be an s3 bucket for each user.

There will be a nuv s3 put command able to write in that bucket.

All we need to do is to generate the taskfile that will invoke that command after buinding.

running nuv devcluster create will create a kind cluster

note that

• the cluster created should be named "nuvolaris", and if it is there it should say it is already running
• it should run some preflight tests that checks there is docker, and there is enough memory available (you can take those preflight tests from io-sdk)
• it should create first a directory in ~/.nuvolaris and write some files in it, most notably the kind.yaml to be used by kind, but other will follow), use this to get the home directory https://github.com/mitchellh/go-homedir
• use go:embed to embed files

Also

nuv devcluster destroy will destroy it

please also add unit tests for the various functions and use the "dryrun" method so you can output the various commands generated, unit testing should only test the commands and the files generated are correct

if we need integration tests, let's add separately, i plan to use jest for integration tests (I mean checking that it actually created a custer and it works).

This task should stop at cluster creation, but to be complete we will have also to deploy the operator that we will do once I complete it

Nice to have. nuv setup --devcluster creates the devcluster with a single command and deploy it...

Fork apache/openwhisk then create a top level nuvolaris-controller that pull the repo and builds and publish in github docker registry. Builds and publish standalone OpenWhisk, to be used by the controller.

This issue should inagurate the in nuvolaris-controller repo.

I was playing around with the Dockerfile and noticed a small oversight.
In the Dockerfile for the remote container, the path used with dpkg to install delta is missing the path "/tmp/".
line 65: sudo dpkg -i "$FILE" ; rm "/tmp/$FILE"
The installation fails because it's not finding the file (the error is also visible in the build logs in github).

Add it to the testing repo

It would be nice (as requirement) for ML datapipelines having the support inside nuvolaris for durable functions using as
backend (https://docs.temporal.io/blog/workflow-engine-principles/) for stateful function. This will leverage nuvolaris not only as
serverless but also as ETL platform since already we've Red Panda and other tools as a platform.

In nuvolaris-cli write a test (using jest) that

• deploy a demo app (it should excercise action cration and also sequences and triggers)
• verify that everything works

add the github actions to execute a test against

• a locally build cluster created with nuv setup --devcluster
• a cluster running microk8s
• an aws cluster running eks

We can assume those clusters are up-and-running, parameters to access them are stored as GitHub Secrets

Build an installer using ansible that builds a set of virtual machines in a server to install Kubernetes in it.

┆Issue is synchronized with this Jira Task by Unito

Use nuvolaris-hello

Contribute to the nuvolaris-controller a demo using RedPanda

Modify this demo.

In detail, copy the demo folder to demo-redpanda, add to deploy the scripts to launch kafka from here, replace the configuration file in the image in /home/etc/standalone-kcf.conf with a configmap so, to task deploy create a cluster using redpanda.

It may be necessary to fix OpenWhisk as it could not be supported! But simply open the bugs in this case, do not fix them.

┆Issue is synchronized with this Jira Task by Unito

Contribute bash script to the [nuvolaris-testing]
(https://github.com/nuvolaris/nuvolaris-testing) folder cloud/k3s a script that creates in AWS a multi-node cluster based on k3s.

The script should be unattended and assume keys for AWS are in environment variables.

Note there is the k3sup tool that apparently creates a single node k3s cluster. We do now know what is the best way to setup a k3s based cluster.

┆Issue is synchronized with this Jira Task by Unito

Contribute to the nuvolaris-controller a demo using Kafka-Kraft

Modify this demo.

In detail, copy the demo folder to demo-kafka, add to deploy the scripts to launch kafka from here, replace the configuration file in the image in /home/etc/standalone-kcf.conf with a configmap so, to task deploy create a cluster using couchdb.

It may be necessary to fix OpenWhisk as it could not be supported! But simply open the bugs in this case, do not fix them.

┆Issue is synchronized with this Jira Task by Unito

Contribute to the nuvolaris-controller a demo using CouchDB

Modify this demo.

In detail, copy the demo folder to demo-couchdb, add to deploy the scripts to launch couchdb from here, replace the configuration file in the image in /home/etc/standalone-kcf.conf with a configmap so, to task deploy create a cluster using couchdb.

┆Issue is synchronized with this Jira Task by Unito

Contribute an ansible script to the nuvolaris-testing repo that builds in a test server accessible with ssh an OKD Kubernetes cluster to be usable as a test environment. It should export the kubernetes cluster access port and make available the key in /etc/kube/config, so with an scp server:/etc/kube/config ~/.kube/config is possible to retrieve it and access the cluster with kubectl

┆Issue is synchronized with this Jira Task by Unito

Implements the requirement

https://github.com/nuvolaris/nuvolaris-cli/blob/main/README.md#static-frontend

Nuv must scan the folders and create an yaml file the static contents.

Format of the YAML:

namespace: <namespace>
path: <path>
resources:
  <file1>: <body1>
  <file2>: <body2> 
  ...

Where the <fileN> is the relative path of under the various web folders.

The <bodyN> file is the base64 encoded of the corresponding file.

Those files will be crd to be deployed to Kubernetes.

The script should include a simple command to launch and destroy, and a configuration file to create and destroy an AKS cluster in Kubernetes to test nuvolaris.

Please follow this guide

┆Issue is synchronized with this Jira Task by Unito

Contribute an ansible script to the nuvolaris-testing repo that builds in a test server accessible with ssh a Kubernetes witk MicroK8S cluster to be usable as a test environment. It should export the kubernetes cluster access port and make available the key in /etc/kube/config, so with an scp server:/etc/kube/config ~/.kube/config is possible to retrieve it and access the cluster with kubectl

┆Issue is synchronized with this Jira Task by Unito

The configurator currenty has an user interface that writes on mongodb, creating users and associating a number of namespaces to it

We need to:

• change it to write in couchdb, creating proper namespaces and users in couchdb

• there is not an user concept in openwhisk, only namespaces and keys, so we will have to add our user table

If you try to use the subcommands nuv kind --name or nuv wsk action invoke -r you cannot because the first level wsk is eating the parameters so you have to use nuv wsk -- action -r.

This should not be necessary and it should be possible to do nuv wsk ... -r or nuv kind ... --name witouth the --

Add a basic subproject using svelte and tailwind and run it with devcontainers.

Add a command version that prints the value of CLI_VERSION.

We need to start the configurator.

The idea is:

• a svelte
• using tailwind
• written with mocks with mirage

We need a frontend adding users and namespaces.

Each user has an username in the form of an email and a password and can have a number of namespaces associated to it (at least one, but it can be more than one)

You should be able to add, remove, edit users and associated namespaces.

You can mock the API as you like as it will implemented later based on the prototype.

You can decide for the UI, but please use tailwind.

┆Issue is synchronized with this Jira Task by Unito

Implement the nuv setup command that

• deploys the nuvolaris operator as in the task deploy here
• setup the wskprops file as in the task wsk here

so you can start using nuv wsk to create actions as in the task test here

Some notes

• you should NOT use the kubectl but talk directly with the kubernetes client so nuv does not have any dependencies on kubectl
• you should first check if there is already a cluster configured by other means (for example with docker desktop) trying to authenticate using local credentials and the create a nuvolaris namespace. If this fails the write an error message suggesting to use nuv devcluster to create a local cluster first
• you should
  • create a namespace first
  • deploy the operator including a service account, a CRD and a RBAC cluster as in the examples
  • wait for the operator to be running
  • create a wsk cluster creating an instance of the crd

Also note that the example uses Kustomize to specify the current image to run. Since we do not want to depend on kubectl and we will use kube descriptors only once, as once deployed the operator we do not interact any more with kubernetes but only with the operator, just create a static kubectl files in ~/.nuvolaris and apply it.

Finally the wskprops file is by default in ~/.wskprops but to make things and not conflict with wsk and others, create one in ~/.nuvolaris/.wskprops. Then when running nuv wsk we set the WHISK_CONFIG_FILE environment to point to the right one.

┆Issue is synchronized with this Jira Task by Unito

Crete devcontainers, setup go and create a basic cli that will start the operator

Every file with code requires a proper license header.
Furthermore we should add a test in CI that check all the files have a proper license header.

Select and deploy an ingress controller that works in kind (and others) to publish static resources and include to the operator.

Each runtime in OpenWhisk has a basic test suite written in scala like this in node. They customize this

Those tests start a runtime image, intialize it and then run a set of validation tests. We need to replace those basic tests with test written in Javascript with Jests as running them with Scala is awkward and slow (to say the least). The protocol a runtime implements is described here

So contribute to the nuvolaris-testing repo a test suite written in jest that remplements the basic nodejs tests.

┆Issue is synchronized with this Jira Task by Unito

After deployment nuv wsk action list fails.
Looks like the WSK_CONFIG_FILE is set only when doing the setup.

Every action need to receive environment variables to find his mongodb database.

This task requires implemeting the needed changes to OpenWhisk so a database configured by the configurator will be accessible by the corresponding actions.

Add to the operator the ability to deploy MongoDB.

You should:

• add the deployment descriptors under deploy/mongodb (you can use the operator)
• write the python code to deploy it (no parameters from crd for now - they will come later) (the code is very simple just copy the one for the standalone deployment and adapt it)
• expose mongodb as nodeport 30017

note that the kind in lastest developer image already exposes the right port

Write a './setup.sh' that will install

• nodenv
• goenv
• pyenv

and check prerequisites

┆Issue is synchronized with this Jira Task by Unito

Each runtime in OpenWhisk has a basic test suite written in scala like this for python.

Those tests start a runtime image, intialize it and then run a set of validation tests. We need to replace those basic tests with test written in Javascript with Jests as running them with Scala is awkward and slow (to say the least).

So contribute to the nuvolaris-testing repo a test suite written in jest that reimplements the basic nodejs tests.

┆Issue is synchronized with this Jira Task by Unito

Build and publish a set of OpenWhisk runtimes, all based on actionloop, to be used by the OpenWhisk runtime buil5 with #2

┆Issue is synchronized with this Jira Task by Unito

In order to provide an environment that works for everyone build an image to be used as a development environment.

The image should work on Intel and ARM 64.

Furthermore, customize the environment of the various subprojects to work with Run in Container feature of VSCode.

Currently .github actions just runs on every push, building an image for each push.
The image is named neo-latest.

We should improve to do that when releasing (tagging) and use the tag to build the image.

Creating a basic operator using kopf that starts standalone Openwhisk, the one built with #2

This is the issue number 1 and it is symbolic as usually you close it when you reach the goal of your project. It will stay open until we reach 1.0.

Requirements for Nuvolaris 1.0:

• supports at leasts EKS, AKS, GKE, Microk8s, K3s, OKD
• installs in Kubernetes with the single command: curl -sL get.nuvolaris.org | kubectl apply -f -
• provides out of the box essential services like databases, object storage and cache

┆Issue is synchronized with this Jira Task by Unito

The outcome is a design document to be implemented:

• define the deployment standard for backend and frontend
• define the architecture to receive and deploy the statics

Each runtime in OpenWhisk has a basic test suite written in scala like this in go.

Those tests start a runtime image, intialize it and then run a set of validation tests. We need to replace those basic tests with test written in Javascript with Jests as running them with Scala is awkward and slow (to say the least).

So contribute to the nuvolaris-testing repo a test suite written in jest that remplements the basic nodejs tests.

┆Issue is synchronized with this Jira Task by Unito

The operator must deploy the database couchdb

It will receive a CRD with all the credentials from the CLI.

OpenWhisk has configuration file that includes database access credentials.

You need to deploy couchdb and openwhisk with the parameters to access the database.

So the operator must:

• Start CouchDB
• Create a configmap with database credentials (specified in the CRD)
• Change the configurations of openwhisk with a configmap to use the credentials and start
• Start Openwhisk with new credentials