openebs-archive / maya Goto Github PK

Reference -
https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units

There are various shell files used in Maya.

• Create a make target to install shell based linting tool
  • i.e. Install https://github.com/koalaman/shellcheck
• Modify make test that will run linting operations on all shell files

vsm-stats should be interactive. The user has to run maya vsm-stats vol-name every time to see the stats.
You can use curses library for this maybe use https://github.com/nsf/termbox-go

Since the replica's and controllers may need to communicate beyond the boundaries of a docker cluster, we will be using "host" based networking scheme. Before a VSM is scheduled, it should be allocated with an IP address for exposing the iSCSI volumes and a set of replica IP/Address and ports.

NOTE - This will require thinking w.r.t workflow & handling of specs.

Possible workflow:

• Step 1

maya profile-list [prefix]

# Will list various profile available
# Profile are same as specs
# They will be made available during install time
# They may reside at /etc/maya.d/profiles/
# CLI will not have create, update profiles

• Step 2

maya vsm-create -name [options] [path-to-specs]

# path-to-specs can be profile or a custom specs
# path-to-specs is optional, i.e. maya will use the default profile
# name will be override the name provided in the specs at runtime
# *in near future* specs will be stored in git

• Step 3

maya vsm-list [prefix]

# This will list down the vsms that were created earlier

• Step 4

maya vsm-specs -name

# A future requirement
# This will dump the latest vsm specs from git
# This is required for editing the vsm specs before presenting it to maya for an update
# -name signifies that this vsm exists in git (& hence assumed to exist in server)

• Step 5

maya vsm-update -name [options] [path-to-specs]

# -name will be used to search for an existing vsm in git
# path-to-specs will be used to override the existing vsm specs
# path-to-specs can be optional; maya will use the existing specs from git
# diff of desired vs. actual vsm specs (via git) will be displayed
# a dry run of update will be displayed (via server)
# a prompt will be displayed for user confirmation

on the osh nodes or minion nodes, where the docker is installed. check what docker network plugins (cni) are configured. This can be added as a new command:

maya network status

Working on testing framework and writing *_test.go files to introduce test cases for better unit testing

Install the requirement components to convert a node into openebs node by installing:

• docker
• nomad/consul client
• configuration of persistent store
• configuration of vsm network interfaces

The location of the persistent store and vsm-network interfaces are specific a node. (For example a one node can have mellanox drivers and other in the same cluster can have intel nics. Though not an ideal case, but stuff happens, when putting together the available hardware into a common cluster.)

The details should be stored in the consul ( DB ) on a per node basis.

maya uses host-based networking for exposing storage. The should be an cli option to specify the network(s) that can be assigned for the VSMs. Whether a network can be used for public (accessing the storage) or private (used for replication). The scope of the networking can be vast, but as part of this issue, it would suffice if the user is able to set the network interface of the node and the range to be used.

The status should include the details like:

• IP address
• Node on which the VSM volume is exposed from
• Replica's and their current status (sync/stopped/lagging)

vsm-stats is showing zero in output, it should be relative.

add tests in vsm-stats

The ~/.profile in master-01 VM had both these entries.
The first entry is the right one, whereas the second entry seems to be from some hard-coded stuff.

export NOMAD_ADDR=http://172.28.128.8:4646
export NOMAD_ADDR=http://172.28.128.3:4646

maya cli should include an option to setup etcd-master. The command could be:

maya setup-etcd-master [args....]

arguments could include the listen/expose/advertise ip addresses, peer etcd-master addresses, certificates etc., Use the following steps as a sample for installing the etcd.

sudo apt-get install etcd

Config parameters could be:

ETCD_NAME=k8s-master-1
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS="https://172.28.128.10:2380"
ETCD_LISTEN_CLIENT_URLS="https://172.28.128.10:2379,http://localhost:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://172.28.128.10:2380"
ETCD_INITIAL_CLUSTER="k8s-master-1=https://172.28.128.10:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-0"
ETCD_ADVERTISE_CLIENT_URLS="https://172.28.128.10:2379"
ETCD_CERT_FILE="/etc/etcd/client.crt"
ETCD_KEY_FILE="/etc/etcd/client.key"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ca.crt"
ETCD_PEER_CERT_FILE="/etc/etcd/client.crt"
ETCD_PEER_KEY_FILE="/etc/etcd/client.key"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ca.crt"

Users should be able to install maya/openebs using the released version, as explained in Step1 of https://github.com/openebs/maya/wiki/Scenario-01---Install

Needs investigation on how release mechanism is enabled via gitub, linking of the released version to a site downloadable file.

Refers to issue - #8

Implement a new command that installs k8s flexvolume plugin on the k8s minion nodes.

say, the cli can be like:

maya k8s volume-plugin-install <plugin-name> <path-to-the-volume-plugin-file>

This should install the volume-plugin-file:

cp <path-to-the-volume-plugin-file> /usr/libexec/kubernetes/kubelet-plugins/volume/exec/openebs~<plugin-name>/<plugin-name>
sudo systemctl restart kubelet

Issue seen while testing:
https://github.com/openebs/openebs/tree/master/k8s-demo
Doc followed for setup:
https://github.com/openebs/openebs/blob/master/k8s-demo/run-mysql-openebs.md
Commands run inside omm-01 VM:
maya vsm-list
Output:

ID         Type     Priority  Status
demo-vsm1  service  50        running

maya vsm-stop demo-vsm1
Output:

==> Monitoring evaluation "4e35aca5"
    Evaluation triggered by job "demo-vsm1"
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "4e35aca5" finished with status "complete"

maya vsm-list
Output:
No running jobs
But the controller and replica are still up and running inside osh-01 VM.

User should be able to use maya to install the right version of the FlexVolume Driver on the kubernetes minion hosts, depending on the version of the kubernetes.

The command could look like:
maya install-driver --type=k8s --name=openebs-iscsi

The iSCSI FrontEnd requires the IP address to be allocated and specified during creation of VSM. There could be a case where the network subnet for storage can change in the environment. Maya should provide a mechanism to change the IP addresses allocated to VSMs.

maya version command should also include a note to check if there is a later version available then the one that is currently installed. This will involve contacting the github releases page to get the latest version.

maya should have option to generate certificates that can be used by a set of hosts (hostname and ip address) forming a kubernetes cluster. These certificates can be used by the k8s master and minion nodes for various services.

certificates and certificate authority files can be created using cfssl, the install and certificate generation steps are as follows:

On any linux client install cfssl

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
chmod +x cfssl_linux-amd64
sudo mv cfssl_linux-amd64 /usr/local/bin/cfssl
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x cfssljson_linux-amd64
sudo mv cfssljson_linux-amd64 /usr/local/bin/cfssljson

Create the files ca-config.json, ca-csr.json and client-csr.json files using the example provided below.

The commands to generate key are :

cfssl gencert -initca ca-csr.json | cfssljson -bare ca
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client-csr.json |  cfssljson -bare client

The above steps will generate : ca.pem, ca-key.pem, client.pem, client-key.pem
Rename them as follows to be used during configuration of etcd or k8s services.

mv ca.pem ca.crt
mv ca-key.pem ca.key
mv client.pem client.crt
mv client-key.pem client.key

For example, if the following hosts are used for creating kuberntes master and minion nodes:
k8s-master-1 : 172.28.128.1
k8s-node-1 : 172.28.128.1

The client-csr.json would be like:

{
  "CN": "kubernetes",
  "hosts": [
    "k8s-master-1",
    "k8s-node-1",
    "172.28.128.1",
    "172.28.128.2",
    "127.0.0.1"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "IN",
      "L": "BLR",
      "O": "Kubernetes",
      "OU": "Cluster",
      "ST": "Karnataka"
    }
  ]
}

The ca-csr.json can be:

{
  "CN": "Kubernetes",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "IN",
      "L": "BLR",
      "O": "Kubernetes",
      "OU": "CA",
      "ST": "Karnataka"
    }
  ]
}

And ca-config.json :

{
  "signing": {
    "default": {
      "expiry": "8760h"
    },
    "profiles": {
      "client": {
        "usages": ["signing", "key encipherment", "server auth", "client auth"],
        "expiry": "8760h"
      }
    }
  }
}

Steps to reproduce:

• Run the command

 maya setup-osh -self-ip=<self-ip> -omm-ips=<master-ip>

Output:

ls: cannot access '/etc/maya.d/scripts/': No such file or directory
Error executing cmd: exit status 2
Install failed: Bootstrap failed: Missing path: /etc/maya.d/scripts/
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease
Get:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Fetched 306 kB in 1s (177 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
unzip is already the newest version (6.0-20ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 31 not upgraded.
Cleaning old maya boostrapping if any ...
Fetching utility scripts ...
Fetching docker scripts ...
Fetching Mayaserver scripts ...
Fetching consul scripts ...
Fetching nomad scripts ...
Fetching Flannel scripts ...
Fetching mayaserver config templates ...
Fetching consul config templates ...
Fetching nomad config templates ...
Error executing cmd: exit status 1
Install failed: Error installing docker
OpenEBS Host setup failed

This is not a recommended configuration, but for development mode and quick prototyping, allow deploying OpenEBS maya master and storage host on a single machine/VM.

maya cli has been tested on ubuntu, test/investigate if the default release binary works on centos. If the binary doesn't work, check if it can be built from source.

Current maya cli based install commands are implemented in a very crude approach.
They are simple wrapper over shell scripts.
However, maya needs to do the overall deploy management (validate, monitor, upgrade, rollback, etc).
In addition it needs to do the deployment over VMs, Bare Metal, & AWS

This issue will serve as the tracking point for all these requirements.

Some References:

• https://github.com/coreos/coreos-baremetal
• https://github.com/coreos/kube-aws

On osh and omm, nomad is installed. On running maya version, verify if there is an update available to the installed nomad and consul versions.

Currently we have a command to add the node to the cluster

$ maya setup-osh -omm-ips -self-ip

There needs to be a command to drain the node from the cluster

$ maya drain-osh <host-id>

@AmitKumarDas @prateekpandey14 @kmova ##39

Refers to issue - #8

This is very similar to #13.

This issue should target linting golang code

Ref - https://blog.gopheracademy.com/advent-2016/some-tools-for-go-that-you-might-not-know-yet/

The primary objectives of the CLI will be:

• It should be intuitive to use
• It should be simple to use
• It should have less learning curve
• It will eventually be inclined to infrastructure storage domain
• It may make use of (i.e. talk to) Nomad CLI or Nomad agent server
• It should not change the way Nomad behaves
• It should compliment Nomad

Stop an existing VSM. This command is used to signal allocations to shut down for the given vsm-ID.

Assumptions:

• Nomad setup in a datacenter with 1 Server & 2 Clients

Steps:

• Create a job specs with 2 tasks
• 1 task will be for jiva
• 2nd task will be for jiva replica
• Each task will reside in different task groups
• Place appropriate constraints that enable placement of these tasks on different nodes

https://github.com/hashicorp/go-checkpoint

maya server depends on consul and nomad server being installed. User should be able to trigger the installation of the maya dependencies using "maya install".

• download the required version of the nomad/consul
• configure nomad/consul as primary server or attach to the existing cluster

error querying node status

Setup Details

• A single nomad server
• 2 nomad clients
• All belonging to a single datacenter as per nomad's nomenclature

GO_VERSION should be updated to new stable version 1.8

The volume creation API should be compatible with EBS.

• Update the launch arguments for replica containers
  • Pass a backing store (default is local directory)
• After a replica is launched, indicate to controller about the replica addition

Things like actual state, desired state, validations, etc should be taken into consideration.

The validations for all CRUD ops in maya api server will make use of 3rd party providers.
e.g. network provider, orchestrators, storage interface agents etc.

vt-x hardware acceleration disabled and unable to boot vm

The following document describes the way to install Flannel on OpenEBS hosts and connect to the etcd that is also managing the k8s - flannel.

https://docs.google.com/document/d/1Ige6SwecYLglxmRT4NDmDr1gYDkoWR_pRF4wiWNA_E8/edit?usp=sharing

The intent is that a command like the below should be support that will automate the steps to install flannel with the required args.

maya setup-nwd flannel <args....>

Add support for the following:

maya mount --type=iscsi --portal=< portal address > --iqn= < iqn > --mntpath=
maya unmount --type=iscsi --mntpath=

These commands can be used from the openebs drivers or just from the command line.

maya version, should be able to detect if the current node is configured as k8s master or minion and print the status of master or minion k8s services.

If the current node is master, print the status of - kube-apiserver, kube-controller-manager and kube-scheduler. On the minion node, print the status of kube-proxy and kubelet.

https://blox.github.io/