gridscale / gscloud Goto Github PK

Right now:

$ gscloud server ls --json | jq | head -n3
[
  [
    {

That's surprising and a bit too verbose. One list of objects is totally enough. Also check the other commands.

Reminder to delete github.com/gridscale/table after 0.4 is out. See #52

Creating a server with: gscloud server create --name test-1 --cores=1 --mem=1 --with-template="CentOS 8 (x86_64)" --hostname test-1

fails with
Server created: 1020a40a-f1d5-42da-8d65-877e9c159e94
Failed generating password: number of digits and symbols must be less than total length

according to @bkircher this is because if you leave the --password flag, a password is automatically generated - and the generated password doesn't pass the API requirements.

we expect to return the password, and the storage UUID

When gscloud is located in $PATH and executed in credential plugin mode with neither absolute nor relative path, but rather through search path lookup, path to gscloud is not correctly determined and stored in kubeconfig.

F.e. with gscloud located in /home/user/bin, /home/user/bin being in $PATH of user user and gscloud being executed with $ gscloud kubernetes cluster ... in current working dir /home/user, the resulting command inside kubeconfig will be /home/user/gscloud instead of /home/user/bin/gscloud

Add a flag --include-related or something that removes attached storages and IPv{4,6} objects.

Example:

$ gscloud server rm --include-related 71385c7f-0cfc-4159-9611-0cc87fba0b34

Maybe with a security yes|no question.

We should always return valid JSON if --json is given or nothing.

I know that we at least use KUBECONFIG but this is documented nowhere. So add a section on environment variables to man-page/help text.

Current:
gscloud uses a fixed expiration time (e.g., 1 hour). https://github.com/gridscale/gscloud/blob/develop/cmd/kubernetes.go#L195

Expected:
gscloud uses the expiration time returned by API and stored in the credentials.

The expiration time is already in the kubeconfig returned from the PaaS service, e.g.,

{
  "paas_services": {
    "1612364a-6c2a-4806-a0b2-50542dd23d6d": {
      "service_template_uuid": "e32b413a-de8d-4380-a826-f2b493480103",
      "resource_limits": [],
      "security_zone_uuid": "da39a112-c195-4178-9875-0843b0bcb708",
      "object_uuid": "1612364a-6c2a-4806-a3b2-50542dd23d6d",
      "name": "gsk-internal-test4",
      "credentials": [
        {
          "type": "kubeconfig",
          "kubeconfig": "apiVersion: v1\nclusters:\n- cluster:\n    certificate-authority-data: .....", 
          "expiration_time": "2020-07-17T07:24:01Z"
        }
      ],
      ....
}

i would love to get access to my linux-vms without going thru the webinterface.

(btw: where can i post feature request for the web-ui? with none of my mobile-browsers i'm able to use the console)

thanx and cheers.ivo

Minor thing here.

Suggestion: we should call fields, columns, references to a resource identifier "ID" rather than "UUID". I believe that this field is internally a UUID4 is actually an implementation detail and we should start referring to it as ID.

Objections? Okay for me to create a PR?

Create GH releases with goreleaser.

gscloud network --help

List, create, or remove networks.

Usage:
  gscloud network [command]

Available Commands:
  ls          List networks
  rm          Remove network

Global Flags:
      --account string   Specify the account used (default "default")
      --config string    Specify a configuration file (default "/Users/mharriss/Library/Application Support/gscloud/config.yaml")
  -h, --help             Print usage
  -j, --json             Print JSON to stdout instead of a table
      --noheading        Do not print column headings
  -q, --quiet            Print only IDs of objects

Use "gscloud network [command] --help" for more information about a command.

but it doens't seem that the "create" command has been implemented yet

so this task is to implement + add documentation

Add a --noheading -- Do not print column headings flag. It should turn off printing column header so that users can use grep and awk to extract data from the output more easily.

Missing:

gscloud iso-image create \
 --url=https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.7.0-amd64-netinst.iso \
 --name "Debian 10.7"

gscloud iso-image ls

gscloud iso-image rm <ID>

Please upgrade this library, it will make the code look much cleaner.

When server create fails somewhere in the middle of the whole thing it leaves objects behind that have already been created. This is unlike the panel which always cleans up nicely.

Panel uses POST /batch with

{
    "sub_calls": {
        "sub_calls": [
            {
                "call_type": "servers_post",
                "name": "ser1",
                "payload": {
                    "auto_recovery": true,
                    "availability_zone": null,
                    "cores": 1,
                    "hardware_profile": "q35",
                    "location_uuid": "8328a5bc-e66e-4edc-8aae-2e2bf07fdb28",
                    "memory": 1,
                    "name": "test-1"
                }
            },
            {
                "call_type": "storages_post",
                "name": "sto1",
                "payload": {
                    "capacity": 10,
                    "location_uuid": "8328a5bc-e66e-4edc-8aae-2e2bf07fdb28",
                    "name": "test-1",
                    "storage_type": "storage",
                    "template": {
                        "hostname": "test-1",
                        "password": "mysecret",
                        "password_type": "plain",
                        "template_uuid": "8d1bb5dc-7c37-4c90-8529-d2aaac75d812"
                    }
                }
            },
            {
                "call_type": "servers_storages_post",
                "name": "sto_x_ser",
                "payload": {
                    "bootdevice": true,
                    "server_uuid": "${SER1:SERVER_UUID}",
                    "storage_uuid": "${STO1:STORAGE_UUID}"
                }
            }
        ]
    }
}

to create a server. Not in API docs. You can observe this kind of behavior when user forgets to set --password in gscloud-server-create: new server object will be left while storage creation fails.

Ideas?

By placing servers in different availability zones, you can control the server's physical distance.

Add --availability-zone flag with 4 possible values:

• the empty string (default), do not consider availability when placing this server object
• a
• b
• c

Add a --quiet flag to server, network, and other resources sub-commands. If given it should only print the UUID of the resource. For example:

$ gscloud server ls --quiet
8bbc3917-a0c0-42c5-b3b7-2087cc33cb82
1eebb354-7cc7-4b4b-8e7a-65ffeccc7359

It only prints the UUID to stdout followed by a newline. No table header is printed.

This would enable us to write things like

$ gscloud server ls --quiet | while read s; do
    gscloud server off -f $s
done

To be released soon.

Work on a better README, esp. the first third needs some love. Make some simple examples that show how to work with this.

gscloud server create --name foo --cores=1 --mem=1 --with-template="CentOS 8 (x86_64)" --password=secret --hostname=foo

Creates a server with pc-i440fx-2.4 machine type (and no public net interface). It is tagged as "Standard" hardware profile in the panel. But actually haven't seen that before.

I would expect the default to be 'pc-q35-2.10' machine type which is tagged as "Q35 Chipset" hardware profile in the panel.

Not sure where this comes from, need to check gsclient-go and gridscale API docs.

After setting up kubectl using gscloud kubectl now complains that it cannot find gscloud in /home/dennis/Downloads for some reason (I copied gscloud to /usr/local/bin):

$ kubectl get nodes
Unable to connect to the server: getting credentials: exec: fork/exec /home/dennis/Downloads/gscloud: no such file or directory

The problem seems to be that the path is hard-coded into the kubernetes config file as command: /home/dennis/Dowloads/gscloud. When I replace this with command: gscloud then kubectl finds the copy in /usr/local/bin and no longer throws an error.

gscloud is using github.com/gridscale/table for the table rendering stuff. Upstream closed rodaine/table#12 so we can switch back to github.com/rodaine/table and remove our fork.

Has been deprecated since v0.7.0. A password is automatically generated.

After executing kubectl in several directories I noticed that a sub-directory "cache" is created every time. If gscloud wants to cache data it should probably create that folder in ~/.config/gridscale instead.

Something like

$ gscloud server on --name <name>
$ gscloud server on <id>

and

$ gscloud server off --name <name>
$ gscloud server off <id>

which would be ACPI and

$ gscloud server off --force <id>

would be hard power plug or something.

Version string and README indicate that this is beta status. I think with 0.4.0 we can say it's okay to use and not experimental anymore.

Implement gscloud storage set command.

Implement gslcoud-paas along with all its sub-commands like list, create, rm.

Make sure to sign release assets.

gscloud ip assign ADDR|ID --to-server=SERVER_ID
gscloud ip release ADDR|ID

maybe?

We're creating the gscloud cache directory in the current working dir at the moment:

$ pwd
/Users/thomas/tmp/gscloud
$ ls -la
total 0
drwxr-xr-x   2 thomas  staff    64B Jan 13 11:57 .
drwxr-xr-x  72 thomas  staff   2.3K Jan 13 11:57 ..
$ kubectl get nodes
NAME           STATUS   ROLES    AGE     VERSION
node-pool0-0   Ready    <none>   4d20h   v1.16.4
node-pool0-1   Ready    <none>   4d14h   v1.16.4
$ ls -la
total 0
drwxr-xr-x   3 thomas  staff    96B Jan 13 11:57 .
drwxr-xr-x  72 thomas  staff   2.3K Jan 13 11:57 ..
drwx------   3 thomas  staff    96B Jan 13 11:57 cache
$ find cache -ls
45820327        0 drwx------    3 thomas           staff                  96 Jan 13 11:57 cache
45820328        0 drwx------    3 thomas           staff                  96 Jan 13 11:57 cache/exec-credential
45820329       16 -rw-------    1 thomas           staff                5442 Jan 13 11:57 cache/exec-credential/1358261b-ca50-4c35-a2b1-xxx.json

Let's move that to an os-specific, standardized location. The freedesktop spec f.e. defines $XDG_CACHE_HOME for this (https://specifications.freedesktop.org/basedir-spec/latest/ar01s03.html). So on linux it would be: ~/.cache/gscloud

@bkircher do you have any ideas on how to make this as compliant as possible. Is there some library we can use?

i'd love to upload my existing kvm-raw vm-images directly as storage into gridscale. best way might be via gscloud instead of webinterface. this would simplify migration of existing vms massively.

thanx and cheers.ivo

Having multiple entries in config yaml and selecting with --account flag does not work. Always seems to select the default (or first) account.

Repro:

• add multiple entries in config file
• do a gscloud --account second-entry server ls

Expected:

• when no --account is given: always chose account 'default' if exists, or, if only one exists in config, that one
• when --account is given, that one
• when --account is given and that one does not exist in config, an error

Add command to list, remove, create IP -4 and -6 addresses. Add a way to assign them to servers.

Maybe something like

gscloud ip ls # lists all
gscloud ip ls -4 # lists only IPv4
gscloud ip ls -6 # lists only IPv6
gscloud ip rm <id> # delete by ID or
gscloud ip rm <address> # delete by address
gscloud ip create --reverse-dns=example.com --assign-to=<server id> # will pull a IPv4 from pool
gscloud ip create -6 --reverse-dns=example.com --assign-to=<server id> # same with IPv6
# supporting additional flags like
 --name=Example
 --failover=yes

In gscloud-server-create, --password is mandatory when creating with storage. This is not enforced by gscloud but API calls fail if user forgets to give a PW. Actually this is fine but annoyingly late.

Second thing is that --password is not used at all when --with-template is not given. Also makes sense since we don't need to set a root PW if we don't create a new storage from a OS template. But again, this is annoyingly silent on this user "error".

Lastly, maybe we just should remove --password flag oand not allow the user to set passwords themselves. For one thing, thse passwords wouldn't be recorded by shell history anymore (good) and user cannot forget to specify one (good). We could use something like https://github.com/sethvargo/go-password to always generate sufficiently secure passwords and return them after the server (erm, storage) has been created.

The error in this line L159 should be handled. And it is better, if the command can be rollbacked when there is an error returned. E.g. if linking a server and a storage fails, the server and the storage should be removed.

Add --auto-recovery. Default on... which mihgth mean that we just might a --no-auto-recovery flag.

Ditch git flow, use GitHub regular work flow. Will make releases much easier and we can automate with tools like GoReleaser.

TODO:

• update CONTRIBUTING.md
• update release-checklist.md
• add goreleaser YAML (#73)
• remote stale release branches from repository (origin/release/*)
• remove develop branch, make master default

Add --timeout flag so we can set timeout when dealing with gs resources. E.g:
./gscloud server --timeout 10m

Please always include ID in table output and remove --id flag, as it is not needed anymore.

Let's improve the error messages. We should be able to give more distinct information to the user, why something might have failed. E.g.:

newKubeConfig := fetchKubeConfigFromProvider(clusterID)
if len(newKubeConfig.Clusters) == 0 || len(newKubeConfig.Users) == 0 {
	fmt.Fprintln(os.Stderr, "Error: Invalid kubeconfig")
	os.Exit(1)
}
c := newKubeConfig.Clusters[0]
u := newKubeConfig.Users[0]

As a user this is quite confusing because it looks like there's something wrong with his local ~/.kube/config, which isn't the case at all. Furthermore, there should be separate messages for both cases, since both seem to have different originating errors. Something like

• "cluster not found"
• "invalid user or token" (if we can tell which is the cause be even more detailed)

This should clear out some initial headache and make the first use even more impressive (because gscloud is already awesome).

Works: $ ./gscloud off --force <id>

Doesn't work: $ ./gscloud off <ID>

During the night i wrote some gocode on my rasppi to see how to build for armv7, during tests i noticed one of my test server was still online

• I’m sure i turned it off the regular way via ./gscloud off <id>
• Gaved it a try on macOS and Manjaro it doesn't work without force on both systems, to shut down the server.

I must be missing something in the code somewhere.
Can someone else please test it as well?

See : 78388d4

Sometimes we print with fmt. and sometimes it is log.. Maybe redirect to whatever cobra offers here. Anyway, kind of in-transparent. Clean this up.

Something like

$ gscloud server events ID

would be nice.

I was thinking about something like

$ gscloud ssh-key ls

$ gscloud ssh-key add --name [email protected] --file ~/.ssh/id_rsa.pub

or even from stdin with

$ cat ~/.ssh/id_rsa.pub | gscloud ssh-key add

Allow listing templates with gscloud template ls.

I think it's neat to have the ability to display the version of a compiled binary. A version subcommand would be canonical to the docker/kubernetes ecosystem, so we should have one, too.

I'd say this can be done by reading the VERSION file on compile time and adding a basic function that prints out the content of mentioned file.