Right now it is only possible to use one marathon framework instance with the marathon cli command. Compare https://github.com/mesosphere/dcos-cli/blob/master/dcos/marathon.py#L31.

As users might install multiple marathons, the cli should support this, i.e. switching via the command line.

if a new master is elected, dcos CLI reports an error because core.mesos_master_url does not point to the correct master.

I get an error when I try to update an app if more than one version exists:

Steps to reproduce:
1.) Create an app with one instance
2.) Execute dcos marathon app update basic-1 instances=4
3.) Wait till scaling is done
4.) Execute dcos marathon app update basic-1 instances=5

Current result:
a.) First update works as expected and app is being scaled
b.) Second update causes an error:

Error: Additional properties are not allowed ('versionInfo' was unexpected)
Value: {"backoffFactor": 1.15, "backoffSeconds": 1, "labels": {}, "cpus": 0.1, "requirePorts": false, "instances": 5, "healthChecks": [], "disk": 0.0, "id": "/basic-1", "maxLaunchDelaySeconds": 3600, "version": "2015-10-13T08:48:27.884Z", "env": {}, "uris": [], "mem": 16.0, "dependencies": [], "storeUrls": [], "versionInfo": {"lastScalingAt": "2015-10-13T08:48:27.884Z", "lastConfigChangeAt": "2015-10-13T08:48:08.480Z"}, "upgradeStrategy": {"maximumOverCapacity": 1.0, "minimumHealthCapacity": 1.0}, "cmd": "sleep 5000", "executor": "", "ports": [10001], "constraints": []}

Expected result:
Latest version of the app is being updated when executing step 4.)

Issue detected when testing mesosphere/marathon#2409

Probably requires adding a path to mesos-cli. I'll give that a try, but instructions may need to change.

ImportError: Entry point ('console_scripts', 'dcos-registry') not found

on: poc-dcos-cli branch

Right now the CLI is asking the package manager for everything and doing the filtering itself. This needs to change so that it is delegated to the package manager.

dcos list
Traceback (most recent call last):
File "/usr/local/bin/dcos-list", line 9, in
load_entry_point('dcos==0.0.1', 'console_scripts', 'dcos-list')()
File "/opt/demo/dcos-cli/dcos/cli.py", line 65, in wrapper
return fn(_args, *_kwargs)
File "/opt/demo/dcos-cli/dcos/log.py", line 28, in timer
return fn(_args, *_kwargs)
File "/opt/demo/dcos-cli/dcos/cli.py", line 43, in wrapper
return fn(cmd_args, _args, *_kwargs)
File "/opt/demo/dcos-cli/dcos/cmds/list.py", line 27, in main
tb.add_row(fw["name"].split('-', 1))
File "/usr/lib/python2.7/dist-packages/prettytable.py", line 818, in add_row
raise Exception("Row has incorrect number of values, (actual) %d!=%d (expected)" %(len(row),len(self._field_names)))
Exception: Row has incorrect number of values, (actual) 1!=2 (expected)

on: poc-dcos-cli branch

Are there plans to implement a ps function like there was in mesos-cli?

https://github.com/mesosphere/mesos-cli#ps

This was a great way of seeing high level statistics on CPU and memory quickly for running tasks.

dcos cli is looking for marathon on port 80 instead of 8080 and path /marathon/v2/info instead of /v2/info

[ec2-user@ip-172-31-18-67 cli]$ dcos package install hdfs
In order for HDFS to start successfully, it requires a minimum of five nodes; each with at least 2 CPU shares and 8GB of RAM available for use by the HDFS Service.
Note that the service is alpha and there may be bugs, including possible data loss, incomplete features, incorrect documentation or other discrepancies.
Continue installing? [yes/no] yes
Installing Marathon app for package [hdfs] version [0.1.7]
URL [http://ip-172-31-23-64.ec2.internal/marathon/v2/info] is unreachable: HTTPConnectionPool(host='ip-172-31-23-64.ec2.internal', port=80): Max retries exceeded with url: /marathon/v2/info (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x32b6e10>: Failed to establish a new connection: [Errno 111] Connection refused',))

During manual tests we noticed that installed package information is not getting merged.

dcos package list
NAME       VERSION  APP              COMMAND    DESCRIPTION
cassandra  0.2.0-1  ---              cassandra  Apache Cassandra running on Apache Mesos
cassandra  0.2.0-1  /cassandra/dcos  ---        Apache Cassandra running on Apache Mesos
marathon   0.11.1   /marathon-user   ---        A cluster-wide init and control system for services in cgroups or Docker containers.

Right now we are ignoring the response status code we need to make sure that we deal with this globally across all of the endpoints.

to give human readable validation errors

Regression introduced in #8.

After setting marathon.url in the dcos cli config (e.g. to access a marathon-user instance, compare https://github.com/mesosphere/dcos-cli/issues/396), the package command will use this as well. This is very confusing for the user.

In addition, services installed this way

• will not show up in the DCOS UI
• will not have a working admin router /service/<framework-name> endpoint.

Just did a dcos package instal kubernetes. According to http://docs.mesosphere.com/services/kubernetes, dcos kubectl should now be available as a sub-command.

It isn't.

I would like to have a way to ssh into the environment of a certain task rather than just a node itself. For example, to ssh in to the environment of any/every elasticsearch node to install plugins etc.

For docker, this would mean using 'docker exec', but for the mesos isolator, this would just be about dropping you in the right folder.

We need to figure out how we are going to support the --endpoint for package list.

I have next issue:

$ bin/dcos-kubectl kubectl --info
dyld: Library not loaded: @executable_path/../.Python
  Referenced from: <HOME>/.dcos/subcommands/kubernetes/env/bin/python2.7
  Reason: image not found
[1]    40818 trace trap <HOME>/.dcos/subcommands/kubernetes/env/bin/dcos-kubectl kubectl --info

$ ./bin/dcos     
dyld: Library not loaded: @executable_path/../.Python
  Referenced from: /Users/miry/.dcos/subcommands/kubernetes/env/bin/python2.7
  Reason: image not found
dyld: Library not loaded: @executable_path/../.Python
  Referenced from: /Users/miry/.dcos/subcommands/kubernetes/env/bin/python2.7
  Reason: image not found
Traceback (most recent call last):
  File "/Users/miry/projects/oos/dcos/bin/dcos-help", line 11, in <module>
    sys.exit(main())
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/dcoscli/help/main.py", line 17, in main
    return _main()
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/dcoscli/main.py", line 112, in wrapper
    result = func(*args, **kwargs)
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/dcoscli/help/main.py", line 31, in _main
    return cmds.execute(_cmds(), args)
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/dcos/cmds.py", line 43, in execute
    return function(*params)
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/dcoscli/help/main.py", line 89, in _help
    .make_command_summary_string(sorted(results))
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/concurrent/futures/_base.py", line 581, in result_iterator
    yield future.result()
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/dcos/subcommand.py", line 132, in documentation
    return (path_noun, info(executable_path, path_noun))
  File "/Users/miry/projects/oos/dcos/lib/python2.7/site-packages/dcos/subcommand.py", line 147, in info
    [executable_path, path_noun, '--info'])
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 573, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/Users/miry/.dcos/subcommands/kubernetes/env/bin/dcos-kubectl', 'kubectl', '--info']' returned non-zero exit status -5

 $ ./bin/dcos --version
dcos version 0.3.0
$ uname -a
Darwin host.local 15.4.0 Darwin Kernel Version 15.4.0: Sun Jan  3 21:35:22 PST 2016; root:xnu-3248.40.155~11/RELEASE_X86_64 x86_64

When spark is running on marathon, and running dcos package install spark again, it shows:

Installing Marathon app for package [spark] version [1.4.1]
App or group is locked by one or more deployments. Override with --force.

Would be really handy when writing tutorials to make it so people don't have to copy / paste into a local document (or do a download) before they are able to follow a command in a tutorial.

These are things meant to serve as examples, rather than stuff which should / would be packaged through formal channels / the universe and multiverse.

[ec2-user@ip-172-31-50-60 dcos]$ dcos --log-level=ERROR package install spark
Note that the Apache Spark DCOS Service is beta and there may be bugs, incomplete features, incorrect documentation or other discrepancies.
We recommend a minimum of two nodes with at least 2 CPU and 2GB of RAM available for the Spark Service and running a Spark job.
Note: The Spark CLI may take up to 5min to download depending on your connection.
Continue installing? [yes/no] yes
Installing Marathon app for package [spark] version [1.5.0-multi-roles-v2]
Installing CLI subcommand for package [spark] version [1.5.0-multi-roles-v2]
2016-01-01 04:51:13,440 /home/ec2-user/dcos/local/lib/python2.7/site-packages/dcos/subcommand.py:_execute_install:411 - Install script's stdout: Collecting dcos-spark==0.4.0-spark-1.5.0-multi-roles-v2-bin-2.4.0 from https://s3.amazonaws.com/downloads.mesosphere.io/dcos/spark/dcos_spark-0.4.0_spark_1.5.0_multi_roles_v2_bin_2.4.0-py2.py3-none-any.whl (from -r /tmp/tmpK6g6X2 (line 1))
Downloading https://s3.amazonaws.com/downloads.mesosphere.io/dcos/spark/dcos_spark-0.4.0_spark_1.5.0_multi_roles_v2_bin_2.4.0-py2.py3-none-any.whl (228.9MB)

2016-01-01 04:51:13,440 /home/ec2-user/dcos/local/lib/python2.7/site-packages/dcos/subcommand.py:_execute_install:412 - Install script's stderr: Exception:
Traceback (most recent call last):
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/basecommand.py", line 211, in main
status = self.run(options, args)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/commands/install.py", line 305, in run
wb.build(autobuilding=True)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/wheel.py", line 705, in build
self.requirement_set.prepare_files(self.finder)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/req/req_set.py", line 334, in prepare_files
functools.partial(self._prepare_file, finder))
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/req/req_set.py", line 321, in _walk_req_to_install
more_reqs = handler(req_to_install)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/req/req_set.py", line 491, in _prepare_file
session=self.session)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 825, in unpack_url
session,
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 673, in unpack_http_url
from_path, content_type = _download_http_url(link, session, temp_dir)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 886, in _download_http_url
_download_url(resp, link, content_file)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 621, in _download_url
for chunk in progress_indicator(resp_read(4096), 4096):
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/utils/ui.py", line 133, in iter
for x in it:
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 586, in resp_read
decode_content=False):
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 307, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 243, in read
data = self._fp.read(amt)
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 54, in read
self.__callback(self.__buf.getvalue())
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/controller.py", line 224, in cache_response
self.serializer.dumps(request, response, body=body),
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/serialize.py", line 49, in dumps
"body": _b64_encode_bytes(body),
File "/home/ec2-user/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/serialize.py", line 12, in _b64_encode_bytes
return base64.b64encode(b).decode("ascii")
File "/usr/lib64/python2.7/base64.py", line 53, in b64encode
encoded = binascii.b2a_base64(s)[:-1]
MemoryError

Error installing 'spark' package.

My dcos setup worked fine. Today "dcos package install kubernetes" failed without any sensible error message:

Error installing 'kubernetes' package

With log-level set to debug I get this:

...
Creating directory: '/Users/sts/.dcos/subcommands/kubernetes'
Calling: ['/Users/sts/Quellen/mesos/dcos-cli/cli/env/bin/virtualenv', '/Users/sts/.dcos/subcommands/kubernetes/env']
Install script's stdout: Using real prefix '/System/Library/Frameworks/Python.framework/Versions/2.7'
New python executable in /Users/sts/.dcos/subcommands/kubernetes/env/bin/python
Installing setuptools, pip...done.

Install script's stderr: 
Calling: ['/Users/sts/.dcos/subcommands/kubernetes/env/bin/pip', 'install', '--requirement', '/var/folders/gg/zn9yfdld21j7xg5jnj5zz5d80000gn/T/tmp1W0OCQ']
Install script's stdout: 
Install script's stderr: You are using pip version 6.1.1, however version 7.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Exception:
Traceback (most recent call last):
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/basecommand.py", line 246, in main
    status = self.run(options, args)
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/commands/install.py", line 324, in run
    finder=finder, options=options, session=session):
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/req/req_file.py", line 135, in parse_requirements
    isolated=options.isolated_mode if options else False,
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/req/req_install.py", line 220, in from_line
    isolated=isolated)
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/req/req_install.py", line 79, in __init__
    req = pkg_resources.Requirement.parse(req)
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2960, in parse
    reqs = list(parse_requirements(s))
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2904, in parse_requirements
    "version spec")
  File "/Users/sts/.dcos/subcommands/kubernetes/env/lib/python2.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2869, in scan_list
    raise ValueError(msg, line, "at", line[p:])
ValueError: ('Expected version spec in', '[email protected]:mesosphere/dcos-kubernetes-cli.git#egg=dcos-kubernetes', 'at', '@github.com:mesosphere/dcos-kubernetes-cli.git#egg=dcos-kubernetes')

Hi all,
I'm using just the CLI without a full DCOS.
I want to set the core.mesos_master_url to a zk URI which is permitted. Is this setting only available with a full DCOS instance or am I missing something? A master failover is not really covered with the hardcoding of a mesos master URI.
Many greets
Michael

When working with dcos package search the SOURCE takes up too much space. Also there's strange newlines between package names.

I propose to shorten the SOURCE URLs for well known repositories to just the name and version. And only print full length URLs for custom repos. That way most users screen will look neat.
E.g. instead of repeating http://github.com/mesosphere/multiverse/archive/version-1.x.zip in every line just print multiverse 1.x.

The deprecation notice at https://github.com/mesosphere/mesos-cli leads here. This project is not usable as a replacement for mesos-cli. In particular, it asks for a mesosphere.com account and assumes the use of DCOS (no longer supports vanilla mesos).

As a developer of a third party subcommand, I easily find https://github.com/mesosphere/dcos-helloworld and base by subcommand on it. I expect it to be in good shape and a good example how to write a good subcommand.

In contrast it looks pretty much abandoned, e.g. tests do not work.

The tests in dcos-cli/cli/tests/integrations/test_node.py which claim to test the ability to SSH into DCOS Master and Slave nodes passed, but did not actually verify the desired functionality. Attempting to SSH to DCOS Mesos nodes does not work. The tests should be rewritten to verify actual functionality.

I believe that current tests only run against Mesos clusters with Marathon running. I do not believe they actually run against DCOS clusters. The test infrastructure for the dcos-cli tests needs to provision a DCOS cluster.

I am installing dcos-cli on Windows 8.1 64-bit using the installer script available here: https://downloads.mesosphere.io/dcos-cli/install.ps1

When running the script, it replaces the user's PATH variable with the install path: C:\Users\user\Desktop\dcos-cli\Scripts\.

This means that the existing paths in the PATH variable is removed, which can break a few things.

It would be great if it's able to concatenate the dcos-cli path instead of replacing the path.

I'm testing marathon's application groups via dcos and created a sample json. When I'm posting it via curl to Marathons REST api, it is successfully started. However via dcos cli I get errors.

If I add a toplevel empty
"apps": [ ]
property into the json, then I get:
RefResolutionError: Unresolvable JSON pointer: u'definitions/pathType'

Without the apps property just a lengthy stacktrace.

This is the command I'm using:
dcos marathon group add multi.json

The json:

{
  "id": "/multi-test7",
  "groups": [ {
    "id": "/multi-test7/persistence",
    "apps": [ {
      "id": "/multi-test7/persistence/mongo",
      "cmd": "while [ true ] ; do echo 'Fake MongoDB server' ; sleep 5 ; done",
      "cpus": 0.1,
      "mem": 64.0,
      "instances": 1
    }]
  },{
    "id": "/multi-test7/persistence2",
    "apps": [ {
      "id": "/multi-test7/persistence2/mongo",
      "dependencies": ["/multi-test7/persistence/mongo"],
      "cmd": "while [ true ] ; do echo 'Fake MongoDB server' ; sleep 5 ; done",
      "cpus": 0.1,
      "mem": 64.0,
      "instances": 1
    }]
  } ]
}

After some testing it turned out that if I remove the dependencies property from the json, it will be posted successfully to Marathon via dcos cli.

In version-2 of universe image information was moved to the resource.json we need to make sure that the CLI honors that when listing installed packages.

Now we revised dcos/dcos-cli code to discover security implemented into dcos and actually we didn’t find a lot. We see OAuth implementation hardcoded into dcos cli.

So I just wondering about your strategy about security.

• What security elements will be implemented into DCOS?
• What approaches will be used?
• Are you planing to implement account management (Identity, Authorisation, Policy, Audit)?

Hello, I found you could use https://github.com/jonathanslenders/python-prompt-toolkit to make dcos-cli more convenient.

Example: "/this/is/a.valid/app-or-group.id"

This regular expression forbids this:

https://github.com/mesosphere/dcos-cli/blob/master/cli/dcoscli/data/marathon-group-schema.json#L210

The regular expression which validates ID path segments in Marathon is here (the things between the slashes):

https://github.com/mesosphere/marathon/blob/releasing-0.11/src/main/scala/mesosphere/marathon/api/v2/ModelValidation.scala#L230

When I have an installed cassandra on a cluster, that it prints out the requirements makes me think it's about to launch a new cassandra cluster

dcos package install --cli cassandra
The Apache Cassandra DCOS Service implementation is alpha and there may be bugs, incomplete features, incorrect documentation or other discrepancies.
The default configuration requires 3 nodes each with 0.3 CPU shares, 1184MB of memory and 272MB of disk.
Continue installing? [yes/no] yes
Installing CLI subcommand for package [cassandra] version [0.2.0-1]
New command available: dcos cassandra
Thank you for installing the Apache Cassandra DCOS Service.

    Documentation: http://mesosphere.github.io/cassandra-mesos/
    Issues: https://github.com/mesosphere/cassandra-mesos/issues

I am not sure if this is a case of marathon hasn't yet implemented group update or if the dcos CLI is doing it wrong but I get no error and yet nothing changes on marathon. Either way a change should be made to the CLI either informing that the feature is not yet supported or being fixed to work as expected.

If this should be working, let me know and I will try to make a simple marathon group to be able to reproduce.

Thanks in advance.

Running

dcos node ssh --master --option IdentityFile=/Users/gabriel/.ssh/gabriel.pem --user=core

against a test/continuous cluster fails with

ssh: connect to host 10.0.4.191 port 22: Operation timed out

It appears the MesosDNSClient is resolving the internal IP of master. This is confirmed when looking at the Mesos UI which reports

Cluster:gabriel-test
Server:10.0.4.191:5050

This command should resolve the public IP.

When I add a app with "dcos marathon app add" and a json, if marathon returns a 400 with a message saying why the json definition is bad, all I see from the terminal is 400 Bad Request. So for a user to figure out what's wrong they still need to curl themselves to marathon to get the exact error message.

We should print out the response body so users can fix their json def.

SSH to slave nodes not work. The comments in the code describe how it should work and then fail to implement what they describe.

def _ssh(master, slave, option, config_file, user):
    """SSH into a DCOS node.  Since only the masters are definitely
    publicly available, we first ssh into an arbitrary master, then
    hop to the desired node.
    ...
    """

Either implement this feature or remove the comment. See the dicussion here. Probably the comment should be removed.

I feel it would be beneficial to have the flexibility to allow for environment variables (or argument variables) to be used to override the default configuration TOML file.

I have a need for this in the circumstance I'm in, where we have a default config that points to our production marathon URI and I would like to use the cli (wrapped in a propriatery docker container) to interface with a local marathon deploy on my machine. I realize that I could just download the vanilla CLI on my machine and have a config point to localhost, but it would be preferable to simply set environment variables and have those override the read config.

Currently dcos-cli supports service log, but similiarly will be great to support task logs especially for subcommands.

A use case that this is great for is Spark, as the results of the Spark job is in the stdout/stderr and it's not easy to get the results from Mesos UI due to DCOS making slaves private.

Will be nice that we can just do "dcos spark log " which then call a library function from dcos package.