embotech / ecos Goto Github PK

Hi,
I managed to crash my MATLAB instance when I select ecos for the following basic linear program. It runs fine with cvx' basic solver (sdpt3 on my machine), i.e., with cvx_solver ecos commented out.

Can you confirm this or help me debug on my side?
Am I missing anything or is there a problem with purely linear programs?

Best,
Matthias

clear all;
%http://people.brunel.ac.uk/~mastjjb/jeb/or/morelp.html
n = 2; c = [1; 1]; A = [50, 24; 30, 33]; b = [40*60; 35*60];

cvx_begin
    cvx_solver ecos
    variables x(n);

    maximize (c' * x);
    subject to
        x >= [45; 5];
        A*x <= b;
cvx_end
x
%correct: x = [45, 6.25]

Looks like you forgot to commit / add "conelp_buildlowrankmatrices" to the repository.

Error in ==> conelp at 256
    [S,Vrank1] = conelp_buildlowrankmatrices(scaling,n,p,dims);

Check the cone module and ecos.c for divisions. Make them safe if possible by checking the divisor before division.

Running valgrind on ecostester reveals a slew of memory leaks. Requires further investigation.

When using ecos with matlab/cvx it prints to stdout. Can it be made to obey the cvx quiet command to suppress printing?

Pandas seems to be the new numpy/scipy for data analysis; might as well add Pandas data frames to the ECOS Python module.

I am experiencing an issue where my problem can be solved by the two solvers shipped with CVX 2.0 b, but not with ECOS. Below is the output from CVX for this particular issue.

Calling ECOS unknown: 870 variables, 429 equality constraints
   For improved efficiency, ECOS is solving the dual problem.
------------------------------------------------------------
------------------------------------------------------------
Status: Error
Optimal value (cvx_optval): NaN

Reference to non-existent field 'r'.

Error in cvx_ecos>solve (line 107)
            reord.q.r = [ reord.r.r ; temp(:) ];

Error in cvxprob/solve (line 422)
            [ x, status, tprec, iters, y ] = shim.solve( At, b, c, cones, quiet, prec, solv.settings, eargs{:} );

Error in cvx_end (line 87)
        solve( prob );

Error in main_no_sparse (line 253)
cvx_end

I would be happy to share the relevant code in a private message to trace this issue.

Hi,

I have an instance after solving which, the value of objective function for x is better than for best_x (x contains the correct solution when compared to CVX). Shouldn't best_x always have at least as good as the last x? Or best_x is used only when some criterion is violated? Or best_x is an internal variable and should not be used by end-users? It's all a bit confusing, and comments in the source and documentation do not say much about it...

/Alex

We're trying to bundle Win64 shared library version of ECOS with ECOS.jl, but hitting issues (jump-dev/ECOS.jl#8)

As you can see, there are some odd things going on with types, even ecostester seems to be not working right. Any ideas?

Some atoms that can be represented as SOCPs are represented as SDPs in CVX. Allow the ECOS shim to handle SDPs that can be "downgraded" to SOCPs.

A cvxpy user ran into a very simple problem that causes a numerical exception in ECOS. Here's the problem:

import cvxpy as cp
x=cp.Variable()
p = cp.Problem(cp.Minimize(0), [0 <= x, x <= 1])
print p.solve()
print p.status

and here's code that creates the same problem matrices:

import numpy as np
import scipy.sparse as sp
import ecos
c = np.array([0.])
h = np.array([1., -0.])
G = sp.csc_matrix([1., -1.]).T
dims = {'q': [], 's': [], 'l': 2}
print ecos.solve(c, G, h, dims)

All of a sudden all my regression tests failed mysteriously, and it turned out ecos is the culprit

which u -all
C:\work\solvers\ecos\ecos-master\matlab\tools\u.m

Not a good choice on a function name...

To install ECOS for use with Python on Windows 7 I had to do the following things:

1. Install Visual C++ 2008 Express Edition from go.microsoft.com/?linkid=7729279 which gives the compiler used to compile Python 2.7.6 (I'm using Python(x,y) which I highly recommend for anyone wanting to install Python and all the crap needed for a useful scientific python installation.)

2. Make (ISO C or Microsoft specific?) changes to ecosmodule.c:
   -- All 3 "inline" modifiers must be changed to "__inline".
   -- All variables must be declared at the top of functions, they can't be declared all throughout the body.

Afterwards the "setup.py install" command works.

How hard is it to get SDP support on this? Also, see

https://github.com/sedumi/sedumi

It's possible we should just focus on getting a working C version of SeDuMi.

Could you please include a documented example on how to use the library from a C/C++ standalone application? The runecos.c file has no documentation about the problem.

Many thanks,

To ease deployment of ECOS on Windows machines, we should consider packaging the Python distributions into wheels. If anyone has good tutorials on how to do this, we're all ears!

I am experiencing a bug with ECOS where the solver just hangs as soon as it is called by CVX. This does not happen in the same situation with SeDuMi, for example.

The code that I have built around the particular piece of CVX code is quite substantial, and I do not feel comfortable distributing it on this side. How should I handle this bug? Is there still a way for me to report it and contribute to its solution?

ecos_mex crashes Matlab with segfault if the dims argument doesn't have the specified fields (l and q). Should be checked.

Is ecos compatible with Numpy 1.8.0? When I run "pyhton setup.py install" I get the following warning:
Using deprecated Numpy API

Also, I get this error: undefined struct 'timespec'
It seems that ecos is only compatible with Visual Studio 2008. I didn't work with Visual Studio 2010 and 2013.

Hi,
I have been trying to get ecos running on one of our Olimex STM32-P407 boards ( https://www.olimex.com/Products/ARM/ST/STM32-P407/ )
Unfortunately, I have been unsuccessful thus far even though the compilation goes fine after restructuring the files a little bit. I threw headers and sources into a single folder because the cross compiler wouldn't find them otherwise.

I suspect that I am running out of RAM (196KB here) during the ECOS_setup call.
I tested the runecos executable on Linux with valgrind and found that it has about 300KB on the heap and that's unfortunately the best estimate I have right now.

I am wondering if you have a guideline for how much RAM is required to run a very small problem? Have you actually run this on a small microcontroller already? Any recommendation for a platform with, say, 1MB RAM?

Best regards,
Matthias

Not actually causing trouble as far as I can see, but probably this should be mxMalloc

Hi,

In the ECC paper from 2013, it is mentioned that ECOS has the three routines setup, solve, and cleanup. Furthermore, it is stated that
"Once the problem has been set up (possibly accomplished by a code generator),
different problem instances can be solved without the need
for dynamic memory allocation (and without the Setup overhead)."

This is something that I am interested in, but I haven't yet understood if there is a routine that updates the required mywork members based on new information from my problem matrices c, A, b, G, h, without allocating additional memory, or if this is something I would need to carry out "manually" before a repeated solve (e.g., by "copying" ECOS_setup but stripping it from the MALLOCs).

So, my question is: Do you have a recommended procedure to carry out repeated solves without additional memory allocation?

Best,
Magnus Nilsson, Viktoria Swedish ICT

The solution to both of these problems is -K/2. In both cases ECOS runs into numerical issues for large K. The get_problem_data lines give the arguments CVXPY passes to ECOS.

import cvxpy
import ecos

# Numerical problems with quad_form.
K=500
u = cvxpy.Variable()
cost_optimizer=K*u+cvxpy.quad_form(u, 1)
objective = cvxpy.Minimize(cost_optimizer)
problem=cvxpy.Problem(objective)
problem.solve(verbose=True)
# Should be near 0.
print u.value + K/2

c, G, h, dims, A, b = problem.get_problem_data(solver=cvxpy.ECOS)

# Numerical problems with square.
K=850
u = cvxpy.Variable()
cost_optimizer=K*u+cvxpy.square(u)
objective = cvxpy.Minimize(cost_optimizer)
problem=cvxpy.Problem(objective)
problem.solve(verbose=True)
# Should be near 0.
print u.value + K/2

c, G, h, dims, A, b = problem.get_problem_data(solver=cvxpy.ECOS)

How accurate is ECOS supposed to be? In cvxpy I generally find that ECOS's values for the problem variables are accurate to within 4 significant figures. My understanding is that interior point methods are usually more accurate than this.

ecos.m refers to default dims, but this doesn't seem to be implemented?

This part:
x <= u
-x <= v
Should be
x <= u
-x <= u

You don't need to introduce the second variable v.

In the README it says the possible exit flags are
0=OPTIMAL, 1=PRIMAL INFEASIBLE, 2=DUAL INFEASIBLE, -1=MAXIT REACHED
But I've also gotten -2 and -3. What do these exit flags mean? What is the true range of possible exit flags?

Here's a simple CVX script that throws an error:

cvx_begin
    variable x 
    cvx_solver ecos
    minimize x
    x >= 1
cvx_end

Here's the output:

Calling ECOS unknown: 1 variables, 1 equality constraints
------------------------------------------------------------
------------------------------------------------------------
Status: Error
Optimal value (cvx_optval): NaN

??? Error using ==> ecos
G and h do not match in dimension

I think I know what's going on, but if anyone else has some spare time and wants to look into this, they should feel to.

Some of the students in the convex optimization class were having an issue with problem A3.19, in which none of the cvxpy solvers was stable. I have this code, which is the literal translation of the matlab solution we have. On my machine (Windows x64, ecos 1.0.4, cvxpy 0.2.6, numpy 1.8.1, scipy 0.14.0), this code gives the expected answer (5077.5), but when the lines marked with ### were swapped, it gives a solver_error with optimal value of None. Steven suggested I post this issue here. Thanks for the help!

# exercise A3.19
import numpy as np
import cvxpy as cvx

# data
a = [0.5, -0.5, 0.2, -0.7, 0.6, -0.2, 0.7, -0.5, 0.8, -0.4]
l = [40, 20, 40, 40, 20, 40, 30, 40, 30, 60]
Preq = np.arange(a[0], a[0]*(l[0]+0.5), a[0])
for i in range(1, len(l)):
    Preq=np.r_[ Preq, np.arange(Preq[-1]+a[i], Preq[-1]+a[i]*(l[i]+0.5), a[i]) ]
Preq = np.asmatrix(Preq).T
T = sum(l)
Peng_max = 20.0
Pmg_min = -6.0
Pmg_max = 6.0
Ebatt_max = 100.0
eta = 0.1
gamma = 0.1

# solve
Peng = cvx.Variable(T)
Pmg = cvx.Variable(T)
Pbr = cvx.Variable(T)
E = cvx.Variable(T+1)

obj = cvx.Minimize(cvx.sum_entries(Peng) + \
          gamma * cvx.sum_squares(Peng) + \
          1e-5 * cvx.sum_entries(cvx.neg(Pmg)))
cons = [Preq == Peng + Pmg - Pbr, 
        E[0] == E[T],
        Pmg_min <= Pmg, Pmg <= Pmg_max,        ###
        Pbr >= 0, 0 <= Peng, Peng <= Peng_max, ###
        0 <= E, E <= Ebatt_max]

for t in range(T):
    cons += [E[t+1] <= E[t]-Pmg[t]-eta*cvx.abs(Pmg[t])]


prob = cvx.Problem(obj, cons)
result1 = prob.solve(solver=cvx.ECOS) # expected: 5077.5
print result1
print prob.status

It seems that ecos currently does not work with the newly released CVX 2.1 (unannounced, but can be downloaded here: http://cvxr.com/cvx/download/).

I get the following error message:

>> cvx_install_ecos
Compiling LDL...Building with 'Microsoft Windows SDK 7.1 (C)'.
MEX completed successfully.
            [done]
Compiling AMD...Building with 'Microsoft Windows SDK 7.1 (C)'.
MEX completed successfully.
            [done]
Compiling ecos...Building with 'Microsoft Windows SDK 7.1 (C)'.
MEX completed successfully.
            [done]
Compiling ecos_mex...Building with 'Microsoft Windows SDK 7.1 (C)'.
MEX completed successfully.
            [done]
Linking...     Building with 'Microsoft Windows SDK 7.1 (C)'.
MEX completed successfully.
                [done]
ecos successfully compiled. Happy solving!
Error in cvx_version (line 11)
global cvx___

Output argument "varargout" (and maybe others) not assigned during call to "C:\cvx\cvx\cvx_version.m>cvx_version".

Error in cvx_install_ecos (line 2)
[ ver, isoctave, fs, ps ] = cvx_version;

I am trying to run a CVX model in Linux that works fine in my Windows setup. However, on Linux I get the following error message:

Undefined function 'ecos' for input arguments of type 'struct'.

Error in cvx_run_solver (line 50)
   [ varargout{1:nargout} ] = sfunc( inputs{:} );

Error in cvx_solver_shim>solve (line 185)
[ yy, xf, info, zz, xK ] = cvx_run_solver( @ecos, ecos_c, ecos_G, ecos_h, K, ecos_A, ecos_b, opts, varnames{:}, settings, 7 ); %#ok

Error in cvxprob/solve (line 422)
            [ x, status, tprec, iters, y ] = shim.solve( At, b, c, cones, quiet, prec, solv.settings, eargs{:} );

Error in cvx_end (line 87)
        solve( prob );

Error in convex_guidance_core (line 261)
cvx_end

Error in myscript.....

I suspect that ECOS is not on the path or so?! Do you have any ideas how I could fix this?

Since 1.0.3 seems to be the "standard", including https://pypi.python.org/pypi/ecos/1.0.3, it seems like this release should be tagged (I don't see it in the github releases).

I don't have access to a Windows machine, so I can't exactly compile ECOS (and the associated Python bindings) for it. We should provide Windows support at some point, especially since CVXPY is beginning to rely heavily on ECOS.

So far the ordering is computed in preproc.c by directly calling AMD by Tim Davis.

To allow for other orderings, we need to separate this out and create a new ordering function in the SPLAMM (sparse linear algebra with memory manager) module. It would be great to just call

ok = matrix_order(&K, &P, "symamd")

to find the ordering &P with "symamd" (or "amd" or "metis" etc.). Also useful for code generation.

Here's the CVXPY code to generate the problem:

from cvxpy import *
x = Variable(5)
obj = Maximize(sum_entries(sqrt(x)))
prob = Problem(obj, [x >= 1])
prob.solve(solver=ECOS, verbose=True)
c, G, h, dims, A, b = prob.get_problem_data(ECOS)

It's late. I should be sleeping.

Anyway, since CVX pioneered the successive approximation approach to handling logarithms, etc. I wonder if for a future goal (say, ECOS 1.2) it might be a good idea for us to bake in the successive approximation directly into the solver. So whenever ECOS sees an exponential cone, it uses a successive approximation technique (presumably by maintaining pointers to the KKT system). Of course, we would print the requisite warning. But this would let us (quickly) solve problems with logarithms in, say, CVXPY.

Of course, we could construct the approximation in CVXPY instead of directly in the solver.... Thoughts?

This is an old artefact, but in case someone wants to develop with vstudio we need to fix this project first to match the current files etc.

Add portfolio problem data and formulation and rewriting code from the ECC paper to the examples.

I could not install ecos on Python 3.3, windows 7, due to the error described above in function ECOS_solve of ecos.c. Cold you, please, help me on it?

When I run sudo pip install ecos, I get the following error:

------------------------------------------------------------                    
/usr/bin/pip run on Wed Jan 15 22:30:03 2014
Downloading/unpacking ecos
  Getting page http://pypi.python.org/simple/ecos
  URLs to search for versions for ecos:
  * http://pypi.python.org/simple/ecos/
  Getting page http://pypi.python.org/simple/ecos/
  Analyzing links from page https://pypi.python.org/simple/ecos/
    Found link https://pypi.python.org/packages/2.7/e/ecos/ecos-1.0.1.macosx-10.9-x86_64.tar.gz#md5=9e9cc677817ba72f8255aac8b349301f (from https://pypi.python.org/simple/ecos/), version: 1.0.1.macosx-10.9-x86_64
    Found link https://pypi.python.org/packages/source/e/ecos/ecos-1.0.1.tar.gz#md5=5ada9e047ff91ffc02d9d995afb30320 (from https://pypi.python.org/simple/ecos/), version: 1.0.1
  Using version 1.0.1.macosx-10.9-x86_64 (newest of versions: 1.0.1.macosx-10.9-x86_64, 1.0.1)
  Downloading ecos-1.0.1.macosx-10.9-x86_64.tar.gz
  Downloading from URL https://pypi.python.org/packages/2.7/e/ecos/ecos-1.0.1.macosx-10.9-x86_64.tar.gz#md5=9e9cc677817ba72f8255aac8b349301f (from https://pypi.python.org/simple/ecos/)
  Running setup.py egg_info for package ecos
    Traceback (most recent call last):
      File "<string>", line 14, in <module>
    IOError: [Errno 2] No such file or directory: '/home/fitze/build/ecos/setup.py'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 14, in <module>

IOError: [Errno 2] No such file or directory: '/home/fitze/build/ecos/setup.py'

----------------------------------------
Command python setup.py egg_info failed with error code 1
Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 126, in main
    self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 223, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 980, in prepare_files
    req_to_install.run_egg_info()
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 216, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 255, in call_subprocess
    % (command_desc, proc.returncode))
InstallationError: Command python setup.py egg_info failed with error code 1

Basically, it tries to download the newer 'osx' release, which it shouldn't do for my system.

Running pip install ecos should also install scipy. Mark include scipy in the install_requires keyword argument in setup.py.

The CVX shim is there, but the readme misses a section on how to set it up and use it.

This problem is unbounded, but ECOS returns infeasible. Here's the CVXPY code to generate the problem and extract the representation passed to ECOS:

from cvxpy import *
x1 = Variable()
x2 = Variable()
obj = Minimize(-x1-x2)
constraints = [2*x1 + x2 >= 1, x1 + 3*x2 >= 1, x1>= 0, x2>=0]
prob = Problem(obj, constraints)
prob.solve(solver=ECOS)
c, G, h, dims, A, b = prob.get_problem_data(ECOS)

ecos.m refers to dense or sparse A and G, but only seems to accept sparse.

Hi. I am using MATLAB2013B on 64-bit windows 7. I Installed CVX and it works properly. I tried to install ECOS by unzipping the *.zip archive ecos-master.zip and placing the folder ecos-master in a directory. I opened matlab and navigated to the ecos-master\matlab directory. Typed cvx_install_ecos at the MATLAB command prompt and got the following dump:

CVX, version 2.0 (beta) (c) 2012, CVX Research, Inc.

Software for Disciplined Convex Programming

Version info:
Code: build 1023, 2013-12-03 11:04:38
Documentation: build 994, 2013-07-24 10:54:53
Installation info:
Path: C:\Users\mountcastle\Downloads\cvx
MATLAB version: 8.2 (R2013b)
OS: Windows 7 amd64 version 6.1
Java version: 1.7.0_11
Verfying CVX directory contents:
WARNING: The following extra files/directories were found:
C:\Users\mountcastle\Downloads\cvx\ecos/
C:\Users\mountcastle\Downloads\cvx\ecos/ecos.mexw64
C:\Users\mountcastle\Downloads\cvx\shims/cvx_ecos.m

These files may alter the behavior of CVX in unsupported ways.

Setting CVX paths...already set!
Saved preferences...found.
Searching for solvers...3 shims found.
2 solvers initialized (* = default):

• SDPT3 4.0 {cvx}\sdpt3
  SeDuMi 1.34cvx {cvx}\sedumi
  1 solver skipped:
  ECOS
  Could not find an ECOS installation.
  Saving updated preferences...done.
  Testing with a simple model...done!

To change the default solver, type "cvx_solver <solver_name>".
To save this change for future sessions, type "cvx_save_prefs".
Please consult the users' guide for more information.

cvx_install_ecos
Compiling LDL... [done]
Compiling AMD... [done]
Compiling ecos... [done]
Compiling ecos_mex... [done]
Linking... [done]
ecos successfully compiled. Happy solving!
Warning: Directory already exists.
In cvx_install_ecos at 5

CVX, version 2.0 (beta) (c) 2012, CVX Research, Inc.

Software for Disciplined Convex Programming

Version info:
Code: build 1023, 2013-12-03 11:04:38
Documentation: build 994, 2013-07-24 10:54:53
Installation info:
Path: C:\Users\mountcastle\Downloads\cvx
MATLAB version: 8.2 (R2013b)
OS: Windows 7 amd64 version 6.1
Java version: 1.7.0_11
Verfying CVX directory contents:
WARNING: The following extra files/directories were found:
C:\Users\mountcastle\Downloads\cvx\ecos/
C:\Users\mountcastle\Downloads\cvx\ecos/ecos.mexw64
C:\Users\mountcastle\Downloads\cvx\shims/cvx_ecos.m

These files may alter the behavior of CVX in unsupported ways.

Setting CVX paths...already set!
Saved preferences...found.
Searching for solvers...3 shims found.
2 solvers initialized (* = default):

• SDPT3 4.0 {cvx}\sdpt3
  SeDuMi 1.34cvx {cvx}\sedumi
  1 solver skipped:
  ECOS
  Could not find an ECOS installation.
  Saving updated preferences...done.
  Testing with a simple model...done!

To change the default solver, type "cvx_solver <solver_name>".
To save this change for future sessions, type "cvx_save_prefs".
Please consult the users' guide for more information.

Is there a simple fix?

Is there a way to specify solver options from Python besides verbose=True? For example, could I change the maximum number of iterations?

For example, feasibility problems, linear algebra testers, etc. etc.

This would enable passing a settings struct from matlab directly to ecos, without the need to recompile.

Fedora 18 repo provides cvxopt 1.1.5. The cvxopt.h in ECOS is 1.1.6. The matrix struct (and probably others) aren't compatible.

Would it be possible to include some of the older headers in ECOS and use setup.py to check what version of cvxopt is installed?

Hi,
I have started experimenting with your solver today and so far everything is great!

I am wondering if I can somehow construct and test a problem in MATLAB using CVX or yalmip and then export the problem matrices in such a way that I can later re-use them in a C program.
Is there a convenient way that doesn't involve constructing the A,G,h matrices by hand?