belson17 / modred Goto Github PK

For example:
FAIL: test_compute_proj_coeffs (testdmd.TestDMDHandles)
FAIL: test_compute_model (testera.testERA)

Since Python 2 will soon be deprecated, we probably don't need to support it anymore. We can get rid of the py2to3 module, see what errors arise, and then fix them, which should make the code compatible with Python 3.

There can be a division by zero error in line 340 of
vectorspace.py when vecs_per_proc = 2:

    num_gets =  (num_rows*num_cols) / ((vecs_per_proc-2) *
        parallel.get_num_procs()**2) + num_rows/parallel.get_num_procs()

From what Jonathan said to me via email, vecs_per_proc should always be at least 3, so perhaps all that is needed is an error check.

creating .../lib/python3.7/site-packages/modred-2.0.4.post6-py3.7.egg
Extracting modred-2.0.4.post6-py3.7.egg to .../lib/python3.7/site-packages
File ".../lib/python3.7/site-packages/modred-2.0.4.post6-py3.7.egg/modred/tests/scaling.py", line 350
s.barriers / s.total))
^
SyntaxError: invalid syntax

This came up in an email from a user in June 2014. It was only a line or two of code, but requires the user to know more than they would otherwise, both in terms of math and the data variables in the classes (like POD). We should decide if we want to add a method.

Need to check code before new release.

There are a few warnings in vectorspace.py related to how many vectors can be loaded, etc. I think it makes sense for these to be printed to stderr, not stdout, as that is where warnings and errors typically go. There is also an estimate for the amount of time that all the inner products will take. I think it might make sense for this to print to stderr as well, as it is sort of an auxiliary/unnecessary message.

Hi,

I'm having issues getting modred installed with Python 3.6.23 (pip version 21.0.1) .

Package Version

certifi 2020.12.5
numpy 1.19.5
pip 21.0.1
pypi 2.1
setuptools 52.0.0.post20210125
wheel 0.36.2
wincertstore 0.2

The pip install fails.

ImportError: modaldecomp requires python version 2.7.x

If I try the setup.py (from modred-2.0.4.post6) it fails with an error

TypeError: 'version' object is not iterable

Do I have anything wrong with my environment configuration?

Thanks

Clean up docstrings so it's all on the same level. Either include math in docstrings or don't. Use matrix or vector space notation depending on arrays or handles implementations. Maybe include an "algorithms" section.

In emails from Milos, it looked like our type check that the arguments to call are numpy arrays didn't work as expected.

Consider making inner_product a keyword argument. It is always needed for performing the decomposition step, but one might want to create a POD/BPOD/DMD object to only perform the compute_modes step, which shouldn't require any inner products to be computed.

Allow for larger col chunks if extra memory.

The default should be None to take advantage of the smartness of np.loadtxt when passed a delimiter of None. This came up in an email to me (Brandt) from a user in April 2014.

Make a style guide for future developers.

Something about an IndexError.

python version 3.7.0
numpy version 1.17.4
Platform: windows

pip install modred
Collecting modred
Using cached https://files.pythonhosted.org/packages/ff/d9/9b65b1ffee0015b7191c63da691f9899e13f23728fc40449206e2f96e470/modred-2.0.4.post6.tar.gz
ERROR: Command errored out with exit status 1:
command: 'c:\users\XXXX\appdata\local\programs\python\python37-32\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\XXXX\AppData\Local\Temp\1\pip-install-k4ro1oo_\modred\setup.py'"'"'; file='"'"'C:\Users\XXXX\AppData\Local\Temp\1\pip-install-k4ro1oo_\modred\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\XXXX\AppData\Local\Temp\1\pip-install-k4ro1oo_\modred\pip-egg-info'
cwd: C:\Users\XXXX\AppData\Local\Temp\1\pip-install-k4ro1oo_\modred
Complete output (5 lines):
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\XXXX\AppData\Local\Temp\1\pip-install-k4ro1oo_\modred\setup.py", line 20, in
if any(w in ['*a', '*alpha'] for w in parsed_version):
TypeError: 'Version' object is not iterable
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

line 43:
self.data_dir = 'files_ERA'
test with
python -c 'import modred.tests; modred.tests.run()'
reports IOerror:
IOError: files_ERA/A_in1_out1.txt not found.

changed to absolute path solves the problem.

os is ubuntu18.04(python is actually python2.7)

Look into whether Pytest would add useful functionality to testing framework.

Hello everybody,

I’d like to use the Python package ‘modred’ for my master thesis. For this I’d like to install the package first but it doesn’t work.
Attached you can find the following information:

1. Copy of the entire output of the installation
   
2. Python version
   
3. Numpy version
   
4. My operating system
   

I already installed some other packages in the same way.

Do you know where the problem is that I can’t do the installation?

Thanks and Regards
Markus

Following the instructions in the README,

To check the installation, you can run the unit tests (parallel requires mpi4py):

in a fresh conda env without matplotlib (which isn't in install_requires), trouble is hit at

----------------------------------------------------------------------
ERROR: testera (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: testera
Traceback (most recent call last):
  File "/home/gmcbain/miniconda3/envs/modred/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/gmcbain/miniconda3/envs/modred/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/gmcbain/src/modred/modred/tests/testera.py", line 11, in <module>
    import matplotlib.pyplot as plt
ModuleNotFoundError: No module named 'matplotlib'

ImportError: Failed to import test module: testera

I saw that pytest #33 is under consideration, but meanwhile this might be avoided with unittest.skipIf.

On closer inspection, it seems that plt is only referred to in a string

and a comment

so perhaps the offending import could just be omitted.

I see no reason for Parallel to be a class. In fact, we have to do funny stuff to ensure or encourage the user not to have multiple instances of it. It seems to me simpler if we just make it a module that has a number of useful methods. This avoids the whole instance issue altogether, and should provide the same functionality.

There seems to be an error in the balanced_truncation function (modred.util), it does not take orders that are lower than Len(E).

It can be fixed by changing:

to

SL = Lo.dot(U[:,:order]).dot(np.diag(E[:order]**-0.5))
SR = Lc.dot(V[:,:order]).dot(np.diag(E[:order]**-0.5))

This came up in an email Jon and I replied to in June 2014. We gave them instructions and an explanation, but it required a few lines of code. We should decide if we want to include methods to do this for users.

vec * array uses vec's mul, which can be written to handle arrays. However, array * vec uses arrays' mul, which doesn't always do what a user wants on the vector (namely it broadcasts the multiplication onto every element of mul, potentially). We should add documentation that says to do array * vec, it's best to actually wrap the array into a vec so the operation is vec * vec.

If the python custom vector object wraps an object from C++ or another language, and the data associated with the vector is stored in that foreign object, then python's copy.deepcopy method may not copy the data into a new python object. This point was raised by Philip Sakievich in an email in December 2016. We should at least mention this in the documentation. One possible solution is to check for a custom vector "deepcopy" method.

Hi everybody,

after installing the modred package I'd like to use it for a model order reduction. For this want to apply the Proper Orthogonal Decomposition (POD) and Eigensystem Realization Algorithm (ERA) on my dataset. This consists of an input (m x n) and an output matrix (n x p). Unfortunately I have a problem with the function variables:

1. According to the modred.pod-documentation I need a inner_product as an input for the calculation: https://modred.readthedocs.io/en/stable/pod.html. How do I get the inner_product from my dataset?

2. According to the modred.era-documentation I need a put_mat as an input for the calculation: https://modred.readthedocs.io/en/stable/era.html. How do I get the put_mat from my dataset?
   Where do I get the Markov parameters in the next step? As I understand the documentation the function expects it as an input but I don't know how to calculate it.

Thanks and regards
Markus

Hej,

Here is the other problem I was mentioning in my previous issue. I can run all of the examples except for the Complex Ginzburg-Landau one. Here is the output when running this example.

----- Parameters ------
nx = 220
dt = 1.0
U = 2.0
c_u = 0.2
c_d = -1.0
mu_0 = 0.38
mu_2 = -0.01
s = 1.6
x_s = 8.24621125124
nu = (2+0.4j)
gamma = (1-1j)
chi = (0.239159529837+0.0475717882612j)
-----------------------
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/home/jean-christophe/Codes/modred/modred/examples/runall.py in <module>()
     25 
     26 if not parallel.is_distributed():
---> 27     execfile('main_CGL.py')

/home/jean-christophe/Codes/modred/modred/examples/main_CGL.py in <module>()
    139 r = 10
    140 direct_modes, adjoint_modes, sing_vals = mr.compute_BPOD_matrices(
--> 141     q, q_adj, list(range(r)), list(range(r)), inner_product_weights=weights)
    142 
    143 # Plot the first 3 modes

/home/jean-christophe/anaconda2/lib/python2.7/site-packages/modred-2.0.1-py2.7.egg/modred/bpod.pyc in compute_BPOD_matrices(direct_vecs, adjoint_vecs, num_inputs, num_outputs, direct_mode_indices, adjoint_mode_indices, inner_product_weights, atol, rtol, return_all)
     79     # Compute first column (of chunks) of Hankel matrix
     80     all_adjoint_first_direct = np.array(vec_space.compute_inner_product_mat(
---> 81         adjoint_vecs, direct_vecs[:, :num_inputs]))
     82     all_adjoint_first_direct_list = [
     83         all_adjoint_first_direct[

/home/jean-christophe/anaconda2/lib/python2.7/site-packages/numpy/matrixlib/defmatrix.pyc in __getitem__(self, index)
    316 
    317         try:
--> 318             out = N.ndarray.__getitem__(self, index)
    319         finally:
    320             self._getitem = False

TypeError: slice indices must be integers or None or have an __index__ method

I am using:

• Python 2.7.13 :: Anaconda custom (64-bit)
• Numpy 1.12.1
• Scipy 0.19.0

Hi all,

I believe tutorials have not been updated :

mr.range needs to be replace with range, with the following at the beginning
from future.builtins import range

and compute_POD_arrays_snaps_method needs to be replaced by mr.compute_POD_matrices_snaps_method

Hey,
I might be completely off base here, but the documentation seems to reference the eigenvectors as both "U" (L149) and then later on line 139 as "V" i was wondering if this is a mistake on my understanding or a simple typo.

please let me know what comes of this and thanks so much for creating this. has certainly helped me substantially!
best,
Phil

parallelization bug.pdf

In
https://modred.readthedocs.io/en/stable/tutorial_modaldecomp.html
an example for pod decomposition is given. To my understanding the last line
codes, eig_vals = mr.compute_POD_matrices_snaps_method( vecs, list(range(num_modes)))

should rather read

modes, eig_vals = mr.compute_POD_arrays_snaps_method( vecs, list(range(num_modes)))

It is great that Pierre Augier updated the code to make it compatible with Python 3. However, now the package requires the future module, both in python2 and python3. It would be good to remove this dependency. (I see no reason why we need this, as we are doing nothing terribly fancy in our code, so it should be straightforward to make it compatible with both python 2.7 and python3 without an additional package.)

Hey all,
seems that the sanity check is using absolute error (difference in results) in checking definition of the vectorspace. When using a large-valued vector, this error is sometimes not achievable due to machine precision.

In my example, the inner products are of the order of 1e8 with a 1e-9 absolute error. This causes the check to fail with the default tolerance of 1e-12.

To me, it makes sense to turn this into a check on relative error (ratio of results) rather than absolute (difference of results) as to avoid machine precision issues.

Remove requirements that arguments are a certain type. There are a few checks for lists, but we probably don't need to and it's not pythonic.

Hej,

I have just git-cloned the latest version of modred. Install went smoothly but then, I get some test failures. Here is the output of modred.tests.run()

In [14]: modred.tests.run()
............ss..............F......sssssss................sss.......
======================================================================
FAIL: test_all (testdmd.TestDMDArraysFunctions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jean-christophe/anaconda2/lib/python2.7/site-packages/modred-2.0.1-py2.7.egg/modred/tests/testdmd.py", line 171, in test_all
    rtol=rtol, atol=atol)
  File "/home/jean-christophe/anaconda2/lib/python2.7/site-packages/numpy/testing/utils.py", line 1392, in assert_allclose
    verbose=verbose, header=header)
  File "/home/jean-christophe/anaconda2/lib/python2.7/site-packages/numpy/testing/utils.py", line 739, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-10, atol=1e-12

(mismatch 100.0%)
 x: [repr failed]
 y: [repr failed]

----------------------------------------------------------------------
Ran 68 tests in 3.188s

FAILED (failures=1, skipped=12)

I am using Python 2.7.13 :: Anaconda 4.3.1 (64-bit) and Numpy 1.11.3. Note that I also get some errors whenever I try to trun the GInzburg-Landau example (although all other examples run fine). It might be a separate issue I guess however.

++
JC

Hello, is there a correct way to cite modred in a scientific paper? Is there a paper or a conference talk I can cite?

Automatically detect the max_vecs_per_node. Or in the future, max_vecs_per_communicator. This would require some fancier things, like determining the available RAM and the size of a vector object. It might not be possible since some memory is used by the vector objects when doing inner products, etc. We could leave a safety factor though, and maybe leave space for 1-2 vecs/modes for scratch space for the user's functions.

It isn't always clear what is an array and what is a matrix in the tests, especially because with certain methods arrays go in and matrices come out. We should try to be consistent and clear about this so that the tests are easier to read and modify.

Right now we have the restriction that a few vectors must fit in one node's local memory. Ideally we'd remove this restriction so that vectors can be arbitrarily large. The way to do this seems to be to generalize the parallelization so that users can have parallel, distributed, Vector classes. Then, modred would work in units of larger groups of processors/nodes, MPI communicators, and the lower level user-implemented parts would operate within one of these communicators. The user-defined Vector class could then have its memory distributed amongst multiple nodes within one communicator. This would make modred useful for huge datasets. It might not be that hard either; we would need to generalize the node/processor structure to communicators. We should think about and discuss this a bit more before jumping in though.

A user reported that it failed to install properly using python 2.7 and a package manager or the code on https://pypi.org/project/modred, but that he could install it from the source code on git.

DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Defaulting to user installation because normal site-packages is not writeable
Collecting modred
Using cached modred-2.0.4.post6.tar.gz (109 kB)
ERROR: Command errored out with exit status 1:
command: /usr/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mTJBTf/modred/setup.py'"'"'; file='"'"'/tmp/pip-install-mTJBTf/modred/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-VlUsHC
cwd: /tmp/pip-install-mTJBTf/modred/
Complete output (5 lines):
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-mTJBTf/modred/setup.py", line 20, in
if any(w in ['*a', '*alpha'] for w in parsed_version):
TypeError: 'Version' object is not iterable
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.