Git Product home page Git Product logo

Comments (21)

gwenchee avatar gwenchee commented on July 29, 2024

can you try running a d3ploy input file. So go into the input directory and run cyclus linear_power_demand.xml. I think your d3ploy did install fine. Because when I run python setup.py install, this is my output:

gwenchee ~/github/d3ploy (transitionscenarios) $ python setup.py install running install running build running build_py package init file 'd3ploy/__init__.py' not found (or not a regular file) package init file 'd3ploy/__init__.py' not found (or not a regular file) running install_lib running install_egg_info Removing /home/gwenchee/anaconda3/lib/python3.6/site-packages/d3ploy-0.0.1-py3.6.egg-info Writing /home/gwenchee/anaconda3/lib/python3.6/site-packages/d3ploy-0.0.1-py3.6.egg-info

I usually ignore the 'd3ploy/__init__.py' not found (or not a regular file) warning.

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

cyclus linear_power_demand.xml gives:

terminate called after throwing an instance of 'cyclus::IOError'
what(): The file 'linear_pow_demand' could not be loaded.
Aborted (core dumped)

But running cyclus recycle.xml from within cycamore/input results in a successful cyclus run.

from d3ploy.

katyhuff avatar katyhuff commented on July 29, 2024

did you run cyclus linear_power_demand.xml or cyclus linear_pow_demand.xml? The error message would indicate you ran the latter.

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

There's no linear_power_demand.xml file in d3ploy/input, just linear_pow_demand.xml- so my command was cyclus linear_pow_demand.xml

from d3ploy.

gwenchee avatar gwenchee commented on July 29, 2024

Just to check, are you on the master branch of arfc/d3ploy?

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

Yes

sonata@gwenchee:~/fuelcycle/d3ploy$ git diff
sonata@gwenchee:~/fuelcycle/d3ploy$ git branch
* master
sonata@gwenchee:~/fuelcycle/d3ploy$ git pull upstream master
From https://github.com/arfc/d3ploy
* branch master -> FETCH_HEAD
Already up-to-date.

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

I'm also having an issue where cycamore seems to install correctly, then fails cycamore_unit_tests with a seg fault. cyclus/cycamore#488. Could this be related?

from d3ploy.

gwenchee avatar gwenchee commented on July 29, 2024

Yeah it is possible since we use cycamore archetypes in these d3ploy tests.

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

I installed cyclus and cycamore (passing all unit tests) on a MacOS system rather than Ubuntu. After installing d3ploy, the d3ploy tests still seem to be having trouble importing the d3ploy python modules. cyclus linear_pow_demand.xml returns ERROR(core ):No module found for path libd3ploy.demand_driven_deployment_inst.dylib. Running python deploy_solver_unit_tests.py from tests/unit_tests returns
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x10) frame #0: 0x0000000111b503d4 libpython3.6m.dylibPyImport_AddModuleObject + 20
libpython3.6m.dylibPyImport_AddModuleObject

And pytest returns errors like

`===================================================================== FAILURES ======================================================================
________________________________________________________________ test_backdeployment ________________________________________________________________

    def test_backdeployment():
        output_ = 'test_backdeployment.sqlite'
        input_path = os.path.abspath(__file__)
        find = 'd3ploy/'
        indx = input_path.rfind('d3ploy/')
        input_ = input_path.replace(
            input_path[indx + len(find):], 'input/linear_pow_demand_backdeployment.xml')
        s = subprocess.check_output(['cyclus', '-o', output_, input_],
>                                   universal_newlines=True, env=ENV)

tests/integration_tests/back_deployment_test.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/anaconda3/lib/python3.6/subprocess.py:336: in check_output
    **kwargs).stdout
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

input = None, timeout = None, check = True
popenargs = (['cyclus', '-o', 'test_backdeployment.sqlite', '/Users/sonata/fuelcycle/d3ploy/input/linear_pow_demand_backdeployment.xml'],)
kwargs = {'env': {'Apple_PubSub_Socket_Render': '/private/tmp/com.apple.launchd.2e5eHuiyYA/Render', 'CMAKE_MODULE_PATH': '/opt/...ude:', 'CMAKE_PREFIX_PATH': '/opt/local:', 'CONDA_DEFAULT_ENV': 'base', ...}, 'stdout': -1, 'universal_newlines': True}
process = <subprocess.Popen object at 0x118083320>
stdout = '              :                                                               \n          .CL:CC CC             _Q   ...                             \n ERROR(core  ):No module found for path libd3ploy.demand_driven_deployment_inst.dylib\n'
stderr = None, retcode = 1

    def run(*popenargs, input=None, timeout=None, check=False, **kwargs):
        """Run command with arguments and return a CompletedProcess instance.
    
        The returned instance will have attributes args, returncode, stdout and
        stderr. By default, stdout and stderr are not captured, and those attributes
        will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.
    
        If check is True and the exit code was non-zero, it raises a
        CalledProcessError. The CalledProcessError object will have the return code
        in the returncode attribute, and output & stderr attributes if those streams
        were captured.
    
        If timeout is given, and the process takes too long, a TimeoutExpired
        exception will be raised.
    
        There is an optional argument "input", allowing you to
        pass a string to the subprocess's stdin.  If you use this argument
        you may not also use the Popen constructor's "stdin" argument, as
        it will be used internally.
    
        The other arguments are the same as for the Popen constructor.
    
        If universal_newlines=True is passed, the "input" argument must be a
        string and stdout/stderr in the returned object will be strings rather than
        bytes.
        """
        if input is not None:
            if 'stdin' in kwargs:
                raise ValueError('stdin and input arguments may not both be used.')
            kwargs['stdin'] = PIPE
    
        with Popen(*popenargs, **kwargs) as process:
            try:
                stdout, stderr = process.communicate(input, timeout=timeout)
            except TimeoutExpired:
                process.kill()
                stdout, stderr = process.communicate()
                raise TimeoutExpired(process.args, timeout, output=stdout,
                                     stderr=stderr)
            except:
                process.kill()
                process.wait()
                raise
            retcode = process.poll()
            if check and retcode:
                raise CalledProcessError(retcode, process.args,
>                                        output=stdout, stderr=stderr)
E               subprocess.CalledProcessError: Command '['cyclus', '-o', 'test_backdeployment.sqlite', '/Users/sonata/fuelcycle/d3ploy/input/linear_pow_demand_backdeployment.xml']' returned non-zero exit status 1.

/anaconda3/lib/python3.6/subprocess.py:418: CalledProcessError`

from d3ploy.

FlanFlanagan avatar FlanFlanagan commented on July 29, 2024

At first glance it looks like d3ploy is not installed properly. Since the I don't think there is an install issue on current branches my guess is you are missing a dependency. Do you have both statsmodels and arch?

from d3ploy.

FlanFlanagan avatar FlanFlanagan commented on July 29, 2024

Also pmdarima need to be installed as well. All three are python packages, and have done a installs

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

Yes, all three dependencies have been installed.

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

Just an update - essentially the same issue is coming up with Python2.7 on a clean ubuntu system. I was using Python3.6 on the Mac, so I don't think it's a Python version issue. d3ploy modules aren't being loaded because shared libraries (".so" files in this case) aren't being found.

sonata@sonata-VirtualBox:~/fuelcycle/d3ploy$ pytest
=========================================================== test session starts ===========================================================
platform linux2 -- Python 2.7.15rc1, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /home/sonata/fuelcycle/d3ploy, inifile:
collected 19 items                                                                                                                        

tests/integration_tests/back_deployment_test.py F                                                                                   [  5%]
tests/integration_tests/buffer_test.py FFFF^C

================================================================ FAILURES =================================================================
___________________________________________________________ test_backdeployment ___________________________________________________________

    def test_backdeployment():
        output_ = 'test_backdeployment.sqlite'
        input_path = os.path.abspath(__file__)
        find = 'd3ploy/'
        indx = input_path.rfind('d3ploy/')
        input_ = input_path.replace(
            input_path[indx + len(find):], 'input/linear_pow_demand_backdeployment.xml')
        s = subprocess.check_output(['cyclus', '-o', output_, input_],
>                                   universal_newlines=True, env=ENV)

tests/integration_tests/back_deployment_test.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

popenargs = (['cyclus', '-o', 'test_backdeployment.sqlite', '/home/sonata/fuelcycle/d3ploy/input/linear_pow_demand_backdeployment.xml'],)
kwargs = {'env': {'BASH_ENV': '/usr/share/lmod/lmod/init/bash', 'CLUTTER_IM_MODULE': 'xim', 'COLORTERM': 'truecolor', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', ...}, 'universal_newlines': True}
process = <subprocess.Popen object at 0x7f6ab64cc050>
output = '              :                                                               \n          .CL:CC CC             _Q   ...                                \n ERROR(core  ):No module found for path libd3ploy.demand_driven_deployment_inst.so\n'
unused_err = None, retcode = 1
cmd = ['cyclus', '-o', 'test_backdeployment.sqlite', '/home/sonata/fuelcycle/d3ploy/input/linear_pow_demand_backdeployment.xml']

    def check_output(*popenargs, **kwargs):
        r"""Run command with arguments and return its output as a byte string.
    
        If the exit code was non-zero it raises a CalledProcessError.  The
        CalledProcessError object will have the return code in the returncode
        attribute and output in the output attribute.
    
        The arguments are the same as for the Popen constructor.  Example:
    
        >>> check_output(["ls", "-l", "/dev/null"])
        'crw-rw-rw- 1 root root 1, 3 Oct 18  2007 /dev/null\n'
    
        The stdout argument is not allowed as it is used internally.
        To capture standard error in the result, use stderr=STDOUT.
    
        >>> check_output(["/bin/sh", "-c",
        ...               "ls -l non_existent_file ; exit 0"],
        ...              stderr=STDOUT)
        'ls: non_existent_file: No such file or directory\n'
        """
        if 'stdout' in kwargs:
            raise ValueError('stdout argument not allowed, it will be overridden.')
        process = Popen(stdout=PIPE, *popenargs, **kwargs)
        output, unused_err = process.communicate()
        retcode = process.poll()
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd, output=output)
E           CalledProcessError: Command '['cyclus', '-o', 'test_backdeployment.sqlite', '/home/sonata/fuelcycle/d3ploy/input/linear_pow_demand_backdeployment.xml']' returned non-zero exit status 1

/usr/lib/python2.7/subprocess.py:223: CalledProcessError

from d3ploy.

FlanFlanagan avatar FlanFlanagan commented on July 29, 2024

That's just how cyclus builds modules. They are built into shared objects.

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

We fixed the d3ploy module import error by downgrading scipy to version 1.2 (for future reference). Running d3ploy/input files still results in a seg fault. Result of backtrace (Python 3.6 on the mac) from cyclus linear_pow_demand.xml within input:

Process 53857 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
    frame #0: 0x00007fff76848fbf libc++.1.dylib`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::assign(char const*, unsigned long) + 19
libc++.1.dylib`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::assign:
->  0x7fff76848fbf <+19>: movb   (%rbx), %cl
    0x7fff76848fc1 <+21>: testb  $0x1, %cl
    0x7fff76848fc4 <+24>: jne    0x7fff76848fcd            ; <+33>
    0x7fff76848fc6 <+26>: movl   $0x16, %esi
Target 0: (cyclus) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x68)
  * frame #0: 0x00007fff76848fbf libc++.1.dylib`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::assign(char const*, unsigned long) + 19
    frame #1: 0x00007fff76848f9b libc++.1.dylib`std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::operator=(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 51
    frame #2: 0x00000001089a2559 libcyclus.dylib`cyclus::Agent::spec(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 25
    frame #3: 0x00000001089a2356 libcyclus.dylib`cyclus::DynamicModule::Make(cyclus::Context*, cyclus::AgentSpec) + 1350
    frame #4: 0x000000010941cb2f libcyclus.dylib`cyclus::BuildMasterSchema(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 3375
    frame #5: 0x0000000109422536 libcyclus.dylib`cyclus::XMLFileLoader::master_schema() + 134
    frame #6: 0x00000001094225f1 libcyclus.dylib`cyclus::XMLFileLoader::LoadSim() + 65
    frame #7: 0x000000010000e161 cyclus`main + 13601
    frame #8: 0x00007fff788bf015 libdyld.dylib`start + 1
    frame #9: 0x00007fff788bf015 libdyld.dylib`start + 1

from d3ploy.

FlanFlanagan avatar FlanFlanagan commented on July 29, 2024

This is that same EXC_BAD_ACCESS.

Can you try it on your virtual machine of ubuntu with python 3.6?

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

The module import error re-occurs on the ubuntu virtual machine where everything has been installed with Python 3.6 and cyclus/cycamore pass their unit tests. cyclus linear_pow_demand.xml produces:
ERROR(core ):No module found for path libd3ploy.demand_driven_deployment_inst.so. The installed scipy is version 1.2.0.

from d3ploy.

FlanFlanagan avatar FlanFlanagan commented on July 29, 2024

If this is fixed can you close the issue @sonatav2 .

Thanks

from d3ploy.

katyhuff avatar katyhuff commented on July 29, 2024

from d3ploy.

FlanFlanagan avatar FlanFlanagan commented on July 29, 2024

Good idea!

from d3ploy.

sonatav2 avatar sonatav2 commented on July 29, 2024

There are still no .so or .dylib files in the given directories, but Python 3.6 (not downloaded through conda) on Ubuntu with scipy 1.2.0 and all of the dependencies installed using pip3 is able to successfully run the xml files in the input directory. One test in pytest fails (tests/integration_tests/continuous_integration_test), but that's a separate issue:

        for prototype in ['reactor', 'source']:
            row_count = cur.execute(query.replace('_', prototype)).fetchone()[0]
>           assert (row_count >= 1)
E           assert 0 >= 1

from d3ploy.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.