gafusion / omas Goto Github PK

View Code? Open in Web Editor NEW

30.0 8.0 14.0 24.25 MB

Ordered Multidimensional Array Structure

Home Page: http://gafusion.github.io/omas

License: MIT License

Python 0.57% Makefile 0.01% CSS 0.01% HTML 99.42% Shell 0.01% Cython 0.01%

storage-systems data-storage-infrastructure imas iter fusion omfit

omas's People

Contributors

Stargazers

Watchers

Forkers

coobas karel-van-de-plassche gkdb yangye4ever reksoatr pfbuxton stefsmeets odstrcilt jyang19 plasmafair hetsko hassec

omas's Issues

ODS leaf type consolidation and arrayification

For a non-standard use-case (parsing a ahead of time unknown nested structure) I needed to convert all leaf values from strings to ints/floats if possible, and to consolidated arrays of ODS's into a single array of floats/ints. I'm happy to open a pull request for these functions, but I'm not sure if you want to support this functionality and if so, where, e.g. as method of the ODS structure? Or function in omas_utils.

Sample equilibrium data are transposed

2d profile data in samples/sample_equilibrium_ods.json are transposed

Things in ods['equilibrium']['time_slice'][0]['profiles_2d'][0] that should be transposed back:

phi
psi
b_field_tor

load_omas_imas breaks if data_error_{upper, lower} not filled correctly

Dear All,

I was trying to read a set of IDS which contains data_error_{upper, lower} nodes that are not filled correctly. When OMAS tries to build an uarray (uncertainties) it will throw na exception NegativeStdDev: The standard deviation cannot be negative
breaking the load process. Could we add an option to just give a warning in those cases. Errors would be ignored. Actually this should be the default behaviour otherwise it can be a bummer for the user.
I am using omas version 0.29

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/afs/gw/switm/omfit/atom/omas/omas/omas_imas.py", line 456, in load_omas_imas
    data = uarray(data,stdata)
  File "/gw/switm/omfit/atom/miniconda2/lib/python2.7/site-packages/uncertainties/unumpy/core.py", line 287, in uarray
    (nominal_values, std_devs))
  File "/gw/switm/omfit/atom/miniconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2576, in __call__
    return self._vectorize_call(func=func, args=vargs)
  File "/gw/switm/omfit/atom/miniconda2/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2652, in _vectorize_call
    outputs = ufunc(*inputs)
  File "/gw/switm/omfit/atom/miniconda2/lib/python2.7/site-packages/uncertainties/unumpy/core.py", line 286, in <lambda>
    lambda v, s: uncert_core.Variable(v, s), otypes=[object])
  File "/gw/switm/omfit/atom/miniconda2/lib/python2.7/site-packages/uncertainties/core.py", line 2585, in __init__
    self.std_dev = std_dev  # Assignment through a Python property
  File "/gw/switm/omfit/atom/miniconda2/lib/python2.7/site-packages/uncertainties/core.py", line 2606, in std_dev
    raise NegativeStdDev("The standard deviation cannot be negative")
uncertainties.core.NegativeStdDev: The standard deviation cannot be negative

List IMAS issues to be discussed at upcoming IMAS code-camp

@gafusion/omas_developers I will be attending the IMAS code camp in early April. Please let me know here if you have issues/complaints with the IMAS data structure that you would like me to bring up when I am there.

use of equilibrium.time_slice[:].profiles_1d.psi as coordinate is a terrible idea #45
summary IDS is a catch-all
rotation (simulations use rotation frequencies, experiments use (midplane) velocities):
- core_profiles.profiles_1d[:].omega0 (http://gafusion.github.io/doc/rotation.html)
- core_profiles.profiles_1d[:].electrons.rotation.perpendicular
- core_profiles.profiles_1d[:].electrons.rotation.diamagnetic
- core_profiles.profiles_1d[:].electrons.parallel_stream_function
- core_profiles.profiles_1d[:].electrons.velocity.poloidal
- core_profiles.profiles_1d[:].electrons.velocity.radial
- core_profiles.profiles_1d[:].electrons.velocity.toroidal
- core_profiles.profiles_1d[:].ion[:].rotation.perpendicular
- core_profiles.profiles_1d[:].ion[:].rotation.diamagnetic
- core_profiles.profiles_1d[:].ion[:].rotation.parallel_stream_function
- core_profiles.profiles_1d[:].ion[:].velocity.poloidal
- core_profiles.profiles_1d[:].ion[:].velocity.radial
- core_profiles.profiles_1d[:].ion[:].velocity.toroidal
geometry
- equilibrium.time_slice[:].profiles_1d.centroid
- equilibrium.time_slice[:].profiles_1d.centroid.r_max
- equilibrium.time_slice[:].profiles_1d.centroid.r_min
- equilibrium.time_slice[:].profiles_1d.centroid.r
- equilibrium.time_slice[:].profiles_1d.centroid.z
per-quantity cocos transform rule https://github.com/gafusion/omas/blob/master/omas/omas_cocos.py should be part of IMAS data structure
obsolescent fields should have a pointer to what they got renamed to (if it's only a rename: eg. b_tor-->b_field_tor)
equilibrium_2d R/Z coordinates are flipped for some cases: eg. pulse 130501
Are machine parameters (such as pf_active) in COCOS11

equilibrium.time_slice[:].profiles_1d.psi

One of the fundamental coordinates for the equilibrium IDS is equilibrium.time_slice[:].profiles_1d.psi. This is a strange choice, however, as the limits of the psi grid can change as an equilibrium evolves. This is causing a problem in which I try to write a new gEQDSK to an existing ODS, but SIBRY has decreased. Thus, something like rho_tor_norm, which should always go from 0 to 1 but uses psi as a coordinate, gets truncated when interpolating onto the old psi grid. A few options for how to fix this:

A coordinate shouldn't change in time, so psi is a terrible choice. We could open a JIRA issue and request that the equilibrium IDS use something fixed and/or normalized. psi_norm would make sense, though it doesn't exist in that IDS right. rho_tor_norm does, but is not a coordinate typically used in MHD equilibrium calculations.
Unlike core_profiles, in which different quantities could come from different codes and diagnostics, an equilibrium should probably be entirely self-consistent. If something is writing to that IDS, perhaps we should assume that everything is getting updated self-consistently. In OMFIT, this would take the form of wiping the equilibrium before writing a new one. This would be good coding practice for users, but probably wouldn't need to be done at the framework level.

invalid pf_active geometry_type used

pf_active.coil[:].element[:].geometry.geometry_type = 0 is undefined
valid options are: {1: 'outline', 2: 'rectangle', 4: 'arcs of circle'}

This came up with ac95bca

load_omas_imas should check for ids_properties.homogeneous_time

if not len(getattr(ids, ds).time):

should become

if getattr(ids, ds).ids_properties.homogeneous_time not in [0,1]:

but needs testing

plot_core_profiles_summary broken

Executing in OMFIT

OMFIT['ods'] = load_omas_iter_scenario_remote(pulse=131034, run=0)
OMFIT['ods'].plot_core_profiles_summary()

yields

Traceback (most recent call last):
  File "/Users/smithsp/OMFIT-source/omfit/classes/OMFITx.py", line 3845, in manage_user_errors
    tmp=command(**kw)
  File "/Users/smithsp/OMFIT-source/omfit/utils_tk.py", line 1072, in GlobLoc
    return tmp.run(_relLoc=self.namespace, _OMFITscriptsDict=False, _OMFITconsoleDict=True, noGUI=None)
  File "/Users/smithsp/OMFIT-source/omfit/classes/omfit_python.py", line 864, in run
    result = self.__run__(**kw)
  File "/Users/smithsp/OMFIT-source/omfit/classes/omfit_python.py", line 746, in __run__
    tmp=execGlobLoc(self, kw, _relLoc, OMFITconsoleDict, {}, prerun=_prerun, postrun=_postrun)
  File "/Users/smithsp/OMFIT-source/omfit/classes/omfit_python.py", line 83, in f_locked
    return f(*args,**kw)
  File "/Users/smithsp/OMFIT-source/omfit/classes/omfit_python.py", line 290, in execGlobLoc
    exec (compile(exeString, filename, "exec"), GlobLoc)
  File "/tmp/smithsp/OMFIT/OMFIT_2019-03-11_13_49_10_583409/project/OMFIT command box #3", line 1, in <module>
    OMFIT['ods'].plot_core_profiles_summary()
  File "/Users/smithsp/omas/omas/omas_plot.py", line 606, in core_profiles_summary
    ax0.set_xlabel('$\\rho$')
AttributeError: 'NoneType' object has no attribute 'set_xlabel'

I looked at the code, and it seems that ax0 is not defined for some condition, but I am not sure of the solution.

Running omas make tests on iris

I am trying to test a small regression test in omas_plot that I added but have some issues with running the test suite on IRIS. It doesn't seem to get further than the init method of make tests.

From the omas folder I am running (I cloned the omas repo a couple of months ago and was never able to omas testing and always used a PR to do this):

>>>> module load omfit/unstable
>>>> make tests

Which gives the following error:

Makefile:42: warning: overriding commands for target `tests_examples'
Makefile:36: warning: ignoring old commands for target `tests_examples'
python3 -m unittest discover --pattern="*.py" -s omas/tests -v -f
/fusion/projects/codes/atom/omfit_py3/atom_SATURN_GCC/miniconda3_20200421/lib/python3.7/site-packages/xarray/core/merge.py:16: FutureWarning: The Panel class is removed from pandas. Accessing it from the top-level namespace will also be removed in the next version
Setting up OMAS warnings for user slendebroekt
OMAS warnings set to hard mode
Setting up OMAS warnings for user slendebroekt
OMAS warnings set to hard mode
__init__ (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: __init__ (unittest.loader._FailedTest)
----------------------------------------------------------------------
TypeError: __init__() missing 2 required positional arguments: 'method_name' and 'exception'

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)
make: *** [tests] Error 1

I was wondering if you have seen this before @eldond and/or if you know what I can try.

Thanks!

Populate omas_cocos.py with missing COCOS transformations

@gafusion/omas_developers the OMAS utility
https://github.com/gafusion/omas/blob/master/utilities/generate_cocos_signals.py
is designed to identify IMAS signals that may require COCOS transforms :) (Link edited.)

This utility populates the https://github.com/gafusion/omas/blob/master/omas/omas_cocos.py with likely candidates, while retaining the transform information for the entries that already had one.

Please, as you start using the different IDSs, edit the omas/utilities/generate_cocos_signals.py with the missing COCOS transforms.

This system should also be able to let us keep up with possible future new entries in the IMAS data structure.

equilibrium.time_slice[].profiles_1d.centroid is no part of IMAS

Hello
the centroid structure is present in the equilibrium ods but not in the corresponding ids.

When I use IMASgo to send my ods to IMAS, there is a warning message :
equilibrium.time_slice[].profiles_1d.centroid is no part of IMAS
It's not a problem because it is only a warning.

But it becomes a problem when I want to use the load_omas_imas routine to generate a geqdsk file from my IMAS data.
The error is: AttributeError: 'time_slice_profiles_1d__structure' object has no attribute 'centroid'.
It is not a warning, it is an error so it's blocking my program.

Can I deactivate/bypass the centroid part?

For now, I found a temporary solution. I modified the filled_paths_in_ids routine (in omas_imas.py). In the traverse loop, I added the test:
if kid in ['centroid']:
continue
It makes the job...

My OMAS version is a little old: 0.49.1
IMAS version 3.25.0-4.2.0

Thank you.

Laurent

profiles2d dimension order

All over the OMAS/OMFIT code, there is consistently assumed that the 2D equilibrium variables has shape [equilibrium.time_slice[:].profiles_2d[:].grid.dim2, equilibrium.time_slice[:].profiles_2d[:].grid.dim1]. However, according to the presented IMAS data-structure (https://gafusion.github.io/omas/schema/schema_equilibrium.html), 2d variables should have shape [equilibrium.time_slice[:].profiles_2d[:].grid.dim1, equilibrium.time_slice[:].profiles_2d[:].grid.dim2], i.e. to be transposed with respect to the implemented convention.

My colleague used OMAS to store data to IMAS on Gateway to run the ASCOT code (I think it required IMAS v 3.17) and he obtained correct results only if he used [dim1, dim2] shape of the psi matrix.

For example the order [dim2, dim1] is required here:

omas/omas/omas_plot.py

Lines 419 to 434 in c827e37

 if 'phi' in eq['profiles_2d'][0] and 'phi' in eq['profiles_1d']: 

 value2D = numpy.sqrt(abs(eq['profiles_2d'][0]['phi'])) 

 value1D = numpy.sqrt(abs(eq['profiles_1d']['phi'])) 

 else: 

 value2D = eq['profiles_2d'][0]['psi'] 

 value1D = eq['profiles_1d']['psi'] 

 value2D = (value2D - min(value1D)) / (max(value1D) - min(value1D)) 

 # contours 

 line = numpy.array([numpy.nan, numpy.nan]) 

 for item1 in contourPaths(eq['profiles_2d'][0]['grid']['dim1'], eq['profiles_2d'][0]['grid']['dim2'], value2D, 

 levels, smooth_factor=contour_smooth): 

 for item in item1: 

 line = numpy.vstack((line, item.vertices, numpy.array([numpy.nan, numpy.nan]))) 

 # internal flux surfaces w/ or w/o masking

omas/omas/omas_plot.py

Lines 63 to 68 in c827e37

 [X, Y] = numpy.meshgrid(x, y) 

 if compare_version(matplotlib.__version__, '2.1') >= 0: 

 contour_generator = _contour.QuadContourGenerator(X, Y, Z, None, True, 0) 

 else: 

 Cntr = matplotlib._cntr.Cntr(X, Y, Z)

Coordinate consistency

Right now, if I have an existing ODS and try to update data in it, there's no check that the coordinates I'm using for the new data are the same as what's already in the ODS. I propose the following, though I mainly want to get a discussion started.

Change cocos_environment to omas_environment
Add a new "coordinate_consistency" keyword (or similar) with the following options and behavior:

False: does nothing, assumes user will ensure consistency
True
- Coordinate values must be written before data can be entered
- If a coordinate value is entered and it doesn't exist, do nothing
- If it does exist, throw an exception if the grid is not the same as already in place
- If data is entered, ensure the dimensionality conforms with the coordinates
'new'
- New coordinates can be declared, overwriting old ones
- If a new coordinate is declared, existing data in tree is automatically interpolated
- All new data entered is checked for dimensionality conformity with the new coordinate
'old'
- New coordinates can be declared but do not overwrite old ones
- All new data entered gets interpolated onto the old coordinates

More robust tempfile settings for suite test

Can't refer to os.environ['HOME'] in creation of temporary directories: https://ci.appveyor.com/project/conda-forge/staged-recipes/build/1.0.19322/job/h1s0b2ik8588hkx6

Documentation fails

In the main directory (after make json), make html yields:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/sphinx_gallery/gen_rst.py", line 460, in _memory_usage
    out = func()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/sphinx_gallery/gen_rst.py", line 442, in __call__
    exec(self.code, self.fake_main.__dict__)
  File "/Users/smithsp/omas/omas/examples/plot_omas.py", line 15, in <module>
    ods = load_omas_s3('OMFITprofiles_sample', user='omas_shared')
  File "/Users/smithsp/omas/omas/omas_s3.py", line 129, in load_omas_s3
    remote_uri(_base_S3_uri(user) + filename, os.path.abspath(tmp_dir) + os.sep + os.sep + os.path.split(filename)[1], 'down')
  File "/Users/smithsp/omas/omas/omas_s3.py", line 65, in remote_uri
    obj.download_file(filename, Config=TransferConfig(use_threads=False))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/boto3/s3/inject.py", line 314, in object_download_file
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/boto3/s3/inject.py", line 172, in download_file
    extra_args=ExtraArgs, callback=Callback)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/boto3/s3/transfer.py", line 307, in download_file
    future.result()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/s3transfer/download.py", line 345, in _submit
    **transfer_future.meta.call_args.extra_args
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/botocore/client.py", line 272, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/botocore/client.py", line 576, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Installation of OMAS on NERSC

Has anybody tried installing OMAS on NERSC? Specifically, I am trying to install it on Cori and running into the following error when I try to run the "pip install ..." command when installing from a pull of the git repo. The error is the following

ERROR: Invalid requirement: 'build_documentation]'

Any help would be appreciated.

core_profiles_pressures() is broken

As noted in #31, core_profiles_pressures() is broken. https://github.com/gafusion/omas/pull/31/files#diff-cb60549dd1fcc7aca8fea8e20f1305cbR59
Once core_profiles_pressures() is fixed, it could be used in omas_sample.py to help build sample profiles.

conda build failed

https://circleci.com/gh/conda-forge/omas-feedstock/3?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

How can I plot the equilibrium cross section with contours of normalized psi or normailzed rho?

It used to be that the equilibrium_CX method in https://github.com/gafusion/omas/blob/master/omas/omas_plot.py could use 'phi' to determine 'rho' and also I think it provided normalized psi. Now gathering of contour_quantity usually goes through this get2d() function:

omas/omas/omas_plot.py

Line 642 in bf0b06b

def get2d(contour_quantity):

which relies on having 1d profiles present, but there's apparently no 1d profile of psi_norm, so this quantity seems unavailable. It's an easy calculation, I just don't know how to ask for it.

This change also defeats https://github.com/gafusion/OMFIT-source/blob/62cf92b7680050b9e06b6db30c71d9550e893265/omfit/classes/omfit_omas_utils.py#L143 , which tries to provide rho by writing phi, which worked under the old rules.

So I think I have to update multi_efit_to_omas() (should it include a version check and retain the old functionality for backwards compatibility?), and then equilibrium_CX() needs a special case to handle psi_N from psi in profiles_2d and psi_axis and psi_boundary in global_quantities.

pFile-IMAS translation issues

I've been working to put the pFile into an ODS, but there are some issues getting all the information there. We should figure out the best way to handle these, then open a JIRA issue:

Core profiles defines velocities, not rotations, on a rho_tor_norm grid. Velocities are not flux quantities, so it is not clear at what major radius the velocities should be taken. Either this should be declared (e.g., outer midplane), an R_major grid should be populated, or rotation frequency fields should be available.
core_profiles.profiles_1d[:].e_field asks for the flux-surface-averaged electric field, but the pFile only has the field at the outer midplane.
I don't see any place for the parallel stream function (Kpol in the pFile). I could derive this from the rotations/velocities if I have corresponding R and B_tor arrays on the rho_tor_norm grid.
I see no place for the ExB shearing rate. In particular, the pFile gives the Hahm-Burell expression for this. It looks like it could be derived given info about the magnetic field and flux, but would lose accuracy since I'd have to take a flux derivative.

omas_plot uses deprecated matplotlib module _cntr

While trying to work through samples/omas_omfit.py, I encountered an ImportError because omas/omas_plot.py is using a deprecated module from an old version of matplotlib. There is no matplotlib version specified in requirements.txt. See flatironinstitute/CaImAn#174.

Exception in script of module OMFIT['PCS']['_PCS_prad_control']:
Error in "OMFIT command box #1" at line  9
    omas_plot.equilibrium_summary(ods,linewidth=1)
ImportError: cannot import name _cntr

Press <F6> to see full error report...

Traceback (most recent call last):
  File "/home/eldond/OMFIT-source/omfit/OMFITx.py", line 3590, in manage_user_errors
    tmp=command(**kw)
  File "/home/eldond/OMFIT-source/omfit/utils_tk.py", line 835, in GlobLoc
    return tmp.run(_relLoc=self.namespace, _OMFITscriptsDict=False, _OMFITconsoleDict=True, noGUI=None)
  File "/home/eldond/OMFIT-source/omfit/omfit_tree.py", line 1277, in run
    result = self.__run__(**kw)
  File "/home/eldond/OMFIT-source/omfit/omfit_tree.py", line 1185, in __run__
    tmp=execGlobLoc(self, kw, _relLoc, OMFITconsoleDict, {}, prerun=_prerun, postrun=_postrun)
  File "/home/eldond/OMFIT-source/omfit/omfit_tree.py", line 127, in f_locked
    return f(*args,**kw)
  File "/home/eldond/OMFIT-source/omfit/omfit_tree.py", line 283, in execGlobLoc
    exec(compile(exeString, filename, "exec"),GlobLoc)
  File "/tmp/eldond/OMFIT/OMFIT_2018-05-31_15_40_36_986220/project/OMFIT command box #1", line 9, in <module>
    omas_plot.equilibrium_summary(ods,linewidth=1)
  File "/usr/lib/python2.7/site-packages/omas/omas_plot.py", line 287, in equilibrium_summary
    ax = equilibrium_CX(ods, time_index=time_index, ax=ax, **kw)
  File "/usr/lib/python2.7/site-packages/omas/omas_plot.py", line 227, in equilibrium_CX
    levels, smooth_factor=1):
  File "/usr/lib/python2.7/site-packages/omas/omas_plot.py", line 37, in contourPaths
    from matplotlib import _cntr
ImportError: cannot import name _cntr

Failing import of OMAS

I am running pytests in gitlab CI and it seems that the enviroment has not USER enviroment and as a consequence the test fails.
Not sure whether is this bug or not.

_____________________ ERROR collecting tests/omas_test.py ______________________
tests/omas_test.py:4: in <module>
    import omas
/opt/conda/envs/pleque/lib/python3.6/site-packages/omas/__init__.py:1: in <module>
    from .omas_core import *
/opt/conda/envs/pleque/lib/python3.6/site-packages/omas/omas_core.py:1219: in <module>
    from .omas_imas import *
/opt/conda/envs/pleque/lib/python3.6/site-packages/omas/omas_imas.py:364: in <module>
    def load_omas_imas(user=os.environ['USER'], machine=None, pulse=None, run=0, paths=None,
/opt/conda/envs/pleque/lib/python3.6/os.py:669: in __getitem__
    raise KeyError(key) from None
E   KeyError: 'USER'

Failed tests for 0.10.3

The error raised by the tests (see https://circleci.com/gh/conda-forge/omas-feedstock/34?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link) was:

running run_test.py
OMAS plotting function are not available: ModuleNotFoundError("No module named 'matplotlib'",)
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 462, in ods_sample
    print(ods['wall.description_2d.0.limiter.unit.0.outline.r'])
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 243, in __getitem__
    return self.omas_data[key[0]][l2o(key[1:])]
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 243, in __getitem__
    return self.omas_data[key[0]][l2o(key[1:])]
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 243, in __getitem__
    return self.omas_data[key[0]][l2o(key[1:])]
  [Previous line repeated 3 more times]
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 235, in __getitem__
    dynamic_path_creation=self.dynamic_path_creation))
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 157, in __setitem__
    raise(ValueError('`%s` has no data'%location))
ValueError: `wall.description_2d.limiter.unit.outline.r` has no data

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/test_tmp/run_test.py", line 11, in <module>
    omas.test_omas_suite()
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 570, in test_omas_suite
    ods = ods_sample()
  File "/home/conda/feedstock_root/build_artifacts/omas_1525321528628/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/python3.6/site-packages/omas/omas_core.py", line 464, in ods_sample
    assert 'wall.description_2d.0.limiter.unit.0.outline' not in ods
AssertionError

I recommend adding the testing to the release procedure.

loading an IDS from a previous data versio can fail

Recently I was trying to load IDSs written with an older data version than the current one, and found this issue:

at the ITM/gateway try:

module load imasenv
module load omfit
python -c "from omas import *; ods = load_omas_imas(shot=92436,run=172,machine=u'jet',user=u'g2jofe',imas_version=u'3.20.0')"

but it fails with:

core_transport.model.7.profiles_1d.0.ion.2.element.0.multiplicity is in OBSOLESCENT state
core_transport.model.8.profiles_1d.0.ion.0.element.0.multiplicity is in OBSOLESCENT state
core_transport.model.8.profiles_1d.0.ion.1.element.0.multiplicity is in OBSOLESCENT state
core_transport.model.8.profiles_1d.0.ion.2.element.0.multiplicity is in OBSOLESCENT state
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/afs/gw/switm/omfit/atom/omas/omas/omas_imas.py", line 460, in load_omas_imas
    h = h[step]
  File "/afs/gw/switm/omfit/atom/omas/omas/omas_core.py", line 580, in __getitem__
    cocos=self.cocos, cocosio=self.cocosio, coordsio=self.coordsio))
  File "/afs/gw/switm/omfit/atom/omas/omas/omas_core.py", line 423, in __setitem__
    spaces + '^' * len(structure_key) + '\n' + '%s' % options)
LookupError: `ec_antennas.antenna.0.power` is not a valid IMAS 3.21.0 location
                                    ^^^^^
Did you mean: [u'launching_angle_pol', u'launching_position', u'launching_angle_tor', u'name', u'frequency_error_lower', u'power_launched', u'beam', u'frequency', u'frequency_error_upper', u'identifier', u'frequency_error_index', u'mode']

which should not happen since I requested a load with imas_version=u'3.20.0'

My 92436/172 pulse was created with the IMAS data structure 3.20.0.

The power in ec_antennas.antenna.%.power, nbi.unit.%.power or ic_antennas.antenna.%.power changed to power_launched in 3.21.0, the current/latest data structure.

Seems that imas_version is not propagated to the consistency check loop.

Function to find time_slice at a given time

Typically I might know a time for which I want data, but I won't know what time_slice that occurs at within an ODS. There's a slice_at_time function, but that appears to literally slice the ODS at that time (at labels it time_slice=0). Could you please write a function that would just return the integer time_slice that a given time corresponds to?

There's also a complication of where the time get set. For example, within OMFIT's OMFITgeqdsk.to_omas, the time from a gEQDSK gets set to ods['equilibrium.time_slice'][time_index]['time'], but does not get put in the ods['equilibrium.time'] array. It looks like slice_at_time is only looking for the former.

No access to repository

@orso82 I worked on the COCOS functionality for core_profiles and core_sources. While I was able to clone the repository, I can't push to remote. I'd like to make a remote development branch and pull request my changes. Could you add me please?

pip install of omas on iris

pip install of omas on iris is failing for me with the following error:

pip install --upgrade -e .[build_structures,build_documentation]
....
....
Using cached https://files.pythonhosted.org/packages/b2/4c/b6f966ac91c5670ba4ef0b0b5613b5379e3c7abdfad4e7b89a87d73bae13/pandas-0.24.2.tar.gz
   Complete output from command python setup.py egg_info:
   Traceback (most recent call last):
     File "<string>", line 1, in <module>
     File "/tmp/pip-install-Y9FfVI/pandas/setup.py", line 746, in <module>
       **setuptools_kwargs)
     File "/fusion/usc/opt/python/2.7.11/lib/python2.7/distutils/core.py", line 111, in setup
       _setup_distribution = dist = klass(attrs)
     File "build/bdist.linux-x86_64/egg/setuptools/dist.py", line 269, in __init__
     File "build/bdist.linux-x86_64/egg/setuptools/dist.py", line 313, in fetch_build_eggs
     File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 825, in resolve
   
     File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 1070, in best_match
   
     File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 1082, in obtain
   
     File "build/bdist.linux-x86_64/egg/setuptools/dist.py", line 380, in fetch_build_egg
     File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 640, in easy_install
     File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 670, in install_item
     File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 853, in install_eggs
     File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1081, in build_and_install
     File "build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py", line 1067, in run_setup
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 246, in run_setup
     File "/fusion/usc/opt/python/2.7.11/lib/python2.7/contextlib.py", line 35, in __exit__
       self.gen.throw(type, value, traceback)
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 195, in setup_context
     File "/fusion/usc/opt/python/2.7.11/lib/python2.7/contextlib.py", line 35, in __exit__
       self.gen.throw(type, value, traceback)
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 166, in save_modules
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 141, in resume
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 154, in save_modules
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 195, in setup_context
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 243, in run_setup
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 273, in run
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 242, in runner
     File "build/bdist.linux-x86_64/egg/setuptools/sandbox.py", line 45, in _execfile
     File "/tmp/easy_install-duGNUu/numpy-1.19.0rc1/setup.py", line 182
       raise ValueError(f'Submodule {p} missing')
                                               ^
   SyntaxError: invalid syntax
   
   ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-Y9FfVI/pandas/
You are using pip version 18.1, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

consistency_check in omas_environment

Is the following supposed to work?

import omas
ods = omas.ODS()
with omas.omas_environment(ods, consistency_check=False):
    ods['nbi']['unit']['this_path_does_not_exist'] = 2

I get the error:

Traceback (most recent call last):
  File "test_omas.py", line 4, in <module>
    ods['nbi']['unit']['this_path_does_not_exist'] = 2
  File "/fusion/projects/codes/atom/omfit_py3/atom_SATURN_GCC/miniconda3/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/home/smithsp/omas/omas/omas_physics.py", line 1007, in omas_environment
    setattr(ods, item, bkp_args[item])
  File "/home/smithsp/omas/omas/omas_core.py", line 371, in consistency_check
    self.getraw(item).consistency_check = consistency_value_propagate
  File "/home/smithsp/omas/omas/omas_core.py", line 371, in consistency_check
    self.getraw(item).consistency_check = consistency_value_propagate
  File "/home/smithsp/omas/omas/omas_core.py", line 358, in consistency_check
    raise LookupError(underline_last(text, len('LookupError: ')) + '\n' + options)
LookupError: Not a valid IMAS 3.25.0 location: nbi.unit.this_path_does_not_exist
                                                        ^^^^^^^^^^^^^^^^^^^^^^^^
A numerical index is needed with n>=0

when the context exits.

Failing to read ITER data

In [1]: import omas

In [2]: import imas

In [3]: ods = omas.load_omas_iter_scenario(pulse=131034, run=0)
-  amns_data
-  barometry
-  bolometer
-  charge_exchange
-  coils_non_axisymmetric
-  controllers
-  core_instant_changes
*  core_profiles
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-b7ce734437e0> in <module>()
----> 1 ods = omas.load_omas_iter_scenario(pulse=131034, run=0)

/work/imas/opt/EasyBuild/software/OMAS/0.35.0-intel-2018a-Python-3.6.4/lib/python3.6/site-packages/omas-0.35.0-py3.6.egg/omas/omas_imas.py in load_omas_iter_scenario(pulse, run, paths, imas_version, verbose)
    586 
    587             # load data from imas
--> 588             ods = load_omas_imas(user=None, machine=None, pulse=pulse, run=run, paths=paths, imas_version=imas_version, verbose=verbose)
    589 
    590         finally:

/work/imas/opt/EasyBuild/software/OMAS/0.35.0-intel-2018a-Python-3.6.4/lib/python3.6/site-packages/omas-0.35.0-py3.6.egg/omas/omas_imas.py in load_omas_imas(user, machine, pulse, run, paths, imas_version, verbose)
    434                     if verbose:
    435                         print('* ', ds)
--> 436                     available_paths = filled_paths_in_ids(ids, load_structure(ds, imas_version=imas_version)[1], [], [])
    437                     if joined_paths is None:
    438                         # if joined_paths is None, this means the user requested everything

/work/imas/opt/EasyBuild/software/OMAS/0.35.0-intel-2018a-Python-3.6.4/lib/python3.6/site-packages/omas-0.35.0-py3.6.egg/omas/omas_imas.py in filled_paths_in_ids(ids, ds, path, paths, assume_uniform_array_structures)
    623     # keys
    624     keys = ds.keys()
--> 625     if keys[0] == ':':
    626         keys = range(len(ids))
    627         if len(keys) and assume_uniform_array_structures:

TypeError: 'dict_keys' object does not support indexing

Loaded modules:

[kripnel@hpc-login02 ~]$ module list
Currently Loaded Modulefiles:
  1) GCCcore/6.4.0                                       31) Szip/2.1.1-GCCcore-6.4.0
  2) binutils/2.28-GCCcore-6.4.0                         32) HDF5/1.10.1-intel-2018a
  3) GCC/6.4.0-2.28                                      33) XZ/5.2.3-GCCcore-6.4.0
  4) icc/2018.1.163-GCC-6.4.0-2.28                       34) libxml2/2.9.7-GCCcore-6.4.0
  5) ifort/2018.1.163-GCC-6.4.0-2.28                     35) MDSplus/7.46.1-intel-2018a
  6) iccifort/2018.1.163-GCC-6.4.0-2.28                  36) MDSplus-Java/7.46.1-intel-2018a-Java-1.8.0_162
  7) impi/2018.1.163-iccifort-2018.1.163-GCC-6.4.0-2.28  37) Blitz++/0.10-GCCcore-6.4.0
  8) iimpi/2018a                                         38) libMemcached/1.0.18-GCCcore-6.4.0
  9) imkl/2018.1.163-iimpi-2018a                         39) PCRE/8.41-GCCcore-6.4.0
 10) intel/2018a                                         40) SWIG/3.0.12-intel-2018a-Python-2.7.14
 11) numactl/2.0.11-GCCcore-6.4.0                        41) Boost/1.66.0-intel-2018a
 12) PGI/18.4-GCC-6.4.0-2.28                             42) cURL/7.58.0-GCCcore-6.4.0
 13) Java/1.8.0_162                                      43) netCDF/4.6.0-intel-2018a
 14) Anaconda3/5.0.1                                     44) MDSplus-Python/7.46.1-intel-2018a-Python-2.7.14
 15) bzip2/1.0.6-GCCcore-6.4.0                           45) UDA/2.2.2-intel-2018a-Java-1.8.0_162-Python-2.7.14
 16) zlib/1.2.11-GCCcore-6.4.0                           46) ant/1.10.1-Java-1.8.0_162
 17) ncurses/6.0-GCCcore-6.4.0                           47) IMAS/3.21.0-3.8.9
 18) libreadline/7.0-GCCcore-6.4.0                       48) cftime/1.0.0-intel-2018a-Python-3.6.4
 19) Tcl/8.6.8-GCCcore-6.4.0                             49) netcdf4-python/1.4.0-intel-2018a-Python-3.6.4
 20) SQLite/3.21.0-GCCcore-6.4.0                         50) xarray/0.10.8-intel-2018a-Python-3.6.4
 21) GMP/6.1.2-GCCcore-6.4.0                             51) BeautifulSoup/4.6.3-intel-2018a-Python-3.6.4
 22) libffi/3.2.1-GCCcore-6.4.0                          52) Sphinx/1.8.3-intel-2018a-Python-3.6.4
 23) libyaml/0.1.7-GCCcore-6.4.0                         53) NASM/2.13.03-GCCcore-6.4.0
 24) PyYAML/3.12-intel-2018a-Python-2.7.14               54) libjpeg-turbo/1.5.3-GCCcore-6.4.0
 25) libpng/1.6.34-GCCcore-6.4.0                         55) LibTIFF/4.0.9-GCCcore-6.4.0
 26) freetype/2.9-GCCcore-6.4.0                          56) Pillow/5.0.0-intel-2018a-Python-3.6.4
 27) expat/2.2.5-GCCcore-6.4.0                           57) OMAS/0.35.0-intel-2018a-Python-3.6.4
 28) fontconfig/2.12.6-GCCcore-6.4.0                     58) Python/3.6.4-intel-2018a
 29) X11/20180131-GCCcore-6.4.0                          59) matplotlib/2.1.2-intel-2018a-Python-3.6.4

Querying unknown quantities enters them in ODS

If I have an ODS and ask for the value of a location that doesn't exist, for example:
ods['equilibrium']['time_slice'][0]['global_quantities']['randomcharacters']
it will tell me that that's not a valid location, but then create it inside the ODS with value None. If I then query it again it won't throw any warning or error.

Suggest that this should raise a KeyError exception.

Directly use COCOS information from data dictionary

Now IMAS DD includes COCOS information. We need to decipher their nomenclature and interface it with what we already have in OMAS.

fc7fc3d#diff-63fbd44b1acff31692fb109a7e658549

Ensure omas writes in COCOS format 11

Not sure if omas is only writing in COCOS format 11, which seems to be a convention for IMAS.
Consult with Olivier Sauter and Rui (among others).

Conversion between tabular/tensor-like data and nested tree data

Posting an issue as I couldn't find it clearly in the documentation. Are there axis-aware tools already in OMAS that can convert between tabular data and nested data? I know IMAS likes nestedness, but I tend to prefer pandas DataFrames and/or xarray DataSets. A column in a DataFrame would be equivalent to a leaf node in nested tree, which is not the same as the current omas to_hdf5 functions if I understand correctly, where every leaf is still a leaf, and not matched to its respective time/radial basis.

Python 3 + CI

Hi omas-devs,

what about ensuring Python 3 compatibility of omas in CI (Travis @ github)? I think omas is a reasonably small and fresh project to be Python 3 compatible :) I'm hitting Python 3 errors with omas every now and then :-Z

Cheers
Jakub

ODS needs deepcopy ability

As mentioned in https://github.com/gafusion/OMFIT-source/commit/f4d328b9379c27eeb7c346108b83c858347bebf9#commitcomment-32071184, the ODS class needs the ability to turn off checks during deepcopy.

add OMAS_DEBUG_STDOUT environmental variable

to write debug statements to stdout rather than stderr

Add documentation example showing use of OMAS plot functions

Add core_transport sample

Request adding equilibrium.prof1d.centroid to IMAS datastructure

gfile.to_omas and equilibrium codes populate:

ods['equilibrium']['time_slice'][0]['profiles_1d']['centroid']

with the following 1d arrays:

r 
r_max
r_min
z

This is however not implemented in the IMAS datastrucute (also not in https://gafusion.github.io/omas/schema/schema_equilibrium.html).

ods_sample fails for 0.44.1

I installed omas 0.44.1 with pip. The command v = ods_sample() fails:
python3 v0.44.1

~/OMFIT-source 53 : python
Python 3.7.3 (default, Mar 30 2019, 03:37:43) 
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
/Users/smithsp/.pyhistory
>>> import omas
>>> import omas.omas_sample
>>> omas.omas_sample.ods_sample()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/omas/omas_sample.py", line 35, in ods_sample
    ods = eval(ds)(ods)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/omas/omas_sample.py", line 69, in equilibrium
    eq = load_omas_json(imas_json_dir + '/../samples/sample_equilibrium_ods.json', consistency_check=False)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/omas/omas_json.py", line 70, in load_omas_json
    with open(filename, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/omas/imas_structures/../samples/sample_equilibrium_ods.json'

python2 v0.44.2

>>> import omas
^[[A>>> import omas.omas_sample
>>> omas.omas_sample.ods_sample()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/omas/omas_sample.py", line 35, in ods_sample
    ods = eval(ds)(ods)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/omas/omas_sample.py", line 69, in equilibrium
    eq = load_omas_json(imas_json_dir + '/../samples/sample_equilibrium_ods.json', consistency_check=False)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/omas/omas_json.py", line 70, in load_omas_json
    with open(filename, 'r') as f:
IOError: [Errno 2] No such file or directory: u'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/omas/imas_structures/../samples/sample_equilibrium_ods.json'
>>> omas.__version__
'0.44.2'

I was going to use this to demonstrate OMFITcollection for #74 .

IMAS error setting equilibrium.time_slice[0].profiles_2d[0].grid_type=1

I suspect this is an IMAS issue, since I am assigning this entry a scalar, as I am already doing for other quantities.

Error setting: [u'time_slice', 0, u'profiles_2d', 0, u'grid_type']
Traceback (most recent call last):
  File "geqdsk_to_from_imas.py", line 21, in <module>
    shot, run, new)
  File "/home/meneghini/atom/omas/omas/omas_imas.py", line 213, in save_omas_imas
    imas_set(ids,path,ods[path],True)
  File "/home/meneghini/atom/omas/omas/omas_imas.py", line 104, in imas_set
    m.put(0)
  File "/fusion/projects/codes/imas/core/imas/3.10.1/ual/bugfix/IMAS-994-failed-make-install-after-make/python/lib.linux-x86_64-2.7/imas_3_10_1_ual_3_5_4_dev48/equilibrium.py", line 90, in put
    self.time_slice.put(path, cpopath, homogeneous_time)
  File "/fusion/projects/codes/imas/core/imas/3.10.1/ual/bugfix/IMAS-994-failed-make-install-after-make/python/lib.linux-x86_64-2.7/imas_3_10_1_ual_3_5_4_dev48/equilibrium.py", line 1006, in put
    obj1 = self.array[i].putInObject( path, '', 0, obj1)
  File "/fusion/projects/codes/imas/core/imas/3.10.1/ual/bugfix/IMAS-994-failed-make-install-after-make/python/lib.linux-x86_64-2.7/imas_3_10_1_ual_3_5_4_dev48/equilibrium.py", line 1209, in putInObject
    obj = self.profiles_2d.putInObject(path, cpopath + 'profiles_2d', i, obj)
  File "/fusion/projects/codes/imas/core/imas/3.10.1/ual/bugfix/IMAS-994-failed-make-install-after-make/python/lib.linux-x86_64-2.7/imas_3_10_1_ual_3_5_4_dev48/equilibrium.py", line 19793, in putInObject
    obj = self.array[i].putInObject(path, '', i, obj)
  File "/fusion/projects/codes/imas/core/imas/3.10.1/ual/bugfix/IMAS-994-failed-make-install-after-make/python/lib.linux-x86_64-2.7/imas_3_10_1_ual_3_5_4_dev48/equilibrium.py", line 20182, in putInObject
    obj = self.grid_type.putInObject(path, cpopath + 'grid_type', i, obj)
AttributeError: 'numpy.ndarray' object has no attribute 'putInObject'

Error with load_omas_imas_remote module has no attribute fortran_environment

Hi OMAS devs,

I'm attempting to use the load_omas_imas_remote functionality from freia at UKAEA. However it is throwing an exception before telling me sensible reasons as to why it won't work, like incorrect pulse number or something.

What I did:

I use module load omfit-dev/20190417 which gives omas and omfit to python 2.7. Then in ipython I do the following

In [1]: from omas import *
In [2]: from omfit.classes.omfit_omas import load_omas_imas_remote
In [3]: ods = load_omas_imas_remote(serverPicker='iter_login', machine='ITER',user='cookj', pulse=123456,
   ...:  run=1)

which gives the error message AttributeError: 'module' object has no attribute 'fortran_environment'. Complete output below:

Loading OMFIT APIs...
Setting up environment...
External imports...
Loading utility functions...
Loading plot utility functions...
Loading widgets utility functions...
Loading tk functions...
Loading fit utility functions...
--------------------
Linux freia053 3.10.0-862.3.3.el7.x86_64 #1 SMP Thu Jun 14 15:28:39 CDT 2018 x86_64 x86_64
v0.26.13-1995-g9af095f6c on branch unstable
Installation type     : PERSONAL
OMFITstartDir         : /home/cookj
OMFITsrc              : /common/transp_shared/omfit/atom-dev/OMFIT-source/omfit
OMFITsettingsDir      : /home/cookj/.OMFIT
OMFITsessionsDir      : /tmp/cookj/OMFIT_local/sessions
OMFITcontrolmastersDir: /tmp/cookj/OMFIT_local/controlmasters
OMFITscriptsBackupDir : /tmp/cookj/OMFIT_scripts_backup
OMFITautosaveDir      : /tmp/cookj/OMFIT_projects_autosave
OMFITtmpDir           : /tmp/cookj/OMFIT
OMFITsessionDir       : /tmp/cookj/OMFIT/OMFIT_2019-04-24_08_58_59_912441
OMFITcwd              : /tmp/cookj/OMFIT/OMFIT_2019-04-24_08_58_59_912441/project
--------------------
Time to load startup_framework: 1.29012 seconds
Loading OMFIT classes...
Loading widgets utility functions...
Loading plot utility functions...
UserWarning: No `boutdata` and `boututils` support (BOUT_TOP environment variable): ImportError('No module named boutdata',) (omfit_bout.py@37)
UserWarning: No `pact.pdb` format support: ImportError('No module named pact.pdb',) (omfit_pdb.py@21)
UserWarning: No `CDB` support (CDB_PATH environment variable): ImportError('No module named pyCDB',) (omfit_cdb.py@33)
UserWarning: No GACODE support ($GACODE_ROOT=''): ImportError('No module named tgyro.data',) (omfit_gacode.py@59)
Offending module: omfit_namelist
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-50092e1bdf4b> in <module>()
----> 1 ods = load_omas_imas_remote(serverPicker='iter_login', machine='ITER',user='cookj', pulse=123456, run=1)

/common/transp_shared/omfit/atom-dev/OMFIT-source/omfit/classes/omfit_omas.pyc in load_omas_imas_remote(serverPicker, **kw)
    159     :param \**kw: all other parameters are passed to the load_omas_imas function
    160     """
--> 161     return _base_omas_remote(serverPicker, 'load_omas_imas', **kw)
    162 try:
    163     load_omas_imas_remote.__doc__ += load_omas_imas.__doc__.replace('\n    ', '\n        ')

/common/transp_shared/omfit/atom-dev/OMFIT-source/omfit/classes/omfit_omas.pyc in _base_omas_remote(serverPicker, target_function, python_script, python, quiet, forceRemote, IMAS_module, OMAS_module, UDA_module, **kw)
    122 python'''
    123 
--> 124     import OMFITx
    125     return OMFITxAttributeError: 'module' object has no attribute 'fortran_environment'.remote_python(None,
    126                                 executable=python + ' -u %s',

/common/transp_shared/omfit/atom-dev/OMFIT-source/omfit/classes/OMFITx.py in <module>()
      5 print('Loading OMFIT APIs...')
      6 
----> 7 from omfit_tree import *
      8 from utils_widgets import _defaultFont
      9 from classes.utils_base import _streams

/common/transp_shared/omfit/atom-dev/OMFIT-source/omfit/omfit_tree.py in <module>()
     59         if _python_module != 'omfit_base':
     60             for item in getattr(classes, _python_module).__all__:
---> 61                 setattr(classes.omfit_base, item, getattr(getattr(classes, _python_module), item))
     62 
     63         if _python_module != 'omfit_python':

AttributeError: 'module' object has no attribute 'fortran_environment'

Any ideas how to fix this?

Thanks,
James

tests directory missing in pip package

When I install the 0.18.1 pip package, there is no omfit.tests module. Please add it in the next release.

add core_sources sample

Not use pickles for remote transfers

As pointed out by @jsferreira in https://github.com/gafusion/OMFIT-source/issues/2616#issuecomment-377180597 there can be problems transferring data as pickles, due to possible inconsistencies in versions of packages installed. Can't one of the other formats be used: netcdf or json? I know they can be slower, but they are more robust to this type of issue.

Timing load of IMAS data

see 0f7285a#commitcomment-32969860

ITER case: plotting is failing

Failing plotting for ITER baseline:

ods = omas.load_omas_iter_scenario(pulse=130501, run=1)
In [28]: omas.omas_plot.equilibrium_summary(ods)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-28-913587c8af9d> in <module>()
----> 1 omas.omas_plot.equilibrium_summary(ods)

~/.local/lib/python3.6/site-packages/omas/omas_plot.py in equilibrium_summary(ods, time_index, fig, **kw)
    483 
    484     ax = fig.add_subplot(1, 3, 1)
--> 485     ax = equilibrium_CX(ods, time_index=time_index, ax=ax, **kw)
    486     eq = ods['equilibrium']['time_slice'][time_index]
    487 

~/.local/lib/python3.6/site-packages/omas/omas_plot.py in equilibrium_CX(ods, time_index, contour_smooth, levels, ax, **kw)
    422     line = numpy.array([numpy.nan, numpy.nan])
    423     for item1 in contourPaths(eq['profiles_2d'][0]['grid']['dim1'], eq['profiles_2d'][0]['grid']['dim2'], value2D.T,
--> 424                               levels, smooth_factor=contour_smooth):
    425         for item in item1:
    426             line = numpy.vstack((line, item.vertices, numpy.array([numpy.nan, numpy.nan])))

~/.local/lib/python3.6/site-packages/omas/omas_plot.py in contourPaths(x, y, Z, levels, remove_boundary_points, smooth_factor)
     71     [X, Y] = numpy.meshgrid(x, y)
     72     if compare_version(matplotlib.__version__, '2.1') >= 0:
---> 73         contour_generator = _contour.QuadContourGenerator(X, Y, Z, None, True, 0)
     74     else:
     75         Cntr = matplotlib._cntr.Cntr(X, Y, Z)

ValueError: x, y and z must all be 2D arrays with the same dimensions

Check PINT performance tips

https://pint.readthedocs.io/en/latest/performance.html

Add new nbi plot

@eldond Can you point me to the right workflow for adding a plot for NBI related stuff in an ODS (especially omas vs OMFIT)? I want to plot beam powers/voltage/energy fraction per 'energy harmonic' vs time and possibly also top-down and poloidal cross-section plots.

	if 'phi' in eq['profiles_2d'][0] and 'phi' in eq['profiles_1d']:
	value2D = numpy.sqrt(abs(eq['profiles_2d'][0]['phi']))
	value1D = numpy.sqrt(abs(eq['profiles_1d']['phi']))
	else:
	value2D = eq['profiles_2d'][0]['psi']
	value1D = eq['profiles_1d']['psi']
	value2D = (value2D - min(value1D)) / (max(value1D) - min(value1D))

	# contours
	line = numpy.array([numpy.nan, numpy.nan])
	for item1 in contourPaths(eq['profiles_2d'][0]['grid']['dim1'], eq['profiles_2d'][0]['grid']['dim2'], value2D,
	levels, smooth_factor=contour_smooth):
	for item in item1:
	line = numpy.vstack((line, item.vertices, numpy.array([numpy.nan, numpy.nan])))

	# internal flux surfaces w/ or w/o masking

	[X, Y] = numpy.meshgrid(x, y)
	if compare_version(matplotlib.__version__, '2.1') >= 0:
	contour_generator = _contour.QuadContourGenerator(X, Y, Z, None, True, 0)
	else:
	Cntr = matplotlib._cntr.Cntr(X, Y, Z)

gafusion / omas Goto Github PK

omas's People

Contributors

Stargazers

Watchers

Forkers

omas's Issues

Recommend Projects

Recommend Topics

Recommend Org