tofuproject / tofu Goto Github PK
View Code? Open in Web Editor NEWProject for an open-source python library for synthetic diagnostics and tomography for Fusion devices
Home Page: https://tofuproject.github.io/tofu/index.html
License: MIT License
Project for an open-source python library for synthetic diagnostics and tomography for Fusion devices
Home Page: https://tofuproject.github.io/tofu/index.html
License: MIT License
Useful for tIGNITRON on WEST
Wait for imas to handle events
Add the possibility of plotting Equilibrium and 2D map on :
tf.data.Data.plot_spectrogram()
tf.data.Data.plot_svd()
All tests should be updated with:
Since I merged with the latests version of master
(with the new module for imas) I get the following error when doing python setup.py nosetests
. I don't get it when just calling nosetests [options]
. It seems when nosetests is ran from distutils
there can be this kind of problems (see this SO post).
Error:
======================================================================
ERROR: Failure: Exception (IMAS python API issue
imas could not be imported into tofu ('import imas' failed):
it may not be installed (optional dependency)
or you not have loaded the good working environment
=> the optional sub-package tofu.imas2tofu is not usable
)
Traceback (most recent call last):
File "/home/u2/mendoza/.conda/envs/tofu3/lib/python3.7/site-packages/nose/failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "/home/u2/mendoza/.conda/envs/tofu3/lib/python3.7/site-packages/nose/loader.py", line 417, in loadTestsFromName
addr.filename, addr.module)
File "/home/u2/mendoza/.conda/envs/tofu3/lib/python3.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/home/u2/mendoza/.conda/envs/tofu3/lib/python3.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/home/u2/mendoza/.conda/envs/tofu3/lib/python3.7/imp.py", line 244, in load_module
return load_package(name, filename)
File "/home/u2/mendoza/.conda/envs/tofu3/lib/python3.7/imp.py", line 216, in load_package
return _load(spec)
File "", line 696, in _load
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/u2/mendoza/tofu/tofu/imas2tofu/init.py", line 16, in
raise Exception(msg)
Exception: IMAS python API issue
imas could not be imported into tofu ('import imas' failed):
it may not be installed (optional dependency)
or you not have loaded the good working environment
=> the optional sub-package tofu.imas2tofu is not usable
Make the input polygons of Ves and Struct more flexible (type):
The dependency on git during setup is an issue wen installing with easyconfig:
see https://jira.iter.org/browse/IMAS-2287?filter=-3 for context
Explanation:
I got your email Vezinet Didier, asking how to make the package easier (i.e. getting rid of the patch).
What the patch does is remove the part that requires git commands during the python setup.py build step. Why? because the .eb file downloads the tarball – so there is no git repository inside.
How the packaging would be better: remove the need for git. You can probably include a test that inspects whether setup.py is run from a git repo or not. E.g. check for the .git directory's existence, or make git describe fail gracefully and continue with the default version – now it breaks
It is not possible now to choose which LOS is used for computing the LOS signal.
Idem dans :
GDetect.calc_Sig()
GDetect.plot_Sig()
The longitudinal sampling of a large number of LOS can be used for:
When possible the sampling is vectorized.
But in many cases it cannot be vectorized (if the number of samples it different on each LOS).
In this case a for loop is used (Cython)
This algorithm is not optimal and can be optimized and parallelized.
Furthermore, 3 different cases are possible for the output format. This is not necessary, a single format (concatenated array) would result in simpler code and more efficient computing.
The concerned routines are (in tf.geom._GG02.pyx):
Two algorithms are envisaged for this:
1/ make semi-plane from LOS in any direction, count intersections with polygon (even vs odd)
2/ triangulate polygon from barycenter, count intersections of LOS with triangulated surface (even vs odd)
Tasks:
There's an error when cam is IN device with "optimized" algo, the results are always NaN
To answer need expressed by TFL on WEST
Created branch feature/magfieldlines for this
When calling _tfg_c._Detect_SigSynthDiag() with PreComp=False and non-standard resolution parameters, the following error was raised:
>> NameError: global name 'ConeWidth_k' is not defined
Indeed, checking _tfg_c._Detect_SigSynthDiag() showed that variable ConeWidth_k had not been defined as one of the input arguments !
The runtime for the current method is longer than a minute or more depending on the file size
Identical to without unit vector, one extra output
The function works fine when unprocessed video is used i.e., the raw data from camera but has problems reading the metadata of videos previously processed my OpenCV
Cython is necessary here for memory / speed performance.
We will intergate toroidally over a potentially large plasma sub-domain (or the whole domain).
It implies:
Important : for each point, only compute solid angle if there is no vignetting due to structral elements!
i.e.: use algorithms for LOS_intersection where appropriate ! (to be discussed)
The output should be a 2D (R,Z) map of toroidally integrated solid angles (one map for each M point on the trajectory, so in fact the output should be a (M,R,Z) array).
The question of where to put the loop on the M points remains to be determined.
Maybe inside (innermost loop) so the parallelization is done on R ?
The KeyHandler class in tofu should be improved to:
Include the possibility of directly providing the shot, user.... (and keep dids, just convert to it in the function)
Make sure diag names match the imas diag names
Create default Name for objects if not provided
A segmentation fault appeared
VideoCapture class reads videos through default 4 channels and any forced tweaking have not worked yet. This results in an additional grayscale conversion step before doing any operation. Although it does not provide any noticeable increased processing time still a fix would render dumpro more efficient.
To provide the background image for field line tracing
A base method shall be included in Struct, then Config just loops over it
Take attribute Poly_closed from Struct subclass instances
No tool (neither distutils nor setuptools) has been used yet to ensure easy installation of the release (to be defined)
In its current form, it is not possible to choose the parameters of Detect.set_SigPreComp() at the creation of the Detect.
Several problems:
When trying to re-create the WEST SXR camera from the real geometry, tf.geom.CamLOS1D() mixes up 1D vs D in _complete_dX12(self, dgeom) (line 2982).
Probably due to the fact that the Ds are not aligned but on a berth
This version can be done with numpy, no need for cython, a priori
Take time to implement the exact computation and the approximation
Support for matlab users
Create / optimize / parallelize an algorithm for computing the distance between N LOS and M closed polygons (with LIm=None, e.g.: isoflux surfaces).
As specified in Milestone "LOS-to-isoflux distance computation", should exist in several versions (distance, bool, closest LOS, closest isoflux)
Tests are not fully passed on travis, due to hard-coded path for tests inputs
They are also too verbose (print of all save / loads)
In some geometrical configurations, the direction in which the vector perpendicular to a Detect of Optics instance (attribute nIn) may not be obvious and the algorithm may point it towards the wrong direction upon creation of the instance.
This is a problem for later computations, which rely on nIn to estimate the volume of sight (e.g.: k*nIn) and lay thus look in the wrong direction.
Profile, optimize and parallelize the best algorithm
Make the structure of the code (inlined basic functions...) allows for creating other version later if necessary
Make available through a single python function, for N LOS and M polygons, return a (N,M) array of booleans
Create a new IMAS sub-package for handling tofu-IMAS generic compatibility
Inspired from the prior tfou_west plugin
Should allow to load and visualize interactively:
Should also allow to compute synthetic diag data from core_profiles and equilibrium
To allow direct issue opening by users
That issue was revealed by an ITER test case implemented by Gabor.
On figure01, page 1 one notices that there is part of the VOS on the opposite side of the Tokamak.
How is this possible ?
When estimating the VOS of a detector+aperture, one of the first steps that ToFu takes is to plot many LOS from the detector through the aperture, to ‘sample’ the VOS.
Now, since we are dealing with a torus, there is always a risk that a LOS will enter the torus twice : once in the poloidal cross-section of interest, and once more in the opposite poloidal cross-section (cf. figure01 page 2 where several examples of LOS are represented)
Of course ToFu can handle this, and it ‘knows’ that each LOS should enter the Torus only once, thus, it disregards the second part in the opposite cross-section.
But detector 1805-00 is in a very special configuration : it is such that some LOS can enter the opposite cross-section without having first entered the relevant one, because the detector is so close to the edge and looking up.
Thus, for these LOS ToFu does not know that they should be disregarded because the usual criterion (i.e. disregard if it is the second time they enter the torus) is not fulfilled.
To solve this problem in the former ToFu version available at IPP, two types of routines were added to General_Geom_cy.pyx (each in 1D and 2D version).
One returns the three planes and 2 points defining the forbidden area.
One check whether a points to this area.
See modified part attached => to be implemented
tofu is not yet available as a pre-compiled conda package for windows 64 bits, so Arpan tried to install it using pip instead.
After creating a virtual environment dedicated to that installation, and running pip (from the conda pip), the following happened (see png attached):
> pip install tofu
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\napra\AppData\Local\Temp\pip-install-6rkhfbb8\tofu\setup.py", line 70, in
not_openmp_installed = check_for_openmp(os.environ['CC'])
File "C:\Users\napra\AppData\Local\Temp\pip-install-6rkhfbb8\tofu\setup.py", line 43, in check_for_openmp
stdout=fnull, stderr=fnull)
File "C:\Users\napra\Anaconda3\envs\tofu140\lib\subprocess.py", line 317, in call
with Popen(*popenargs, **kwargs) as p:
File "C:\Users\napra\Anaconda3\envs\tofu140\lib\subprocess.py", line 769, in init
restore_signals, start_new_session)
File "C:\Users\napra\Anaconda3\envs\tofu140\lib\subprocess.py", line 1172, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\napra\AppData\Local\Temp\pip-install-6rkhfbb8\tofu\
When creating a 1D cam with only one ray, ie:
D = [0,0,0]
u = [0.1, 0.1, 0.1]
cam = tf.geom.CamLOS1D(Exp="Dummy", Name="Dummy", dgeom=(D, u),
config=config, Diag='Test', method="ref", plotdebug=False)
There is an error, and Cam2D is called
I was trying to configure Travis tests with Python version 3.7, after some configurations, I just realized there is a bottom line:
conda.exceptions.UnsatisfiableError: The following specifications were found to be in conflict:
- polygon3 -> python[version='>=3.6,<3.7.0a0']
- python=3.7
So we either try to move forward without the polygon library or we drop the support of python 3.7 (knowing that for the future this still will have to be solved)
versus:
Hi @Didou09,
I'm currently trying to install tofu.
I tried to install using
python setup.py develop
I first had to correct setup.py since line 128 should be commented out.
Then the cython build process failed on the following:
File "/var/folders/r9/s1k2nd110xzfxyfym7jmyb8r0000gn/T/easy_install-v02kmvno/Polygon2-2.0.8/setup.py", line 29
print "Using NumPy extension!"
I'm using Python 3. So this explains the above error.
When I run 2to3 on the code I get this output:
RefactoringTool: Files that need to be modified:
RefactoringTool: tofu/init.py
RefactoringTool: tofu/defaults.py
RefactoringTool: tofu/helper.py
RefactoringTool: tofu/pathfile.py
RefactoringTool: tofu/setup.py
RefactoringTool: tofu/geom/init.py
RefactoringTool: tofu/geom/_compute.py
RefactoringTool: tofu/geom/_core.py
RefactoringTool: tofu/geom/_plot.py
RefactoringTool: tofu/geom/setup.py
RefactoringTool: tofu/plugins/ITER/init.py
RefactoringTool: tofu/treat/init.py
RefactoringTool: tofu/treat/_compute.py
RefactoringTool: tofu/treat/_core.py
RefactoringTool: tofu/treat/_plot.py
I also had to modify some of the dependencies to get them to work with Python 3.
I'll submit a PR with the modifications for Python 3 compatibility.
Are there any tests I can run to see if everything works correctly now that it seems installed?
Best regards,
Florian
Write the fastest possible algorithm for computing the solid angle subtended by an arbitrary (flat, non-complex) polygon as seen from any point, with a side-discrimination (unit perpendicular vector provided).
Discuss necessity to write several versions for:
Possible strategy : decompose into triangles (with barycenter as common summit) and use exact formulation (see existing algo in tofu and this page :
https://en.wikipedia.org/wiki/Solid_angle
)
Inline that algorithm
A saved GDetect instance for which only the LOS and Etendues had been computed did not have a Sino_RefPt (None).
When trying to load it, an error was raised because the saving routine had turned it into:
np.array(None)
That array has shape=(), but it is not None, thus raising an error in the _Detect_checkinputs() routine called when re-creating the Detect instances from the saved file.
ZeroDivisionError raised from tf.geom._GG02.pyx in kInOut calculation with optimized algorithm in Linear case due to double identical points in concatenated Struct polys.
Reproduce with (on WEST):
import tofu as tf
dids = dict(shot=53069)
interf = tf.imas2tofu.load_Diag(diag='interfero', dids=dids, Name='test')
bolo = tf.imas2tofu.load_Diag(diag='bolo', dids=dids, geom=False, Name='test')
ece = tf.imas2tofu.load_Diag(diag='ece', dids=dids, geom=False, Name='Test')
ece.plot_combine([interf, bolo])
_Ves_mesh_dlfromL_cython
_Ves_meshCross_FromD
Improve the travis meta to handle a larger diversity of builds using a build matrix to get more control of:
Apply to conda recipe and pip wheels
Not very urgent
Think about a replacement strategy
indentation error when defining methods for video class.
There is problem with checking whether path exists or not
On certain configurations, it appears that the projections of the viewing cone computed via tofu.geom._compute._Detect_set_ConePoly() are not consistent with a computation of the solid angle of those points taking into account Colis=True by tofu.geom._compute._Detect_SAngVect_Points().
See figure attached for an example on ITER, in the divertor region (the points and thoses stored in Detect._SynthDiag_Points)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.