Comments (13)
Thanks to Felix for the solution. I had not specified the window for optimization correctly. My record length is 1,440,000 but I had left the window at the coded default of 100,000. I did experiment with this previously but I had thought that the window length needed would be defined by the range of scales requested --- and so I had constrained the scales but still couldn't get the optimization wisdom file to be created. Bill
from fcwt.
@moonj94 check the tutorial.ipynb from the cell that says "Optimization plans"
This will generate the file that you are missing (you need to set the threads to 1 to get a file with *_t1.wis)
Hope that helps :)
from fcwt.
Hi @felixdollack for getting back! I must have missed that part of the tutorial, thanks for pointing me to it. I generated the necessary files and created my own class to pass through an optimized fcwt object. However, for some reason it is causing my kernel to crash instantly. My nthreads is 1 at the moment.
Here is the crash error message:
Canceled future for execute_request message before replies were done
The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click [here](https://aka.ms/vscodeJupyterKernelCrash) for more info. View Jupyter [log](command:jupyter.viewOutput) for further details.
I usually get this message when my computer runs out of memory, but I have 64GB RAM. Is there anything else I can do? Any help would be appreciated!
Thanks!
from fcwt.
@moonj94 the error message sounds like you are using the processing in an async way?
Canceled future for execute_request message before replies were done
If not, then maybe there is some issue with the linux implementation.
In that case @fastlib should have a look at this.
from fcwt.
@felixdollack Hmm, I just tried checking if the error persists after turning fast=False, and it does. The problem seems to be when I call self.fcwt_optim.cwt(...)
. Weird thing is that it should be fairly identical to the boilerplate version of fcwt.cwt. I think the issue was how I made the class. When I put the initialization and actual usage of the optimzation object into one function the kernel doesn't crash, but I now get this error: TypeError: in method 'FCWT_create_FFT_optimization_plan', argument 2 of type 'int'
EDIT
I increased the signal length and the optimization plans are being initailized but for some reason not recognized/found. I suppose this leads me to an important question: How does enabling optimization work for fCWT? Is the only way to use optmization to utilize the initailized fcwt_optim object? Or does simply running create_FFT_optimization_plan
create the e.g. n4095t1 files? And if so how can I help the code find these objects?
from fcwt.
The error you mentioned sounds like the max_size argument was not an integer.
From what I understand the max_size argument needs to be larger than 1024 (if it is less or equal to 1024 you should see an error different from the one you saw).
To enable optimization you need to set the use_optimization_plan to True when instantiating the FCWT object.
To create the optimization files you need to run the method create_FFT_optimization_plan with the FCWT object that has optimization enabled. If you execute it on another instance without optimization enabled it will do nothing (no files are created).
Each FCWT instance is supposed to look for the created wisdom files (if optimization was enabled for this specific instance).
Could you post the class you wrote to get another pair of eyes checking what could be the culprit?
from fcwt.
Here is the class:
class MyCWT(object):
def __init__(self,scaling=config.scaling,signal_length=100000):
super().__init__()
morl = fcwt.Morlet(2.0)
# nthreads = 1
# use_normalization=True
#create a new fcwt object
nthreads=config.nthreads
fs,f0,f1,fn=config.fs,config.f0,config.f1,config.fn
# scaling='log'
use_optimization_plan=True
use_normalization=True
if scaling == "lin":
self.scales = Scales(morl,FCWT_LINFREQS,fs,f0,f1,fn)
elif scaling == "log":
self.scales = Scales(morl,FCWT_LOGSCALES,fs,f0,f1,fn)
else:
self.scales = Scales(morl,FCWT_LOGSCALES,fs,f0,f1,fn)
self.fcwt_optim = fcwt.FCWT(morl, nthreads, use_optimization_plan, use_normalization)
# signal_length=100000
if config.fast:
self.fcwt_optim.create_FFT_optimization_plan(signal_length,'FFTW_ESTIMATE')
self.fn=fn
def cwt(self,input):
if input.ndim > 1:
raise ValueError("Input must be a vector")
#check if input is single precision and change to single precision if not
if input.dtype != 'single':
input = input.astype('single')
output = np.zeros((self.fn,input.size), dtype='csingle')
freqs = np.zeros((self.fn), dtype='single')
# input=map(float,input)
# input=input.astype(float)
self.fcwt_optim.cwt(input,self.scales,output)
self.scales.getFrequencies(freqs)
return freqs, output
This class actually causes the kernel to crash for some reason.
So instead I tried to make a function that instantiates a new object everytime it's called. The code didn't crash, but it didn't end up using the optimization for some reason:
def cwtfunc(input, fs=config.fs, f0=config.f0, f1=config.f1, fn=config.fn, nthreads=1, scaling=config.scaling, fast=config.fast, norm=True):
input_sz=list(np.shape(input))
signal_length=np.max(np.array(input_sz))
print(signal_length)
#check if input is array and not matrix
if input.ndim > 1:
raise ValueError("Input must be a vector")
#check if input is single precision and change to single precision if not
if input.dtype != 'single':
input = input.astype('single')
morl = Morlet(2.0) #use Morlet wavelet with a wavelet-parameter of 2.0
#Generate scales
if scaling == "lin":
scales = Scales(morl,FCWT_LINFREQS,fs,f0,f1,fn)
elif scaling == "log":
scales = Scales(morl,FCWT_LOGSCALES,fs,f0,f1,fn)
else:
scales = Scales(morl,FCWT_LOGSCALES,fs,f0,f1,fn)
fcwt = FCWT(morl, nthreads, fast, norm)
if config.fast:
fcwt.create_FFT_optimization_plan(signal_length,"FFTW_MEASURE")
output = np.zeros((fn,input.size), dtype='csingle')
freqs = np.zeros((fn), dtype='single')
fcwt.cwt(input,scales,output)
scales.getFrequencies(freqs)
return freqs, output
from fcwt.
@felixdollack Wanted to follow up on this. Thanks for your recommendations but I believe I am alraedy doing that. I instantiate a optimization=true fcwt object and then I create the fft optimization plan using that object. But I still get the warning that the wisdom file cannot be found. I am not currently using the class above at the moment as I get a separate error.
from fcwt.
@moonj94 I have currently no linux to test this so I am doing it on macOS.
... When I put the initialization and actual usage of the optimzation object into one function the kernel doesn't crash, but I now get this error:
TypeError: in method 'FCWT_create_FFT_optimization_plan', argument 2 of type 'int'
The reason for this is that you are passing a numpy.int64 instead of int.
In your function cwtfunc
you need to wrap you signal_length calculation with int(...)
signal_length=int(np.max(np.array(input_sz)))
Alternatively length(input_sz)
should do the same as the first two lines in your function.
Using the function works for me and it also finds and uses the optimization files when config.fast=True.
However, using the class produces a segmentation fault.
I don't know exactly why yet, but I will check now
EDIT: The crash seems related to the Scales. If I move it into the cwt function of the class it works.
from fcwt.
@moonj94 Both the problem and the solution are simple.
Since you are working with C-objects that you can only access as pointers,
you need to keep a reference to all objects that are needed (also by other objects).
Here the Scales need a reference to the Morlet object. Since you only create it during init and it is already destroyed when you are passing Scales to the cwt function it crashes.
Change your code so it uses self.morlet
and it will run without crash!
from fcwt.
In my case, I believe I have followed the correct prescription to generate the .wis file namely:
...
Threads:8
Calculating optimal scheme for forward FFT with N:65536
Calculating optimal scheme for backward FFT with N:65536
Optimization schemes for N: 65536 have been calculated. Next time you use fCWT it will automatically choose the right optimization scheme based on number of threads and signal length.
Threads:8
Calculating optimal scheme for forward FFT with N:131072
Calculating optimal scheme for backward FFT with N:131072
Optimization schemes for N: 131072 have been calculated. Next time you use fCWT it will automatically choose the right optimization scheme based on number of threads and signal length.
WARNING: Optimization scheme 'n2097152_t8.wis' was not found, fallback to calculation without optimization.
but, as you can see, the .wis file isn't found. What am I missing?
from fcwt.
Hey @bjunor, it seems that your optimization is not covering your use case.
Your log only shows optimization until N: 131072,
but your input seems to be in the order of N: 2097152.
Can you increase the signal length used in the optimization?
# signal_length should be as long as your actual signal to get the correct optimization files
create_FFT_optimization_plan(signal_length,'FFTW_ESTIMATE')
from fcwt.
from fcwt.
Related Issues (20)
- Linking problem: undefined reference to log@GLIBC_2.29 HOT 7
- installing the fCWT on python HOT 1
- Sample frequency limitation
- FR: Benchmarking against IRCAM "wavelet" Library
- libomp ubuntu build issues
- Installation issues on Apple M1
- Support for Custom Frequency Array Input HOT 1
- Wavelet coherence
- Template for `double`
- Dose it can be used for real-time application in processing low frequency signal?
- CMake function find_package() does not find fCWT-config.cmake configuration files HOT 1
- Generate fCWT for a short signal to be used as feature HOT 1
- Other mother wavelets HOT 1
- Continuous update for real-time application
- can't import FCWT and use it Google Colab python HOT 1
- Integration fCWT with Java
- fCWT on ARM Cortex M7 or so?
- memory error
- The CWT result???? c++
- Problem of installing fCWT on linux
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fcwt.