Comments (6)
How are you planning to do this? It seems to me that matlab positional-only argument specification (that is, the lack of support for named parameters) makes this change (and in general any other parameter addition/removal) challenging.
Options that come to my mind:
- Make seed the first parameter of the function after the time series. I think this is most sensible as seed is a required parameter. It also would make adapting the python bindings very easy.
- Make seed the last parameter of the function. I think this is not really an option.
- Start using inputParsers (but please, not varargins ;-))
- Change all HCTSA functions signature to accept two parameters: the time series and a struct of parameters. If no parameters are passed, a struct of default parameters can be returned. That, I think, would make easier to maintain the library.
from hctsa.
Thanks for the ideas, Santi. I'm now preparing to do this next, as a priority. Yes, this is where Matlab functions are a bit restrictive -- would be great to have named parameters in this case. Anyway, this is where we are -- in Matlab land for now.
For the purposes of how the engine works, I don't see a problem with just resetting the random seed at the beginning of every function call. This could be done: (i) in TSQ_brawn (i.e., not requiring any changes to any functions at all), (ii) inside all stochastic functions. As long as the seed is reset the same way each time, this would ensure reproducibility.
Regarding your options for (ii):
- This works if we reckon that seed is a required parameter, but I see most users not wanting to set this, but for it to default to a particular seed.
- The last parameter could workย (if we view it as optional), but yeah, this could get messy if inputs to functions grow/change with time (although this will probably be minimal -- ideal is for function files not to change too much, at least in their structure).
3./4. Yeah, could use inputParsers (thanks for the link -- have never used this previously). Could also use a standardized first input, which could be a structure, containing the raw time series, the z-scored time series (potentially a whole string of processed versions of the time series that could then be accessed by all functions), a random seed, etc. and other potential generic data inputs. But this would limit the generality of these functions for people wanting to use them outside this context (making their format quite opaque for quick and ready use). Could get around this by having a general function that parses this input data structure, but also deals with the case that a time-series vector is provided (as the current first input). This could be a good investment going forward, when we might want to start using more details about the time-series object (e.g., unevenly sampled data, or providing a custom sampling rate that could be used in some functions, etc.)
I think for now I'm favouring using (i), if it's not too slow just for a quick-to-implement solution (or 1./2.). Looking to a future in which the first input is a structure (that can also deal with vectors if just a vector is provided).
Of course, external C++/fortran mex functions that use randomization will not be reproducible, unless I go into their sources and impose seed resetting...
from hctsa.
This is underway in the DeterministicStochasticity branch (sub-branch of OperationChanges). Will happen for now through a uniform BF_ResetSeed function, that all stochastic operations will reference for inputs specifying a seed reset (the default will be the Matlab default: reset the seed to Mersenne Twister with seed 0). Am yet to specify defaults or incorporate input arguments controlling this, but hopefully I'll find time to get to this by the end of the week.
Like I said in the longer term, looking to implement a consistent 1st-input argument data structure that could specify something like this (also sampling rate, or a time vector for unevenly sampled data, for some custom additional operations for specific domain applications, etc.), but for now this is a workable solution that doesn't require large-scale changes to the current architecture.
from hctsa.
If I understand correctly, touching TSQ_Brawn would not help library users. So not useful to me ;-). But that is indeed the approach I would initially take in pyopy: reset the global rng state to the specified seed (in python land) before running the computation. That is ugly in two ways: 1) touching global state is ugly per se 2) having seed-unaware random functions makes a tad more difficult to batch operation calls deterministically (e.g. one would need to interleave calls to set the seed between calls to the operator) and with clear provenance (as the seed is always part of the signature of the operation, even if implicit).
So for me, the only way to get this correct is to get seed-aware functions (ii).
Making the seed the last parameter of the functions is definitely less work, so probably the best way to go.
My thinking is: seed at the end means enforcing users to specify all other parameters if they want to play with stochasticity (with what that means in functions were the defaults are something more elaborated than a number or a string). This is always true for any nth parameter when using vanilla matlab function dispatch. So the question would be: what will a user want to change from the default parameters more often? That depends on the scenario, so there is no good answer. I often will want to change the seed when I find an interesting (aka discriminative) feature that is stochastic, instead of touching any of the other parameters. Check if the appeal of the feature is just because Mars was aligned with Pluto or if it really encodes something useful.
What I would definitely try, regardless of where the seed parameter goes is:
- making absence or negative seed to mean "use clock-based seed"
- using a local random number generator initialised with whatever seed whenever possible - that is, for these operators in which you control the random sampling bit.
from hctsa.
My tests tell me that these operators need to be tagged as stochastic:
- DN_SimpleFit
- MF_FitSubsegments
- MF_arfit
- MF_CompareAR
- MF_hmm_fit
- MF_hmm_CompareNStates
- MF_GARCHcompare
- MF_StateSpace_n4sid
- NW_VisibilityGraph
- PP_Compare
- PP_Iterate
- SB_TransitionpAlphabet
- SP_Summaries
There can be false positives, I might have bugs and other sources of stochasticity might be in place. Also I might miss some more because of the simplicity of the series I'm using to test. I still have some operators failing to compute too.
I will give a second look, hopefully, at the end of the week, but of course, nothing can replace careful thinking.
from hctsa.
The only remaining (labeled) stochastic operations are the fractaldimensions operations (which rely on mexed code from TSTOOL). I don't know how to control random seeds in C++ (e.g., in gendimest
)
However, everything using random numbers in Matlab now has a controlled random seed (using the function BF_ResetSeed), which is specified as input to most functions, and defaults to 'default', which resets the seed as rng(0,'twister');
from hctsa.
Related Issues (20)
- Warning: No available confusion matrix plotting functions HOT 1
- TS_GetIDs returns brace indexing error HOT 5
- Wrong_TS_Classify HOT 3
- will it work for multivariate time series HOT 1
- Error when running TS_LocalClearRemove HOT 1
- Multichannel EEG analysis HOT 1
- Consider a cloud CI service HOT 1
- nn_prepare.m as a function is not supported and a whole host of other errors HOT 8
- Not able to skip User input "y" in TS_Init HOT 3
- Init only with notLengthDependent features HOT 1
- calling hctsa from python with matlab engine HOT 2
- TS_Classify produces an "Unrecognized function or variable 'foldLosses'" error when trying to save the classifier HOT 3
- Typo in TS_Classify and errors running 'simpleNull' = false in ComputeCVAccuracies? HOT 1
- compile_mex error from range_search.cpp file HOT 1
- Error using compile_mex HOT 3
- Multichannel EEG signal analysis HOT 1
- TS_Cluster produces an 'Unknown distance metric: 'corr_fast''error HOT 2
- parallel operation HOT 2
- Duplicate Function Names
- TS_InitiateParallel passes incorrect information to BF_CheckToolBox HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hctsa.