Comments (7)
@patrickherring-TRI could you send me the exact file the stacktrace is from? I believe the name is PreDiag_000358_000165_structure.json
but I cannot find it on s3. I don't think this was as simple of a problem as I originally thought, since I couldn't reproduce the error with any of the legacy files I had on hand... also, after looking at the code more closely, any legacy files should still actually work with featurization, so I need an actual failing file to debug more
from beep.
I think this is happening even when there are multiple featurizers getting applied, but it could have something to do with the runtime environment
from beep.
I need to see the full traceback (should be in the status json, if it's not I'll need to fix that) to debug it
from beep.
Yes, I am running this on the notebook and it seems like the status.json
is not getting written to the temporary directory where the features are getting written.
This is the input command :
!beep featurize -f IntracellFeatures -d /home/ec2-user/SageMaker/notebooks/TRI/temp --save-intermediates /home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json
And the output:
2021-10-14 12:30:15 INFO Featurizing 1 files
2021-10-14 12:30:15 INFO Applying 1 featurizers to each of 1 files
2021-10-14 12:30:15 DEBUG Hashing file '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json' to MD5
2021-10-14 12:30:19 DEBUG File 1 of 1: Loading processed run '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json'.
2021-10-14 12:30:35 DEBUG File 1 of 1: Loaded processed run '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json' into memory.
2021-10-14 12:30:35 INFO File 1 of 1: Featurizer IntracellFeatures valid with params {'diagnostic_cycle_type': 'rpt_0.2C', 'step_type': 0, 'anode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/anode_test.csv', 'cathode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/cathode_test.csv'} for '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json'
/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/scipy/optimize/_numdiff.py:519: RuntimeWarning: invalid value encountered in true_divide
J_transposed[i] = df / dx
2021-10-14 12:30:48 INFO File 1 of 1: Featurizer IntracellFeatures applied with params {'diagnostic_cycle_type': 'rpt_0.2C', 'step_type': 0, 'anode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/anode_test.csv', 'cathode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/cathode_test.csv'} for '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json'
2021-10-14 12:30:48 INFO File 1 of 1: Featurizer IntracellFeatures features for '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json' written to '/home/ec2-user/SageMaker/notebooks/TRI/temp/IntracellFeatures-PreDiag_000358_000165_structure.json'
2021-10-14 12:30:48 DEBUG Hashing sub-operation output file '/home/ec2-user/SageMaker/notebooks/TRI/temp/IntracellFeatures-PreDiag_000358_000165_structure.json' to MD5
CRITICAL:root:Feature matrix could not be created: 'BEEPFeatureMatrixError'!
2021-10-14 12:30:48 INFO Featurization report:
2021-10-14 12:30:48 INFO All 1 featurizers succeeded: 1/1
2021-10-14 12:30:48 INFO - /home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json
2021-10-14 12:30:48 INFO No featurizers succeeded or file failed: 0/1
2021-10-14 12:30:48 INFO Featurization matrix created: False
from beep.
Is there someplace else that I should look for the status.json?
from beep.
By default no status json is created. It needs to be specified as part of the base command
$ beep --output-status-json /path/to/where/i/want/status.json featurize --foo bar
See https://tri-amdd.github.io/beep/Command%20Line%20Interface/1%20-%20overview/ under "Output streams"
We can also change the default to put a status json in the current working directory with a filename timestamped for uniqueness, but that might clog up things unnecessarily. Seems more straightforward to just specify a status json by whatever filename you want if you do indeed want it.
from beep.
Yeah, sorry, I just mis-understood the usage :D . Running with that argument gives me a status.json, I will share that with you over slack since .json is not supported here.
from beep.
Related Issues (20)
- [Feature Request] Documentation on How to Contribute to BEEP Codebase
- Calculate Diagnostic Features at Every Diagnostic Cycle HOT 2
- [Feature Request] Integrate with amplabs.ai API Read/GET endpoints
- How to load Biologic raw data file in beep ? [Question] HOT 2
- [Feature Request] Novonix Datapath HOT 2
- Add Novonix data requirements to the documentation
- Featurize the Structured Data @ardunn HOT 2
- Create ability for datapaths to produce plots
- Noxonix protocol generation HOT 1
- [Feature Request] Usability of working with data
- Incorporate unifying scheme for featurization
- [Feature Request] Decouple the modelling from the data
- Include `step_type_name` in structured data
- [Bug] Allow for structured summary to account gracefully for steps without cv segment HOT 1
- New Arbin Log HOT 2
- Struggling with featurization HOT 2
- [Question]
- [Bug] pip install crashes HOT 1
- `indeterminate_step_charge_default` needs test
- multiple steps with same step-index are collated together within a single cycle
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from beep.