Comments (7)
ardunn
commented on August 19, 2024
1
@patrickherring-TRI could you send me the exact file the stacktrace is from? I believe the name is PreDiag_000358_000165_structure.json but I cannot find it on s3. I don't think this was as simple of a problem as I originally thought, since I couldn't reproduce the error with any of the legacy files I had on hand... also, after looking at the code more closely, any legacy files should still actually work with featurization, so I need an actual failing file to debug more
from beep.
patrickherring-TRI
commented on August 19, 2024
I think this is happening even when there are multiple featurizers getting applied, but it could have something to do with the runtime environment
from beep.
ardunn
commented on August 19, 2024
I need to see the full traceback (should be in the status json, if it's not I'll need to fix that) to debug it
from beep.
patrickherring-TRI
commented on August 19, 2024
Yes, I am running this on the notebook and it seems like the status.json is not getting written to the temporary directory where the features are getting written.
This is the input command :
!beep featurize -f IntracellFeatures -d /home/ec2-user/SageMaker/notebooks/TRI/temp --save-intermediates /home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json
And the output:
2021-10-14 12:30:15 INFO Featurizing 1 files
2021-10-14 12:30:15 INFO Applying 1 featurizers to each of 1 files
2021-10-14 12:30:15 DEBUG Hashing file '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json' to MD5
2021-10-14 12:30:19 DEBUG File 1 of 1: Loading processed run '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json'.
2021-10-14 12:30:35 DEBUG File 1 of 1: Loaded processed run '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json' into memory.
2021-10-14 12:30:35 INFO File 1 of 1: Featurizer IntracellFeatures valid with params {'diagnostic_cycle_type': 'rpt_0.2C', 'step_type': 0, 'anode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/anode_test.csv', 'cathode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/cathode_test.csv'} for '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json'
/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/scipy/optimize/_numdiff.py:519: RuntimeWarning: invalid value encountered in true_divide
J_transposed[i] = df / dx
2021-10-14 12:30:48 INFO File 1 of 1: Featurizer IntracellFeatures applied with params {'diagnostic_cycle_type': 'rpt_0.2C', 'step_type': 0, 'anode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/anode_test.csv', 'cathode_file': '/home/ec2-user/anaconda3/envs/beep-newcli/lib/python3.8/site-packages/beep/protocol_parameters/intracell_info/cathode_test.csv'} for '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json'
2021-10-14 12:30:48 INFO File 1 of 1: Featurizer IntracellFeatures features for '/home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json' written to '/home/ec2-user/SageMaker/notebooks/TRI/temp/IntracellFeatures-PreDiag_000358_000165_structure.json'
2021-10-14 12:30:48 DEBUG Hashing sub-operation output file '/home/ec2-user/SageMaker/notebooks/TRI/temp/IntracellFeatures-PreDiag_000358_000165_structure.json' to MD5
CRITICAL:root:Feature matrix could not be created: 'BEEPFeatureMatrixError'!
2021-10-14 12:30:48 INFO Featurization report:
2021-10-14 12:30:48 INFO All 1 featurizers succeeded: 1/1
2021-10-14 12:30:48 INFO - /home/ec2-user/SageMaker/efs-readonly/structure/PreDiag_000358_000165_structure.json
2021-10-14 12:30:48 INFO No featurizers succeeded or file failed: 0/1
2021-10-14 12:30:48 INFO Featurization matrix created: False
from beep.
patrickherring-TRI
commented on August 19, 2024
Is there someplace else that I should look for the status.json?
from beep.
ardunn
commented on August 19, 2024
By default no status json is created. It needs to be specified as part of the base command
$ beep --output-status-json /path/to/where/i/want/status.json featurize --foo bar
See https://tri-amdd.github.io/beep/Command%20Line%20Interface/1%20-%20overview/ under "Output streams"
We can also change the default to put a status json in the current working directory with a filename timestamped for uniqueness, but that might clog up things unnecessarily. Seems more straightforward to just specify a status json by whatever filename you want if you do indeed want it.
from beep.
patrickherring-TRI
commented on August 19, 2024
Yeah, sorry, I just mis-understood the usage :D . Running with that argument gives me a status.json, I will share that with you over slack since .json is not supported here.
from beep.
Related Issues (20)
- [Feature Request] Documentation on How to Contribute to BEEP Codebase
- Calculate Diagnostic Features at Every Diagnostic Cycle HOT 2
- [Feature Request] Integrate with amplabs.ai API Read/GET endpoints
- How to load Biologic raw data file in beep ? [Question] HOT 2
- [Feature Request] Novonix Datapath HOT 2
- Add Novonix data requirements to the documentation
- Featurize the Structured Data @ardunn HOT 2
- Create ability for datapaths to produce plots
- Noxonix protocol generation HOT 1
- [Feature Request] Usability of working with data
- Incorporate unifying scheme for featurization
- [Feature Request] Decouple the modelling from the data
- Include `step_type_name` in structured data
- [Bug] Allow for structured summary to account gracefully for steps without cv segment HOT 1
- New Arbin Log HOT 2
- Struggling with featurization HOT 2
- [Question]
- [Bug] pip install crashes HOT 1
- `indeterminate_step_charge_default` needs test
- multiple steps with same step-index are collated together within a single cycle
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from beep.