Comments (7)
Hi,
For the first point, I will have a look and improve robustness.
Regarding the second point, it seems you try to read a file that is asking too much memory. You can try to use convertAfterRead=False parameter or ask to load only a set of channels you are interested in instead of the full file data.
from mdfreader.
Hey ratal, thanks for the input! I tried convertAfterRead=False which didn't help though. The error occurs before reaching line 1367 of mdf4reader where the boolean is checked:
if convertAfterRead:
self._convertAllChannel4()
is never reached.
I would like to try to load only a set of channels, which can be done via argument channelList = ['channel', 'list']. However, I don't know the data and am not sure what to put as "channel" or "list"
EDIT: I tried mdfreader.mdfinfo("./file.MF4").listChannels("./file.MF4") to get the channel names. However, I once again received a KeyError at mdfinfo4 line 990:
for cn in range(nChannel):
if not toChangeIndex[cn]:
# offset all indexes of indexes to be moved
self['CNBlock'][dg][cg][cn + nChannel] = self['CNBlock'][dg][cg].pop(cn)
self['CCBlock'][dg][cg][cn + nChannel] = self['CCBlock'][dg][cg].pop(cn)
File "/Programme/anaconda/lib/python2.7/site-packages/mdfreader/mdfinfo4.py", line 990, in readCGBlock
self['CCBlock'][dg][cg][cn + nChannel] = self['CCBlock'][dg][cg].pop(cn)
KeyError: 1
from mdfreader.
Indeed documentation is maybe not so clear about channel listing. Below is the correct procedure:
from mdfreader.mdfinfo4 import info4
yop=info4()
yop.listChannels4('yourfile.mf4')
Looking a bit more deeply, there is possibility your file uses nested channel conversions. It was not so much tested so far (no file to experiment) and not really implemented.
It could then create a wrong representation of stored data and ask for too much data to be loaded ending with memory issues.
If you still have problem, best way would be to send an example of this file or at least a representation of its structure, like a info4 dict.
from mdfreader.
I still receive the same error as mentioned above, even if I use the approach you just described. I don't know whether the file is nested but it's possible. Sadly, I can't send you the data file as it is confidential from a project I work with several partners. A info4 dict might be possible though, if you tell me how I create it.
from mdfreader.
After thinking, it might be complicated for you and me to use this info4 dict. Instead, could you use mdfvalidator from vector (free use) and export as an xml the file structure ? It will help me to understand it.
from mdfreader.
2 feedback since 2 weeks, closed
from mdfreader.
I have the same issue but with a relatively small file 2,46 Mb. I would think it is a nested data format problem.
self['cc_ref'][i] = CCBlock(fid, self['cc_ref'][i])['name']['Comment']
does not exist.
self['cc_ref'][i] = CCBlock(fid, self['cc_ref'][i])['unit']['Comment'] does but I do not know if I should make the change.
from mdfreader.
Related Issues (20)
- Can we specify interpolation method while resampling? HOT 2
- Not able to extract data from different devices with same channel name HOT 1
- GigE Vision (GVSP) does not read last byte of Ethernet payload HOT 1
- [Improvement] Create Python package from dataRead extension. HOT 2
- CSV export without empty row between data? HOT 4
- Index problem HOT 1
- AttributeError: 'list' object has no attribute 'dtype' HOT 2
- MemoryError reading 4,11GB MF4 file
- export to csv adding an extra line HOT 2
- readmdf read network file HOT 2
- export_to_csv(): Is it possible to remove the 100hz resample? HOT 15
- [Improvement] - unzip to RAM instead of disk HOT 2
- UserWarning: Could not parse CG block names tag HOT 1
- Data conversion inconsistent HOT 5
- Incorrect data conversion HOT 2
- where is the sample.mdf file? HOT 1
- Can i read .mat file using mdfreader? HOT 1
- check for right endianness HOT 3
- Improved concatination in mdf.return_pandas_dataframe() HOT 2
- UnboundLocalError: local variable 'record_size' referenced before assignment HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mdfreader.