loudnate / openaps-mmhistorytools Goto Github PK
View Code? Open in Web Editor NEWTools for cleaning, condensing, and reformatting openaps history data
License: MIT License
openaps-mmhistorytools's People
Stargazers
Watchers
Forkers
bewest divishinating channemann oskarpearson audiefile jasoncalabrese dougtakeuti thebookins docprojectcatalogue nightscou2018 pettypavlow hdbaproopenaps-mmhistorytools's Issues
Change --zero-at parameter to accept a clock output file
Be a better openaps reporting citizen!
TempBasal entries over midnight are mis-ordered, lack descriptions
[
{
"amount": -0.018000000000000016,
"start_at": "2015-07-10T22:08:06",
"description": "",
"type": "TempBasal",
"unit": "U/hour",
"end_at": "2015-07-10T23:59:59"
},
{
"amount": -0.018000000000000016,
"start_at": "2015-07-11T00:00:00",
"description": "",
"type": "TempBasal",
"unit": "U/hour",
"end_at": "2015-07-11T02:08:06"
},
]
Handle Reservoir + Set (Set Change) as Suspend
Pump was removed and the set change process started, then I left it in the fully rewound state and took a shower. After getting out, completed set change (replaced reservoir, primed), but cleaned history did not reflect this effective "suspend". I believe there may have been a scheduled basal change that occurred in the middle, so the rewind and two prime history entries may have been nonconsecutive.
I don't have the history files to show this as my SD card died before I uploaded them, unfortunately.
Don't use `munge` as a device name in README
'date' epoch is no longer in history records
Traceback (most recent call last):
File "/usr/local/bin/openaps-report", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/root/src/openaps/bin/openaps-report", line 82, in <module>
app( )
File "/root/src/openaps/openaps/cli/__init__.py", line 44, in __call__
self.run(self.args)
File "/root/src/openaps/bin/openaps-report", line 75, in run
output = app(args, self)
File "/root/src/openaps/openaps/cli/subcommand.py", line 50, in __call__
return self.method.main(args, app)
File "/root/src/openaps/openaps/reports/invoke.py", line 40, in main
output = task.method(args, app)
File "/root/src/openaps/openaps/uses/use.py", line 26, in __call__
output = self.main(args, app)
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/__init__.py", line 205, in main
tool = ReconcileHistory(*args)
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/historytools.py", line 170, in __init__
self.add_history_event(event)
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/historytools.py", line 174, in add_history_event
decoded = getattr(self, "_decode_{}".format(event["_type"].lower()))(event)
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/historytools.py", line 217, in _decode_pumpresume
new_event[key] = event[key]
KeyError: 'date'
Openaps reporting support
@bewest you mentioned optional args aren't compatible with report serialization, is that accurate? Should I change all use arguments to '-' in this plugin?
Travis CI
I haven't been able to log into travis-ci.org over the past week, so this is on hold.
Parse Exercise Markers in Pump History
Bridging bewest/decoding-carelink#126 and loudnate/openaps-predict#13
Temp Basals that start at 11:59:59 aren't properly reconciled
Great bug!
[
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-24T02:10:45 head[2], body[0] op[0x16]",
"timestamp": "2016-03-24T02:10:45",
"_body": "",
"_head": "1603",
"duration (min)": 90,
"_date": "2dca021810"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-24T02:10:45 head[2], body[1] op[0x33]",
"timestamp": "2016-03-24T02:10:45",
"_body": "00",
"_head": "3302",
"rate": 0.05,
"_date": "2dca021810"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-24T01:49:40 head[2], body[0] op[0x16]",
"timestamp": "2016-03-24T01:49:40",
"_body": "",
"_head": "1600",
"duration (min)": 0,
"_date": "28f1011810"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-24T01:49:40 head[2], body[1] op[0x33]",
"timestamp": "2016-03-24T01:49:40",
"_body": "00",
"_head": "3300",
"rate": 0.0,
"_date": "28f1011810"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-24T01:37:19 head[2], body[0] op[0x16]",
"timestamp": "2016-03-24T01:37:19",
"_body": "",
"_head": "1603",
"duration (min)": 90,
"_date": "13e5011810"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-24T01:37:19 head[2], body[1] op[0x33]",
"timestamp": "2016-03-24T01:37:19",
"_body": "00",
"_head": "3300",
"rate": 0.0,
"_date": "13e5011810"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:59:59 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:59:59",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "3bfb175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:59:59 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:59:59",
"_body": "00",
"_head": "332c",
"rate": 1.1,
"_date": "3bfb175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:59:57 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:59:57",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "39fb175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:59:57 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:59:57",
"_body": "00",
"_head": "332c",
"rate": 1.1,
"_date": "39fb175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:59:55 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:59:55",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "37fb175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:59:55 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:59:55",
"_body": "00",
"_head": "332c",
"rate": 1.1,
"_date": "37fb175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:54:57 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:54:57",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "39f6175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:54:57 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:54:57",
"_body": "00",
"_head": "3332",
"rate": 1.25,
"_date": "39f6175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:54:55 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:54:55",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "37f6175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:54:55 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:54:55",
"_body": "00",
"_head": "3332",
"rate": 1.25,
"_date": "37f6175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:54:53 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:54:53",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "35f6175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:54:53 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:54:53",
"_body": "00",
"_head": "3332",
"rate": 1.25,
"_date": "35f6175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:44:55 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:44:55",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "37ec175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:44:55 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:44:55",
"_body": "00",
"_head": "337d",
"rate": 3.125,
"_date": "37ec175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:44:53 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:44:53",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "35ec175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:44:53 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:44:53",
"_body": "00",
"_head": "337d",
"rate": 3.125,
"_date": "35ec175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:44:52 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:44:52",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "34ec175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:44:52 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:44:52",
"_body": "00",
"_head": "337d",
"rate": 3.125,
"_date": "34ec175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:39:57 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:39:57",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "39e7175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:39:57 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:39:57",
"_body": "00",
"_head": "3383",
"rate": 3.275,
"_date": "39e7175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:39:55 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:39:55",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "37e7175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:39:55 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:39:55",
"_body": "00",
"_head": "3383",
"rate": 3.275,
"_date": "37e7175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:39:53 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:39:53",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "35e7175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:39:53 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:39:53",
"_body": "00",
"_head": "3383",
"rate": 3.275,
"_date": "35e7175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:34:54 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:34:54",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "36e2175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:34:54 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:34:54",
"_body": "00",
"_head": "335f",
"rate": 2.375,
"_date": "36e2175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:34:51 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:34:51",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "33e2175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:34:51 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:34:51",
"_body": "00",
"_head": "335f",
"rate": 2.375,
"_date": "33e2175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:34:50 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:34:50",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "32e2175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:34:50 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:34:50",
"_body": "00",
"_head": "335f",
"rate": 2.375,
"_date": "32e2175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:29:54 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:29:54",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "36dd175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:29:54 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:29:54",
"_body": "00",
"_head": "3363",
"rate": 2.475,
"_date": "36dd175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:29:51 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:29:51",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "33dd175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:29:51 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:29:51",
"_body": "00",
"_head": "3363",
"rate": 2.475,
"_date": "33dd175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:29:50 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:29:50",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "32dd175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:29:50 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:29:50",
"_body": "00",
"_head": "3363",
"rate": 2.475,
"_date": "32dd175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:24:52 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:24:52",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "34d8175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:24:52 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:24:52",
"_body": "00",
"_head": "3373",
"rate": 2.875,
"_date": "34d8175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:24:50 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:24:50",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "32d8175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:24:50 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:24:50",
"_body": "00",
"_head": "3373",
"rate": 2.875,
"_date": "32d8175710"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-03-23T23:24:48 head[2], body[0] op[0x16]",
"timestamp": "2016-03-23T23:24:48",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "30d8175710"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-03-23T23:24:48 head[2], body[1] op[0x33]",
"timestamp": "2016-03-23T23:24:48",
"_body": "00",
"_head": "3373",
"rate": 2.875,
"_date": "30d8175710"
}
]
Re-naming/re-org work
Per dev convo tonight, requesting a look at the names and organization . Consider "clean" and "prepare"?
historytools.py reconcile crashes on temp to zero then suspend
pi@rpi4 ~/openaps-dev $ cat clean.json
[
{
"timestamp": "2015-09-06T15:09:51",
"_type": "PumpResume"
},
{
"_type": "PumpSuspend",
"_description": "PumpSuspend 2015-09-06T15:09:03 head[2], body[0] op[0x1e]",
"timestamp": "2015-09-06T15:09:03",
"_body": "",
"_head": "1e00",
"_date": "83490f060f"
},
{
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2015-09-06T15:09:02 head[2], body[0] op[0x16]",
"timestamp": "2015-09-06T15:09:02",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "82490f060f"
},
{
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2015-09-06T15:09:02 head[2], body[1] op[0x33]",
"timestamp": "2015-09-06T15:09:02",
"_body": "00",
"_head": "3300",
"rate": 0.0,
"_date": "82490f060f"
}
]pi@rpi4 ~/openaps-dev $ cat clean.json | openaps use munge reconcile
Traceback (most recent call last):
File "/usr/local/bin/openaps-use", line 5, in <module>
pkg_resources.run_script('openaps==0.0.6', 'openaps-use')
File "/usr/local/lib/python2.7/dist-packages/setuptools-1.0-py2.7.egg/pkg_resources.py", line 492, in run_script
File "/usr/local/lib/python2.7/dist-packages/setuptools-1.0-py2.7.egg/pkg_resources.py", line 1350, in run_script
File "/usr/local/lib/python2.7/dist-packages/openaps-0.0.6-py2.7.egg/EGG-INFO/scripts/openaps-use", line 63, in <module>
app( )
File "/usr/local/lib/python2.7/dist-packages/openaps-0.0.6-py2.7.egg/openaps/cli/__init__.py", line 44, in __call__
self.run(self.args)
File "/usr/local/lib/python2.7/dist-packages/openaps-0.0.6-py2.7.egg/EGG-INFO/scripts/openaps-use", line 57, in run
output = app(args, self)
File "/usr/local/lib/python2.7/dist-packages/openaps-0.0.6-py2.7.egg/openaps/uses/__init__.py", line 92, in __call__
return self.method.selected(args)(args, app)
File "/usr/local/lib/python2.7/dist-packages/openaps-0.0.6-py2.7.egg/openaps/uses/__init__.py", line 31, in __call__
return self.method(args, app)
File "/usr/local/lib/python2.7/dist-packages/openaps-0.0.6-py2.7.egg/openaps/uses/use.py", line 26, in __call__
output = self.main(args, app)
File "/usr/local/lib/python2.7/dist-packages/openapscontrib.mmhistorytools-0.0.3-py2.7.egg/openapscontrib/mmhistorytools/__init__.py", line 205, in main
tool = ReconcileHistory(*args)
File "/usr/local/lib/python2.7/dist-packages/openapscontrib.mmhistorytools-0.0.3-py2.7.egg/openapscontrib/mmhistorytools/historytools.py", line 170, in __init__
self.add_history_event(event)
File "/usr/local/lib/python2.7/dist-packages/openapscontrib.mmhistorytools-0.0.3-py2.7.egg/openapscontrib/mmhistorytools/historytools.py", line 174, in add_history_event
decoded = getattr(self, "_decode_{}".format(event["_type"].lower()))(event)
File "/usr/local/lib/python2.7/dist-packages/openapscontrib.mmhistorytools-0.0.3-py2.7.egg/openapscontrib/mmhistorytools/historytools.py", line 217, in _decode_pumpresume
new_event[key] = event[key]
KeyError: 'date'
pi@rpi4 ~/openaps-dev $
KeyError when working with cleaned resume entries
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/__init__.py", line 206, in main
tool = ReconcileHistory(*args)
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/historytools.py", line 170, in __init__
self.add_history_event(event)
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/historytools.py", line 174, in add_history_event
decoded = getattr(self, "_decode_{}".format(event["_type"].lower()))(event)
File "/root/src/openaps-mmhistorytools/openapscontrib/mmhistorytools/historytools.py", line 217, in _decode_pumpresume
new_event[key] = event[key]
KeyError: '_date'
Handle No Delivery flows
It's happened to me before where I've forgotten to act on the Fixed Prime screen and the pump remains "suspended". It would be good if this could be reflected in the reconciled/resolved history.
New Feature: Merging (and deduping) history arrays
Useful for prepending only new entries to an existing history array.
$ openaps use pump read_history_data 0 | openaps use munge group - read_history_data.json
From there, the result can be passed to clean to trim any file excess, but maybe clean should be split into clean and trim, since clean adds Suspend/Resume end-caps.
Split trim operations from `clean` into a `trim` command
The ability to trim by timestamp range is useful in isolation, like just maintaining a current history file.
Better meal descriptions
{
"start_at": "2015-07-10T21:21:09",
"description": "BolusWizard",
"end_at": "2015-07-10T21:21:09",
"amount": 20,
"type": "Meal",
"unit": "g"
}
"BolusWizard: 20g"?
New command: Append
Provide the ability to append a dosing record and re-munge history without having to query an entire page again.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.