openaps / oref0 Goto Github PK

Now that AMA carb absorption calculations are being done using the same data used to detect sensitivity, it should be fairly straightforward to improve the autosens algorithm to directly exclude deviations caused by carb absorption (when carb info is avilable), thereby allowing us to use a much narrower percentile range and detect excess sensitivity or resistance with much more precision.

It seems completely random when this happens, but I get it more frequently when rebooting or after turning my pi on after I shut it down, properly or improperly. It also happens randomly throughout the day. This is what it looks like after it goes dead, and then calling oref0-reset-usb:

pi@raspberrypi:~/myopenaps $ sudo oref0-reset-usb
Power-cycling USB to fix dead stick
pi@raspberrypi:~/myopenaps $ lsusb
Bus 001 Device 022: ID 1d50:8001 OpenMoko, Inc.
Bus 001 Device 021: ID 22a3:0047
Bus 001 Device 020: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 019: ID 0424:9514 Standard Microsystems Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

pi@raspberrypi:~/myopenaps $ ls -la /dev/mmeowlink
lrwxrwxrwx 1 root root 7 May  5 10:39 /dev/mmeowlink -> ttyACM1

As you can see it, it finds it just find! It's worth mentioning also, that I can physically see it doing the reset, as the dexcom says it's not charging for a few seconds when it happens. I can also call openaps use pump mmtune, which seems contrary to what you would think I'd be able to do. Next time it goes dead, I'll try more and add more information.

When deviation is <= 0 after a meal, we should still assume carbs are absorbing (slowly), and that BG is failing to rise for some other reason. This will prevent COB from staying high indefinitely until the carbs fall off the back of the pumphistory. For starters, perhaps use a minimum carb impact of 3 mg/dL per 5m. This would only apply for historical carb absorption, not for calculating current and future carb impact.

Currently we return early from determine basal and let the high temp run to completion.

Per @jasoncalabrese: With regular target of 115, autosens raises the target too high when detecting lots of sensitivity. Need to limit autosens target adjustments to 1.5x instead of 2x.

Would be nice to have oref0 automatically detect when the PWD is more or less sensitive to insulin than normal. Could have it periodically pull 24h of pump and glucose history, calculate BGI for each 15m period and compare it to avgDelta for that period to get each 15m deviation. Take the ~30th percentile deviation (median of the non-meal-absorbing hours of the day), *4 to get hourly, and divide by sens to get basal offset required to neutralize sensitivity/resistance. Apply that to scheduled basal when calculating IOB.

Hi

Note: this bug could possibly lead to carb values being "doubled-up" in the meal-assist algorithm, leading to problems.*

There are apparently situations where openaps-mmhistorytools finds and removes duplicate carb values. (See below for details.)

Currently, if you supply the dev branch with a history that's been cleaned with openaps-mmhistorytools, the meal code doesn't correctly interpret carb values.

The mmhistorytools code ejects a value like this:

  {
    "amount": X, 
    "start_at": "2016-02-08T21:54:19", 
    "description": "BolusWizard: Xg", 
    "type": "Meal", 
    "unit": "g", 
    "end_at": "2016-02-08T21:54:19"
  }, 

More examples are at https://github.com/loudnate/openaps-mmhistorytools/blob/master/tests/historytools_tests.py#L970-L979 and https://github.com/loudnate/openaps-mmhistorytools/blob/master/tests/historytools_tests.py#L165 shows a case with bolus wizard duplicates.

https://github.com/openaps/oref0/blob/dev/lib/meal/history.js#L20-L34 is currently where the parsing happens in oref0.

This would need to change the oref parsing code to:

1. Expect a 'munged' data file (ie, by removing the existing code that contains 'Bolus' or 'BolusWizard', so as to break compatibility with files that could contain duplicate information
2. See if there are any differences in the 'munged' data for Bolus vs BolusWizard values (one expects a bolus value, one expects a carb value)
3. Find all entries of type "Meal"
4. Use the 'start_at' value as a replacement for timestamp
5. Use 'amount' as the 'carbs' value
6. Check the unit is always 'g' (in case some strange measurement is presented down the line in a different region

CC @loudnate

This first treatment entry is NOT picked up by the OpenAPS minutes ago pill, but the second entry below it is... Ben pointed out that non-working entry is missing " "duration": 30," at Line 5
{
"_id": {
"$oid": "569d4d4c2bb7e86eaf13996f"
},
"raw_duration": {
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-01-18T14:36:40 head[2], body[0] op[0x16]",
"timestamp": "2016-01-18T14:36:40-06:00",
"_body": "",
"_head": "1600",
"duration (min)": 0,
"_date": "28640e5210"
},
"timestamp": "2016-01-18T14:36:40-06:00",
"absolute": 0,
"rate": 0,
"raw_rate": {
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-01-18T14:36:40 head[2], body[1] op[0x33]",
"timestamp": "2016-01-18T14:36:40-06:00",
"_body": "00",
"_head": "3300",
"rate": 0,
"_date": "28640e5210"
},
"eventType": "Temp Basal",
"medtronic": "mm://openaps/mm-format-ns-treatments/Temp Basal",
"created_at": "2016-01-18T14:36:40-06:00",
"enteredBy": "openaps://medtronic/722"
}
Working entry:
{
"_id": {
"$oid": "569d1e562bb7e86eaf139930"
},
"duration": 30,
"raw_duration": {
"_type": "TempBasalDuration",
"_description": "TempBasalDuration 2016-01-18T11:16:55 head[2], body[0] op[0x16]",
"timestamp": "2016-01-18T11:16:55-06:00",
"_body": "",
"_head": "1601",
"duration (min)": 30,
"_date": "37500b5210"
},
"timestamp": "2016-01-18T11:16:55-06:00",
"absolute": 4.65,
"rate": 4.65,
"raw_rate": {
"_type": "TempBasal",
"temp": "absolute",
"_description": "TempBasal 2016-01-18T11:16:55 head[2], body[1] op[0x33]",
"timestamp": "2016-01-18T11:16:55-06:00",
"_body": "00",
"_head": "33ba",
"rate": 4.65,
"_date": "37500b5210"
},
"eventType": "Temp Basal",
"medtronic": "mm://openaps/mm-format-ns-treatments/Temp Basal",
"created_at": "2016-01-18T11:16:55-06:00",
"enteredBy": "openaps://medtronic/722"
}

There should be a level that cuts of the basal no matter what below a certain level. There has been discussion to set this somewhere between 55 and threshold

Need to make it possible to switch from dev to AMA branch without it breaking existing loops.

@johnmales noticed that he has "on several occasions run into the problem of the medtronic CGM data being stale - sometimes hours old - and oref0 going ahead and setting temp basals. Usually this circumstance happens with that decocare page loading bug and the downloaded cgm data getting “stuck” redownloading old data repeatedly." It looks like he's seeing the issue @ktomy is trying to fix in #54, namely that https://github.com/openaps/oref0/blob/master/bin/oref0-determine-basal.js#L39-L54 does not return early after https://github.com/openaps/oref0/blob/master/bin/oref0-determine-basal.js#L47 ("Could not determine last BG time"). That is a safety-critical bug that we need to get fixed ASAP and merged to both dev and master. If we can do so by merging #54 that would be great; otherwise adding a return line might be sufficient.

@iainct reports at openaps/openaps#88 (comment) that oref0 erroneously calculated a negative IOB based on a basal_profile.json that increases from 0.2 U/hr to 1.4 U/hr over the course of 60 minutes:

{ "i": 17, "start": "17:00:00", "rate": 0.2, "minutes": 1020 },
{ "i": 18, "start": "18:00:00", "rate": 0.8, "minutes": 1080 },
{ "i": 19, "start": "19:00:00", "rate": 1.4, "minutes": 1140 }

As oref0 currently calculates IOB based on the assumption that the currently scheduled basal profile is "close enough" to use for the last DIA hours' worth of pumphistory, this kind of basal profile is dangerous to use with oref0.

For now, we should detect when ( max_daily_basal / min_daily_basal ) > 3, and error out in that case. In the long term, we should probably make oref0-calculate-iob smarter about using the basal scheduled at the time of each entry in pumphistory to calculate IOB as well. I suspect we might want to keep the max/min > 3 check in place even after doing that, though.

Currently oref0 requires that the Pi, pump, and CGM be set to the same timezone, and errors out if (minAgo > 10 || minAgo < -5). This causes problems when traveling and changing timezones, and every spring and fall when the Pi's clock automatically changes for DST. I pushed a workaround (89b609c) that will help temporarily for tonight's DST change, but at the expense of continuing to operate on stale data for up to an hour (or up to 2 hours if timezones are off to begin with). We need to figure out a better solution for dealing with timezones.

Given the frequency of communication errors, there are many times when a full loop cannot be completed but there is still some communication happening between openaps and the pump. There are times therefore that it would be nice to have a simple command that can issue a temp basal to 0.

The scenario may be that the PWD is at school and the parent notices that bg is falling quickly. The parent sees either via a Nightscout integration or other monitoring means that the loop isn't being completed. Therefore, the parent can log into the openaps system remotely and issue a command to set a temp basal to 0. This command may be able to get through to the pump even though the loop may fail at a more intensive communication step like pulling the pump-history

As of today, when the settings/auto-sens.json report encounters an entry in monitor/glucose.json that doesn't have a "sgv" entry in it, it dies. Creating a new report that lives alongside monitor/glucose.json and pointing settings/auto-sens.json at that seems to resolve the issue.

When eating a large meal and only bolusing for part of it, meal assist does a good job of giving more insulin as BG is initially rising, but once BG flattens out in range, it will let IOB and insulin activity drop almost to zero (if no manual boluses are performed), and only belatedly deliver the rest of the meal insulin when BG starts to rise quickly again.

It would be better to continue delivering a slightly elevated basal rate after a meal, as long as deviation is high and snoozeBG is still above target, to simulate an extended bolus. It could set the basal high enough to cover the carbs that haven't yet been bolused for over the course of DIA. So for example, if you bolus 6U and get 1U of meal assist insulin during the initial rise from a 100g meal (with an IC of 10 and a DIA of 3), it would want to deliver 3U of additional insulin over 3h, so it would set the basal to 1U/hr higher than normal if BG is flat. As with "high and flat" situations, we could reduce that insuilinReq proportionally as avgDelta drops from 0 down to expectedDelta. We might also want to increase it proportionally up to full mealAssist levels as avgDelta increases from 0 up to the full mealAssist threshold.

I'm not yet happy with Meal Assist: I think we need to switch to using an advanced meal assist algorithm that is aware of COB / carb absorption and can tell when to high-temp aggressively for carbs being absorbed, and when insulin activity is already high enough to risk going low, and we need to preemptively low-temp instead.

To accomplish this, we need to determine how many meal carbs have been absorbed, at any given point after a meal, so we know how much more of a rise we can eventually expect from the remaining meal carbs, and whether we can expect any dips as insulin activity picks up. This needs to be determined based on the actual observed effect on BG of carb absorption, not based on any assumptions about how fast carbs "should" absorb for any given meal. By doing so, we should be able to safely high-temp for ~half the remaining unabsorbed carbs, as long as carbs are currently absorbing fast enough to outpace insulin activity. If carb absorption is too slow for insulin activity and a low is projected to occur before all carbs absorb, we can instead low-temp until BG and/or deviation picks up sufficiently to make it safe to high-temp again.

To calculate carbs absorbed to date, we can sum up the 5m deviations, for each 5m period since the meal carbs were entered, to determine how much effect carb absorption has actually had on BG so far, and by using the IC ratio and ISF, determine how many carbs have been absorbed, and conversely how many g of COB are remaining.

We can then extrapolate that the current deviation (the current impact of carb absorption) will continue into the future until most of those carbs have absorbed. To simplify the calculations, I'll probably extrapolate the current rate until those projected deviations have accounted for half the remaining COB. (In reality, deviations tend to gradually return to zero as carb absorption slows down, but the simpler math should be sufficient for estimating the total area under that curve.) We can then use the projected BGI and deviations for the next few hours to determine the projected minimum and eventual BG. If projected minimum BG is below targeted min_bg (and below current BG), we can low-temp as needed to help bring that up. If projected minimum BG is above target (or above current BG), we can high-temp as required to bring eventual BG down to target.

Per discussion with @cjo20 and @eyim, we should use a more intelligent formula at
https://github.com/openaps/oref0/blob/master/bin/oref0-determine-basal.js#L166
to ensure that threshold drops more slowly than min_bg, and is never less than about 70mg/dL.
Something like https://files.gitter.im/nightscout/intend-to-bolus/gqge/blob maybe.

Note that https://github.com/openaps/oref0/blob/master/lib/profile/targets.js#L29 already ensures that min_bg can't be lower than 90.

In order to recover from a low, many times OpenAPS will turn off his basal for a long time - 1 to 3 hours. If during the night, this will result in a net negative iob - sometimes a net negative 2+ units before his blood sugars start to rise.

The issue is just as he is getting back to a normal range, OpenAPS will give him insulin to offset the net negative iob which results in pushing him back down to another low. And the cycle continues.

In our case, the preferred behavior would be to ignore the net negative iob. This often happens after he has a lot of exercise that day or studies a lot that evening. The body seems to want to recover extra sugar leading to the low and requiring a significant net negative iob to recover.

@loudnate's Loop app for iOS does an excellent job of calculating IOB and looping using only the pump reservoir information (refreshed every 5 minutes), without ever having to pull a full pumphistory. While there are lots of things (like NS uploads) we still want pumphistory for, it would be good to be able to use reservoir data as an alternative when we're having trouble refreshing pumphistory (or want to do a quick enact without waiting to refresh it). This would probably require building some sort of reservoirhistory.json file and using whatever portions of it are more recent than the most recent pumphistory.json

glucose.json.txt
Attached is my glucose.json file. This file resulted in openaps running a temp basal even well after the 15 old threshold. I've had old data from hours ago that openaps still thinks is current data.

Obviously, this is a major concern as if his last bg was high or low it could cause a serious problem

Do we need some sort of limit on how long oref0 will temp to zero, to avoid bad rebounds and potential ketones?

Submitted at the request of bewest. mm-format-* tools don't seem to have Dexcom counterparts.

Received this message when running loop:
get-profile://text/shell/settings/profile.json Could not parse carbratio_data. Feature Meal Assist enabled but cannot find required carb_ratios. I don't have a meal.json report, since I haven't done any Meal Assist setup yet.

Removing remainder = [] from my settings/profile.json and enact/suggested.json report configurations fixed this. Perhaps it was feeding [] to oref0 when it was expecting a meal.json?

For reference:

Profile.json report output of: {"error":{"errno":34,"code":"ENOENT","path":"[]","syscall":"open"},"msg":"Could not parse carbratio_data. Feature Meal Assist enabled but cannot find required carb_ratios.","file":"[]"}

Profile.json report configuration of: [report "settings/profile.json"] use = shell bg_targets = settings/bg_targets.json settings = settings/settings.json basal_profile = settings/basal_profile.json reporter = text json_default = True max_iob = max_iob.json device = get-profile remainder = [] insulin_sensitivities = settings/insulin_sensitivities.json

gather-profile alias of: gather-profile = report invoke settings/settings.json settings/bg_targets.json settings/insulin_sensitivities.json settings/basal_profile.json settings/profile.json

Related to a recent discussion on gitter, one way to minimize my editing the openaps.ini file would be to provide a way to tweak current entries. For example, currently I can:
openaps alias show myalias
but if I want to tweak it I need to type (or copy paste) the entire string in again with
openaps alias add myalias "long string here"
Perhaps the addition of an openaps edit myalias where the string is echo-ed (with 'add'?) on to the command line and the use simply can make a few tweaks, then hit enter? Can this be universal for reports as well?

After upgrading to the advanced meal assist branch, I've noticed that the nightscout device no longer functions as intended. I tried to re-set it up using autoconfigure-device-crud but that didn't work. Checking nightscout -h now only yields

  Usage:
nightscout <cmd>

rather than what I was seeing before.

Don't blindly calculate from first or biggest carb treatment: properly figure out when COB drops to zero and start over after that.

All times and time zones were set correctly but pump history wasn't properly zoned so treatments were displayed in Nightscout as off by one hour (one hour into the future).

In /lib/meal/history.js, the following loop causes COB to disappear because the BolusWizard and the Bolus entry are in the pumphistory.js at exactly the same time, and it kills the BolusWizard entry, assuming it's a duplicate, but that's where the COB comes from:

        for (var j=0; j < mealInputs.length; j++) {
          var element = mealInputs[j];
          if (element.timestamp == current.timestamp) dupeFound = true;
      }

Commenting out this loop fixed the issue for me, but that's just a hack.

This is the meal.json setup:

openaps report add monitor/meal.json text meal shell monitor/pumphistory-zoned.json settings/profile.json monitor/clock-zoned.json monitor/glucoseclean.json settings/basal_profile.json

Not using a separate carb history file, because on Medtronic, it's in the pumphistory-zoned.json.

For example, here a low temp basal was ended because Avg Delta > Expected Delta, even though Eventual BG was much lower than min set.

I currently see openaps upload it in mg/dL format.
But my nightscout is in mmol/L mode and interprets the bloodvalues wrong.

Example upload:
"amount": 140,
"eventType": "",
"glucose": 140,
"glucoseType": "Finger",
"notes": "Pump received finger stick.",
"medtronic": "mm://openaps/mm-format-ns-treatments/CalBGForPH",
"created_at": "2016-01-30T21:36:06+01:00",
"enteredBy": "openaps://medtronic/754"

According to @bewest on gitter:
that's a bug and should be logged/tracked
yup that's a gap in the current tools
issue here is reverse though
the pump-history, the bg stuff seems to always be in mg/dL

I din't get adding ?units=mg/dL to your NS endpoint working
it could be argued that a patch to mm-format-ns-treatments should handle this maybe

After callibration delta should be calculated in a different way or we should stop the loop for a time to understand the trend after callibration.

Okay another example of where openaps not doing what I would think it would do. The situation is where he is low and stays low. It looks like openaps simply tries to cancel a high temp if there is one but doesn't try to implement a low temp basal. Basically when Dexcom shows a low, it just reads 39 the entire time. Therefore tick and avg delta = 0 if you stay low for a while and you get this:
{"delta":0,"glucose":39,"avgdelta":0}
{"duration":33,"rate":0,"temp":"absolute"}
{"iob":1.2243434921930496,"activity":0.06642615555555557,"bolusiob":0}
{"max_iob":6,"type":"current","dia":5,"current_basal":0.8,"max_daily_basal":1.1,"max_basal":3.5,"min_bg":120,"max_bg":120,"sens":80}
IOB: 1.22, Bolus IOB: 0.00
Avg. Delta: 0.0, BGI: -26.6
15m deviation: 80
BG: 39+0 -> 21-21 (Unadjusted: -59--59)
BG 39<110, avg delta 0>0; no high-temp to cancel
If he stays in this situation for more than the 30 minutes then any temp basal implemented as he is dropping runs out and no new low temp basal in enacted which I think it should. Several ways to address this. The first to deal with this exact scenario of it reading 39 and staying 39 but I think openaps should have some feature that says if below some cutoff it recommends a temp basal of 0 regardless of any projections or other calculations. This would be safest. The temp basal in the calculation above was one we put on manually.
Here is the chart. The scenario is that he goes to bed at around 125 but with a lot of insulin. My wife gives him 12 carbs of sugar and turns off his basal for 2 hours. The openaps system was sitting at this desk where he was studying so when he went to bed it was out of range but presumably it would low temp him like my wife as done. He begins to drop a little while later and we begin to feed him a ton of sugar at that point but regardless he goes low on the Dexcom and stays there for more than an hour. I move the openaps into range around 12:30 am once he has already hit low and it gives the recommendation above. I even temporarily turn off the low basal that we manually put on to see if that would make a difference but it gives the same recommendation - no high-temp to cancel rather than recommending a low temp basal.
Luckily by blood his bg had already shot up but I think this is the case if someone goes low and stays low, openaps should be recommending a 0 temp basal the entire time no matter what the other metrics are saying

scottleibrand 01:53
Yeah, you're probably right. FWIW, the situation you're seeing is that the BGI says he "should" be dropping by 26 (!) mg/dL every 5 minutes (no wonder he hit 39!) but it sees him dropping far less than that, so it figures something is keeping his BG up and doesn't low-temp. That logic works pretty well at 70, but not so much at 50, and definitely not at 39, where the +0 is a complete artifact.
What would you say the "extend 0 temp no matter what" threshold should be?

eyim 01:58
For us, it would be 70. If he is below 70 we leave off his basal. It can cause him to shoot higher than we like on the rebound but it seems safer to me. Others without the issues we have to deal with may be more comfortable with something lower say 50 or 60

scottleibrand 01:58
And thanks for reporting this. This is the kind of thing we really want to get fixed ASAP to be as safe as possible when it matters most.
I'm inclined to go with 55, since it's Dexcom's hypo alarm threshold.

eyim 01:58
That makes sense to me

scottleibrand 02:00
Fortunately this 39+0 thing does not apply above 39: you have to actually have carbs working in order to keep from dropping, and those are likely to finish absorbing before any temp could ever take effect.
So if you're 60+0 and BGI is -20, we know the carbs are working.
And the minute we see they're not and he down-ticks, we will low-temp again.
another simple fix would be to change the < to <= to just handle the +0 case
but I think I like the 55 threshold better.

eyim 02:03
Yes definitely an outlier problem with the 39 artifact. I agree <= would handle this as well

scottleibrand 02:03
Or more likely both.

A probably throw away branch with a test:
dev...jasoncalabrese:wip/mystery-iob

Test with what I think is the exact pump history doesn't produce the same result, but somehow the bolus snooze of 0.005226074186666674 matches exactly.

IOB 0.04
2015-12-16T15:02:44-08:00
Tree: https://github.com/jasoncalabrese/indy/tree/f04d3b041086e7e0f45492e0320b9d788676fb84
IOB Result: https://raw.githubusercontent.com/jasoncalabrese/indy/f04d3b041086e7e0f45492e0320b9d788676fb84/predict/iob.json
Pump History: https://raw.githubusercontent.com/jasoncalabrese/indy/f04d3b041086e7e0f45492e0320b9d788676fb84/monitor/pump-history-zoned.json

IOB 0.80
2015-12-16T15:07:43-08:00
Tree: https://github.com/jasoncalabrese/indy/tree/daa04a8417d3875fd9cb18be90227e99bcf56f63
IOB Result: https://raw.githubusercontent.com/jasoncalabrese/indy/daa04a8417d3875fd9cb18be90227e99bcf56f63/predict/iob.json
Pump History: https://raw.githubusercontent.com/jasoncalabrese/indy/daa04a8417d3875fd9cb18be90227e99bcf56f63/monitor/pump-history-zoned.json

adding git diff > /dev/null || ( mv .git /tmp/.git-date +%s; openaps init . ) that I have tested fixed a git corruption that the standard test did not detect.

https://github.com/iainct/oref0/blob/dev/bin/oref0-reset-git.sh#L40

original issue show here:
pi@raspberrypi:~/slicaps $ git diff error: object file .git/objects/64/637af367202e8101856fb78ad32f7ed41691d6 is empty fatal: unable to read 64637af367202e8101856fb78ad32f7ed41691d6

After upgrading the advanced-meal-assist branch to the latest and greatest, my monitor/meal.json is failing. The line in my meal.ini file was fields = pumphistory profile clock carbhistory glucose basal_profile and that worked previously. My settings/carbhistory.json file is empty because reasons. Running the command manually, but with my previous order like so: oref0-meal monitor/pumphistory.json settings/profile.json monitor/clock.json settings/carbhistory.json monitor/glucose.json settings/basal_profile.json gives me input data: [SyntaxError: Unexpected end of input]. However, if I make the order of arguments match the help prompt, then the report runs fine, both manually and from within my loop.

Currently oref0-meal.js ignores any carbs more than DIA hours ago when calculating COB. For really large meals that take more than DIA hours to absorb, would be good to consider older carbs as well.

Was testing out the auto sens part lately. One edge case was that I was running it using my backup pump with real bg data but fake pump data - everything was really screwed up and it came out with some really crazy results like 10X the sensitivity or something like that. Given that sometimes people switch out pumps and other things like that, maybe worthwhile to put some safeguards on the range that it can adjust things

Here are the commits from two iter_pump_hours 4 reports, one loop apart. The first commit adds a bolus wizard entry to history, the second adds the bolus itself to history (presumably the record was created after delivery finished?). Because the bolus has the same timestamp as the already-uploaded bolus wizard entry, it is excluded by nightscout cull-latest-openaps-treatments:

First commit: https://gist.github.com/mddub/f64b5ae8cacbaa1b67b308654450ec46
Second commit: https://gist.github.com/mddub/ec20634e5a579a7225655b715b2bf930

From my openaps.ini:

latest-ns-treatment-time = ! bash -c "nightscout latest-openaps-treatment $NIGHTSCOUT_HOST | json created_at"
format-latest-nightscout-treatments = ! bash -c "nightscout cull-latest-openaps-treatments monitor/pumphistory-zoned.json settings/model.json $(openaps latest-ns-treatment-time) > upload/latest-treatments.json"
upload-recent-treatments = ! bash -c "openaps format-latest-nightscout-treatments && test $(json -f upload/latest-treatments.json -a created_at eventType | wc -l ) -gt 0 && (ns-upload $NIGHTSCOUT_HOST $API_SECRET treatments.json upload/latest-treatments.json ) || echo \"No recent treatments to upload\""
upload = ! bash -c "openaps upload-recent-treatments 2>/dev/null >/dev/null && echo -n \"Uploaded; most recent treatment event @ \" && openaps latest-ns-treatment-time || echo \"Error; could not upload\""

I set these aliases up a long time ago according to the docs. Is there a newer way to upload treatments which prevents this issue?

(Running openaps 0.1.0, oref0 0.1.4, decocare 0.0.23)

After watching COB stay way to high for a long time I finally found the issue was caused by using my short glucose.json with 10 entries instead of the 24h glucose. The result was that carbs older than 50 minutes were never getting decayed.

Per the discussion in #58 and many preceding and subsequent discussions in various fora, it would be very useful to implement an algorithm to provide suggestions for tuning a user's current basal schedule, insulin sensitivity factor (ISF), insulin to carb (IC) ratio, and duration of insulin action (DIA).

To do so, we could use the OpenAPS automatic sensitivity detection algorithm to calculate, for each 5m data point, how much BG deltas deviated from what would’ve been predicted based on net insulin activity (relative to current schedule basals) and the user's current ratios.

For each carb entry, we can determine the sum of subsequent positive deviations (above a threshold used to detect when carbs are absorbing, perhaps 0.5 mg/dL/min). This can then be directly compared to the carb estimate to get a carb sensitivity factor (CSF, units of mg/dL per g), which can be combined with estimated ISF (units of mg/dL per U) to get IC ratio (units of g per U). (Note: while it would appear that this relies on correct carb counting, it actually does not. If the user's carb counts are biased high or low, the resulting ratios will essentially correct for that, resulting in what you might call perceived-carb ratios.)

To calculate ISF/DIA and tune basal schedules, we need to first exclude any data between carb announcement and 15-30 minute after the last carb absorption is detected. We can then bucket the remaining data points according to whether their insulin activity is dominated by boluses / high-temps or by basal insulin. The former can be used to calculate ISF, and the latter to tune basal schedules. We can initially assume that ISF is constant, and then come back and fine-tune based on time of day or day-to-day variation, assuming we have sufficient data, after we’ve tuned basal schedules and DIA.

To calculate ISF, we need to take the subset of data with no carb absorption whose insulin activity is bolus / high-temp dominated as described above, and perform a parameter estimation function to solve for ISF. One simple (but maybe not ideal) method to do so would be to perform a simple goal seek algorithm, recalculating the deviations for each new ISF to find the ISF that converges on a median or mean deviation (for this subset of data) of zero.

To determine the best DIA, we want to select the one that minimizes the sum (or perhaps the RMS?) of all the deviations, in order to choose the insulin curve that best fits the actual observed insulin activity profile over time. This parameter could be estimated from the same subset of data used to estimate ISF, using simple goal seek or more sophisticated parameter estimation methods.

To tune basal schedules, we can use the data points whose insulin activity is dominated by basal insulin (where net IOB is small), bucket the data by time of day, and look at the BG deviations for the timeframe ~90m later. (So for a 12am-2am basal, the deviations of interest would be for 1:30am-3:30am.) To determine how far off the schedule basal is for each time bucket, we can divide the previously-estimated ISF (mg/dL per U) by the median deviation (units of mg/dL per hour) to get a basal offset in U/hr.

IOB is impacted by square waves, and suspend/resumes. It'd be nice to have them considered.

it is possible to change the setting between "exchange" and "grams" in the bolus wizard menue of the pump under Carb. Units. Exchange is used to express how many insulin units are necessary to be given for 1 BE (10 to 12 grams of carbohydrate). In our case it was 0.6 units for 10 grams of carbs. If the option "grams" is selected the input is how many grams of carbs are necessary for 1 unit of insulin. In this example 16. Meal-assist uses the carb_ratios.json for its calculations and expects the value entered in "grams". If the setting "exchange" is used, it calculates way to aggressive. It should be made clear in the docs how important it is to use the "grams" setting with meal-assist and to implement a verification which setting is used when the JSON-file is generated. If the setting "exchange" is used, the values should be refused and the error message should point to set the setting to "grams"

Can we build in some sort of alerting framework into oref0 that makes it easy to configure alerts for things we should be taking manual action on?

Regardless of pumphistory and clock, the version of oref0-calculate-iob in dev always reports:
{"iob":0,"activity":0,"bolusiob":0}

Currently determine-basal will allow a temp to continue running if it's within 0.1U/hr of the desired temp, or if insulinReq is within 0.1U. Rather than using a hard-coded value, which is too small for basals >1U/hr and possibly too large for very small basal rates, it would be better to let the current temp run if it is within a certain percentage (perhaps 20%) of the desired temp/quantity.

@cjo20 thoughts?

I have encountered an issue where inadequate blood glucose data to produce an auto-sens.json file is stopping determine-basal reporting from producing a report, and therefore stopping looping.

It would appear to be more logical to continue to have determine-basal produce a report if auto-sens.json is missing or has no data, rather than stopping the report from proceeding at all.

If BG is not yet rising from carbs (like in eating soon mode), don't reset snoozeBG.