Git Product home page Git Product logo

logger's Introduction

Logger

Join the chat at https://gitter.im/thebookins/xdrip-js

Please note this project is neither created nor backed by Dexcom, Inc. Logger is not a product. Logger comes with no warranty or official support. Anyone using Logger is doing so at their own risk and must take responsibility for their own safety. The use of Logger for therapy is not FDA approved and comes with inherent risks.

Logger is a Bash (shell) based Dexcom g5 / g6 glucose pre-processor for OpenAPS. Logger runs on the OpenAPS rig and connects to the g5 or g6 transmitter via bluetooth, waits for the first bg, logs a json entry record, processes the entry record based on calibrations, updates NightScout and OpenAPS indepently, then exits. Logger is a wrapper shell script to xdrip-js that is called from cron every minute. The user interface is a mixture between unix command line scripts and NightScout. The current Logger features are below:

  • Preparation and sending of the blood glucose data to Nightscout and to OpenAPS.

  • Alternate Bluetooth connection - allows Logger to run alongside one other CGM app (besides the receiver). With this setting set to true, Logger can run alongside Xdrip+ or the Dexcom App.

  • Offline mode - Logger runs on the rig and sends bg data directly to openaps through via xdripAPS. Logger queues up NS updates while internet is down and fills in the gaps when internet is restored.

  • Reset Transmitter - Use the following command, wait > 10 minutes, and your expired transmitter is new again! Careful, though it will reset everything on your transmitter including your current session. Note this feature is only available via the command line. cgm-reset

  • Stop Sensor - Use the following command, wait > 5 minutes, and your current sensor session will stop. Use this command before changing out your sensor so that Logger will stop sending glucose information temporarily (while the sensor is stopped) This feature is also available via the command line only. cgm-stop

  • Start Sensor - Use the following command, wait > 5 minutes, and your sensor session will start. Use this command after inserting your sensor so that Logger will start the process for sending glucose information. This feature is also available via the command line and you can also do it via Nightscout as a BG Treatment, entry type of Sensor Start. cgm-start

  • Set Transmitter - Use the following command, wait > 5 minutes, and your new transmitter id will be used. This feature is only available if using the xdripjs.json configuration file & not calling Logger with the transmitter id as an argument. cgm-transmitter

  • Calibration via linear least squares regression (LSR) (similar to xdrip plus)

    • Calibrations must be input into Nightscout as BG Check treatments or command line calibrate bg_value.
    • Logger will not calculate or send any BG data out unless at least one calibration has been done in Nightscout. Calibration is not required for a g6 when using no-calibration by sending it a valid sensor serial code in the cgm-start message.
    • LSR calibration only comes into play after 3 or more calibrations. When there one or two calibrations, single point linear calibration is used.
    • The calibration cache will be cleared for the first 15 minutes after a Nightscout "CGM Sensor Insert" has been posted as Nightscout treatment.
    • After 15 minutes, BG data will only be sent out after at least one calibration has been documented in Nightscout.
  • fakemeter - Additional offline ability where Logger sends the blood glucose to the pump. To enable, simply go into the pump and add a meter with id 000000 through the Utilities -> Meter Options menu

Warning!

Logger LSR calibration is a new feature as of Feb/2018. Only those who closely monitor and check blood glucose and regularly review the Logger logfiles should use this program at this time.

  • /var/log/openaps/logger-loop.log
  • /var/log/openaps/cgm.csv
  • /root/myopenaps/monitor/xdripjs/calibrations.csv - the current list of calibrations (unfiltered, BG check, datetime, BG Check ID)
  • /root/myopenaps/monitor/xdripjs/calibration-linear.json - the current calibration values (slope, yIntercept). Please note that other fields in this file are for informational purposes at this time.

Join the chat at https://gitter.im/thebookins/xdrip-js

Please note this project is neither created nor backed by Dexcom, Inc.

Prerequisites

  • Openaps must be installed using the CGM method of xdrip.
  • Logger does not currently support token-based authentication with Nightscout

Update node version. Follow these steps in this order.

Set Nightscout environment variables in ~/.bash_profile. Make sure the following 4 lines are in this file. If not carefully add them paying close attention the values (xxxx is your hashed Nightscout API_SECRET and yyyy is your Nightscout URL):

API_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxx
export API_SECRET
NIGHTSCOUT_HOST=https://yyyyyyy.herokuapp.com
export NIGHTSCOUT_HOST

For the edison/explorer board (jubilinux), the version of Node that ships with jubilinux is old (v0.10.something). Here are the instructions for updating the edison/explorer rig's Node:

sudo apt-get remove nodered -y
sudo apt-get remove nodejs nodejs-legacy -y
sudo apt-get remove npm  -y # if you installed npm
sudo curl -sL https://deb.nodesource.com/setup_8.x | sudo bash -
sudo apt-get install nodejs -y

cd ~/src/Logger && npm rebuild # if Logger exists already and you are upgrading node from 6.x to 8.x

If you are using the RPI zero hat, the version of Node that ships with jubilinux also needs updating. Here are the instructions for updating the rpi zero hat's Node:

mkdir node && cd node
wget https://nodejs.org/dist/v8.10.0/node-v8.10.0-linux-armv6l.tar.xz
tar -xf node-v8.10.0-linux-armv6l.tar.xz
cd *6l
sudo cp -R * /usr/local/
# add /usr/local/bin to PATH env variable in .bash_profile
node -v
# should show 8.10.0 at this point

Installation

cd ~/src
git clone https://github.com/xdrip-js/Logger.git
cd Logger
sudo npm run global-install
sudo apt-get install bluez-tools

Set your transmitter (replace 403BX6 with your actual 6 digit transmitter serial number)...

cgm-transmitter 403BX6

Add cron job entry ...

* * * * * cd /root/src/Logger && ps aux | grep -v grep | grep -q Logger || /usr/local/bin/Logger >> /var/log/openaps/logger-loop.log 2>&1

Logger Configuration Options:

You can edit the following options values in the configuration file (~/myopenaps/xdripjs.json)

{"transmitter_id":"8XXXXX"}, Required - 6 character tx serial number (i.e. 403BX6).  Should be set using cgm-transmitter cmd.
{"sensor_code":"1234"}, Optional - Sensor code used for G6 only. Start a transmitter using this to use the G6 no-calibration mode. If set to a non-empty string, calibrations are not sent to the transmitter. To set a new sensor code you must set it using the cgm-start cmd or NS.
{"mode":"native-calibrates-lsr"}, Optional - if you specify "read-only" then Logger still sends BG values to OpenAPS, but all start/stop/calibrations must be done by the receiver, the Dexcom App, or XdripPlus. The default is "native-calibrates-lsr" then it will use the Dexcom algorithms, unless not available, and it will also calibrate the local LSR algorithm based on the values of the Dexcom algorithm, every 6 hours. "native-calibrates-lsr" is useful to allow expired sessions and extended sensors to automatically use recent calibration data. 

{"pump_units":"mg/dl"}, Optional - pumpUnits default is "mg/dl"
{"fake_meter_id":"000000"}, Optional - meterid for fakemeter sending glucose records to pump, default is "000000"
{"alternate_bluetooth_channel":true or false} Optional - Default is true. If set to true then Logger uses the alternate channel to connect to the Dexcom transmitter. If set to true the receiver cannot be used. However, when set to true, either the Xdip Plus android app or the Dexcom Iphone app can be used alongside Logger. Keep in mind that there is a higher chance for bluetooth conflict when connecting to the transmitter with both channels. You may be able to avoid some reconnects by keeping the rig and the phone app physically separated by several inches.
{"watchdog":true or false}, Optional - Default is true. If set to true then Logger will automatically reboot the rig to resolve bluetooth issues if no glucose is seen from the transmitter in more than 14 minutes.
{"utc":true or false}, Optional - Default is true. If set to true then Logger will assume UTC date/time strings coming from the curls to NightScout. 
{"fakemeter_only_offline":true or false}, Optional - Default is false. If set to false then Logger will not attempt to call fakemeter unless the rig is offline. 

Getting Started

First, make sure you've checked the Prerequisites above and completed the Installation steps. Afterwords, perform the following steps:

Review the Logger logfile to ensure the proccess is proceeding without errors. Run the following command on the rig and keep it running for debugging purposes for the first time you go through the procedures below:

tail -f /var/log/openaps/logger-loop.log

Insert the new Dexcom g5 or g6 sensor. Notify Logger of this insertion by using Nightscout Treatment CGM Sensor Insert

Start the new Dexcom g5 sensor by using one of the two methods (Nightscout Treatment CGM Sensor Start) or (run cgm-start from the command line)

If starting a g6, specify the sensor serial code by putting the 4 digit code in the Nightscout Sensor Start Note or passing the the command line as follows: cgm-start -c 1234 where 1234 is the sensor serial code.

Within 15 minutes, the sensor state should show "Warmup" in Nightscout and in the Logger log. At this point, you have 2 options:

Prefered: Wait to calibrate until the 2 hour warmup period is complete and the sensor state in NS shows "First Calibration". Calibrate again once the sensor state in NS shows "Second Calibration"

Not Prefered / Advanced mode: Calibrate after > 15 minutes since CGM start.

Calibrate by using one of the two methods (Nightscout Treatment BG Check and put calibrated glucose in the Glucose Reading field) or (run calibrate bgvalue from the command line)

After calibration(s), you should see BG values in Nightscout and in the log.

Troubleshooting

G5 Tx unfiltered / filtered values showing 0

If both unfiltered and filtered values are showing up as 0 in the Logger logfile, then you may be able to solve the problem by doing a cgm-reset. Note: This will reset the session and you will lose any transmitter stored calibrations so this technique is probably best used when unfiltered is 0 upon new sensor insertion.

Tx Communication Timing Out Issues

If timing out, recheck the configuration of the transmitter id, make sure the bt-device command is available, ensure there are no conflicting bluetooth connections on the same channel (i.e. Dexcom App, Receiver, or XDrip+).

Since the timer only allows communications for a few seconds every 5 minutes, isolation of any timeout issues are key. The following are some suggestions:

  1. Turn off Logger and receiver, run Xdrip+ and see if it can connect to the tx. If Xdrip+ can connect, then it may be a rig issue. If Xdrip+ cannot connect, then it's mostly isolated to a tx issue.
  2. Run the following command. If it fails for any reason, then the install of bluez-tools may have an issue. If bluez-tools is not installed, then bt timeouts will likely be the result. bt-device -l
  3. Check your network log. BT PAN may be trying to do something and restarting bluetoothd. Check /var/log/openaps/network.log and any bluetooth log (if it exists) in that directory. Also, check for bluetooth errors in /var/log/syslog as well.
  4. Turn off every other possible dexcom connection and try connecting with the tx with the official Dexcom app. This will only work if you have a non-expired tx or have successfully reset it earlier.
  5. Try a different transmitter (if you have one). If this works, then the other tx has an issue, usually battery related.
  6. Try a different rig (if you have one). If this works, then the other rig or it's install/configuration is likely the culprit.
  7. If on an RPI, check the mode in /etc/bluetooth/main.conf. At least one user found that setting mode to Controller mode resolved the timeouts.

If you have network connectivity on the rig, but BG values are not showing up on NightScout, then run the following command which should retrieve the last BG Check Treatment posted to NightScout. Review any errors that the command returns and re-check your NIGHTSCOUT_HOST and API_SECRET environment variables.

curl --compressed -m 30 -H "API-SECRET: ${API_SECRET}" "${NIGHTSCOUT_HOST}/api/v1/treatments.json?find\[eventType\]\[\$regex\]=Check&count=1"

Checking Sensor Noise Levels

To check recent reported BG noise levels, run the command line utility cgm-noise

Install issues

If during installation you get stuck after running this command sudo npm run global-install at a point starting fetchMetadata it suggests you have an issue connecting to the git reposistory. E.g.:

> [email protected] global-install /root/src/Logger
> rm -rf ./node-modules/xdrip-js && ./bin/upgrade-node.sh && npm install && ./bin/logger-setup.sh

Node version already at v8 - good to go
[  ................] / fetchMetadata: sill install loadAllDepsIntoIdealTree

To fix this, run the following git config --global url."https://".insteadOf git:// which points the downloader to the https:// instance and run sudo npm run global-install again and carry on from that point.

If this doesn't work, delete the Logger folder and all it's contents rm -rf /root/src/Logger and then re-run the installation from the start.

logger's People

Contributors

alimhass avatar cfomodz avatar cluckj avatar efidoman avatar foxy7 avatar j4cbo avatar jonathan-cole avatar jpcunningh avatar luucv avatar scottleibrand avatar thebookins avatar tynbendad avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

logger's Issues

Failure to Run when Nightscout isn't Running

When Logger can access Nightscout everything works well. When the device isn't connected to the Internet things remain running well, but if the device is connected to the Internet, the Nightscout host is up and accessible, but Nightscout isn't running the following error is given repeatedly and Logger doesn't collect readings:

07/11 06:38:03 Starting Logger
parse error: Invalid numeric literal at line 1, column 10
07/11 06:38:08 You must record a "Sensor Insert" in Nightscout before Logger will run
07/11 06:38:08 exiting

To fix the situation where the device is connected to the Internet, the Nightscout host is up and accessible, but Nightscout isn't running you have to blackhole the Nightscout host for the entire device by adding a bogus entry to /etc/hosts. Ideally, this would not be necessary and Logger would continue to work in this situation.

Calibration issues

Following the issues I saw with the calibrations on logger, below is the content of the requested files:

g5.csv

epochdate datetime unfiltered filtered trend calibratedBG meterbg slope yIntercept slopeError yError rSquared Noise NoiseSend
1519301888 ########## 140064 140544 Flat 131 131 1069 0 0 0 0 0 1
1519302188 ########## 140288 140416 Flat 131 null 1069 0 0 0 0 0 1
1519302487 ########## 140064 140224 Flat 131 null 1069 0 0 0 0 0 1
1519302788 ########## 140160 140096 Flat 131 null 1069 0 0 0 0 0.00014 1
1519303088 ########## 139712 140000 Flat 130 null 1069 0 0 0 0 0.00037 1
1519303388 ########## 138880 139840 Flat 129   1069 0 0 0 0 0.00083 1
1519303688 ########## 137408 139296 Flat 128 null 1069 0 0 0 0 0.00197 1
1519303988 ########## 135616 138272 Flat 126 null 1069 0 0 0 0 0.00302 1
1519304288 ########## 134400 136800 Flat 125   1069 0 0 0 0 0.00285 1
1519304588 ########## 132544 135104 Flat 123 null 1069 0 0 0 0 0.0033 1
1519304888 ########## 131392 133440 Flat 122 null 1069 0 0 0 0 0.00302 1
1519305187 ########## 130528 132032 Flat 122 null 1069 0 0 0 0 0.00274 1
1519305488 ########## 128864 130784 Flat 120 null 1069 0 0 0 0 0.00225 1
1519305788 ########## 127680 129632 Flat 119 null 1069 0 0 0 0 0.00186 1
1519306088 ########## 125680 128320 Flat 117 null 1069 0 0 0 0 0.00122 1
1519306388 ########## 126160 127008 Flat 118 null 1069 0 0 0 0 0.00253 1
1519306688 ########## 126896 126128 Flat 118 null 1069 0 0 0 0 0.00446 1
1519306988 ########## 133184 126688 Flat 124 null 1069 0 0 0 0 0.0288 1
1519307288 ########## 138208 129472 Flat 129 null 1069 0 0 0 0 0.04 1
1519307587 ########## 141280 134048 Flat 132 null 1069 0 0 0 0 0.04202 1
1519307888 ########## 143936 139104 Flat 134   1069 0 0 0 0 0.03979 1
1519308187 ########## 144096 142848 Flat 134 null 1069 0 0 0 0 0.03761 1
1519308488 ########## 143968 144576 Flat 134 null 1069 0 0 0 0 0.03628 1
1519308788 ########## 145632 144896 Flat 136 null 1069 0 0 0 0 0.03145 1
1519309087 ########## 147552 145056 Flat 138 null 1069 0 0 0 0 0.02718 1
1519309388 ########## 146976 145728 Flat 137 null 1069 0 0 0 0 0.0225 1
1519309688 ########## 149536 147008 Flat 139 null 1069 0 0 0 0 0.02134 1
1519309988 ########## 150176 148416 Flat 140 null 1069 0 0 0 0 0.02147 1
1519310288 ########## 150688 149568 Flat 140 null 1069 0 0 0 0 0.01404 1
1519310588 ########## 152416 150624 Flat 142 null 1069 0 0 0 0 0.00803 1
1519310888 ########## 151808 151424 Flat 142 null 1069 0 0 0 0 0.00772 1
1519311189 ########## 152000 151936 Flat 142 null 1069 0 0 0 0 0.00625 1
1519311489 ########## 152480 152192 Flat 142 null 1069 0 0 0 0 0.0061 1
1519311788 ########## 153120 152320 Flat 143 null 1069 0 0 0 0 0.00564 1
1519312088 ########## 152128 152480 Flat 142 null 1069 0 0 0 0 0.00676 1
1519312388 ########## 151232 152416 Flat 141 null 1069 0 0 0 0 0.00662 1
1519312688 ########## 150304 151936 Flat 140   1069 0 0 0 0 0.00704 1
1519312987 ########## 150272 151104 Flat 140 null 1069 0 0 0 0 0.00401 1
1519313287 ########## 153696 150720 Flat 143 null 1069 0 0 0 0 0.00946 1
1519313587 ########## 154528 151424 Flat 144   1069 0 0 0 0 0.00958 1
1519313888 ########## 155552 153120 Flat 145   1069 0 0 0 0 0.00876 1
1519314188 ########## 155200 154848 Flat 145   1069 0 0 0 0 0.00854 1
1519314487 ########## 156608 155872 Flat 146 null 1069 0 0 0 0 0.00911 1
1519314789 ########## 159968 156608 Flat 149 null 1069 0 0 0 0 0.01317 1
1519315087 ########## 163168 158048 Flat 152 null 1069 0 0 0 0 0.01614 1
1519315387 ########## 163456 160352 Flat 152 null 1069 0 0 0 0 0.0143 1
1519315687 ########## 166592 163072 Flat 161 171 974 0 0 0 0 0.01371 1
1519315988 ########## 168000 165408 FortyFiveUp 172 171 974 0 0 0 0 0.01053 1
1519316288 ########## 169376 167200 FortyFiveUp 173 null 974 0 0 0 0 0.00901 1
1519316587 ########## 171488 168768 Flat 176 null 974 0 0 0 0 0.00763 1
1519316887 ########## 173152 170368 Flat 177 null 974 0 0 0 0 0.00726 1
1519317187 ########## 173120 171872 Flat 177 null 974 0 0 0 0 0.00857 1
1519317487 ########## 175520 173280 Flat 180 null 974 0 0 0 0 0.00649 1
1519317787 ########## 175776 174432 Flat 180 null 974 0 0 0 0 0.00767 1
1519318088 ########## 176800 175424 Flat 181 null 974 0 0 0 0 0.00645 1
1519318387 ########## 176544 176224 Flat 181 null 974 0 0 0 0 0.00618 1
1519318687 ########## 176512 176640 Flat 181 null 974 0 0 0 0 0.00656 1
1519318987 ########## 175104 176576 Flat 179 null 974 0 0 0 0 0.0073 1
1519319287 ########## 174816 176000 Flat 179 null 974 0 0 0 0 0.00757 1
1519319587 ########## 173024 175040 Flat 177 null 974 0 0 0 0 0.00995 1
1519319888 ########## 172704 173952 Flat 177 null 974 0 0 0 0 0.00817 1
1519320188 ########## 171712 172928 Flat 176 null 974 0 0 0 0 0.00716 1
1519320488 ########## 169696 171904 Flat 174 null 974 0 0 0 0 0.00889 1
1519320787 ########## 167264 170624 Flat 171 null 974 0 0 0 0 0.00614 1
1519321087 ########## 166496 168960 Flat 170 null 974 0 0 0 0 0.00552 1
1519321387 ########## 164704 167104 Flat 169 null 974 0 0 0 0 0.00368 1
1519321687 ########## 161536 165184 Flat 165 null 974 0 0 0 0 0.00509 1
1519321987 ########## 157152 162912 Flat 161 null 974 0 0 0 0 0.00765 1
1519322287 ########## 155104 160000 Flat 159 null 974 0 0 0 0 0.00761 1
1519322588 ########## 152224 156672 Flat 156 null 974 0 0 0 0 0.00659 1
1519322887 ########## 146720 153152 Flat 150 null 974 0 0 0 0 0.01121 1
1519323189 ########## 142688 149440 Flat 146 null 974 0 0 0 0 0.00955 1
1519323488 ########## 129808 144352 FortyFiveDown 133 null 974 0 0 0 0 0.03253 1
1519323789 ########## 128736 137760 FortyFiveDown 132 null 974 0 0 0 0 0.03443 1
1519324087 ########## 127056 131264 FortyFiveDown 130 null 974 0 0 0 0 0.03554 1
1519324387 ########## 118976 125952 FortyFiveDown 122   974 0 0 0 0 0.03547 1
1519324687 ########## 117280 122208 Flat 120 null 974 0 0 0 0 0.03569 1
1519324987 ########## 111216 118688 Flat 114 null 974 0 0 0 0 0.03553 1
1519325287 ########## 110368 114800 Flat 113 null 974 0 0 0 0 0.04097 1
1519325587 ########## 109120 111392 Flat 112   974 0 0 0 0 0.04273 1
1519325887 ########## 108272 109104 Flat 111 null 974 0 0 0 0 0.04667 1
1519326187 ########## 105184 107856 Flat 107   974 0 0 0 0 0.04639 1
1519326488 ########## 103424 106672 Flat 106   974 0 0 0 0 0.0482 1
1519327087 ########## 96560 102736 Flat 99   974 0 0 0 0 0.02109 1
1519327387 ########## 91680 99456 Flat 94 null 974 0 0 0 0 0.02083 1
1519327687 ########## 85776 94960 Flat 88 null 974 0 0 0 0 0.02197 1
1519327987 ########## 80928 89408 FortyFiveDown 73 61 779 34917 27 3928 0.97854 0.01581 1
1519328287 ########## 77808 83872 FortyFiveDown 64 null 779 34917 27 3928 0.97854 0.01479 1
1519328587 ########## 76816 79456 SingleDown 53 null 779 34917 27 3928 0.97854 0.01365 1
1519328887 ########## 81120 77376 Flat 59 null 779 34917 27 3928 0.97854 0.0342 1
1519329187 ########## 93088 79120 Flat 74 null 779 34917 27 3928 0.97854 0.10314 1
1519329487 ########## 113568 86624 SingleUp 87 null 779 34917 27 3928 0.97854 0.19946 1
1519336386 ########## 219040 212640 DoubleUp 241 243 747 38358 12 2633 0.98081 0.19946 1
1519336688 ########## 221056 216704 DoubleUp 244 244 748 38270 6 1691 0.98643 0.0841 1
1519336988 ########## 223392 219872 DoubleUp 247 null 748 38270 6 1691 0.98643 0.07189 1
1519337287 ########## 227136 222592 DoubleUp 252 null 748 38270 6 1691 0.98643 0.06057 1
1519337587 ########## 226880 224896 Flat 252 null 748 38270 6 1691 0.98643 0.04889 1
1519337887 ########## 227200 226592 Flat 252 null 748 38270 6 1691 0.98643 0.039 1
1519338187 ########## 225408 227296 Flat 250 null 748 38270 6 1691 0.98643 0.03631 1
1519338487 ########## 217280 225920 Flat 239   748 38270 6 1691 0.98643 0.05381 1
1519338787 ########## 218528 222816 Flat 240 null 748 38270 6 1691 0.98643 0.05511 1
1519339087 ########## 215360 219072 Flat 236 null 748 38270 6 1691 0.98643 0.05784 1
1519339387 ########## 211776 215712 Flat 231 null 748 38270 6 1691 0.98643 0.04693 1
1519339687 ########## 195872 211744 FortyFiveDown 220 null 748 38270 6 1691 0.98643 0.1094 1
1519339987 ########## 131552 198720 DoubleDown 124 null 748 38270 6 1691 0.98643 0.1094 1
1519340286 ########## 189408 180352 FortyFiveDown 202 null 748 38270 6 1691 0.98643 0.1094 1
1519340587 ########## 157600 164512 SingleDown 180 null 748 38270 6 1691 0.98643 0.50102 3
1519340887 ########## 196800 161696 FortyFiveDown 195 null 748 38270 6 1691 0.98643 0.62192 0
1519341187 ########## 147904 168960 FortyFiveUp 146 null 748 38270 6 1691 0.98643 0.62192 4
1519341487 ########## 138336 169312 DoubleDown 133 null 748 38270 6 1691 0.98643 0.58585 3
1519341787 ########## 180512 163488 Flat 190 null 748 38270 6 1691 0.98643 0.58585 3
1519342087 ########## 167648 157856 SingleUp 181 null 748 38270 6 1691 0.98643 0.68729 0
1519342388 ########## 173376 161152 DoubleUp 181 187 747 37690 5 1709 0.98896 0.69879 0
1519342687 ########## 170528 169824 Flat 177 null 747 37690 5 1709 0.98896 0.69944 0
1519342987 ########## 167648 174016 Flat 173 null 747 37690 5 1709 0.98896 0.70419 0
1519343286 ########## 165248 172384 Flat 170 null 747 37690 5 1709 0.98896 0.64238 0
1519343587 ########## 161728 167648 Flat 166 null 747 37690 5 1709 0.98896 0.5777 3
1519343887 ########## 159136 163008 Flat 162 null 747 37690 5 1709 0.98896 0.54951 3
1519344187 ########## 153536 159424 Flat 155 null 747 37690 5 1709 0.98896 0.43187 3
1519344487 ########## 148768 156032 FortyFiveDown 148 null 747 37690 5 1709 0.98896 0.34277 2
1519344786 ########## 142304 151744 FortyFiveDown 140 null 747 37690 5 1709 0.98896 0.33326 2
1519345088 ########## 136512 146176 FortyFiveDown 132 null 747 37690 5 1709 0.98896 0.06732 1
1519348090 ########## 106048 104864 Flat 91   747 37690 5 1709 0.98896 0.02672 1
1519348387 ########## 108624 105728 Flat 95 97 750 36963 4 1349 0.98994 0.01481 1
1519348688 ########## 111344 107344 Flat 99   750 36963 4 1349 0.98994 0.02395 1
1519348986 ########## 114832 109696 Flat 103 null 750 36963 4 1349 0.98994 0.03472 1
1519349287 ########## 115408 112432 Flat 104 null 750 36963 4 1349 0.98994 0.03745 1
1519349586 ########## 116304 114816 Flat 105 null 750 36963 4 1349 0.98994 0.04022 1
1519349886 ########## 115648 116192 Flat 104 null 750 36963 4 1349 0.98994 0.04092 1
1519350186 ########## 113904 116208 Flat 102 null 750 36963 4 1349 0.98994 0.03828 1
1519350486 ########## 111840 115088 Flat 99 null 750 36963 4 1349 0.98994 0.03643 1
1519363085 ########## 249856 134752 Flat 250 135 874 31295 61 18623 0.95513 0.0388 1
1519363386 ########## 189440 164064 DoubleDown 180 null 874 31295 61 18623 0.95513 0.0388 1
1519363686 ########## 164896 192448 DoubleDown 166 null 874 31295 61 18623 0.95513 0.16299 1
1519363986 ########## 149312 199520 DoubleDown 148 118 848 37862 56 18071 0.94517 0.17895 1
1519364286 ########## 140928 177248 DoubleDown 134 null 848 37862 56 18071 0.94517 0.18455 1
1519364586 ########## 135840 148096 SingleDown 124 null 848 37862 56 18071 0.94517 0.1863 1
1519364886 ########## 134528 131040 SingleDown 113 null 848 37862 56 18071 0.94517 0.18656 1
1519365186 ########## 133568 128288 FortyFiveDown 112 null 848 37862 56 18071 0.94517 0.18668 1
1519365486 ########## 132224 131840 Flat 111 null 848 37862 56 18071 0.94517 0.18683 1
1519365786 ########## 130784 134368 Flat 109 null 848 37862 56 18071 0.94517 0.18674 1
1519366086 ########## 132832 133952 Flat 111   848 37862 56 18071 0.94517 0.18645 1
1519366386 ########## 133120 132416 Flat 112 null 848 37862 56 18071 0.94517 0.16099 1
1519366686 ########## 132704 131744 Flat 111   848 37862 56 18071 0.94517 0.12481 1
1519366986 ########## 131840 132096 Flat 110 null 848 37862 56 18071 0.94517 0.0786 1
1519367286 ########## 131584 132448 Flat 110 null 848 37862 56 18071 0.94517 0.03557 1
1519367586 ########## 132480 132320 Flat 111 null 848 37862 56 18071 0.94517 0.01651 1
1519367886 ########## 134720 132320 Flat 114 null 848 37862 56 18071 0.94517 0.00958 1
1519368186 ########## 136768 133184 Flat 116 null 848 37862 56 18071 0.94517 0.01073 1
1519368486 ########## 137152 134912 Flat 117 null 848 37862 56 18071 0.94517 0.00991 1
1519368786 ########## 134112 136288 Flat 113 null 848 37862 56 18071 0.94517 0.01431 1
1519369086 ########## 133856 136320 Flat 115 147 862 34231 63 18926 0.92196 0.01293 1
1519369386 ########## 132672 135040 Flat 114 null 862 34231 63 18926 0.92196 0.0119 1
1519369686 ########## 129664 133056 Flat 110 null 862 34231 63 18926 0.92196 0.01606 1
1519369986 ########## 128144 131040 Flat 108 null 862 34231 63 18926 0.92196 0.01677 1
1519370286 ########## 127296 129216 Flat 107 null 862 34231 63 18926 0.92196 0.01675 1
1519370586 ########## 125056 127536 Flat 110 138 893 26262 63 19794 0.91372 0.0183 1
1519370886 ########## 125968 126272 Flat 111 138 893 26262 63 19794 0.91372 0.01833 1
1519371186 ########## 125920 125584 Flat 111 null 893 26262 63 19794 0.91372 0.01391 1
1519371486 ########## 124512 125312 Flat 110 null 893 26262 63 19794 0.91372 0.00909 1
1519371786 ########## 123104 125008 Flat 108 null 893 26262 63 19794 0.91372 0.00774 1
1519372086 ########## 124512 124480 Flat 110 null 893 26262 63 19794 0.91372 0.00914 1
1519372386 ########## 125200 124080 Flat 110 null 893 26262 63 19794 0.91372 0.01023 1
1519372686 ########## 126112 124304 Flat 111 null 893 26262 63 19794 0.91372 0.01153 1
1519372986 ########## 126736 125232 Flat 112 null 893 26262 63 19794 0.91372 0.00864 1

Calibration-Linear.json

[{"slope":893, "yIntercept":26262, "formula":"calibratedbg=(unfiltered-yIntercept)/slope", "yError":19794, "slopeError":63, "rSquared":0.91372, "numCalibrations":8, "calibrationType":"LeastSquaresRegression"}]

calibrations.csv

140064 131 ########## 1519301888 5a8eb46718f7ba5efbccc254
166592 171 ########## 1519315687 5a8eead118f7ba5efbcf8123
80928 61 ########## 1519327987 5a8f1a2e18f7ba5efbd231a3
219040 243 ########## 1519336386 5a8f3a9018f7ba5efbd3f079
221056 244 ########## 1519336688 5a8f3c4118f7ba5efbd40613
173376 187 ########## 1519342388 5a8f528818f7ba5efbd53502
108624 97 ########## 1519348387 5a8f691d18f7ba5efbd65af5
249856 135 ########## 1519363085 5a8fa38f18f7ba5efbd95453
149312 118 ########## 1519363986 5a8fa6aa18f7ba5efbd98179
133856 147 ########## 1519369086 5a8fba6a18f7ba5efbda982f
125056 138 ########## 1519370586 5a8fc13318f7ba5efbdaf677

bad internet connection causes missed readings

my router has been having some issues lately causing us to lose internet overnight for a few hours. i noticed that after the internet is back, logger uploads spotty info to nightscout. looking into it a bit more there were actual missed readings, and the cause is due to incorrectly timing the "Waiting " in check_last_glucose_time_smart_sleep().

the code in that function uses the modification time of the file last-entry.json, however that file is modified after post-nightscout-with-backfill() call in main(), and I assume when internet connection drops that function has a long timeout (the curl's appear to use -m 30, and there are up to 3 of them, which would delay the next start by 90 seconds if all timeout).

the fix would be simple, just move the cp before the call to post-nightscout-with-backfill(), however there is another cp to last-entry.json at the end of capture_entry_values() and i'm not sure how this affects the modification timestamp.

also tacking on an unrelated issue here, from gitter, just to make sure we don't miss it:
@efidoman >A simpler solution may be to just add tx-expired to every status where tx age > 90. Is 90 days the same expire time frame for g5 and g6?

fidoman @efidoman Aug 11 04:22
Yes, I confirmed via Dexcom's web site, both are 90 days.
https://www.dexcom.com/faq/how-long-does-dexcom-g6-transmitter-last?category=2641

Feature Request: Glucose Listen Only

As discussed in Gitter with @efidoman, here is my feature request:

I use a G6 but I guess it's the technically the same as for G5. I'd like to use Logger just for listening to the transmitted glucose values. Everything else (calibration, sensor-start, sensor-stop, error management, etc) I want to handle with the app or the Dexcom reader. As I understood the documentation, Logger needs to send a sensor start command to the transmitter to listen to glucose values. I'm worried multiple start commands could cause issues with the auto calibration. However, even if not I guess it would be better to not send unnecessary commands if I just want the glucose values.

Cheers!

Constraints on unfiltered stop us from accepting valid native glucose

The native algorithm works down to lower values of unfiltered than logger currently allows... Ideally we should accept the native glucose even if we don't want to come up with our own calculations at this point, where logger said unfiltered was out of range..

,"date":1561635060883,"dateString":"2
019-06-27T11:31:00.883Z","sgv":53,"direction":"None","ty
pe":"sgv","filtered":17034,"unfiltered":25568,"rssi":-77
,"noise":"1","trend":2,"state":"OK","status":"OK","gluco
se":53}]

Please remove dependency on $HOME/myopenaps/pump.ini

With 0.7.0-dev and PR openaps/oref0#1176 there is no need for oref0 to create a pump.ini. Currently oref0 must create it because xdripjs-js Logger uses it for the fakemeter, see

export MEDTRONIC_PUMP_ID=`grep serial ~/myopenaps/pump.ini | tr -cd 0-9`

Can the Logger code be altered to comply with the 0.7.x oref0 way of doing it to read these settings from the preferences.json?

See @cluckj review at openaps/oref0#1176 (review) for details.

Install git vs https

Hi,

Regarding install url for logger and the issue i've had previously.

Is there any benefit in using git instead of https?

"url": "git://github.com/xdrip-js/Logger.git"

My install fails, i believe, due to this one and only instance of "git" in the code hence why i had to find a work around previously and additional comment in the troubleshooting help.

Also i notice that Oref0 uses "https" for the repo location in the package.json, https://github.com/openaps/oref0/blob/acb0150021e7bf284b6741a9510076a13f690747/package.json#L11.

I propose that we change package.json line 17 to be

"url": "https://github.com/xdrip-js/Logger.git"

I'm happy to make the pull request just unsure why it's git// in the first place.

thanks,

Feature Request: Bypass Watchdog if NS has recent BG

I'm using the read-only mode of Logger to offline loop. However at home the loop is online and can read the BG values from NS (where xdrip on the phone is pushing to). As I don't carry the loop with me all the time, it reboots quite often which and drains the battery but is not nessecary. Would be nice if the watchdog would check the monitor/glucose.json if recent values are available and prevent the reboot if so (and thus only reboots if no recent values are available).

Bluetooth Timout

Hi,

This isn't really an issue but more a comment. In the troubleshooting section in the readme it says to change the controller mode in the settings, but not to which mode to set. I found out that it needs to be on "bredr" mode.

It would be a nice addition to the troubleshooting section to state this.

Clear Calibrations unless restart specifically requested

Lookout does the following logic --> a Sensor Insert, Sensor Start or Sensor Stop will invalidate any BG Checks prior to that event. Only the BG Checks after the most recent of any of those events will be used, and if there are not enough BG Checks to calculate an LSR calibration, it will delete the calibration it has if it had one.

Logger does the following: a Sensor Insert clears the cache. It also means that Logger will only keep the latest calibration record for the first 12 hours (which are usually unstable and not good inputs for LSR). Logger doesn't clear cache for Sensor Start nor Sensor Stop.
and we don't want starts/stops to clear the cache since that would ruin restarts

A separate restart nightscout treatment would help with that

Someone should not have to do extra work to indicate something is not a restart. we should default to the (safer) new-sensor behavior unless someone specifically notes that they're doing a restart
that to me is the strongest reason for needing a separate sensor restart treatment in NS

A Sensor Insert, Sensor Start, and a Sensor Stop will invalidate any BG Checks prior to that event (same as Lookout). Then, if in NS, you add a "restart sensor" treatment type that will Stop/Start the sensor without deleting the calibration cache. Add restart sensor to the Logger command line utilities as well.

alternate bt channel should be automatically picked when configuring logger

After running the oref0-setup.sh, the "cgm-transmitter" command creates the ~/myopenaps/xdripjs.json config file needed by the logger; however, the logger is supposed to work in "parallel" with the main G5/G6 user (phone?).
The json file created by the "cgm-transmitter" command does not set the "alternate_bluetooth_channel" as true.

"Exit - Invalid response from g5 transmitter"

I had been using Logger for about a week (along side a receiver) with phenomenal results - thank you guys for all your hard work! One problem I'm running into now though is that Logger runs, connects to the transmitter, and then (for reasons I can't seem to get to the bottom of) doesn't like what it's getting out of the transmitter. I've been getting the following output since this morning:

06/24 15:24:04 Calling xdrip-js ... node logger 41D61F
Sun, 24 Jun 2018 19:24:09 GMT bluetooth-manager on -> stateChange: poweredOn
Sun, 24 Jun 2018 19:24:09 GMT bluetooth-manager starting scanning
Sun, 24 Jun 2018 19:24:09 GMT bluetooth-manager on -> scanStart
Sun, 24 Jun 2018 19:28:25 GMT bluetooth-manager Sun Jun 24 2018 15:28:25 GMT-0400 (EDT): peripheral: Dexcom1F with rssi -72
Sun, 24 Jun 2018 19:28:25 GMT bluetooth-manager on -> scanStop
Sun, 24 Jun 2018 19:28:25 GMT bluetooth-manager on -> connect
Sun, 24 Jun 2018 19:28:25 GMT bluetooth-manager on -> peripheral services discovered
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager on -> service characteristics discovered
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager Tx 01406b5f2a0e1f4f7002
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager Rx 030b05b5db0c815b22454f9b62b02bcb79
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager Tx 04cf4374cc9545ab9b
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager Rx 050102
Sun, 24 Jun 2018 19:28:26 GMT transmitter transmitter responded with status message 050102
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager Tx 0619
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager Tx 07
Sun, 24 Jun 2018 19:28:26 GMT bluetooth-manager setting notify to true
Sun, 24 Jun 2018 19:28:26 GMT smp got data 0b01
Sun, 24 Jun 2018 19:28:26 GMT smp got security request
Sun, 24 Jun 2018 19:28:28 GMT bluetooth-manager disconnected peripheral
Sun, 24 Jun 2018 19:28:28 GMT bluetooth-manager scanning again in 1 minute

06/24 15:28:28 after xdrip-js bg record below ...
cat: /root/myopenaps/monitor/logger/entry.json: No such file or directory

cat: /root/myopenaps/monitor/logger/entry.json: No such file or directory
06/24 15:28:28 Exit - Invalid response from g5 transmitter
ls: cannot access '/root/myopenaps/monitor/logger/entry.json': No such file or directory
rm: cannot remove '/root/myopenaps/monitor/logger/entry.json': No such file or directory
06/24 15:28:28 Removing existing Dexcom bluetooth connection = Dexcom1F

entry.json doesn't get created presumably because the data it got was bad, but the receiver continues to show good and valid readings. How can I get more information on what data it's getting back so I can try to diagnose what it doesn't like about it?

Rebooting in 15 seconds to fix BT and xdrip-js

after activating Logger and BT tethering, I received the following error and the PI will restart. it happened several times a day!
"Broadcast message from root@dorrarig1 (somewhere) (Tue May 28 03:31:34 2019):
Rebooting in 15 seconds to fix BT and xdrip-js - save your work quickly!"

any idea?

Feature request: multiple calibrations in one cycle.

One feature/improvement needed to Logger is the ability to send multiple calibrations in one wake up cycle. This avoids waiting 10 minutes on initial sensor calibrations.

Also when something fails in the logger loop, on the cycle where the calibration was supposed to be sent, then we currently need to do another blood finger prick. Instead, we should allow slightly backdated calibrations. I think up to 20-30 minutes seems reasonable.

Auto Restarts

Support automatic restarts. Raw notes / requirements copied from gitter to here:

  • Restarts should be on day 7 for g5 and day 10 for g6.
  • Don't waste 3 days of a coded no-cal start.
  • Restarts should not use a code to restart.
  • Restarts should not happen while a user is in the process of starting a new sensor.
  • So, if you stop the sensor (or issue any NS or command line driven cgm control - start/stop/reset/ then auto-restart will be delayed by 3 hours.
  • Auto-restarts would be to restart immediately (w/o a code) within 30 minutes of when the transmitter reports it will reach the end of the session and a new sensor needs to be started. when the transmitter reports it needs calibration, it would automatically send a synthetic calibration based on the current unfiltered glucose reading and the LSR calibrations from before the last session of the same sensor ended.
  • There will need to be a minimum of 3 LSR-based calibration records for autostart to work.
  • if a new sensor is inserted, the user can simply start a new session with a code, and that wouldn't trigger any of the auto-restart or automatic first calibration behavior.
  • Question: Will auto restart be done by Logger or by xdrip-js?
  • The restart at 30 minutes before the end of a session will backdate the start/stop messages by 2 hours, thereby avoiding the 2 hour warmup period. A similar implementation has been done before (iOS in CGMBLEKit) and has been used sucessfully.
  • then, once the native algorithm has been restarted with a single calibration, leave it up to the user to enter real calibrations to get it to re-learn the slope and intercept. Until that happens, Logger will continue to use the last 8 calibrations (mostly from the last session on the same sensor) to calculate glucose (the current expired behavior).
  • Pick up real calibrations as they're entered, and add them to the LSR data, such that the LSR and native algorithms would gradually converge over the course of multiple calibrations.
  • the trick is to send both the stop and start commands during an active session.
  • Question, do we need a restart message in xdrip-js that does both the start/stop in the same 5 minute wake-up session?
  • the only potential downside to auto-restarts would be in the scenario where the user is pulling BGs into NS with the Share bridge, and the sensor session has an abnormal slope/intercept, such that the native algorithm's single point calibration becomes highly inaccurate when BG diverges significantly from the point at which it was calibrated. that's not an issue for Logger-uploaded BGs, as long as we use the expired behavior, because we have the full LSR data from the last sensor session. To mitigate that, Logger could refuse to calibrate an auto-restarted sensor if the slope/intercept is too abnormal, thereby forcing the loop to run off of Logger data until the user calibrates manually. Or do something smarter like wait to do the first calibration until BG is at the low or high end of the target range, depending on whether the slope/intercept is way higher or lower than normal, such that we ensure that if we end up looping on the incorrect single point calibrated data, we always err on the side of too little insulin rather than too much.
  • Another potential issue with auto-restarts is that I vaguely remember a discussion where it was suggested that Dexcom expects the raw readings to continue to settle down over a longer period than 2 hours - so they apply some kind of exponential correction for a number of hours afterwards. Since in our case we don't really have a new sensor that would result in a inaccurate reported drift. The Dexcom readings on a newly restarted sensor drift low (compared to BG actually being higher) over the first 2 days. Yes, very clearly shows up when doing staggered multiple receiver restarts on a G4. The correction is the problem. The correction is applied to handle the normal case of an actually new sensor bedding into tissue. When the sensor is actually 7d old, the corrections cause falsely low readings. The easiest fix is just to calibrate twice a day when it asks you to. If you fail to do that, you get slightly less insulin than you'd otherwise need, and BG runs a bit high. not a safety concern. There is one mitigation that we'd be doing: using LSR-based expired mode for Logger-reported glucose readings until we had a few calibrations.
  • stop/start/calibrate will all be done in one 5 minute interval. The sensor immediately goes into 2nd Calibration.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.