markmpn / autotuneweb Goto Github PK

This is an incredible tool. Thank you so much for simplifying this process. Would it be possible to automatically run autotune every week (or alternate range of time) so I could have an email in my inbox with my results every Sunday for instance?

Hello Mark,

In OpenAPS I have created some code for improving the error message on Autotune. This, because sometimes I mention Facebook help requests because of Autotune error message. See openaps/oref0#1404 for code and background information. It is approved by Scott Leibrand and merged into dev. Perhaps this is something to include in AutotuneWeb? If so, users will see a improved error message when there are no BG-values for selected dates. Love to hear.

All the best,
Peter

This morning I have submitted my information but have not received a mail with my result. When I resubmit, it says that I am number 8 in the queue....
Never had this issue before , is there a way to solve this?

A few days ago there was some error while starting a new report und it doesn't seem to resolve on its own. If you need the exact URL, please let me know.

I tried site https://autotuneweb.azurewebsites.net/ yesterday again since bout a year. The autotune job keeps running but never finishes.

Hope you can help or provide an alternative (because I do not have one; e.g. I also tried running autotune on WSL Ubuntu but failed during installation).

I'm getting the email back with this error and it sends me the logs. The error message in the logs is :

Could not parse input data: SyntaxError: Unexpected end of JSON input
at JSON.parse ()
at Object. (/usr/src/oref0/bin/oref0-autotune-prep.js:63:37)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
oref0-autotune-core autotune.2024-03-21.json profile.json profile.pump.json > newprofile.2024-03-21.json
/usr/src/oref0/lib/autotune/index.js:151
CRData.forEach(function(CRDatum) {
^

TypeError: Cannot read property 'forEach' of undefined
at tuneAllTheThings (/usr/src/oref0/lib/autotune/index.js:151:12)
at Object. (/usr/src/oref0/bin/oref0-autotune-core.js:67:27)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:188:16)
at bootstrap_node.js:609:3
false
Warning: API_SECRET is not set when calling oref0-autotune.sh
(this is only a problem if you have locked down read-only access to your NS).
Could not run oref0-autotune-core autotune.2024-03-21.json profile.json profile.pump.json

Currently, in order to use autotuneweb, it's necessary to be able to pull data from NightScout for it to run.

Many non-looping users store pump and CGM data in Diasend, and would be potentially better served by being able to simply upload data to autotuneweb from Diasend. There are currently tools available to upload a Diasend .csv file to NightScout. Potentially these could be repurposed for use with autotuneweb to allow the same file to be uploaded and run through the functionality here.

NightscoutLoader: https://github.com/gh-davidr/NightscoutLoader

Equivalent enhancement request for NightScoutLoader: gh-davidr/NightscoutLoader#12 (comment)

Could the dataspan be increased beyond 30 days ?
I appreciate that heroku costs would increase as this would increase the load, however you could do 1 to 30 days free as it currently is is and and 30 days plus as a paid service for a small price which would cover the cost ? Just a thought, I love autotune web and run it regularly but a 90day average would be great to rule out the anomalies that an off week can bring in a 30-day period.

Look forward to hearing your thoughts

After entering my URL and hitting "Start", this error message gets displayed:
Unexpected character encountered while parsing value: E. Path '', line 0, position 0.

This is my URL:
https://host.ns.10be.de:1234/

I have been using AutotuneWeb with this host frequently before, so this is new.

Hi!

I just tried getting AutotuneWeb to run with my Heroku hosted Nightscout but for some reason i constantly got "403 (Forbidden)" and AutotuneWeb wouldn't be able to grab the profile from the Nightscout API.
After a while of troubleshooting, i actually noticed that cloudflare, which i am routing my web based Nightscout traffic through, seemed to log unusually many 'Bot fight mode' firewall events. It looks like cloudflare's firewall (specifically the 'Bots' module under "Security" -> "Bots" to be exact) rejects all of AutotuneWeb's requests towards the Nightscout API, therefore causing the 403.

Now, I am aware this is not an issue with AutotuneWeb in and of itself. If anyone is to blame, it is me for using cloudflare CDN and failing to recognize it as the culprit. But i hereby suggest extending the FAQ by mentioning the possibility of "Foul play" by intermediate services like cloudflare's bot protection. Especially because I never explicitly told cloudflare to block requests from AutotuneWeb. It just automatically classified the request as coming from a bad bot, therefore being fraudulent.

This may or may not also apply to other CDN's, I'm not sure.

OpenAPS has documentation that list this option but warns it is experimental. Getting accurate prediction data, especially for people using your tool meaning they aren't on a closed loop, is very difficult. Insulin absorption rate has a large impact on predictions provided.

Can this be added as an optional flag to include in the output just like you have UAM? Both are optional CLI flags. Thank you for the consideration.

Hi, AutotuneWeb is amazing! I'm a Loop user and using the Nightscout on HTTP protocol. But AutotuneWeb project only support HTTPS protocol. Is it possible to support HTTP? Thanks!

Hi Mark every time I try to run auto tune it sends be an email saying an error occurred??
https://hockeymom3.herokuapp.com/

Hello,

I've seen that you added Fiasp. It would be great if you could add Lyumjev, too!

Here is the data that AndroidAPS uses:
"Lyumjev: Peak 45min (+/- 5min) and DIA 5h (5h-7h or 4h-6h)"

Source: nightscout/AndroidAPS#269

Does this help? nightscout/AndroidAPS@05125af

Love the website, thanks for all the work!

Hello,

I have been using autotuneweb successfully in the past (AAPS user). But recently and also today I only receive an error message:
An error occured while processing your request.

Could someone help me please? :)

I don't want my nightscout to be open to the world, so it would be great if you could support the authentication tokens that nightscout can generate and use.

I run Nightscout on heroku with the

AUTH_DEFAULT_ROLES denied

setting to keep the site private.

I created a readonly user (roles 'Readable') to allow AutotuneWeb to access the data. However when I supply a tested and valid URL+token to https://autotuneweb.azurewebsites.net,

The remote server returned an error: (401) Unauthorized

is returned. I wondered if the app is stripping off the auth token?

Using aaps 3.0 rc3 and trying to run autotune web but it constantly fetches the wrong profile! Instead of the current profile, it tries to run on a temporary profile (125%) that an automation has created a while ago even though aaps is back on normal profile.

I tried to save current profile again in NS but that didn't make a difference... I usually don't touch the NS profiles since aaps is only run on locale profiles which it only uploads to NS for display purposes.
Is there a way of tricking autotune to select correct profile, or is this a "bug" in aaps?

Thanks for an amazing tool! One of the issues that I've faced is that the results often don't match a manual autotune. Part of the reason, I think, is that the date ranges of the data used don't necessarily match because:

a) autotuneweb's UI doesn't indicate the date range that is used, so it's not clear to the user how to generate matching data via a manual autotune. This can be fixed with some additional messaging on the input screens. e.g. 'Autotune web will analyze this along with your data from <dd/mm/yyyy> until <dd/mm/yyyy> (inclusive).'

b) autotuneweb doesn't allow the user to set a date range. Fixing this is preferable to the above approach because in some cases the user will want to choose to set a range to analyze for which the data is relatively 'clean' (e.g. so as to exclude days when a child forgets to make carb entries OR to exclude days with intense exercise that would skew the analysis).

e.g. something like:
Choose date range: dd/mm/yyyy to dd/mm/yyyy

(with default values preset to past 1/2 full weeks)

Thanks!

Line should be something like:

setTimeout(() => window.location.reload(), 10000);

to avoid "Uncaught TypeError: Illegal invocation" in some modern browsers like Edge.

I have submitted my information and have not received an email with my results and cannot resubmit. It has been running for 2 days. I have never had an issue prior to this.

That code assumed the unit name is "mmol". I am using @Kdisimone's Loop "anna" branch that automatically uploads profiles to nightscout. If I access mysite/api/v1/profile/current, my unit name is instead "mmol/L". this results in autotuneweb running with an isf that is not multiplied with 18 even though is should.

Suggested fix:

 private decimal ToMgDl(decimal value)
        {
            if (this.Units.ToLower().Contains("mmol"))
                return value * 18;

            return value;
        }

Update Autotune image to latest dev for Omnipod users

Hello,
I’ve used AutotuneWeb regularly in the past few weeks. Today it keeps returning errors. https://autotuneweb.azurewebsites.net/Home/RunJob

I get this error every time I try to run the webapp, not sure what other details I should give but hopefully the logs will be helpful enough :) Thanks for your help!

Compressing old json and log files to save space...
gzip: ns-*.json: No such file or directory
gzip: autotune*.json: No such file or directory
Autotune disk usage:
16K	.
Overall disk used/avail:
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdb1        69G   53M   65G   1% /mnt
Grabbing NIGHTSCOUT treatments.json and entries/sgv.json for date range...
Query: https://ido-cgm.herokuapp.com entries/sgv.json find%5Bdate%5D%5B%24gte%5D=1608170400000&find%5Bdate%5D%5B%24lte%5D=1608256800000&count=1500
-rw-r--r-- 1 _azbatch _azbatchgrp 3 Dec 24 13:05 ns-entries.2020-12-17.json
Query: https://ido-cgm.herokuapp.com treatments.json find%5Bcreated_at%5D%5B%24gte%5D=2020-12-16T06:00+02:00&find%5Bcreated_at%5D%5B%24lte%5D=2020-12-18T18:00+02:00
-rw-r--r-- 1 _azbatch _azbatchgrp 3 Dec 24 13:05 ns-treatments.2020-12-17.json
oref0-autotune-prep   ns-treatments.2020-12-17.json profile.json ns-entries.2020-12-17.json profile.pump.json > autotune.2020-12-17.json
undefined:1
undefined
^

SyntaxError: Unexpected token u in JSON at position 0
    at JSON.parse (<anonymous>)
    at categorizeBGDatums (/usr/src/oref0/lib/autotune-prep/categorize.js:73:28)
    at generate (/usr/src/oref0/lib/autotune-prep/index.js:23:30)
    at Object.<anonymous> (/usr/src/oref0/bin/oref0-autotune-prep.js:131:27)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
Warning: API_SECRET is not set when calling oref0-autotune.sh
(this is only a problem if you have locked down read-only access to your NS).
Could not run oref0-autotune-prep ns-treatments.2020-12-17.json profile.json ns-entries.2020-12-17.json

I'm noticing an issue where the Autotune Web results that are e-mailed do not have the correct basal rates for the evening. In the screenshots attached, you'll notice that the 8PM basal rate change is not reflected but the 11PM change is.

This could be user error but I believe everything is set up correctly. See the attached images showing the basal rates in Nightscout and the basal rates in the e-mail.

For additional background we are using the Jojo-Beeps branch of Loop that automatically updates our Nightscout profile when basal changes are made within the Loop app.

Happy to provide more context if helpful. I did try changing the basal rates in Nightscout manually this morning (resetting them to match the correct basal profile in Loop) but that change also did not accurately reflect in the Autotune e-mail.

Please let me know if you think this is software related or user related.

Thsi tool is really good. I am using http to access the nightscout website, it is hard for me to use https, can you help me support http? Thank you,

Hi Mark, submitted autotune request this morning and I've not received the results. Thought I'd try again and I get this message, "Your job is already running. It started at 09:17, and has been running for 748 minutes.
Please be patient and you will get your results by email as soon as it is complete."
Is there a problem with it at the moment?

Currently autotune expects profile data in a different format than nightscout returns. How are you generating the profile for autotune to use?

I am trying to start up the application on an Ubuntu box. I have created the SQL Server table and had to add the extra column for DaysDuration.

I thought it would work, but I am still getting this error:

Unhandled Exception: System.Data.SqlClient.SqlException: Invalid column name 'TimeZone'.

Is there another column that I need to add to the Job table?

Can you add a policy or information to the site about how emails and NS URL are used, and stored, if at all?

Would it be possible to be able to bypass the FAQ requirement listed at https://autotuneweb.azurewebsites.net/Home/About which says Nightscout must be set with readable in AUTH_DEFAULT_ROLES with a read access token as part of the Nightscout URL like ?token=report-my_token_id?

Some of us have AUTH_DEFAULT_ROLES set to denied in order to save bandwidth on the free usage tier of cloud providers.
Thanks for all the work you do to provide this tool on the web :)

Hi, it would be helpful if we could update the NS profile from the results, in an semi automated way.

Most common user error I've seen this week (up to more than a dozen) has been people getting an error because they're running with http in their URL instead of https. I'm wondering if we can add a check on the URL before it goes off to run a job, and if it fails with http, retry with https?

I am using NSClient2 2.0 Build 4f0f947ed-2018.11.03-00:28 and NightScout 0.10.3-master-20180805 I am still getting the message about temp basal entries in NightScout not having the "rate" property. have aslo tried AndroidAPS 2.0 master and had the same problem. Any ideas?

I tested and found that I get exactly the same result either with or without UAM categorized as basal. Could this be the truth? I'm announcing all of my meals (I hope) but the same result down to the 3rd decimal place?
But thank you and Dana for this great work; all of this will result in more and more acceptance at our doctors.

Thanks for this wonderful tool.

Getting some weird ISF recommendations. For every iteration it wants to increase it. I suspect this is related to mmol/L - mg/dL conversion.

Our current ISF is 9 mmol/L.

From the profile generated by autotuneweb:
"sensitivities": [
{
"i": 0,
"start": "00:00:00",
"sensitivity": 162.0,

From the log file after running:
Limiting adjusted ISF of 159.28 to 12.86 (which is pump ISF of 9 / 0.7 )
p50deviation: -1.775 p50BGI -0.14 p50ratios: 12.649 Old ISF: 12.592 fullNewISF: 159.276 adjustedISF: 12.857 newISF: 12.645 newDIA: 6 newPeak: 75

Resulting in this recommendation:
Parameter | Pump | Autotune | Days Missing

ISF [mg/dL/U] | 9.000 | 12.645 |

Hey Mark,

Super thanks for all your effort and for sharing your work!!

I have been running into a problem. I don't get the same results when I run autotune manually from my computer:

I copied my profile.json to openaps/settings/autotune.json, openaps/settings/pumpprofile.json. and ran:
oref0-autotune-prep myopenaps/autotune/ns-treatments.json myopenaps/autotune/profile.json myopenaps/autotune/ns-entries.{date}.json myopenaps/autotune/profile.json --tune-insulin-curve=true=true --categorize_uam_as_basal=true

I use as startdate "2021-12-30" and as enddate "2022-01-04" (the day of yesterday to synchronize with autotuneWeb) and I get these results:
00:00 | 0.360 | 0.432 | 2
01:00 | 0.360 | 0.432 | 2
02:00 | 0.360 | 0.432 | 2
03:00 | 0.360 | 0.252 | 2
04:00 | 0.360 | 0.252 | 2
05:00 | 0.360 | 0.252 | 2
06:00 | 0.360 | 0.432 | 2
07:00 | 0.380 | 0.456 | 2
08:00 | 0.380 | 0.266 | 2
09:00 | 0.380 | 0.266 | 2
10:00 | 0.380 | 0.456 | 2
11:00 | 0.380 | 0.266 | 2
12:00 | 0.320 | 0.384 | 2
13:00 | 0.290 | 0.348 | 2
14:00 | 0.280 | 0.336 | 2
15:00 | 0.340 | 0.238 | 2
16:00 | 0.360 | 0.432 | 2
17:00 | 0.300 | 0.360 | 2
18:00 | 0.320 | 0.384 | 2
19:00 | 0.320 | 0.384 | 2
20:00 | 0.380 | 0.456 | 2
21:00 | 0.350 | 0.245 | 2
22:00 | 0.330 | 0.231 | 2
23:00 | 0.330 | 0.231 | 2

When I run autotuneWeb on the azure platform I get these results, which seem better

"Based on data entered between 2021-12-30 and 2022-01-04":

Parameter | Pump | Autotune | Days Missing

ISF [mg/dL/U] | 45.000 | 49.923 |
Carb Ratio[g/U]| 16.000 | 14.852 |
00:00 | 0.360 | 0.432 | 1
01:00 | 0.360 | 0.432 | 1
02:00 | 0.360 | 0.432 | 1
03:00 | 0.360 | 0.432 | 1
04:00 | 0.360 | 0.432 | 1
05:00 | 0.360 | 0.432 | 1
06:00 | 0.360 | 0.434 | 2
07:00 | 0.380 | 0.454 | 2
08:00 | 0.380 | 0.454 | 2
09:00 | 0.380 | 0.456 | 1
10:00 | 0.380 | 0.436 | 0
11:00 | 0.380 | 0.369 | 1
12:00 | 0.320 | 0.286 | 0
13:00 | 0.290 | 0.241 | 1
14:00 | 0.280 | 0.259 | 1
15:00 | 0.340 | 0.291 | 0
16:00 | 0.360 | 0.330 | 0
17:00 | 0.300 | 0.280 | 1
18:00 | 0.320 | 0.320 | 3
19:00 | 0.320 | 0.320 | 3
20:00 | 0.380 | 0.358 | 1
21:00 | 0.350 | 0.387 | 1
22:00 | 0.330 | 0.396 | 1
23:00 | 0.330 | 0.396 | 1

Do you have an idea what I'm doing wrong?

Hello my autotune is stuck in running I hope you can help me out

Your job is already running. It started at 08:49 UTC, and has been running for 438 minutes.

Please be patient and you will get your results by email as soon as it is complete.

93bc42435d4dec9fb86a061deb9eb6e649e284b2-b1121879-55b0-47d4-bd7f-cd8b4d39d9b7

When I enter my Nightscout URL I get the following message:
The requested name is valid, but no data of the requested type was found.
What could it be?

I would be very happy about an answer.