rcpch / digital-growth-charts-server Goto Github PK

On our Windows builds, the Instructions page errors with:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

the Traceback was:
127.0.0.1 - - [08/Jul/2020 12:01:44] "?[35m?[1mGET /instructions HTTP/1.1?[0m" 500 -
Traceback (most recent call last):
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\json\decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
StopIteration: 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 2464, in call
return self.wsgi_app(environ, start_response)
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 2450, in wsgi_app
response = self.handle_exception(e)
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 1867, in handle_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask_compat.py", line 39, in reraise
raise value
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask_cors\extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask_compat.py", line 39, in reraise
raise value
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\flask\app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "C:\Users\andrewp\Documents\GC\digital-growth-charts-flask-client\app.py", line 150, in instructions
return render_template("instructions.html", fill=html.json())
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\json_init_.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\andrewp\AppData\Local\Programs\Python\Python38-32\Lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Implement internationalisation and localisation of advice strings to be available in different languages. Any subsequent change in the text, or added support for new languages would then changes in only one place.

PR incoming

do we need a test suit covering the client?

• Testing the Flask app with quick unit tests designed to pick out regressions quickly when developing
• Testing the API responses as a full round trip
• python hypothesis library which can be used to spot-check a large 'solution space' rather than having to brute-force cover every possible input

We need to explain the decisions we have made where RCPCHGrowth deviates from LMSGrowth. I have started a paragraph in developer-documentation in the docs folder.

• Andrew and Marcus need access to control and configure this
• There should be an API management system (we are probably going to use Amazon's own 'API Gateway' product, but if this disappoints then there are a number of other options.
• The 'front end' of the API management platform is accessible from the Internet.
• The 'back end' of the API management platform should connect to a private internal network (which is not accessible to the Internet)
• Inside the Private Network we can set up 'serverless' infrastructure which runs the dGC API software.

https://github.com/rcpch/digital-growth-charts/blob/fa8ad104925e56a7be3ff31d300a2902de157bc6/rcpchgrowth/rcpchgrowth/date_calculations.py#L173

1. datetime.strptime(convert_string, '%d/%m/%Y') in our code is nearly as short as string_to_date(convert_string), while the former is more standard Python and clearer to the reader how the date is going to be parsed.

2. In the API I am using ISO8601 dates, which will arrive in the form 'YYYY-MM-DD' (note the hyphen separator, and the order). It would be good to work towards using just one string format for dates, for maximum consistency

https://github.com/rcpch/digital-growth-charts/blob/8b3e68c964a5e393438ab636df785bb9ff43ea5f/app.py#L14

We can safely leave a default key in the codebase for development purposes.

But for production this configuration variable needs to be changed and moved out of the GitHub committed code. This SO answer gives a good description of how it should be done, with environment variables.

In response to feedback from the Project Board at the Teams meting Monday 13/07/2020:

The project board felt that we should not be sending advice strings for single measurements in any circumstances, therefore this feature will be removed.

Instead there will be 'generic' advice which the Project Board will create, which will be along the lines of: "Single measurements on growth charts and in centile form are hard to interpret, so growth charts and centiles should always be interpreted as part of the trend over time. If unsure then see your GP"

@magdaumerska can I ask you to ensure the Project Team come up with a form of words for this, that they are happy with, as part of their review of the existing documentation. I wouldn't be surprised if the paper charts already have something along these lines in place, which can be reviewed and repurposed. Once we have the text we can insert it into the API.

It would be useful to automate a process which creates serial data points that mimic the growth of a child for testing purposes

Child centile results are reported with a string value interpretation for the user, one as a lay message to guide families on interpretation of the centile result, while the clinician message assumes some medical knowledge and training and may make referral recommendations. Are these appropriate in each case?

We need to decide which cloud platform provider we are going to use. There are a lot of options, however two that the College already use are: Microsoft Azure and Amazon Web Services.

Both are advanced platforms and will have all the features we need.
Both are reliable.

In my opinion the key feature is being able to get into the admin panel ourselves and make changes. I think @andrewpalmeruk mentioned that only one of these we have direct access to (Azure??) and the other we would have to ask a third party to do any changes for us (AWS??).

I'm very much in favour of having direct access because even when things are working well communication-wise, this introduces a significant delay in getting changes made. And if comms are slow, delayed, or people are away from keyboard or on holiday, then everything stops.

aim for an initial 10 unit tests for the API responses, based on hitting a growth reference point excatly so there is no interpolation required.

Assigned to @pacharanero

These need refactoring and changing to measurement_method (only an enum ['height', 'weight', 'bmi', 'ofc']) on the client side

There should be a disclaimer which makes it clear that the API is in alpha and cannot be used for clinical purposes

Once discussions have concluded regarding the exact license to use we should have a license file in the source code. I am happy to do this once we are decided on which OSI-approved license we are using.

Refactor API to use measurement_type

Update README.md into small chapters that will make it easier to use

We should consider how we are going to approach internationalisation
Internationalisation allows us to vary the advice text that is returned by the API
This would be necessary for international users who need the text in their preferred language.

Additionally it may provide a mechanism to customise the returned text for specific non-International users, such as Devolved nations of the UK or other regions who require different return values.

Thank you to @statist7 for identifying this issue. Although the default_to_youngest_reference flag has appropriately been set to false, it is still selecting the younger reference.

Should we be using gender or sex against measurement and dates?

Thank you to @statist7 for identifying this issue.

Currently the SDS calculation for all term babies > 37 < 42 weeks are assumed to be 40 weeks. This is incorrect and needs to use the averaged LMS values from the reference data

Andrew and Joanne set up Windows Dev Envs
Edit and update README if any windows issues noted
Submit PR

• There should be a CONTRIBUTING.md file in the root of the project, or in the documentation folder, containing:
• details of how to discuss potential improvements and design their implementation
• how to develop the improvements, coding style, etc
• submission instructions (pull request)
• details of contributor covenant

We require a list of validated test identities to send to the API and check the return values are exactly what the validated response should be. This is important for internal testing during development, and also as a resource for implementers.

@eatyourpeas has a CSV file containing ~1000 rows of synthetic test data for a single fictional individual over the course of the 0-1 year age range of the MVP, which would be a good starting point.

@statist7 - if @eatyourpeas provides you with the file, would you be able to create validated SDS and centile data for the rows?

(As a side-note, I would like to get test data such as this into machine-readable form, in GitHub and under version control as soon as possible, possibly in the API repository under a subfolder like /test-data)

This seems harder than it should be. I am always impressed by how well Amazon make things hard to do and, rather than driving users away to other products, it's seems to have conversely created for them a 'specialist skill area' of AWS experts.

On Heroku (an independent platform which does serverless deployment) deployment is as simple as git push heroku master. And you can rig it to update the deployment automatically when you push new code to your repo. However on AWS Lambda there seems to be no hook to 'pull' code in from a repository, it seems to want a ZIP file upload (really?) or for you to edit the code directly in-browser (no good for an open source project or a codebase as large as ours)

There is a tool that GitHub themselves produce as part of their Actions which might help here:

https://github.com/marketplace/actions/aws-lambda-deploy

needs to be looked at and fixed assigning @pacharanero

A bit too perky for my liking. Need to find out where to change this text in the API Management platform and change it.

Postman is a tool for interacting with APIs, and can be used within our team for testing. It can also be used by implementers as a quick way to interact with our API and to learn how it works.

We can make all of this easier by distributing a Postman configuration file (known as a 'Collection') which contains all the setup needed to interact with the API.

The UK-WHO reference was constructed using the LMS method, which defines the centiles in terms of the median M, coefficient of variation S and skewness L at each age, and summarised as a table with the LMS values in three columns at a series of ages in the rows.

The LMS method is a special case of GAMLSS (generalised models for location, scale and shape), and charts are now being created using more complex GAMLSS models that involve four not three columns of data, where the extra column adjusts for kurtosis (heaviness of tails).

If the longer term plan is to internationalise the API, we should recognise that this would be a useful extension. It is not too difficult to implement, involving four look-up columns rather than three and a slightly more complex z-score to centile conversion, but I think it's worth considering.

This feature is not an MVP feature. But there are standard reference data for growth in Down's Syndrome (Trisomy 21), which we could add to the project in the future. Steps required to do this properly would be:

• obtain accurate Down's Syndrome growth references data from the official source

• convert this from the format it will come in (likely CSV or Excel) into our standard format for LMS growth reference data

• add this reference data to the main references repository (currently this is in the Growth Chart API Server repo, but I'm expecting to move it out into its own repo in the future.

• create a new API endpoint at something like api/v1/downs-calculations and configure this in the server. Much of the standard centile code will be able to be reused for this, but it may require custom handling hence being at a separate endpoint. Using another endpoint also completely removes the possibility of inadvertently requesting Down's references for a non-Down's patient. (for example, if we handled this feature through the main /calculations/ endpoint and had an additional query string parameter for the reference selection - eventually someone would manage to do it wrong)

• document the new endpoint in the OpenAPI specification in the server repo root

• add a view to the Client(s) flask to demonstrate this new feature (likely to be possible to reuse much of the other views)

At some stage we are likely to need a discussion channel for implementers (developers, companies, teams who are using the API) to be able to contact the team for support as they develop their client applications.

There are a number of considerations to this, which we should be thinking about early, so that we are ready. I'd appreciate your thoughts on this @eatyourpeas @andrewpalmeruk @magdaumerska

Resource requirements for support

• Answering support queries does require some resource (time mainly, on the part of whoever answers the query).
• Initially, the Project Team may be able to answer some of these queries as part of their routine work, however there may need to be some resource allocation for this in the longer term.
• Q: Should support be free for the implementers? (My preference would be yes - we want to egt people up and running as soon as possible, not make it hard for them)
• Q: If free, is there a point at which would we need to limit this to avoid overwhelming the team? And how should that be done.
• Q: Is there also some kind of 'premium' or tiered commercial support offering here which the College could provide? A clinical and technical advice service, perhaps even help with validation and testing of your interface, etc? Video advice and support calls?

Technical options for the actual channel

• We could use an existing free chat tool for support. This is cheap, easy, and requires essentially no maintenance as a platform. Examples of these which are frequently used for software support channels include:

  • Gitter - integrates well with GitHub repositories, less well-known
  • Telegram - works a bit like WhatsApp, can create groups, join with a link, very open.
  • Slack - feature rich, more difficult to manage groups, need a 'new-user-adder' app to enable onboarding.
  • Keybase - obscure but very good and cross-platform.

• We could also set up something more comprehensive than a chat platform, such as a web forum. This has the advantage of being under our direct control and can be themed in RCPCH colours and branded with the logo. It also allows us to have a number of related channels covering different APIs, if we develop more of these in the future. I have considerable experience hosting and operating the Discourse forum platform. My preferred option would be a Discourse web forum.

See #24 - As per @statist7 comment: It is desirable to implement calculation of conditional velocity centiles which are age specific and guide on an individual child's speed of growth.

@statist7 suggests that while the unit testing should match LMSGrowth to 10^-5, the values returned to the end user should be round to 2 decimal places. For the purposes of testing it is easier for the moment not to make this change.

This feature most likely of interest only to growth chart builders/enthusiasts but would allow the researcher to view a cohort of children as a scatter plot against the centiles as a backdrop.

This is currently not possible as the create_centiles function receives 2 parameters (sex: str, born_preterm: bool)
These are both flags which are used to select the correct references to return against a single child.

To implement this the unique_child flag (which already exists in the calling function) would become a new parameter which would default to the whole preterm dataset if true.

In future the calculation / calculations endpoint will have less meaning if different datasets are used. This should be refactored now to reflect UK-WHO

FHIR ('Fast Healthcare Interoperability Resources', pronounced 'Fire') is a standard interchange format for medical data. It has been suggested that the response from the Growth Chart API should be in the form of a FHIR Resource.

More information on FHIR can be found here: https://en.wikipedia.org/wiki/Fast_Healthcare_Interoperability_Resources

The FHIR specifications are here: http://hl7.org/fhir/

At present the data returned by the API is in a structured data format called JSON (JavaScript Object Notation) and this is also the most common way that FHIR resources are transmitted. From initial discussions between myself and @eatyourpeas we think it is likely that we would be able to return a FHIR Resource as the API response.

• create a repo for the new client (existing repo will be renamed for the server at the end of the process)
• ensure functions for MVP are fully exposed via REST API endpoints (ie not being accessed via internal code)
• test API endpoints via Postman
• build new Flask client
• gradually remove all Client functions over to the new client
• delete those client functions from the Flask app

The API is designed to accept dates and measurements and convert them to z-scores. This represents the typical clinical scenario, where a child has been measured and needs to be assessed.

However there is also a need to do the reverse conversion, from z-score to measurement. The most obvious example of this is to obtain the values need to draw the chart centiles, where each centile curve corresponds to a particular z-score. This might be viewed as more a research than a clinical requirement, and hence non-MVP, but I'm not so sure.

We have already discussed returning the measurements corresponding to specified centiles, to be used to draw the chart. I argue that we could generalise this by allowing the request to include a set of centiles (and corresponding set of ages) for which the corresponding measurements would be returned.

For the MVP the z-scores could default to the nine-centile format and the age range to the first year, but it would be worth keeping the spec general for non-MVP.

Calculate growth velocity expressed as a conditional velocity centile using TC correlation matrices

Babies born preterm should be plotted on UK90 preterm data up until 2 weeks of age. Babies born at term should not be plotted on those data, instead from 37-42 weeks there should be no centiles, and the WHO 2006 data should pick up at this point.

At this stage, and with such a simple API structure, we probably don't need to worry about GraphQL, however in the future it may be something to consider adding if it is necessary

A client library is a piece of software that we produce, as the creators of the API which we distribute to consumers of the API. The library allows them to integrate with the API more easily, by providing utility functions in a language that the developer is familiar with.

Many APIs in the wider tech world have 'official' client libraries available in the major common languages - such as JavaScript, Java, Python, Ruby, Go, Kotlin, Swift, etc. This aids adoption of the API by end users.

Initially I thought this would be something we should produce for our API consumers. However, the more I think about this, the more I'm actually thinking we may not actually need client libraries because the type of consumers we are expecting to have are going to be integrating the API into a wide range of primarily existing clinical software.

Also, our API is relatively simple for consumers to use, and a client library may simply not be necessary.

• One thing we could do is ask consumers who develop code for our API in a particular language to 'donate' their client code to the project.

• If we are involved in supporting a consumer to integrate with the API, then we could consider using that as an opportunity to develop a client library in the requisite language.

The convention for British growth charts is to use the nine-centile format that I proposed in the BMJ (1994), extending from the 0.4th to the 99.6th and defined by a set of z-scores spaced two-thirds of a unit apart. So the nine centiles, 0.4, 2, 9, 25, 50, 75, 91, 98, 99.6, are (except for the 50th) rounded for presentation, and their position on the chart should be defined by the underlying z-score (compactly defined as -4:4*2/3).

Other countries tend to define their chart centiles as whole centiles rather than underlying z-scores, e.g. selecting from the set 1, 3, 5, 10, 25, 50, 75, 90, 95, 97, 99 (an exception is WHO 2006 which uses whole z-scores). For the purposes of internationalization we should make this choice of centiles easy to set up, by defining it as a global preference.

@eatyourpeas I've labelled this mvp as the code change is minor but important.

• We have 2 main repositories for the dGC code:

https://github.com/rcpch/digital-growth-charts-server
https://github.com/rcpch/digital-growth-charts-flask-client

• At the moment these are both Private repositories

• We had, in initial discussions, agreed that we would open source them, and this is really fundamental to the 'trustability' of our code and our API - that anyone can look at and comment on it, improve and develop it. (Note: even when open sourced, ONLY the dGC Developer Team can make changes to our GitHub repository code or to the running code on the API server.)

• I would like to set a date for when we will open the repositories up to the public, as I feel that it's important we do this sooner rather than later.

• I would therefore like to propose the date of 1st August 2020 for <OPEN SOURCE DAY> This is an auspicious date as it is Yorkshire Day 🍾