sjoerdk / pimsclient Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
pimsclient version: 0.5.1
Currently pimsclient can only send identifiers to PIMS to retrieve a pseudonym, or send in pseudonyms to retrieve identifiers.
This is enough for projects starting clean with PIMS.
For projects with existing data this is not sufficient however. These often require historic pseudonym/identifier pairs to be added to PIMS so that new data for historic patients will be pseudonymized consistently with already acquired data.
For this there needs to be a way to set manual pseudonym/identifier pairs in PIMS.
Special care needs to be taken to handle conflicts:
Add a requests AuthBase object that does re-login when msal auth token runs out.
Currently this is holding up dependent projects. It's a good addition here
The PIMS API endpoint
api/Keyfiles/{keyfile_id}/Files/Deidentify
is the main way of setting pseudonyms via pims API. (see https://github.com/sjoerdk/pimsclient/issues/159)
It turned out in #10042 that in case identifiers already exist with a different pseudonym, PIMS will silently skip these. The call finishes as succesful. However the pseudonyms will not be set.
This is unwanted behavior. PIMS itself needs to be changed to return some kind of warning or error. However, in the mean time, consider having pimsclient check everything manually beforehand to avoid these errors.
We want to be able to save AccessionNumber pseudonyms in PIMS. Add this as a datatype
Hey Sjoerd,
Really nice client, I have been trying to get it to work with our downloading scripts.
Currently, the automatic download for e.g. COVID CT has per patient ID both a pseudonym and another random number, the anonymization salt. This is used to perturb data randomly, e.g. moving dates around etc.
I would therefore like to be able to not only have "PatientID" | "Pseudonym" in PIMS, but actually rows like "PatientID" | "Pseudonym" | "Salt".
I tried hacking around the KeyFiles.deidentify method and also tried fiddling around on the swagger UI of PIMS. I think /api/Keyfiles/{KeyfileKey}/Pseudonyms/{PseudonymId}/Data
should be suitable, but I get an insufficient rights error.
Do you know if this is already possible with pimsclient now and I am missing something?
For Project.set_keys()
there should be an 'ignore errors' flag. Currently an error is thrown if any pseudonym to set already exists. When adding many pseudonyms however, upon hitting any error, often part of the pseudonyms you were setting aready exist. Having to prune these away each time before re-running the script is annoying.
Add 'ignore_errors=False' which will print warnings instead of raise exceptions when trying to set existing pseudonyms
For debugging server interactions it is often useful to output the full content of post requests done with pimsclient, including full json content.
As far as I have found you can get the requests lib to output a lot of http debug info (https://stackoverflow.com/questions/16337511/log-all-requests-from-the-python-requests-module/57325050#57325050), but not the json content sent. I think this should be added in pimsclient code.
There is an unwanted schroedingers behaviour in the current PIMS API. By querying whether an identity has a pseudonym, you create that pseudonym.
There should be an API endpoint to get a yes or no answer without creating pseudonym. This is important when doing larger scale operations on legacy archives. You don't always want to create pseudonyms for everything you query.
Related to https://github.com/ResearchBureau/tickets/issues/210
Server errors codes are caught in the check_response(). The error text is being added to the exception message unbounded.
It turns out the exception message can be very long. For 503 service unavailable errors, PIMS2 can return a very large string of bytes. This takes up 100s of pages of logging each time.
Bound exception message length please.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.