Performance and Load Tips about rwslib HOT 3 OPEN

Hi John. A bit of background first. Rave is primarily a transactional system with what I would describe as a follow-along reporting subsystem called Clinical Views. The Clinical View system scans for changes in the transactional tables and pulls them into a de-normalized reporting table. The data entry to available for reporting lag is generally low but if there is a lot going on you can get longer delays, up to hours. The services you are talking about all read from these clinical views. A header included with any response tells you the last time the views were updated which can help with the scheduling of further calls. i.e. if you see that a clinical view has not been updated in 4 hours you might choose to repeat a request at intervals until you get more up-to-date data.

Since all the approaches you have suggested read from these clinical views the load on Rave should not be significant, you are making requests from what are essentially materialized views in the database, pre-computed extract tables. These views are organized by form so a subject-level request or a study level request has to do a lot more joins. That said, for a very large study requesting all data for a particular form in a single hit could get you a timeout if it takes more than 1 hour to stream the data. If you think the studies won't become so large that you'll hit this limit then that may be the most efficient, otherwise by subject would likely be safer and you can do by subject/form combination but that may leave you a lot of requests.

There are ways to get just incremental changes from a last date/time but the client has to set up their clinical view configuration in a certain way and that is not guaranteed so I would not rely on it.

Lastly, you could combine the ClinicalAuditRecord dataset with these requests. You'd poll the ClinicalAuditRecords service for data changes (see the AuditEvent sub-project in the extras directory of rwslib) to detect changes in subjects/forms and then on your polling interval you could request those forms/subjects that you know have had changes which could reduce the total number of requests. Bear in mind my caveat about the update timestamps on the clinical views, the audits will be written before the clinical views reflect those changes.

But my advice overall, from knowledge of how the system works rather than really extensive experience of using these various services, is to do by-form if you know data volumes are not hundreds of thousands of records or by subject otherwise.

I know the above doesn't give you a definitive answer but hopefully helps to understand the risks and tradeoffs.

from rwslib.

I am looking to see sample filled eCRFs (forms) for the reference and better understanding. Please suggets from where i can get the same?

from rwslib.

@vagarwal77 a google image search for "medidata rave eCRF" will provide a lot of example screenshots. I don't think we can help you further than that.

from rwslib.