SMP: CFE_TIME_GetReference() has insufficient protection against update while reading about cfe HOT 6 CLOSED

Imported from trac issue 15. Created by jphickey on 2014-12-30T22:23:15, last modified: 2019-03-05T14:58:28

from cfe.

Trac comment by glimes on 2016-11-08 14:17:08:

Current crop of cfe-next are all going into CFE 6.6

from cfe.

Trac comment by jphickey on 2017-10-16 13:47:18:

This is currently in progress and it depends on #74 being fixed first.

The issue described here is related to reading the reference time by other threads. The protection is inadequate and allows a ready to get a partially updated object. However, the "garbage in garbage out" rule applies, there are deeper issues related to updating the shared object in the first place. Once the writers are synchronized, then the issue of readers can be addressed.

from cfe.

Trac comment by jphickey on 2017-10-16 14:39:46:

CONFIRMED ISSUE

I have confirmed I am able to produce this issue in the real system by subscribing to the "1Hz" MID in an app, and calling CFE_TIME_GetUTC() within that handler. Sometimes this comes back with bad UTC information.

Note in order to //identify// when the UTC information is bad, it required a bit of a hack to the update code to first "poison" the global MET value before writing the correct value in the global variable. If it gets returned from CFE_TIME_GetRefence(), the poisoned MET value then creates an incorrect UTC conversion, which is in turn identifiable in the application.

This proves that applications can indeed read the MET during update, even though the updater thread runs at a higher priority than the reader thread.

from cfe.

Trac comment by jphickey on 2017-10-17 14:56:46:

Note - it was easier to include a patch for the CFE_TIME_GetReference() in with the rest of the changes for #74, rather than keeping it a separate changeset.

The fix is to keep two version counters, instead of just one. The "pending" counter is incremented before any changes start, and the "complete" counter is incremented when all changes are complete. A reader task then samples them in the //opposite// order, where the "complete" version counter is read before any copying, and then it is checked against the "pending" counter after copying is complete. If these values are equal, then the copied data can be trusted. If they are not equal, then it must be retried.

This "version" paradigm was preserved (rather than using a lock) since it allows multiple reader tasks to execute concurrently, but a mutex lock would force all reader threads to be serialized, negatively affecting performance.

Testing with the #74 fix, no corrupt UTC samples were observed using the same "stress test" approach described in my previous comment. Therefore nothing more beyond the fix for this other ticket needs to be done.

from cfe.

Trac comment by jhageman on 2019-03-05 14:58:28:

Milestone renamed

from cfe.