Question about authentication security about cmi-5_spec_current HOT 3 CLOSED

This is what we (at Rustici Software) affectionately refer to as the Pizza Problem (I even mentioned it on the last WG call). Original reference that is still accurate and can really be applied to any interoperability standard: https://scorm.com/blog/scorm-security-some-perspective/

Essentially the answer is, it isn't sufficiently secure to prevent cheating.

Requiring the AU to have a server side component isn't sufficient either, particularly when you start from the premise of "a sufficiently advanced user" because they could still craft requests to the server to have it trigger the mechanism that is used to trigger the statements that fulfill the requirements of the spec.

In fact, you can do cmi5 100% server side, there is nothing to stop you from proxying the launch URL request to a different application endpoint and have the application handle all interaction with the end user and perform the session elements server side, the application just won't be directly interoperable as cmi5. This is where cmi5 truly differs from SCORM in that there is no requirement for there to be a JS (really ECMAScript) runtime available. As the article suggests there are ways to improve the detection of cheating, particularly if the AU is already known to the detecting system, and statement signatures would certainly help with trusting the data is unaltered from the original source (and could be leveraged on either or both the LMS and AU statements (you'd want different private keys)).

Does cmi5 / xAPI offer any protection against cheating beyond the easily defated basic auth token? Is there something I'm missing?

Directly, no. Does it provide a path to something better than SCORM did, IMO yes. Have we seen much uptake or even concern over it, not really (yet).

from cmi-5_spec_current.

Thanks for taking the time to respond! I agree that delivering packaged, interactive content without a server-side component is always going to necessitate sacrificing control. Indeed, the online games industry has struggled with this for years and the answer hasn't changed in decades: adjudicate everything server-side, trust as little as possible from the client, and only trust the client to report inputs, not the results of their actions.

Requiring the AU to have a server side component isn't sufficient either, particularly when you start from the premise of "a sufficiently advanced user" because they could still craft requests to the server to have it trigger the mechanism that is used to trigger the statements that fulfill the requirements of the spec.

This is a bit disingenuous. Sure, a user could click through the pages of an AU without reading them, just as easily as they could craft a JSON request saying they clicked through the pages. But the real problem is when the user reports a 100% result on taking a quiz. At least with a server-side grading component, the user would have to do the work of finding the right answers and submitting them to the server for grading.

Still, I broadly agree with your assessment, that client-only packaged content will always involve trust and can't effectively prevent cheating.

I also agree that cmi5 courses could be implemented server-side and use a shared secret to communicate server-to-server with the LMS / LRS. This seems like the only surefire way to protect against cheating. But at that point, cmi5 wouldn't be necessary - actually, you'd just be working around it anyway. This is really just a regular OAuth implementation between two server-side systems based on a shared secret - something that happens all the time these days and doesn't necessitate a new standard.

There is a so-called "nuclear" option here: streaming. Hear me out.

If I trust the courseware authors (and do an audit of the AU code and deem it safe), I can host a server-side headless browser and stream the course content to the user, essentially as video (although it could be streamed as rendered HTML - there are some products out there that do this), and have the client's input streamed back to the server. This would let us use fully packaged cmi5 content from trusted providers without also opening the door to cheaters and abusers. The LMS / LRS could only accept statements from the server-side cmi5 player, refusing to accept xAPI statements from end users.

Has anyone done that before - streamed cmi5 courses from a server-running browser? It would be an interesting exercise for sure!

from cmi-5_spec_current.

Reviewed in cmi5 working group meeting 6/17/2022.

There is no practical way to guarantee cheating cannot take place with an interoperability standard (like cmi5).
There are multiple methods to reduce the risks of cheating, but that requires control of the learning environment (which is something outside of the scope of cmi5).

from cmi-5_spec_current.