aicc / cmi-5_spec_current Goto Github PK

As per the 11-15-2013 discussion regarding Art W's pull request. Please add the following example and modify the wording to indicate that only one Activity Type can be used per AU identified. (i.e. An AU can only have one activity type)

"object": {
  "id":"<AU identifier>"
  "definition": {

         "type": "http://aicc.org/cmi5/activities/<activity type>",

         "name": {

            "en-US": "<activity type>"

           }
    },

   "objectType": "Activity"       
},

Would it be helpful to reference the course structure xml from the statement? I suggest this goes in an extension of the activity definition.

Any thoughts?

Hi,

In the xsd, it seems to that the maxOccurs="unbounded is not needed.

<xs:complexType name="BlockType">
<xs:sequencemaxOccurs="unbounded">

This issue thread was started to consider what may be required in CMI5 for a Personal Needs Profile (PNP)

I found the IMS has an “Access For All Specification” with detail on
a) Display: how resources are to be presented and structured;
b) Control: how resources are to be controlled and operated; and,
c) Content: what supplementary or alternative resources are to be supplied.

Which is based on:

ISO/IEC 24751-1 Information technology -- Individualized adaptability and
accessibility in e-learning, education and training -- Part 1: Framework and
reference model

ISO/IEC 24751-2 Information technology -- Individualized adaptability and
accessibility in e-learning, education and training -- Part 2: "Access for all" personal needs and preferences for digital delivery

This is the link to the IMS document:
http://www.imsglobal.org/accessibility/accpnpv2p0/spec/ISO_ACCPNPinfoModelv2p0.html
It was interesting that they had Needs and Preferences tagged as “required”, “preferred” , “Optionally use” and “prohibited”. The example for “prohibited” were animations which flash and can cause seizure.

The IMS Class “Access_For_All_User” puts the Language attribute under this parent container. It also has a ‘Speech Rate’ which I recall Boeing used in the courseware. The link above can be scrolled through to see parameters accounted for.

I believe we need a file name for the course structure XML file. If it is part of a zipped package, the LMS will need to know which file contains the course structure.

The group continued the discussion last week on aspects of Complete vs. Passed with regards to the Course Structure.

There seems to be general acceptance for added the following fields to the metadata for AU’s in a course structure:

A “Move On” Completion Requirement.

If the AU reports a completion/mastery related state to the LMS that meets this requirement, then the LMS can consider the AU completed for purposes of determining course completion or prerequisites.
The vocabulary values for this field would cover the following states:
• Passed
• Completed
• Completed AND Passed
• Completed OR Passed.

Duration
• Normal Calculation of Duration – Time between “Started” and “Exited” statements
• Calculation for Duration in an “abandoned session” – Time between the “Started” statement and the last statement issued by the AU.

Actor Definition for LMS written Statements

Nik Hruska raised the question about which “actor” would be used in XAPI Statements written by the LMS.

The group agreed that it should be the learner. In all AU written statements it is assumed that the actor is the learner also but it is not expressly written in the CMI-5 spec.

It was agreed that some language should be added to “Statement API requirements” sections to confirm that.

So in looking through the requirements, suspend has a requirement to be tightly coupled with terminate. If this is the case, do we need two Statements? Can't suspend indicate that the session is terminated on its own? This way "terminated" could actually mean the logical end of an attempt without the intention to resume (and therefore if summary data were placed only in terminates, it would also make more sense with the knowledge that this really means the learner is done). It would make querying cleaner too. Abandoned would have to look for both suspended and terminate in its behavior, but I think this is the only detriment.

Section 7.2.1. Says “AU shall retrieve the value of session_id…as specified in section 8." But section 8 has no mention of session_id. Should it refer instead to Section 10: State API?

On the calls we have discussed the notion of defining core concepts within behavior of AUs (or any other content type, should they arise). The following list has been agreed upon (taken from Minutes April 26, 2013):

· There must be a notion of “Complete” – learner viewing all the material or completing the all exercises in the entire AU

· There must be a notion of “Incomplete” – learner viewing a portion of the material or completing a portion exercises within the AU

· There must be a notion of “Passed” – being judged/measured in the AU and doing well enough to meet the objectives of the AU

· There must be a notion of “Failed” – being judged/measured in the AU and failing to do well enough to meet the objectives of the AU

· There must be a notion of a “Score” – a percentage correct for the measured activity in the AU

· There must be a notion of “Time spent” in an AU’s session.

The group discussed the “Waived” verb/statement for reporting completion of an AU by alternative means.

It was determined that, rather than having multiple verbs for different scenarios (“collateral credit”, “testing out”, or “administrative reasons” like an AU crash) that there be just one verb with an extension to define the “reason” for the LMS marking an AU complete.

These “reasons” were discussed as follows:

• Testing Out – Taking an Assessment to waive the AU

• Equivalent AU in the course – Completing an Equivalent AU (in the same course) to waive the AU

• Equivalent Outside Activity – Completing an Equivalent Activity outside of the course to waive the AU

• Administrative – The LMS administrative user marking the AU complete for various reasons. (AU execution failure, etc.)

There's a lot of stuff duplicated across the run time and course structure documents. I suggest removing it and putting it all in a separate document, at least whilst it's being developed.

While the MoveOn criteria is specified in the Course structure, it is possible that an LMS Administrator overrides the criteria originally imported. In that event, the AU has no knowledge of what criteria is required to advance to the next AU. Should the value be placed in the State document?

Hi,
From my reading, an AU is a subset of what the XAPI spec calls an AP, but which is launchable from an LMS. Is this correct? If so I propose:

State the relationship between AUs and APs in the AU section.

Replace references to learning activities with AU to avoid confusion.

Demographic Data (Profile API)

After some discussion it was made clear that Demographic data will have to “optional” due to privacy rules (“Safe Harbor”, EU privacy laws, etc.) that LMS vendors have to work with.
The group will begin to define Profile API and Actor Object (Learner) properties for this.

We (Jhorlin and Nick at Riptide) met about this a few times this week...

Can we get some input?
In defining the purpose of launch in other environments. In the web environment launch tells the LMS that the content from the LMS has been delivered to the learner.

In the context of an OS do we want to place strict rules stating that the binary (executable) and the content have to be delivered from the LMS to the learner? Is it sufficient to state that the binary can reside within the OS but the content must be downloaded from the LMS?

Thoughts?

The notion of an “incomplete” verb appears to have been dropped and it seems that an AU is considered “incomplete” as soon as it has been started. (and remains incomplete until it is “completed’)

Use "targetNamespace" in place of "target-Namespace" attribute in schema tag, XSD course schema definition.

I suggest we change the style and formatting to match the experience API. This means for example replacing "shall" with MUST and putting more of the information in tables.

Thoughts?

I’m working with a client to implement a secure (or as secure as possible) mechanism for launching e-learning content from an LMS. We’re adopting the CMI5 launch mechanism, but I want to specify a mechanism whereby the LMS can be integrated with the LRS to generate new Basic credentials for each launch. Primarily I just need with to work with Moodle and a bespoke LRS being developed for this project, but it would be beneficial for everybody if we can agree a standard way of doing this so that the work we do on Moodle will plug into any LRS, or at least any CMI5 compliant LRS. I’ve come up with the following high level approach and appreciate any input before I work on the details.

1. The LRS will generate a set of OAuth credentials for the LMS to use. These credentials will only ever be used for server to server communication between the LMS and LRS and will never be exposed to the learner in any way. The LRS administrator should only supply these credentials to an LMS it has a high level of trust for.
2. The LRS will provide an endpoint for the LMS to request a set of basic credentials. The LMS will request a new set of credentials every time it launches an experience.
3. When requesting a set of basic credentials the LMS will pass certain parameters to the LRS, for example: expiry, authority, allowed actors, allowed verbs, allowed activity ids, allowed registration, allowed timestamp, allowed endpoints etc. (We’ll refer to the list of OAuth authorisation scopes in the Tin Can specification: https://github.com/adlnet/xAPI-Spec/blob/1.0.1/xAPI.md#642-oauth-authorization-scope as a starting point)
4. The LRS will return a freshly generated set of basic credentials and certain permissions based on the parameters passed by the LMS.
5. The LRS will refer to these permissions when it receives a request using the basic credentials it supplied and will reject requests outside those permissions. Where a set of credentials has expired, error code 401 Unauthorized will be returned. 403 Forbidden will be returned

I need to move fairly rapidly with this; if anybody would like to discuss this in a Google Hangout or something, please let me know a good time for you.

Per Bill's request, my thoughts on verbs proposed. My comments are specific to CMI-5 and not commentary on usage of these verbs in XAPI.

•Passed
Personally, I don't see a use for this specific to CMI-5. The "result" in XAPI will indicated success of failure.

•Failed
Ditto.

•Completed
This is critcal to me. Once we see a completed, we can check the "result" property for success or failure.

•Mastered
Not sure how this is different from a Completed verb plus a "success" result.

•Progressed
I am ambivalent on this one. I guess it could be used to report progress.

•Commented
Could be used to provide feedback to the AU developer.

•Resumed
•Suspended
I believe these are important as they could allow us to suspend and resume an in-progress AU.

•Asked
Could be used to forward questions to the AU developer or content owner, but not critical.

•Responded
Could be used for detailed tracking of results.

•Preferred
Can be used to store student preferences (Volume, window size, etc)

•Scored
How is this different from the score in the "result" data?

The progress extension is listed in the context section but should be on the result section.

Reporting tools may want to get the status of an e-learning course, rather than the activity stream of events. For example a reporting tool might want to know the overall success status of a piece of e-learning without having to calculate this based on a series of statements about several attempts in which the learner had a mix of successes and failures.

I propose we include a standard state document as part of CM5. This document can be titled 'CMI5Status' and contain a JSON object with a property of "result". The value of this property should be a result object as defined here: https://github.com/adlnet/xAPI-Spec/blob/1.0.1/xAPI.md#415-result

Thoughts?

We need to find a way to allow the launched content to be able to request demographic data from the LMS. This is a current capability of AICC HACP, and useful for any content that needs context or extended information about the learner/agent.

The LMS passes the content the agent ID. The content asks the LMS for extended information on the agent. For example location, organization, department and so on.

Could we discuss with the group. There seem two issues

1. How do we structure the data, we might use HR-XML or we might have name/value pairs
2. What technical method should be used? We potentially might extend one of the existing xAPI APIs, e.g. extend Agent Profile API probably best or modify Document API https://github.com/adlnet/xAPI-Spec/blob/master/xAPI.md#docapis

In 9.3 Verbs, we specify:

LMS system must record and provide reporting for all statements regardless of which verbs AU’s use in statement.

This statement works well as a general outline for LMS accountability, but doesn't go very far into specifying the actual reporting requirement / use case for the LMS.

Background

(apologies, tl;dr - if your eyes are starting to water, please skip to the questions, below)

In my work, we offer course products as well as an LMS, and also often deal with content integration for third-party LMSes. Custom reporting and non-standard metrics support has become a common request for clients, and while we can take care of this in our LMS, currently, a content provider has little to no control over how the data (especially non-standard data such as values stored in suspend data) can be reported on in a third party system.

We're finding more and more that the LMS administrators that we support are often:

• Less technical than we would desire (it's rare to find one that can tell us which version of scorm their LMS supports)
• Less likely to interact with complex systems in their own LMS, or:
• Very interested in the reporting side of things but frustrated by:
  • Lack of granularity in the data (especially for single-sco content)
  • Lack of a reasonable reporting / report generation process
  • Lack of capability for their LMS' standard reporting to match our LMS' custom tailored reports
• More likely to point the finger at the content when the LMS fails to support a reporting feature (scorm 1.2's non-mandatory objectives and interactions, for example, and especially 1.2's lack of interactions.n.description, which I'm sure we've all grappled with when reporting on interactions).
• More likely to ask the content provider for LMS investigation and support than contacting their LMS provider (as LMS vendors often provide per-hour support fees, while content providers will usually go further to ensure that their content is perceived as functional, even if the LMS partially or fully at fault)

XPAPI/TinCan alleviate many (or as some might say, too many) of these restrictions. With CMI-5 defining the 'standard' verb/activity types, but with the goal of allowing the use of non-standard objects wherever required, I am very excited from both a content provider and LMS provider standpoint to see what kinds of data we can start making available to our clients. All the data in the world, however, won't help those clients if they can't make effective use of it for any of the reasons above (especially if we as content providers do not have any control over the LMS itself).

Questions about LMS reporting requirments

A.K.A. What are you trying to say, Chris?

• What is the most basic requirement for a reporting mechanism? We've talked about searching / filtering using the IRI / contextActivity. Is the LMS reporting requirement stopping at this level, or are we thinking that the reports should be more formalized, specifically purposed things that exist as named entities on the LMS (i.e. "WHMIS User Completion Report with session time and interaction results"?
• Is the LMS required to provide automatic reporting / full visibility around all statements/verbs used by a course/AU, or is specific input required from the LMS admin to tailor a custom report per course/AU?
• Does prescribing a base-level reporting mechanic go too far into implementation?
• From a content provider perspective: Wouldn't it be great to be able to have a mechanism to suggest to the LMS how best it might report on the course data? (Especially non-standard data that a course might make use of?)

Thanks Team! (Sorry for the wall of text! I guess I'm a bit of a reporting nerd! 😅)

The spec, in 10.0 says that ONLY one State document for the combination of activityid, agent, registration, and stateid can exist. It says the Document Name (State ID) must be "LMSLaunchData". Is this the same stateid? If so, we need to clarify it by calling it out in the ONLY requirement and by using syntax consistently (State ID vs stateid). Finally, the State API PUT Properties calls the stateid LMS.LaunchData (notice the extra period).

It's not clear what the document ids of the documents are from the examples given. For example, is 'CMI5LearnerPreferences' the document id or a property of the object contained in the document?

I think it should be the document id, but the example suggests it's a property of the object.

The question was asked about how the mastery and verb reporting may be affected with a “proctored” activity. It was determined that “proctoring” (having some in the room to verify learner identity) was an “access” issue and should not affect the data (verbs, etc.) being recorded.

The group continued the discussion last week on aspects of Complete vs. Passed with regards to the Course Structure.

There seems to be general acceptance for added the following fields to the metadata for AU’s in a course structure:

Mastery_Score

If a mastery_score is specified in the course structure, the AU has been designed to have a scored activity judge mastery. The LMS will use the value to write the Mastery_Score in the LRS initialization data for the AU. The AU must report a Passed verb if the Mastery_Score was met or exceeded. AU’s that do not have a “scored” judgment will not be affected by Mastery_Score.

Activity Types

The following were proposed as Activity Types:
• Assessment
• Tutorial
• Simulation
• Reference
• Job Aid
• Video

We need a base URI to define verbs, activity types, extensions and document names.

Is everybody happy with http://www.aicc.org/cmi5/ ?

With the way the spec is currently set up, the possibility of an activity being both failed and completed cannot exist. I think we should at least justify this in the spec. I believe the line of thinking is that if you failed something, it can't be considered that you did it to a point of completion and you'd need remediation.

Hi all,
first of all: Thank you for being inclusive and helpful at your Work Group Meetings. This causes me to continue being "the new guy" and therefore I'm going to ask some stupid questions while reading the specifications. Please stop me, if it is annoying or just close my issues!;-)

To get started I read the course structure specification. And one thing that I noticed, is that there aren't any XML attributes used although if I look at it from a programmers perspective, they would be vey useful for some reason:

• Some of the child elements are unique, like courseId, objectiveId, moveOn, url, etc. Defining them as attribute would enforce the uniqueness in a more implicit and natural manner.
• Additionally it is a lot easier to crawl the XML when you search an element with an specific attribute.
• And finally, imho it's what people expect, if you look at html for example
• If I would convert the XML structure into an object oriented representation there would be an object "courseId" or "moveOn", whereas they should be simple properties of the parent object.

Here are two different examples:

<course>
<courseId>http://www.yoursite.com/identifiers/course/005430bf-b3ba-45e6-b47b-d629603d83d2</courseId>
</course>

vs

<course id="http://www.yoursite.com/identifiers/course/005430bf-b3ba-45e6-b47b-d629603d83d2"
</course>

We "Id" in all places except courseIdentifier in the Course Structure. For consistency, I believe it should be "courseId"

7.2.2 indicates that "Initialized" is a "should" for the AU, while 9.3.2 indicated "Initialized" verb is a "must" for the AU.

ADL will need to provide a URL structure for verbs and activities. Once this is done, the spec should be updated so that there is more certainty of the definitions resolving.

Current List of Verbs
Based on this meeting’s discussion (May 10), these are the verbs currently proposed.

Launched – the LMS launched the AU
Started – The AU is fully initialized and ready to be used
Completed – The learner viewed or did all of the relevant activities in an AU presentation.
Failed – The learner attempted and failed a judged activity in the AU. If the activity was scored, the statement should have a score in the result.
Passed – The learner attempted and succeed a judged activity in the AU. If the activity was scored, the statement should have a score in the result.
Suspended – The learner exited an AU (uncompleted) with the intention of returning to the AU.
Resumed – The learner resumed an AU that was intentionally left uncompleted.
Abandoned – The learner or the AU exited the session abnormally (not sending an “Exit” verb statement to the LMS) to end the AU session
Exited – The Learner exited the AU normally (sending and exit message to the LMS to close the AU session)/.

Problem
If an AU is launched in an existing window, it needs to know that this has happened and what URL to return the learner to when they exit the activity. I assume this will always be a URL on the LMS. Without this information, the AU might simply close the window when the learner exits.

Proposed solution
The LMS adds a parameter to the State at launch to say what the launch method is, and the return URL.

In a situation where the AU does not properly terminate a session (by sending a statement with an “Exit” verb), what the LMS should do ?
There were 2 approaches proposed :
#1 – have the LMS create a statement in the LRS with an Exit Verb (and extension data indicating that the session was “abandoned” – not exited correctly)
#2 – Define a new Verb called “Abandoned” that the LMS would use to create a statement (indicating that the session was “abandoned” – not exited correctly)

Of the two approaches, it seems that #2 appeared to be the most appropriate to the group, since it was consistent with the LMS created statement in the LRS for the Verb “Launched”. (i.e. LMS created statements should have verbs that only the LMS uses)

The current proposal is to define a verb called “Abandoned”

The following proposal was made for verbs on April 26, 2013. Sorry for the poor formatting, I would recommend that someone who doesn't have a PR out make this a Pull Request.

Proposed Verbs (all mandatory)

· Launched – AU launched from the LMS. LMS must record this statement in the LRS just prior to actually launching the AU.

· Started – AU has been launched and is fully initialized and ready for learner interaction. This represents the initial launch of the AU and must be provided immediately on the first launch. Must be made only once during a session.

· Resumed – AU has been launched and is fully initialized and ready for learner interaction. This represents a subsequent launch of the AU. This must be provided immediately upon all subsequent launches of the AU. This statement must be made only once during a session.

     o   Results 

     §  Success. (mandatory)  Success must be set to false (unless the AU 
     was launched after a “completion” statement was recorded.) 

· Completed – The AU has determined that the learner has met all of the learning objectives specific to the AU. This may have been determined by viewing all of the material, successfully completing an assessment, or some combination. This statement must be made only once during a session.

      o   Results 

      §  Success = true (mandatory) – Success must be set to true

      §  Score (optional) – should be recorded if used to determine verb “completed”.

· Exited – The AU has exited the launch session. Must be recorded just prior to learner exiting the AU. This statement must be made only once during a session (and it must be the last statement). The LMS/LRS must not accept any more requests from the AU until the next launch.

            o   Results

           §  Duration (mandatory). The duration of the launch session must be recorded.
           The AU determines the duration. (i.e. It may be less than or equal to the actual 
           time between “started/resumed” and “exited”).

Has there been any thought to how course structures might be defined?

I'd previously put some rough thoughts on a similar matter together here:
http://tincanapi.co.uk/pages/The_Learning_Plan_Framework.html

Does this look like the direction we want to go in? I could rework that to use CMI5 terms and URIs as a starting point if the group thinks this will be valuable.

The following item was not fully flushed out and will likely require more discussion.

“Test Out” (Collateral Credit)

In some courses the learner may “test out” of a course by passing an assessment AU and not being required to take some or all of the other AU’s in a course. This notion will require the course structure to indicate “completion requirements” for each AU. It will also require the LMS to write in the LRS statements for the “externally completed” AU’s. (to allow the AU’s to discover if they were “externally completed” when reviewed and to provide a comprehensive record in the LRS).

Amplify definition to describe AU reading Launch Parameters and Entitlement documents from the State API

The was an extensive discussion on “mastery” of an AU and the potential use of a “Mastery_Score” (similar to SCORM and AICC) extension. In most cases it is assumed the AU design will determine criteria for mastery. However, there are many use cases where the companies that use 3rd party content wish to establish a higher or lower level of performance required (usually on scored tests) that the “default” determined by AU designer. The proposed Verb “Passed” is associated with mastery (based on learner being judged in the AU).

Based on the discussion, the following was proposed:
If an AU is scored and the LMS issues a “Mastery_Score”, then the AU must use the “Mastery_Score” to determine whether a “Passed” or “Failed” verb is issued in when the learner performs the judged activity. If the AU’s judged activity does not use a “score” then mastery (whether “Passed” or “Failed” verb is issued for judged activity) is solely determined by the AU since there is no clear convention for the many different kinds of criteria that could be used.

Just to clarify, any Statement with the verbs "completed, passed, waived, failed, or abandoned" (the IRIs, not just the words of course) MUST use completion and success. The verb will dictate the allowed value of each.

Please read the thread below on the xapi forums:

https://groups.google.com/a/adlnet.gov/forum/#!topic/xapi-spec/qcq6gP_FmRs

I'd be interested on hearing the views of the CMI5 working group on this. Thanks!

Mastery_Score
• Add as an AU property in the Course Structure
• The LMS will record the Mastery_Score in the State API just prior to launch
• If the AU is scored and a Mastery_Score is issued then the AU must use mastery_score to determine “Passed” or “Failed”.

In CMI5 we say The AU must not issue multiple statements with "Completed" for the same AU within a given AU session or course registration for a given learner. We say the same about passed.

I think this is sensible, but do we need to add more detail about how this will be achieved in the scenario where something goes wrong?

See this comment on the xAPI spec: adlnet/xAPI-Spec#497 (comment)