Skip to content

Home

Jump to bottom
jsewall edited this page Feb 15, 2023 · 17 revisions

Table of Contents

  1. Overview
  2. Installation and Build
  3. Architecture
  4. Using the built-in logging library
    1. Basic Usage Example: Initializing the library
    2. Logging a Student's Attempt
    3. Logging a Tutor's Evaluation
    4. Anatomy of a logging message
  5. API
  6. References

Overview

DataShop is a learner record store (LRS) specifically designed to take in transactional student input evaluation data. In other words, the database is ideally suited for data that records a learning session between a student and an intelligent tutor. However, the library itself can be used in a number of other student logging scenarios and other data formats.

The logging library is geared towards tutors and activities that communicate interactions using Selection-Action-Input triples (SAIs), each of which can be thought of as a Subject-Verb-IndirectObject sentence describing a student's interaction. The library is meant to log both the student's interactions (encoded as SAIs) and the tutor's responses about SAIs, as well as other interactions with the tutor interface.

Each student/tutor session consists of a session start, one or more sets of student-to-tutor/tutor-to-student message pairs and a session end. Please see the diagram below that illustrates the most basic series of logging steps:

E.g. for most student/tutor interaction you will be using the code to document the following state transitions:

The actions in steps 2 and 3 form a pair linked together by a Transaction ID. We log interactions this way so that later log analysis can match up a student's action with the proper tutor response or evaluation. Therefore the execution diagram above can be thought of as a series of student-action/tutor-evaluation transactions.

In the diagram Attempt marks the event that a student acted on an interface to generate an SAI. For example if a student typed in "Hello World" into text input area "textinput1" the interface will generate an SAI with S("textinput1"),A("UpdateTextField"),I("Hello World").

Installation and Build

First, make sure you have NPM installed on your machine:

https://www.npmjs.com/get-npm

After pulling from the git repository run:

npm install

To do a continuous build for development purposes:

npm run dev

To do a build that will be saved into ./lib,

npm run build

We encourage new developers and users to first try out our sample code. To see a page using the library bring up this file in a browser:

index.html

Architecture

  1. The singular point of contact for any learning activity developer is the semantic layer. This is where you express a student's interactions with your learning software in a way that can be readily analyzed by both machines and humans.

  2. Data and messages going to the LRS (in a lot of cases DataShop) need to be encoded such that a machine can read them and process them. By default the logging library supports the DataShop format but its design allows other encoders to be used. Currently xAPI and Caliper encoders are in the works.

  3. Finally there is the transport layer, which ensures that messages get transmitted to the log (message capture) server. In this part of the library we use mechanisms such as bundling to make sure we do not flood any server we send to.

Using the logging library

Most users will not need to do more than listen to log message traffic that gets sent from the interface to the log server (or your own logging script). We've made this quite easy and can be done by adding a few lines of Javascript code in your existing tutor. Take a look at the example below:

<script>	
var vars =
{
  question_file: "1416.brd",
};

function ctatOnload ()
{
  initTutor(vars);
		
  var logLibrary=commShell.getLoggingLibrary ();
  logLibrary.assignLogListener (function logListener (aMessage)
  {
    console.log ("Log message: " + aMessage);
  });
}
</script>

All that happens is that the normal tutor startup sequence is augmented with an extra call to obtain a reference to the logging library, after which a listener function is assigned. When you add this to your tutor you will receive a copy of every log XML message that gets sent out.

For those who want to have full control over logging or who want to modify how logging works, a more detailed mechanism is available. Most of the time the approach described below is used for html pages that do not have a CTAT tutor but want to use the CTAT logging capabilities. In every other case it is recommended to use the built-in logging capabilities.

Basic Usage Example: Initializing the library

Initialization (Default CTAT Version):

<html>
<head>
<script src="https://cdn.ctat.cs.cmu.edu/releases/latest/jquery.min.js"></script>
<script src="https://cdn.ctat.cs.cmu.edu/releases/latest/ctat.min.js"></script>

<script>
	CTATConfiguration.set("SessionLog", "true");
	CTATConfiguration.set("Logging", "ClientToLogServer");
	CTATConfiguration.set("dataset_name", "HTML5LoggingTest");
	CTATConfiguration.set("dataset_level_name1", "Unit1.0");
	CTATConfiguration.set("dataset_level_type1", "unit");
	CTATConfiguration.set("dataset_level_name2", "Section1.0");
	CTATConfiguration.set("dataset_level_type2", "section");
	CTATConfiguration.set("problem_name", "AFractionsProblem");
	CTATConfiguration.set("school_name", "CMU");
	CTATConfiguration.set("instructor_name", "A Teacher Name");
	CTATConfiguration.set("user_guid", "student1");
	CTATConfiguration.set("class_name", "22-512");
   
	var loggingLibrary=new CTATLoggingLibrary (true);
		
	function testLogging ()
	{
		useDebugging=true;
		
		loggingLibrary.start ();
	}

</script>

<body>
    <button type="button" onClick="testLogging ()">Start Log Session</button> 
</body>
</html>

When this JavaScript code completes 1 or 2 XML messages will have been sent to the log service, depending on the configuration settings of the library instance. By default you would see the following two messages:

<?xml version="1.0" encoding="UTF-8"?><log_session_start timezone="undefined" date_time="2015/09/09 13:40:49.323" auth_token="" session_id="ctat_session_d2d49363-a08a-dfd8-6afa-9101b1dcbc6c" user_guid="calvin" class_id="" treatment_id="" assignment_id="" info_type="tutor_message.dtd"/>
<?xml version:"1.0" encoding="UTF-8"?>
<log_action auth_token="s=cbe1bc160a2d8e4a67e350e073bc4c39&u=calvin&h=913b9333a84f2082478223caea39f10e" session_id="ctat_session_73c47bf7-edfb-05f4-8851-ba71280e3d4f" action_id="EVALUATE_QUESTION" user_guid="calvin" date_time="2015/09/09 15:41:00.332" timezone="America/New_York" source_id="CTATTutor" external_object_id="" info_type="tutor_message.dtd">
  <?xml version="1.0" encoding="UTF-8"?>
  <tutor_related_message_sequence version_number="4">
    <context_message context_message_id="9fb401fb-a7ca-5557-a38b-5f344d56d925" name="START_PROBLEM">
      <class>
        <name>22-512</name>
        <school>CMU</school>
        <period>1st</period>
        <description>CTAT Logging Test</description>
        <instructor>Miss Wormwood</instructor>
      </class>
      <dataset>
        <name>HTML5LoggingTest</name>
        <problem tutorFlag="tutor">
          <name>AHardFractionsProblem</name>
          <context>undefined</context>
        </problem>
      </dataset>
    </context_message>
  </tutor_related_message_sequence>
</log_action>

Logging a Student's Attempt

var transactionID=loggingLibrary.logInterfaceAttempt ("textinput1","UpdateTextField","Hello World");

Which is equivalent to:

var transactionID=loggingLibrary.logInterfaceAttemptSAI (new CTATSAI ("textinput1","UpdateTextField","Hello World"));

<?xml version:"1.0" encoding="UTF-8"?>
<log_action auth_token="s=cbe1bc160a2d8e4a67e350e073bc4c39&u=calvin&h=913b9333a84f2082478223caea39f10e" session_id="ctat_session_73c47bf7-edfb-05f4-8851-ba71280e3d4f" action_id="EVALUATE_QUESTION" user_guid="calvin" date_time="2015/09/09 15:41:01.632" timezone="America/New_York" source_id="CTATTutor" external_object_id="" info_type="tutor_message.dtd">
  <?xml version="1.0" encoding="UTF-8"?>
  <tutor_related_message_sequence version_number="4">
    <tool_message context_message_id="9fb401fb-a7ca-5557-a38b-5f344d56d925">
      <semantic_event transaction_id="af560eca-c709-ffef-8973-c7d3b47a1f65" name="ATTEMPT"/>
      <event_descriptor>
        <selection>textinput1</selection>
        <action>UpdateTextField</action>
        <input>
          <![CDATA[Hello World]]>
      </input>
      </event_descriptor>
    </tool_message>
  </tutor_related_message_sequence>
</log_action>

Logging a Tutor's Evaluation

The value for the logResponse function's transactionID argument is the that returned by the logInterfaceSAI() call for the student action above.

loggingLibrary.logResponse (transactionID,"textinput1","UpdateTextField","Hello World","RESULT","CORRECT","You got it!");

Which is equivalent to calling:

loggingLibrary.logResponseSAI (transactionID,new CTATSAI ("textinput1","UpdateTextField","Hello World"),"RESULT","CORRECT","You got it!");

<?xml version:"1.0" encoding="UTF-8"?>
<log_action auth_token="s=cbe1bc160a2d8e4a67e350e073bc4c39&u=calvin&h=913b9333a84f2082478223caea39f10e" session_id="ctat_session_73c47bf7-edfb-05f4-8851-ba71280e3d4f" action_id="EVALUATE_QUESTION" user_guid="calvin" date_time="2015/09/09 15:41:01.632" timezone="America/New_York" source_id="CTATTutor" external_object_id="" info_type="tutor_message.dtd">
  <?xml version="1.0" encoding="UTF-8"?>
  <tutor_related_message_sequence version_number="4">
    <tutor_message context_message_id="9fb401fb-a7ca-5557-a38b-5f344d56d925">
      <semantic_event transaction_id="af560eca-c709-ffef-8973-c7d3b47a1f65" name="RESULT"/>
      <event_descriptor>
        <selection>textinput1</selection>
        <action>UpdateTextField</action>
        <input>
          <![CDATA[Hello World]]>
        </input>
      </event_descriptor>
      <action_evaluation >CORRECT</action_evaluation>
      <tutor_advice>
        <![CDATA[You got it!]]>
      </tutor_advice>
    </tutor_message>
  </tutor_related_message_sequence>
</log_action>

Anatomy of a logging message

Every logging message consists of two parts, the envelope and the payload. Both parts are XML messages with the payload URL-encoded within the envelope XML, as per the DataShop's Guide to the Tutor Message Format:

Logging Message

For example, if the XML below constitutes the content part of a log message, like so:

<?xml version="1.0" encoding="UTF-8"?>
<tutor_related_message_sequence version_number="4">
  <context_message context_message_id="9fb401fb-a7ca-5557-a38b-5f344d56d925" name="START_PROBLEM">
    <class>
      <name>22-512</name>
      <school>CMU</school>
      <period>1st</period>
      <description>CTAT Logging Test</description>
      <instructor>Miss Wormwood</instructor>
    </class>
    <dataset>
      <name>HTML5LoggingTest</name>
      <problem tutorFlag="tutor">
        <name>AHardFractionsProblem</name>
        <context>undefined</context>
      </problem>
    </dataset>
  </context_message>
</tutor_related_message_sequence>

Then you will see the final message look like the XML message below. Note the URL encoded payload (content) inside the log_action envelope.

<?xml version="1.0" encoding="UTF-8"?>
<log_action auth_token="" session_id="ctat_session_73c47bf7-edfb-05f4-8851-ba71280e3d4f" action_id="EVALUATE_QUESTION" user_guid="calvin" date_time="2015/09/09 15:41:01.632" timezone="undefined" source_id="tutor" external_object_id="" info_type="tutor_message.dtd">

%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%3Ctutor_related_message_sequence%20version_number%3D%224%22%3E%3Ccontext_message%20context_message_id%3D%22%22%20name%3D%22START_PROBLEM%22%3E%3Cclass%3E%3Cname%3E22-512%3C%2Fname%3E%3Cschool%3ECMU%3C%2Fschool%3E%3Cperiod%3E1st%3C%2Fperiod%3E%3Cdescription%3ECTAT%20Logging%20Test%3C%2Fdescription%3E%3Cinstructor%3EMiss%20Wormwood%3C%2Finstructor%3E%3C%2Fclass%3E%3Cdataset%3E%3Cname%3EHTML5LoggingTest%3C%2Fname%3E%3Cproblem%20%20tutorFlag%3D%22tutor%22%3E%3Cname%3EAHardFractionsProblem%3C%2Fname%3E%3Ccontext%3Eundefined%3C%2Fcontext%3E%3C%2Fproblem%3E%3C%2Fdataset%3E%3C%2Fcontext_message%3E%3C%2Ftutor_related_message_sequence%3E

</log_action>

API

Interaction Methods

  • start()

    • Arguments:
      • None
    • Returns:
      • String The generated session id
    • Description: Convenience function, which is functionally identical to calling the following sequence: generateSession() followed by startProblem().

  • generateSession()

    • Arguments:
      • None
    • Returns:
      • String The generated session id
    • Description: Generates a unique session id.

  • logSessionStart()

    • Arguments:
      • None
    • Returns:
      • None
    • Description: Sends a session-start message to the logging service.

  • startProblem()

    • Arguments:
      • None
    • Returns:
      • None
    • Description: Sends a session message and a context message to the logging service.

  • logInterfaceAttempt(aSelection, anAction, anInput)

    • Arguments:
      • _aSelection _- String Selection
      • anAction - String Action
      • anInput - String Input
    • Returns:
      • String The transaction id.
    • Description: Send a tool message to the logging service for the student's attempt.

  • logInterfaceAttemptSAI(saiObj)

    • Arguments:
      • saiObj CTATSAI A CTATSAI object containing the selection, action, and input.
    • Returns:
      • String The transaction id.
    • Description: Send a tool message to the logging service for the student's attempt.

  • logInterfaceHintRequest(aSelection, anAction, anInput)

    • Arguments:
      • _aSelection _- String or Array<String> Selection(s): use an array to send multiple selections.
      • anAction - String Action(s): use an array to send multiple actions.
      • anInput - String Input(s): use an array to send multiple inputs.
    • Returns:
      • String The transaction id.
    • Description: Send a tool message to the logging service for a student's hint request.

  • logHintResponse(txID, aSelection, anAction, anInput, aSemanticName, anEvaluation, anAdvice)

    • Arguments:
      • txID - String Transaction id of the corresponding hint request; if null, will use transaction id from last attempt.
      • aSelection - String The component selection.
      • anAction - String The component action.
      • anInput - String The component input.
      • currentHintNumber - Integer Position of this advice text among all hint texts available; first text is number 1.
      • totalHintsAvailable - Integer Total number of advice texts available for this hint request.
      • anAdvice - String Success or bug message.
    • Returns:
      • None
    • Description: Send a tutor message to the logging service for the tutor's response to hint request.

  • logResponse(txID, aSelection, anAction, anInput, aSemanticName, anEvaluation, anAdvice)

    • Arguments:
      • txID - String Transaction id of the corresponding attempt; if null, will use transaction id from last attempt.
      • aSelection - String The component selection.
      • anAction - String The component action.
      • anInput - String The component input.
      • aSemanticName - String A semantic name (e.g., "RESULT").
      • anEvaluation - String The evaluation of the attempt, "CORRECT" or "INCORRECT".
      • anAdvice - String Success or bug message.
    • Returns:
      • None
    • Description: Send a tutor message to the logging service for the tutor's response corresponding to the student's attempt.

  • logResponseSAI(txID, saiObj, aSemanticName, anEvaluation, anAdvice)

    • Arguments
      • txID - String Transaction id of the corresponding attempt; if null, will use transaction id from last attempt.
      • saiObj - CTATSAI A CTATSAI object containing the selection, action, and input.
      • aSemanticName - String A semantic name (e.g., "RESULT").
      • anEvaluation - String The evaluation of the attempt, "CORRECT" or "INCORRECT".
      • anAdvice - String Success or bug message.
    • Returns
      • None
    • Description: Send a tutor message to the logging service for the tutor's response corresponding to the student's attempt.

  • setLoggingURL(aURL)

    • Arguments
      • aURL - String A URL.
    • Returns
      • None
    • Description: Set the URL of the logging service.

References

Clone this wiki locally