Export data

#Exporting data.

All studies are saved in the database, and can be exported once the session is done.

The system allows for two different kind of exports:

• Raw data export in the session and study level
• Condensed data export in the study level

Raw data export

This should be used if a researcher wants to save all the data generated in a study. It consists of exports of all entities involved in the session execution.

The following files will be generated:

• CompletedTask_session_(SESSIONID)_DATE
• CompletedTaskAttribute_session_(SESSIONID)_DATE
• CompletedTaskScore_session_(SESSIONID)_DATE
• EventLog_session_(SESSIONID)_DATE
• SubjectExport_session_(SESSIONID)_DATE
• SubjectAttribute_(SUBJECTID)session(SESSIONID)_DATE
• TaskExport_session_(SESSIONID)_DATE
• TaskExecutionAttribute_(TASKCONFIGID)session(SESSIONID)_DATE

And the folders:

• CompletedTaskAttribute
• EventLog
• TaskExecutionAttribute

To better explain the structure, this is a summary of how the system stores the data.

In a session the researcher can create the subjects by hand, or can use a landing page to automatically create subjects. Sometimes this subjects can also have attributes, either set by other sessions or by the researcher. The entity that holds the subjects is : Subject and its export file is: SubjectExport_session(SESSIONID)DATE. The subjects attributes are stored in SubjectAttributes and the export is : SubjectAttribute(SUBJECTID)session(SESSIONID)DATE.

Before a session starts, the tasks must have been configured to use a plugin and a configuration data. This is stored in the TaskExecutionAttributes and the export is on the TaskExecutionAttribute(TASKCONFIGID)session(SESSIONID)DATE files, one for each task configuration. The tasks also have configurations related to the pages they will use and how long each page will last and so on. This is stored in the Task entity and the export is on the TaskExport_session(SESSIONID)DATE. This information is included in the export because the configuration is often a condition to the experiment.

Once the condition to start a session or its start time is reached, the system creates for each of the tasks CompletedTask entries. If the task is set to be run in solo mode, one CompletedTask will be created for EACH of the subjects in the session, if not, only one per task.

This CompletedTask entity is the equivalent of an empty answer sheet. If task is in group mode, one blank page for the group, if in solo one blank page per subject. The export for this entity is in CompletedTask_session(SESSIONID)DATE

As soon as the first task starts, the system will store all answers that the users provide inside the empty sheets we just mentioned. However we use a separate entity to save the data: CompletedTaskAttributes and the export file is : CompletedTaskAttribute_session(SESSIONID)DATE.

If the task has scoring turned on, and the plugin provider implemented the scoring script, the calculations will be done by the system and the result will be saved in the entity CompletedTaskScore whose file is : CompletedTaskScore_session(SESSIONID)DATE.

All the data sent between the server and the subjects is stored in a form of event. We do it like that so that we can replay the events, in case a user refreshes its browser, or in the future if we implement a REPLAY SESSION widget. By all data, we mean all answers to the tasks, all chat messages, all interactions to collaboration widgets and so on. The entity that saves all of this data is: EventLog and its file is: EventLog_session(SESSIONID)DATE.

The system provides the ability for any of the attributes to be integers, doubles, booleans and strings.

Since strings can have any length, the system checks if the current item being exported has new lines or a JSON structure. If it does, it does not export it inline in the .CSV file. It exports it in an external file saved in one of the folders:

• CompletedTaskAttribute
• EventLog
• TaskExecutionAttribute

Condensed data

The ideia for this export is to put in one file the results for all sessions within a study. The new files that get exported are:

• CompletedTaskScore_study_DATE - This holds all of the SCORES for all of the sessions listed one after the other. (Used to calculate global session scores.)
• SubjectContribution_Aggregate_(STUDYID)_DATE - This file aggregates all of the subjects interactions and lists them in one line per session, the system automatically calculates all of the communications and interactions from the EventLogs.