Novel Objects Experiment

This repository contains code for the web-based human experiments for the paper Learning to Learn: How to Continuously Teach Humans and Machines.

Hosting the experiment

To run the experiment locally, make sure node.js is installed and run:

$ npm install
$ npm run dev

and visit http://localhost:5173.

To build a production version of the experiment, run:

$ npm run build

after which the bundled experiment will be written to dist/ which can be uploaded online.

Collecting Data

We host the experiment on a JATOS server to collect the experiments. Instructions to install and get started with the server can be found here.

To setup the experiment in JATOS, we provide jzip files for all six curriculums in build/ which can be imported into JATOS as instructed here.

Computing Results

The raw results for the experiment can be viewed in the JATOS server. To compute the accuracy and forgetfulness, we provide the parse-results.js file which writes the results to a JSON file.

In order get the accuracy and forgetfulness, export the raw result files for the desired participants as instructed here. Make sure to export export a JATOS Results Archive (JRZIP) file.

In an empty directory, unzip the jrzip file and run the parse-results.js script from there.

For instance, assuming you are at the directory of this repository in the terminal.

$ mkdir results && cd results
$ cp results.jrzip .
$ unzip results.jrzip
$ node ../parse-results.js