Learning States

3. Learning states

3.1. Experimental design

Data collection was centered on a learning activity involving an ECG lesson presented in a Jupyter Notebook. This lesson included the same theoretical content as a prior study but incorporated open-ended questions designed to prompt active reflection on the learning process. Participants were required to recall and apply the material they had reviewed to formulate thoughtful responses. To monitor reading behaviors, all text and images were initially blurred, becoming visible only when participants hovered their mouse over specific sections. This method enabled precise tracking of their reading progress.

3.2. Data recording

This study involved 18 volunteers, including 13 females, aged between 18 and 28 (M = 21.7, SD = 3.2). These participants were enlisted from the NOVA School of Science and Technology. All were right-handed, reported no history of psychological disorders, and were not on any regular medication. Written informed consent was obtained before participating and all Ethical Procedures approved by the Ethics Committee of NOVA University of Lisbon were thoroughly followed.

During data collection, electromagnetic noise saturated one of the EEG channels in one participant, and excessive sweat caused saturation of the EDA signals in two participants. These signals were excluded from further analysis. Additionally, HCI variables were not recorded for subject 2. In this case, the files don't have timestamps (because there's no need to synchronise the biosignals with the HCI features) but they do have nseq.

3.3. Data labelling

For the data labelling, participants (biomedical engineering students or researchers) observed a video with their screen activity, facial expressions, and biosignals during the experiment. They were tasked with self-reporting their cognitive states by selecting from a predefined list of learning states, accompanied by their definition:

• Neutral: A state without any discernible emotion or cognitive engagement.
• Interest/Flow: A state of focused interest or deep involvement in the task.
• Surprise: The reaction to encountering something unexpected or new.
• Boredom: A lack of interest or disengagement with the task.
• Distraction: A loss of focus due to internal or external interruptions.
• Confusion: Difficulty understanding the content or task at hand.
• Eureka: A moment of insight or discovery after a period of struggle.
• Frustration: A feeling of dissatisfaction or disappointment caused by unmet expectations or goals.

Participants were instructed to label their cognitive states whenever they experienced a notable change in their learning process or at regular two-minute intervals if no changes occurred. This self-reported labeling, documented in the learning_states_labels.txt files and overseen by the principal investigator, ensured that the recorded states closely reflected the participants' internal experiences.

3.4. Data description

Biosignals include EEG, fNIRS (not converted to oxi and deoxiHb), ECG, EDA, respiration (RIP), accelerometer (ACC), and push-button data (PB). In all biosignal files, the first column represents the timestamps.

HCI features encompass keyboard, mouse, and screenshot data. Below is a Python code snippet for extracting screenshot files from the screenshots CSV file.​

import base64
from os import mkdir
from os.path import join

file = '...'

with open(file, 'r') as f:
    lines = f.readlines()

for line in lines[1:]:
    timestamp = line.split(',')[0]
    code = line.split(',')[-1][:-2]
    imgdata = base64.b64decode(code)
    filename = str(timestamp) + '.jpeg'
   
    mkdir('screenshot')

    with open(join('screenshot', filename), 'wb') as f:
        f.write(imgdata)

A characterization file containing age and gender information for all subjects in each dataset is provided within the respective dataset folder (e.g., D3_subject-info.csv). Other complementary files include (i) description of the pushbuttons to help segment the signals (e.g., D3_S2_PB_description.json) and (ii) labelling (e.g., D3_S2_learning_states_labels.txt).