P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Summary

This article presents a method for estimating same-day P300 speller Brain-Computer Interface (BCI) accuracy using a small testing dataset.

Abstract

Performance estimation is a necessary step in the development and validation of Brain-Computer Interface (BCI) systems. Unfortunately, even modern BCI systems are slow, making collecting sufficient data for validation a time-consuming task for end users and experimenters alike. Yet without sufficient data, the random variation in performance can lead to false inferences about how well a BCI is working for a particular user. For example, P300 spellers commonly operate around 1-5 characters per minute. To estimate accuracy with a 5% resolution requires 20 characters (4-20 min). Despite this time investment, the confidence bounds for accuracy from 20 characters can be as much as ±23% depending on observed accuracy. A previously published method, Classifier-Based Latency Estimation (CBLE), was shown to be highly correlated with BCI accuracy. This work presents a protocol for using CBLE to predict a user's P300 speller accuracy from relatively few characters (~3-8) of typing data. The resulting confidence bounds are tighter than those produced by traditional methods. The method can thus be used to estimate BCI performance more quickly and/or more accurately.

Introduction

Brain-computer interfaces (BCIs) are a noninvasive technology that allows individuals to communicate through machines directly without regard for physical limitations imposed by the body. BCI can be utilized as an assistive device operated directly by the brain. BCI uses the brain activity of a user to determine if the user intends to choose a certain key (letter, number, or symbol) displayed on the screen¹. In a typical computer system, a user physically presses the intended key on a keyboard. However, in a BCI system with a visual display, the user needs to focus on the desired key. Then, BCI will select the intended key by analyzing the meas....

Protocol

The "CBLE Performance Estimation" GUI was applied to two datasets: "BrainInvaders" dataset and Michigan dataset. For the "BrainInvaders" dataset, the data collection was approved by the Ethical Committee of the University of Grenoble Alpes²⁰. Michigan data were collected under the University of Michigan Institutional Review Board approval¹⁹. Data were analyzed under Kansas State University exempt protocol 7516. If collecting new data, follow the user's IRB-approved process for collecting informed consent. Here, the proposed protocol is evaluated using offline analysis of previously-recorded, d....

Results

The proposed protocol has been tested on two different datasets: "BrainInvaders" and the Michigan dataset. These datasets are already introduced briefly in the Introduction section. The parameters used for this two datasets are mentioned in Table 1. Figures 2-4 depict the findings obtained using the "BrainInvaders" dataset, whereas Figures 5-7 demonstrate the results achieved fr.......

Discussion

This article outlined a method for estimating BCI accuracy using a small P300 dataset. Here, the current protocol was developed based on the "bi2014a" dataset, although the efficacy of the protocol was confirmed on two different datasets. To successfully implement this technique, it is crucial to establish certain variables, such as the epoch window for the original data, the window for time shifting, the down-sampling ratio, and the size of both the training and testing datasets. These variables are determined b.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
MATLAB 2021	Matlab	N/A	Any recent MATLAB version can be used.