Objectively Assessing Sports Concussion Utilizing Visual Evoked Potentials

Summary

A portable system capable of measuring steady-state visual-evoked potentials was developed and trialed on 65 amateur rugby players over 18 weeks to investigate SSVEP as a potential electrophysiological biomarker for concussion. Players' baselines were measured pre-season, with retesting for reliability, concussion, and recovery assessment being conducted within controlled time-periods, respectively.

Abstract

A portable system capable of measuring steady-state visual-evoked potentials (SSVEP) was developed to provide an objective, quantifiable method of electroencephalogram (EEG) testing following a traumatic event. In this study, the portable system was used on 65 healthy rugby players throughout a season to determine whether SSVEP are a reliable electrophysiological biomarker for concussion. Preceding the competition season, all players underwent a baseline SSVEP assessment. During the season, players were re-tested within 72 h of a match for either test-retest reliability or post-injury assessment. In the case of a medically diagnosed concussion, players were reassessed again once deemed recovered by a physician. The SSVEP system consisted of a smartphone housed in a VR-frame delivering a 15 Hz flicker stimulus, while a wireless EEG headset recorded occipital activity. Players were instructed to stare at the screen's fixation point while remaining seated and quiet. Electrodes were arranged according to the 10-20 EEG-positioning nomenclature, with O1-O2 being the recording channels while P1-P2 the references and bias, respectively. All EEG data was processed using a Butterworth bandpass filter, Fourier transformation, and normalization to convert data for frequency analysis. Players SSVEP responses were quantified into a signal-to-noise ratio (SNR), with 15 Hz being the desired signal, and summarized into respective study groups for comparison. Concussed players were seen to have a significantly lower SNR compared to their baseline; however, post-recovery, their SNR was not significantly different from the baseline. Test-retest indicated high device reliability for the portable system. An improved portable SSVEP system was also validated against an established EEG amplifier to ensure the investigative design is capable of obtaining research quality EEG measurements. This is the first study to identify differences in SSVEP responses in amateur athletes following a concussion and indicates the potential for SSVEP as an aid in concussion assessment and management.

Introduction

People now-a-days are greatly aware of the morbidity caused by brain injuries in sport¹. A sports related concussion (SRC) is a form of mild traumatic brain injury (mTBI) that is frequently reported in contact sports such as football, rugby, and boxing^2,3,4. The biomechanical transduction of impulsive force to the brain following an impact on the field results in a disruption of neuronal function, leading to both immediate and transient symptoms that affect an athlete's physical, cognitive, and emotional state^1,5. In most cases, these symptoms subdue within a short period of time, granted the athlete is appropriately treated and not exposed to further impacts⁶.

As SRC is detrimental to the neurological health of players, sports' governing bodies face the challenge of employing accurate and timely concussion diagnosis to allow for a safe return-to-play protocol^5,7,8,9. However, concussion detection can be precluded by athletes who minimize or deny symptoms to avoid a concussion diagnosis, thus accelerating their return to play. These actions can potentially increase their risk of Second Impact Syndrome, a condition in which rapid cerebral edema forms following a second head injury during the concussion recovery phase¹⁰. Additionally, due to the lack of education around concussion diagnosis and the variable nature of its physiological definition, it is not uncommon for SRC to go unreported or misdiagnosed¹¹. Unfortunately, long periods of repeated and inappropriately managed concussions may lead to a range of chronic neurological impairments, such as chronic traumatic encephalopathy (CTE), which is strongly associated with SRC^12,13,14.

In an effort to combat the challenges associated with SRC, sports organizations utilize a variety of concussion assessment tools. The most commonly used and accessible tool, sports concussion assessment tool (SCAT), is a standardized paper test that incorporates physical and cognitive assessments in combination with scaled symptom reporting^15,16. However, previous studies have demonstrated that symptom reporting is subjective and unreliable by identifying gender differences within mTBI groups and outliers in the control group^17,18. More advanced tools that are utilized at a professional level, such as the Immediate Post-Concussion Assessment Tool (ImPACT), which operates as a Computerized Neurocognitive Test (CNT), also fall victim to manipulation as they require active participation and effort from the athlete. Despite built-in checks for manipulation in CNTs, research has shown that they are prone to ceiling effects and suffer poor reliability^19,20. The limitations of these existing assessment tools in combination with a more public understanding of the significant health effects of SRC have resulted in a critical need for an objective biomarker that can accurately and timely diagnose a concussion.

One field that has shown promise in identifying an objective biomarker for concussion is electrophysiology. There is emerging evidence that event-related potentials, in particular visual evoked potentials (VEP) are impaired following a concussion^21,22. One subset of VEP; steady-state visual-evoked potentials (SSVEP) are an objective, quantifiable fluctuation of electrical activity that occurs in the brain in response to a specific set of visual stimuli, as measured by electroencephalogram (EEG) technology^23,24. SSVEP offer an improved resistance to noise artifacts and variable contact impedance to conventional VEP measurements. Also, due to the controlled frequency of the visual stimulus, there is a reduction in synchronicity between EEG recordings and stimulus, resulting in a more simplified electrical model^25,26. This approach has been validated with frequencies between the 12-15 Hz range producing an optimal response of salience for flicker type stimuli²⁷. Overall, these advantages mean SSVEP offer a more robust electrophysiological measurement that can be used in a non-clinical setting such as sports fields and physician offices. This sideline application possibility in combination with the technology's positive results in previous literature makes it a promising candidate for the identification of an objective biomarker for SRC.

The goal of this study was to investigate potential differences in SSVEP that were recorded from athletes who were assessed by an experienced sports doctor as healthy, concussed, or recovered from a recent concussion. The methodology of the study entailed 65 male amateur rugby union players being routinely assessed with a portable SSVEP system over an 18-week competitive season. Players are to be assessed for a baseline prior to the commencement of full-contact training and re-assessed within 72 h following competitive games. Players who were injured during the season were evaluated for concussions by the team's physician and re-assessed with the SSVEP system for post-injury and recovery readings. Additionally, this study extends its protocol to validate portable SSVEP system's ability to obtain research-quality EEG readings that can potentially aid in the sideline assessment of SRC.

Protocol

Approvals for the first part of the protocol was obtained from the South Eastern Sydney Local Health District (ESLHD) Human Research Ethics Committee (HREC). All players were provided with detailed participant information sheets and consent was obtained prior to participation (SESLHD-HREC reference number: 17/039 HREC/17/POWH/91).

Approvals for the use of improved portable system study was obtained from Bellberry Human Research Ethics Committee (HREC). All control subjects were provided with detailed participant information sheets and consent was obtained prior to participation (HREC Reference number: 1802VEPEEG-CER).

1. Participant screening and consent

1. Recruit participants from a single rugby union club prior to the commencement of a competitive season. Participants must be healthy, native English-speaking, male adults (over the age of 18) who are members of an amateur rugby union team (Figure 1).
   1. Due to the flickering nature of the visual stimuli, ensure that the participants do not meet any of the following strict exclusion criteria: diagnosis or symptoms of epilepsy, existing and/or previous brain injuries, or legal blindness.
   2. Inform the participants to report any symptoms following an injury on the sporting field truthfully to the medical practitioner who forwards the information to the associated study investigator(s). The participants must understand that the study involvement requires a minimum of two SSVEP testing sessions during the season, with the potential for further testings in the event of a concussive injury.

2. Investigative SSVEP equipment setup

1. Fit a charged liquid-crystal display (LCD) smartphone into a Virtual Reality (VR)-frame (Figure 2A; see Table of Materials).
   1. Generate a .mp4 video file consisting of a sequence of black and white screens alternating at a frequency of 15 Hz, for a total of 30 s. Place a random number in the center of the video frame (occupying less than 2% of the screen with a visual angle of 1.5°) for the participants to focus centrally upon during the stimulation. Ensure that the number is altered at 5 s intervals to encourage sustained attention (Figure 2B).
   2. Upload the constructed video file to the smartphone and display it at full brightness (minimum ~490 nits), acting as the visual stimulus of the SSVEP system.
2. Charge the wireless 14-channel EEG headset, which will be used as the primary EEG recording device (Figure 3A).
   1. Pair the headset to a nearby computer via the supplied universal USB Receiver (dongle). Insert the dongle into the computer's USB port, turn on the headset via the power button, open the 14 channel EEG headset software on a computer and select the Connect button on the application next to the displayed headset ID (see Table of Materials).
3. Completely saturate the supplied felt sensors with saline solution.
   1. Install the saturated sensors into the black plastic arms of the headset by gently turning each sensor clockwise until a "click" is felt and the sensor feels secure.

3. Participant baseline assessment

1. Obtain consent from all participants who have passed the screening questions to be involved in the study before the rugby season commences.
2. Ask consented participants to be seated in a chair in a quiet enclosed environment with natural ambient light, such as an office room. While the participants are seated, fit the 14 channel EEG headset to the participants' heads by sliding the headband down from the top of their head. Arrange the electrodes according to the International 10-20 system (Figure 3B)^28,29. Position the two front sensors of the headset in line with the participants' hairline or roughly three finger widths above the participants' eyebrows.
   NOTE: If the participants have thick hair, work the sensor under the hair, and add additional saline. If the subject moves from the stationary seated position or there is a change in environmental settings during the assessment, an artifact (Figure 4) can occur, and the SSVEP data should not be collated for the analysis (discarded).
   1. Use the occipital electrodes (O1 and O2) as the main recording electrodes and the parietal electrodes (P3 and P4) as the ground and common-mode (reference) electrodes (Figure 3B).
3. Ensure there is an adequate connection between the headset and the participants' head by utilizing contact quality software before proceeding with testing. Green electrodes indicate good contact quality (<20 kOhms) according to the manufacturer's standards.
   1. In the event a good contact quality was not achieved for all electrode channels of interest (O1, O2), re-saturate the sensors with saline solution and re-check positioning to ensure the sensors are as flush against the scalp as possible.
      NOTE: Good contact quality must be obtained to ensure electrical connectivity between the scalp and electrodes, minimizing potential artifacts (Figure 4).
4. Ask the participants to hold the testing smartphone within the VR-frame up to the eyes, flush against their faces and nasal bridge, ensuring both eyes are completely covered.
   1. Have the participants confirm that the focal number is visible at the center of the screen and that the frame is blocking out environmental light.
5. Remove the VR-frame from the participants' face and inform them that the visual stimulus video will now be initiated. Then hand them back the VR-frame to reapply in the same position as step 3.4.1.
   1. Remind the participants that they are to focus on the focal number and remain still and quiet during the duration of the SSVEP testing.
6. Initiate the visual stimulus video by pressing the Play button on the smartphone screen, and then have the participants place the VR-frame on his face (as per step 3.4).
7. Ask the participants to confirm they have the VR-frame in the correct position as per step 3.4. Following participant's confirmation, simultaneously begin a countdown stopwatch for 30 s and then select the Start Recording button on the 14-channel EEG headset software to begin recording on the EEG headset.
8. Once the 30 s period is over, stop the EEG recording by selecting the Stop button on the 14-channel EEG headset software.
   1. Remove the VR-frame from the participants' eyes and inform them that the first SSVEP assessment is complete.
9. Save the recorded SSVEP response to the local account by clicking on the Saved Sessions button on the 14-channel EEG headset software. Export the saved file as a European Data Format (EDF) file to a preferred storage hardware device.
   NOTE: EDF files are to be labeled with the participant's initials, date, time, and the type of test conducted (baseline, post-injury, or recovery) for identification and storage purposes.
10. Repeat SSVEP assessment protocol (steps 3.2 to 3.9) in quick succession following step 3.9.
    ​NOTE: This is conducted as the second reading from the two assessments and yields a consistently clearer EEG reading.
11. Once the pair of SSVEP assessments have been successfully completed, have the participant remove the VR-frame from their eyes, and carefully remove the headset from the participants' heads.
    1. Remove the electrode sensors from the headset and store them in an appropriate protective case until their next use to ensure that the saline does not corrode the headset's electrodes.
    2. Clean both the headset and smartphone + frame with isopropyl alcohol (70%) wipes before storing both pieces of equipment in their respective cases until their next use.
12. Ask the participant whether they experienced any abnormal reactions to the stimulus, including the presence of a headache or dizziness. Record the response in a study log.
13. Inform the participant that they have completed their baseline testing and will be assessed again within 72 h of the next competitive game of the season.

4. Post-injury assessment

1. Ensure that any impact on the field suspected to be a concussion is recorded and reported back to the study investigator by either the team physician or player.
2. Obtain post-injury assessments of any players who were reported in step 4.1 within 72 h of the incident using the SSVEP protocol detailed in section 3.
3. Ensure all players undergo a post-injury SSVEP assessment, which is evaluated by the team physician (see section 5) to determine when the player has recovered from the injury and can return to full-contact training and games. Ensure the team physician's diagnosis is recorded for investigational purposes.
   1. Inform the player that they will be reassessed with the SSVEP system once deemed recovered by the team physician.

5. Clinical concussion evaluations

1. Ensure all players are evaluated by the team physician within 72 h of a potential concussion-generating impact.
   NOTE: The team physician must be a licensed practitioner with clinical experience in assessing sports-related injuries and approved by a relevant sport's governing body.
   1. Have the team physician perform a neurological exam in addition to using elements of the Sport Concussion Assessment Tool (SCAT) to determine the player's state of consciousness, and presence of any of the common signs and symptoms associated with concussion¹⁵.
      NOTE: Assessments are made with the player's typical demeanor and behavior as a reference.
2. As per Rugby Australia's Graduated Return to Play (GRTP) guidelines³⁰, ensure the player is re-evaluated by the same team physician within 48 h of a concussion diagnosis, and again once per practice session until cleared to return to play³⁰.
3. Ensure players undergo a final evaluation with the team's physician to determine whether they have clinically recovered, before allowing them to return to full-contact practice or games. The physician is to determine the player's injury state based on the improvement in SCAT testing and reduction in associated symptoms and any atypical behavior, which was previously noted.
   NOTE: This final evaluation is to occur following a minimum of 12 days post-injury as per GRTP.
   1. Record the clinician's determination of the player's injury state (concussed or recovered) for investigational purposes.

6. Recovery assessment

1. Assess the players who were deemed recovered in step 5.3 using the SSVEP protocol detailed in section 3 to obtain a recovery SSVEP response.

7. Test-retest reliability assessment

1. Ensure all non-injured participants are retested within 72 h following each competitive game of the season with the portable SSVEP system in the same setting as detailed in section 3 to obtain data for test-retest reliability of the system.

8. SSVEP data processing

1. Generate a numerical computing script that can import and process all recorded EDF data files from sections 3-7 (see Table of Materials).
   1. Apply a Butterworth band-pass filter with corner frequencies at 5 Hz and 40 Hz to the raw SSVEP voltages obtained with the 14 channel EEG headset to minimize effects of lower-frequency noise, DC voltage offset, and mains power³¹.
   2. Apply a Fast Fourier Transformation (FFT) to the filtered SSVEP values to analyze data in the frequency domain³².
   3. Normalize FFT values from electrode channels O1 and O2 to generate a single array of values for graphical plotting as a power spectrum density (Figure 5).
2. Once the EEG data has been processed, calculate the signal-to-noise ratio (SNR) of each data file by dividing the magnitude at 15 Hz by the average magnitude between 5-40 Hz. The SNR formula is:
   
   Where:
   Amplitude_15Hz = The voltage of the 15 Hz Frequency band (µV)
   Amplitude_Average = The average voltage of all Frequency bands between 5-40 Hz (µV)
3. Ensure all data (raw EDF, processed EDF, SNR value, comments, and dates of the players injury state) is saved in appropriately labeled folders for investigative purposes.

9. Statistical analysis

1. Combine the players' data files in a comma-separated values (CSV) sheet categorically (separate columns for baseline, concussion, and recovery reading).
   1. Use the SNR of each SSVEP reading to compare the different groups. Express all summarized data as medians with 25^th to 75^th interquartile range (IQR), with a statistical significance level of alpha (α) = 0.05 set, and all levels of variance displayed in standard deviation (± S.D.). Use statistical analytics software (see Table of Materials) for analysis of the group SNR values.
   2. Evaluate the normality of all players' SNR values using a Shapiro-Wilk test.
   3. Compare the mean SNR between all three assessment groups using paired t-tests (between baseline-concussed readings, concussed-recovered readings, and baseline-recovered readings) for players who experienced all three types of assessments. Ensure a multiple comparison correction is applied by utilizing a Bonferroni correction.
   4. Calculate the effect size of the t-test comparisons by utilizing Cohen's D³².
2. Estimate test-retest reliability using a model 3, type k intra-class correlation coefficient (ICC); ICC(3,k) with a 95% confidence interval (CI) to examine agreement between baseline and repeated testing throughout the season.
   ​NOTE: The experiment can be paused here and restarted at a later point in time.

10. Improved portable SSVEP system setup

NOTE: The system described in this section is an all in one set up which has been designed to enable for a more efficient measurement process, both quicker and with improved comfort for the participant. Please note that the set up described in sections 2-6 is a simplistic prototype consisting of numerous pieces of equipment merged to generate a proof-of-concept system. The protocol schematic using this system is presented in Figure 6.

1. Obtain a portable SSVEP system that contains a visual stimulus and electrode configuration equivalent to the system described in section 2 (see Figure 7A).
   1. Download the system's SSVEP application from the iOS app store onto a device that operates iOS (see Table of Materials).
2. Ensure the investigational team is familiarized with the instructions for use (IFU) of the portable SSVEP system and its supplied iOS application, which enables the activation of the SSVEP device and local storage of the EEG readings.
   1. Generate subject accounts on the iOS application by following the instructions for use(IFU) of the portable SSVEP system. Open the app, select the Subjects tab, then press the Add New Subject button.

11. Reference EEG system setup

1. Obtain a validated clinical-grade EEG amplifier unit with a modular electrode configuration that does not contain a visual stimulus component (Figure 7C).
2. Ensure the investigational team is familiarized with the instructions for use (IFU) of the clinical EEG system and its supplied clinical neurology software application (see Table of Materials), which enable activation of the amplifier and local storage of EEG readings.

12. EEG validation of an improved portable SSVEP system

1. Recruit healthy, English-speaking adult participants using the same exclusion criteria detailed in section 1. Consent these participants for the SSVEP system validation study.
2. Randomly assign participants a number from 01 to 20. Evaluate participants with an odd number assigned first with the portable SSVEP system, and then with the clinical EEG system (steps 12.3-12.16). Vice versa for participants assigned with even numbers (steps 12.10-12.14, then 12.3-12.9, then 12.15-12.16) (Figure 6).
   1. Set up the SSVEP headset as per instructions for use (IFU): fully charge the headset and Bluetooth pair it to the SSVEPiOS application. Insert the supplied polyurethane sensor cylinders into the electrode channels and saturate using normal saline solution.
3. Place the SSVEP headset on the participant's head as per the system's IFU, with the inferior of the rear housing unit situated directly above the participant's inion, the front visor housing placed securely over their eyes and nasal bridge and the headset tightened via the elastic adjustment straps and securing buckles.
4. Check the SSVEP iOS application's impedance indicator to ensure an adequate connection between the headset and the participant's head before testing. Green visual indicators on the associated application (impedance <15 kOhms) indicate good contact quality according to manufacturer's standards.
   1. If a good contact quality is not achieved for all electrode channels of interest, re-saturate the sensors with normal saline solution and re-check positioning to ensure the sensors are as flush against the scalp as possible.
      NOTE: Good contact quality must be obtained to ensure electrical connectivity between the scalp and electrodes, minimizing potential artifacts (Figure 4).
   2. Ensure that the participant is comfortable. Instruct them to remain still, calm, and quiet while seated and gazing forward into the lights, and to only blink when required. If the subject moves from stationary seated position or there is a change in environmental settings during the assessment, an artifact (Figure 4) can occur, and the SSVEP data should not be collated for the analysis (discarded).
5. Initiate the visual stimulus by pressing the Begin Test button on the iOS application. When prompted, press the Continue button to move to the next stage of testing. The visual stimulus is run twice as per the system's IFU, obtaining a preliminary and primary SSVEP reading.
6. Following completion of the SSVEP assessment, remove the SSVEP headset from the participant's heads. Allow the subjects to relax for a minimum of 30 s.
7. In order to obtain a test-retest reliability value, repeat the SSVEP assessment protocol with the SSVEP system (steps 12.3 to 12.7) following the end of the rest period.
8. Allow the participants to rest for 5 min before proceeding with the next EEG system.
9. Set up the clinical EEG amplifier as per IFU: power on the amplifier by connecting to a computer via USB dongle provided and power cable, connect five electrode leads into the respective 10-20 EEG positions of 01, O2, OZ, P1, and P2, open clinical neurology EEG software on a computer and create a new study (via the Record tab).
   1. Apply the scalp preparation gel to the occipital region of the participant's head, rubbing the gel in a clockwise motion with fingers to spread into a thin layer.
   2. Fit the electrode cap over the participant's head as per the IFU of the clinical EEG system. Apply the conductive gel to the five electrode patches carefully and with clean hands.
   3. Place the five electrode patches onto the participant's scalp in the respective O1, O2, O3, P1, and P2 positions as per the 10-20 EEG positioning nomenclature.
10. Check the clinical EEG amplifier software application's impedance indicator to ensure an adequate connection between the headset and the participant's head before testing. Impedance values of <15 kOhms are considered to be acceptable for EEG application.
    1. In the instance a good contact quality is not achieved for all electrode channels of interest, re-apply the preparation gels and re-check positioning to ensure the sensors are as flush against the scalp as possible.
11. Instruct the participant to hold the front visor of the SSVEP device over their eyes as per step 12.4, while an investigative assistant holds the rear housing unit away from the scalp.
    1. Repeat the instructions outlined in section 12.5.2.
12. Initiate the EEG amplifier's recording by pressing the red Record symbol located on the top-left corner of the clinical neurology software. Immediately following the initiation of the recording, initiate the visual stimulus by pressing the Begin Test button on the iOS application.
    1. Press the red Record symbol to stop the EEG amplifier recording once prompted that the SSVEP system has ceased the first round of stimulus and is awaiting the command to continue.
    2. Restart the EEG amplifier recording by pressing the red Record symbol on the clinical neurology software, and then continue the visual stimulus by selecting the Continue button on the SSVEP iOS app.
    3. Press the red Record symbol to stop the EEG amplifier recording once prompted that the SSVEP system has ceased the visual stimulus.
13. Allow the subject to relax for a minimum of 30 s. In order to obtain a test-retest reliability value, repeat the SSVEP assessment protocol with the clinical EEG system (steps 12.10 to 12.13) following the end of the rest period (two total tests: Test 1 / Test 2).
14. Once the pair of SSVEP assessments have been successfully completed, have the participant remove the front visor from their eyes, and carefully detach the electrode patches from their scalp.
    1. Clean both the SSVEP headset and electrode leads with isopropyl alcohol (70%) wipes before storing both pieces of equipment in their respective cases until their next use.
15. Ask the participant whether they experienced any abnormal reactions to the stimulus, including the presence of a headache or dizziness. Record their responses in a study log before informing them they have completed their participation in the study.

13. Test-retest reliability calculation of the portable SSVEP system

1. Download the raw SSVEP values of both SSVEP primary test sets from the iOS device using a mobile management utility software; results will be outputted as comma-separated value (CSV) files. For identification and storage purposes, save the files using the participant's full name, the type of test conducted and date.
2. Modify the numerical computing script generated in section 8 to process the CSV files with an additional array of SSVEP values from the O_z recording channel.
3. Collate the processed data files in an array of CSV sheets, with values summarized according to their respective category of Test 1 or Test 2.
4. Use the SNR of each SSVEP reading to compare the results of the different tests. Estimate test-retest reliability using a model 2, type 1 interclass correlation coefficient ICC (2,1) with a 95% confidence interval (CI), and α significance level set to 0.05.

14. Test-retest calculation of the clinical EEG system

1. Download the raw SSVEP values of both EEG amplifier primary test sets from the clinical neurology software ; results will be outputted as CSV files. For identification and storage purposes, save the files using the subject name and the type of test conducted.
2. Repeat section 13.2-13.4 using the downloaded EEG amplifier CSV files to estimate test-retest reliability of the Clinical EEG System.

15. Agreement calculation of the portable SSVEP and clinical EEG systems

1. Outline the criteria for determining whether an SSVEP was detected by the EEG system (recommended criteria: if the main signal peaks between 5-35 Hz is 15 ± 0.1 Hz, and if the Z-score of the main peak is above 5).
2. Modify the numerical computing script generated in sections 12 and 13 to additionally determine the peak frequency and Z-score of the SSVEP files that are processed. Formula for Z-score is.
   
   Where:
   Amplitude_Peak = The voltage at the peak frequency (µV)
   Amplitude_Average = The average voltage of all Frequency bands between 5-35 Hz (µV)
3. Conduct a binomial analysis of the EEG systems' ability to successfully detect a SSVEP using the outlined criteria, with binomial probability (%) to be taken as the agreement between the two systems. Preliminary and primary recordings for each system are to be used to calculate the level of agreement across two sets of data, respectively.
4. Use the Z-score of each SSVEP reading to compare the different EEG systems. Express all summarized data as medians with 25^th to 75^th interquartile range (IQR). Use statistical analytics software for statistical analysis of the system processed values.
5. Modify the numerical computing script generated in sections 12 and 13 to additionally determine the average SNR for each frequency that resulted in a data point ranging from 0-25 Hz, of ALL (preliminary and primary) SSVEP files generated by the SSVEP and Clinical EEG System.
   1. Create a power spectrum density (PSD) of both systems normalized average SNR results between 0-25 Hz (see Representative Results).
      NOTE: Create a separate data series for each EEG system and overlay it onto the same PSD.

Results

A total of 65 male rugby players (20.9 ± 2.3 years old) were successfully enrolled in the first section of this study, with all players undergoing a baseline SSVEP assessment (Figure 1). Through the course of the rugby season, 12 participants sustained a potential concussive injury on the field of play and were re-evaluated with the SSVEP system for a post-injury assessment. The team physician evaluated these players using a clinical concussion evaluation protocol and diagnosed these 12 participants as concussed. All twelve were deemed recovered by the physician within the 12 day GRTP time period³⁰. Following the physician's determination that the players were recovered, 8 players were available for an additional SSVEP; categorized as a recovery assessment. Twenty-two players who were not concussed were retested for the study reliability purposes over the course of the season. The remaining participants who were not retested were lost to follow-up. No adverse events following SSVEP stimulation were reported for the duration of the study. The reliability of the SSVEP system used on the rugby players was confirmed by a high intra-class correlation coefficient (ICC) with a 95% confidence interval, equalling to 0.91 (0.79-0.96) for the retested non-injured players (n = 22) and 0.96 (0.74-0.99) for the retested recovered players³³. Datasets in which a good contact quality were achieved were considered for this calculation. This is a result of a few occasions in which participants' hair or skin potential impacted the EEG systems' ability to obtain clean SSVEP (Figure 4).

In order to determine whether the SSVEP produced via this investigative system can be utilized as a biomarker for concussion, the SNR values of the processed results were grouped into baseline (control), concussed, and recovery assessments for comparison (Figure 1). Overall, the median SNR for all control players (n = 65) was 4.80 [IQR: 4.07-5.68], with the average processed EEG of the control group showing a clear 15 Hz peak signal in the respective frequency spectrum³³. A similar response was seen when the average SSVEP values of a separate control group (n = 20; healthy general population) produced by the same visual stimulus but recorded with a different EEG system, were graphed as a power spectrum density (Figure 5). This median distribution and power spectrum density allowed for a clear control to be set for a SSVEP response of a non-injured (non-concussed) player to the investigative setup (Figure 2, Figure 3). The median SNR of all concussed players (n = 12) and the recovered players with available SSVEP assessments (n = 8), was 2.00 [IQR: 1.40-2.32] and 4.82 [IQR: 4.13-5.18], respectively³³. The pilot study observed significant differences in median SNR values (+4.03; p < 0.0001) between the control (baselined) and concussed players. A concussion had a large effect on an SSVEP signal (Cohens, d = 4.03). Interestingly, the recovered group of players were seen to have a minute SNR variance (+0.02; p = 0.0495) just at significance (α < 0.05), with trivial effect compared to the control group (Cohens, d = 0.17)³³. This indicates that following a full recovery, as per the GRTP guidelines³⁰, SSVEP values are equivalent for a concussed and non-injured player. Further, the median SNR was seen to be significantly different (+2.80; p = 0.0002) between the concussed and recovered group of players, demonstrating that the recovery period has had a large effect on the concussed player's SSVEP signal (Cohens, d = 3.60)³³.

A similar response in median SNR variance was seen when comparing only the players who underwent all three forms of testing (n = 8; baselines, concussed and, recovery). A significant change between baselines vs concussed (-2.34; p = 0.0001) and concussed vs recovery (-2.72; p = 0.0002) was observed, whereas minor variance was seen between baselines vs recovery (+ 0.28; p = 0.0495), with a trivial effect between these groups (Cohens d = 0.17). These findings were reinforced when taking the average SNR values of the players who underwent all three forms of testing. The average SNR of these players' baseline, concussed, and recovery readings were 4.45, 2.20, and 4.33, respectively. A significant difference was seen between baseline vs concussed (p = 0.0001) and concussed vs recovery groups (p = 0.0002). The variation in average SNR values between the recovery and baseline group was small, but just at significance (p = 0.0495). Overall, the response to the stimulus was lower in concussed players when compared to their baseline assessment. Following a monitored recovery period, these players were eventually able to generate a response equivalent to their initial (baseline) assessment³³. This demonstrates that a sports-related concussion has an impact on an individual's ability to generate SSVEPs for a minimum period of 12 days. If an individual's SSVEP response was routinely measured in a similar fashion to this protocol (Figure 1): baseline, post-injury, recovery, a health practitioner could potentially utilize the SSVEPs as a biomarker for concussion.

The all-in-one portable SSVEP system (Figure 7A), was used on (n = 20) healthy control subjects from the general population, non-specified to the sport of rugby. Since this is an investigational device with a different electrode system (Figure 7B) and slightly varied stimuli from the initial SSVEP setup, the median and average SNR values were not valid for comparison (Table 1). Similarly, since participants did not engage in sports with a high occurrence of concussion, the SSVEP system was not assessed as a SSVEP marker for concussion. Instead, a test-retest reliability study was conducted to validate the system for future use in large-scale trials (Figure 6). The SSVEP system returned a high correlation of 0.81 (CI: 0.59-0.92), indicating the device is reliable at obtaining SSVEPs (Table 2). Additionally, the accuracy of the systems' EEG technology was validated through an agreement study against a traditional clinical-grade EEG system (Figure 7C), which returned a similar ICC value of 0.83 (CI: 0.63-0.93) (Table 2). The first repetition of testing (preliminary) resulted in 18/20 participants displaying an agreement across both systems for a binomial probability of 95%. For one participant, the devices did not agree due to the SSVEP system detecting a more prominent alpha rhythm than the desired 15 Hz signal response (Figure 8). For the other participant, no SSVEP was identified with the clinical EEG system (Figure 9). However, In the second repetition (primary), all 20 participants had an agreement across both systems for a binomial probability of 100%. The overall accuracy of the two systems to produce a SSVEP is illustrated in Figure 10, which depicts both systems having a prominent SNR solely at the stimulated frequency: 15 Hz. This validates the portable system as functionally equivalent to clinical grade devices that are used to record EEG signals. When taken in combination with the SSVEP system's portability and ease of use, it opens up a variety of applications for the capturing of research quality SSVEP outside of the clinical setting such as in a large-scale SRC case study.

Figure 1: Flowchart methodology of athlete participation in SRC-SSVEP study. The flowchart details screening for participant eligibility and group-allocation throughout the SRC-SSVEP study duration of amateur rugby union players. SRC; sports related concussion. SSVEP; steady state visual evoked potentials. Please click here to view a larger version of this figure.

Figure 2: Visual stimulus component of portable SSVEP system. (A) LCD smartphone with video loaded and displayed, fitted within a cardboard VR frame. The participant is required to hold the VR frame flush against their face and nasal bridge ensuring both eyes are completely enclosed by the frame. (B) Illustration of the visual stimulus; video loop created of alternating white (top row) and black screens (bottom row) at a frequency of 15 Hz. Each screen contained two frames separated by a vertical dividing line aligned with the left and right eye field of view of the VR frame. Each frame contained a focal point in the form of a number at its center which alternated within a range of 1-9 at 5 s intervals. SSVEP; steady state visual evoked potentials. LCD; liquid crystal display. VR; virtual reality. Please click here to view a larger version of this figure.

Figure 3: Wireless EEG component of portable SSVEP system. (A) A 14-channel EEG headset capable of transmitting data wirelessly to a nearby receiver attached to a computer. (B) A visual map of the 14 electrode positions with respect to the international 10-20 EEG system for electrode placement in human EEG studies. Two occipital electrodes (O1 and O2) were utilized as recording electrodes, while two parietal electrodes were utilized as the common-mode subtraction and ground (P3 and P4, respectively) in the SRC-SSVEP study. EEG; electroencephalography. SSVEP; steady state visual evoked potentials. SRC; sports related concussion. Please click here to view a larger version of this figure.

Figure 4: Illustration of the importance of contact quality in SSVEP measurements. SSVEP responses of a single control (healthy general population) subject measured with the SSVEP system, with a set stimulus frequency of 15 Hz and a sampling rate of 250 Hz, when: (A) atypical saline solution is used on electrodes, (B) electrodes are not adequately worked through subject hair to contact the scalp, (C) electrodes are saturated with saline and worked through the hair. Saline is essential to ensure electrical connectivity between the patients' head and electrodes; without it, large-amplitude skin-potential artifacts are observed in a harmonic fashion. Hair acts as a resistor minimizing electrical connectivity between the patient's scalp and the electrodes and hence results in increased noise. Please click here to view a larger version of this figure.

Figure 5: Average SSVEP response of 20 control subjects in EEG validation study. SSVEP responses of control (healthy general population) subjects (n = 20) measured with the SSVEP system, with a set stimulus frequency of 15 Hz and a sampling rate of 250 Hz. Individual SSVEP values were filtered between 5-40 Hz before being fast Fourier transformed and normalized. Average SSVEP of the population are illustrated as a power spectrum density, with the y-axis representing signal amplitude in microvolts (uV). SSVEP; steady state visual evoked potentials. Please click here to view a larger version of this figure.

Figure 6: Flowchart methodology of EEG validation study between two systems. The flowchart details the methodology of validating a portable EEG system against an established EEG reference system: the SSVEP and clinical EEG systems, respectively. Control (healthy general population) participants are screened and randomly assigned a test order and two tests conducted on each system in a test-retest fashion. EEG; electroencephalography. Please click here to view a larger version of this figure.

Figure 7: Electrode overview for EEG validation study between two systems. (A) The improved portable SSVEP system. (B) The international standard 10-20 EEG modified combinatorial nomenclature system. (C) The established clinical EEG reference system. The SSVEP system measures EEG signals through its three occipital electrode channels (O1, O2, and Oz) while utilizing the two partial electrode channels (P1 and P2) as the reference and bias, respectively. The clinical EEG system allows for the measurement of EEG signals through its 40-channel amplifier, which can manually be positioned in the same O1, O2, Oz, P1, P2 arrangement as the SSVEP system for comparison. EEG; electroencephalography. Please click here to view a larger version of this figure.

Figure 8: Power spectrum density of a single control participants' (participant 09) SSVEP response as measured by two EEG systems. (A) The SSVEP system. (B) The clinical EEG system. Both measurements were obtained using the same visual stimulus (from the SSVEP system): a 15 Hz flicking stimulus of white LEDs in an enclosed case. Note how despite the prominent 15 Hz response seen in both systems, the absolute highest peak for the SSVEP system was at 10.5 Hz rather than at the stimulated 15 Hz. As per the criteria of the agreement study, in which the systems must detect the stimulated frequency as the peak (primary) amplitude, this constitutes as a failure. SSVEP; steady state visual evoked potentials. LED; light-emitting diodes. Please click here to view a larger version of this figure.

Figure 9: Power spectrum density of a single control (healthy general population) participants' (participant 19) SSVEP response as measured by two EEG systems. (A) The SSVEP system. (B) The clinical EEG system. Both measurements were obtained using the same visual stimulus (from the SSVEP system); a 15 Hz flicking stimulus of white LEDs in an enclosed case. Note the lack of a prominent 15 Hz response for the clinical EEG system as it is surrounded by noise of a similar magnitude. As per the criteria of the agreement study, in which the systems must produce a response with a Z-score greater than 5, this constitutes as fail. SSVEP; steady state visual evoked potentials. LED; light-emitting diodes. Please click here to view a larger version of this figure.

Figure 10: A visual illustration of agreement between two EEG systems measuring SSVEP of control participants. The average SSVEP response of (n = 20) control (healthy general population) subjects were plotted as the SNR against the frequency range of interest; 5-25 Hz for measurements with the SSVEP (green) and clinical EEG (red) systems. Each control subject produced two datasets for each system in the EEG validation study generating a total of 40 SSVEP datasets for each system. The two systems' illustrated responses were superimposed to visualize how closely they agree in SSVEP measurement when stimulated by the same visual stimulus: white LEDs flickering at 15 Hz for 30 s. The frequency range is filtered to below the predicted 30 Hz harmonic as to solely focus on the primary stimulus response. EEG; electroencephalography. SSVEP; steady state visual evoked potentials. SNR; signal-to-noise ratio. Please click here to view a larger version of this figure.

EEG System	Minimum	IQR 25	Median	IQR 75	Maximum	Mean	Std. Dev.
NC 1	4.402	8.187	9.829	13.667	20.703	11.148	4.577
NC 2	4.509	9.123	11.055	12.586	23.225	11.615	4.213
Grael 1	4.335	7.99	10.171	13.238	21.758	11.36	4.897
Grael 2	4.979	9.002	10.619	12.667	20.177	11.22	3.865

Table 1: SSVEP statistical summary of control participants as measured by two EEG systems. Two SSVEP measurements were conducted on (n = 20) control (healthy general population) subjects using a portable EEG system, and an established clinical EEG system; SNR values were calculated from the SSVEP (with 15 Hz being taken as the primary signal). Statistics were calculated for each measurement dataset, including the minimum, maximum, 25^th and 75^th interquartile range (IQR), median, mean, and standard deviation (std dev). EEG; electroencephalography. SSVEP; steady state visual evoked potentials.

EEG System	Group	N	ICC (95% CI)	Mean time between Test (min)
Nurochek	Control	20	0.81 (0.59-0.92)	0.5
Grael	Control	20	0.83 (0.63-0.93)	0.5

Table 2: Test-retest reliability of the portable SSVEP System and clinical EEG systems. Reliability of the integrated SSVEP and clinical EEG system, were calculated based on inter-class correlation coefficient (ICC) with a 95% confidence interval (CI) from two tests conducted 30 s apart, using the same set of control (healthy general population) individuals (n = 20); ICC (2,1). SNR values (with 15 Hz being taken as the primary signal) of the SSVEP tests were used as the parameter of interest for the ICC calculation. EEG; electroencephalography. SSVEP; steady state visual evoked potentials.

Discussion

This is the first study to develop a protocol that identifies differences in SSVEP responses in healthy male amateur ruby union players at the three stages of a concussion; pre-injury (baseline), concussed, and recovered (Figure 1). The method included the recruitment and screening of 65 participants who were routinely assessed with an investigational SSVEP setup over the course of a competitive season. As the SSVEP setup was relatively simple and portable, all assessments were conducted in a non-clinical environment, demonstrating the potential use as a point-of-care concussion assessment. The study successfully demonstrated that an individual's ability to generate SSVEPs is attenuated following a diagnosed concussion. The depressive impact of a concussion was seen to diminish following a defined recovery period, as seen when SSVEP values returned to a pre-concussed level for each individual. Statistical analysis between the participant groups showed a significance in the SSVEP attenuation effects. The high test-retest reliability in non-concussed participants highlighted the stability of the electrophysiological biomarker in simple and more refined portable SSVEP systems (Table 2). Additionally, the absolute agreement between an SSVEP system and a traditional EEG amplifier, validates the technology for use as a medical aid capable of obtaining research quality EEG signals (Figure 10).

Since this study was dependent on participants volunteering for post-injury as well as repeated assessments during the rugby season, some logistical modifications had to be made to the method. The estimated time periods between baseline and retests had to be flexible to accommodate the participant's schedules. Despite these measures, some players were still lost to follow up for a variety of miscellaneous reasons, including non-related injuries or lack of interest. This resulted in the use of a more comprehensive statistical calculation, ICC, for the device's reliability across weeks. No adverse events to the SSVEP setup were observed. Some logistic issues were encountered that required minor modifications of the protocol: long or thick hair in particular proved troublesome in acquiring good contact between the headset and the participant's scalp. As poor contact would decrease the quality of the EEG readings (Figure 4), participants with long or thick hair were required to brush and hold their hair up and out to the side of their head while the sensors were placed. An additional exclusion criterion was generated due to this issue, where individuals with complex hairstyles (e.g., dreadlocks) were excluded from this study.

As previously outlined in this paper, current concussion assessment tools are highly subjective and are at the risk of manipulation by an athlete that can ultimately hinder a clinician's ability to make a crucially important diagnosis³⁴. Some athlete tracking studies have attempted to investigate a more objective biomarker for concussion through the use of radiological modalities such as magnetic resonance imaging (MRI) and computed tomography (CT). However, these methods only provide information about macroscopic structural injuries such as hemorrhages which vary from the definition of concussion as a functional brain injury^6,35. The results of this study are supported by previous studies that demonstrated VEP to be a functional biomarker³⁶, which is attenuated or delayed in the presence of concussion^21,37,38. While there are similarities in these previous VEP study methods with respect to our physical setup and hypothesis, our study expands on the literature via the use of SSVEP over VEP. Furthermore, the protocol varies by investigating real-time assessments of players at the three stages of a concussion compared to traditional control vs concussed case studies. Additionally, the method extends its investigative power by comparing innovative and traditional EEG systems to distinguish potential dissimilarities that may limit their accuracy in obtaining objective electrophysiological measurements. Thus, the protocol used in this study provides a unique and valuable addition to existing literature on objective concussion biomarkers.

Despite the overall success of this protocol, there are several limitations to note. For example, a small degree of intra-participant variability in background EEG noise for assessment conducted in immediate succession was noted. Two protocol-design limitations may prove at fault for this first variability: the first being the 14-channel EEG system's lack of high-fidelity impedance feedback and loose constraints for the effects of fatigue and environmental influence on the subject's attentiveness. While this intra-participant variability was not seen with the other EEG systems used in this protocol, it is worth exploring these effects in more detail to confirm their cause is a result of the headset's design and not an unidentified natural occurrence. Second, most participants had larger SSVEP signals after the second assessment versus the first (Table 1). This may be the result of participants becoming more familiar with the assessment process and consequent behavioral adaptations to the equipment setup, including reduced blinking and restlessness during repeat stimulus presentation. Further studies are needed to determine whether there is indeed a familiarization effect to the SSVEP protocol and, if so, what potential modifications need to be made to reduce its occurrence in future studies. Lastly, it is important to note that due to the extensive dependence on volunteers from a relatively small population of individuals (those at high risk of concussion occurrence with the willingness to be repeatably examined), this study was limited to a small sample size of 65 participants, 12 of whom suffered a concussion. Studies with a larger cohort size will be needed in order to evaluate the robustness of this protocol's assessment of concussion, particularly its sensitivity and specificity. It would also be interesting to see this protocol replicated in a range of age groups whose brain development states vary, from those still developing (adolescents) to those with potential cognitive decline (elderly) and delineate whether or not responsivity significantly differs. With respect to the improved SSVEP system, its comparative study highlighted the in-built limitations of the device in comparison to traditional EEG systems. Traditional EEG systems generally adopt the full 10-20 system of montages, which comprise 21 electrode sites (Figure 7B). The SSVEP system on the other hand only uses three electrode channels (O1, O2, and Oz) corresponding to the visual cortex (Figure 7A). This reduction in capability means the system has a narrower scope of EEG applications and limits the potential analysis that can be conducted on the electrophysiological data obtained within this protocol.

As previously mentioned, further research is required to overcome the limitations of this protocol and test its strength on a larger cohort to assess whether its outcomes are able to be generalized. More importantly, additional studies are required to better understand the mechanisms underlying our finding in SSVEP attenuation. For instance, the changes in SSVEP response found in our concussed participants are most likely representations of disturbances in neuronal function, but it is not yet established whether these are primary (e.g., damaged white matter) or secondary (e.g., neuroinflammatory) phenomena. One potential future application of this method is the investigation into the recovery period associated with neuronal depression and concussion individualized to the subject. A deeper insight into this recovery period may see amendments made to sports return to play (RTP) rules and regulations that better protect an injured athlete. This method also introduces the practicality of a portable SSVEP system applied in non-clinical settings, such as a concussion assessment delivered expediently on the sideline of a sporting field. This has the potential to provide significant benefit to not only medical professionals, but coaches, athletes, and their respective families to address the negative physiological effects of concussion and Second Impact Syndrome^10,11. The generation of improved SSVEP systems, such as the portable SSVEP system utilized in this study, may see more advanced equipment and technological applications arise in the field of neurophysiology and SRC that will prove beneficial for the success of future studies.

In summary, this protocol proved successful in its aim of identifying SSVEP as an objective biomarker for concussion in contact sport athletes. The study as a whole provides evidence that SSVEP are significantly attenuated in the presence of a concussion and are capable of being reliably produced at a research quality level through a simplified portable EEG system. We, therefore, propose that SSVEP may be used as a supplemental aid for the assessment of concussive injuries, in particular, the sideline assessment of SRC. Further studies with more refined protocols, advanced techniques, and improved equipment may build upon this study and provide critical information to combat the detrimental effects of concussions on athletes' lives.

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Ag-AgCl Electrodes	Compumedics	97000153	Disposable EEG electrode Wires
Cardboard VR	Google	87002823-01	VR Frame
CaviWipes	Metrex	13-1100	Disinfectant Wipes
Emotiv Xavier	Emotiv	EMO-BCI-ONET-MAC-01	EEG Headset Software / Contact Quality
EPOC Felt Sensors	Emotiv	EMO-EPO-FELT-00	EEG soft electrode contacts
USB Reciever Universal Model	Emotiv	EMO-EPO-USB-04	Signal Reciever for 14 channel EEG Headset
EPOC+	Emotiv	EPOC+ V1.1A	14 Channel EEG headset
Excel 2016	Microsoft	KB4484437	Spreadsheet Software
Grael 4K EEG Amplifier	Compumedics	928-0002-02	Clinical EEG / 40 Channel EEG Amplifier Unit
iPad 5th Generation	Apple	A1822	iOS Device
iPhone 6s	Apple	A1633	iOS Device
iTunes	Apple	V12.5.5.5	Mobile Device Management Utility
MATLAB	MathWorks	R2015b	Numerical Computing Software
Nurochek iOS App	HeadsafeIP	HS02	SSVEP iOS App Software
Nurochek System	HeadsafeIP	HS01	Portable SSVEP System
Polyurethane Sensor Cylinders	Headsafe	HSIP01-213	EEG soft electrode contacts
Profusion EEG 5	Compumedics	AH744-00	Clinical Neurology Software for EEG Amplifier
Quik-Gel Electrolyte	Compumedics	92000016	EEG Conductive Gel
Renu Fresh Solution	Bausch+Lomb	435720	Saline Solution
SPSS 24	IBM	CRZ0WML	Statistical Analytics Software
Ten20 Paste	Weaver	92100031	EEG Skin Prep Gel/Paste
Vaio Pro 11	Sony	SVP1132A1CL	Computer / Laptop
Xperia Z1	Sony	C6906	LCD Smartphone