Utilizing Electroencephalography Measurements for Comparison of Task-Specific Neural Efficiencies: Spatial Intelligence Tasks

Summary

This manuscript describes an approach to measure neural activity of humans while solving spatially focused engineering problems. The electroencephalogram methodology helps interpret beta brain wave measurements in terms of neural efficiency, with the aim of ultimately enabling comparisons of task performance both between problem types and between participants.

Abstract

Spatial intelligence is often linked to success in engineering education and engineering professions. The use of electroencephalography enables comparative calculation of individuals' neural efficiency as they perform successive tasks requiring spatial ability to derive solutions. Neural efficiency here is defined as having less beta activation, and therefore expending fewer neural resources, to perform a task in comparison to other groups or other tasks. For inter-task comparisons of tasks with similar durations, these measurements may enable a comparison of task type difficulty. For intra-participant and inter-participant comparisons, these measurements provide potential insight into the participant's level of spatial ability and different engineering problem solving tasks. Performance on the selected tasks can be analyzed and correlated with beta activities. This work presents a detailed research protocol studying the neural efficiency of students engaged in the solving of typical spatial ability and Statics problems. Students completed problems specific to the Mental Cutting Test (MCT), Purdue Spatial Visualization test of Rotations (PSVT:R), and Statics. While engaged in solving these problems, participants' brain waves were measured with EEG allowing data to be collected regarding alpha and beta brain wave activation and use. The work looks to correlate functional performance on pure spatial tasks with spatially intensive engineering tasks to identify the pathways to successful performance in engineering and the resulting improvements in engineering education that may follow.

Introduction

Spatial ability is vital to Science, Technology, Engineering, and Math (STEM) fields and education and correlates with success in these areas^1,2,3. Therefore, it is important to understand the development of how spatial ability impacts problem solving⁴. Spatial ability has been linked to interest⁵, performance⁶, success in engineering academics⁷ and success in engineering professionals⁸. However, there is not a lot of work indicating specific neural processes in solving problems typical to many spatial ability instruments, nor specific engineering content that is highly spatial.

This paper provides an introduction to methods used for data collection and analysis of spatial ability instrument scores combined with neural measurements. The intent of publishing with JoVE is to make these methods more accessible to a broader audience. General public hardware and software were utilized in this study. As a methods paper, full results/data sets are not reported, nor are multiple samples provided. All images were captured specifically for this publication. The methods detailed below were utilized in preparing a preliminary conference report⁹ based on data from eight college sophomore-aged participants, three of whom were female.

Many existing instruments are used to indicate levels of spatial ability inherent to or learned by individuals. Two valid and reliable^10,11 instruments that are commonly used are the Mental Cutting Test (MCT)¹² and the Purdue Spatial Visualization test of Rotations (PSVT:R)¹³. While originally occupationally designed¹⁴ these instruments test different stages of spatial visualization development described by Piagetian theory^10,15. The use of these instruments creates a need to understand the underlying physiological cognitive phenomena existing when individuals work through these problems. For this reason, this study aims to showcase methods utilizing empirical physiological data that may ultimately improve the analysis and understanding of spatial thought, verify existing metrics testing capabilities, and increase the applicability of spatial assessments to more complex problems typical to engineering education. Many of these problems can be encountered in engineering Statics.

Statics is a foundational mechanics course delivered to most engineering students (e.g., Biological, Mechanical, Civil, Environmental, Aerospace Engineering)^16,17. It is one of the first extensive problem solving experiences that students are given in core engineering content¹⁸. Statics involves the study of the interaction of forces on a rigid body that is at rest or moving at a constant velocity. Unfortunately Statics has high dropout, withdrawal, and failure rates (14% as seen in the investigated University) and this may be related to traditional lecture and curriculum delivery models that omit key avenues of support such as spatially enhanced approaches to education. For example, spatially enhanced approaches in Statics can target the visualization of how forces interact outside of typical analytical analysis and reinforce students' procedural knowledge with grounded conceptualization. The effectiveness of such interventions needs to be investigated from a cognitive neuroscientific perspective.

Electroencephalography (EEG) presents a unique and mobile method of measuring students' brainwave activity. Individuals performing tasks who elicit beta activation are generally very engaged with the task specifics and are attentive to what they are doing^19,20. As task demands increase, the amplitude of the beta wave increases, as does the size of the cortical area the bandwidth frequencies occupy. The more neurons that fire within the beta frequency range (alpha: 8 - 12 Hz, beta: 12 - 24Hz) can be defined as greater beta power. Relatedly, as one becomes more experienced in a task, the amplitude of beta waves decreases, generating less beta power. This is part of the neural efficiency hypothesis^21-28, in which greater task experience when performing a task is related to a decrease in frequency power. Although EEG has previously been used in the study of spatial abilities (often for mental rotation and spatial navigation tasks) — and applicable data have been identified in the alpha, beta, and theta bands^27-33 — alpha and beta bands were observed for this study, and beta was selected for further representative analysis in this paper and in the preliminary conference report⁹. The procedures defined below thus focus on beta band analysis, but an investigation into all three bands, depending on the logged data, is recommended in the future.

The neural efficiency hypothesis has been tested on various tasks, including chess, visuospatial memory, balancing, and resting. All have indicated task experience as a factor in decreased frequency power when performing familiar tasks. One particular study²⁵ has presented evidence that, although the intelligence of a person (as measured by IQ) can help the individual acquire the skills to perform a task, experience with the task outweighs intelligence in its contribution to neural efficiency. In other words, the more experienced an individual is, the more neurally efficient he or she becomes.

Existing neural efficiency studies involving spatial ability have primarily focused on spatial rotation, and different problem sets have been used to compare different populations (e.g., male/female)^27-28. EEG studies of spatial ability tasks have also provided insight by comparing performance to other task types (e.g., verbal tasks)^27,29,30. The methods discussed in this paper focus on and compare problems from the MCT, PSVT:R, as well as static equilibrium tasks, which are related to spatial ability but are not limited to spatial rotation and navigation. Other spatial tasks may be used in place of the ones given as examples in this manuscript. In this way, additional insight may be obtained in the future regarding different populations (e.g., male/female or expert/novice) to ultimately help improve engineering educational practices.

In an effort to investigate spatial ability and engineering aptitude, we have developed a protocol utilizing EEG measurements to identify the beta wave activations of low performing to high performing participants during a limited battery of specific spatial and engineering tasks. In this case, the term high performer is related to the performance of the participant, and is not reflective of the amount of time spent in the field by the learner, as all participants were at approximately the same point in their education. Additionally, the problem set involved is quite specific and basic; thus the terms "expert" or "high performing" herein must not be viewed in the sense of an expert, professionally employed engineer, but representing only high performance in this narrow slice of engineering mechanics curriculum and spatial ability instruments. The neural measurements can also be used to identify any gross trends for which task types may recruit more cognitive resources than others, with possible interpretation regarding levels of difficulty. This information may potentially provide insight into future assessment and intervention with regard to spatial ability. Other future insight may be derived by considering more specific regions of the brain, which was not possible in this study due to the limited number of channels available in the EEG hardware used.

Protocol

Ethical Statement Regarding Use of Human Participants

Procedures involved in this work have been approved by the Institutional Review Board (IRB) at Utah State University for the study of human subjects. It is recommended that any similar work should also be approved by the relevant IRB. Participants are allowed to stop or withdraw from the study at any time during the experiment.

1. Selection of Participants

1. Select participants on a voluntary basis from students currently enrolled in a Statics course. Ensure that the participants have been previously exposed to the Statics content they will see while in the study.
   Note: Ideally participants should have exposure to the material via lecture at least four weeks prior to participation in the study so that there is time for consolidation of concepts and development of differential performance levels.
2. Conduct recruitment following all IRB protocols. Explain all procedures and restrictions to potential participants in detail. Describe the scope of future contact and how the results of the study may be published and/or made available to the participants. Define any compensation for participation in the study. If a volunteer chooses to enroll in the study, provide him or her with a double-coded identification code only known to and controlled by the principal investigator.
   Note: Specific population recruitment will require statistical analysis to verify the statistical power of the sample and number of required participants to analyze individual differences in the brainwave data. If the researcher desires to conduct a comparative study between populations, then a power analysis should be conducted to develop appropriate participant group sizes for both populations.
3. Select activities or problems that are representative of the experiences desired for the participants.
   Note: In this protocol, sophomore-level engineering students enrolled in a Statics course were selected. The activities deemed relevant were problems from introductory Statics content as well as two commonly used spatial ability instruments: the Mental Cutting Plane Test (MCT) and the Purdue Spatial Visualization Test (PSVT-R). Each instrument tests different constructs of spatial ability and the level of spatial development in this selected student population.
4. Establish a calendar to organize the laboratory sessions for the participants.

2. Preparation of Instruments

1. Set up the EEG headsets (aka caps) per manufacturer instructions. Complete this preparation before the participant arrives for the study. To reiterate, this procedure is specific to general mobile headsets such as Emotiv, as opposed to medical grade EEG setups.
   1. Charge the EEG headsets — ideally at least a 1-hr charge per session. For a regularly used laboratory, have at least two headsets charging at all times.
   2. Place all required liquids in an accessible area, including the liquid for dampening the EEG electrodes (e.g., water) and an abrasive cleaner for ensuring good contact for the EEG reference nodes.
   3. Insert the felts into each casing (ensuring the gold contact is seated securely in each casing). Saturate the felts with the dampening liquid using a syringe. Allow dampened electrodes to rest.
2. Set up any required video cameras to measure participant behavior. For the current protocol, use two video cameras per participant. Re-adjust cameras once the participant is in place. Be sure that the video is time-stamped.
   1. Focus one camera on the participant's face if aiming to record facial expressions and obtain high quality audio.
   2. Focus the other camera on the area in front of the participant to capture hand movement, including handwriting actions (if handwriting is expected, make sure to use a writing instrument that is dark and/or thick enough to be captured by the camera), and enough of the computer monitor to provide insight as to which task is being solved at which time.
   3. Turn on the computer and verify that the software is on to record brainwave data. Verify that all software and data collection devices are interfacing sufficiently for data collection.

3. Preparation of Study Participants and Session Commencement

1. Verify the receipt of documented consent from participants per the IRB agreement discussed above. Answer any questions the participants have prior to beginning the study. Remind the participants that data collected will be referenced by ID code and there will be no identifying information that ties the data to the participant, and that they may withdraw at any time.
2. Ask each participant to fill out a demographics survey prior to participation in the study. This survey may ask about gender, age, previous experience that may impact their abilities in the study (e.g., past engineering or spatially intensive courses, spatial ability enhancing hobbies, and questions regarding exclusion criteria such as traumatic brain injuries they may have suffered, which hand they use).
   1. Exclude participants from the volunteer group for the EEG analysis if any of the following conditions exist: (a) the participant is left handed or ambidextrous, in order to control for brain laterality confounds; (b); the individual cannot participate in the lab sessions due to a physical disability; or (c) the individual has suffered serious brain injury. Notify potential participants of these limitations during the recruitment process, or as early as possible to avoid spending unnecessary time and resources.
3. Upon arrival, ensure the participant is comfortable and resolve any remaining questions or concerns.
   1. Demonstrate the syringe used to saturate the node and explain that it will only be used to keep the EEG felts damp. If the participant has an extreme fear of needles, consider implementing other precautions (e.g., keeping the syringe out of their focal plane when re-wetting the felts).
   2. Ask the participant to remove any electronics from their person.
4. Place the EEG headset on the participant.
   1. Check the felts for dampness and place the felt/casing combinations into the EEG headset.
   2. Clean the reference points (e.g., mastoid process) of the participant with the abrasive cleanser. Wipe away any residue.
   3. Place the headset on the participant with the reference nodes appropriately aligned with the reference points. Do not excessively bend the arms of the headset. Leave a gap between the reference node and the back of the ear so as to not cause discomfort, and align and space the headset appropriately with the participant's head.

4. Software Execution within the Session

1. Start the EEG-logging software. Ensure that good connectivity exists between the logging device (e.g., personal computer) and EEG headset by checking that all of the channels are displayed on the logging device. Check that all channels initially display similar behavior with low amplitude oscillations. Check the EEG to ensure good connectivity with the participant — re-wetting and adjusting the felts so as to achieve consistent patterns on the logging device — immediately preceding the rest periods and before the start of each new problem type.
   Note: The EEG operates at 128 Hz. Electrooculography was not used to record eye movement, and linked-ear reference was not used.
2. Instruct the participant to remain as still and quiet as possible during the task exercises.
3. Initiate the task-presentation software.
   Note: During data collection, all pre-planned visual communication with the participant occurs via the computer monitor. In this case, a series of spatial and engineering problems will appear on the computer screen, and participants will be asked to solve them. Correct answers were not provided to the participants during data collection. The problem images advanced based on user input, so timing was based on problem-solving duration.
   1. Display spatial problem type 1 (e.g., PSVT:R — a multiple choice test, or true-false rotation problems — see Figure 1)¹³. Note: The duration of these problems (e.g., 30 sec) will be used as the time range for the data analysis. Five problems were included in this set.
   2. Display spatial problem type 2 (e.g., MCT — a multiple choice test, or true-false mental cutting problems — see Figure 2)¹². Note: The duration of these problems (e.g., 30 sec) will be used as the time range for the data analysis. Five problems were included in this set.
   3. Display engineering problem type¹⁷ (e.g., Statics problems — broken down to focus on specific principles of engineering Statics, or any other applied problem type hypothesized to have spatial components — see Figure 3). Note: These problems take significantly more time to solve than the spatial problems. The number of problems shown to participants varied from four to ten.
   4. Assign rest periods at the beginning and the end of data collection — used for obtaining baseline data. Ensure that each of these have the same duration (e.g., 120 sec).
4. If desired, conduct an exit interview with the participant. This may include their thoughts on the experimental presentation, wearing the EEG headset, the communication process used throughout recruitment and participant preparation, and/or any protocol requiring verbal answers mentioned above. A validated user questionnaire may be provided to the participants instead of conducting an interview.
5. Turn off the task-presentation software, the EEG-logging software, remove the EEG headset, and turn off the video-recording equipment.

5. Conclusion of the Session

1. Dismiss the study participant. Thank the participant and provide them an overview of any future contact (e.g., for follow-up interviews or subsequent sessions of the study), explain how the results of the study may be published and/or made available to the participants, and provide any refreshments or payment (or an explanation of how the payment will be provided) agreed upon as part of the compensation for participation in the study.
2. Transfer data logs to any required long-term or transfer storage devices. Store the signed consent form appropriately and as designated by IRB protocol.
3. Clean up the instruments and laboratory space.
   1. Remove the felts from the headset and sanitize or dispose of them.
   2. Return the EEG electrode casings and headset to the proper storage location.
   3. Dispose of used syringes and trash appropriately.
   4. Return liquids to appropriate storage locations.
   5. Secure the lab if not being used by other researchers.

6. Data Analysis

1. Identify and extract the raw data for each channel and the marker data from the EEG data logs. Use ASCII bit markers to identify the beginning and end of data collection, as well as the transitions between different phases of data collection (e.g., problem types) and individual problems. Ensure that each phase type has a different marker value so as to enable differentiation during analysis. Name the data in a manner that references the participant ID code as the source.
   Note: EEGLAB commands are defined herein, but EEGLAB requires MATLAB for this execution.
   1. Click File > Import Data > Using EEGLAB functions and plugins > From EDF/EDF + GDF files (BIOSIG toolbox)
   2. Select the appropriate data file. Click Open to load the data.
   3. Select the Channel List. Click Ok to accept.
   4. Provide a Dataset Name. Specify a descriptive name for the data that reflects the source and date of collection. In this case, PSF1448 indicates data from Participant ID 48 in the Fall of 2014.
2. Map the extracted data to the montage (i.e., the layout of the EEG nodes) by selecting the montage provided by the EEG headset vendor (e.g., a 10 - 20 system). Ensure the montage being used for the analysis matches the layout of the EEG headset utilized during the session. This is manufacturer-specific.
   1. Edit channel locations by clicking Edit > Channel Locations.
   2. Select Montage. In this case the default montage is appropriate, so simply click Ok to accept.
   3. Select Channel Information Specification. In this case the default is appropriate, so simply click Ok to accept.
3. Reduce the EEG channel data to that which is most representative of brain activity, as defined below.
   1. Apply an initial filter to the data. Typically, apply a high-pass, low-pass filter (with 0.1 Hz as the lower bound of the high-pass filter and 59 Hz as the upper bound of the low-pass filter). Applying a low-pass filter less than 60 Hz removes the noise from the U.S. electrical grid. Save the dataset with a new name as a restore point.
      1. Click Tools > Basic FIR Filter (new, default).
      2. Set Basic Filter Parameters. Set Lower edge to 0.1 Hz, Higher edge to 59 Hz, do not plot the frequency response, and click Ok to accept.
      3. Specify a new name for the filtered data (by appending "_filtered" to the existing dataset name). Check the box to save the data as a file, and use the same name for the filename. Click Ok to accept.
   2. Remove any data that lie before the first EEG marker or after the last EEG marker — keeping in mind any latency in recording the EEG markers. Record the latency (time) of the marker indicating the initiation of data recording and the latency of the marker indicating the end of the data. Save the dataset with a new name as a restore point.
      1. The latency values can be found in the "Edit event values - pop_editeventvals()" screen; press the ">>" button to go to the final marker signifying the end of the EEG data. No change needs to accepted, so click Cancel once the values have been recorded.
      2. Click Edit > Select data.
      3. Enter the beginning and end latency (time) values, separated by a space, in the "Time range [min max] (s)" field, and click Ok to accept.
      4. Specify a new name for the cropped data (by appending "_cropped" to the existing dataset name). Check the box to save the data as a file, and use the same name for the filename. Click Ok to accept.
   3. Reject sections of data with large artifacts. The steps included below describe how to do this manually while visually inspecting the data. Note: the removal of data will also cause artifacts^34,35.
      1. Normalize the data in each channel (remove the mean and put each channel on the same scale). Also remove the DC offset (this alters the data, but not the visualization).
         1. Click Plot > Channel data (scroll).
         2. Click Settings > Time range to display.
         3. Specify the time range (e.g., 30 sec) to be shown in the plot in the "New window length (s):" field. The time range is based on the time between markers for problems within a given phase (or within the two spatial problem phases). The time range can be based on the maximum, minimum, or average time between markers.
         4. Click the "Norm" button to normalize the data in the plot (this is cosmetic only and does not alter the underlying data).
         5. Click Display > Remove DC offset to remove the DC offset in the plot (this is cosmetic only and does not alter the underlying data).
      2. Remove large artifacts that are not repeated regularly over time.
         1. Mark all of the abnormal looking artifact data. Once all artifact data have been marked, click the Reject button.
            Note: These can appear as abnormally high or broad peaks in the data — in single or multiple channels — or as long trends that appear in a small number of channels. Data are suspect if data from separate channels appear to cross one another in the plot. These represent artifacts that are not part of the brainwave spectrum and most likely represent muscle movement by the participant or a node(s) with poor connectivity. Anything resembling a square wave is not representative of human brain activity.
      3. Save the dataset with a new name as a restore point.
         1. Click File > Save current dataset as.
         2. Specify a new name for the cropped data (by appending "_manRej" to the existing dataset name). Click Save to accept.
      4. If a particular channel appears to be faulty, remove the data from it individually. This represents a big loss in data, so do so with great caution. Look at the data from the channel over a long time period, as it often settles over time and provides useful data.
   4. Run an Independent Component Analysis (ICA) and select the best representations of brainwave activity.
      Note: This aids in the removal of sets of repetitive artifacts in the data. These sets contain artifacts that will appear multiple times at roughly regular intervals with a repeated shape. Typically they are a result of biological functions such as blinking or pulse — each of which will have its own set.
      1. Map the ICA-separated data to a representation of the cranium based on the montage. Reject results associated with blinking, pulse, or muscle tension — which will often appear in the ICA results as areas of emphasis above the eyes, near the temples, or over the ears, respectively. Reject any component that shows the entire cranium as being engaged since it not representative of brain activity (see Figure 4).³⁵ Accept other results (see Figures 5 - 6).
         1. Click Tools > Run ICA.
         2. Select the default (runica) ICA algorithm. Click Ok to accept.
         3. Click Plot > Component properties.
         4. Select the component indices (the 14 EEG electrode channels loaded into memory) and the spectral options. As before, the lower edge is 0.1 Hz, and the higher edge is 59 Hz. Click Ok to accept entries.
         5. Within the Accept/Reject window, click on the Accept button to change status to Reject (and click it again to change it back to Accept). Click Ok to log the Accept/Reject labeling.
      2. Plot the ICA-separated data in a 2-D color plot. Reject results that appear streaky, blank, or peppered with discontinuities, then save the dataset with a new name as a restore point (see Figures 5 - 6).
         1. Within the Accept/Reject window, click on the Accept button to change status to Reject (and click it again to change it back to Accept). Click Ok to log the Accept/Reject labeling.
         2. Click Tools > Remove Components — to actually remove the data marked for rejection previously.
         3. Click ok to continue. The indices of components logged for rejection are shown in the "Remove components" window.
         4. Click Accept in the "Confirmation" window to continue with the pruning of the data.
         5. Specify a new name for the pruned data (by appending "_manRejPruned" to the existing dataset name). Check the box to save the data as a file, and use the same name for the filename. Click Ok to accept.
            Note: Streaks that last longer than 0.5 sec are considered reasonable for rejection. Relative "goodness" may need to be utilized here, depending on how good other data sets appear to be — it is desirable to keep at least half of the components. Good results are often represented by continuous gradations on a 2-D Continuous Data color plot³⁴.
4. Remove the boundary values left in the data. Save the dataset with a new name as a restore point.
   1. Click Edit > Event values.
   2. Scroll through the events and click the Delete event button when the event type is a boundary. When all have been removed, click Ok.
   3. Specify a new name for the deleted-boundary data (by appending "_deleteBoundaries" to the existing dataset name). Click Save to accept. Use same menu item selection as before to get to this screen (see step 6.3.3.3.1).
5. Calculate absolute power metrics for each exercise type. This is a power-based logarithmic transform based on the microvolt measurement and the time — calculated for each frequency band (Delta, Theta, Alpha, Beta, and Gamma).²²
   1. Chunk the data into blocks, using markers to indicate the beginning and end of each task.
      1. Click Edit > Select data using events.
      2. Utilize appropriate timeframes for each task type. Define the timeframe for the rest periods by the duration of a rest period. For the spatial problems (which are roughly similar in duration), use either the average duration of all spatial tasks or the maximum duration of all spatial tasks. For the applied (e.g., engineering Statics) problems, identify the average duration for each problem. Save the dataset with a new name as a restore point.
         1. Enter the marker type in the "Event type(s) ([]=all)" field, (e.g., marker type 50 was used to mark rest events). Rest events had a duration of 120 sec in this case, so enter "1 120" for the time limits array. Click Ok to accept.
         2. Specify a new name for the event data (by appending "_rest" to the existing dataset name in this case). Check the box to save the data as a file, and use the same name for the filename. Click Ok to accept.
            Note: If the applied portions can be reduced so they take roughly the same amount of time as the spatial tasks, then use the same timeframe size as the spatial tasks. Since EEG is a time-sensitive measure, the more accurate the time epochs are for each condition, the less confounded the data are in the end (i.e., the number of samples collected for each condition will be more consistent).
6. Compare results for final analysis.
   1. Calculate the percentage for each chunk relative to the baseline rest measurements. See the Supplemental Code File, and Tables 1 - 8.
      1. Open the Absolutepower Script in MATLAB and click the Run button to run the script on the data loaded in the workspace during step 6.6 (e.g., the rest data).
      2. Select absolutepowermatrix data in the MATLAB Workspace for transfer to a spreadsheet program (e.g., MS Excel).
   2. Repeat steps 6.5 - 6.6.1.2 for every exercise/marker type.
   3. Compare results with similar timeframes (e.g., spatial tasks) to one another for insight into relative difficulty.
   4. Compare results across participants to identify higher relative performers versus lower performers in the skills being assessed. Note: High performers may show very little increase in the beta activation relative to the baseline, while low performers may show an increase on the order of 70%^21-26.

Representative Results

In this section, the preceding steps are illustrated with sample figures as described below. Full data summaries with statistical tests are not provided, as the objective of this paper is to focus on methods. Examples of potential PSVT:R, MCT, and Spatial problems are given in Figure 1, Figure 2, and Figure 3, respectively.

The EEG cap will collect brain activation via electrical potentials for each given channel, which can be viewed in parallel as shown in Figure 7. As mentioned previously, certain artifacts within the data need to be manually removed, while others can be removed via ICA. At times a faulty channel can be identified. Such artifacts are visible in Figure 7. In the analysis software, the large, non-repetitive artifacts can be manually marked in sequence and then removed by clicking the "REJECT" button (as in step 6.3.3.2.1). All figures with images of EEG data analysis are from the analysis software tool listed in the Table of Materials.

Following the ICA, the analysis software maps the data in two ways: 1) A scalp-mapped representation of activation, and 2) A 2-D Continuous Data plot of activation arrayed by Trials and Time. An example of acceptable data can be observed in Figure 5. An example of rejected scalp mapped data indicating activity not associated with the brain for three cases can be seen in Figure 4. 2D Continuous Data plots for those same rejected three cases can be seen in Figure 6. The streaking observed in the first two plots warrants consideration for removal. The streaking in the third plot may be considered borderline — 2-D Continuous Data plots of this quality may be considered for inclusion, and the researcher must consider the balance between including spurious signals and discarding valuable data. Streaks longer than 0.5 seconds are considered grounds for rejection. For more insight, refer to the EEGLAB website (http://sccn.ucsd.edu/eeglab/).

Once all confounding data have been rejected — either through manual rejection while looking at the brainwave plots or after ICA — and the data have been chunked in time for the appropriate activity type, the absolute power calculations can be made for each frequency band and each activity type via the MATLAB script (based on the analysis software functions) given in the Supplemental Code File. The summary data then generated by the function are shown in the tables below. Table 1 contains the data from the Rest time periods — which are used as the baseline for the efficiency calculations. Table 2, Table 3, and Table 4 contain the absolute power data for the PSVT:R, MCT, and Statics problems, respectively. By dividing by the cell value for the corresponding channel and frequency band in the Rest table, the relative absolute power ratios are shown in Table 5, Table 6, and Table 7 for PSVT:R, MCT, and Statics problems, respectively.

Ultimately, the average value across all channels is taken for the beta frequency band for each activity type, and the results are shown in Table 8. This type of data can be used to identify ROIs for future research. From these data for the participant in question, we see that the relative absolute power appears lower for the PSVT:R than for the MCT. Decisive conclusions regarding this statement, though, remain dependent on a larger sample size to establish possible statistical significance. The relative absolute power for Statics tasks may be compared to the value from other participants, and estimates of high performer vs low performer cognitive exertion may be identified which could be correlated with functional scores on the Statics problems for validation.

Although this is specifically a methods paper, and presents examples of data from only one participant, the preliminary report's statistical analysis used a Levene's test to assess normality, followed by Friedman's test comparing group x task x EEG channel. Finally, a follow-up Wilcoxon test was performed on significant Friedman effects and interactions. The comparison between high and low performers showed significantly higher beta activation levels for low performers than for high performers (For PSVT:R, F3: χ²(1,6) = 5.33, p < .03; T8: χ²(1,6) = 4.08, p < .05; FC6: χ²(1,6) = 4.08, p < .05; F8: χ²(1,6) = 4.08, p < .05; AF4: χ²(1,6) = 5.33, p<.03. For MCT, F3: χ²(1,6) = 5.33, p < .03; T8: χ²(1,6) = 5.33, p < .03; FC6: χ²(1,6) = 5.33, p < .03; AF4: χ²(1,6) = 4.08, p < .05. For Statics, FC6: χ²(1,6) = 4.08, p < .05). ⁹

Figure 1: PSVT:R Example Problem. Part A demonstrates a single sample PSVT:R Problem as seen by participants. (Source: Guay (1976)) The correct answer is C. Part B provides a visual explanation of the solution. Please click here to view a larger version of this figure.

Figure 2: MCT Example Problem. Part A demonstrates a single sample MCT Problem as seen by participants. The correct answer is D. Part B provides a visual explanation of the solution. (Source: CEEB (1939)) Please click here to view a larger version of this figure.

Figure 3: Statics Example Problem. Illustrates a single example Statics problem given to participants. This problem is for in-plane (i.e., 2-D) equilibrium given three forces and a common connection structure. Please click here to view a larger version of this figure.

Figure 4: Examples of Non-Brain Scalp-Mapped Activity. Three examples of post-ICA scalp-mapped data are shown from an individual, 23-year-old, male participant. Full-scalp activation, activation above a single eye/temple, and activation focused on eyes and temples are indicative of corporal activity, not brain activity, as shown after ICA in IC2, IC3, and IC4, respectively. Please click here to view a larger version of this figure.

Figure 5: Acceptable Post-ICA Data Images. Illustration of acceptable scalp map and 2-D Continuous Data plot after ICA for a sample case, Independent Component 13 (IC13), from an individual, 23-year-old, male participant. Activation appears to be centered on a region of the brain in the scalp-mapped view, and no large streaks are visible in the Continuous data plot. Please click here to view a larger version of this figure.

Figure 6: 2-D Continuous Data Plots Matching Scalp-Mapped Images. Three examples of post-ICA continuous-data plots are shown from an individual, 23-year-old, male participant. Thick bands or streaks in the 2-D Continuous Data plots from ICA indicate discontinuities not indicative of normal brain function in IC2, IC3, IC4 — particularly in IC2 and IC3 plots. Please click here to view a larger version of this figure.

Figure 7: Brainwave Data with Artifacts. Screenshot of an artifact (channel F7) manually marked for rejection with a sample time range from an individual, 23-year-old, male participant. Note the event across multiple channels between 132 and 133: Similar events are repeated multiple times (approximately the same shape and size at regular intervals) — and thus are assumed to represent a non-brain biological function (e.g., blinking) — and can be removed via Independent Component Analysis (ICA). Please click here to view a larger version of this figure.

Supplemental Code File: MATLAB Script and Alteration. Displays the scripts (spectopo.m and absolutepower.m) for the transformation based on the microvolt measurement and the time — calculated for each frequency band (Delta, Theta, Alpha, Beta, and Gamma) — to obtain the absolute power at each frequency²². The code changes required for proper functionality in pop_chanedit.m are also included. Please click here to download this file.

REST		AF3	F7	F3	FC5	T7	P7	O1	O2	P8	T8	FC6	FC4	FC8	AF4
delta		2.92885	4.08477	3.54998	2.34592	2.70998	2.32691	2.68544	4.27085	2.98234	8.86292	6.23237	4.78013	10.8036	3.25063
theta		0.97171	1.37529	1.31051	0.80067	0.86828	0.72737	0.89545	1.47262	0.9612	2.62535	1.81392	1.50252	3.17255	1.07803
alpha		1.05352	1.3154	1.1847	0.65468	0.80063	0.67154	1.02715	2.07336	1.08513	2.66165	1.57996	1.34778	3.03508	1.16919
beta		0.43161	0.90384	0.50791	0.53479	0.50098	0.38674	0.38319	0.58092	0.31785	1.01047	0.56527	0.49346	0.90616	0.48072
gamma		0.5045	1.34183	0.62215	0.84909	0.70052	0.51585	0.43051	0.67612	0.34162	1.03946	0.64008	0.5726	0.91932	0.51616

Table 1: Rest Absolute Power. Contains the absolute power values for the baseline Rest time periods. Values are shown for each EEG cap channel and each neural frequency band. Please click here to download this table as an Excel spreadsheet.

PSVT:R		AF3	F7	F3	FC5	T7	P7	O1	O2	P8	T8	FC6	FC4	FC8	AF4
delta		3.20159	4.9235	4.45167	2.34879	2.42221	2.02463	2.94513	5.43045	4.42694	12.7964	11.31	6.487	21.8189	4.09331
theta		0.96945	1.59045	1.37746	1.03259	0.84002	0.66437	1.07593	1.74327	1.17321	3.7199	2.85166	1.53374	5.03852	1.18174
alpha		0.85227	1.13582	1.02927	0.58288	0.67936	0.58545	0.74962	1.66418	0.99799	2.75755	2.02905	1.36223	3.80233	1.0266
beta		0.35494	0.678	0.40734	0.36971	0.37595	0.30512	0.31952	0.50253	0.28369	0.75791	0.71554	0.42837	1.01529	0.34922
gamma		0.30691	0.74519	0.41486	0.43652	0.39229	0.30623	0.30822	0.4174	0.22447	0.66889	0.70126	0.36895	0.90685	0.30268

Table 2: PSVT:R Absolute Power. Contains the absolute power values for the time periods when the participant was solving PSVT:R problems. Values are shown for each EEG cap channel and each neural frequency band. Please click here to download this table as an Excel spreadsheet.

MCT		AF3	F7	F3	FC5	T7	P7	O1	O2	P8	T8	FC6	FC4	FC8	AF4
delta		4.25246	7.54329	5.08043	5.52389	3.73567	3.26572	3.76397	5.8437	4.62085	18.7991	16.4444	6.24405	28.1184	4.59798
theta		1.19953	1.84997	1.70135	1.27424	1.30572	1.08925	1.09528	1.91699	1.34909	4.19652	3.73398	2.04338	6.21749	1.33753
alpha		1.18154	1.41989	1.23333	0.76868	0.8051	0.6844	1.02368	2.53414	1.29356	2.94347	2.26038	1.4973	3.94919	1.1579
beta		0.44047	0.89503	0.54	0.51125	0.46215	0.36589	0.3884	0.61918	0.35962	1.03223	0.89744	0.54226	1.35175	0.47197
gamma		0.41897	1.05133	0.51015	0.64259	0.51855	0.39244	0.41827	0.52564	0.29925	0.87269	0.84818	0.4996	1.08765	0.41331

Table 3: MCT Absolute Power. Contains the absolute power values for the time periods when the participant was solving MCT problems. Values are shown for each EEG cap channel and each neural frequency band. Please click here to download this table as an Excel spreadsheet.

Statics		AF3	F7	F3	FC5	T7	P7	O1	O2	P8	T8	FC6	FC4	FC8	AF4
delta		7.21032	12.8557	8.50834	7.09116	5.75386	4.80761	6.79589	9.11056	7.39437	23.7659	18.5893	11.7132	32.0165	8.38173
theta		1.64049	3.16334	1.98263	1.70548	1.52057	1.25686	1.61864	2.35557	1.6244	4.85163	3.79464	2.53764	6.50266	1.809
alpha		0.86505	1.37518	1.00568	0.72506	0.76361	0.6491	0.95616	1.63483	0.9386	2.56892	1.67092	1.18895	3.13664	0.98499
beta		0.35583	0.55288	0.41326	0.30866	0.34607	0.29362	0.357	0.59991	0.34927	1.04345	0.66066	0.44385	1.21395	0.42598
gamma		0.24587	0.43744	0.31831	0.23404	0.25428	0.2218	0.26349	0.39275	0.22939	0.7927	0.507	0.29891	0.94462	0.3172

Table 4: Statics Absolute Power. Contains the absolute power values for the time periods when the participant was solving Statics problems. Values are shown for each EEG cap channel and each neural frequency band. Please click here to download this table as an Excel spreadsheet.

PSVT:R %		AF3	F7	F3	FC5	T7	P7	O1	O2	P8	T8	FC6	FC4	FC8	AF4	average
delta		1.09312	1.20533	1.254	1.00122	0.89381	0.8701	1.0967	1.27152	1.48439	1.44382	1.81472	1.35708	2.01959	1.25924
theta		0.99766	1.15645	1.05108	1.28965	0.96746	0.91339	1.20155	1.18379	1.22056	1.41692	1.5721	1.02078	1.58816	1.09621
alpha		0.80897	0.86348	0.86881	0.89032	0.84853	0.8718	0.7298	0.80265	0.9197	1.03603	1.28424	1.01072	1.2528	0.87804
beta		0.82237	0.75013	0.80199	0.69131	0.75043	0.78897	0.83383	0.86506	0.89252	0.75005	1.26584	0.86809	1.12043	0.72645	85.2%
gamma		0.60836	0.55535	0.66682	0.5141	0.56	0.59365	0.71594	0.61734	0.65707	0.6435	1.09557	0.64435	0.98644	0.5864

Table 5: PSVT:R Relative Absolute Power. Contains the relative absolute power values — that is, the ratio compared to the Rest baseline — for the time periods when the participant was solving PSVT:R problems. Values are shown for each EEG cap channel and each neural frequency band. Please click here to download this table as an Excel spreadsheet.

MCT %		AF3	F7	F3	FC5	T7	P7	O1	O2	P8	T8	FC6	FC4	FC8	AF4	average
delta		1.45192	1.84669	1.43111	2.35468	1.37849	1.40346	1.40162	1.36828	1.54941	2.12109	2.63855	1.30625	2.60268	1.41449
theta		1.23445	1.34515	1.29823	1.59146	1.5038	1.49751	1.22317	1.30176	1.40354	1.59846	2.05851	1.35997	1.95978	1.24072
alpha		1.12151	1.07944	1.04106	1.17413	1.00557	1.01915	0.99661	1.22223	1.19207	1.10588	1.43065	1.11093	1.30118	0.99034
beta		1.02052	0.99025	1.06317	0.95599	0.9225	0.9461	1.01359	1.06585	1.13138	1.02154	1.58762	1.09891	1.49174	0.9818	109.2%
gamma		0.83046	0.78351	0.81998	0.7568	0.74023	0.76077	0.97157	0.77744	0.87596	0.83956	1.32511	0.87252	1.1831	0.80073

Table 6: MCT Relative Absolute Power. Contains the relative absolute power values — that is, the ratio compared to the Rest baseline — for the time periods when the participant was solving MCT problems. Values are shown for each EEG cap channel and each neural frequency band. Please click here to download this table as an Excel spreadsheet.

Statics %		AF3	F7	F3	FC5	T7	P7	O1	O2	P8	T8	FC6	FC4	FC8	AF4	average
delta		2.46182	3.14723	2.39673	3.02277	2.12321	2.06609	2.53064	2.1332	2.47939	2.6815	2.9827	2.45039	2.96349	2.57849
theta		1.68824	2.30012	1.51286	2.13005	1.75125	1.72794	1.80763	1.59958	1.68997	1.84799	2.09195	1.68893	2.04966	1.67807
alpha		0.82111	1.04545	0.84889	1.1075	0.95375	0.96658	0.93089	0.78849	0.86496	0.96516	1.05757	0.88215	1.03347	0.84245
beta		0.82441	0.6117	0.81364	0.57716	0.69079	0.75922	0.93164	1.03269	1.09885	1.03264	1.16874	0.89947	1.33966	0.88613	90.5%
gamma		0.48736	0.326	0.51162	0.27564	0.36299	0.42997	0.61205	0.58088	0.67146	0.76261	0.79208	0.52202	1.02753	0.61453

Table 7: Statics Relative Absolute Power. Contains the relative absolute power values — that is, the ratio compared to the Rest baseline — for the time periods when the participant was solving Statics problems. Values are shown for each EEG cap channel and each neural frequency band. Please click here to download this table as an Excel spreadsheet.

	average
PSVT:R %	85.2%
MCT %	109.2%
Statics %	90.5%

Table 8: Averaged Relative Absolute Power. Contains the relative absolute power values — that is, the ratio compared to the Rest baseline — averaged across all EEG cap channels for the time periods when the participant was solving PSVT:R, MCT, and Statics problems. Percentages are shown for the beta frequency band only. Please click here to download this table as an Excel spreadsheet.

Discussion

The protocol discusses the application of electroencephalography to measure brain activity for participants working problems from two typical spatial ability instruments and highly spatial engineering Statics problems. The methods detailed here may ultimately be able to help understand the neural efficiency of high and low performers engaged in working these problems. It is vital to understand any differences in neural efficiencies of engineering students working on the MCT and PSVT:R, as these tests are often used to assess spatial ability. Comparing them to each other allows us to better assess their applicability to success in engineering and their position in foundational engineering curricula.

The protocol establishes procedures for research on neural efficiencies associated with spatial cognition tasks. It is important that reliable and valid instruments are used to assess spatial abilities connected to engineering content. It is also important that engineering problems target the representative engineering content for a specific course. EEG measurements offer a distinct non-intrusive capability to triangulate cognitive component data from students engaged in spatial aspects of engineering problem solving. Proper time stamping should be used for such data collection, ensuring triangulation with video-archived events. IRB protocols should be stringently followed, ensuring the anonymity of participant data and analysis.

Most troubleshooting concerns occur while collecting EEG data as detailed below, and the majority of those are handled before data are recorded. Corrections for poor impedance and noise are best handled during setup. Following the EEG headset manufacturer's instructions is critical, and in our experience the indications by the manufacturer's software can direct users to check specific electrodes. Typically the connection between the felt pad and the participant's head needs to be dampened more, or the connection between each electrode and the headset may need to be checked. If some connectivity is visible, but the quality is poor, using the syringe to re-dampen the felt is often sufficient, and at times the headset needs to be adjusted physically to ensure solid contact with the scalp. In a couple of cases, we had to ask participants to rinse their hair in a sink before we were able to obtain a good connection. When the electrode appeared to not be transmitting data, it was often remedied by removing the electrode and then reinserting it. At times, the plastic case for the electrode may crack, in which case it will need to be replaced.

Other troubleshooting may occur during data analysis, and is discussed in the protocol. Data preprocessing involves the filtering and removal of artifacts. Often the data analysis software supports manual rejection as well as scripts that can be run during preprocessing and processing of the data.

Modifications were made to a script within the analysis software. Those changes are documented in the supplemental code file. Modifications to the protocol may also be made. A concurrent protocol has been used in which verbal answers are required during the study. This will introduce more artifacts into the EEG data, but will provide more insight into the participant's functional knowledge during the tests. An alternative has also been used in which the participant participates in a video-recorded interview with the researcher after the session.

Other recommended potential modifications include utilizing different spatial ability tests¹⁴, different engineering questions¹⁷, or other educational assessments. Different brain activity metrics, possible via EEG and other instrumentation, could also shed light on the difficulty, or other characteristics, of skill assessments.

We recognize that there are limitations with the technique defined in this document. The constructs of spatial ability (rotation and cutting planed surface) measured by the PSVT:R and MCT are only two of many potential constructs measurable with other spatial metrics. In addition, different spatially intensive tasks (i.e., different types of problems or different courses and coursework) may also be assessed. Research into neural efficiency should of course also be conducted on a broader scope than just fundamental engineering courses such as Statics. For example, it should be investigated within the many STEM fields acknowledged in the literature to depend on spatial reasoning³. Also, neural efficiency studies should not be limited to skills directly linked only to spatial ability^21-28. Even within the research involved in brainwave measurement, the practice of averaging power measurements over the duration of a task prohibits investigation into other correlations that may occur within the patterns of brain activity. EEG measurements, due to their temporal responsivity, are not limited to neural efficiency studies. And EEG instrumentation is itself limited by the depth of brainwave activity it can detect, particularly when compared to the higher spatial resolution of functional near-infrared spectroscopy or functional magnetic resonance imaging, although its temporal responsivity remains among the best³⁶.

Ultimately, the potential of using physiological measurements to provide insight to educational theory and practice appears immense^37,38. The technical approach and goals of this protocol are different than the biofeedback approach using EEG in educational/training studies³⁹, but all are worth consideration as insight is gained in phenomena such as spatial ability development and engineering skill development. This approach of using EEG to examine neural efficiency between spatial tasks inherent within specific spatial ability instruments defines another method of segregating spatial ability tests. This exemplifies a new application of a neuroscientific approach for investigating spatial ability tests, as well as opening a neuroscientific approach towards the investigation of existing educational theory. Finding methods for verification and validation is part of engineering culture. Within this new application, physiological brainwave testing can open a new realm of understanding and refining educational theory. Indeed, if viewed as a potential avenue of validation, a novel and new generation of engineering educational research may arise.

Materials

Name	Company	Catalog Number	Comments
Emotiv EPOC Model 1.0	Emotiv	Model: Emotiv Premium	"High resolution, multi-channel, portable EEG system."
Emotiv Control Panel (software)	Emotiv		Used for data collection.
Emotiv Testbench (software)	Emotiv		Used for data collection.
Virtual Serial Port Emulator - VSPE (software)	ETERLOGIC.COM		Used COM10 in data collection. Available as a free download, depending on the operating system.
E-Prime 2.0 (software)	Psychology Software Tools		Used for data collection (presentation of problems to participants and collection of markers for different phases).
EEGLab 13.4.4b (software)	Swartz Center for Computational Neuroscience (SCCN)		Used for data analysis. "An open source environment for electrophysiological signal processing". SCCN is a Center of the Institute for Neural Computation, the University of California San Diego.
MATLAB R2014b	The Mathworks, Inc.		Used to run EEGLab
Microsoft Excel 2013	Microsoft		Used to assemble and compare tabulated results from EEGLab & MATLAB, to create tables
Camcorder with built in Mic	Canon	CNVHFR50	Used to record sessions
Syringe Kit (5 cc syringe & 16 G blunted needles x 2)	Electro-Cap Intnl. Inc.	E7	For keeping the EEG cap's felts damp.
Nuprep EEG Skin Prep Gel	Weaver and Company	10-30	For cleaning the mastoid process.
Sanitizer	Purell	S-12808	For sanitizing hands