MazeSuite is a complete toolset to prepare, present and analyze navigational and spatial experiments. Functional near-infrared spectroscopy (fNIR) is an optical brain imaging technique that enables noninvasive and portable monitoring of cerebral blood oxygenation changes. This paper summarizes collective use of MazeSuite and fNIR within a cognitive processing learning paradigm.
MazeSuite is a complete toolset to prepare, present and analyze navigational and spatial experiments1. MazeSuite can be used to design and edit adapted virtual 3D environments, track a participants' behavioral performance within the virtual environment and synchronize with external devices for physiological and neuroimaging measures, including electroencephalogram and eye tracking.
Functional near-infrared spectroscopy (fNIR) is an optical brain imaging technique that enables continuous, noninvasive, and portable monitoring of changes in cerebral blood oxygenation related to human brain functions2-7. Over the last decade fNIR is used to effectively monitor cognitive tasks such as attention, working memory and problem solving7-11. fNIR can be implemented in the form of a wearable and minimally intrusive device; it has the capacity to monitor brain activity in ecologically valid environments.
Cognitive functions assessed through task performance involve patterns of brain activation of the prefrontal cortex (PFC) that vary from the initial novel task performance, after practice and during retention12. Using positron emission tomography (PET), Van Horn and colleagues found that regional cerebral blood flow was activated in the right frontal lobe during the encoding (i.e., initial naïve performance) of spatial navigation of virtual mazes while there was little to no activation of the frontal regions after practice and during retention tests. Furthermore, the effects of contextual interference, a learning phenomenon related to organization of practice, are evident when individuals acquire multiple tasks under different practice schedules13,14. High contextual interference (random practice schedule) is created when the tasks to be learned are presented in a non-sequential, unpredictable order. Low contextual interference (blocked practice schedule) is created when the tasks to be learned are presented in a predictable order.
Our goal here is twofold: first to illustrate the experimental protocol design process and the use of MazeSuite, and second, to demonstrate the setup and deployment of the fNIR brain activity monitoring system using Cognitive Optical Brain Imaging (COBI) Studio software15. To illustrate our goals, a subsample from a study is reported to show the use of both MazeSuite and COBI Studio in a single experiment. The study involves the assessment of cognitive activity of the PFC during the acquisition and learning of computer maze tasks for blocked and random orders. Two right-handed adults (one male, one female) performed 315 acquisition, 30 retention and 20 transfer trials across four days. Design, implementation, data acquisition and analysis phases of the study were explained with the intention to provide a guideline for future studies.
Maze-Suite consists of three main applications; an editing program to construct maze environments (MazeMaker), a visualization/rendering module (MazeWalker), and finally an analysis and path visualization tool (MazeAnalyzer). The following sections describe the use of MazeSuite and more details are available in the three supplementary videos, one for each MazeSuite module.
1. Designing mazes
The MazeMaker application within Maze-Suite, allows creating three-dimensional (3D) environments by simply drawing them on a two-dimensional (2D) canvas from a birds' eye view. Walls and floors can be drawn by mouse clicks to indicate coordinates, and users can import 3D object files or edit properties of items within the maze such as position, orientation, color, texture and lights.
Maze start and end regions can be assigned in addition to other exit criteria such as a timeout period. Separate text messages can also be displayed to inform a participant for each exit condition. During the design phase, mazes can be tested by using the ‘quick run' function within MazeMaker.
For functional imaging studies, participants are usually asked to perform a series of tasks with repeated trials. To facilitate this testing of repeated trials, MazeMaker can create maze list files, which are sorted lists of mazes and text messages for an experimental session. It is critical that the maze list file for the experiment is prepared and thoroughly pilot-tested before the day of the experiment.
A cursory tutorial for the use and development of environments with MazeMaker is provided in 'Supplementary video II – MazeMaker'.
2. fNIR Setup and placing the fNIR sensor-pad
Preparing the fNIR Box
- There are two cable connectors behind the fNIR box. One of the slots is for USB connection and the other connection is for the power cord.
- Connect the fNIR device box through the USB cable to a PC or Laptop that will be used for data acquisition.
- Connect the power adapter to the device and turn on the switch.
- The ribbon cable is used to connect the sensor pad with the fNIR box.
- The sensor pad houses light source (LEDs) and photo detectors.
- The LEDs emit infrared light at 730nm and 850nm wavelengths that are absorbed mainly by deoxygenated- and oxygenated- hemoglobin, respectively and thus can penetrate through biological tissue.
Placing the Sensor-pad
- Ask the participant to lift their hair off the forehead before sensor placement. Place the sensor strip just above the eyebrows. Match the center of the sensor with the vertical axis of symmetry that passes also through the nose
- Press the sensor pad firmly against the forehead and use a clip to hold the cables together at the back of the head. While not strictly necessary, head-wrap or tennis bandana is recommended to secure the sensor pad.
- Upon placing the sensor pad on the forehead, attach the two ends of the ribbon cable to the fNIR box. The ribbon cables should be attached by matching the 'I' and 'II' sides of the ribbon cable with the corresponding 'I' and 'II' connectors on the device.
Starting the COBI Studio Software for data collection
- Click on the Cognitive Optical Brain Imaging (COBI) Studio13 icon on the desktop to open the main window.
- The program will come with certain preset settings for data collection and visualization.
- It is important to check and confirm data acquisition parameters and triggering (marker synchronization) at device settings dialog if needed.
- COBI Studio can automatically name all related data files if "Experiment mode" is activated by using the "New Experiment" wizard.
- In the data folder, 3 types of files will be created: (*.nir) for fNIR data, (*.mrk) for marker data and (*.txt) for a log of the experiment session.
3. Running the experiment: Rendering the mazes
Device Setting and starting fNIR data acquisition
- LED Drive Current defines how bright each LED shines. The default value for LED Drive Current is 20mA. This value may have to change based on the skin pigmentation and other characteristics of the participant. Suggested range for the LED current is between 5mA to 20mA
- The default value for initial gain for all channels is 20. Suggested values for gain are 1,5,10,15,20.
- Click on the 'Start current device' link, and examine the signal quality. If the raw intensity values shown in COBI approach 4000 or are below 1000, click 'Stop Device', and adjust the LED Drive Current and Device Gain until suitable values are obtained. In low signal circumstances, increase LED Drive Current before increasing Device Gain. Under saturated signal circumstances reduce the Device Gain before reducing the LED Drive Current. Once signal quality is acceptable, continue to Step 4.
- Start Baseline. This will collect 10 seconds of data and use it as baseline in a modified Beer-Lambert equation set to calculate concentration changes for oxy- and deoxy-hemoglobin8.
- Let the Baseline complete (may take 10-20 seconds)
- Click 'Start recording'. This will start saving all the data. Experiment protocol should start after this.
- Experimenter may choose to add manual markers throughout the experiment session to denote certain events by pressing the ‘add manual marker' buttons using the manual marker menu located in the lower left corner of the screen.
- Start the Maze Suite to present visual stimuli.
Using MazeWalker to render mazes
- Run MazeWalker from menu at Start>Maze Suite>MazeWalker
- Turn on marker synchronization by selecting 'Enable Serial Port' from the in the menu under Advanced>Serial Port Options. Ensure that the correct COM port address is selected.
- Select the maze list file (that was created by MazeMaker), and also select name for the new log file for this session. Name or participant number can be recorded within walker field. Optionally the autolog may be used to automatically record to a time-stamped log file instead of manually specifying the log file.
- Click 'Start' to initiate the process. Depending on the video settings, protocol can run full screen or in windowed mode.
4. Visualizing the subject's path
Using MazeAnalyzer, the researcher can visualize the maze and the participant's path from the maze and log files. Additionally, a summary report including total path length and time to completion of the each maze are produced as cursory behavioral measures. Log files contain information in millisecond time resolution about the path that a subject traveled as well as subject's view vector and interactions with objects.
A tutorial video is provided in 'Supplementary video I – MazeAnalyzer' and describes the use of MazeAnalyzer's basic functionality along with methods to produce the behavioral metrics described in the results section.
5. Processing the fNIR data and analysis
Noise removal is the first step for processing the data. Sources of noise include 1) Head Movement 2) Physiological signals such as heart rate and respirations and 3) Instrument and environment-related noise.
Head movement can cause the fNIR detectors to shift and lose contact with the skin, exposing them to: 1) ambient light; 2) light emitted directly from the fNIR sources; or 3) light reflected from the skin, rather than being reflected from tissue in the cortex. This type of motion artifact is readily recognizable because it causes sudden, large spikes in the fNIR data. A more subtle artifact of head movement is due to the effects of gravity on the cerebral blood. Rapid head movement can cause the blood to move toward (or away from) the area that is being monitored, rapidly increasing (or decreasing) blood volume with a concomitant skewing of the data. Since the dynamics of this type of motion artifact are slower than LED "pop" they can be confused with the actual hemodynamic response due to brain activation. Hence, removing motion artifact from fNIR data is an important and necessary step if fNIR is to be deployed as a brain monitoring technology in natural environments16.
Physiological signals such as heart rate (over 0.5 Hz) and respiration (over 0.2 Hz) are at higher frequency ranges than hemodynamic responses, thus, they can be eliminated using a linear phase low-pass FIR filter with cut-off frequency between 0.1 to 0.15 Hz9.
Instrument and environmental noise can originate due to ambient light such as daylight (DC) and room light (60Hz) or light from a computer monitor (60-75Hz). It is suggested that the best way to eliminate this type of noise is to prepare the experimental environment and data acquisition accordingly. An analog low pass filter (anti-aliasing filter) has been implemented in the fNIR box to eliminate folding of high frequency noise onto sampling frequency range.
There are many advanced noise reduction algorithms available that utilize different characteristics of the signal17-20. However if a subject or session can be excluded if data is irrecoverable (i.e. saturated).
Figure 1. fNIR signal components
The fNIR raw signals are light intensity measurements (See Figure 1). By measuring optical density (OD) changes at two wavelengths, the relative change of oxy-Hb and deoxy-Hb versus time can be obtained using the modified Beer–Lambert Law21-23. OD at a specific input wavelength (λ) is the logarithmic ratio of input light intensity (Iin) and output (detected) light intensity (Iout). OD is also related to the concentration (c) and extinction coefficient (e) of chromophores, the corrected distance (d) between the light source and detector, plus a constant attenuation factor (G).
Having the same Iin at two different time instances and detected light intensity during baseline (Irest) and during performance of the task (Itest), the difference in OD for wavelength λ is
Measuring the OD at two different wavelengths gives
This equation set can be solved for concentrations if the 2x2 matrix is non-singular. Typically, the two wavelengths are chosen i) within 700-900nm where the absorption of oxy-Hb and deoxy-Hb are dominant as compared to other tissue chromophores, and ii) below and above the isosbestic point (~805nm where absorption spectrums of deoxy- and oxy-Hb cross each other) to focus the changes in absorption to either deoxy-Hb or oxy-Hb, respectively. The fNIR instrument used in this study employs 730nm and 850nm wavelengths.
Finally, using the markers (time synchronization signals), periods of baseline/rest and tasks are labeled and select features from oxygenation data are extracted such as mean, min and max values. COBI Studio saves both raw light intensity measurements and calculated oxygenation values (using modified beer lambert law) to text based files as well as time synchronization (marker) files. COBI Studio can also be used to apply noise removal methods such as finite impulse response low-pass or band-pass filters. Output files can be exported to common analysis software such as (Matlab, Excel, SPM and SPSS) or dedicated analysis software such as fnirSoft24 for further processing.
6. Representative Results:
fNIR Signal Examples
Figures 2-5 preset fNIR raw signals from a single measurement location with two wavelength components (730nm and 850nm) shown separately. Figure 2 represents a valid and acceptable signal epoch whereas Figure 3 and 4 are unacceptable and needs to be discarded. Figure 5 presents raw signal that was contaminated with motion artifact and needs to be cleaned or discarded.
Figure 2. A good fNIR raw signal sample
Figure 3. A bad fNIR raw signal sample where 850nm channel is saturated.
Figure 4. A bad fNIR raw signal sample where there's hardware problem or cable connectivity issue.
Figure 5. A bad fNIR raw signal sample where there are motion artifacts.
Assessments of learning are best exemplified by retention (i.e., memory) and transfer (i.e., generalizability) tests. In our study, three mazes (maze1, maze2 and maze3) were practiced during the acquisition phase for a total of 105 trials each across three days. The two conditions; random practice order (RND) and blocked practice order (BLK) are summarized in Figure 6. Two subjects volunteered for the four day study. On the 4th day, 10 retention trials were conducted for each maze and the mazes were presented in a random order. Two novel mazes (maze4 and maze5) were created that had additional corridors and different starting and ending points than the mazes originally practiced. Subjects completed ten trials for each of the two new mazes. These mazes constituted a transfer phase, and were presented in a random order to all. The transfer phase mazes were used to determine the extent to which each subject was able to generalize their learning and practice with acquisition mazes.
Figure 6. Representative experiment protocol outline.
Study Behavioral Results
The following figures 7 through 9, display average feature values (path length, total time of completion, and velocity) of subjects across each day. First, results from maze1, maze2 and maze3 are listed for both RND and BLK practice. Next, maze4 and maze5 results are listed to compare random versus blocked practice results. All error bars are standard error of the mean (SEM).
Figure 7. RND practice behavioral performance summary for acquisition and retention tasks including total path length, total time of completion and average velocity.
Figure 8. BLK practice behavioral performance summary for acquisition and retention tasks including total path length, total time of completion and average velocity.
Figure 9. Behavioral performance comparison of BLK vs. RND practice for transfer tasks including total path length, total time of completion and average velocity.
Study fNIR Results
Oxygenated hemoglobin concentration changes from baseline were averaged over the time course of each individual maze using marker data received from MazeWalker (indicates start and end of each maze). Larger magnitude concentration changes are taken to represent higher levels of regional brain activation. A higher level of activation was observed during both retention and transfer tasks for BLK practice compared to RND practice. During transfer tasks, a higher level of activation was seen under BLK practice as compared with RND practice (See Figure 10).
For the BLK practice subject, transfer trials required higher activation compared to retention trials. For the RND practice subject, activation level DID NOT differ for transfer and retention tasks as opposed to BLK practice subject (See Figure 11).
Figure 10. Comparison of average oxy-Hb concentration changes for BLK vs. RND practice for both retention tasks (left) and transfer tasks (right).
Figure 11. Comparison of average oxy-Hb concentration changes for retention vs. transfer tasks for both BLK (left) and RND (right) practice.
The prefrontal cortex (PFC) in the human brain facilitates cognitive control over the coordination of thoughts and actions in relation to internal goals. In particular, the anterior/dorsolateral PFC is known to mediate higher cognitive functions such as task management, planning and spatial navigation25. fNIR is a portable, safe and noninvasive brain monitoring tool that has been used in clinical, laboratory and natural settings to study brain activation. This exploratory study demonstrated the use of Maze Suite and fNIR to study neurobehavioral aspects of spatial navigation. In this exploratory study, the MazeSuite platform is used in conjunction with fNIR to study the neurobehavioral aspects of spatial navigation in the dorsolateral PFC and to demonstrate the combination of these two tools.
MazeSuite is an experimental design, presentation and analysis platform. It enables the creation and application of simple 3D environments with a user friendly graphical interface and automatically records behavioral measures for within subject or across subject comparisons. During the presentation of MazeSuite environments, simultaneous time-synchronized fNIR measurements were taken using a commercially available, portable, continuous wave fNIR system (Imager 1000, fNIR Devices, LLC) and COBI Studio software15. fNIR previously has been established as a safe and effective noninvasive brain monitoring tool in clinical, laboratory and natural settings for the study of brain activation7,11 and is used in this study to investigate cognitive responses associated with contextual interference during of spatial navigation tasks.
To examine the effects of practice order contextual interference, subjects were presented with either a low interference (BLK) or high interference (RND) practice order. These distinct practice schedules were used to test the effect of learning multiple virtual spatial navigation maze tasks across acquisition, retention and transfer tests. The behavioral results indicate that for both practice orders, there is a monotonic decreasing trend in the total time necessary to complete maze, suggesting that as subjects practice, they completed each maze in shorter periods of time. In addition, the average speed with which subjects navigated the maze (maze velocity) increased with practice. These improvements in behavioral measures across time are expected inferences of learning. Mean oxy-Hb concentration changes during practice trials indicate that the BLK practice required higher brain activation when compared with RND practice order.
RND practice resulted in faster completion time and shorter path lengths when compared to the BLK practice order for both retention and transfer, respectively. A decreasing trend observed in the mean oxy-Hb for BLK and RND practice across retention trials indicating reduced activity in the PFC. This finding is expected as previous research has suggested that there is a reduction in the activity of the PFC during the later stages of learning12,26.
Moreover, the navigation in new mazes during the transfer phase required higher brain activation for BLK practice subject compared to RND practice subject. Given that the practice order was different for the tasks already learned (i.e., maze1, maze2, and maze3), this stratified random practice order for the subject that learned the tasks in a sequential (BLK practice) order may have been sufficiently novel to require additional effort and cognitive resources to perform the tasks12,26. However, for the RND practice, the transfer phase of neural activation was not higher than the retention phase. These findings corroborate the PET findings with spatial navigation of virtual mazes reported by Van Horn and colleagues12.
In summary, we described the use of MazeSuite in combination with COBI Studio for a study on the effects of contextual interference related to practice order when learning spatial navigational tasks. The fNIR methods discussed here are not limited to spatial navigation tasks and can be used for a variety of tasks in other neuroimaging studies. Design, implementation, data acquisition and analysis phases of the study were explained with the intention to provide a guideline for future studies.
fNIR Devices, LLC manufactures the optical brain imaging instrument and licensed IP and know-how from Drexel University. H. Ayaz, M. Izzetoglu, K. Izzetoglu and B. Onaral were involved in the technology development and thus offered a minor share in the new startup firm fNIR Devices, LLC.
Funding for this work was provided in part by the Commonwealth of Pennsylvania # 4100037709 subcontract #240468 and Drexel University subcontract #280773.
|fNIR Imager 1000
- Ayaz, H., Allen, S. L., Platek, S. M., Onaral, B. Maze Suite 1.0: a complete set of tools to prepare, present, and analyze navigational and spatial cognitive neuroscience experiments. Behav. Res Methods. 40, 353-359 (2008).
- Chance, B., Zhuang, Z., UnAh, C., Alter, C., Lipton, L. Cognition-activated low-frequency modulation of light absorption in human brain. Proceedings of the National Academy of Sciences of the United States of America. 90, 3770-3774 (1993).
- Villringer, A., Planck, J., Hock, C., Schleinkofer, L., Dirnagl, U. Near infrared spectroscopy (NIRS): a new tool to study hemodynamic changes during activation of brain function in human adults. Neuroscience letters. 154, 101-104 (1993).
- Hoshi, Y. Non-synchronous behavior of neuronal activity, oxidative metabolism and blood supply during mental tasks in man. Neuroscience letters. 172, 129-133 (1994).
- Strangman, G., Boas, D. A., Sutton, J. P. Non-invasive neuroimaging using near-infrared light. Biological psychiatry. 52, 679-693 (2002).
- Coyle, S., Ward, T. E., Markham, C. M. Brain-computer interface using a simplified functional near-infrared spectroscopy system. Journal of neural engineering. 4, 219-226 (2007).
- Ayaz, H. Optical brain monitoring for operator training and mental workload assessment. Neuroimage. , (2011).
- Izzetoglu, M. Functional near-infrared neuroimaging. IEEE Trans Neural Syst Rehabil Eng. 13, 153-159 (2005).
- Izzetoglu, M., Bunce, S. C., Izzetoglu, K., Onaral, B., Pourrezaei, K. Functional brain imaging using near-infrared technology. IEEE Eng Med Biol Mag. 26, 38-46 (2007).
- Shalinsky, M. H., Kovelman, I., Berens, M. S., Petitto, L. A. Exploring Cognitive Functions in Babies, Children & Adults with Near Infrared Spectroscopy. J. Vis. Exp. (29), e1268-e1268 (2009).
- Izzetoglu, K. The evolution of field deployable fNIR spectroscopy from bench to clinical settings. Journal of Innovative Optical Health Sciences. 4, 1-12 (2011).
- Van Horn, J. D. Changing patterns of brain activation during maze learning. Brain Res. 793, 29-38 (1998).
- Shewokis, P. A. Memory consolidation and contextual interference effects with computer games. Perc Motor Skills. 97, 581-589 (2003).
- Magill, R. A., Hall, K. G. A review of the contextual interference effect in motor skill acquisition. Human Movement Science. 9, 241-289 (1990).
- Ayaz, H., Onaral, B. Analytical software and stimulus-presentation platform to utilize, visualize and analyze near-infrared spectroscopy measures Masters Degree thesis [dissertation]. , Drexel University. (2005).
- Ayaz, H. Advances in Understanding Human Performance: Neuroergonomics, Human Factors Design, and Special Populations. Marek, T., Karwowski, W., Rice, V. 3, CRC Press Taylor & Francis Group. 21-31 (2010).
- Izzetoglu, M., Chitrapu, P., Bunce, S., Onaral, B. Motion artifact cancellation in NIR spectroscopy using discrete Kalman filtering. Biomedical engineering online. 9, 16-16 (2010).
- Huppert, T. J., Diamond, S. G., Franceschini, M. A., Boas, D. A. HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl Opt. 48, 280-298 (2009).
- Zhang, Q., Strangman, G., Ganis, G. Adaptive filtering to reduce global interference in non-invasive NIRS measures of brain activation: How well and when does it work? Neuroimage. 45, 788-794 (2009).
- Izzetoglu, M., Devaraj, A., Bunce, S., Onaral, B. Motion artifact cancellation in NIR spectroscopy using Wiener filtering. IEEE Trans Biomed Eng. 52, 934-938 (2005).
- Cope, M. The development of a near infrared spectroscopy system and its application for non invasive monitoring of cerebral blood and tissue oxygenation in the newborn infant. , University College London. London. (1991).
- Elwell, C. Quantification of adult cerebral hemodynamics by near-infrared spectroscopy. Journal of Applied Physiology. 77, 2753-2753 (1994).
- Wyatt, J. Quantitation of cerebral blood volume in human infants by near-infrared spectroscopy. Journal of Applied Physiology. 68, 1086-1086 (1990).
- Ayaz, H. Functional Near Infrared Spectroscopy based Brain Computer Interface PhD thesis [dissertation]. , Drexel University. (2010).
- Wood, J. N., Grafman, J. Human prefrontal cortex: processing and representational perspectives. Nat Rev Neurosci. 4, 139-147 (2003).
- Shadmehr, R., Holcomb, H. H. Neural correlates of motor memory consolidation. Science. 277, 821-825 (1997).