The Topography of High-order Human Object Areas

Trends in Cognitive Sciences. Apr, 2002  |  Pubmed ID: 11912041

Cortical topography is one of the most fundamental organizing principles of cortical areas. One such topography - eccentricity mapping - is present even in high-order, ventral stream visual areas. Within these areas, different object categories have specific eccentricity biases. In particular, faces, letters and words appear to be associated with central visual-field bias, whereas buildings are associated with a peripheral one. We propose that resolution needs are an important factor in organizing object representations: objects whose recognition depends on analysis of fine detail will be associated with central-biased representations, whereas objects whose recognition entails large-scale integration will be more peripherally biased.

Eccentricity Bias As an Organizing Principle for Human High-order Object Areas

Neuron. Apr, 2002  |  Pubmed ID: 11988177

We have recently proposed a center-periphery organization based on resolution needs, in which objects engaging in recognition processes requiring central-vision (e.g., face-related) are associated with center-biased representations, while objects requiring large-scale feature integration (e.g., buildings) are associated with periphery-biased representations. Here we tested this hypothesis by comparing the center-periphery organization with activations to five object categories: faces, buildings, tools, letter strings, and words. We found that faces, letter strings, and words were mapped preferentially within the center-biased representation. Faces showed a hemispheric lateralization opposite to that of letter strings and words. In contrast, buildings were mapped mainly to the periphery-biased representation, while tools activated both central and peripheral representations. The results are compatible with the notion that center-periphery organization allows the optimal allocation of cortical magnification to the specific requirements of various recognition processes.

Analysis of the Neuronal Selectivity Underlying Low FMRI Signals

Current Biology : CB. Jun, 2002  |  Pubmed ID: 12123569

A prevailing assumption in neuroimaging studies is that relatively low fMRI signals are due to weak neuronal activation, and, therefore, they are commonly ignored. However, lower fMRI signals may also result from intense activation by highly selective, albeit small, subsets of neurons in the imaged voxel. We report on an approach that could form a basis for resolving this ambiguity imposed by the low (mm range) spatial resolution of fMRI. Our approach employs fMR-adaptation as an indicator for highly active neuronal populations even when the measured fMRI signal is low.

Large-scale Mirror-symmetry Organization of Human Occipito-temporal Object Areas

Neuron. Mar, 2003  |  Pubmed ID: 12670430

We have combined functional maps of retinotopy (eccentricity and meridian mapping), object category, and motion in a group of subjects to explore the large-scale topography of higher-order object areas. Our results reveal seven consistent category-related entities situated in the occipito-temporal cortex adjoining early visual areas. These include two face-related regions, three object-related regions, and two building-related regions. Interestingly, this complex category-related pattern is organized in a large-scale dorso-ventral mirror symmetry of object category. Furthermore, correlating this pattern to the map of visual field eccentricity, we found that the entire network of areas could be related to a single and unified eccentricity map. We hypothesize that this large-scale organization points to a possible development of high-order object areas through extension and specialization of a single proto-representation.

Face-selective Activation in a Congenital Prosopagnosic Subject

Journal of Cognitive Neuroscience. Apr, 2003  |  Pubmed ID: 12729493

Congenital prosopagnosia is a severe impairment in face identification manifested from early childhood in the absence of any evident brain lesion. In this study, we used fMRI to compare the brain activity elicited by faces in a congenital prosopagnosic subject (YT) relative to a control group of 12 subjects in an attempt to shed more light on the nature of the brain mechanisms subserving face identification. The face-related activation pattern of YT in the ventral occipito-temporal cortex was similar to that observed in the control group on several parameters: anatomical location, activation profiles, and hemispheric laterality. In addition, using a modified vase-face illusion, we found that YT's brain activity in the face-related regions manifested global grouping processes. However, subtle differences in the degree of selectivity between objects and faces were observed in the lateral occipital cortex. These data suggest that face-related activation in the ventral occipito-temporal cortex, although necessary, might not be sufficient by itself for normal face identification.

Counterexamples in Sentential Reasoning

Memory & Cognition. Oct, 2003  |  Pubmed ID: 14704025

How do logically naive individuals determine that an inference is invalid? In logic, there are two ways to proceed: (1) make an exhaustive search but fail to find a proof of the conclusion and (2) use the interpretation of the relevant sentences to construct a counterexample--that is, a possibility consistent with the premises but inconsistent with the conclusion. We report three experiments in which the strategies that individuals use to refute invalid inferences based on sentential connectives were examined. In Experiment 1, the participants' task was to justify their evaluations, and it showed that they used counterexamples more often than any other strategy. Experiment 2 showed that they were more likely to use counterexamples to refute invalid conclusions consistent with the premises than to refute invalid conclusions inconsistent with the premises. In Experiment 3, no reliable difference was detected in the results between participants who wrote justifications and participants who did not.

Intersubject Synchronization of Cortical Activity During Natural Vision

Science (New York, N.Y.). Mar, 2004  |  Pubmed ID: 15016991

To what extent do all brains work alike during natural conditions? We explored this question by letting five subjects freely view half an hour of a popular movie while undergoing functional brain imaging. Applying an unbiased analysis in which spatiotemporal activity patterns in one brain were used to "model" activity in another brain, we found a striking level of voxel-by-voxel synchronization between individuals, not only in primary and secondary visual and auditory areas but also in association cortices. The results reveal a surprising tendency of individual brains to "tick collectively" during natural vision. The intersubject synchronization consisted of a widespread cortical activation pattern correlated with emotionally arousing scenes and regionally selective components. The characteristics of these activations were revealed with the use of an open-ended "reverse-correlation" approach, which inverts the conventional analysis by letting the brain signals themselves "pick up" the optimal stimuli for each specialized cortical area.

Functional Analysis of the Periphery Effect in Human Building Related Areas

Human Brain Mapping. May, 2004  |  Pubmed ID: 15083523

Several studies have shown that a region in the anterior collateral sulcus (CoS) and a region in the vicinity of the transverse occipital sulcus (TOS) are preferentially activated by images of buildings and scenes. We have found recently that these regions show a strong activation bias to stimuli located in the peripheral visual field. We explore in detail the source of this "periphery" effect. Our results show that the periphery effect can be generated by a large single object occupying the peripheral visual field as well as by multiple small peripheral objects. We also investigated whether the periphery effect was related to the annular shape used in conventional mapping of the visual field periphery and found that the mere presence of a stimulus in the visual field periphery, regardless of object shape, is sufficient to enhance activation. We also found that a small bias toward the peripheral visual field was shown even when the stimulated areas in the central and peripheral parts of the visual field are equated. Finally, our results demonstrate that the periphery effect shows object selectivity that can be obtained even with face images, which are the non-optimal stimulus for this region. In summary, our study shows that the building-related CoS and TOS manifest a true but graded retinotopic bias toward the peripheral visual field.

One Picture is Worth at Least a Million Neurons

Current Biology : CB. Jun, 2004  |  Pubmed ID: 15182673

How many neurons participate in the representation of a single visual image? Answering this question is critical for constraining biologically inspired models of object recognition, which vary greatly in their assumptions from few "grandmother cells" to numerous neurons in widely distributed networks. Functional imaging techniques, such as fMRI, provide an opportunity to explore this issue, since they allow the simultaneous detection of the entire neuronal population responding to each stimulus. Several studies have shown that fMRI BOLD signal is approximately proportional to neuronal activity. However, since it provides an indirect measure of this activity, obtaining a realistic estimate of the number of activated neurons requires several intervening steps. Here, we used the extensive knowledge of primate V1 to yield a conservative estimate of the ratio between hemodynamic response and neuronal firing. This ratio was then used, in addition to several cautious assumptions, to assess the number of neurons responding to a single-object image in the entire visual cortex and particularly in object-related areas. Our results show that at least a million neurons in object-related cortex and about two hundred million neurons in the entire visual cortex are involved in the representation of a single-object image.

The Importance of Being Nonalignable: a Critical Test of the Structural Alignment Theory of Similarity

Journal of Experimental Psychology. Learning, Memory, and Cognition. Sep, 2004  |  Pubmed ID: 15355137

The structural alignment theory of similarity distinguishes 2 types of difference that may occur between stimuli: Alignable differences are those related to a commonality, whereas nonalignable differences are not related to a commonality. Alignment theory predicts that alignable differences should be more heavily weighted than nonalignable differences in similarity judgment. Experiments 1 and 2 demonstrate that, contrary to this prediction, nonalignable differences exerted a greater impact than alignable differences in similarity and difference judgments of geometric stimuli. Experiment 3 revealed that the relative weight accorded a given difference was also affected by contextual constraints. Thus, although the experiments supported the validity of the distinction between alignable and nonalignable differences, results were discordant with the specific prediction of structural alignment theory.

Believe It or Not: on the Possibility of Suspending Belief

Psychological Science. Jul, 2005  |  Pubmed ID: 16008791

We present two experiments that cast doubt on existing evidence suggesting that it is impossible to suspend belief in a comprehended proposition. In Experiment 1, we found that interrupting the encoding of a statement's veracity decreased memory for the statement's falsity when the false version of the statement was uninformative, but not when the false version was informative. This suggests that statements that are informative when false are not represented as if they were true. In Experiment 2, participants made faster lexical decisions to words implied by preceding statements when they were told that the statements were true than when the veracity of the statements was unknown or when the statements were false. The findings suggest that comprehending a statement may not require believing it, and that it may be possible to suspend belief in comprehended propositions.

Coupling Between Neuronal Firing, Field Potentials, and FMRI in Human Auditory Cortex

Science (New York, N.Y.). Aug, 2005  |  Pubmed ID: 16081741

Functional magnetic resonance imaging (fMRI) is an important tool for investigating human brain function, but the relationship between the hemodynamically based fMRI signals in the human brain and the underlying neuronal activity is unclear. We recorded single unit activity and local field potentials in auditory cortex of two neurosurgical patients and compared them with the fMRI signals of 11 healthy subjects during presentation of an identical movie segment. The predicted fMRI signals derived from single units and the measured fMRI signals from auditory cortex showed a highly significant correlation (r = 0.75, P < 10(-47)). Thus, fMRI signals can provide a reliable measure of the firing rate of human cortical neurons.

Detailed Exploration of Face-related Processing in Congenital Prosopagnosia: 2. Functional Neuroimaging Findings

Journal of Cognitive Neuroscience. Jul, 2005  |  Pubmed ID: 16102242

Specific regions of the human occipito-temporal cortex are consistently activated in functional imaging studies of face processing. To understand the contribution of these regions to face processing, we examined the pattern of fMRI activation in four congenital prosopagnosic (CP) individuals who are markedly impaired at face processing despite normal vision and intelligence, and with no evidence of brain damage. These individuals evinced a normal pattern of fMRI activation in the fusiform gyrus (FFA) and in other ventral occipito-temporal areas, in response to faces, buildings, and other objects, shown both as line drawings in detection and discrimination tasks and under more naturalistic testing conditions when no task was required. CP individuals also showed normal adaptation levels in a block-design adaptation experiment and, like control subjects, exhibited evidence of global face representation in the FFA. The absence of a BOLD-behavioral correlation (profound behavioral deficit, normal face-related activation in the ventral occipito-temporal cortex) challenges existing accounts of face representation, and suggests that activation in these cortical regions per se is not sufficient to ensure intact face processing.

Widespread Functional Connectivity and FMRI Fluctuations in Human Visual Cortex in the Absence of Visual Stimulation

NeuroImage. May, 2006  |  Pubmed ID: 16413791

To what extent does the visual system's activity fluctuate when no sensory stimulation is present? Here, we studied this issue by examining spontaneous fluctuations in BOLD signal in the human visual system, while subjects were placed in complete darkness. Our results reveal widespread slow fluctuations during such rest periods. In contrast to stimulus-driven activity, during darkness, functionally distinct object areas were fluctuating in unison. These fMRI fluctuations became rapidly spatially de-correlated (39% drop in correlation level, P < 0.008) during visual stimulation. Functional connectivity analysis revealed that the slow spontaneous fluctuations during rest had consistent and specific neuro-anatomical distribution which argued against purely hemodynamic noise sources. Control experiments ruled out eye closure, low luminance and mental imagery as the underlying sources of the spontaneous fluctuations. These results demonstrate that, when no stimulus is present, sensory systems manifest a robust level of slow organized fluctuation patterns.

Human Brain Activation During Viewing of Dynamic Natural Scenes

Novartis Foundation Symposium. , 2006  |  Pubmed ID: 16649716

To what extent do brains of different human individuals operate in a similar manner? Here we explored the organization and function of different brain regions under progressively more natural conditions. Applying an unbiased analysis, in which spatiotemporal activity patterns in one brain were used to 'model' activity in another brain, we found a striking level of voxel by voxel synchronization between individuals during free viewing of an audio-visual movie. This intersubject correlation was evident not only in primary and secondary visual and auditory areas, but also in association cortices. The results reveal a surprising tendency of individual brains to 'tick collectively' during natural vision. Moreover, our results demonstrate that the unitary nature of conscious experience in fact consists of temporally interleaved and highly selective activations in an ensemble of specialized regions, each of which 'picks-up' and analyses its own unique subset of stimuli according to its functional specialization. Applying reverse correlation to the movie stimuli provides a powerful methodology for revealing both known and unexpected functional specializations in those cortical areas activated by the movie.

Repetition Suppression for Spoken Sentences and the Effect of Task Demands

Journal of Cognitive Neuroscience. Dec, 2006  |  Pubmed ID: 17129188

We examined whether the repeated processing of spoken sentences is accompanied by reduced bold oxygenation level-dependent response (repetition suppression) in regions implicated in sentence comprehension and whether the magnitude of such suppression depends on the task under which the sentences are comprehended or on the complexity of the sentences. We found that sentence repetition was associated with repetition suppression in temporal regions, independent of whether participants judged the sensibility of the statements or listened to the statements passively. In contrast, repetition suppression in inferior frontal regions was found only in the context of the task demanding active judgment. These results suggest that repetition suppression in temporal regions reflects facilitation of sentence comprehension processing per se, whereas in frontal regions it reflects, at least in part, easier execution of specific psycholinguistic judgments.

Brain Networks Subserving the Extraction of Sentence Information and Its Encoding to Memory

Cerebral Cortex (New York, N.Y. : 1991). Dec, 2007  |  Pubmed ID: 17372276

Sentences are the primary means by which people communicate information. The information conveyed by a sentence depends on how that sentence relates to what is already known. We conducted an fMRI study to determine how the brain establishes and retains this information. We embedded sentences in contexts that rendered them more or less informative and assessed which functional networks were associated with comprehension of these sentences and with memory for their content. We identified two such networks: A frontotemporal network, previously associated with working memory and language processing, showed greater activity when sentences were informative. Independently, greater activity in this network predicted subsequent memory for sentence content. In a separate network, previously associated with resting-state processes and generation of internal thoughts, greater neural activity predicted subsequent memory for informative sentences but also predicted subsequent forgetting for less-informative sentences. These results indicate that in the brain, establishing the information conveyed by a sentence, that is, its contextually based meaning, involves two dissociable networks, both of which are related to processing of sentence meaning and its encoding to memory.

Accelerating Medical Research Using the Swift Workflow System

Studies in Health Technology and Informatics. , 2007  |  Pubmed ID: 17476063

Both medical research and clinical practice are starting to involve large quantities of data and to require large-scale computation, as a result of the digitization of many areas of medicine. For example, in brain research - the domain that we consider here - a single research study may require the repeated processing, using computationally demanding and complex applications, of thousands of files corresponding to hundreds of functional MRI studies. Execution efficiency demands the use of parallel or distributed computing, but few medical researchers have the time or expertise to write the necessary parallel programs. The Swift system addresses these concerns. A simple scripting language, SwiftScript, provides for the concise high-level specification of workflows that invoke various application programs on potentially large quantities of data. The Swift engine provides for the efficient execution of these workflows on sequential computers, parallel computers, and/or distributed grids that federate the computing resources of many sites. Last but not least, the Swift provenance catalog keeps track of all actions performed, addressing vital bookkeeping functions that so often cause difficulties in large computations. To illustrate the use of Swift for medical research, we describe its use for the analysis of functional MRI data as part of a research project examining the neurological mechanisms of recovery from aphasia after stroke. We show how SwiftScript is used to encode an application workflow, and present performance results that demonstrate our ability to achieve significant speedups on both a local parallel computing cluster and multiple parallel clusters at distributed sites.

They Saw a Movie: Long-term Memory for an Extended Audiovisual Narrative

Learning & Memory (Cold Spring Harbor, N.Y.). Jun, 2007  |  Pubmed ID: 17562897

We measured long-term memory for a narrative film. During the study session, participants watched a 27-min movie episode, without instructions to remember it. During the test session, administered at a delay ranging from 3 h to 9 mo after the study session, long-term memory for the movie was probed using a computerized questionnaire that assessed cued recall, recognition, and metamemory of movie events sampled approximately 20 sec apart. The performance of each group of participants was measured at a single time point only. The participants remembered many events in the movie even months after watching it. Analysis of performance, using multiple measures, indicates differences between recent (weeks) and remote (months) memory. While high-confidence recognition performance was a reliable index of memory throughout the measured time span, cued recall accuracy was higher for relatively recent information. Analysis of different content elements in the movie revealed differential memory performance profiles according to time since encoding. We also used the data to propose lower limits on the capacity of long-term memory. This experimental paradigm is useful not only for the analysis of behavioral performance that results from encoding episodes in a continuous real-life-like situation, but is also suitable for studying brain substrates and processes of real-life memory using functional brain imaging.

Brain Areas Selective for Both Observed and Executed Movements

Journal of Neurophysiology. Sep, 2007  |  Pubmed ID: 17596409

When observing a particular movement a subset of movement-selective visual and visuomotor neurons are active in the observer's brain, forming a representation of the observed movement. Similarly, when executing a movement a subset of movement-selective motor and visuomotor neurons are active, forming a representation of the executed movement. In this study we used an fMRI-adaptation protocol to assess cortical response selectivity to observed and executed movements simultaneously. Subjects freely played the rock-paper-scissors game against a videotaped opponent, sometimes repeatedly observing or executing the same movement on subsequent trials. Numerous brain areas exhibited adaptation (repetition suppression) during either repeated observations or repeated executions of the same movement. A subset of areas exhibited an overlap of both effects, containing neurons with selective responses for both executed and observed movements. We describe the function of these movement representation areas in the context of the human mirror system, which is expected to respond selectively to both observed and executed movements.

Abstract Coding of Audiovisual Speech: Beyond Sensory Representation

Neuron. Dec, 2007  |  Pubmed ID: 18093531

Is there a neural representation of speech that transcends its sensory properties? Using fMRI, we investigated whether there are brain areas where neural activity during observation of sublexical audiovisual input corresponds to a listener's speech percept (what is "heard") independent of the sensory properties of the input. A target audiovisual stimulus was preceded by stimuli that (1) shared the target's auditory features (auditory overlap), (2) shared the target's visual features (visual overlap), or (3) shared neither the target's auditory or visual features but were perceived as the target (perceptual overlap). In two left-hemisphere regions (pars opercularis, planum polare), the target invoked less activity when it was preceded by the perceptually overlapping stimulus than when preceded by stimuli that shared one of its sensory components. This pattern of neural facilitation indicates that these regions code sublexical speech at an abstract level corresponding to that of the speech percept.

Extrinsic and Intrinsic Systems in the Posterior Cortex of the Human Brain Revealed During Natural Sensory Stimulation

Cerebral Cortex (New York, N.Y. : 1991). Apr, 2007  |  Pubmed ID: 16699080

When exposing subjects to a continuous segment of an audiovisual movie, a large expanse of human cortex, especially in the posterior half of the cerebral cortex, shows stimulus-driven activity. However, embedded within this widespread activity, there are cortical regions whose activity is dissociated from the external stimulation. These regions are intercorrelated among themselves, forming a functional network, which largely overlaps with cortical areas previously shown to be deactivated by task-oriented paradigms. Moreover, the network of areas whose neuronal dynamics are associated with external inputs and the network of areas that appears to be intrinsically driven complement each other, providing coverage of most of the posterior cortex. Thus, we propose that naturalistic stimuli reveal a fundamental neuroanatomical partition of the human posterior cortex into 2 global networks: an "extrinsic" system, comprising areas associated with the processing of external inputs, and an "intrinsic" system, largely overlapping with the task-negative, default-mode network, comprising areas associated with--as yet not fully understood--intrinsically oriented functions.

Enhanced Intersubject Correlations During Movie Viewing Correlate with Successful Episodic Encoding

Neuron. Feb, 2008  |  Pubmed ID: 18255037

While much has been learned regarding the neural substrates supporting episodic encoding using highly controlled experimental protocols, relatively little is known regarding the neural bases of episodic encoding of real-world events. In an effort to examine this issue, we measured fMRI activity while observers viewed a novel TV sitcom. Three weeks later, subsequent memory (SM) for the narrative content of movie events was assessed. We analyzed the encoding data for intersubject correlations (ISC) based on subjects' subsequent memory (ISC-SM) performance to identify brain regions whose BOLD response is significantly more correlated across subjects during portions of the movie that are successfully as compared to unsuccessfully encoded. These regions include the parahippocampal gyrus, superior temporal gyrus, anterior temporal poles, and the temporal-parietal junction. Further analyses reveal (1) that these correlated regions can display distinct activation profiles and (2) that the results seen with the ISC-SM analysis are complementary to more traditional linear models and allow analysis of complex time course data. Thus, the ISC-SM analysis extends traditional subsequent memory findings to a rich, dynamic and more ecologically valid situation.

A Hierarchy of Temporal Receptive Windows in Human Cortex

The Journal of Neuroscience : the Official Journal of the Society for Neuroscience. Mar, 2008  |  Pubmed ID: 18322098

Real-world events unfold at different time scales and, therefore, cognitive and neuronal processes must likewise occur at different time scales. We present a novel procedure that identifies brain regions responsive to sensory information accumulated over different time scales. We measured functional magnetic resonance imaging activity while observers viewed silent films presented forward, backward, or piecewise-scrambled in time. Early visual areas (e.g., primary visual cortex and the motion-sensitive area MT+) exhibited high response reliability regardless of disruptions in temporal structure. In contrast, the reliability of responses in several higher brain areas, including the superior temporal sulcus (STS), precuneus, posterior lateral sulcus (LS), temporal parietal junction (TPJ), and frontal eye field (FEF), was affected by information accumulated over longer time scales. These regions showed highly reproducible responses for repeated forward, but not for backward or piecewise-scrambled presentations. Moreover, these regions exhibited marked differences in temporal characteristics, with LS, TPJ, and FEF responses depending on information accumulated over longer durations (approximately 36 s) than STS and precuneus (approximately 12 s). We conclude that, similar to the known cortical hierarchy of spatial receptive fields, there is a hierarchy of progressively longer temporal receptive windows in the human brain.

Improving the Analysis, Storage and Sharing of Neuroimaging Data Using Relational Databases and Distributed Computing

NeuroImage. Jan, 2008  |  Pubmed ID: 17964812

The increasingly complex research questions addressed by neuroimaging research impose substantial demands on computational infrastructures. These infrastructures need to support management of massive amounts of data in a way that affords rapid and precise data analysis, to allow collaborative research, and to achieve these aims securely and with minimum management overhead. Here we present an approach that overcomes many current limitations in data analysis and data sharing. This approach is based on open source database management systems that support complex data queries as an integral part of data analysis, flexible data sharing, and parallel and distributed data processing using cluster computing and Grid computing resources. We assess the strengths of these approaches as compared to current frameworks based on storage of binary or text files. We then describe in detail the implementation of such a system and provide a concrete description of how it was used to enable a complex analysis of fMRI time series data.

Cortical Patterns of Category-selective Activation for Faces, Places and Objects in Adults with Autism

Autism Research : Official Journal of the International Society for Autism Research. Feb, 2008  |  Pubmed ID: 19360650

Autism is associated with widespread atypicalities in perception, cognition and social behavior. A crucial question concerns how these atypicalities are reflected in the underlying brain activation. One way to examine possible perturbations of cortical organization in autism is to analyze the activation of category-selective ventral visual cortex, already clearly delineated in typical populations. We mapped out the neural correlates of face, place and common object processing, using functional magnetic resonance imaging (fMRI), in a group of high-functioning adults with autism and a typical comparison group, under both controlled and more naturalistic, viewing conditions. There were no consistent group differences in place-related regions. Although there were no significant differences in the extent of the object-related regions, there was more variability for these regions in the autism group. The most marked group differences were in face-selective cortex, with individuals with autism evincing reduced activation, not only in fusiform face area but also in superior temporal sulcus and occipital face area. Ventral visual cortex appears to be organized differently in high-functioning adults with autism, at least for face-selective regions, although subtle differences may also exist for other categories. We propose that cascading developmental effects of low-level differences in neuronal connectivity result in a much more pronounced effect on later developing cortical systems, such as that for face-processing, than earlier maturing systems (those for objects and places).

Co-speech Gestures Influence Neural Activity in Brain Regions Associated with Processing Semantic Information

Human Brain Mapping. Nov, 2009  |  Pubmed ID: 19384890

Everyday communication is accompanied by visual information from several sources, including co-speech gestures, which provide semantic information listeners use to help disambiguate the speaker's message. Using fMRI, we examined how gestures influence neural activity in brain regions associated with processing semantic information. The BOLD response was recorded while participants listened to stories under three audiovisual conditions and one auditory-only (speech alone) condition. In the first audiovisual condition, the storyteller produced gestures that naturally accompany speech. In the second, the storyteller made semantically unrelated hand movements. In the third, the storyteller kept her hands still. In addition to inferior parietal and posterior superior and middle temporal regions, bilateral posterior superior temporal sulcus and left anterior inferior frontal gyrus responded more strongly to speech when it was further accompanied by gesture, regardless of the semantic relation to speech. However, the right inferior frontal gyrus was sensitive to the semantic import of the hand movements, demonstrating more activity when hand movements were semantically unrelated to the accompanying speech. These findings show that perceiving hand movements during speech modulates the distributed pattern of neural activation involved in both biological motion perception and discourse comprehension, suggesting listeners attempt to find meaning, not only in the words speakers produce, but also in the hand movements that accompany speech.

Task-dependent Organization of Brain Regions Active During Rest

Proceedings of the National Academy of Sciences of the United States of America. Jun, 2009  |  Pubmed ID: 19541656

The human brain demonstrates complex yet systematic patterns of neural activity at rest. We examined whether functional connectivity among those brain regions typically active during rest depends on ongoing and recent task demands and individual differences. We probed the temporal coordination among these regions during periods of language comprehension and during the rest periods that followed comprehension. Our findings show that the topography of this "rest network" varies with exogenous processing demands. The network encompassed more highly interconnected regions during rest than during listening, but also when listening to unsurprising vs. surprising information. Furthermore, connectivity patterns during rest varied as a function of recent listening experience. Individual variability in connectivity strength was associated with cognitive function: more attentive comprehenders demonstrated weaker connectivity during language comprehension, and a greater differentiation between connectivity during comprehension and rest. The regions we examined have generally been thought to form an invariant physiological and functional network whose activity reflects spontaneous cognitive processes. Our findings suggest that their function extends beyond the mediation of unconstrained thought, and that they play an important role in higher-level cognitive function.

Shared and Idiosyncratic Cortical Activation Patterns in Autism Revealed Under Continuous Real-life Viewing Conditions

Autism Research : Official Journal of the International Society for Autism Research. Aug, 2009  |  Pubmed ID: 19708061

Although widespread alterations in cortical structure have been documented in individuals with autism, the functional implications of these alterations remain to be determined. Here, we adopted a novel inter-subject correlation (inter-SC) and intra-subject correlation (intra-SC) technique to quantify the reliability of the spatio-temporal responses of functional MR activity in adults with autism during free-viewing of a popular audio-visual movie. Whereas these complex stimuli evoke highly reliable shared response time courses in typical individuals, cortical activity was more variable across individuals with autism (low inter-SC). Interestingly, when we measured the responses within an autistic individual across repeated presentations of the movie, we observed a unique, idiosyncratic response time course that was reliably replicated within each individual (high intra-SC). Encouragingly, after filtering out the idiosyncratic responses from each individual time course, we were able to uncover a more typical response profile, which resembles the shared responses seen in the typical subjects. These findings indicate that, under conditions approximating real-life situations, the neural activity of individuals with autism is characterized by individualistic responses that, although reliable within an autistic individual, are both highly variable across autistic individuals and different from the responses observed within the typical subjects. These idiosyncratic responses may underlie the atypical behaviors observed in autism. At the same time, we are encouraged by the presence of the more typical activation pattern lurking beneath these idiosyncratic fluctuations. Taken together, these findings may pave the way to future research aimed at characterizing the idiosyncratic response profiles, which, in turn, might contribute to a better understanding of the heterogeneity of the autism spectrum and its diagnosis.

Database-managed Grid-enabled Analysis of Neuroimaging Data: the CNARI Framework

International Journal of Psychophysiology : Official Journal of the International Organization of Psychophysiology. Jul, 2009  |  Pubmed ID: 19233234

Functional magnetic resonance imaging (fMRI) has led to an enormous growth in the study of cognitive neuroanatomy, and combined with advances in high-field electrophysiology (and other methods), has led to a fast-growing field of human neuroscience. Technological advances in both hardware and software will lead to an ever more promising future for fMRI. We have developed a new computational framework that facilitates fMRI experimentation and analysis, and which has led to some rethinking of the nature of experimental design and analysis. The Computational Neuroscience Applications Research Infrastructure (CNARI) incorporates novel methods for maintaining, serving, and analyzing massive amounts of fMRI data. By using CNARI, it is possible to perform naturalistic, network-based, statistically valid experiments in systems neuroscience on a very large scale, with ease of data manipulation and analysis, within reasonable computational time scales. In this article, we describe this infrastructure and then illustrate its use on a number of actual examples in both cognitive neuroscience and neurological research. We believe that these advanced computational approaches will fundamentally change the future shape of cognitive brain imaging with fMRI.

Outsourcing Neuroimaging Data Analysis Implications for Scientific Accountability and Issues in the Public Interest

Trends in Cognitive Sciences. Jan, 2010  |  Pubmed ID: 19846333

Reliability of Cortical Activity During Natural Stimulation

Trends in Cognitive Sciences. Jan, 2010  |  Pubmed ID: 20004608

Response reliability is complementary to more conventional measurements of response amplitudes, and can reveal phenomena that response amplitudes do not. Here we review studies that measured reliability of cortical activity within or between human subjects in response to naturalistic stimulation (e.g. free viewing of movies). Despite the seemingly uncontrolled nature of the task, some of these complex stimuli evoke highly reliable, selective and time-locked activity in many brain areas, including some regions that show little response modulation in most conventional experimental protocols. This activity provides an opportunity to address novel questions concerning natural vision, temporal scale of processing, memory and the neural basis of inter-group differences.

Multiple Sources of Competence Underlying the Comprehension of Inconsistencies: A Developmental Investigation

Journal of Experimental Psychology. Learning, Memory, and Cognition. Mar, 2010  |  Pubmed ID: 20192531

How do children know the sentence "the glass is empty and not empty" is inconsistent? One possibility is that they are sensitive to the formal structure of the sentences and know that a proposition and its negation cannot be jointly true. Alternatively, they could represent the 2 state of affairs referred to and realize that these are incommensurate, that is, that a glass cannot simultaneously be empty and contain something. In 2 studies, the authors investigated how children (N = 186; ages 4-8) acquire competence to notice inconsistencies. The authors found that children could determine that 2 states of affairs were incommensurate before being able to determine that statements of the form p and not-p were inconsistent. The results demonstrate that competence in understanding inconsistent relations depends on (a) the ability to represent 2 states of affairs and (b) the ability to process negation in the context of an inconsistency. The authors discuss these results in relation to sources of competence that may underlie the assessment of such simple inconsistencies.

Human-monkey Gaze Correlations Reveal Convergent and Divergent Patterns of Movie Viewing

Current Biology : CB. Apr, 2010  |  Pubmed ID: 20303267

The neuroanatomical organization of the visual system is largely similar across primate species, predicting similar visual behaviors and perceptions. Although responses to trial-by-trial presentation of static images suggest that primates share visual orienting strategies, these reduced stimuli fail to capture key elements of the naturalistic, dynamic visual world in which we evolved. Here, we compared the gaze behavior of humans and macaques when they viewed three different 3-minute movie clips. We found significant intersubject and interspecies gaze correlations, suggesting that both species attend a common set of events in each scene. Comparing human and monkey gaze behavior with a computational saliency model revealed that interspecies gaze correlations were driven by biologically relevant social stimuli overlooked by low-level saliency models. Additionally, humans, but not monkeys, tended to gaze toward the targets of viewed individual's actions or gaze. Together, these data suggest that human and monkey gaze behavior comprises converging and diverging informational strategies, driven by both scene content and context; they are not fully described by simple low-level visual models.

I Can Make Your Brain Look Like Mine

Harvard Business Review. Dec, 2010  |  Pubmed ID: 21188897

Speaker-listener Neural Coupling Underlies Successful Communication

Proceedings of the National Academy of Sciences of the United States of America. Aug, 2010  |  Pubmed ID: 20660768

Verbal communication is a joint activity; however, speech production and comprehension have primarily been analyzed as independent processes within the boundaries of individual brains. Here, we applied fMRI to record brain activity from both speakers and listeners during natural verbal communication. We used the speaker's spatiotemporal brain activity to model listeners' brain activity and found that the speaker's activity is spatially and temporally coupled with the listener's activity. This coupling vanishes when participants fail to communicate. Moreover, though on average the listener's brain activity mirrors the speaker's activity with a delay, we also find areas that exhibit predictive anticipatory responses. We connected the extent of neural coupling to a quantitative measure of story comprehension and find that the greater the anticipatory speaker-listener coupling, the greater the understanding. We argue that the observed alignment of production- and comprehension-based processes serves as a mechanism by which brains convey information.

A Neuronal Basis for Task-negative Responses in the Human Brain

Cerebral Cortex (New York, N.Y. : 1991). Apr, 2011  |  Pubmed ID: 20805236

Neuroimaging studies have revealed a number of brain regions that show a reduced blood oxygenation level-dependent (BOLD) signal during externally directed tasks compared with a resting baseline. These regions constitute a network whose operation has become known as the default mode. The source of functional magnetic resonance imaging (fMRI) signal reductions in the default mode during task performance has not been resolved, however. It may be attributable to neuronal effects (neuronal firing), physiological effects (e.g., task vs. rest differences in respiration rate), or even increases in neuronal activity with an atypical blood response. To establish the source of signal decreases in the default mode, we used the calibrated fMRI method to quantify changes in the cerebral metabolic rate of oxygen (CMRO₂) and cerebral blood flow (CBF) in those regions that typically show reductions in BOLD signal during a demanding cognitive task. CBF:CMRO₂ coupling during task-negative responses were linear, with a coupling constant similar to that in task-positive regions, indicating a neuronal source for signal reductions in multiple brain areas. We also identify, for the first time, two modes of neuronal activity in this network; one in which greater deactivation (characterized by metabolic rate reductions) is associated with more effort and one where it is associated with less effort.

Interpretation-mediated Changes in Neural Activity During Language Comprehension

NeuroImage. Apr, 2011  |  Pubmed ID: 21232614

Using functional magnetic resonance imaging (fMRI), we identified cortical regions mediating interpretive processes that take place during language comprehension. We manipulated participants' interpretation of texts by asking them to focus on action-, space-, or time-related features while listening to identical short stories. We identify several cortical regions where activity varied significantly in response to this attention manipulation, even though the content being processed was exactly the same. Activity in the posterior and anterior sections of the left inferior frontal gyrus (IFG), which are thought to have different sensitivities to high-level language processing, was modulated by the listeners' attentional focus, but in ways that were quite different. The posterior left IFG (Pars Opercularis) showed different activity levels for the three conditions. However, a population coding analysis demonstrated similar distributions of activity across conditions. This suggests that while the gain of the response in the Pars Opercularis was modulated, its core organization was relatively invariant across the experimental conditions. In the anterior left IFG (Pars Triangularis), the analysis of population codes revealed different activity patterns between conditions: there was little similarity between activity during time-attention and action- and space-attention, however there were similar activity patterns while attending to space and action information. In addition, both the left superior temporal gyrus and sulcus showed greater activity in the space and action attention conditions when contrasted with time attention. We discuss these findings in light of work on the role of left IFG in processing semantic information in language, and in light of theories suggesting that temporal information in language is processed in the brain using similar mechanisms as spatial information. Our findings suggest that a substantial source of variance in neural activity during language comprehension emerges from the internally-driven, information-seeking preferences of listeners rather than the syntactic or semantic properties of a text.

Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story

The Journal of Neuroscience : the Official Journal of the Society for Neuroscience. Feb, 2011  |  Pubmed ID: 21414912

Real-life activities, such as watching a movie or engaging in conversation, unfold over many minutes. In the course of such activities, the brain has to integrate information over multiple time scales. We recently proposed that the brain uses similar strategies for integrating information across space and over time. Drawing a parallel with spatial receptive fields, we defined the temporal receptive window (TRW) of a cortical microcircuit as the length of time before a response during which sensory information may affect that response. Our previous findings in the visual system are consistent with the hypothesis that TRWs become larger when moving from low-level sensory to high-level perceptual and cognitive areas. In this study, we mapped TRWs in auditory and language areas by measuring fMRI activity in subjects listening to a real-life story scrambled at the time scales of words, sentences, and paragraphs. Our results revealed a hierarchical topography of TRWs. In early auditory cortices (A1+), brain responses were driven mainly by the momentary incoming input and were similarly reliable across all scrambling conditions. In areas with an intermediate TRW, coherent information at the sentence time scale or longer was necessary to evoke reliable responses. At the apex of the TRW hierarchy, we found parietal and frontal areas that responded reliably only when intact paragraphs were heard in a meaningful sequence. These results suggest that the time scale of processing is a functional property that may provide a general organizing principle for the human cerebral cortex.

The Relationship Between BOLD Signal and Autonomic Nervous System Functions: Implications for Processing of "physiological Noise"

Magnetic Resonance Imaging. Dec, 2011  |  Pubmed ID: 21543181

Functional magnetic resonance imaging (fMRI) research has revealed not only important aspects of the neural basis of cognitive and perceptual functions, but also important information on the relation between high-level brain functions and physiology. One of the central outstanding questions, given the features of the blood oxygenation level-dependent (BOLD) signal, is whether and how autonomic nervous system (ANS) functions are related to changes in brain states as measured in the human brain. A straightforward way to address this question has been to acquire external measurements of ANS activity such as cardiac and respiratory data, and examine their relation to the BOLD signal. In this article, we describe two conceptual approaches to the treatment of ANS measures in the context of BOLD fMRI analysis. On the one hand, several research lines have treated ANS activity measures as noise, considering them as nothing but a confounding factor that reduces the power of fMRI analysis or its validity. Work in this line has developed powerful methods to remove ANS effects from the BOLD signal. On the other hand, a different line of work has made important progress in showing that ANS functions such as cardiac pulsation, heart rate variability and breathing rate could be considered as a theoretically meaningful component of the signal that is useful for understanding brain function. Work within this latter framework suggests that caution should be exercised when employing procedures to remove correlations between BOLD data and physiological measures. We discuss these two positions and the reasoning underlying them. Thereafter, we draw on the reviewed literature in presenting practical guidelines for treatment of ANS data, which are based on the premise that ANS data should be considered as theoretically meaningful information. This holds particularly when studying cortical systems involved in regulation, monitoring and/or generation of ANS activity, such as those involved in decision making, conflict resolution and the experience of emotion.

Brain-to-brain Coupling: a Mechanism for Creating and Sharing a Social World

Trends in Cognitive Sciences. Feb, 2012  |  Pubmed ID: 22221820

Cognition materializes in an interpersonal space. The emergence of complex behaviors requires the coordination of actions among individuals according to a shared set of rules. Despite the central role of other individuals in shaping one's mind, most cognitive studies focus on processes that occur within a single individual. We call for a shift from a single-brain to a multi-brain frame of reference. We argue that in many cases the neural processes in one brain are coupled to the neural processes in another brain via the transmission of a signal through the environment. Brain-to-brain coupling constrains and shapes the actions of each individual in a social network, leading to complex joint behaviors that could not have emerged in isolation.

Temporal Eye Movement Strategies During Naturalistic Viewing

Journal of Vision. , 2012  |  Pubmed ID: 22262911

The deployment of eye movements to complex spatiotemporal stimuli likely involves a variety of cognitive factors. However, eye movements to movies are surprisingly reliable both within and across observers. We exploited and manipulated that reliability to characterize observers' temporal viewing strategies while they viewed naturalistic movies. Introducing cuts and scrambling the temporal order of the resulting clips systematically changed eye movement reliability. We developed a computational model that exhibited this behavior and provided an excellent fit to the measured eye movement reliability. The model assumed that observers searched for, found, and tracked a point of interest and that this process reset when there was a cut. The model did not require that eye movements depend on temporal context in any other way, and it managed to describe eye movements consistently across different observers and two movie sequences. Thus, we found no evidence for the integration of information over long time scales (greater than a second). The results are consistent with the idea that observers employ a simple tracking strategy even while viewing complex, engaging naturalistic stimuli.

Multiple Sensitivity Profiles to Diversity and Transition Structure in Non-stationary Input

NeuroImage. Jan, 2012  |  Pubmed ID: 22285219

Recent formalizations suggest that the human brain codes for the degree of order in the environment and utilizes this knowledge to optimize perception and performance in the immediate future. However, the neural bases of how the brain spontaneously codes for order are poorly understood. It has been shown that activity in lateral temporal cortex and the hippocampus is linearly correlated with the order of short visual series under tasks requiring attention to the input and when series order is invariant over time. Here, we examined if sensitivity to order is manifested in both linear and non-linear BOLD response profiles, quantified the degree to which order-sensitive regions operate as a functional network, and evaluated these questions using a paradigm in which performance of the ongoing task could be completed without any attention to the stimulus whose order was manipulated. Participants listened to a 10-minute sequence of tones characterized by non-stationary order, and fMRI identified cortical regions sensitive to time-varying statistical features of this input. Activity in perisylvian regions was negatively correlated with input diversity, quantified via Shannon's Entropy. Activity in ventral premotor, lateral temporal, and insular regions was correlated linearly, parabolically, or via a step-function with the strength of transition constraints in the series, quantified via Markov Entropy. Granger-causality analysis revealed that order-sensitive regions form a functional network, with regions showing non-linear responses to order associated with more afferent connectivity than those showing linear responses. These findings identify networks that spontaneously code and respond to diverse aspects of order via multiple response profiles, and that play a central role in generating and gating predictive neural activity.

Interspecies Activity Correlations Reveal Functional Correspondence Between Monkey and Human Brain Areas

Nature Methods. Feb, 2012  |  Pubmed ID: 22306809

Evolution-driven functional changes in the primate brain are typically assessed by aligning monkey and human activation maps using cortical surface expansion models. These models use putative homologous areas as registration landmarks, assuming they are functionally correspondent. For cases in which functional changes have occurred in an area, this assumption prohibits to reveal whether other areas may have assumed lost functions. Here we describe a method to examine functional correspondences across species. Without making spatial assumptions, we assessed similarities in sensory-driven functional magnetic resonance imaging responses between monkey (Macaca mulatta) and human brain areas by temporal correlation. Using natural vision data, we revealed regions for which functional processing has shifted to topologically divergent locations during evolution. We conclude that substantial evolution-driven functional reorganizations have occurred, not always consistent with cortical expansion processes. This framework for evaluating changes in functional architecture is crucial to building more accurate evolutionary models.

Syntactic Structure Building in the Anterior Temporal Lobe During Natural Story Listening

Brain and Language. Feb, 2012  |  Pubmed ID: 20472279

The neural basis of syntax is a matter of substantial debate. In particular, the inferior frontal gyrus (IFG), or Broca's area, has been prominently linked to syntactic processing, but the anterior temporal lobe has been reported to be activated instead of IFG when manipulating the presence of syntactic structure. These findings are difficult to reconcile because they rely on different laboratory tasks which tap into distinct computations, and may only indirectly relate to natural sentence processing. Here we assessed neural correlates of syntactic structure building in natural language comprehension, free from artificial task demands. Subjects passively listened to Alice in Wonderland during functional magnetic resonance imaging and we correlated brain activity with a word-by-word measure of the amount syntactic structure analyzed. Syntactic structure building correlated with activity in the left anterior temporal lobe, but there was no evidence for a correlation between syntactic structure building and activity in inferior frontal areas. Our results suggest that the anterior temporal lobe computes syntactic structure under natural conditions.