Thomas W. James
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Thomas W. James has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
Karin Harman James
3 shared publicationsLindsay R. Arcurio
Department of Psychological and Brain Sciences, Indiana University, Bloomington, USA
3 shared publicationsPeter R. Finn
Department of Psychological & Brain Sciences, Indiana University, Bloomington, USA
3 shared publications- Abolfazl Alipour
Young Researchers and Elite Club
2 shared publications - Daniel Levitas
Indiana University, Bloomington, USA
2 shared publications - Isabel Gauthier
Department of Psychology, Vanderbilt University
2 shared publications
Latest Publications
- Open Access
- 11 July 2025
- Published by MDPI AG in Preprints.org
Sensorimotor Mechanisms of Decisions and Actions
The field of decision neuroscience is a contributor to the replication crisis in psychology. Here, it was argued that a key reason is the confusion, misrepresentation, and conflation of the roles and definitions of mental phenomena/psychological constructs, behaviors/actions, and neural mechanisms/systems. Four common assumptions about decisions and their underlying systems were outlined and critiqued: 1) the underlying system has discrete processing stages, 2) the stages are linearly causal, 3) they are volitional, and 4) they rely on internal representations. Next, the definitions of mental phenomena (decisions), actions, and neural mechanisms were critiqued, finding that “neural mechanisms” in current decision neuroscience are merely reflections of the mental phenomena they purport to explain. The next, and most radical, argument was that decisions cannot be the cause of actions (without accepting a dualist perspective). Decisions and actions are both caused by neural mechanisms and the resulting correlation makes them appear to be causally related. Finally, it was recommended that the approach taken in autonomous robotics be used as a scaffold for conceiving of the neural mechanisms of decisions and actions.
- 11 July 2025
- Published by MDPI AG in Preprints.org
Open AccessSensorimotor Mechanisms of Decisions and Actions
The field of decision neuroscience is a contributor to the replication crisis in psychology. Here, it was argued that a key reason is the confusion, misrepresentation, and conflation of the roles and definitions of mental phenomena/psychological constructs, behaviors/actions, and neural mechanisms/systems. Four common assumptions about decisions and their underlying systems were outlined and critiqued: 1) the underlying system has discrete processing stages, 2) the stages are linearly causal, 3) they are volitional, and 4) they rely on internal representations. Next, the definitions of mental phenomena (decisions), actions, and neural mechanisms were critiqued, finding that “neural mechanisms” in current decision neuroscience are merely reflections of the mental phenomena they purport to explain. The next, and most radical, argument was that decisions cannot be the cause of actions (without accepting a dualist perspective). Decisions and actions are both caused by neural mechanisms and the resulting correlation makes them appear to be causally related. Finally, it was recommended that the approach taken in autonomous robotics be used as a scaffold for conceiving of the neural mechanisms of decisions and actions.
- 15 March 2024
- Published by SAGE Publications in Brain Connectivity
- Vol. 14 (2) , 92-106
Increased Segregation in Functional Connectivity Networks When Watching Unpleasant Arousing Videos: A Generalized Psychophysiological Interaction Analysis
Background: Properties of functional connectivity (FC), such as network integration and segregation, are shown to be associated with various human behaviors. For example, Godwin et al. and Sun et al. found increased integration with attention allocation, whereas Cohen and D'Esposito and Shine et al. observed increased segregation with simple motor tasks. The current study investigated how viewing video clips with different valence and arousal influenced integration–segregation properties in task-based FC networks. Methods: We analyzed an open dataset collected by Kim et al. We performed a generalized psychophysiological interaction (gPPI) analysis paired with network analysis and community detection to investigate changes in brain network dynamics when people watched four types of videos that differed by affective valence (unpleasant or pleasant) and arousal (arousing or calm). Results: Results showed that unpleasant arousing videos produced greater FC deviation from the baseline (task-induced FC deviation [tiFCd]) and perturbed the brain into a more segregated state than other kinds of video. Increased segregation was only observed in association systems, not sensorimotor systems. Discussion: Unpleasant arousing content perturbed the brain to a functionally distinct state from the other three types of affective videos. We suggest that the change in brain state was related to people disengaging from the unpleasant arousing content or, alternatively, staying alert while exposed to unpleasant arousing stimuli. The study also added to our understanding of how combining task-based gPPI analysis with community detection methods and network segregation measures can advance our knowledge of the links between behavior and brain state changes. Impact statement: Network integration and segregation is an important property of the human brain. We address the question of how affective stimuli influence brain dynamics from a functional connectivity (FC) network integration–segregation perspective. By conducting a whole-brain generalized psychophysiological interaction (gPPI) analysis paired with community detection methods, we found that highly aversive video content induced significant FC changes and perturbed the brain to a more segregated state.
- 15 March 2024
- Published by SAGE Publications in Brain Connectivity
- Vol. 14 (2) , 92-106
Increased Segregation in Functional Connectivity Networks When Watching Unpleasant Arousing Videos: A Generalized Psychophysiological Interaction Analysis
Background: Properties of functional connectivity (FC), such as network integration and segregation, are shown to be associated with various human behaviors. For example, Godwin et al. and Sun et al. found increased integration with attention allocation, whereas Cohen and D'Esposito and Shine et al. observed increased segregation with simple motor tasks. The current study investigated how viewing video clips with different valence and arousal influenced integration–segregation properties in task-based FC networks. Methods: We analyzed an open dataset collected by Kim et al. We performed a generalized psychophysiological interaction (gPPI) analysis paired with network analysis and community detection to investigate changes in brain network dynamics when people watched four types of videos that differed by affective valence (unpleasant or pleasant) and arousal (arousing or calm). Results: Results showed that unpleasant arousing videos produced greater FC deviation from the baseline (task-induced FC deviation [tiFCd]) and perturbed the brain into a more segregated state than other kinds of video. Increased segregation was only observed in association systems, not sensorimotor systems. Discussion: Unpleasant arousing content perturbed the brain to a functionally distinct state from the other three types of affective videos. We suggest that the change in brain state was related to people disengaging from the unpleasant arousing content or, alternatively, staying alert while exposed to unpleasant arousing stimuli. The study also added to our understanding of how combining task-based gPPI analysis with community detection methods and network segregation measures can advance our knowledge of the links between behavior and brain state changes. Impact statement: Network integration and segregation is an important property of the human brain. We address the question of how affective stimuli influence brain dynamics from a functional connectivity (FC) network integration–segregation perspective. By conducting a whole-brain generalized psychophysiological interaction (gPPI) analysis paired with community detection methods, we found that highly aversive video content induced significant FC changes and perturbed the brain to a more segregated state.
- Open Access
- 6 March 2024
- Published by Wiley in Human Brain Mapping
- Vol. 45 (4) , e26648
Dynamic threat–reward neural processing under semi‐naturalistic ecologically relevant scenarios
Studies of affective neuroscience have typically employed highly controlled, static experimental paradigms to investigate the neural underpinnings of threat and reward processing in the brain. Yet our knowledge of affective processing in more naturalistic settings remains limited. Specifically, affective studies generally examine threat and reward features separately and under brief time periods, despite the fact that in nature organisms are often exposed to the simultaneous presence of threat and reward features for extended periods. To study the neural mechanisms of threat and reward processing under distinct temporal profiles, we created a modified version of the PACMAN game that included these environmental features. We also conducted two automated meta‐analyses to compare the findings from our semi‐naturalistic paradigm to those from more constrained experiments. Overall, our results revealed a distributed system of regions sensitive to threat imminence and a less distributed system related to reward imminence, both of which exhibited overlap yet neither of which involved the amygdala. Additionally, these systems broadly overlapped with corresponding meta‐analyses, with the notable absence of the amygdala in our findings. Together, these findings suggest a shared system for salience processing that reveals a heightened sensitivity toward environmental threats compared to rewards when both are simultaneously present in an environment. The broad correspondence of our findings to meta‐analyses, consisting of more tightly controlled paradigms, illustrates how semi‐naturalistic studies can corroborate previous findings in the literature while also potentially uncovering novel mechanisms resulting from the nuances and contexts that manifest in such dynamic environments.
- 6 March 2024
- Published by Wiley in Human Brain Mapping
- Vol. 45 (4) , e26648
Open AccessDynamic threat–reward neural processing under semi‐naturalistic ecologically relevant scenarios
Studies of affective neuroscience have typically employed highly controlled, static experimental paradigms to investigate the neural underpinnings of threat and reward processing in the brain. Yet our knowledge of affective processing in more naturalistic settings remains limited. Specifically, affective studies generally examine threat and reward features separately and under brief time periods, despite the fact that in nature organisms are often exposed to the simultaneous presence of threat and reward features for extended periods. To study the neural mechanisms of threat and reward processing under distinct temporal profiles, we created a modified version of the PACMAN game that included these environmental features. We also conducted two automated meta‐analyses to compare the findings from our semi‐naturalistic paradigm to those from more constrained experiments. Overall, our results revealed a distributed system of regions sensitive to threat imminence and a less distributed system related to reward imminence, both of which exhibited overlap yet neither of which involved the amygdala. Additionally, these systems broadly overlapped with corresponding meta‐analyses, with the notable absence of the amygdala in our findings. Together, these findings suggest a shared system for salience processing that reveals a heightened sensitivity toward environmental threats compared to rewards when both are simultaneously present in an environment. The broad correspondence of our findings to meta‐analyses, consisting of more tightly controlled paradigms, illustrates how semi‐naturalistic studies can corroborate previous findings in the literature while also potentially uncovering novel mechanisms resulting from the nuances and contexts that manifest in such dynamic environments.
- Open Access
- 15 November 2023
- Published by Wiley in Brain and Behavior
- Vol. 13 (12) , e3312
Impact of aversive affect on neural mechanisms of categorization decisions
Introduction: Many theories contend that evidence accumulation is a critical component of decision‐making. Cognitive accumulation models typically interpret two main parameters: a drift rate and decision threshold. The former is the rate of accumulation, based on the quality of evidence, and the latter is the amount of evidence required for a decision. Some studies have found neural signals that mimic evidence accumulators and can be described by the two parameters. However, few studies have related these neural parameters to experimental manipulations of sensory data or memory representations. Here, we investigated the influence of affective salience on neural accumulation parameters. High affective salience has been repeatedly shown to influence decision‐making, yet its effect on neural evidence accumulation has been unexamined.Methods: The current study used a two‐choice object categorization task of body images (feet or hands). Half the images in each category were high in affective salience because they contained highly aversive features (gore and mutilation). To study such quick categorization decisions with a relatively slow technique like functional magnetic resonance imaging, we used a gradual reveal paradigm to lengthen cognitive processing time through the gradual “unmasking” of stimuli.Results: Because the aversive features were task‐irrelevant, high affective salience produced a distractor effect, slowing decision time. In visual accumulation regions of interest, high affective salience produced a longer time to peak activation. Unexpectedly, the later peak appeared to be the product of changes to both drift rate and decision threshold. The drift rate for high affective salience was shallower, and the decision threshold was greater. To our knowledge, this is the first demonstration of an experimental manipulation of sensory data or memory representations that changed the neural decision threshold.Conclusion: These findings advance our knowledge of the neural mechanisms underlying affective responses in general and the influence of high affective salience on object representations and categorization decisions.
- 15 November 2023
- Published by Wiley in Brain and Behavior
- Vol. 13 (12) , e3312
Open AccessImpact of aversive affect on neural mechanisms of categorization decisions
Introduction: Many theories contend that evidence accumulation is a critical component of decision‐making. Cognitive accumulation models typically interpret two main parameters: a drift rate and decision threshold. The former is the rate of accumulation, based on the quality of evidence, and the latter is the amount of evidence required for a decision. Some studies have found neural signals that mimic evidence accumulators and can be described by the two parameters. However, few studies have related these neural parameters to experimental manipulations of sensory data or memory representations. Here, we investigated the influence of affective salience on neural accumulation parameters. High affective salience has been repeatedly shown to influence decision‐making, yet its effect on neural evidence accumulation has been unexamined.Methods: The current study used a two‐choice object categorization task of body images (feet or hands). Half the images in each category were high in affective salience because they contained highly aversive features (gore and mutilation). To study such quick categorization decisions with a relatively slow technique like functional magnetic resonance imaging, we used a gradual reveal paradigm to lengthen cognitive processing time through the gradual “unmasking” of stimuli.Results: Because the aversive features were task‐irrelevant, high affective salience produced a distractor effect, slowing decision time. In visual accumulation regions of interest, high affective salience produced a longer time to peak activation. Unexpectedly, the later peak appeared to be the product of changes to both drift rate and decision threshold. The drift rate for high affective salience was shallower, and the decision threshold was greater. To our knowledge, this is the first demonstration of an experimental manipulation of sensory data or memory representations that changed the neural decision threshold.Conclusion: These findings advance our knowledge of the neural mechanisms underlying affective responses in general and the influence of high affective salience on object representations and categorization decisions.
- 26 September 2023
- Published by Elsevier in Neuropsychologia
- Vol. 190, 108695
Neural segregation and integration of sensory, decision, and action processes during object categorization
- 26 September 2023
- Published by Elsevier in Neuropsychologia
- Vol. 190, 108695
Neural segregation and integration of sensory, decision, and action processes during object categorization
- 10 August 2021
- Published by Springer Nature in Cognitive Neurodynamics
- Vol. 16 (1) , 149-165
A computational examination of the two-streams hypothesis: which pathway needs a longer memory?
The two visual streams hypothesis is a robust example of neural functional specialization that has inspired countless studies over the past four decades. According to one prominent version of the theory, the fundamental goal of the dorsal visual pathway is the transformation of retinal information for visually-guided motor behavior. To that end, the dorsal stream processes input using absolute (or veridical) metrics only when the movement is initiated, necessitating very little, or no, memory. Conversely, because the ventral visual pathway does not involve motor behavior (its output does not influence the real world), the ventral stream processes input using relative (or illusory) metrics and can accumulate or integrate sensory evidence over long time constants, which provides a substantial capacity for memory. In this study, we tested these relations between functional specialization, processing metrics, and memory by training identical recurrent neural networks to perform either a viewpoint-invariant object classification task or an orientation/size determination task. The former task relies on relative metrics, benefits from accumulating sensory evidence, and is usually attributed to the ventral stream. The latter task relies on absolute metrics, can be computed accurately in the moment, and is usually attributed to the dorsal stream. To quantify the amount of memory required for each task, we chose two types of neural network models. Using a long-short-term memory (LSTM) recurrent network, we found that viewpoint-invariant object categorization (object task) required a longer memory than orientation/size determination (orientation task). Additionally, to dissect this memory effect, we considered factors that contributed to longer memory in object tasks. First, we used two different sets of objects, one with self-occlusion of features and one without. Second, we defined object classes either strictly by visual feature similarity or (more liberally) by semantic label. The models required greater memory when features were self-occluded and when object classes were defined by visual feature similarity, showing that self-occlusion and visual similarity among object task samples are contributing to having a long memory. The same set of tasks modeled using modified leaky-integrator echo state recurrent networks (LiESN), however, did not replicate the results, except under some conditions. This may be because LiESNs cannot perform fine-grained memory adjustments due to their network-wide memory coefficient and fixed recurrent weights. In sum, the LSTM simulations suggest that longer memory is advantageous for performing viewpoint-invariant object classification (a putative ventral stream function) because it allows for interpolation of features across viewpoints. The results further suggest that orientation/size determination (a putative dorsal stream function) does not benefit from longer memory. These findings are consistent with the two visual streams theory of functional specialization.
- 10 August 2021
- Published by Springer Nature in Cognitive Neurodynamics
- Vol. 16 (1) , 149-165
A computational examination of the two-streams hypothesis: which pathway needs a longer memory?
The two visual streams hypothesis is a robust example of neural functional specialization that has inspired countless studies over the past four decades. According to one prominent version of the theory, the fundamental goal of the dorsal visual pathway is the transformation of retinal information for visually-guided motor behavior. To that end, the dorsal stream processes input using absolute (or veridical) metrics only when the movement is initiated, necessitating very little, or no, memory. Conversely, because the ventral visual pathway does not involve motor behavior (its output does not influence the real world), the ventral stream processes input using relative (or illusory) metrics and can accumulate or integrate sensory evidence over long time constants, which provides a substantial capacity for memory. In this study, we tested these relations between functional specialization, processing metrics, and memory by training identical recurrent neural networks to perform either a viewpoint-invariant object classification task or an orientation/size determination task. The former task relies on relative metrics, benefits from accumulating sensory evidence, and is usually attributed to the ventral stream. The latter task relies on absolute metrics, can be computed accurately in the moment, and is usually attributed to the dorsal stream. To quantify the amount of memory required for each task, we chose two types of neural network models. Using a long-short-term memory (LSTM) recurrent network, we found that viewpoint-invariant object categorization (object task) required a longer memory than orientation/size determination (orientation task). Additionally, to dissect this memory effect, we considered factors that contributed to longer memory in object tasks. First, we used two different sets of objects, one with self-occlusion of features and one without. Second, we defined object classes either strictly by visual feature similarity or (more liberally) by semantic label. The models required greater memory when features were self-occluded and when object classes were defined by visual feature similarity, showing that self-occlusion and visual similarity among object task samples are contributing to having a long memory. The same set of tasks modeled using modified leaky-integrator echo state recurrent networks (LiESN), however, did not replicate the results, except under some conditions. This may be because LiESNs cannot perform fine-grained memory adjustments due to their network-wide memory coefficient and fixed recurrent weights. In sum, the LSTM simulations suggest that longer memory is advantageous for performing viewpoint-invariant object classification (a putative ventral stream function) because it allows for interpolation of features across viewpoints. The results further suggest that orientation/size determination (a putative dorsal stream function) does not benefit from longer memory. These findings are consistent with the two visual streams theory of functional specialization.
- 1 April 2021
- Published by Springer Nature in Brain Imaging and Behavior
- Vol. 15 (2) , 1007-1024
Neural Modulation in Approach-Avoidance Conflicts in Externalizing Psychopathology
Externalizing psychopathology (EXT) is characterized by poor decision-making in situations that involve simultaneous cues for approach and avoidance behavior (i.e. approach-avoidance conflicts). Previous studies of EXT have examined these deficits primarily using tasks involving decisions between positive reward and negative punishment, suggesting that EXT is characterized by a general bias towards high salience (e.g. temporally proximal or reward) cues relative to low salience (e.g. temporally distal or loss) cues. However, in order to better characterize decision-making in approach-avoidance conflicts, the present study utilized a novel task to examine neural activation in contexts involving both positive reward and negative punishment as well as positive punishment and negative reward by manipulating physical proximity of affective cues. Neuroimaging results indicated that EXT was associated with deficits related to cue prioritization based on salience, suggesting that failure to differentiate relevant from less relevant information contributes to poor decision-making among individuals with EXT.
- 1 April 2021
- Published by Springer Nature in Brain Imaging and Behavior
- Vol. 15 (2) , 1007-1024
Neural Modulation in Approach-Avoidance Conflicts in Externalizing Psychopathology
Externalizing psychopathology (EXT) is characterized by poor decision-making in situations that involve simultaneous cues for approach and avoidance behavior (i.e. approach-avoidance conflicts). Previous studies of EXT have examined these deficits primarily using tasks involving decisions between positive reward and negative punishment, suggesting that EXT is characterized by a general bias towards high salience (e.g. temporally proximal or reward) cues relative to low salience (e.g. temporally distal or loss) cues. However, in order to better characterize decision-making in approach-avoidance conflicts, the present study utilized a novel task to examine neural activation in contexts involving both positive reward and negative punishment as well as positive punishment and negative reward by manipulating physical proximity of affective cues. Neuroimaging results indicated that EXT was associated with deficits related to cue prioritization based on salience, suggesting that failure to differentiate relevant from less relevant information contributes to poor decision-making among individuals with EXT.
- 19 March 2021
- Published by Springer Nature in Psychopharmacology
- Vol. 238 (7) , 1867-1883
Neural mechanisms of sexual decision-making in women with alcohol use disorder
Rationale The co-occurrence of alcohol consumption and sexual activity is associated with increased risk for sexual assault, sexually transmitted disease, and unplanned pregnancy among young adult women with alcohol use disorder (AUD). There is considerable previous work demonstrating neural reactivity to alcohol cues in AUD. Because alcohol consumption and sexual behavior are both rewarding and tend to co-occur, sexual cues may produce similar neural reactivity in women with AUD, possibly indicating a shared mechanism underlying reactivity to both types of cues. Alternatively, reactivity to alcohol versus sexual cues may be distinct, suggesting domain-specific mechanisms. Objectives We investigated whether the decision vulnerabilities in AUD women regarding sexual activity were related to differences in brain activation compared to control women. Methods Women with (n = 15) and without (n = 16) AUD completed a hypothetical decision-making task during fMRI that presented low- or high-risk scenarios involving visual sexual, appetitive, and neutral cues. Results Results showed that sexual cues were more often endorsed by women with AUD compared to controls and elicited differential brain activation patterns in frontal, visual, and reward regions. During high-risk decisions, women with AUD failed to downregulate activation, causing hyperactivation compared to controls. Conclusions Visual sexual cues produced reactivity like that previously demonstrated for alcohol cues, suggesting a shared or domain-general mechanism for alcohol and sexual cue reactivity in women with AUD. Riskier sexual decisions in women with AUD may be a consequence of repeatedly pairing alcohol use and sexual activity, a characteristic behavior of this population.
- 19 March 2021
- Published by Springer Nature in Psychopharmacology
- Vol. 238 (7) , 1867-1883
Neural mechanisms of sexual decision-making in women with alcohol use disorder
Rationale The co-occurrence of alcohol consumption and sexual activity is associated with increased risk for sexual assault, sexually transmitted disease, and unplanned pregnancy among young adult women with alcohol use disorder (AUD). There is considerable previous work demonstrating neural reactivity to alcohol cues in AUD. Because alcohol consumption and sexual behavior are both rewarding and tend to co-occur, sexual cues may produce similar neural reactivity in women with AUD, possibly indicating a shared mechanism underlying reactivity to both types of cues. Alternatively, reactivity to alcohol versus sexual cues may be distinct, suggesting domain-specific mechanisms. Objectives We investigated whether the decision vulnerabilities in AUD women regarding sexual activity were related to differences in brain activation compared to control women. Methods Women with (n = 15) and without (n = 16) AUD completed a hypothetical decision-making task during fMRI that presented low- or high-risk scenarios involving visual sexual, appetitive, and neutral cues. Results Results showed that sexual cues were more often endorsed by women with AUD compared to controls and elicited differential brain activation patterns in frontal, visual, and reward regions. During high-risk decisions, women with AUD failed to downregulate activation, causing hyperactivation compared to controls. Conclusions Visual sexual cues produced reactivity like that previously demonstrated for alcohol cues, suggesting a shared or domain-general mechanism for alcohol and sexual cue reactivity in women with AUD. Riskier sexual decisions in women with AUD may be a consequence of repeatedly pairing alcohol use and sexual activity, a characteristic behavior of this population.
- Open Access
A computational examination of the two-streams hypothesis: which pathway needs a longer memory?
The two visual streams hypothesis is a robust example of neural functional specialization that has inspired countless studies over the past four decades. According to one prominent version of the theory, the fundamental goal of the dorsal visual pathway is the transformation of retinal information for visually-guided motor behavior. To that end, the dorsal stream processes input using absolute (or veridical) metrics only when the movement is initiated, necessitating very little, or no, memory. Conversely, because the ventral visual pathway does not involve motor behavior (its output does not influence the real world), the ventral stream processes input using relative (or illusory) metrics and can accumulate or integrate sensory evidence over long time constants, which provides a substantial capacity for memory. In this study, we tested these relations between functional specialization, processing metrics, and memory by training identical recurrent neural networks to perform either a viewpoint-invariant object classification task or an orientation/size determination task. The former task relies on relative metrics, benefits from accumulating sensory evidence, and is usually attributed to the ventral stream. The latter task relies on absolute metrics, can be computed accurately in the moment, and is usually attributed to the dorsal stream. To quantify the amount of memory required for each task, we chose two types of neural network models. Using a long-short-term memory (LSTM) recurrent network, we found that viewpoint-invariant object categorization (object task) required a longer memory than orientation/size determination (orientation task). Additionally, to dissect this memory effect, we considered factors that contributed to longer memory in object tasks. First, we used two different sets of objects, one with self-occlusion of features and one without. Second, we defined object classes either strictly by visual feature similarity or (more liberally) by semantic label. The models required greater memory when features were self-occluded and when object classes were defined by visual feature similarity, showing that self-occlusion and visual similarity among object task samples are contributing to having a long memory. The same set of tasks modeled using modified leaky-integrator echo state recurrent networks (LiESN), however, did not replicate the results, except under some conditions. This may be because LiESNs cannot perform fine-grained memory adjustments due to their network-wide memory coefficient and fixed recurrent weights. In sum, the LSTM simulations suggest that longer memory is advantageous for performing viewpoint-invariant object classification (a putative ventral stream function) because it allows for interpolation of features across viewpoints. The results further suggest that orientation/size determination (a putative dorsal stream function) does not benefit from longer memory. These findings are consistent with the two visual streams theory of functional specialization.
Open AccessA computational examination of the two-streams hypothesis: which pathway needs a longer memory?
The two visual streams hypothesis is a robust example of neural functional specialization that has inspired countless studies over the past four decades. According to one prominent version of the theory, the fundamental goal of the dorsal visual pathway is the transformation of retinal information for visually-guided motor behavior. To that end, the dorsal stream processes input using absolute (or veridical) metrics only when the movement is initiated, necessitating very little, or no, memory. Conversely, because the ventral visual pathway does not involve motor behavior (its output does not influence the real world), the ventral stream processes input using relative (or illusory) metrics and can accumulate or integrate sensory evidence over long time constants, which provides a substantial capacity for memory. In this study, we tested these relations between functional specialization, processing metrics, and memory by training identical recurrent neural networks to perform either a viewpoint-invariant object classification task or an orientation/size determination task. The former task relies on relative metrics, benefits from accumulating sensory evidence, and is usually attributed to the ventral stream. The latter task relies on absolute metrics, can be computed accurately in the moment, and is usually attributed to the dorsal stream. To quantify the amount of memory required for each task, we chose two types of neural network models. Using a long-short-term memory (LSTM) recurrent network, we found that viewpoint-invariant object categorization (object task) required a longer memory than orientation/size determination (orientation task). Additionally, to dissect this memory effect, we considered factors that contributed to longer memory in object tasks. First, we used two different sets of objects, one with self-occlusion of features and one without. Second, we defined object classes either strictly by visual feature similarity or (more liberally) by semantic label. The models required greater memory when features were self-occluded and when object classes were defined by visual feature similarity, showing that self-occlusion and visual similarity among object task samples are contributing to having a long memory. The same set of tasks modeled using modified leaky-integrator echo state recurrent networks (LiESN), however, did not replicate the results, except under some conditions. This may be because LiESNs cannot perform fine-grained memory adjustments due to their network-wide memory coefficient and fixed recurrent weights. In sum, the LSTM simulations suggest that longer memory is advantageous for performing viewpoint-invariant object classification (a putative ventral stream function) because it allows for interpolation of features across viewpoints. The results further suggest that orientation/size determination (a putative dorsal stream function) does not benefit from longer memory. These findings are consistent with the two visual streams theory of functional specialization.
- Open Access
- 8 July 2020
- Published by Springer Nature in Learning & Behavior
- Vol. 48 (4) , 393-394
Cross-modal tactile-visual neural representations in bumble bees
The centuries-old philosophical question posed by William Molyneux and referred to by John Locke (1689) concerns the transfer of object information from touch to vision. Solvi, Al-Khudhairy, and Chittka (2020) tested this question comparatively in bumble bees and found evidence of crossmodal object recognition between touch and vision.
- 8 July 2020
- Published by Springer Nature in Learning & Behavior
- Vol. 48 (4) , 393-394
Open AccessCross-modal tactile-visual neural representations in bumble bees
The centuries-old philosophical question posed by William Molyneux and referred to by John Locke (1689) concerns the transfer of object information from touch to vision. Solvi, Al-Khudhairy, and Chittka (2020) tested this question comparatively in bumble bees and found evidence of crossmodal object recognition between touch and vision.