Projects Funded in the 2015-2016 Award Period


Using neuro-cognitive multi-modal techniques to assess mental workload in real-world language contexts

Investigator Team: Phillip Holcomb, Psychology; Matthias Scheutz, Computer Science

Primary NSRDEC Collaborator: Marianna Eddy

This proposal links directly to Topic B: Monitor, Characterize and Optimize Cognitive and Non-Cognitive States, by establishing and testing EEG/behavioral (i.e., multi-modal) measures of mental workload in ecologically valid cognitive paradigms. Much of the prior work using neuro-cognitive methods (most prominently ERPs) to assess mental workload have used artificial secondary task paradigms (typically the oddball task) and have measured the P300 component. Here we propose to use well worked out naturalistic language processing tasks, which more closely parallel real-world multi-tasking environments, along with multi-modal techniques (behavior and EEG/ERPs) to assess mental workload under varying demand scenarios. One long term goal of this research is use neuro-cognitive measures of naturalistic language processing to optimize language comprehension in demanding scenarios.


A Pilot Study of the Influence of Different Urban Environments on Mental States

Investigator Team: Justin Hollander, Urban and Environmental Policy and Planning; Robert Jacob, Computer Science; Holly A. Taylor, Psychology

Primary NSRDEC Collaborators: Tad Brunyé, Marianna Eddy

Just as the Center’s work is around monitoring, characterizing, and optimize cognitive and non-cognitive states (Topic B), this project will use urban context to frame a series of research questions, including: How does urban context (weighting on four Cognitive Architecture principles) influence a soldier’s mental states? Can we predict how certain urban contexts will influence them and their abilities to respond to sudden tasks? In these different contexts, how can we optimize soldier performance by inducing higher levels of attention and meditative state? This research can go far in helping to understand the influence of the built environment on mental states, developing techniques for monitoring these effects, predicting how these effects will influence emotion, cognition and behavior, and seeking to optimize the fit between an individual’s task objectives and supporting mental states. There may be strategies for cueing particular emotional and cognitive states that can overcome potential performance decrements induced by environmental context.


Adding Interactivity to Brain Stimulation and Measurement

Investigator Team: Robert Jacob, Computer Science; Holly A. Taylor, Psychology

Primary NSRDEC Collaborator: Tad Brunyé

This undertaking directly relates to Topic B of the center’s objectives (Monitor, Characterize and Optimize Cognitive and Non-Cognitive States). Using functional near-infrared spectroscopy as input, we will explore the possibility of teaching machine learning software to automatically infer when a user is mentally taxed. We will dissect the learned patterns of the resulting machine learning algorithms for the purpose of better understanding the characteristics of cognitive workload, but our focus as computer scientists and human-computer interaction researchers will be exploring the engineering aspects of evaluating workload from a filtered fNIRS signal in terms of a minimal set of readily extractable features that can power a machine learning algorithm in real-time. Furthermore, we will explore the nature and time-course of interactions between monitoring, metrics development, and the delivery of optimization strategies such as low-current brain stimulation.


Neuromodulation of the Anterior Cingulate Cortex (ACC) via transcranial Direct Current Stimulation

Investigator Team: Lisa Shin, Psychology; Navneet Kaur, Psychology

Primary NSRDEC Collaborator: Tad Brunyé

Our research project is directly linked to Topic B of the Center’s topics of interest. Our project will entail establishing and testing multi-modal measures (i.e., combined tDCS and EEG) and metrics for monitoring and characterizing relevant cognitive states (i.e, behavioral tasks assessing cognitive and emotional interference). Furthermore, our project will foster collaborations to advance the field of neuromodulation and cognitive neuroscience. The results from this pilot study will provide a novel basis of comparison between two popular neurostimulation techniques and inform us which technique, traditional tDCS or HD-tDCS, may allow for stimulation of medial regions of the brain. Furthermore, the behavioral tasks will provide a basis for understanding the relationship between brain stimulation and cognitive and emotional interference. The next step would be to take the knowledge from the pilot study to assess whether it is possible to facilitate or inhibit specific medial frontal brain regions (i.e., dorsal and rostral ACC) and to assess the effects on cognitive and emotional interference by using tDCS combined with precise brain coordinates and computational models.


Measuring and Applying Cognitive Load

Investigator Team: Holly A. Taylor, Psychology; Matthias Scheutz, Computer Science; William C. Messner, Engineering

Primary NSRDEC Collaborator: Joe Moran

Topic B and Phase I: We develop a technique for assessing multiple dimensions of cognitive load applicable to a wide range of situations. Using a mobile eye tracker, we will monitor performance through think aloud and eye movements. Co-registration provides insights into specific mental processes underlying cognitive load. The data will allow us to model the relationship between the ambient light and pupillary responses. We will obtain a clear signal of cognitive load that accounts for ambient light by developing a mathematical model to remove the noise related to the light reflex. Relying on a lightweight mobile eye tracker for data collection and accounting for environmental noise during data analysis will enable assessment of cognitive load in real life settings. This will expand the range of possible experimental paradigms from strictly controlled laboratory studies to naturalistic designs.

Topic A and Phase II: Phase II involves utilizing the pupillometric signal developed in Phase I as a real-time reflection of cognitive load within an interactive user interface. The interactive system uses an adaptive learning system recognizing patterns of cognitive load in supporting multitasking during a human-robot activity.


Analyzing users' gaze and mouse interactions in Bayesian reasoning tasks

Investigator Team: Remco Chang, Computer Science; Paul Han, School of Medicine; Holly Taylor, Psychology

Primary NSRDEC Collaborators: Tad Brunyé, Joe Moran

In addition to establishing measures and characterizing risk decision making behaviors (Topic Area B), our long term goal is to develop an operational decision-making support tool (Topic Area A) to improve people’s Bayesian reasoning skills in high-stakes situations. Specifically, if combining visual illustration with textual description leads to worse performance in reasoning tasks, what is a better strategy? Furthermore, could this strategy be generalized to both printed (static) information as well as dynamic (computer-based) interactive tools? Through the eye-tracking and mouse-tracking experiments, we will collect large-scale data that can be used to derive quantitative models that will lead to the development of usable and operational decision-making support tools.