Projects Funded in the 2018-2019 Award Period

 

Developing Noninvasive Sensors for Wireless Monitoring and Transmission of Physiological Status

Investigator Team: Fiorenzo Omenetto, Ph.D. (PI, Biomedical Engineering) and Susan Roberts, Ph.D. (Co-PI, USDA HNRCA)

Primary NSRDEC Collaborators: Grace Giles, Ph.D.

The overall objective is to develop a platform for a surface-mounted, conformal RF antenna and an associated readout/data interpretation system that will be able to extract physiological information from bodily fluids such as sweat and saliva. There will be two lines of body interfaces that will be pursued that are predicated on sampling and analyzing (i) human sweat and (ii) saliva. The goal is to expand approaches to further increase the sensitivity and reliability of these sensors with electrochemical strategies by printing and/or functionalizing the surfaces of the metals with appropriate reagents or antibodies for detection of fatigue relevant markers, additionally exploring the possibility of surface mounted oxymetry as a possible alternative strategy.

 

Image Processing Based Framework for Semantic Gaze Mapping

Investigator Team: Karen Panetta, Ph.D. (PI, Electrical and Computer Engineering) and Holly Taylor, Ph.D. (co-PI, Psychology)

Primary NSRDEC Collaborator: Erika Hussey, Ph.D.

The goal of this project is to deliver a user-friendly, robust approach to collect and analyze video data using advanced image processing algorithms. The proposed framework will lay groundwork for automating and coordinating image and verbal processing streams. The project refines an existing algorithm developed in Dr. Panetta’s Vision and Sensing System Lab. Currently, the prototype system architecture to automate mobile eye tracking video data collection and analysis has been developed for use in a laboratory setting. The system’s reliability will be tested with more difficult video data sets obtained in outdoor environments from mobile participants and refine its accuracy to account for environmental noise. This project also explores possibilities for multi-modal data analysis, starting with prosody analysis and possibly speech-to-text processing. The ideal outcome would be a reliable analytic framework to enhance future data mining and real-time performance monitoring, enhanced with multi-user applications and spoken word detection. 

 

Identifying Novel Biomarkers of Inattention to Promote Learning and Memory

Investigator Team: Elizabeth Race, Ph.D. (PI, Psychology); Holly Taylor, Ph.D. (Co-PI, Psychology); Robert Jacob, Ph.D. (Co-PI, Computer Science)

Primary NSRDEC Collaborator: Tad Brunyé, Ph.D., Aaron Gardony, Ph.D.

Currently, there is no reliable means of measuring internal fluctuations in attention that impair learning and memory. Behavioral markers of inattention are not always present in the moment (e.g., one can appear completely on-task even when attention is off-task), and performance decrements due to inattention may only become evident at later times (e.g., when memory is probed). Behavior analysis on human participants reports attention fluctuations between on-task and off-task focus with off-task focus 67% of the time. This research uses EEG and machine learning to charaterize and monitor attentional states that optimize learning and promote long-term retention of information. Our goal is to then use this neural biomarker to drive a brain-computer interface that will provide passive neurofeedback to enhance memory in a variety of vigilance situations without burdening primary task performance. An additional set of analyses will be conducted using machine learning algorithms and pattern classification to identify moment-to-moment brain states that predict subsequent memory. 
 

A Virtual Platform for Evaluating Human-Robot Teaming

Investigator Team: Matthias Scheutz, Ph.D. (PI, Computer Science) and JP de Ruiter, Ph.D. (Co-PI, Psychology)

Primary NSRDEC Collaborator: Matthew Cain, Ph.D.

Increasingly, military teams are starting to incorporate robots into their organizational structures, to work alongside humans in shoulder-to-shoulder interaction (Jentsch, 2016). These complex team structures offer numerous advantages, but also present unsolved challenges to manage and coordinate the actions of all the agents. Currently, it is far from clear as to how such coordination should take place, as there are no studies that investigate mixed-initiative human-robot teams using autonomous robots. This project aims to directly address this gap in the literature by developing and testing a novel platform for human-robot teaming in virtual reality (VR) environments. The long term goal involves conducting a formal investigation and constructing a corpus to be used as a research tool and evaluation platform.

 

Comparing Spatial Awareness During Use of Wearable Navigational Aids

Investigator Team: Holly Taylor, Ph.D. (PI, Psychology) and William Messner, Ph.D. (Co-PI, Mechanical Engineering)

Primary NSRDEC Collaborator: Aaron gardony, Ph.D.

The primary objectives of this project involve optimizing navigation performance by increasing spatial location and orientation awareness through wearable navigational aids. The navigational aids provide input through different sensory modalities, including two auditory formats (spatialized audio and verbal), two tactile formats (vibrotactile and heat), and visual, and research studies assessed effectiveness and usability based on modality. Next steps are to interface the wearable navigation aids with virtual environments, thus allowing us to explore the impact of the aids for learning new environments to be later navigated, and to develop alternative versions of the vibrotactile belt to vary signal processing location and signal meaning. 

 

Persistent Memory Through Stress

Investigator Team: Ayanna Thomas, Ph.D. (PI, Psychology) and Michael Romero, Ph.D. (Co-PI, Biology)

Primary NSRDEC Collaborator: Caroline Davis, Ph.D.

This projects aims to 1) examine the relationship between the physiological acute stress response and memory retrieval, 2) determine how retrieval enhanced learning moderates the negative effects of the delayed stress response on memory, and 3) to examine the value of retrieval practice in facilitating schema creation and flexible schema implementation in the context of a novel complex learning paradigm. Towards this end, our goal is to: (1) investigate the value of retrieval practice in developing schemas; (2) investigate the value of retrieval practice in facilitating the ability to incorporate new members of a particular class of stimuli into a previously learned group; (3) investigate the value of retrieval practice in facilitating discrimination processes. A further line of experimentation includes developing tasks and materials that allows for the investigation of retrieval practice on increasingly more difficult schema construction--for example, materials that require verbal, semantic, and spatial processing in order to be learned and understood.

 

A Test of Two Brief Cognitive Mindsets to Improve Performance in High-Stakes Environments

Investigator Team: Heather Urry, Ph.D. (PI, Psychology/CABCS); Shuchin Aeron, Ph.D. (Co-PI, Electrical and Computer Engineering); Eric Miller, Ph.D. (Co-PI, Electrical and Computer Engineering); Andrew Thompson, Ph.D.

Primary NSRDEC Collaborator: Caroline Davis, Ph.D., Tad Brunye, Ph.D.

The overall goal of this research is to examine whether invoking two, simple cognitive mindsets - mindfulness and cognitive reappraisal - can enhance cognitive and motor performance under conditions of high stress. The paradigm includes stress manipulation and mindfulness training for subjects engaged in a virtual reality simulation of tactical room clearing. The objective will be to use the large dataset collected to populate a machine learning (ML) algorithm that predicts performance outcomes, additionally collecting new data to cross-validate that algorithm.

 

Altering Multitasking Behavior Using Low Current Brain Stimulation

Investigator Team: Nathan Ward, Ph.D. (PI, Psychology); Holly Taylor, Ph.D. (Co-PI, Psychology) & Rob Jacob, Ph.D. (Co-PI, Computer Science)

Primary NSRDEC Collaborator: Tad Brunye, Ph.D.

Our current program of research investigates how transcranial direct current stimulation (tDCS) and transcranial alternating current stimulation (tACS) can be used to alter multitasking performance. Using tDCS, the aim is to temporarily and selectively boost task-switching and dual-tasking abilities by up-regulating neuronal excitability in brain areas uniquely supporting each. On the other hand, tACS will be used to target brain networks rather than specific regions of interest (ROIs). Administering both techniques using the same StarStim-8 devices allows for a novel comparison between ROI- versus network-based approaches to altering multitasking sub-processes. This targeted comparison has not yet been performed within the same experimental paradigm.

 

Integrating Eye-Tracking Into Soldier-Approved Eyewear

Investigator Team: Sameer Sonkusale, Ph.D. (PI, Electrical and Computer Engineering); Karen Panetta, Ph.D. (PI, Electrical and Computer Engineering)

Primary NSRDEC Collaborator: Tad Brunye, Ph.D.

The objective of this pilot project is to develop a laboratory-grade quality (in terms of reliability and spatial and temporal resolution) eye tracking device based on SensoMotoric Instrument eye equipment (model ETG2) to accommodate the form factor and fit of unique constraints of PEO-Soldier approved eyewear, making it possible to wear eye tracking devices during field training exercises.

 

Numerical Modeling of Transcranial Direct Current Stimulation

Investigator Team: Luis Dorfmann, Ph.D. (PI, Civil and Environmental Engineering); David Kaplan, Ph.D. (Co-PI, Biomedical Engineering) 

Primary NSRDEC Collaborator: Erika Hussey, Ph.D. 

The primary objective of this research is to address some of the limitations of transcranial direct current stimulation. Currently there are no in vivo measurements of tDCS-induced electric fields in humans, so current work relies heavily on neurocomputational models to estimate the electric field produced by tDCS. This creates issues where only preselected head shapes are available, for example. Other issues include the actual conductivity and relative permittivity of tissues in the head. These are cerrently treated as constants, but may not be. The research will determine the extent of these limitations and inconsistencies within them. 

 

Development of a Bidirectional Probe Integrating fNIRS and tDCS for Closed Loop Brain Stimulation

Investigator Team: Valencia Koomson, Ph.D. (PI, Electrical and Computer Engineering); Robert Jacob, Ph.D. (PI, Computer Science)

Primary NSRDEC Collaborator: Erika Hussey, Ph.D., Grace Giles, Ph.D.

The objective of this proposal is to develop a brain machine interface probe to enable simultaneous transcranial direct current simulation (tDCS) and measurement of associate changes in oxygenated hemoglobin using functional near infrared spectroscopy (fNIRS) tools. Previously reported experimental data and clinical studies have shown the benefits of tDCS for a variety of Soldier-relevant cognitive use cases, including sustained attention and vigilance, working memory, response inhibition, and multitasking. 

 

fNIRS for Assessing Neurocognitive Synchrony in Small Teams

Investigator Team: Angelo Sassaroli, Ph.D. (PI, Biomedical Engineering); Nathan Ward, Ph.D. (Psychology)

Primary NSRDEC Collaborator: Tad Brunyé, Ph.D. 

This work is motivated by the importance of characterizing and assessing the level of cooperation and shared understanding of team members during team interactions in realistic settings. Functional near infrared spectroscopy (fNIRS) will be used to assess the covariation of hemodynamic brain signals from individual team members during cooperation towards a common task goal. The goal is to try to determine if covariation is predictive of quantitative team performance outcomes. 

 

An Augmented Reality System and Framework for Human-Machine Collaborative Teaming

Investigator Team: Jivko Sinapov, Ph.D. (PI, Computer Science); Chris Rogers, Ph.D. (PI, Mechanical Engineering)

Primary NSRDEC Collaborator: Matt Cain, Ph.D. 

The overall goal of this project is to develop and evaluate an Augmented Reality (AR) framework that bridges the gap between a robot's representation of the world, and that of the human. Humans and robots interact often both in the home and on the battlefield or in disaster scenarios, but humans rarely understand how a robot interprets our world which limits the overall effectiveness of the human robot coordination. Our hypothesis is that by "seeing the unseen", humans will coordinate with robots more effectively in high stress and cognitive load situations.

 

Reality Augmentation and Learning: How to Successfully Employ Technology to Enhance Learning

Investigator Team: Ayanna K. Thomas, Ph.D.; Remco Chang, Ph.D.; Marianna Eddy, Ph.D.

Primary NSRDEC Collaborator: Aaron Gardony, Ph.D.

This research team will examine factors that promote efficient and effective learning. To accomplish this aim, we will employ well-established findings in cognitive psychology (retrieval enhanced learning, transfer appropriate processing) to examine their influence on complex learning in technologically enhanced environments. This research is important, because the military is required to quickly master complex information and has access to technological tools that may enhance learning. However, it is unlcear whether learning using those tools will constrain how that information can later be retrieved. We will investigate how technological tools can be most effectively used to promote retrieval in novel contexts. 

 

Measuring and Applying Cognitive Load

Investigator Team: Holly A. Taylor, Ph.D. (PI, Psychology); Matthias Scheutz, Ph.D. (Co-PI, Computer Science); William C. Messner, Ph.D. (Co-PI, Engineering)

Primary NSRDEC Collaborator: Joe Moran, Ph.D. 

As technology becomes more pervasive and powerful, handling a complex user interface and coping with information overload imposes a workload often exceeding the user's cognitive capacity. High cognitive load increases stress and decreases performance. Ideally an interactive system could recognize changes in a user's cognitive state and adjust worload demands appropriately, resulting in better task performance and improved user experience. Current cognitive state can be monitered using physiological measures such as galvanic skin response, electromyography, and brain sensing techniques such as EEG or fNIRS. An interactive system can account for the user's limitations by modifying current demands based on such measurements, instead using pupillometry, a robust indicator of mental load. We aim to develop an experimental and analytic paradigm that will overcome the challenge of human-computer interaction and the pupil's sensitivity to luminance during the collection of data.