Projects Funded in the 2017-2018 Award Period


Understanding and Improving Augmented Reality Based Spatial Learning in a Networked Environment

Investigator Team: Fahad Dogar, Ph.D. (PI, Computer Science) and Nathan Ward, Ph.D. (Co-PI, Psychology)

Primary NSRDEC Collaborator: Aaron Gardony, Ph.D.

The focus of this project on using AR is an example of using innovative tools to better understand and enhance small team cognition. Focus on a networked environment allows multiple users to collaborate together on a spatial learning task, providing a new dimension to existing projects at CABCS which currently focus mostly on non-networked, single user scenarios. While this project focuses on collaborative spatial learning and navigation, insights gained from this effort will be applicable to a wide range of projects that involve VR/AR-based collaboration in a networked environment.


Developing noninvasive sensors for wireless monitoring and transmission of physiological status

Investigator Team: Fiorenzo Omenetto, Ph.D. (PI, Biomedical Engineering) and Susan Roberts, Ph.D. (Co-PI, USDA HNRCA)

Primary NSRDEC Collaborators: Grace Giles, Ph.D.

The overall objective is to develop a platform for a surface-mounted, conformal RF antenna and an associated readout/data interpretation system that will be able to extract physiological information from bodily fluids such as sweat and saliva. There will be two lines of body interfaces that will be pursued that are predicated on sampling and analyzing (i) human sweat and (ii) saliva. The goal is to expand approaches to further increase the sensitivity and reliability of these sensors with electrochemical strategies by printing and/or functionalizing the surfaces of the metals with appropriate reagents or antibodies for detection of fatigue relevant markers, additionally exploring the possibility of surface mounted oxymetry as a possible alternative strategy.


Image Processing Based Framework for Semantic Gaze Mapping

Investigator Team: Karen Panetta, Ph.D. (PI, Electrical and Computer Engineering) and Holly Taylor, Ph.D. (co-PI, Psychology)

Primary NSRDEC Collaborator: Erika Hussey, Ph.D.

The goal of this project is to deliver a user-friendly, robust approach to collect and analyze video data using advanced image processing algorithms. The proposed framework will lay groundwork for automating and coordinating image and verbal processing streams. The project refines an existing algorithm developed in Dr. Panetta’s Vision and Sensing System Lab. Currently, the prototype system architecture to automate mobile eye tracking video data collection and analysis has been developed for use in a laboratory setting. The system’s reliability will be tested with more difficult video data sets obtained in outdoor environments from mobile participants and refine its accuracy to account for environmental noise. This project also explores possibilities for multi-modal data analysis, starting with prosody analysis and possibly speech-to-text processing. The ideal outcome would be a reliable analytic framework to enhance future data mining and real-time performance monitoring.


Identifying novel biomarkers of inattention to promote learning and memory

Investigator Team: Elizabeth Race, Ph.D. (PI, Psychology); Holly Taylor, Ph.D. (Co-PI, Psychology); Robert Jacob, Ph.D. (Co-PI, Computer Science)

Primary NSRDEC Collaborator: Tad Brunyé, Ph.D., Aaron Gardony, Ph.D.

Currently, there is no reliable means of measuring internal fluctuations in attention that impair learning and memory. Behavioral markers of inattention are not always present in the moment (e.g., one can appear completely on-task even when attention is off-task), and performance decrements due to inattention may only become evident at later times (e.g., when memory is probed). The goal of this project is to identify an EEG biomarker of inattention that can predict whether incoming information will be retained in memory to inform the design of a passive, closed-loop brain-computer interface (BCI) that dynamically adapts a user’s environment in real-time to optimize learning and memory. 


A virtual platform for evaluating human-robot teaming

Investigator Team: Matthias Scheutz, Ph.D. (PI, Computer Science) and JP de Ruiter, Ph.D. (Co-PI, Psychology)

Primary NSRDEC Collaborator: Matthew Cain, Ph.D.

Increasingly, military teams are starting to incorporate robots into their organizational structures, to work alongside humans in shoulder-to-shoulder interaction (Jentsch, 2016). These complex team structures offer numerous advantages, but also present unsolved challenges to manage and coordinate the actions of all the agents. Currently, it is far from clear as to how such coordination should take place, as there are no studies that investigate mixed-initiative human-robot teams using autonomous robots. This project aims to directly address this gap in the literature by developing and testing a novel platform for human-robot teaming in virtual reality (VR) environments. The long term goal involves conducting a formal investigation and constructing a corpus to be used as a research tool and evaluation platform.


Multimodal Flexible Wireless Platforms for Psycho-physiological monitoring

Investigator Team: Sameer Sonkusale, Ph.D. (PI, Electrical and Computer Engineering) and Remco Chang, Ph.D. (Co-PI, Computer Science)

Primary NSRDEC Collaborators: Tad Brunyé

This project seeks to utilize nano and bioengineering technology to develop a novel, integrative, real-time psychophysiological sensor suite in a flexible fabric patch. By providing a platform that accomplishes what previously required multiple systems it will reduce the time required to collect and integrate several streams of information. Many of these measures were previously not observable in real-time and the quantification of respiration volume and sweat biomarkers is currently not an available capability. Additionally, many of the currently employed techniques only provide enough power for several hours of data streaming or recording. Through utilizing an adaptive sampling approach this project will enable long-term monitoring of user status, a valuable asset for extended events and operations.


Comparing Spatial Awareness During Use of Wearable Navigational Aids

Investigator Team: Holly Taylor, Ph.D. (PI, Psychology) and William Messner, Ph.D. (Co-PI, Mechanical Engineering)

Primary NSRDEC Collaborator: Aaron gardony, Ph.D.

The primary objectives of this project involve optimizing navigation performance by increasing spatial location and orientation awareness through wearable navigational aids. The navigational aids provide input through different sensory modalities, including two auditory formats (spatialized audio and verbal), two tactile formats (vibrotactile and heat), and visual, and research studies assessed effectiveness and usability based on modality. Next steps are to interface the wearable navigation aids with virtual environments, thus allowing us to explore the impact of the aids for learning new environments to be later navigated, and to develop alternative versions of the vibrotactile belt to vary signal processing location and signal meaning. 


Persistent Memory Through Stress

Investigator Team: Ayanna Thomas, Ph.D. (PI, Psychology) and Michael Romero, Ph.D. (Co-PI, Biology)

Primary NSRDEC Collaborator: Caroline Davis, Ph.D.

This projects aims to 1) examine the relationship between the physiological acute stress response and memory retrieval, 2) determine how retrieval enhanced learning moderates the negative effects of the delayed stress response on memory, and 3) to examine the value of retrieval practice in facilitating schema creation and flexible schema implementation in the context of a novel complex learning paradigm. Towards this end, our goal is to: (1) investigate the value of retrieval practice in developing schemas; (2) investigate the value of retrieval practice in facilitating the ability to incorporate new members of a particular class of stimuli into a previously learned group; (3) investigate the value of retrieval practice in facilitating discrimination processes. A further line of experimentation includes developing tasks and materials that allows for the investigation of retrieval practice on increasingly more difficult schema construction--for example, materials that require verbal, semantic, and spatial processing in order to be learned and understood.


A Test of Two Brief Cognitive Mindsets to Improve Performance in High-Stakes Environments

Investigator Team: Heather Urry, Ph.D. (PI, Psychology/CABCS); Shuchin Aeron, Ph.D. (Co-PI, Electrical and Computer Engineering); Eric Miller, Ph.D. (Co-PI, Electrical and Computer Engineering); Andrew Thompson, Ph.D.

Primary NSRDEC Collaborator: Caroline Davis, Ph.D., Tad Brunye, Ph.D.

The overall goal of this research is to examine whether invoking two, simple cognitive mindsets - mindfulness and cognitive reappraisal - can enhance cognitive and motor performance under conditions of high stress. The paradigm includes stress manipulation and mindfulness training for subjects engaged in a virtual reality simulation of tactical room clearing. The objective will be to use the large dataset collected to populate a machine learning (ML) algorithm that predicts performance outcomes, additionally collecting new data to cross-validate that algorithm.


Altering Multitasking Behavior Using Low Current Brain Stimulation

Investigator Team: Nathan Ward, Ph.D. (PI, Psychology); Holly Taylor, Ph.D. (Co-PI, Psychology) & Rob Jacob, Ph.D. (Co-PI, Computer Science)

Primary NSRDEC Collaborator: Tad Brunye, Ph.D.

Our current program of research investigates how transcranial direct current stimulation (tDCS) and transcranial alternating current stimulation (tACS) can be used to alter multitasking performance. Using tDCS, the aim is to temporarily and selectively boost task-switching and dual-tasking abilities by up-regulating neuronal excitability in brain areas uniquely supporting each. On the other hand, tACS will be used to target brain networks rather than specific regions of interest (ROIs). Administering both techniques using the same StarStim-8 devices allows for a novel comparison between ROI- versus network-based approaches to altering multitasking sub-processes. This targeted comparison has not yet been performed within the same experimental paradigm.


Epidermal Sensor prototype and novel strategies for data analysis

Investigator Team: Shuchin Aeron, Ph.D. (PI, Electrical and Computer Engineering); Eric Miller, Ph.D. (Co-PI, Electrical and Computer Engineering) & Fiorenzo Omenetto, Ph.D. (Co-PI, Biomedical Engineering)

Primary NSRDEC Collaborator: Tad Brunye, Ph.D.

This project has three primary objectives: 1) to accelerate the development of prototype epidermal sensors for field deployment in the form of electrochemically active sensing RF patches, sensing bracelets and chemoresponsive tattoo-barcodes, 2) to develop innovative analytical methods for the processing and interpretation of biosensors developed, and 3) to develop machine learning methods and real-time data processing and interpretation capabilities (including software and algorithms) to enable real-time Cognitive State Estimation (CSE) for the CABCS virtual reality laboratories, and ultimately, field deployment and testing.


Integrating eye-tracking into soldier-approved eyewear

Investigator Team: ameer Sonkusale, Ph.D. (PI, Electrical and Computer Engineering)

Primary NSRDEC Collaborator: Tad Brunye, Ph.D.

The objective of this pilot project is to develop a laboratory-grade quality (in terms of reliability and spatial and temporal resolution) eye tracking device based on SensoMotoric Instrument eye equipment (model ETG2) to accommodate the form factor and fit of unique constraints of PEO-Soldier approved eyewear, making it possible to wear eye tracking devices during field training exercises.


Innovative sham electrodes for transcranial direct current stimulation

Investigator Team: Sameer Sonkusale, Ph.D. (PI, Electrical and Computer Engineering)

Primary NSRDEC Collaborator: Tad Brunye, Ph.D.

The objective of this pilot project is to develop a more effective sham stimulation with innovation in electrode design and miniaturized circuitry. Current sham procedures involve briefly turning stimulation on then off at the beginning and end of an experimental session; this method does not reliably prevent participants from determining which experimental condition they are receiving (Brunye et al., 2014). For effective controls, there is a need to introduce more effective sham stimulation procedures that administer electrical current without penetrating the surface of the scalp, while having the effect to mimic the tingling and itching sensation so that participants cannot reliably distinguish active versus control experimental conditions.