Projects Funded in the 2016-2017 Award Period


Mobile sign and gesture recognition system for communication in impoverished environments

Investigator Team: Ariel Goldberg, Psychology; Robert Jacob, Computer Science

Primary NSRDEC Collaborator: Tad Brunyé

The challenge of constructing systems to mediate gestural communication is determining how one should interact with a computer that typically plays no role in the interaction: e.g., how to report failed recognition of a gesture? How to indicate what gesture was recognized? This project thus links to Topic A: Understanding Humans and Supporting Systems. The ultimate goals of this project are to 1) create a mobile hardware-agnostic gesture recognition system that can be easily adapted to recognize new gestural systems (possibly even natural sign languages) and 2) investigate how users may best interact with a recognition system while gesturally communicating with other individuals (e.g., how can a police officer using hand signals to direct manned and driver-less vehicles in traffic be informed that the signals were not recognized by the system?) The purpose of this pilot is to tackle the technological challenges involved in capturing gestural information and using this information to recognize the intended gesture.


Developing noninvasive sensors for wireless monitoring and transmission of physiological status

Investigator Team: Fiorenzo Omenetto, Biomedical Engineering; Susan Roberts, USDA Human Nutrition Research Center on Aging

Primary NSRDEC Collaborator: Tad Brunyé

The center seeks to find paths to monitor and characterize cognitive and non-cognitive states (Topic B). The proposed approach hinges on the development of a skin mounted system sensing platform that will serve as a universal starting point to monitor physiological parameters. While we aim to leverage the large expertise existent at the nutrition school to validate a real-life scenario on markers of dietary anomalies through sweat monitoring, we will extend the approach to fatigue by adapting the platform to relevant markers such as lactic acid (in sweat) and look to extend to trace hormone detection in saliva. An electronic interface of this kind would provide continuous feedback on heath state of an individual under duress or intense physical activity allowing monitoring of the well being of an individual/a troop. The overall objective of this proposal is to develop a platform for a surface-mounted, conformal RF antenna and an associated readout/data interpretation system that will be able to extract physiological information from bodily fluids such as sweat and saliva.


Stereotype Threat and Performance of Mixed-Gender Groups on Spatial Reasoning Tasks

Investigator Team: Jessica Remedios, Psychology; Linda Tickle-Degnen, Occupational Therapy

Primary NSRDEC Collaborator: Joe Moran

As the Army explores mixed-gender units across operational specializations, the proposed research will examine whether groups as a whole underperform on spatial reasoning tasks when female group members constitute the gender minority. Anecdotes about personal experiences with stereotypes can inform this discussion, but scientific investigations are necessary to provide empirically-vetted strategies for promoting optimal group decision making. Performance in male-majority groups may be impeded (relative to gender-equal and female-majority groups) because women in male-majority groups are expected to contribute fewer and less optimal solutions to spatial problems. In such situations, women may be distracted by stereotypes that others may use to judge their performance. This work links to the objectives of the Center for Applied Brain and Cognitive Sciences (CABCS) because it aims to understand social factors involved in translating optimal individual performance into real-world, team-based contexts (Topic D). Specifically, the findings from this work may contribute to policy designed to address the challenges of gender diversity in groups in order to harness the many benefits of greater gender diversity in the Army.


Evaluation of Continuously Adjustable Autonomy for Line-of-sight Navigation of a Small Quadrotor

Investigator Team: Jason Rife, Mechanical Engineering; Holly Taylor, Psychology

Primary NSRDEC Collaborator: Matt Cain

The proposed research addresses Topic A (Understanding Humans and Supporting Systems). We will examine supervisory control of a small, partially automated Unmanned Aerial System (UAS). Specifically, we will test a system that adjusts the responsibilities of UAS autonomy based on real-time sensing of operator engagement, inferred from operator control commands and head tracking. The study has applications for a wide range of robotic support technologies that might be used in situations where operator engagement changes due to variable workload, fatigue, or stress. The outcome of the proposed work will be to test whether our new concept – that of continuously adjustable autonomy – has advantages over traditional adjustable autonomy (featuring discrete rather than continuous transitions between control modes). A second important outcome of the proposed work will be the development of a prototype system and test capability, which will enable additional future experiments related to UAS autonomy.


Interpreting verbal commands in visual and emotional context: A multimodal investigation

Investigator Team: Gina Kuperberg, Psychology; Remco Chang, Computer Science; Holly Taylor, Psychology; Meredith Brown, Psychology

Primary NSRDEC Collaborator: Marianna Eddy

The proposed research characterizes how the cognitive processes supporting real-time language understanding interact with non-cognitive emotional states. Characterizing these links has considerable implications for predicting how Soldiersunder emotional stress will interpret verbal instructions in context, and for tailoring commands under different circumstances (for example, optimizing tradeoffs between explicitness and economy of speech). This could help anticipate and avoid miscommunication between Soldiers. Our multi-modal experimental approach capitalizes on task-based eye-tracking paradigms linking language understanding to real-world visual environments, actions, and outcomes (the “visual world” paradigm) and on well-established neurophysiological signatures of language prediction (e.g. the N400 ERP component).


The Positive and Negative Consequences of Acute Stress on Memory

Investigator Team: Ayanna Thomas, Psychology; Michael Romero, Biology

Primary NSRDEC Collaborator: Caroline Davis

One of the Center’s objectives is to monitor, characterize, and optimize cognitive states. This project will directly address this objective by systematically examining the memory processes, physiological responses, and subjective feelings that are characterized by the cognitive state of acute psychological stress. Stress has typically been shown to have detrimental effects on memory. However, we aim to optimize the effects of stress on memory by capitalizing on two potential mechanisms: (1) the memory-enhancing immediate stress response, and (2) the use of retrieval enhanced learning to strengthen memory against the negative effects of stress. This research can go far in specifying the underexplored relationship between the biphasic stress response and memory, developing an applicable learning technique to help memory persist through stress (i.e., retrieval enhanced learning), and determining the cognitive
mechanism underlying the effectiveness of retrieval enhanced learning. Although stress is most commonly associated with poor memory performance, the proposed project will demonstrate that this does not always have to be the case. We propose that memory performance under acute stress can be managed by understanding the temporal relationship between stress and memory and by using learning strategies that serve to strengthen memory.


A Test of Two Brief Cognitive Mindsets to Improve Performance in High-Stakes Environments

Investigator Team: Heather Urry, Psychology; Jason Rife, Mechanical Engineering

Primary NSRDEC Collaborator: Caroline Davis

We aim to enhance and characterize cognitive and motor performance under varying conditions of acute psychological stress and physical fatigue using mindfulness and cognitive reappraisal techniques. More specifically, we seek to improve marksmanship performance in a virtual close quarters combat scenario. Multimodal dependent variable measures will include behavioral outcomes reflecting decision making (target selection) and motor execution (shot precision), autonomic nervous system response as measured via electrodermal and electrocardiographic measures, subjective self-report, and stress hormone quantification (salivary alpha-amylase). This interdisciplinary examination into the potential benefits of proven methods for redressing fear and anxiety could offer a significant advantage to people working in high-stress, physically-demanding professions. Given current scientific understanding of mindfulness and cognitive reappraisal, research into direct, real-world application is the next step in making immediate and impactful contributions. Funding the proposed project is imperative to launch the first steps in an innovative line of research geared towards improving the performance and health of people working in high-stakes environments, including military warfighters, fire fighters, police officers, surgeons, and other first responders.


Altering Multitasking Behavior Using Low Current Brain Stimulation

Investigator Team: Nathan Ward, Psychology; Holly Taylor, Psychology; Rob Jacob, Computer Science

Primary NSRDEC Collaborator: Erika Hussey

We propose a way to understand these multitasking deficits by asking how individuals may falter (e.g., via task-switching or dual-tasking or both) and how we can reduce these deficits using brain stimulation. We will use an experimental task that minimally introduces task-switching and dual-tasking demands. Different from prior work in these areas, this novel task presents these scenarios within-subjects and preserves low-level stimulus and response features that are often confounded in multitasking research. The proposed research has several theoretical and practical implications. First, pinpointing the contribution of multitasking processes will guide the development of individualized and targeted cognitive interventions aimed at improving individuals’ performance. Next, this work will provide evidence pertinent to the design of future neurostimulation interventions as a means to enhance cognition and behavior (i.e., beyond just multitasking). Finally, this line of research will have wide reaching applications for common real-world multitasking scenarios).


Comparing Spatial Awareness During Use of Wearable Navigational Aids

Investigator Team: Holly Taylor, Psychology; William Messner, Mechanical Engineering

Primary NSRDEC Collaborator: Aaron Gardony

Spatial awareness, defined as knowledge of oneself in space, allows people to place themselves in a larger world context. Having a good sense of self and environment locations within an environment framework should allow individuals to determine more efficient paths to target locations. This project explores how wearable technology using different sensory modalities might improve spatial awareness and problem solving. Wearables may allow individuals to develop sensory substitution, wherein they interpret sensory signals to have additional meaning, here using sounds or tactile stimuli to interpret cardinal directions. Through the project we are developing an auditory and a vibrotactile navigational wearable that provides cardinal direction information. We will gauge these wearables for the extent to which wearers gain increased awareness of cardinal directions within an environment. Further analyses will take into account wearer's assessments of device usability.