Steeling the team: Assessing individual and team functioning “at a distance”

Welcome to the Spotlight on Research column. This column showcases research activities and projects underway in many of the Research Laboratories within the U.S. Department of Defense, partnering organizations, and the academic and practitioner community in military psychology. Research featured in the column includes a wide variety of studies and programs, ranging from preliminary findings on single studies to more substantive summaries of programmatic efforts on targeted research topics. Research described in the column is inclusive of all disciplines relevant to military psychology—spanning the entire spectrum of psychology including clinical and experimental, as well as basic and applied. If you would like your work to be showcased in this column, please contact Krista Ratwani at (202) 552-6127.

The following article details an approach to unobtrusively monitoring team interaction behaviors. The unique approach involves analyzing the communication patterns of team members to pick out key phrases and words related to the cognitive and emotional states of team members. The ultimate goal of the presented research is to assess communication so as to develop real-time interventions that can enhance the effectiveness of teams operating within dynamic environments.

Research Overview

Real-world teams in military, aviation, space, and other demanding environments operate in a context that is dynamic, complex, and stressful. Teams are exposed to an array of environmental, task, and interpersonal stressors that can negatively impact performance as well as jeopardize the safety and well-being of team members. These demands may result in increased anxiety, negative emotion, distraction, conflict, and loss of team orientation—all of which can compromise mission effectiveness. Steeling or strengthening the team requires the capability to assess the cognitive/emotional state of team members and subsequently target interventions to counter these negative effects. This research describes an approach to dynamically and unobtrusively detect stress and related social, emotional, and cognitive deficits in ongoing team interactions, allowing interventions that mitigate and manage potential decrements in performance to be implemented should the need arise. The overall approach is described, and a use case of a spaceflight team is used to illustrate how ongoing team interactions can be assessed.

Problem to Solve

Considerable research has been conducted to examine the effects of stress on performance and to develop mitigation strategies to overcome these effects. There is also a rich legacy of research on team performance in military field settings, polar settings, submarines, submersibles, and in space (see Berkun, Bialek, Kern, & Yagi, 1962; J. E. Driskell & Olmstead, 1989; Harrison & Connors, 1984; Radloff & Helmreich, 1968).

However, unlike laboratory teams that can be examined “under a microscope,” teams in the real world operate autonomously, apart from direct observation and supervision, and operate in a fluid, dynamic manner to achieve the team's objective. Therefore, the requirement exists to develop non-obtrusive means of detecting cognitive performance deficits, stress, fatigue, or anxiety in situ without the intrusion of the psychologist's typical array of questions and questionnaires. One problem with many existing assessment methods is that most require direct observation of behavior or self-assessment by a pen and paper-type instrument (Brannick, Salas, & Prince, 1997). The requirement to assess individual and team functioning “at a distance” suggests the potential efficacy of a methodology to assess cognitive and emotional states in real-time from ongoing or spontaneous verbal output. In brief, we believe that we can track stress, anxiety, and related cognitive and emotional states in team performance settings via non-obtrusive monitoring of lexical output.

Some political psychologists (see Winter, Hermann, Weintraub, & Walker, 1991) have conducted significant work on the quantitative assessment of leadership “at a distance.” For example, Winter et al. (1991) attempted to assess characteristics such as distrust, power, or affiliation as exhibited by world leaders (typically political or military leaders) by performing content analysis of spontaneous verbal output drawn from interviews or other verbal records. It is interesting to note that in various settings, verbal or textual output has been used extensively to gauge a variety of cognitive, perceptual, and motivational constructs (for a review, see Pennebaker, Mehl, & Niederhoffer, 2003). For example, J. E. Driskell, Salas, and Driskell (2012) were able to assess deception through the analysis of communication between suspected coconspirators. Waller and Zimbelman (2003) have observed that the use of these types of textual/verbal materials allow the researcher to identify the “cognitive footprint” of ongoing, internal psychological processes from textual or verbal records.

The goal of the present research is to develop a methodology to assess team members' cognitive and emotional states “at a distance” through spontaneous verbal output in real-time communications. One product of this research will be an assessment tool to detect cognitive performance deficits, stress, fatigue, and anxiety that will provide an unobtrusive, real-time indicator of individual and team functioning as well as provide guidance for interventions to mitigate performance deficits.

Why Examine Lexical Output?

Lexical analysis refers to a research approach that analyzes speech or text in order to draw inferences regarding the text itself or the speaker's intentions, attitudes, or cognitions. Central to this approach is the emphasis on the importance of language as a means to draw inferences regarding the psychological state of the speaker. Recent research has documented the value of lexical analysis approaches in analyzing affect and cognition in real-world communications. For example, Khawaja, Chen, and Marcus (2012) examined the communication output of bushfire management teams to assess cognitive load and other indices of collaborative communications. T. Driskell, Blickensderfer, and Salas (2013) conducted a lexical analysis of dyadic and triadic communication to examine rapport in law enforcement investigative interviews.

The basic premise of this work is that spontaneous verbal output provides a natural and valid indicator of basic cognitive processes (Pennebaker et al., 2003). Natural word use is not prone to the typical limitations of self-report measurements. That is, natural language use is less subject to social desirability bias and can be derived in real-time without interfering with the cognitive processes being measured, and without interrupting team performance. Moreover, natural word use is reliable and consistent across time and context and can be meaningfully measured in individuals and teams (Glesser & Gottschalk, 1959; Mehl & Pennebaker, 2003). However, to date, there have been limited efforts to develop a lexical-based approach to tracking and mitigating cognitive/emotional deficits in team performance.

Solution and Approach

What is needed is a means of extracting valid indicators of the relevant elements of cognitive processes occurring during spaceflight from team members' spontaneous verbal output. In other words, how can one extract valid operational measures of stress, fatigue, or anxiety? The approach developed in this research is derived from recent developments in the study of associative meaning in linguistics and information science (e.g., Heylighen, 2001; Turney, 2001). Specifically, for any given construct or process, a lexicon of words indicative of that construct or process is developed. The relative prevalence of those words is used as an indicator of the degree to which that construct is engaged/that process is occurring.

The simplest and most straightforward approach would be to employ standard corpora of word association norms (e.g., Palermo & Jenkins, 1964). These word association norms have been derived by soliciting free associations from large samples of participants. For example, when presented with the stimulus word “sickness,” frequent free-associate response words are “ill,” “fever,” and “nausea.” An indicator of the construct of “sickness” in some ongoing interaction would be the relative prevalence in spontaneous verbal output of these high-frequency associate words.

However, there are several difficulties with the standard corpora of word association norms. For example, the largest and most highly cited of these corpora (Palermo & Jenkins, 1964) is now 50 years out of date. Recent research has resulted in alternative approaches to the study of associative meaning by using an Internet-based approach to examine lexical co-occurrence (Bardi, Calogero, & Mullen, 2008). Using readily available Internet search engines, the lexical co-occurrence of any array of words can be defined in terms of conditional probabilities. That is, for any given construct, a lexicon of words most indicative of that construct can be defined by examining the lexical co-occurrence of those words in online usage. In other words, rather than relying on a sample of research participants to indicate which words co-occur in their thoughts, this approach uses the massive lexical corpus of linguistic output on the Internet to determine which words actually co-occur in use. Spence and Owens (1990) have shown that conditional probabilities derived from lexical co-occurrence in Internet websites render results that are highly consonant with the patterns obtained from the standard corpora of word association norms. However , this approach has the advantages of being current, being capable of handling exhaustively large lexicons for scores of different constructs, and of being tailored to an y specific language of interest. Once such lexicons are developed, then analysis can be focused on automated tabulation of the relative prevalence of construct- or process-relevant lexicon words as a real-time, unobtrusive indicator of cognitive processes.

Findings

Broadly speaking, the primary problem to be addressed is that the demands or stress of the operational environment may result in cognitive/emotional deficits that are detrimental to team performance and well-being. We believe that there are a limited number of cognitive, emotional, and social mechanisms through which stress impacts performance. These “Big Five” stress mechanisms include the following: (a) stress increases distraction and decreases attentional focus, (b) stress increases cognitive load and demand on capacity, (c) stress increases negative affect and frustration, (d) stress increases fear and anxiety, and (e) stress increases social impairment. Therefore, a comprehensive operational measurement of stress would require, at a minimum, separate indices of attentional focus, cognitive demand, negative emotion, anxiety, and social impairment.

It may be useful to consider some of the ways that such indicators could be employed. For illustrative purposes, we have conducted a preliminary and limited analysis examining the Apollo 13 flight crew communications, drawn from the Apollo 13 Technical Air-to-Ground Voice Transcription (available from http://www.jsc.nasa.gov/history/). We selected samples of flight crew communications from three time periods during this mission. Time 1 was a sample of communication from Day 1 of the mission. Time 2 was a sample of communication taken after the explosion and rupture of oxygen tank number 2 in the service module on Day 2, approximately 56 hr into the mission. This time period is after the point in which Commander (CDR) Jim Lovell uttered the iconic phrase, “Houston, we've had a problem.” Time 3 was a sample of communications taken 5 days into the mission on the return.

In the following figures, we present a preliminary analysis of communications from CDR Lovell, Command Module Pilot (CMP) Jack Swigert, and Lunar Module Pilot Fred Haise. For each conversation sample, we used the Linguistic Inquiry and Word Count (LIWC) system as a tool for lexical analysis (Pennebaker, Chung, Ireland, Gonzales, & Booth, 2007). LIWC is a simple word count program that takes text files as input and attempts to match each word to an internal 4,500 word dictionary. Each word is then incremented into one of over 80 preexisting categories, including standard dimensions of linguistic style (e.g., percentage of words in the text that are pronouns, articles, quantifiers, etc.) and word content categories tapping psychological constructs (e.g., positive or negative affect, cognition, and social processes). In the following, we examined the use of positive emotion words and words related to anxiety (for this illustration, we simply used the preexisting default dictionary list of word associates provided by the LIWC program).

First, consider the use of these lexical indicators to gauge variations in basic cognitive processes over time. Figure 1 illustrates how the prevalence of Anxiety lexicon words in these samples varied over time. This figure indicates that anxiety peaks for all crew members after the explosion at Time 2 and decreases over time at Time 3. Examining communication at the individual level, Figure 1 indicates that at Time 2, the highest level of anxiety is exhibited by CMP Swigert. These real-time, unobtrusive indicators of process could be used to capture the topography of individual cognitive processes and cognitive performance deficits in a heretofore unrealized level of precision and specificity.

 

Figure 1 . Variation in Anxiety lexicon word prevalence for each crew member over time.
 

Second, consider the use of these lexical indicators to gauge convergence (or the lack thereof) over time for multiple crew members. Figure 2 illustrates how the relative prevalence of Negative Emotion lexicon words for the three crew members converge over the course of team collaboration. In this case, the fact that the team itself is experiencing high levels of Negative Emotion may indicate that the optimal strategy for mitigation may be a team-level intervention.

Figure 2 . Convergence in Negative Emotion for each crew member over time.

Alternatively, Figure 3 illustrates how the relative prevalence of Positive Emotion lexicon words for the three crew members fails to converge over time (i.e., CDR Lovell exhibits a distinctly lower level of Positive Emotion at Time 3 relative to the other crew members). In this case, the fact that a single crew member exhibits a lower level of Positive Emotion relative to other crew members suggests that that the optimal strategy for mitigation may be an individual-level intervention.

Figure 3 . Lack of convergence in Positive Emotion at Time 3.

Third, it is interesting to note the pattern of results shown in Figure 4, representing the usage of first person plural pronouns such as “we,” “us,” or “our” during crew interaction. J. E. Driskell, Salas, and Johnston (1999) found that the proportional usage of first-person plural pronouns correlated with high group-focus or team orientation in Naval decision-making teams. Figure 4 suggests a high level of team orientation and group-oriented communications peaking at the time of the explosion, especially on the part of CDR Lovell (note that we stretched the time periods out in Figure 4 for greater detail; Time Periods 1 and 2 preceded the accident at Time 3, and Time Periods 4 and 5 followed the accident). These types of analyses can provide an unobtrusive yet insightful window into leadership processes.

Figure 4 . Proportionate use of first person plural pronouns by crew members over time.

Finally, based on the type of data shown in the previous figures, we envision the development of data visualization tools that will allow us to track core variables at an individual and team level over time, as shown in Figure 5.

Figure 5 . Crew member status visualization tool.

Although the foregoing indicators of anxiety, emotional state, and team processes are preliminary and tentative, they are plausible elements that might be used to provide real-time assessment of stress and related cognitive deficits during ongoing team interactions from spontaneous verbal output.

Implications

The research describes a methodology to assess cognitive and emotional state “at a distance” through spontaneous verbal output in ongoing team interaction. The goal is to develop a real-time assessment tool to detect specific stress effects in the team operational setting and, ultimately, target countermeasures for ameliorating stress effects. These real-time, unobtrusive indicators of cognitive processes could be used to measure individual cognitive and emotional state without interfering with the process and performance of the team. This approach can be employed to gauge a full complement of constructs relevant to team performance, including stress, fatigue, and anxiety as well as team collaboration processes. Finally, this approach should be applicable to a variety of teams in high-demand environments such as military, space, and aviation settings.

References

Bardi, A., Calogero, R., & Mullen, B. (2008). A new archival approach to the study of values and value–behavior relations: Validation of the value lexicon. Journal of Applied Psychology, 93, 483–497.

Berkun, M., Bialek, H., Kern, R., & Yagi, K. (1962). Experimental studies of psychological stress in man. Psychological Monographs, 76 (15), 1–39.

Brannick, M. T., Salas, E., & Prince, C. (Eds.). (1997). Team performance assessment and measurement: Theory, methods, and applications. Hillsdale, NJ: LEA.

Driskell, J. E., & Olmstead, B. (1989). Psychology and the military: Research applications and trends. American Psychologist, 44, 43–54.

Driskell, J. E., Salas, E., & Driskell, T. (2012). Social indicators of deception. Human Factors, 54, 577–588.

Driskell, J. E., Salas, E., & Johnston, J. H. (1999). Does stress lead to a loss of team performance? Group Dynamics, 3, 291–302.

Driskell, T., Blickensderfer, E. L., & Salas, E. (2013). Is three a crowd? Examining rapport in investigative interviews. Group Dynamics: Theory, Research, and Practice, 17, 1–13.

Glesser, G. C., & Gottschalk, L. A. (1959). The relationship of sex and intelligence to choice of words: A normative study of verbal behavior. Journal of Clinical Psychology, 15, 182–191.

Harrison, A. A., & Connors, M. M. (1984). Groups in exotic environments. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 17, pp. 49–87). New York, NY: Academic Press.

Heylighen, F. (2001). Mining associative meanings from the Web: From word disambiguation to the global brain. In R. Temmerman (Ed.), Proceedings of trends in special language and language technology. Brussels, Belgium: Standard Publishers.

Khawaja, M., Chen, F., & Marcus, N. (2012). Analysis of collaborative communication for linguistic cues of cognitive load. Human Factors, 54, 518–529.

Mehl, M. R., & Pennebaker, J. W. (2003). The sounds of social life: A psychometric analysis of students' daily social environments and natural conversations. Journal of Personality and Social Psychology, 84, 857–870.

Palermo, D. S., & Jenkins, J. J. (1964). Word association norms: Grade school through college. Minneapolis: University of Minnesota Press.

Pennebaker, J. W., Chung, C. K., Ireland, M., Gonzales, A., & Booth, R. J. (2007). The development and psychometric properties of LIWC2007. Austin, TX: LIWC.net.

Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003). Psychological aspects of natural language use: Our words, our selves. Annual Review of Psychology, 54, 547–577.

Radloff, R., & Helmreich, R. (1968). Groups under stress: Psychological research in Sealab II. New York, NY: Appleton-Century-Crofts.

Spence, D. P., & Owens, K. C. (1990). Lexical co-occurrence and associative strength. Journal of Psycholinguistic Research, 19, 317–330.

Turney, P. (2001). Mining the Web for synonyms: PMI-IR versus LSA on TOEFL. In Proceedings of the 12th European Conference on Machine Learning (ECML 2001). Freiburg, Germany: Springer.

Waller, W. S., & Zimbelman, M. F. (2003). A cognitive footprint in archival data: Generalizing the dilution effect from laboratory to field settings. Organizational Behavior & Human Decision Processes, 91, 254–268.

Winter, D., Hermann, M., Weintraub, W., & Walker, S. (1991). The personalities of Bush and Gorbachev measured at a distance. Political Psychology, 12, 215–245.

 

Author Note
This research is supported by award NCC-9-58-401/NBPF03402 from NASA through the National Science Biomedical Research Institute.

For further information, please contact James E. Driskell, Florida Maxima Corporation.