Correspondence concerning this article should be addressed to:
Danka Maric, M.A.
STEM Education Innovation and Research Institute
Indiana University
755 W. Michigan, UL1123
Indianapolis, 46202
Email Danka Maric, MA
In 1971, Paul Meehl and his colleagues describe a general uneasiness felt by many around factor analysis. They argue that the cause is the largely subjective nature of factor interpretation. That is, when it comes to factor interpretation and naming, it is “an after-the-fact, ad hoc enterprise” (Meehl et al., 1971). At the time, there were no satisfactory criteria for evaluating factor interpretation.
Today, measurement is a critical issue, and adequate psychometric evidence is imperative to obtaining valid and reliable research results (DeVellis, 2017). There is a plethora of resources guiding researchers on factor retention, loading strength, and rotation options (e.g., DeVellis, 2017; Flora & Flake, 2017; Tabachnick & Fidell, 2014; Yong & Pearce, 2013). Aside from the Recapture-Item Technique suggested by Meehl and colleagues (1971), there is still a dearth of satisfactory guidance on factor interpretation. Indeed, factor naming is described as “more of an ‘art’” and as having no rules “except to give names that best represent the variables within the factors (Yong & Pearce, 2013).” Although we respect this method, we wish to offer an alternative approach for those that want something more systematic.
We suggest a mixed-methods approach, applying content analysis principles to the factor naming process. Content analysis is an analytical technique that involves detailed and systematic examination and interpretation of material with the goal of identifying patterns, themes, and meanings (Lune & Berg, 2017). It allows one to examine words, phrases, concepts, sentence structure, and semantics while considering theoretical frameworks and context. To illustrate, we describe our factor naming process for the Civic-minded Graduate (CMG; Steinberg et al., 2011) scale below.
Table 1
| Factor | Item |
|
Interdependence of Self and Others |
I want to dedicate my career to improving society. |
|
Valuing Civic Duty |
I intend to stay current with the local and national news. |
|
Applying Interpersonal Knowledge |
I listen to others and understand their perspective on controversial issues. |
|
Macro-awareness to Intervene |
I am prepared to write a letter to the newspaper or community leaders about a community issue. |
|
Micro-awareness to Intervene |
I know a lot about opportunities to become involved in the community. |
As part of a place-based experiential learning intervention study (Angstmann et al., 2019), we conducted a factor analysis of the CMG scale and found a 25-item, 5 factor structure (Maric et al., in press). Table 1 below displays the factors and items were retained in our structure. Initially, we (authors 1, 3, and 4) met and decided on preliminary factor names based on thematic interpretations and theoretical considerations. At this point, we decided that these names could be improved and wanted to be more systematic in our interpretations. Thus, using standard principles from content analysis (see Lune and Berg, 2017: 184), we examined items for sentence structure, coded them inductively, and then sorted them based on thematic categories. We (authors 1, 2, and 3) independently considered each factor and their items and met to discuss our interpretations. Through deliberation, we each explained our reasoning behind our interpretations according to perceived patterns and theoretical considerations until consensus was reached. Theoretical frameworks included process philosophy (Whitehead, 1978), postcolonial theories of subjectivity (Nyamnjoh, 2017), and interdependent self-construal (see Markus and Kitayama, 1991). Table 2 below displays factor names, key words/phrases, and the themes that were extracted.
Table 2
Factor Names, Key Words/Phrases Identified, and Themes Extracted
|
Factor Name |
Key Words/Phrases |
Themes |
|
Interdependence of Self and Others |
“I”; “careers”; “purpose of work”; “conviction”; “responsibility”; “sincere desire”; “improving society”; “serv[ing] others”; “complex” |
Connection to others and community; serving one’s community |
|
Valuing Civic Duty |
“intend”; “vote”; “involved”; “stay”; “I” or “me”; “important” |
Involvement with current events and politics |
|
Applying Interpersonal Knowledge |
I”; “listen”; “discuss…with civility”; “build consensus” |
Applying interpersonal skills such as consensus building and listening to others |
|
Macro-Awareness to Intervene |
“prepared”; “aware”; “convinced”; “I am”; “write”; “help solve”; “address”; issues”; “problems” |
Knowledge of issues, engagement, and intervention relative to larger societal structures |
|
Micro-Awareness to Intervene |
“I”; “community”; “community issues”; “plan”; “encourage”; “support”; “involved” |
Knowledge of issues, engagement, and intervention relative to local community structures |
For all the factors, we paid close attention to sentence structure, including verbs and subjects. We noticed that in all the items, that the subject was the survey-taking student (denoted by the words “I” or “me”). We considered how these verbs related to the subject and what that meant. For example, in Factor 1, the items’ verbs articulate the intentionality, desire, feelings, sense of identity, and belief of the subject (“I”). In some items, one’s work, career, and specialized knowledge is directed toward others and society. The “I” is connecting their identity, desires, values, and work toward external relationships consolidated within the notions of society and community, leading to the name “Interdependence of Self and Others.”
In Factor 2, we noted that each question focused on intent to be either aware or involved in current local and national news and politics. Coupled with an expression of perceived importance in one item, we interpreted this as implying a valuing of some kind. We determined that these items were each articulating a relationship between one’s valuing of and intent to stay informed and engaged with larger structures of civil society (e.g., news and politics) and named this factor “Valuing Civic Duty”. In Factor 3, we found that there was significant emphasis on interpersonal relationality in diverse contexts, where unified solutions to complex or controversial issues were to be fashioned. Given that the items were all referring to interpersonal skills but also emphasized the application of those skills, we agreed upon “Applying Interpersonal Knowledge”.
The last two factors were admittedly difficult to differentiate. In Factor 4 the verbs were each preceded by an auxiliary verb articulating a state of being (“am”) that referred to a subject (“I”) – the student, connecting the person to an action. Each item also contained “issues” or “problems” in the predicates. Thus, these items were each articulating a relationship between one’s awareness and know-how relative to engagement with larger structures of civil society (e.g., newspapers, leaders), as well as problems and issues that are multiple and “complex.” Thus, we agreed upon “Macro-awareness to intervene. In Factor 5, the subject (“I”) was consistently in relation to the sentences’ objects, such as “community” and “community issues.” Like Factor 4, the verbs in these items referred to actions, this time taken in relation to community, which is on a more micro-level scale. Thus, we settled on the factor name: “Building Micro-Awareness to Intervene.”
Through this technique, we were able to align our factor names more closely with Dewey’s ethical and democratic theories (e.g., Dewey, 1976) underpinning the scales items. Previous iterations of the CMG’s structure (i.e., Steinberg et al., 2011) were not so explicitly aligned with these theories, despite their importance to the items. This also brough the CMG closer to recent conceptualizations of civic-minded education (Kreber, 2016). This paper serves as a primer for others to take a more systematic approach to interpreting and naming factors and a foundation for future work to build upon. Above all, this work can encourage researchers to engage in interdisciplinary collaboration, finding creative ways to incorporate and apply quantitative and qualitative methods into their work.
Authors
Danka Maric, MA1
Grant A. Fore, PhD1
STEM Education Innovation and Research Institute, Indiana University, Indianapolis
Brandon H. Sorge, PhD2
O'Neill School of Public and Environmental Affairs, Indiana University, Indianapolis
Francesca A. Williamson, PhD3
University of Michigan School of Medicine
Julia L. Angstmann, PhD4
Office of Sustainability, Butler University