This column provides a brief synopsis and reference information for select recent journal publications. Note that this is only a short sampling of publications and is not intended as a comprehensive listing.
The Occupational Performance Assessment–Response Distortion (OPerA-RD) scale: Measuring bias in performance evaluation
The ubiquity and consequences of job performance evaluations (Campbell & Wiernik, 2015) necessitate accurate responding. However, response distortion or response bias can produce systematic tendency to respond to a range of questionnaire items on some basis other than the specific item content, and represents a significant threat to the veracity of performance evaluations. Examples of response bias include positive impression management and negative impression. Self-reported bias is often related to pervasive dispositional factors or temporary situational pressures (e.g., job interviews), whereas other-report bias (e.g., evaluations from supervisors or coworkers) may be influenced by ratees’ impression management (IM) tactics, raters’ self-interest or to motivate ratees (Murphy, 2008; Murphy & Cleveland, 1995), or unrepresentative observations (Berry et al., 2012; Murphy, 2008). Bias indicators may lack appropriate relevance or specificity, given that most bias indicators utilized in workplace research include noncontextualized, generic statements. This author aims to address these concerns by developing a response bias assessment contextualized to workplace settings. Converting non-contextualized statements (e.g., “I am a responsible person”) to workplace-contextualized statements (e.g., “I am a responsible employee”) may detect workplace response bias more accurately. Meta-analyses and reviews suggest that contextualizing assessments increases their predictive validity (e.g., Heggestad & Gordon, 2008; Shaffer & Postlethwaite, 2012). Although instruments such as personality assessments have been contextualized in this manner, it appears that contextualization has yet to be applied to bias indicators.
Williams, K. M. (2022). The Occupational Performance Assessment–Response Distortion (OPerA-RD) scale: Measuring bias in performance evaluation. Journal of Personnel Psychology, 21(4), 185–196. https://doi.org/10.1027/1866-5888/a000301opens in new window
Dynamic Fit Index Cutoffs for Confirmatory Factor Analysis Models
Despite their overwhelming popularity, methodological studies have cautioned against fixed cutoffs, noting that the meaning of fit indices varies based on a complex interaction of model characteristics like factor reliability, number of items, and number of factors. Criticism of fixed cutoffs stems primarily from the fact that they were derived from one specific confirmatory factor analysis model and lack generalizability. These authors propose a simulation-based method called dynamic fit index cutoffs such that derivation of cutoffs is adaptively tailored to the specific model and data characteristics being evaluated. To facilitate usage, an open-source, Web based Shiny software application is provided that automates the entire process so that users neither need to manually write any software code nor be knowledgeable about foundations of Monte Carlo simulation. The authors also provide an approach specifically designed for the nuances of 1-factor models, which have received surprisingly little attention in the literature despite frequent substantive interests in unidimensionality
McNeish, D., & Wolf, M. G. (2023). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychological Methods, 28(1), 61–88. https://doi.org/10.1037/met0000425opens in new window
Parsimony in Model Selection: Tools for Assessing Fit Propensity
There is a vast literature on model selection and multi-model inference that focuses on how to assess which statistical model, and therefore which theory, best fits the available data. For example, given some data, one can compare models on various information criterion or other fit statistics. However, what these indices fail to capture is the full range of counterfactuals. That is, some models may fit the given data better not because they represent a more correct theory, but simply because these models have more fit propensity—a tendency to fit a wider range of data, even nonsensical data, better. The authors offer a toolkit for researchers to better study and understand parsimony through the fit propensity of structural equation models. An R package (ockhamSEM) built on the popular lavaan package is described for this usage.
Falk, C. F., & Muthukrishna, M. (2023). Parsimony in model selection: Tools for assessing fit propensity. Psychological Methods, 28(1), 123–136. https://doi.org/10.1037/met0000422opens in new window
Deep Learning: A Primer for Psychologists
Deep learning has revolutionized predictive modeling in topics such as computer vision and natural language processing but is not commonly applied to psychological data. In an effort to bring the benefits of deep learning to psychologists, the authors provide an overview of deep learning for psychological researchers. The benefits of the deep learning approach to predictive modeling is described. Three basic deep learning models generalize linear regression: the feedforward neural network (FNN), the recurrent neural network (RNN), and the convolutional neural network (CNN). R code is provided to demonstrate how each model may be applied to answer prediction-focused research questions using common data types collected by psychologists. Urban, C. J., & Gates, K. M. (2021). Deep learning: A primer for psychologists. Psychological Methods, 26(6), 743–773. https://doi.org/10.1037/met0000374opens in new window
Guidelines for the “Publication Corner” Column:
Have you seen a recent publication that you think would be of great interest to Division 5 membership? Please feel free to share suggestions for publications to include in this column. Please send your ideas to The Score Editor, Gill Sitarenios. Please be sure to include full reference details and a short synopsis (2-3 sentences) of the publication. Use “Publication Corner” in the subject line of the email.