This column provides a brief synopsis and reference information for select recent journal publications. Note that this is only a short sampling of publications and is not intended as a comprehensive listing.
Journal of Behavioral Data Science (JBDS)
A new interdisciplinary journal aiming to provide a free-of-charge and open-access journal to researchers and practitioners in the area of data science and data analytics. The journal covers topics on data science, data analytics, and data analysis. The journal is open to accept submissions now. Visit their websiteopens in new window for more information about the journal can be found online.
p-Hacking and Publication Bias Interact to Distort Meta-Analytic Effect Size Estimates
A large-scale simulation study was conducted to examine how p-Hacking and publication bias distort meta-analytic effect size estimates under a broad variety of circumstances. Very high levels of publication bias can severely distort the cumulative evidence. Moreover, p-Hacking and publication bias interact – at relatively high and low levels of publication bias, p-Hacking does relatively little harm, however at medium levels of publication bias, p-Hacking can considerably contribute to bias. p-Hacking can also severely increase the rate of false positives. The authors assert that scientific journals need to make the prevention of publication bias a top priority to ensure a trustworthy base of evidence.
Friese, M., & Frankenbach, J. (2020). p-Hacking and publication bias interact to distort meta-analytic effect size estimates. Psychological Methods, 25, 456-471. https://dx.doi.org/10.1037/met0000246
Improved Inference in Mediation Analysis: Introducing the Model-Based Constrained Optimization Procedure
When examining indirect effects popular approaches, including confidence interval-based methods tend to produce poor Type 1 error rates when mediation does not occur and only allow dichotomous decisions (significant or not). A new Likelihood Ratio Test (LRT) is proposed that uses non-linear restraints in what is termed a model-based constrained optimization (MBCO) procedure. This procedure (1) offers a more robust Type 1 error rate, (2) provides a p value which serves as a continuous measure of compatibility of data with the hypothesized null model, (3) allows simple and complex hypotheses about mediation, and (4) allows the mediation model to use observed or latent variables. The MBCO procedure is recommended for testing hypotheses about an indirect effect in addition to reporting a confidence interval to capture uncertainty about the indirect effect because this combination transcends existing methods.
Tofighi, D., & Kelley, K. (2020). Improved inference in mediation analysis: Introducing the model-based constrained optimization procedure. Psychological Methods, 25, 496-515. https://dx.doi.org/10.1037/met0000259
Managing Validity Versus Reliability Trade-Offs in Scale-Building Decisions
The goal of this article is to make clear that the source of the validity versus reliability trade-offs is systematic error that contributes to item communality. Non-trivial trade-offs exist in the scale building process. Six common trade-offs are described. Those related to item content, item construction, item difficulty, item scoring, item order, and item analysis. Scale builders are encouraged to specify how reliability and validity are being prioritized and apply these considerations with consistency.
Clifton, J. D. W. (2020). Managing validity versus reliability trade-offs in scale-building decisions. Psychological Methods, 25, 259-270. https://dx.doi.org/10.1037/met0000236
Guidelines for the ‘Publication Corner’ column
Have you seen a recent publication that you think would be of great interest to Div. 5 membership? Please feel free to share suggestions for publications to include in this column. Please send your ideas to The Score editor, Gill Sitarenios. Please be sure to include full reference details and a short synopsis (2–3 sentences) of the publication. Use “Publication Corner” in the subject line of the email.