Cecil Reynolds, PhD, is a prolific scientist-practitioner, who has published 55 books as an author or editor and over 150 peer-reviewed articles. He is also the author or co-author of 37 psychological assessments covering a broad range of domains, including (but not limited to) personality, behavior, neuropsychology and intelligence. He has been the president of three APA divisions, including Div. 5 (Quantitative and Qualitative Methods), Div. 16 (School Psychology) and Div. 40 (Society for Clinical Neuropsychology). Reynolds has also been the editor-in-chief of Archives of Clinical Neuropsychology, Applied Neuropsychology, Psychological Assessment® and currently edits Archives of Scientific Psychology®, APA’s open access data-sharing journal. In this interview, Reynolds talks about his mentors, career path and what he envisions for the future of psychological assessment.
Was there anyone, or perhaps several people, who had a particularly strong influence on your development as a psychologist?
One person that I point to was an undergraduate professor, Robert Brown, who was at the University of North Carolina at Wilmington, where I did my undergraduate degree. He is a classically trained experimental psychologist, and I took every class he taught in the psychology department. He taught me to be a scientist as much as anybody. He was data driven and data based, but always had theory tied to it. He very much taught you to be driven by where the data were taking you and to reject theories that didn’t have good support or that couldn’t be tested. He used to say, “If it can’t be tested, then it’s not science — it’s philosophy.” He had a tremendous influence on me and how I think about data and research problems. I still think about things he taught me as an undergraduate student. Those things are just as valuable now, or more valuable now, as I mature than they were then. So he had a tremendous influence at the undergraduate level in driving me to want to become a good scientist.
For my graduate studies at the University of Georgia, I was just very fortunate to be able to work with Alan Kaufman. He was in charge of developing the first revision of a Wechsler scale (Wechsler Intelligence Scale for Children — Revised; Wechsler, 1974). He came to Georgia the same year I did, only he came as a professor and I came as a student. Since it was his first year on the faculty, he didn’t have any graduate students assigned to him. He went through the incoming students and I was just one of the ones he picked. I am so fortunate that the fit was so good, and in essence, happenstance forged an entire career.
Alan was very data driven. I got a lot of great experience related to test development working with him. He started up work on the Kaufman Assessment Battery for Children (Kaufman & Kaufman, 1983), and I was fortunate enough to be his graduate assistant and got a lot of good experience with him. One of the things about Alan Kaufman that you don’t hear people talk about much is that he was an incredibly good teacher. You just were enamored by his teaching and whatever he was teaching, you wanted to know more about it. He inspired people to move beyond what they were doing, to do more and to learn more. He was a student of Thorndike’s at Columbia, so I’m proud to claim Bob Thorndike as my academic grandfather.
There are also a couple of other people that I would be remiss if I didn’t mention that I had the opportunity to work with. Every summer of graduate school, I worked for Paul Torrance as his graduate assistant. Paul was just a phenomenal scholar, writer and mentor. He was one of the most remarkable people I’ve ever met. He was an incredibly sophisticated thinker when it came to theory and how to make sense out of the world. He was just a tremendous influence in teaching me how to think, how to think as a scientist, and how to think about how data fit (or how they don’t fit) different theories, and how they fit with the world around you.
The other person who had a real influence on me was my internship supervisor, Larry Hartlage, at the Medical College of Georgia. Larry was one of the early pediatric neuropsychologists. He taught me to think about translational research. He was very interested in how we do research that would translate what we know, and what we think we know, into actual practice with patients. He moved me away from basic science into the area of applied research. While I enjoyed more basic kinds of theoretical research, I found I liked applied research even more. I liked figuring out, “why does this matter?” I wanted more than knowledge for the sake of knowledge, and wanted to move more into that area after being exposed to Larry and seeing how research could affect what clinicians do, and how it could improve patient care. He nudged me enough in that direction that I picked up a different emphasis when I moved into my own research career.
I was just incredibly fortunate to have such wonderful mentors. The confluence or synergy or their influences helped create something unique, that you can see when you look at the cohort of students that came out of Georgia at that time.
How did you develop your interest in psychometrics and test development?
Even as an undergraduate, I was interested in how we measure things and that drove some of my interest. Then, of course, going to Georgia and having Alan Kaufman as my major professor, how could I not be interested in psychometrics and test development and in measuring things better? Measuring things better advances science, it advances practice, it advances everything we do.
I’ve always been driven by this idea: “Is there something here that we can do that will improve practice?” Most of the tests I have authored came from what I saw as clinical need so that we could improve practice. I tried to fill in areas where I saw a gap as a practicing clinician, because I always had a clinical practice, in addition to being at the university. I tried to identify needs that I had in the assessment process where I thought there were gaps or areas where I thought I could do a better job if I had an instrument, or a better instrument. So I ended up filling in some of those areas.
In fact, the very first test I ever published was a personality measure for children, the Revised Children’s Manifest Anxiety Scale (Reynolds & Richmond, 1978). For my dissertation research, I needed a good measure of anxiety in children and I couldn’t find one, so I built one. I realized that this was going to delay my dissertation for quite a while, so I picked a different dissertation project; but I was so deep into developing that test that I went ahead and finished it. I actually published it while I was still a graduate student. So I started in the personality arena and stuck around with it for a while.
My next big project was the Behavior Assessment System for Children (Reynolds & Kamphaus, 1992), an early behavioral assessment, but we also included a self-report of personality as part of that. I always think it’s important to ask, particularly kids, but adults too, how they think and feel about themselves, the world around them, and their relationships with other people. So we developed an integrated system that’s been very popular and worked out very well. The third edition of that test has been out for a couple of years.
At the same time, I was very interested in the cognitive domain. I have a Diplomate in Clinical Neuropsychology and also in Pediatric Neuropsychology. I’m very interested in the neuropsychological domain, so I developed a memory battery, the Test of Memory and Learning (Reynolds & Bigler, 1994). I’ve also authored some, what I refer to as, niche neuropsychological tests (for example, the Comprehensive Trail Making Test, Reynolds, 2002; the Test of Verbal Conceptualization and Fluency; Reynolds & Horton, 2006). These are tests that are not in huge widespread use and they are not the kinds of things that people use every day when they are doing assessments. These tests look at pretty specific neuropsychological functions that are useful in particular assessments you do with particular kids or particular adults.
With Randy Kamphaus, I did an intelligence test, the Reynolds Intellectual Assessment Scales (Reynolds & Kamphaus, 2003). We were unhappy with some of the characteristics of some of the intelligence tests that were being used so we’d been talking for years about doing one of our own. We finally got to the point where a publisher was willing to take that risk with us because those kinds of development projects require a great deal of upfront money to get them going and get them developed. And it’s high risk; you don’t know if other people are going to like it or not. So we went that way for the same reason I had done the other tests I had done, to fulfill what I thought was a clinical need so that we could improve practice.
How did you gain such a deep, yet diverse, breadth of knowledge that allowed for the creation of your wide suite of products?
If you want to work in this area, you can’t underestimate the value of also being a practitioner or at least having done assessments yourself and seeing the need. That also drives the breadth of your knowledge. If you are going to do a good job with patient care, you have to know a lot of different areas, and fortunately, I’m someone who loves to read. I’m a voracious reader. So I read in a lot of different professional areas and I’ve had the opportunity over my career to be the editor-in-chief of four very different journals (“Archives of Clinical Neuropsychology,” “Applied Neuropsychology,” “Psychological Assessment” and “Archives of Scientific Psychology”) which forced me to learn broadly and become adept in areas I wasn’t as interested in, but was forced to learn them to be able to do my job at the right level. Being a scientist-practitioner really helps. That’s a term that you don’t hear as much anymore, but it really, to me, is a key aspect of all of this. I always had a clinical practice. My practice was primarily driven by doing testing and diagnostic work, with a very limited therapy load. But seeing patients and doing diagnostics and having to know the different areas really forces you to have both breadth and depth of knowledge. Being a scientist-practitioner gives you a unique perspective that you can bring to problems. You can really integrate information much better having had those experiences and forcing yourself to learn areas that sometimes are hard for you, even in areas that you are not naturally drawn to.
What advice would you have for someone thinking about publishing their test?
One of the first things I’d tell them — assuming that they’ve done the preliminaries right and they have a good idea that will improve practice in some way (you have to have a good idea!), the biggest variable in my mind is what I call the fit. Even though the different test publishers compete in similar domains, they all have unique interests and capabilities. It’s important to figure out which company is going to do the best job with this project. And it’s not the same answer for every project. So when I have a proposal for something new that I want to do, I go through a process of deciding, based on what I know about the companies and their personnel and how they approach test development. One is going to be the best for this test and that’s who I take it to first. And sometimes my first choice says no and then I may take it somewhere else.
However, my first submission on any project, I try to figure out which is going to be the best fit for this project. That requires some research on your part if you are new to this. You want to read their catalogue and see what tests are proprietary to them and what kinds of things they choose to distribute from other publishers. But you want to take special notice of their proprietary products, because that’s probably where their strengths lie. And then many publishers will have some information on their website about what they are interested in and what they want in a proposal.
Also, don’t be afraid to pick up the phone and call and talk to a project director. Talk to a head of research and development. You can get through to those people on the phone and most of them are happy to have at least brief conversations with you. They want to hear your ideas and they are also looking for a fit too.
We’ll change direction now. How has being involved with APA, and Div. 5 in particular, helped or impacted your work?
I’ve been a Div. 5 member since I became a member of APA, and certainly have enjoyed my interactions with Div. 5. I was president of the division at one time. It was an interesting time then when Div. 5 actually used to be classified by APA as a practice division and not as a science division. As APA thought about things and as they made decisions about asking different divisions for advice and how they were viewed, we had a lot of impact on changing the view within APA of Div. 5 into being both a division that could be counted on for science as well as practice information, because certainly what Div. 5 does impacts both. The work of Div. 5 and how it has reviewed materials and its influence on testing and measurement and the standards (i.e., the Standards for Educational and Psychological Testing; AERA, APA, NCME, 2014) has influenced everybody’s career who works in this area. In that sense, being involved in that more intimately through the executive board and on the occasional committee over the years, it draws more of that to your attention and makes you much more sensitive. My clinical practice now is exclusively forensic and I pass out copies of those standards to lawyers constantly and prepare them to cross-examine other experts who have done any kind of testing and measurement. A lot of that stems from my interactions with Div. 5 over the years; it has sensitized me so much to the need to have those standards and for practitioners to be aware of them and to follow them. So that’s been a huge influence on me.
Relatedly, in 2014, you won the Samuel J. Messick Award for Distinguished Scientific Contributions from Div. 5. What did this award mean to you?
That was a big event in my life. I did not know Messick personally, but I knew his writing intimately. I think I’ve read everything he’s ever written that is publicly accessible. I always admired his thinking about validity in particular. He had a tremendous influence on my own thinking and my own formulations about validity and things that I’ve written about related to the concept of the validity of tests. I’m still trying to move the field into that way of thinking. It’s interesting that the Standards have adopted, fundamentally, Messick’s thinking on that and did so almost 20 years ago, but it’s still not pervasive in textbooks or in clinicians’ training. He was such a tremendous influence, and such a good thinker, that to be put into any category with him, which I can claim a little bit since I got the award named after him (but only a little bit, he was such a superior thinker in the field) was validating in a lot of ways. It was validating to my work, my thinking, and my writing, including the fact that I continue to push for practitioners and trainers of practitioners to learn about, and adopt, these concepts the way they are conceived of in modern psychometrics, which is largely due to Samuel Messick. It was validating, it was a big event in my life, and it feels great to be mentioned in the same sentence as somebody like that.
Having said that, if you could do it over, what would you have done differently, if anything, as you navigated your professional career as a test author?
My life’s ambition was originally to be a beach bum. I certainly blew that (laughs). I grew up on the beach as a kid and thought I’d just hang around boats and fish and be a beach bum and it would all be fine, but it just didn’t work out that way. I like to think that there are some decisions in my life where I could have done things better or could have spent more time learning something. I did graduate school very quickly for financial reasons. I had enough money to pay for a certain number of years of school so that’s the amount of time I spent in graduate school. I took overloads every semester, and I went from bachelor’s to PhD, including a year of internship, in 39 months. If I could do it over again, I would have spent an extra year doing my degree. That extra year would have helped me assimilate everything and understand it better. I eventually got there, but the first couple of years after I finished my degree, my knowledge was still too raw. It took a little longer for my thinking to mature than it would have if it had spent a little more time in graduate school. And who wouldn’t want to spend another year with people like Alan Kaufman, Paul Torrance and Larry Hartlage?
Technology is starting to change how tests are used, scored and developed. Where do you think we will be, or should be, five or even ten years from now?
We have a hard time knowing or predicting what new technology is going to be available six months from now, much less five or 10 years from now. But my thinking, and something I’ve been thinking about for some time is: “When is testing is going to catch up to technology?” Computer-driven adaptive testing is the psychological testing of the future (and some say it always will be) and there is going to be a paradigmatic shift that is going to happen. I can’t predict when, but I think that we are at a point now where the combination of artificial intelligence (AI) and virtual reality (VR) is going to give us the opportunity, once it becomes economically viable, to really change psychological and educational testing in massive ways. This is going to create a different genre of psychological tests that are going to conform much better to the ecology of the examinee and the needs of the person doing the assessment. That we’ll be able to design instruments and use them in ways that are unique to the examinee and the informational needs of the examiner–whether it’s a human resources, rehabilitation or special education environment — we’ll be able to do some very specialized kinds of adaptive testing through the combination of AI and VR that right now we’re not able to do. This is really going to change the face of psychological testing. I don’t think it’s going to replace what we currently do entirely. It’s going to replace some of what we do, but it’s going to create a whole new area of psychological testing where we will be able to do much more refined diagnostics and do things in the assessment process that are greatly going to add to our ability to make accurate predictions about how a person will or will not behave or perform in the future.
As you look ahead toward the future, what are some areas that you think require more research?
I think there are areas, particularly in terms on predicting behavior, where we are going to have to focus more clearly and become more refined. One of the things that people are now intensely interested in is predicting future acts of aggression and violence. We know, if you know something about utility theory and utility functions, that that’s incredibly difficult to do. And there are some mathematical limitations placed on us for that. But I think there are ways to conceptualize this issue and conceptualize how we approach those evaluations to become better at prediction. We’re going to have to rethink and re-conceptualize how we do that. And I think there are other areas of prediction too, where we can really refine what we’re doing. I’m leaning a little bit towards thinking that we need to go back to some throwback ideas, some things that were promising at one time that were poorly executed that probably could be revived and improve what we do. For example, using quantitative biographical data in enhancing prediction models for a variety of future behaviors. Quantitative biographical data has a tremendous amount of promise, particularly when coupled with some of the more traditional approaches that we take. I think we’re going to need to reconceptualize how we go about doing prediction and looking at areas where we think it’s extremely difficult or impossible to make accurate predictions and realize it’s not impossible if we reconceptualize how we approach it and find a way to do it. I think we need to get back into that research more.
References
American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (2014). Standards for Educational and Psychological Testing. Washington, D.C.: AERA.
Kaufman, A.S., & Kaufman, N.L. (1983). Kaufman Assessment Battery for Children. Circle Pines, Minnesota: American Guidance Service.
Reynolds, C.R. (2002). Comprehensive Trail Making Test: Examiner’s manual. Austin, Texas: PRO-ED.
Reynolds, C.R., & Bigler, E.D. (1994). Test of memory and learning. Austin, Texas: PRO-ED.
Reynolds, C.R., & Horton, A.M., Jr. (2006). Test of verbal conceptualization and fluency: Examiner’s manual. Austin, Texas: PRO-ED.
Reynolds, C.R., & Kamphaus, R.W. (1992). Behavior Assessment System for Children: Manual. Circle Pines, Minnesota: American Guidance Service.
Reynolds, C.R., Kamphaus, R.W. (2003). Reynolds Intellectual Assessment Scales. Lutz, Florida: Psychological Assessment Resources.
Reynolds, C.R. and Richmond, B.O. (1978) What I think and Feel: A Revised Measure of Children’s Manifest Anxiety, Journal of Abnormal Psychology, vol. 6(2), pp. 271-280.
Wechsler, D. (1974). Wechsler intelligence scale for children — revised. New York: Psychological Corporation.