The study of excellent teaching in higher education has a long, albeit varied, history (Buskist & Keeley, 2014). The study of excellent college teaching is difficult because there is no technical definition of what constitutes excellence in the college classroom. There are certain competencies that professors should exhibit; but, being competent in the classroom is not the same as demonstrating teaching excellence (Elton, 1998). Likewise, having expertise in a given academic field does not translate into having expertise in the classroom. In fact, the two skill sets are likely independent of each other (Hattie & Marsh, 1996; Marsh & Hattie, 2002).
Buskist et al. (2002) argued that Master Teachers — instructors who have demonstrated a level of rare excellence in the college classroom — possess certain qualities, but any specific subset of these qualities is not sufficient.
Rather, Master Teachers are likely to come in all shapes and sizes, so to speak, and represent different combinations or blends of these qualities. What makes [Professor X] a Master Teacher is not exactly the same as what makes [Professor Y] a Master Teacher, although there may be some overlap in the personal qualities and penchants relevant to teaching that each possesses. . . Master Teachers are as unique as teachers as they are as human beings. (pp. 31-32).
The uniqueness of Master Teachers is not surprising. Studies of expertise indicate that experts are qualitatively different from others in the same field (e.g., Ackerman, 2014; Ericsson, 2014). While there maybe debate about how someone becomes an expert (e.g., Hambrick, Oswald, Altmann, Meinz, Gobet, & Campitelli, 2014), there is little contention that experts both process information in their area expertise differently as well as attend to different information (Dunkin, 1995; Horn & Blankson, 2012). This allows them to plan better and respond to situations more adroitly than non-experts.
Historically, most studies of excellent teaching have focused on perceived qualities of professors who exemplify excellence (Buskist & Keeley, 2014). One common example of this type of study is examining what stakeholders (e.g., students, faculty, alumni) perceive as teaching excellence (e.g., Catano & Harvey, 2011; Keeley, Ismail, & Buskist, 2016). Another example is the study of criteria for teaching awards (e.g., Skelton, 2004). While such studies can provide useful information, methodologically they are problematic. They are detached from the teaching process because investigators do not examine the displays of excellence directly.
A second type of investigation is direct observation of Master Teachers (e.g., Hativa, Barak, & Simhi, 2001). These types of studies are much less common than those that examine perceptions of excellence, but are critically important (Berliner, 1986). The reason is that direct observations provide data about the specific behaviors professors do that make them experts in teaching.
While most Master Teachers deliberately practice effective teaching techniques, this is only a partial explanation of what sets them apart (Macnamara, Hambrick, & Oswald, 2014). Instead, they approach teaching qualitatively different than novices—or even “regular” professors — although they may not necessarily be cognizant of how or why what they do is different (Livingston & Borko, 1989). Thus, direct observation of Master Teachers can provide a wealth of information.
Direct observations of Master Teachers are less detached than interviews or questionnaires, but investigators taking this approach typically only study just a slice of what Master Teachers do in the classroom (e.g., observe a specific class or module). What is largely missing from the study of expert university teaching is a thorough examination of what Master Teachers do throughout a full course.
In this current study, I conducted a case study of a Master Teacher for an entire course. Case studies are detailed examinations of a single example and have been a common part of psychological research for a long time (Bromley, 1986). While they have been used to study expertise in teaching at the pre-college level (e.g., Smith & Strahan, 2004), they are rarely used to study Master Teachers within university settings. This is unfortunate because case study methods are particularly useful for studying experts (Ericsson, 2014). Even Hans Eysenck, a strong advocate of quantitative research, noted the value of case studies, “Sometimes we simply have to keep our eyes open and look carefully at individual cases — not in the hope of proving anything, but rather in the hope of learning something!” (Eysenck, 1976, p. 9).
Method
The Case: Roger Kirk. For this study, I selected a case that was extreme (i.e., much different from what is typical). Extreme cases are useful for exploratory studies that are “open-ended” in the investigation of a phenomenon (Seawright & Gerring, 2008).
Roger Kirk is an extreme — an expert in the field of quantitative methods (Beaujean & Weaver, in press). In terms of scholarship, he has published over 100 pieces in psychology and statistics, including five books. One of his books has received the rare honor of being designated a citation classic.
Kirk’s expertise not only includes scholarship, but also includes pedagogy. This is bit harder to quantify, however. Perhaps the best way to make the case of his expertise in this area is to note the numerous awards he has received for teaching. In 1992, he was named the Outstanding Tenured Teacher in the College of Art and Sciences at Baylor University. In the following year, Baylor gave him the title Master Teacher — the university’s highest teaching award. In 2005, Div. 5 of the American Psychological Association (APA) gave him the Jacob Cohen Award for Distinguished Contributions to Teaching and Mentoring. He won the 2012–2013 Cornelia Marschall Smith Professor of the Year at Baylor for his “superlative teaching and outstanding research contributions.” More recently, in 2015, he won the APA’s Charles L. Brewer Award for Distinguished Teaching of Psychology. This award “celebrates Roger E. Kirk’s dedication to and accomplishments in the teaching of psychology. . . His passion for teaching is legendary, his commitment to his discipline absolute, and his work ethic unmatched” (“Charles l. Brewer Award for Distinguished Teaching of Psychology,” 2015, p. 384).
Design: Case Study. The aim of this particular study was to explore how a Master Teacher behaves in the college classroom, which lends itself to a case study design (Baxter & Jack, 2008). Case study research starts from the desire to derive an in-depth understanding of a case set in a real-world context (Yin, 2009, 2012). It is particularly apropos when: (a) the research questions are about “how” or “why”; (b) it is not possible to manipulate behaviors of those involved in the study; and (c) the boundaries are not clear between the phenomenon and context (i.e., the behaviors of a Master Teacher have to be considered within the context of teaching).
This particular case study was holistic, descriptive, and exploratory. It was holistic because I just focused on a single case throughout the study (Kirk). It was descriptive because my goal was to describe a situation not normally accessible to researchers in its real-life context: the in-class behaviors of a Master Teacher. It was exploratory because I had no clear set of behaviors that I was looking to observe when I started the study.
Context: College classroom. I observed Kirk teaching an introductory statistics course during the spring 2018 semester. The course is designed for undergraduate students (typically sophomores or juniors) and covers topics ranging from descriptive statistics to univariate inferential statistics. It is required for all psychology majors, but students from many other majors also take the course. In the semester I observed, 85 students were enrolled.
Data Collection. There are two components to the introductory statistics course: recitation and lab. I observed the recitation component, which met three days a week for 50 minutes each. I observed approximately 95 percent of the classes Kirk taught for the semester (approximately 40 classes). The classes I missed were those in which an exam was administered, I had a schedule conflict, or Kirk was absent. Before beginning the data collection, I obtained Kirk’s permission and the approval of my institution’s institutional review board.
All classroom data was gathered through direct observation by a single observer. I sat in the same seat in the front row every class, which allowed me to focus on Kirk (as opposed to the students). I have taught similar courses in the past, so the content was already familiar to me. This allowed me to focus on Kirk’s teaching behaviors. For each class, I brought a notepad and made notes about teaching behaviors. As this study was exploratory, I did not have a prioiri criteria about what behaviors I would or would not make note of. Instead, I made field notes about any behaviors I saw related to teaching (Fusch & Ness, 2017).
In addition to classroom observations, I also reviewed documents Kirk (2002, 2015) has authored about teaching statistics.
Data Analysis. For the data analysis, I went through my notes looking for themes (i.e., patterns of behavior that have meaning; Braun & Clarke, 2012). Specifically, I went through the data as I gathered them and noted when I saw repetitions of behaviors. I then went through these repetitions to see if they could be further combined to form meaningful themes. Eventually, I settled on five distinct classes of behavior patterns, which I then named and defined.
Results
In what follows, I provide the five themes that emerged from my observations. They are presented in no particular order.
1. Levity
One of the most frequent behavior I observed was the use of humor. This made the general climate of the course positive and lighthearted. Moreover, it allowed Kirk to regain student focus when he judged that students were not being engaged.
Kirk employed many forms of humor. It ranged from anecdotal (e.g., telling stories about individuals having difficulty with ballroom dancing steps) to self-deprecating (e.g., making light of himself when he made an error) to situational (e.g., making funny statements about difficult content such as “boy, this is hard and confusing, isn’t it!”). It was never caustic or juvenile, and never at the expense of any particular student.
Perhaps the most unique way Kirk introduced levity was one of the ways he regained students’ attention: short “dance breaks.” Kirk — an accomplished ballroom dancer — spontaneously broke into a ballroom dance (e.g., foxtrot, rumba) multiple times throughout the semester. This noticeably piqued students’ attention, and then Kirk would segue back into teaching.
2. Rigor
Kirk approached the content rigorously and never “dumbed down” the material. For example, it was not uncommon for him to work through proofs or derivations when they were applicable and accessible to the students. Moreover, he had very high expectations of his students. He frequently told them that he expected that they be able to demonstrate knowledge about not only how to calculate a given statistic, but also know when to apply it and how to interpret the results. His tests reflected these expectations.
3. Care and Respect for Students
Kirk demonstrated respect for his students and care that they did well in his course. This is not surprising, given its commonality among Master Teachers (Keeley et al., 2016). What was surprising was how Kirk went about doing this given the inherent difficulties in this endeavor with larger class enrollments.
Kirk’s demonstration of respect and care was most frequently evident in his purposeful effort to know his students. He had students complete a seating chart after the first few classes. He did not use this chart to just take attendance or call students by name during class. In addition, he came at least 10 minutes early to every class and used the seating chart to talk, informally, with different sections of students before class began.
One day I got to the lecture hall ten minutes earlier than usual. Instead of standing at the front of the hall, I wandered up and down the isles [sic]. During this time, I had a number of conversations with students who never asked questions during class or called me at home. From that day on, the ten-minute walkaround became a regular part of my pre-lecture routine. (Kirk, 2002, p. 5).
The topics of these student conversations were not random. Before each class, Kirk reviewed a notebook that contained the gist of earlier student contacts that he wanted to follow up with (e.g., asking students who were ill how they were feeling, asking student athletes about upcoming sports event). Thus, over the course of the semester, he made multiple personal contacts with the 85 students taking his course!
Another way he demonstrated respect was the way he made it safe to ask questions. He stated multiple times in class that it was normative for statistics to be confusing or provoke anxiety. Thus, he set aside time in every class for students to raise questions about concepts they did not believe they understood well. In other words, he wanted students to ask questions and planed his lessons on the assumption that part of class time would be devoted to answering questions.
Kirk went beyond setting time aside in class to answer questions, however. In his syllabus, he gives his office and home phone numbers so students could contact him outside of class with questions. Moreover, he encouraged them to contact him, “I enjoy answering questions about statistics. . . Answering your questions is an important and enjoyable part of my job; do not hesitate to contact me.”
The way Kirk answered questions in class also demonstrated respect for students—as well as patience. His body language, tone of voice, and positivity in his responses all gave the impression that every question was important, even those that had been asked before. It was not uncommon for students to ask the same question in consecutive classes; I counted one particular question asked four separate times. Yet, Kirk never gave the impression of appearing frustrated, nor were his answers ever condescending. Instead, he answered every question as if it were being asked for the first time.
4. Anecdotes and Examples
Statistics can sometimes be a dry subject. To make the content more accessible and “come alive,” Kirk frequently provided anecdotes and examples throughout his classes. The anecdotes ranged from personal (e.g., an experience using a given statistic, errors he made) to professional (e.g., background of individuals who developed a statistic). Sometimes he combined the two and provided anecdotes of his personal interactions with the statisticians who developed a statistic!
In addition to anecdotes, Kirk provided examples in nearly every class. Sometimes these examples involved working through data analysis from a real-world situation in which case Kirk worked out the solution via slides (which he provided to students ahead of time). Other times, the examples were more didactic and involved Kirk working through the calculations, by hand, in class. Either way, students were afforded multiple demonstrations of how to calculate, use, and interpret every statistic that was discussed in the course.
5. Material Reviews
The first part of every class started at the same way: 10 minutes devoted to reviewing material previously covered in the course. This review could range from going over terms to working through an example. He was especially apt to repeat a previous example when it was complex and students were likely to be confused or have unasked questions (e.g., power analysis). In conducting these reviews, he often made a point to note how concepts he taught earlier in the semester fit together with more recent topics.
Discussion
In this study, I directly observed the pedagogical behaviors of a Master Teacher: Roger Kirk. Unlike many other pedagogical case studies, I observed Kirk in the same course for an entire semester. Thus, I was able to collect data from approximately 40 separate classes. In those classes, I saw repeated behaviors from Kirk that fit into five themes: levity, rigor, care and respect for students, anecdotes and examples, and material reviews.
The first thing that stands out from this study is that the first two themes, levity and rigor, seem contradictory. How can a course with a lighthearted climate also be one that is rigorous? Honestly, I am not sure how it is done; but I know it can be done because I saw Kirk do it. He was able to somehow weave rigor with due levity, and do so in a way that seemed seamless. A former mentor of mine once described his own teaching philosophy as playing an accordion. Sometimes it needs to be stretched (i.e., students need to be challenged) and sometimes it needs to be relaxed (i.e., students need to have a break). Perhaps that is what Kirk is doing by weaving levity and rigor throughout a course. Knowing when to stretch and when to relax; however, is not intuitive and is likely something that differentiates Master Teachers.
The second thing that stands out from this study is the deliberate actions Kirk took to fostering student relationships. Given Kirk’s professional stature, it could be understood if he did not bother with trying to personally connect with students in courses with larger enrolments. It is not easy to connect with so many of them, nor is it easy to create an atmosphere where students feel comfortable asking questions. Yet, Kirk appeared to accept the challenge and, as a consequence, purposefully devoted considerable time and effort to make this happen. I can probably count on one hand the number of professors I know who have provided their home phone number on a syllabus and asked students to call when they have questions. I can count with one finger the number of professors I know who keep a notebook about student interactions so they can follow up with students throughout the semester. This is likely another thing that differentiates Kirk as a Master Teacher (cf. Keeley, Ismail, & Buskist, 2016).
The third thing that stood out was that Kirk planned time for content review and students asking questions. These were not impromptu events that serendipitously occurred that then threw off the schedule of events. Instead, Kirk developed his curriculum with the idea that reviewing older content and answering students’ questions are just as worthy of class time as providing new content. Such behaviors may not be necessary in other courses, but are vital in statistics. Students often see statistics as a disorganized collection of formulae and concepts, so can get overwhelmed by the, seemingly unrelated, details (Harlow, 2013). Thus, reviewing material can help tie older and newer concepts together and aid in student understanding.
Limitations
The major limitations of this study are the same as those of many other case studies: the sample is purposeful and the approach is exploratory. Thus, the results may or may not generalize to other Master Teachers, nor will following Kirk’s example necessarily make a professor develop into a Master Teacher. Moreover, since there was only one observer and the same individual also coded the data and developed the resulting themes, there could be — and likely is — some bias in the results. These factors do not necessarily make the results invalid, but do indicate that others need to replicate the results before any developing any notion of causality or generalizability from the results.
Implications
Personally, I learned a great deal by observing Kirk. I teach courses similar to the one I observed Kirk teaching and have already started implementing changes. I try to incorporate more humor and anecdotes, plan for content reviews, and make a more purposeful effort to talk with students about non-class related material that is important to them.
For those who do not teach courses in quantitative methods, this study may not seem to have any relevance — and perhaps it does not. What could have relevance for faculty from any discipline is observing Master Teachers in their own departments or universities. They may not have the particular title of Master Teacher, but they exist. I cannot recommend enough learning from them. Ask if you can, observe them teach, whether for a few classes or throughout a semester. It does not really matter if you are familiar with the content or not. If you are, all the better; but, even if you are not you can still learn from observing a master (teacher).
References
Ackerman, P. L. (2014). Nonsense, common sense, and science of expert performance: Talent and individual differences. Intelligence, 45, 6-17. doi:10.1016/j.intell.2013.04.009
Baxter, P., & Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report, 13, 544-559.
Beaujean, A. A. & Weaver, C. (in press). Roger E. Kirk: A Biography. The Score
Berliner, D. C. (1986). In pursuit of the expert pedagogue. Educational Researcher, 15, 5-13. doi:10.2307/1175505
Braun, V., & Clarke, V. (2012). Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. (pp. 57-71). Washington, DC: American Psychological Association.
Bromley, D. B. (1986). The case-study method in psychology and related disciplines. Chichester, England: Wiley.
Buskist, W., & Keeley, J. (2014). Becoming an excellent teacher. In D. Dunn (Ed.), The Oxford handbook of undergraduate psychology education. New York, NY: Oxford University Press.
Buskist, W., Sikorski, J., Buckley, T., Saville, B. K., Davis, S. F., & Buskist, W. (2002). Elements of master teaching. In S. F. Davis & W. Buskist (Eds.), The teaching of psychology: Essays in honor of Wilbert J. McKeachie and Charles L. Brewer (pp. 27-39). Mahwah, NJ: Erlbaum.
Catano, V. M., & Harvey, S. (2011). Student perception of teaching effectiveness: Development and validation of the Evaluation of Teaching Competencies Scale (ETCS). Assessment & Evaluation in Higher Education, 36, 701-717. doi:10.1080/02602938.2010.484879
Charles l. Brewer Award for Distinguished Teaching of Psychology. (2015). American Psychologist, 70, 384-385. doi:10.1037/a0039387
Dunkin, M. J. (1995). Concepts of teaching and teaching excellence in higher education. Higher Education Research \& Development, 14, 21-33. doi:10.1080/0729436950140103
Elton, L. (1998). Dimensions of excellence in university teaching. International Journal for Academic Development, 3, 3-11. doi:10.1080/1360144980030102
Ericsson, K. A. (2014). Why expert performance is special and cannot be extrapolated from studies of performance in the general population: A response to criticisms. Intelligence, 45, 81-103. doi:10.1016/j.intell.2013.12.001
Eysenck, H. J. (1976). Introduction. In H. J. Eysenck (Ed.), Case studies in behaviour therapy (pp. 1-15). London, England: Routledge.
Fusch, G. E., & Ness, L. R. (2017). How to conduct a mini-ethnographic case study: A guide for novice researchers. The Qualitative Report, 22, 923-941.
Hambrick, D. Z., Oswald, F. L., Altmann, E. M., Meinz, E. J., Gobet, F., & Campitelli, G. (2014). Deliberate practice: Is that all it takes to become an expert? Intelligence, 45, 34-45. doi:10.1016/j.intell.2013.04.001
Harlow, L. L. (2013). Teaching quantitative psychology. In T. D. Little (Ed.), The Oxford handbook of quantitative methods in psychology: Volume 1: Foundations (pp. 105-117). New York, NY: Oxford University Press.
Hativa, N., Barak, R., & Simhi, E. (2001). Exemplary university teachers: Knowledge and beliefs regarding effective teaching dimensions and strategies. The Journal of Higher Education, 72, 699-729. doi:10.1080/00221546.2001.11777122
Hattie, J., & Marsh, H. W. (1996). The relationship between research and teaching: A meta-analysis. Review of Educational Research, 66, 507-542. doi:10.3102/00346543066004507
Horn, J. L., & Blankson, A. N. (2012). Foundations for better understanding of cognitive abilities. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed., pp. 73-98). New York, NY: Guilford Press.
Keeley, J. W., Ismail, E., & Buskist, W. (2016). Excellent teachers’ perspectives on excellent teaching. Teaching of Psychology, 43, 175-179. doi:10.1177/0098628316649307
Kirk, R. E. (2015, August). Charles L. Brewer invited address: Teaching statistics: What I have learned so far. Paper presented at the annual meeting of the American Psychological Association, Toronto, Canada.
Kirk, R. E. (2002, August). Teaching introductory statistics: Some things I have learned. In L. L. Harlow (Chair), Engaging others in quantitative psychology. Symposium conducted at the annual meeting of the American Psychological Association, Chicago, IL.
Livingston, C., & Borko, H. (1989). Expert-novice differences in teaching: A cognitive analysis and implications for teacher education. Journal of Teacher Education, 40, 36-42. doi:10.1177/002248718904000407
Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014). Deliberate practice and performance in music, games, sports, education, and professions: A meta-analysis. Psychological Science, 25, 1608-1618. doi:10.1177/0956797614535810
Marsh, H. W., & Hattie, J. (2002). The relation between research productivity and teaching effectiveness. The Journal of Higher Education, 73, 603-641. doi:10.1080/00221546.2002.11777170
Master Teachers at Baylor University (BU-PP 703) (2013). Retrieved from https://www.baylor.edu/bupp/doc.php/211002.pdf
McKeachie, W. J. (1997). Critical elements in training university teachers. International Journal for Academic Development, 2, 67-74. doi:10.1080/1360144970020108
Seawright, J., & Gerring, J. (2008). Case selection techniques in case study research: A menu of qualitative and quantitative options. Political Research Quarterly, 61, 294-308. doi:10.1177/1065912907313077
Skelton, A. (2004). Understanding ‘teaching excellence’ in higher education: a critical evaluation of the National Teaching Fellowships Scheme. Studies in Higher Education, 29, 451-468. doi:10.1080/0307507042000236362
Smith, T. W., & Strahan, D. (2004). Toward a Prototype of Expertise in Teaching: A Descriptive Case Study. Journal of Teacher Education, 55, 357-371. doi:10.1177/0022487104267587
Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Los Angeles, CA: Sage.
Yin, R. K. (2012). Case study methods. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 141-155). Washington, DC: American Psychological Association.