Join Perusall's reading community for Critical Teaching Behaviors: Defining, Documenting, and Discussing Good Teaching starting May 1st, 2023. Register: perusall.com/engage.

Eric Mazur:

Welcome to the Social Learning Amplified podcast, the podcast that brings us candid conversations with educators who are finding new ways to engage and motivate their students inside and outside the classroom. Each episode of Social Learning Amplified will give you real life examples and practical strategies you can put into practice in your own courses. Let's meet today's guests.

Welcome to Social Learning Amplified. I'm your host Eric Mazur, and our guests on the episode today are Dr. Lauren Barbeau and Dr. Claudia Cornejo Happel. Thank you both for joining.

Dr. Lauren Barbeau:

Thank you for having us.

Eric Mazur:

Lauren Barbeau (right) is the Assistant Director of Learning and Technology Initiatives and the Center for Teaching and Learning at Georgia Institute of Technology. She earned her PhD in English from Washington University in St. Louis. Before becoming an educational developer, she taught writing and literature courses, her educational development career at Georgia Southern University, and subsequently served as the Assistant Director for faculty development and the scholarship of Teaching and Learning at the University of Georgia. Her research interests include teaching with technology as well as documenting and assessing teaching. Claudia is an associate director of the Center for Teaching and Learning Excellence at Embry-Riddle Aeronautical University in Daytona Beach. She earned her PhD in Spanish from the Ohio State University and an educational specialist degree from Georgia Southern University.

Claudia (left) has previously held positions in the University Center for the Advancement of Teaching at Ohio State, and at the Center for Teaching Excellence at Georgia Southern University, researching and assessing teaching, as well as inclusive instructional practices. Now, just two months ago, Lauren and Claudia published a book together entitled Critical Teaching Behaviors: Defining, Documenting, and Discussing Good Teaching. I was surprised for our listeners who are interested in this book at the end of this podcast, but don't just jump to the end of the podcast now. Lauren and Claudia, your book provides a guide to evidence-based effective teaching and also a framework for assessing and documenting teaching. In the book, you present what you call the critical teaching Behavior framework. Can you explain to our listeners what you mean by critical teaching behaviors?

Dr. Lauren Barbeau:

Sure, Eric, I'll go ahead and explain that. So the framework itself consists of six research based categories. So it's align, include, engage, assess, integrate, technology, and reflect. And all these are our large broadly defined categories that are behavior based. So the idea is that they're based on what we can see instructors doing in the classroom or what instructors can provide evidence that they're doing outside the classroom. So the first column of the framework has these categories with the definitions associated with them. The second column has a short list of behaviors that we can see, so they're representative behaviors. The book goes in much more depth with the behaviors. However, on the framework itself, we wanted to create a nice two page overview. so we kept it short to six representative behaviors. And then the last column actually provides faculty with guidance on what sort of documentation they can collect as evidence that they're engaging in these types of behaviors.

So the framework started off as this project that helps faculty define what good teaching is and what it looks like providing a common language in which to talk about their teaching. But we eventually expanded beyond just the framework itself and developed tools for peer observations so that faculty can conduct observations of each other's teaching for formative or summative purposes, as well as student feedback forms so that faculty can collect mid-semester student feedback. And the idea is that all of these tools are aligned to the same framework. So when faculty are having these observations conducted or collecting mid-semester feedback, they're then able to write a narrative that is coherent using the framework and the behaviors that we've suggested, as well as the categories to help them find a language to frame what it is that they're doing in their classrooms.

Eric Mazur:

I see. So what inspired you to, to create this book?

Dr. Claudia Cornejo Happel:

Yeah, thank you for that question. So we have seen several institutions who are telling their faculty to be

excellent teachers, but then what they're not doing is actually defining what excellent teaching looks like. And so without that definition, it is really hard for faculty to live up to those expectations. And so we really wanted to provide a framework and guidance that was based on evidence and research, because there's a lot of research out there on what is effective teaching, what is good teaching, what supports student learning, but faculty don't always have the time, understandably. So, to dig into that and really see what is it that I need to do to support my student learning, what can I show to the person evaluating me or asking me to demonstrate effective teaching? So the CTB framework is really an attempt to do that, to provide that guidance that is based in research and still is flexible enough to really allow faculty across disciplines to find a way to demonstrate their accomplishments in each one of the categories.

So for example, if you are teaching physics, you might have one approach to engaging your students, whereas I am teaching Latin American literature and culture, and I might have a completely different way to engage my students. That does not mean one of those approaches is wrong. It's just different. And a lot of times we have frameworks that say, here's a checklist, meet all of the boxes. We want to see them all checked. And so that was really one of the things that inspired us for looking into the research, coming up with a framework that provides flexibility as well as a comprehensive definition of good teaching. And hopefully in that way allow faculty to have a common understanding that they can use to talk about teaching, to demonstrate their teaching effectiveness in a way that is not just, oh, here are some things that I'm doing, but actually here are some things that I'm doing that have been shown to support student learning.

Dr. Lauren Barbeau:

And I'll add to that, Eric, that one of the, Claudia and I had sort of different original motivations that converged on a similar path. this project for me started off more as how do we help faculty talk about their teaching and present it for documentation and assessment purposes? So we were getting a lot of questions at the time. Claudia and I worked in the center together. We were getting a lot of questions from chairs and deans saying, I don't know how to Eva evaluate my faculty in a way that's fair based on the size of class they're teaching or the discipline that they're teaching in. Because even within the same college, we have a variety of disciplines that teach differently based on their needs. And so for me this project really came out of the need to provide some sort of guidance for those who are evaluating faculty as well as for faculty who are coming to us saying, look, I'm going to be evaluated on my teaching, and I need to know how to present that. How do I talk about that in a way that best showcases what it is that I'm doing in my classroom?

Eric Mazur:

So the second part of your title refers to good teaching. And Claudia, you already mentioned you know, the need to define good teaching. So what is it exactly, if you can give a synopsis of that defines good learning?

Dr. Lauren Barbeau:

That's a great question, Eric. So the idea behind the research we conducted was that all of these are practices that have been proven to lead to student success. Now, what's different in each of these studies is how they're defining student success. Now, success might be a four year graduation rite. It might be lower DFWI rates, or a higher test scores, or a greater sense of belonging but in some way, shape or form, all of the research that we pulled on to create this project defines student success in a way that shows that students are moving towards the goalpost of learning. and whatever it is that we're doing in the classroom has been proven to contribute to moving them towards that ultimate goal of learning. Although the studies might define that differently.

Eric Mazur:

Now, you are both involved in Centers for Teaching and Learning, so I'm sure that this framework, this CTB framework, was around way before you wrote the book. So you've had the opportunity to, to observe faculty, what have you found that people are doing with your framework, and to what extent have you been able to observe people implement them?

Dr. Claudia Cornejo Happel:

Yeah, that is a great question. So for me at Embry-Riddle, the tool that we have implemented most here is probably the midterm feedback tool. So the midterm feedback is survey that is facilitated at the middle of the semester. So faculty get feedback on how the class is going for their students in the moment rather than waiting until the end of the term. And the tool we are proposing includes both open-ended responses questions, as well as ranking questions that are aligned to the five first categories of critical teaching behaviors. And so we have implemented that here at Embry-Riddle for the last several years. And one of the things that faculty really appreciate is the combination of quantitative and qualitative feedback that they are receiving, especially faculty who are in the STEM disciplines. Really appreciate the quantitative feedback. We've also done some preliminary data analysis that shows that it has the potential by focusing on those behaviors, concrete behaviors rather than faculty characteristics or course materials. it has the potential to reduce bias, implicit bias in the rankings. That is still an ongoing project, but the, the initial data that we collected seems promising. So that is one way in which I have implemented that in my practice here. Lauren, do you wanna share a little bit about your experience?

Dr. Lauren Barbeau:

Sure. Yes. So the first instrument we developed off of this framework, and, and honestly this was before we ever thought about creating a book, is we'd created this framework. And then we said, you know, what would be great is if we had an observation tool that was aligned to this, because then we're giving faculty a more comprehensive guide to talking about and documenting their teaching. so we ended up developing the observation forms and then the protocol for conducting observations because we're only a few people. And, and it's useful for faculty to be able to conduct peer observations and not always have to rely on a center for teaching and learning to do that. So we created this, and as far as I'm aware, Georgia Southern is still using this protocol to conduct observations. I think they started using it in 2019. So it's been in use for about five years now.

And what we've seen is that faculty really appreciate the opportunity to see both individual categories of feedback as well as sort of holistic, where am I? big picture. So this instrument works by allowing faculty to take a deep dive into, let's say, I wanna really look at my include behaviors. So faculty can ask for intensive feedback in one or a few categories, but they can also get a holistic view of their teaching across categories as well. typically we encourage observers to give feedback that highlights categories for strength as well as potential categories for growth and expansion in terms of teaching practice. So we've gotten a lot of really good feedback. The other thing that this observation protocol does is it incorporates the instructor's voice. So it asks the instructor to provide reflections on the class period or modules that were observed. And those are provided to the observer as they write their report.

So you're not just going on what you see happening either in an online space or in a physical classroom space. You also get to hear from the instructor to understand better what it is that went into the planning of the session, or if a particular moment happened in the class period, an instructor might choose to reflect on that a little bit more in depth. I've found when conducting these observations, it really allows me to get much more 360 view of the instructor and to provide feedback in more areas than I would with a traditional observation protocol.

Dr. Claudia Cornejo Happel:

Mm-hmm. <affirmative>, and just adding on to that, we've also worked with a few faculty and also educational developers at other institutions who have adapted and adopted the CTB framework for observations. Primarily. The interesting thing there is they didn't adapt it exactly as we had published it. Mm-hmm. But that I think was one of our intentions that we wanted to provide a framework to open conversations. And so, at least one of the teams that worked with a tool from University of California Fullerton, they spent a whole year looking at the different categories, looking at the tool and thinking about how they might implement it. And we feel that is really one of the, we, we feel accomplished because it really, it started the conversations and sustained it and allowed faculty to really engage with those ideas around evidence-based instructional practices.

Eric Mazur:

And maybe that's exactly what we need, right? That people take ownership of your materials, right? Because that's when you get the best implementation. And also, if I'm not mistaken, you include the protocols in your book, right? So there is an observation protocol suggested to the readers.

Dr. Lauren Barbeau:

So we provide a protocol, one of the chapters on peer observation, and another is on student feedback.

Dr. Claudia Cornejo Happel:

mm-hmm. <affirmative>.

Dr. Lauren Barbeau:

And each of those chapters provides research on why we should be doing this and what the value is, but it also provides sort of step-by-step instructions on how do you do this to get the maximum benefit out of it, and how do you do this using the instruments that we've created. So I did wanna add, Eric, you, you said, you know, maybe the goal is not to have just one way of doing it, but multiple ways. We have creative commons licensed all of the instruments that we've created except the framework itself with the goal of allowing people to adapt this to their institutional context as needed. What we do ask is that people share what they develop with us so that we can then post it to our website under an adaptations tab with the idea that we're sort of communally generating ways of doing things. It's all based around this same framework, providing a language and a foundation for us to do this kind of work. So we're helping each other as we generate these instruments because we've each done something that fits our institutional context and we can learn together what works and how to improve.

Eric Mazur:

So now that your book is out the and having published books myself, I, I know that this is sort of what happens once, once a book gets published, it, it lives the lives of its own, right? I mean, it's out there and and it's out of your hands, not just the book, but the materials in the book

Dr. Lauren Barbeau:

mm-hmm. <affirmative>.

Eric Mazur:

So if we forward the clock 10 years, what are your hopes for the book and your framework? How, how do you hope the CTB framework will have impacted the way we talk about teaching in higher education?

Dr. Claudia Cornejo Happel:

Yeah, and so I'm following up what, what Lauren just said about faculty at different institutions cross disciplines adapting the, the tools to meet their needs. I think that is one of our hopes that the tools inspire conversations in many different universities, institutions, departments, to think about what does good teaching look like and come up with transparent expectation for faculty. And this is what we're, this is what we mean when we say this is good teaching or this is excellent teaching in our department, in our institution. We think that the CTB framework is providing those foundations by focusing on those six big categories. We know that the research on teaching continues to change. And so what we talk about in details in the chapters now, the different strategies is based on research that is available to us now, but there will be new ideas that come up that allow us to think about engaging students, including students or even assessing students in different ways that we have not yet thought about. But we hope that by providing those big categories, it really serves as a foundation on which we can grow and really yes, build and thrive and have communal conversations around teaching that start on the same basis with the same premise in mind.

Dr. Lauren Barbeau:

Yeah, I think to add to what Claudia is saying, I would really want to see this project become a way we think about and approach teaching. So it's providing a foundation to talk about teaching. And so we've, we've done extensive research to identify these six different categories. We didn't just say, you know, these six categories sound like they would be cool. We did extensive literature reviews and sort of looked for what are the common themes that come up across the literature of effectiveness in student learning practices. and that's how we identify these categories. So we would hope to find in 10 years that the categories themselves still stand and that we're able to add new practices to those categories as we encounter emerging research on teaching and learning. I think we would also hope to see this project overall shift the way we evaluate teaching so that we're not relying so much on. There is that one question on the student feedback form that says, my teacher is effective, yes or no, or on a scale of one to five, they do this thing very well, and nope, you're a good teacher, or nope, you're not. So we hope this project overall is going to allow faculty to document their teaching more effectively, but we're also hoping that it will allow those who are evaluating teaching to look beyond the student feedback alone, to incorporate these other aspects of teaching and look at their faculty more holistically.

Dr. Claudia Cornejo Happel:

Yeah. And also to look beyond what is familiar to them, to really look at what has been demonstrated to support teaching and learning.

Eric Mazur:

So you, you, you emphasize Lauren the word alone there when talking about student feedback questionnaires. I understand from that that you're not in favor of completely abolishing student questionnaire. Am I correct?

Dr. Lauren Barbeau:

No, I, I mean, our reliance on mid-semester student feedback suggests that we wouldn't want to abolish that. I think what we would like to see is more equit equitable evaluation that includes the student voice, but also the faculty voice as well as peers. So we want to be sure that we're getting, you know, more of a 360 view of what faculty are doing. And so we're not relying exclusively on student feedback, which we know based on extensive research can be biased for many different reasons. That doesn't mean we throw it out necessarily, but it means that we need to temper it with all these other perspectives on teaching to make it effective.

Eric Mazur:

And now that the book is out, what's next for you?

Dr. Lauren Barbeau:

Well, this is the part where I get to talk about my super exciting project. So I love to have a project, as does Claudia, and obviously we are passionate about critical teaching behaviors, and now the book is out. I am combining that with my other passion, which is Perusall. I have been wanting to bring these two things into conjunction for years, and I finally found a way to do it. Claudia and I are creating a perusal reading community where faculty, or in fact, anyone who's interested in reading the book can join and be part of an author facilitated conversation. So Claudia and I are in the process of filming videos to add more content to that course. So it's not just going to be reading the book together, but there will be videos as well. But there will also be conversation through Perusall where faculty can talk to each other across institutions, and they're also talking to us. So we'll be in there actively participating. We are really excited about this project, so excited that we actually submitted a Perusall Exchange proposal already to talk about social learning for faculty through Perusall platforms. So if you're not able to join us for this first iteration, which begins May 1st and runs until the end of May, I hope that folks who are listening might consider watching our Perusall Exchange presentation where we talk about how that community went and any plans for future iterations based on how this this initial run goes.

Eric Mazur:

Well, that's absolutely wonderful, and I'm going to say a little bit more about that as we wrap up here. I just have one final question, which I'd like to end most of my podcast with, is if there's one thing you could tell our listeners that they should implement or that they could start implementing tomorrow, so to speak, what would that be?

Dr. Claudia Cornejo Happel:

Ooh, that's a great question.

Dr. Lauren Barbeau:

And a tough question.

Dr. Claudia Cornejo Happel:

Yeah. But I think whatever it is, and I think that was really also one of the driving forces. It has to be something that resonates with them, right? There can be all these wonderful ideas out there for you can engage your students in that way. You can have assessments that are exams or projects or recently you talked about the non replaceable assignments on your podcast. So you can have all of these ideas, but in the end, whatever you adopt or whatever you do it, it has to be something that resonates with you. I think we are all in a place, but we are in incredibly at the limit of what we have capacity to do. And so whatever it is, we probably want to start small and we wanna make it something that brings us joy in addition to supporting our student learning.

Dr. Lauren Barbeau:

Not surprisingly, since Claudia and I have worked together so long, I had a very similar response, which was going to sort of center around the word align, align your teaching. So if there's one thing that faculty could do right now, I would recommend I think it's chapter seven of the book. It's on finding your core value. And the idea is that you sit down and you reflect on who you are as a teacher and why you do the things that you do. If you can figure out what your core value is, then you can figure out the sorts of behaviors that you want to implement as well as the why, so that you're not just going out saying like, Ooh, ungrading sounds like a really cool thing. I wanna do that. Not that there's anything wrong with that. We know that ungrading can be very effective, but does it align with who you are as an instructor and what you want to accomplish in your classroom? So for me, I'll echo what Claudia said and say it's all about alignment and finding out who you are as an instructor before you choose your classroom behaviors.

Eric Mazur:

I love that alignment and what a great way to end this podcast. So thank you all for listening and thank you especially to our guest, Dr. Lauren Barbeau and Dr. Claudia Cornejo Happel. So congratulations on your new book. And here's to hoping that the CTB framework will actually impact how we think about teaching and will transform institutions to really start an, in a sense of scholarship about teaching. Now, I mentioned a surprise at the beginning of this podcast, if you fast forward to the end. We already spoke about it in the middle of the podcast, but I am really excited to announce that Perusall and Stylus Publishing, the publisher of Lauren and Claudia's book, will co-organize a four week long collective asynchronous reading of their book. And as Lauren said, this is going to begin on May 1st and run for four weeks. During this event, can peruse Lauren and Claudia's book and asynchronously interact with other readers, and most importantly with them.

And I'm really excited to know that they're already taping additional materials that they will make available in that Perusall event. You'll learn about evidence-based practices to help you identify good teaching and obtain guidance on how to capture and present teaching accomplishment and be able to connect and interact not only with them, but also with other like-minded individuals if unity of people interested in assessing and guiding good teaching practices. Another exciting bit of news, you heard too, Lauren and Claudia will be giving a talk at the Perusall Exchange. So if you haven't registered yet for this year's exciting Perusall Exchange, go to perusall.com/exchange2023. It'll start on June 5th and run through June 11th. I hope that I will see many of you listeners, and of course, Lauren and Claudia at the Exchange this year. You can find our podcast and more on perusall.com/sociallearningamplified subscribe to join us and see you on our next episode.

Eric Mazur:

Social Learning Amplified is sponsored by Perusall, the social learning platform that motivates students by increasing engagement, driving, collaboration, and building community through your favorite course content. To learn more, join us at one of our introductory webinars. Visit perusall.com to learn more and register.