an associate professor at Moravian college in Bethlehem, Pennsylvania and part of the A&P offering team over at McGraw-Hill. Every other month or so she’ll be bringing us a journal article she’s found that helps enlighten us about evidence based approaches to teaching A&P. I have a link to the first journal article in the show notes and the episode page. If you want to read ahead and perhaps send in your own recorded reaction. But you don’t have to read ahead. Don’t worry about that. Krista will be summarizing the article in the journal club segment, and then she and I will be discussing it a bit. As perhaps, if you have any ideas to share about that or any other feature of this podcast, or perhaps a new feature you’d like to suggest, please let us know so that we can better meet your needs.

Kevin Patton:
Hey, let’s have a peer review. Wait, we already have that. It’s called the listener survey. I don’t have nearly as many of those in as I’d like to have. So why not go theAPprofessor.org/survey right now and tell me what you think. Anonymously, of course. This is professional peer review after all right?

Sponsored by AAA

Kevin Patton:
A searchable transcript and a captioned audiogram of this episode are funded by AAA, the American Association for Anatomy at anatomy.org. Did you know that if you’re looking for microscopic images to use in your remote teaching, AAA has you covered. They have something called the Virtual Microscopy Database, or VMD for short. That has more than 3,500 virtual histology and pathology slides that you can use in your course. Just go to anatomy.org and click on the Resources tab. Then select virtual microscopy database. By the way, this is a free resource for both members and nonmembers.

Revisiting Retrieval Practice

Kevin Patton:
Long time listeners know that I’m a big believer in the power of retrieval practice as a learning strategy. When I say believer, I mean a zealot. But I also mean that there’s a huge body of experimental evidence to support my belief that it’s one of the most powerful ways to learn for the long term. So when I throw out the idea of using retrieval practice, particularly in the form of open book un-timed online tests as I did in the pandemic teaching book I told you about in episode 67. And also in a segment of that same episode that was based on my presentation in the HAPS town hall meeting, I’m advocating a really, really, really powerful strategy. But I also know that there is great resistance to it. I know because I also resisted the idea for a long time. I don’t know, maybe for 10 years or maybe even longer than that. But I do know that when I started doing these oddly designed online tests in my web enhanced face-to-face AP courses way back in 2002, I felt like I might be jumping off the proverbial cliff. I thought I might go splat the bottom of that cliff. And a lot of my colleagues were very, very, very unsure of what I was trying to do.

Kevin Patton:
Now they didn’t try to discourage me or anything like that. But I could tell from the look in their eyes and the things that they said to me that they were worried that I truly lost my mind. That is from quirky and eccentric to full blown mad.

Kevin Patton:
But it turns out that I should have trusted experimental evidence, even though it kind of went against my gut feeling and really flew in the face of the way it’s always been done.

Kevin Patton:
On the other hand, part of my gut also knew it was the right thing to do. You know, that part of my gut that actually trusts the science. And when I thought about how we learn nearly everything we learn outside of the classroom. Things like riding a bike, doing any sport or hobby or even on the job training, like when I was an apprentice lion tamer. I realize that we do it by retrieval practice.

Kevin Patton:
That is we try something, it fails, we fix it. Maybe with someone guiding us and we try again and fail again, and get more advice in coaching. And eventually we get it. And learning things that way, we often never forget it. We don’t read about lion taming or soccer, or I don’t know, trout fishing. Take a quiz on what we’ve read and then we’re able to just go out and do it. That just doesn’t make sense right? Yet that’s what we do in our academic courses and what we’re often afraid to let go of.

Sponsored by HAPI

Kevin Patton:
The free distribution of this podcast is sponsored by the master of science in human anatomy and physiology instruction. The HAPI degree. I’m continuing to get all kinds of feedback from our current and former HAPI graduate students who are out in the trenches during this time of pandemic teaching. And they’re expressing their gratitude for giving them the tools, techniques, and the flexible mindset to cope with the sudden move from on-campus teaching to remote learning. And you can get this too. We have many students who already have doctorates and masters degrees in our program and they love it. And yes, most of them already have other full time responsibilities. It’s kind of hard yeah, but it’s doable. Check out this online graduate program at nycc.edu/hapi. That’s H-A-P-I. Or click the link in the show notes or episode page.

Open-book & Untimed & BEARS—oh my!

Kevin Patton:
I’ve talked about this retrieval practice thing in past episodes. So you may want to go back and listen to the back catalog as you walk your dog around your neighborhood at a safe distance from others. But right now, I do want to summarize the essential way my implementation of retrieval practice works in case you want to try something like it or in case it sparks an idea for some little tweak that you can make to what you’re already doing or are already planning for the next semester.

Kevin Patton:
First, it’s all based on online open book tests. That open book things seems heretical, I know. But really this is real life, so I just needed to get over that. And I’m glad I did, because it turns out that this is the way that students achieve mastery by getting help when they need it. Just like in soccer or trout fishing. And you know what? Isn’t looking things up and consulting with peers, something that the truly competent health professionals do all the time? Hey, if they’re taking care of me, I’d rather they double check the proper dosage of that script they’re about to write for me, or ask a colleague to check my medical imaging before making a diagnosis. Why wouldn’t I want my students to use those options in A&P?

Kevin Patton:
The next thing is that I set my learning management system to serve up the questions one at a time, just like they do in the online board exams that most health professionals have to take to get licensed or to proceed from one stage of professional training to the next. I heard so many of my former students tell me that this one at a time format raised their stress levels even higher than they already were walking into that board exam. Meaning maximum stress. Right?

Kevin Patton:
After I’d been doing that for awhile in my class, I started hearing from former students that they were so glad that they’d gotten used to doing that in my class. And that unlike some of their classmates who didn’t have me for A&P, my students didn’t freak out when they had to do test items one at a time on their board tests. But you know what? That’s just gravy.

Kevin Patton:
The main reason I do the one at a time thing is that I found out that during in-class testing, some students had difficulty focusing when they were faced with many items on one page.

Kevin Patton:
Now I know you and I are used to that. Or more likely, our brains are just wired a certain way to allow us to focus on just one item on a page of many items. But for some students, probably each with their own unique set of neural pathways and connections, the ability to see just one test item at a time helps them focus. Which means that it helps them to succeed.

Kevin Patton:
Next, I don’t strictly time my tests. There’s usually a start date and an end date, but not a limit of a certain number of minutes. Way back in 2002 when I first started doing this, I didn’t plan on giving untimed tests. I think I set them up in my first one with a one hour time limit, and it had to be on a certain date and within a certain limited window of time. Like 90 minutes. Guess what? Because I had one of the highest number of students per semester at our college, and because I was apparently the first professor in the history of our college to give students one test item at a time, there were too many hits on our server per hour. And the learning management system started wheezing and coughing. And well, it simply stopped working for awhile. Not what you’d want to happen when 250 students have a limited time to take a test. Right? Especially an A&P test, because those are always scary. And even more so when it’s an online test.

Kevin Patton:
Because remember, this was almost 20 years ago when not very many students had ever taken an online test, or at least weren’t very comfortable with them then. Not at our college, certainly. So I went to our staff and they advised me to stretch out the dates of my test taking window and make my tests untimed. Of course that sticky, half frozen stay in my comfort zone part of my brain rebelled. What?! An un-timed test that they could take over the course of several hours or days?

Kevin Patton:
But the alternative was the shutdown our server for my students and all the students at our college. So yeah. Okay, I relented. And I think I kind of did pout a little bit. At least in my head, I was pouting.

Kevin Patton:
You know what though? It turned out to be a breakthrough. Untimed testing is so freeing to the learning process and it’s so helpful to students with almost any kind of learning challenge, which is everyone in my opinion. Sure. Eventually medical professionals will need to be able to answer things quickly on the spot. But we should not expect them to start there. Should we?

Kevin Patton:
So okay. Maybe close the books and notebooks and have some kind of time limit on a midterm or final exam, maybe. But really I’m thinking of the closed books and timing should come at or near the end of their degree program. Not in my course at all. In A&P, they’re still beginners. Even at the end of my course.

Sponsored by HAPS

Kevin Patton:
Marketing support for this podcast is provided by HAPS, Human Anatomy and Physiology Society, promoting excellence in the teaching of human anatomy and physiology for over 30 years. As I mentioned in the last episode, if you haven’t been stopping by for the virtual town meetings, you’re missing a great opportunity to visit with other A&P teachers and learn about what they’re doing in their courses. You can just listen in or you can ask questions, or you can share what’s worked for you or what’s not worked for you. It’s really fun and it’s uplifting. It’s a way to stay connected to others in the same boat as you are. If you’re a member, watch your emails. And if not, go visit haps at theapprofessor.org/haps, that’s H-A-P-S. And join today.

Recipe for Success

Kevin Patton:
The next step of my recipe calls for a lot of retrieval practice. More than you are used to. Way way more than you and I ever saw in our own education. Retrieval practice, the topic of the very first episode of this podcast is this decade’s favorite jargon for plain, old fashioned testing, or quizzing, or whatever you want to call it. That is being challenged with test items, case studies, problems, specimens to identify. Any and all manner of challenges to go into our memory banks and find the pertinent facts and sort them out and put them together. And retrieve the solution to the challenge. Answer the test item. That’s the retrieval part.

Kevin Patton:
Then do that retrieval with a slightly different problem again and then again so that we get better and better at it. While also improving our longterm retention of relevant memories. That’s the practice part of retrieval practice.

Kevin Patton:
Looking at retrieval practice from the testing angle, we see that this isn’t the kind of testing that most of us first think of when we think of a test. We usually think about summative testing that measures how well a student has learned something. But testing as retrieval practice is instead a formative kind of testing that is measuring how much a student has learned while they are still learning it. It’s sort of like a practice soccer game. You find out what’s working and what’s not working before you play a regular game that counts for your team’s season record and possibly for a championship. So the practice sessions and practice games are formative, and the regular game is summative.

Kevin Patton:
Back in the olden days when I was an undergraduate, that was done through self-graded homework. It would have been impossible for professors, even those with a large team of TA’s, to immediately grade a big stack of homework coming in every single day including weekends.

Kevin Patton:
But thanks to modern learning management systems, I can do that through automatic grading. A student can take an online formative test and get immediate results. Then take another attempt, and another one. Gaining all the benefits of retrieval practice and being able to truly master a concept and remember it for the long term.

Kevin Patton:
Again, back in the olden days, a lot of what I learned was held onto for I don’t know, a few days or weeks, just long enough to pass that next in class test. And then I’d forget it or forget most of it. What I did hold onto for the longterm were concepts that came up again in a later section or a later course, that came soon after the previous one. I have found that giving students multiple attempts gives them enough retrieval practice for it to really do some good. In some courses, I gave them up to three or four attempts with the highest score counting for the course grade. In other courses, I’ve given them unlimited attempts. And you know what? Both approaches work just fine.

Kevin Patton:
The trick of course is to make sure that each test is different. That is, it still tests the same objectives or learning outcomes, but with a different set of test items.

Sponsored by ADInstruments

Kevin Patton:
Support for this episode comes from our friends at ADInstruments who’ve come out to help support A&P teachers and our time of need. They’ve come up with a plan to let any of us use their Lt online learning platform for free. As I mentioned in the last couple of episodes, Lt is an online environment that you can set up very quickly. And it contains ready to use content for anatomy, physiology, and other biology courses, even medical courses. It has over 360 interactive and fully customizable lessons and labs that are just the thing for remote learning. You already know that ADInstruments provides only very high quality products and have access to a whole set of the remote learning resources right now for free is an incredible thing, especially during our adventures with pandemic teaching. Just go to ADInstruments.com/lt/covid19 to find out more. Once again, that’s ADInstruments.com/lt/covid19.

Mix Ingredients

Kevin Patton:
If we want to have a different version of anyone attempt of anyone test and we want to allow at least three attempts, whoa. That’s a lot of different tests to write, isn’t it? And if we want to allow unlimited tests, which is the direction which I’ve been heading, then wow, we need a bajillion versions of each test. Any of that is pretty much impossible, right? Or is it impossible?

Kevin Patton:
Nope. Not only is it possible, it’s fairly easy. Not super easy, but fairly easy. All the learning management systems have the ability to use what I’m going to call a question set, but each LMS has its own name for this feature. Here’s how it works.

Kevin Patton:
For test item one, I write three versions. They could be three different ways to ask the same question, or they could be three different aspects of the concept that I want to assess and test item one for example, it could be three different examples of cardiac muscle tissue to identify. Or it could be one each of three major types of muscle tissue. When a student takes attempt one, one of those three items in the question set will be presented as test item one.

Kevin Patton:
In their next attempt, it could be the same item presented or more likely one of the other two versions because it’s randomly selected by the learning management system. Even if it’s the same item one that they saw in their first test attempt. Each item in the test, let’s say 50 items total, will be similarly randomized.

Kevin Patton:
So yeah, okay. Test item one is the same as they’ve seen before. But maybe, probably, all or most of the other 49 items on attempt two will be different than they’ve seen before. As with a slot machine, having three items in each question set for 50 test items will generate, now hold onto your seats here, it’ll generate 7.2 times 10 to the 23rd different possible test attempt versions. Yeah, that’s a lot, right? That’s more than a mole of different versions.

Kevin Patton:
The thing is, I usually have more than three items per question set. I divide up what I want my students to know in the 50 categories. Some of which overlap because I want to attack some things at different levels of understanding or application or style of test item. Then once I get going and I’m constructing three items for my first category or my first question set, you know what? I always think of a fourth item which sparks an idea for a fifth item.

Kevin Patton:
Not only that, but for multiple choice and matching items, I can set the LMS to randomize the choices so I’ve got another bajillion, bazillion versions of that test being generated. Because not only is the LMS picking random items out of a question set of three or four or five, it’s also scrambling up the choices within a multiple choice or matching item.

Kevin Patton:
When I first started doing this back in 2002, I got a math professor to help calculate the possible number of versions on any one of my tests, and her calculator could not go that high. It just gave an error. And being a math professor, she had a pretty powerful calculator. And you know what? That’s all I needed to know. I didn’t need to know the exact number. I just needed to know it blew out the math professor’s calculator. So that was good enough for me. That is good enough to ensure that my students weren’t getting the same test and every attempt. And of course I waited until the semester started to begin making my first test. Whoops. Should have started that way in advance so well.

Kevin Patton:
So every day that first semester when I came home from school, I immediately set my timer for 20 minutes and wrote as many test items as I could before the timer went off. Doing that every day, yeah it was a chore. But really 20 minutes was not that big of a chore. And it became a habit. So it really wasn’t that hard to sit down and do it for 20 minutes every day. Because that’s just kind of what I got used to.

Kevin Patton:
After my first test was done, I was really getting good at writing test items and getting faster and faster about coming up with good test items. And as students were taking attempts of that first test while I was working on test two, they were taking the first test and they were giving me feedback on which items were poorly constructed, or just odd or full of typos, or something was wrong with it. I could claim that I did that on purpose, but I didn’t. I just got started late, but it did turn out to have that silver lining.

Kevin Patton:
Then the next time I taught that course, it was easy to add a few more test items to some of the questions sets. Of course, I was by now a seasoned pro at test item writing. Something I had always dreaded because I didn’t feel like I was any good at it. But all that retrieval practice gave me a certain level of mastery. Not only mastery of test item writing, but I also gained insights about the A&P concepts that I was considering for test items. I lived through it, and I’m a better person for it. Or at least a slightly better person for it.

Kevin Patton:
So yeah, like any new project, it seems intimidating. But it’s like a very long walking trip. You take one step at a time. And I don’t know if you walk for 20 minutes a day, four days a week, for two 16 week semesters, you’ll end up having walked more than 125 miles, probably. With no injuries, no collapsing on the roadside, no dehydration. Just a smile on your face. So do it one step at a time, and you can do it too.

Icing & Decorations

Kevin Patton:
I also want to mention some icing that I put on that retrieval practice cake, and decorations. After all, isn’t that cake that much better when it has icing and decorations? One thing I did that helped a lot, the icing, was give students an online pretest before each module of the course. And online attempts taken from that huge test bank that I’d built over that coming modules content. No grade because I told them to expect to fail it. But they had to take it unlike the videos and other learning resources for the coming module. Honestly, it took a while to convince them not to read ahead and not to study, not to prepare for it. It was merely to give them a peek at what they’d be expected to know later, but were not in any way expected to know yet. And you know what, those pretests helped a lot.

Kevin Patton:
Now the cake decorations consisted of making my online tests cumulative. Every test had questions from all the previous tests. The really important questions, the core concepts, the big ideas. Yeah, of course they balked at that when I first told them. Who wouldn’t? We’ve all learned that cumulative tests are scary. But it didn’t take them long to realize that those questions from prior modules were the easy questions to answer on a new test. And by the time they got through a few tests, those questions got even easier because they kept seeing those kinds of questions again and again. Questions on those topics kept coming up. So they grew to love cumulative testing. I know that sounds weird, and I know you don’t believe me, but I swear it’s true.

Kevin Patton:
And I know what you’re thinking. That if we’re doing all the formative testing and it’s open book and students can even consult with each other, then well, can we trust that they’ve actually learned anything? That they have any knowledge in their heads that they can use without a book or without asking anyone else if they’re on the right track? Well, the answer to that is an emphatic and wholehearted yes. How do I know that, and how can I be so confident about it?

Kevin Patton:
It’s because when I first started this and for many years thereafter, I also did two summative exams to see whether all that retrieval practice worked. And you know what? It did. Not only did students walk into their exams with more confidence, they did way better on them than my past classes had done on their exams before I started doing any of this formative testing.

Kevin Patton:
Possibly, part of that improvement came from their confidence, but I’m sure that a lot of it came from all those open book formative tests. And my students told me that they not only felt confident on exam day, they also told me that they tried to cram for the exam but found that there just wasn’t any need to. As they prepared to cram the night before an age old ritual they felt compelled to perform. They realized as they reviewed the course content that they already kind of knew it all and knew it pretty well. So yeah, I don’t have any problem thinking that the formative testing alone gave them what they needed.

Kevin Patton:
There’s more to my story of retrieval practice in the A&P course. And a lot of that story is told in previous episodes and in seminars that I have available in the links in the show notes and episode page. But before I wrap this up for now, I want to mention that I really think it works best if we write our own test items most of the time. Sure, the test items at the end of textbook chapters, in study guides, in online study or quizzing programs and the like are all useful tools, and have their place in learning. But I’ve grown to like making my own. And I think if I use my own test items that integrate easily with the way I tell the story of A&P, it works better for learning. But that’s a discussion for another day.

Pandemic Teaching Book (please share!)

Kevin Patton:
In the previous episode, that is episode 67, I mentioned a free ebook that I recently published. It’s called Pandemic Teaching: A Survival Guide for College Faculty. It’s a little over 100 pages, and it’s a quick, easy read. About two hours. And it’s chock-full of ideas, tips, and mindsets that help us move to remote teaching and be successful at it. I’d really love it if you could do me a favor. Please, could you take a moment and share it with three colleagues at your school, one of them being whoever organizes the professional development for your faculty? Really after going to all the trouble of putting it together, I’d like folks to get some use out of this book. It’s a free download from all the major ebook stores. Just share this link I’m going to give you to see a list of different places to find it like Amazon, Kindle, Barnes & Noble NOOK, Apple Books, and all of the list goes on. So that link is books2read. That’s books, the numeral two, read.com/pandemicteaching. Once again, books2read.com/pandemicteaching. Oh yeah, and could you also take a moment to mention it in social media and share that link when you do it? I sure do appreciate your help with this.

Staying Connected

Kevin Patton:
I always put links in the show notes and at the episode page at theAPprofessor.org/68. In case you want to further explore any ideas mentioned in this podcast or if you want to visit our sponsors. And you’re always encouraged to call in with your questions, comments, and ideas at the podcast hotline. That’s 1(833) LION DEN. Or 1(833) 546-6336. Or send a recording or written message to podcast@theAPprofessor.org. I’ll see you down the road.

Aileen:
The A&P professor is hosted by Dr. Kevin Patton, an award-winning professor and textbook author in human anatomy and physiology.

Kevin Patton:
Any resemblance to actual persons living or dead is unintentional and purely coincidental.