We spend a lot of time talking about the things that we would like to do better. It’s a natural disposition for educators – continually looking for ways to perfect what is, at its core, a fundamentally imperfect enterprise. As long as we keep in mind that our efforts to perfect are really about improvement and not about literal perfection, this mindset can cultivate a healthy environment for demonstrably increasing our educational effectiveness.
However – and I admit that I’m probably a repeat offender here – I don’t think we spend enough time reveling in our success. Often we seem to jump from brushfire to brushfire – sometimes almost frantically so. Though this might come from a genuinely honorable sense of urgency, I think it tends to make our work more exhausting than gratifying. Conversely, taking the time to examine and celebrate our successes does two things. First, it bolsters our confidence in our ability to identify a problem, analyze its cause(s), and implement a successful solution – a confidence that is vital to a culture of perpetual improvement. Second, it helps us more naturally approach problems through a problem-solving lens. There is a lot of evidence to show that examining the nature of a successful effort can be more beneficial than simply understanding every painful detail of how we screwed up.
So this last week before Christmas break, I want to celebrate one such success. If I could hang mistletoe over the campus, I’d likely start doling out kisses (the chocolate kind, or course). In the four terms since we implemented the IDEA Center course feedback process, you have significantly increased the degree to which students report learning in their courses. Between fall of 2011 and fall of 2012, the average Progress on Relevant Objectives (PRO) score for a course has increased from a 3.8 to a 4.1. In addition, on 10 of the 12 individual IDEA learning objectives, students in Augustana courses during the fall of 2012 (last term) reported higher average learning progress scores than students from the overall IDEA data base. More specifically, the average learning gains from our own courses last term were higher than our overall Augustana average from the previous three terms on 10 out of 12 IDEA learning objectives.
Looking deeper into the data, the evidence continues to support the conclusion that our faculty have steadily improved their teaching. Over four terms, faculty have reduced the number of objectives they select and narrowed the gap (i.e., variance – for those of you jonesing for statistical parlance) between progress on individual objectives chosen for a given course. This narrowing precision likely indicates an increasing clarity of educational intent on the part of our faculty. Moreover, this reduction in selected learning objectives has not come at the expense of higher order thinking objectives that might be considered more difficult to teach. On the contrary, the selection of individual learning objectives remains similarly distributed – and equally effective – across surface and deep learning objectives. In addition, students’ responses to the questions regarding “excellent teacher” and “excellent course” went up from 4.2 to 4.3 and from 3.9 to 4.0, respectively. Finally, when asked whether “as a result of this course, I have more positive feelings about this field of study,” students’ average responses increased from 3.9 to 4.0.
Are there some reasons to challenge my conclusions? Maybe. While last year’s participation in the IDEA course feedback process was mandated for all faculty in an effort to develop institutional norms, only about 75% of courses participated this fall. So it’s possible that the courses that didn’t participate in the fall would have pulled down our overall averages. Or maybe our faculty have just learned how to manipulate the system and the increased numbers in both PRO scores, individual learning objectives, and teaching methods and styles are nothing more than our improved ability to game the system.
To both of these counter-arguments, in the spirit of the holiday I say (respectfully) . . . humbug. First of all, although older faculty are traditionally least likely to employ course evaluations (as was the case this fall), I think it is highly unlikely that these faculty are also our worst instructors. On contrary, many of them are master teachers who have found long ago that they needed to develop other methods of gathering course feedback that matched their own approach to teaching. Moreover, even if there were some courses taught by senior faculty in which students would have reported lesser degrees of learning, there were courses with lower PRO scores taught by faculty from all classifications. Second, while there might be some potential for gaming the IDEA system, what I have seen some people refer to as “gaming” has actually been nothing but intentionally designed teaching. If a faculty member decides to select objective 11, “learning to analyze and critically evaluate ideas, arguments, and points of view,” and then tells the students that this is a focus of the course, asked students to develop this skill through a series of assignments, discussions, projects, or papers, and then explains to students when and how they were making progress on this objective . . . that all sounds to me like plain ol’ good teaching. So if that is gaming the system or teaching to the test, then (in the words of every kid who has ever played football in the street), “GAME ON!”
Are there other data points in last term’s IDEA aggregate report that we ought to examine and seek to improve? Sure. But let’s have that conversation later – maybe in January. Right now, let’s revel in the knowledge that we now have evidence to show the fruits of our labor to improve our teaching. You made the commitment to adopt the IDEA course feedback system knowing that it might require us to step up our game. It did, and you responded in kind. Actually, you didn’t just meet the challenge – you rose up and proved yourselves to be better than advertised. So congratulations. You thoroughly deserve it. Merry Christmas.
Make it a great day,
Mark