Well hello, everyone!
It’s great to feel the energy on campus again. And it’s exciting to restart my Delicious Ambiguity blog: Season 4 (I’ve been renewed!). Each week I share some tidbit (data that comes from statistics or focus groups) from our own Augustana student data that will help you do what you do just a little bit better the next time you do it. If you’re new to Delicious Ambiguity, you might also want to know that you can search the three years of previous posts (about 100 in all) for everything from athletes to introverts, Greeks to geeks. In addition to a ton of useful findings, you might even find a few funny quips (AKA bewildering side comments).
By now you’ve probably heard me say on at least one occasion that building assessment efforts around genuine improvement, as opposed to doing assessment to find out what’s already happened (i.e., to prove what you think you are already doing), thoroughly changes every part of the assessment process. More importantly, it’s the only way to actually get better because:
- You’ve backward-designed the entire project around finding out what you need to do to get better instead of just finding out what happened, and
- You’ve humbled yourself to the possibility of improvement and thereby matched your efforts with the way that educational processes actually work.
I’d like to share an example of one program at Augustana that has clearly benefited from an “assessment for improvement” approach. My goal here isn’t to brag, but rather to walk you through an example of such a process in the hope that something I share might be applicable to your own unique context.
Augustana has run some version of freshman orientation for a very long time. And by and large, it’s been a pretty successful program. Yet everyone involved has always wondered what they might do to make it just a little bit better. Much of our prior data only told us the proportion of students who were “satisfied” with the experience. Although we could pat ourselves on the back when the numbers were decent (which they were virtually every year), we had no way of turning that information into specific changes that we could trust would actually make the experience demonstrably more effective.
So a few years ago, folks from Student Affairs, Academic Affairs, and the IR office began applying an improvement-centered assessment approach to orientation. First, we talked at length about drilling down to the core learning goals of freshman orientation. Sure, we have lots of things we’d love to result from orientation, but realistically there are only so many things you can do in four days for a group of 18-year-olds who are some combination of giddy, overwhelmed, and panicked – none of which makes for particularly fertile learning conditions.
So with that in mind, we needed to strip our goals down to a “triage” set of learning goals for orientation. We settled on three concepts.
- Welcome Week will connect new students with the people necessary for success.
- Welcome Week will connect new students with the places necessary for success.
- Welcome Week will connect new students with the values necessary for success.
The people we identified included all of the individuals who might influence a student’s first year experience – other students, student affairs and residential life staff, and specific faculty members. The concept of place involved a) knowing EXACTLY how to walk to one’s classes and specific first-year resources, and b) finding other places on campus that a student might use for emotional rejuvenation as well as intellectual work. The values we discussed focused on clarifying a strategy for getting the most out of a liberal arts college setting. This meant introducing students to a mindset that focuses on actively participating in a process of learning and growth and show them how this approach will increase their likelihood of success, both in the first year and beyond.
Once we spelled out our goals for Welcome Week, then we could set about our work from two directions. First, we could start to alter the design of the experience to meet those goals. Second, we could build a survey that examined the degree to which students came away from Welcome Week connected to the people, places, and values that substantially increase the likelihood of success in the first year.
Over the last two years the survey findings have provided a number of interesting insights into the degree to which certain experiences were already meeting the goals we had set. More importantly, the survey data has become a critical conversation guide for specific improvements. Because the questions were built around specific experiences, it has given everyone – particularly peer mentors – a clear target to shoot for with each student. For example, if the goal was to ensure that each student would say that they knew exactly how to get to their classes on the first day, then the peer mentor could shift from merely pointing at buildings while walking around campus to creating some way for new freshmen to walk right up to the door of the room where their class would be.
At the same time that we were using data to guide specific adjustments, the folks planning Welcome Week examined the design of the entire program. This led them to introduce several changes, including the Saturday morning concurrent sessions titled “Augie 101,” focusing on all kinds of issues that would specifically increase the likelihood of successful academic acclimation.
We will survey the freshmen in the next week or so to find out how the most recent set of changes impacted their experience during Welcome Week. But even without that data, I suspect that the new programming this year improved the experience. My confidence comes from one particularly compelling data point that isn’t a number (I know – sit down and take a deep breath!). During the Augie 101 sessions, peer mentors and other older students who were assisting with Welcome Week kept saying, “I wish we would have had something like this during my Welcome Week experience.” To me, that is a powerful endorsement of our efforts.
We will likely never be perfect, but we have mounting evidence that we keep getting better at what we do. That doesn’t mean that we have any reason to brag or rest on our laurels. It just means that we are doing things right. And that’s what makes doing this work so much fun.
Make it a good day,
Mark