Last week I wrote a piece for Inside Higher Ed titled “Dear College Presidents.” They gave it the headline “A Modest Proposal on Rankings” . . . which also works.Given the wave of responses I have received since the piece was published, I thought I’d point out two things.
Organizational Change
Data, Analysis, ACTION (now the camera’s bright lights shine on you!)
A couple of weeks ago, the Assessment for Improvement Committee (AIC) and Institutional Research and Assessment (IR&A) hosted the third of three Friday Conversations focused on improving our students’ cognitive sophistication. Unless you’ve been living under a rock (or a pile of semester transition documents!), you know by now that one of the primary functions of AIC and IR&A is to foster an organizational culture of perpetual improvement. To that end, we run a perpetual cycle of data collection, analysis, and communication about the relationships between the student learning and student experience to shine a light on the ways in which we can improve what we do as a college.
The cycle that culminated this year (the entire process takes 5-6 years) focused on the category of learning outcomes we have called “cognitive sophistication.” In particular, we explored data gathered from the cohort of students who entered Augustana in the fall of 2013 and graduated in the spring of 2017 to examine the development of our students’ inclination to, and interest in, thinking about complex or complicated issues or ideas. Just in case you need to catch yourself up, have a quick look at the three previous posts about this process:
- Does our Students’ Interest in Complex Thinking Change over Four Years
- What Experiences Improve our Student’s Inclination Toward Complex Thinking
- Doing Something with What We Now Know
In the fall term, we presented what we had found about the nature of our student’s growth and collected your suggestions about student experiences and characteristics that might influence this growth. In the winter term, we presented the results of testing your suggestions to identify the student experiences that appear to be statistically significant predictors (i.e., particularly influential experiences) of our students’ growth. By contrast, during the spring term Friday Conversation, AIC and IR&A changes it up a bit and turn the session over to whoever shows up. Because if we – meaning the Augustana community – are going to convert our findings into demonstrable improvements, then we – meaning AIC and IR&A – need to hand these findings over to you and let you shape the way that we translate evidence into improvement.
If you clicked on the third post linked above, you didn’t find the results of the third Friday Conversation, but rather a plug and a plea for attendees. Fortunately, a healthy number of faculty and staff showed up ready to put their brains to work. Folks broke into three groups and narrowed a range of ideas into one or two practical ways that the college could put our findings to use. So without further ado, here are the focal points of the conversation from the last Friday Conversation.
Learning in Context
The first set of findings from our data suggested that when students engage in hands-on or experiential learning experiences, their inclination toward complex thinking seems to increase. This may be because experiencing learning in real-world or hands-on settings inevitably add a context that often complicates what might have seemed more simple when discussed in the sanitary safety of a classroom. As students get accustomed to learning or applying prior learning in these real-world settings, research on experiential learning reveals that students find this learning more interesting and sometimes even invigorating.
Even though Augustana offers all sorts of hands-on learning experiences (e.g., internships, research with faculty, community involvement, etc.), it seems that the distribution of these opportunities across majors is uneven. As a result, students in some programs have a much higher chance of gaining access to these kinds of experiences than other students. The faculty and staff focused on this topic considered policy or practice ideas that could bring more of these kinds of opportunities to programs where they have not traditionally thrived. At the same time, the faculty and staff who joined this part of the conversation emphasized the need to offer professional development in order to help faculty in these programs imagine or craft an expanded range of hands-on learning opportunities, especially in disciplines where faculty research tends to be a solo endeavor or the nature of that research tends to explore far beyond an undergraduate’s scope of understanding.
Integrative Advising
This discussion focused on the “integrative” part of integrative advising. Our findings suggested that the more students engage in the integrative aspects of advising conversations (i.e. when faculty or staff prod students to weave together the variety of things they’ve done in college – AKA that long list at the bottom of the email signature – into a coherent narrative), the more they tend to develop an inclination toward complex thinking. This may be because asking students to turn their own raw data (after all, a list of disparate activities is very much like a set of raw data) into a story requires them to engage in complex thinking about uncertainty from two directions: 1) what themes are already present throughout my various activities that could form the basis of a compelling narrative and 2) given where I want to end up after college, how should I alter my list of activities to better prepare for success in that setting?
Participants in this discussion honed in on three ideas that are either already in development or could be introduced. First, they talked about the existing FYI proposal that includes a portfolio. This portfolio might be an especially good way to get first-year students to map out their college experience with the end (i.e. who they want to be when they receive their diploma) explicitly in mind. Second, the participants talked about the need for a way to continue this way of thinking beyond the first year portfolio and landed on a common assignment within the Reasoned Examination of Faith course (formerly Christian Traditions) that would focus on vocation-seeking and purpose. Third, they identified a continuing need for faculty development that would help individuals apply holistic/integrative advising practices no matter the advising context.
Interdisciplinary Discussions
The third group of faculty and staff tackled the challenge of increasing student participation in interdisciplinary discussions. It shouldn’t be much of a surprise by now that the experiences that we found to predict greater gains in cognitive sophistication were those that required students to apply one set of perspectives or ideas within a different, and often more tangible, context or framework. Augustana already offers several avenues for these kinds of conversations (e.g., Salon, Symposium Day, etc.), and there is a certain subset of students who continually participate with enthusiasm. But increasing student participation in these events means focusing on the subset of students who don’t jump at these opportunities. One possibility included finding ways for students to attend conferences in the region when they aren’t presenting research. Another possibility included fostering more interdisciplinary student groups. A third intriguing idea involved the conversations about a Creativity Center on campus and the idea that this initiative might be an ideal vehicle to bring together students from disciplines that might not normally intersect.
Now comes the hardest part of this process. There isn’t a lot of reason to collect student learning data and identify the experiences that shape that learning if we don’t do anything with what we find out. AIC and IR&A will continue to encourage the campus to plug these findings into policy, program, or curricular design. But we need you to take these findings and discussion points and champion them within your own work.
When you (notice the “when” rather than “if”?) have implemented something cool and creative, can you send me an email and tell me about it? I’ll be sure to share it with the rest of the college and celebrate your work!
Make it a good day,
Mark
Improving an Inclination toward Complex Thinking – Part III (AKA doing something with what we now know)
So far this year, the Assessment for Improvement Committee (AIC) and the Office of Institutional Research and Assessment (IRA) have hosted two Friday Conversations to explore findings from our 4-year study of intellectual sophistication growth among Augustana students. The first conversation focused on how our students changed from Welcome Week (just before freshmen start their first fall term) to graduation and how different types of students (depending on traits like race, sex, or pre-college academic preparedness), although they may have started in different places, seem to have grown about the same amount over four years (I summarized that presentation in a blog post here). The second conversation examined the different student experiences that appear to influence that change, either positively or negatively (you can read a summary of that presentation in a blog post here). Clearly, our findings suggest that the degree to which students take ideas they’ve learned in one discipline and apply them or vet them in a different disciplinary or real-world setting demonstrably increases our student’s inclination toward complex thinking.
Although the findings we’ve discussed so far are interesting in their own right, they don’t do anything by themselves to help us improve student learning. In fact, without collectively committing to do something with our results, we end up just like most organizations – chock-full of data but unable to turn those insights into actions that actually make them better. If we’re being honest, the fact that we know so much about how our students’ growth and the experiences that shape that growth puts us in the unenviable position of being almost morally obligated to do something with what we know – no matter how daunting that might be.
I know all of that sounds a little heavy-handed (ok – more than a little heavy-handed), but in the 8 years I’ve been at Augustana, the times when we’ve been at our absolute best have been when we’ve let down our defenses, humbly looked in the mirror, and chosen to believe in the best of each other. Then we’ve all put our shoulders to the plow to make the education we provide just a little bit better than it was before.
And that is the focus of the third, and most important, AIC/IRA Friday Conversation at the end of this week. After we briefly review what we have learned from our data, we will organize into smaller groups to come up with 2-4 viable ways in which we can turn these findings into action. This might take the form of professional development sessions, policy for course design or pedagogical nuance, or co-curricular emphases to apply our findings to impact a larger proportion of students.
So please come to the AIC/IRA Friday Conversation this Friday, March 23rd. We will be in the Wilson Center. Food and drinks are available at 3:30 and the conversation will start at 4:00.
We are really good at getting better. I’ve seen us do it over and over again. I, for one, can’t wait to see what we come up with!
Make it a good day,
Mark
What do students do about the textbooks and additional materials we assign?
At first, that might seem like an incredibly dumb question. If you’re in a salty mood, you might snarl, “Buy them and learn or fail the damn course.” For most of us, I suspect the thought of not buying the additional materials required (or even recommended) for a class might seem utterly absurd. When I was an undergraduate, I remember being warned not to buy used books because they would likely have someone else’s notes in the margins, leaving no room for me to write my own (ok, maybe not the most convincing argument). Nonetheless, I definitely remember feeling like a slacker if I didn’t show up to the first day of class with a shiny new version of each required text.
Fast forward 30-odd years and things couldn’t be more different. The cost of textbooks has risen even faster than the cost of college tuition (have a look at this graphic from the Bureau of Labor Statistics), even as the cost of recreational books has gone down.
More and more, it appears that students are renting textbooks, borrowing from friends, or just foregoing some books altogether. The Chronicle of Higher Ed highlighted a study in 2011 suggesting that 7 of 10 students have skipped buying a textbook because of cost. More recent online columns and blogs seem to perpetuate the notion, if not brag outright, that a student can succeed in college without buying books. In January, the Atlantic published a longer piece examining the reality that, despite the surge in online and other edtech resources, the cost of textbooks and/or their online equivalent remains exorbitantly high. And in the context of the financial pressures that many students experience just paying tuition, room, and board, I guess it shouldn’t surprise us much when already financially-strapped students take advantage of any alternative that might save them some money.
A few weeks ago, about forty faculty and staff gathered in the library to kickstart a conversation about Augustana students and textbooks. After discussing the financial realities of textbook costs, the conversation turned toward the ways in which we choose the textbooks and additional materials that we assign. Although this is something that we might take for granted at times (especially if one might be scrambling to put a course together), it’s an issue that more and more folks are trying to address. I’m sure there are plenty of examples, but three impressive efforts include the Open Textbook Library, College Open Textbooks, and the Open Educational Resources Commons. Most recently, 40 colleges have made the move to simply go without textbooks and only use freely available learning resources (see here and here).
At the end of the meeting, it seemed clear that we really need to know more about our student’s engagement with the textbooks and additional materials assigned in our courses. One person posed an exceedingly logical suggestion: could we add a couple of questions to the end of every IDEA course feedback survey at the end of spring term asking about:
- The amount that students spent on textbooks for a given class
- How often they used the textbooks and additional materials they bought for that class
- How effective those materials were in helping the student learn in that class
It seems like this would be particularly useful information. But before acting on any of these ideas, I think it’s important to know what you all think about gathering this information, what questions you might have about what is done with this information, and any other concerns you might have about this project.
So . . . . what do you think? Should we ask these questions? What should we do with the data? If we ask these questions, how do we need to be careful and transparent so that whatever we find, 1) gives us a deeper understanding of our students’ engagement with textbooks and additional materials, and 2) genuinely spurs our perpetual effort to improve in a way that fosters inclusiveness and understanding.
Please – send me your thoughts. If you know my email, you can send them there. If you’d rather post in the comments section below, please post away.
Make it a good day,
Mark
This March, it’s Survey Madness!
Even folks who are barely familiar with social science research know the term “survey fatigue.” It describes a phenomenon, empirically supported now by a solid body of research, in which people who are asked to take surveys seem to have only a finite amount of tolerance for it (shocking, I know). So as a survey gets longer, respondents tend to skip questions or take less time answering them carefully. When the term first emerged, it primarily referred to something that could happen within an individual survey. But now that solicitations to take surveys seem to appear almost everywhere, the concept is appropriately applied in reference to a sort of meta survey fatigue.
But if we want to get better at something, we need information to guide our choices. We ought to know by now that “winging it” isn’t much of a strategy. So we need to collect data, and oftentimes survey research is the most efficient way to do that.
Therefore, in my never-ending quest to turn negatives into positives, I’m going to launch a new phrase into the pop culture ether. Instead of focusing on the detrimental potential of “survey fatigue,” I’m going to ask that we all dig down and build up our “survey fitness.”
Here’s why . . .
In the next couple of months, you are going to receive a few requests for survey data. Many of you have already received an invitation to participate in the “Great Colleges to Work For” survey. The questions in this survey try to capture a sense of the organization’s culture and employee engagement. For all of you who take pride in your curmudgeonly DNA, I can’t argue your criticism of the name of that survey. But they didn’t ask me when they wrote it, so we’re stuck with it. Nonetheless, the findings actually prove useful. So please take the time to answer honestly if you get an email from them.
The second survey invitation you’ll receive is for a new instrument called The Campus Living, Learning, and Work Environment. It tries to tackle aspects of equity and inclusion across a campus community. One of the reasons I signed on for this study is because it is the first that I know of to survey the entire community – faculty, staff, administration, and students. We have been talking a lot lately about the need for this kind of comprehensive data, and here is our chance to get some.
So if you find yourself getting annoyed at the increased number of survey requests this spring, you can blame it all on me. You are even welcomed to complain to me about all the surveys I’ve sent out this term if that is what it takes to get you to complete them. And if you start to worry about survey fatigue in yourself or others during the next few months, think of it as an opportunity to develop your survey fitness! And thanks for putting up with a few more requests for data than usual. I guarantee that I won’t let the data just sit at the bottom of a hard drive.
Make it a good day,
Mark
Can I ask a delicate question?
Since this is a crazy week for everyone, I’m going to try to post something that you can contemplate when you get the chance to relax your heart rate and breathe. I hope that you will give me the benefit of the doubt when you read this post, because I can imagine this question might be a delicate one and I raise it because I suspect it might help us more authentically and more honestly navigate through some obviously choppy waters as we make some key decisions about our new semester design.
Sometimes, when we advocate for the value of double majors and similar, or even improved, access to double majors in the new semester system, it seems like the rationale for this argument is grounded in the belief that double-majoring is advantageous for Augustana graduates and, as a corollary, relatively easy access to a double-major is helpful in recruiting strong prospective students. In other instances, it sounds as if we advocate for ease of access to double-majoring because we are afraid that programs with smaller numbers of majors will not survive if we build a system that produces fewer double majors.
Without question, both rationales come from the best of places. Doing all that we can for the sake of our student’s potential future success or the possibility of attracting a stronger and larger pool of future students seems utterly reasonable. Likewise, ensuring the health of all current academic departments, especially those that currently enjoy a smaller number of majors, and therefore ensuring the employment stability of all current faculty, is also utterly reasonable.
Yet I wonder if our endeavor to design the best possible semester system would benefit from parsing these concerns more clearly, examining them as distinct issues, and addressing them separately as we proceed. Because it seems to me that prioritizing double-majoring because it benefits post-graduate success, prioritizing double-majoring because it improves recruiting, and prioritizing double-majoring because it ensures employment stability for faculty is not the same as more directly identifying the factors that maximize our students’ post-graduate success, optimizing our offerings (and the way we communicate them) to maximize our recruiting efforts, and designing a system that maintains employment stability and quality for all of our current faculty members. The first approach asserts a causal relationship and seems to narrow our attention toward a single means to an end. The second approach focuses our attention on a goal while broadening the potential ways by which we might achieve it.
Certainly we can empirically test the degree to which double-majoring increases our student’s post-graduate success or whether a double-major friendly system strengthens our efforts to recruit strong students. We could triangulate our findings with other research on the impact of double-majoring on either post-graduate success or prospective student recruiting and design a system that situates double-majoring to hit that sweet spot for graduates and prospective students.
Likewise, we could (and I would argue, should) design a new semester system that ensures gratifying future employment for all current faculty (as opposed to asking someone with one set of expertise and interests to spend all of their time doing something that has little to do with that expertise and interest). However, it seems to me that we might be missing something important if we assume, or assert, that we are not likely to achieve that goal of employment stability if we do not maintain historically similar proportions of double-majors distributed in historically similar ways.
Those of you who have explored the concept of design thinking know that one of its key elements is an openness to genuinely consider the widest possible range of options before beginning the process of narrowing toward a final product or concept. At Augustana we are trying to build something new, and we are trying to do it in ways that very few institutions have done before. Moreover, we aren’t building it from an infinite array of puzzle pieces; we are building it with the puzzle pieces that we already have. So it seems that we ought not box ourselves prematurely. Instead, we might genuinely help ourselves by opening our collective scope to every possibility that 1) gives our students the best chance for success, 2) gives us the best chance to recruit future students, AND 3) uses every current faculty member’s strengths to accomplish our mission in a new semester system.
Please don’t misunderstand me – I am NOT arguing against double-majors (on the contrary, I am intrigued by the idea). I’m only suggesting that, especially as we start to tackle complicated issues that tie into very real and human worries about the future, we are probably best positioned to succeed, both in process and in final product, to the degree that we directly address the genuine and legitimate concerns that keep us up at night. We are only as good as our people and our relationships with each other. I know we are capable of taking all of this into account as we proceed into the spring. I hope every one of you take some time to relax and enjoy the break between terms so that you can start the spring refreshed and fully able to tackle the complex decisions that we have before us.
Make it a good day,
Mark
Time to break out the nerves of steel
When I used to coach soccer, other coaches and I would sarcastically say that if you want to improve team chemistry, start winning. Of course we knew that petty disagreements and personal annoyances didn’t vanish just because your team got on a winning streak. But it was amazing to see how quickly those issues faded into the shadows when a team found themselves basking in a winner’s glow. Conversely, when that glow faded it was equally amazing to see how normally small things could almost instantaneously mushroom into team-wide drama that would suck the life out of the locker room.
Even though one might think that in order to win again we just needed to practice harder or find a little bit of luck, almost always the best way to get back to winning was to get the team chemistry right first. That meant deliberately refocusing everyone on being the best of teammates, despite the steamy magma of hot emotion that might be bubbling up on the inside. In the end, it always became about the choice to be the best of who we aspired to be while staring into the pale, heartless eyes of the persons we could so easily become.
You might think that I’m going to launch into a speech about American values, immigration, and refugees. But actually I’m thinking about the choices that face all of us at Augustana College as we start to sort through the more complicated parts of the design process in our conversion to semesters. Like a lot of complex organisms, a functioning educational environment (especially one that includes a residential component) is much more than a list of elements prioritized from most to least important. Instead, a functioning educational environment – especially one that maximizes its impact – is an ecosystem that thrives because of the relationships between elements rather than the elements themselves. It is the combination of relationships that maintain balance throughout the organism and give it the ability to adapt, survive, adjust, recover, and thrive. If one element dominates the organism, the rest of the elements will eventually die off, ultimately taking that dominant element down with them. But if all the elements foster a robust set of relationships that hold the whole thing together, the organism really does become greater than the sum of its parts.
Likewise, we are designing a new organism that is devoted to exceptional student learning and growth. Moreover, we have to design this organism so that each of the elements can thrive while gaining strength from (and giving strength to) each other. We give ourselves the best chance of getting it right if we keep the image of an ecosystem fresh in our minds and strive to design an ecosystem in which all of the relationships between elements perpetuate resilience and energy.
But in order to collaboratively build something so complex, we have to be transparent and choose to trust. And this is where we need to break out the nerves of steel. Because we all feel the pressure, the anxiety, the unknown, and the fear of that unknown. The danger, of course, is that in the midst of that pressure it would be easy, even human, to grab on to one element that represents certainty in the near-term and lose sight of 1) the relationships that sustain any given element (including the one you might currently be squeezing the air out of), and 2) the critical role of all of those relationships in sustaining the entire organism.
As we embark toward the most challenging parts of this semester conversion design, I hope we can find a way, especially when we feel the enormity of it all bearing down on us, to embody transparency and choose to trust. That will mean willingly deconstructing our deepest concerns, facing them openly, and straight-forwardly solving them together.
Think about where we were a year ago and where we are now. We’ve done a lot of impressive work that can’t be understated. (Based on the phone calls I’ve received from other institutions asking us how we are navigating the conversion to semesters, we might just be the golden child of organizational functionality!). Now, as the more complex challenges emerge and the pressure mounts, let’s remember what got us here, what will get us through this stretch of challenging decisions, and what will get us safely to the other side.
Make it a good day,
Mark