Author: markhsalisbury

Why should our seniors participate in the Wabash National Study?

When thing get really hectic, I have a hard time remembering what month it is.  Judging by the snow falling outside as a write this first column of the spring term, it’s not just me.  Fortunately, we all have our anchoring mechanisms – our teddy bear or our safe space that keeps us grounded.  For me, it’s the Wabash National Study senior data collection that will occur in March and April.  At long last, it’s time to find out from our seniors how their Augustana experience impacted their development on many of the primary intended outcomes of a liberal arts education.  (I know.  Own it!)

 

I believe that the data we gather from the Wabash National Study could be the most important data that Augustana has collected in its 150+ year history.  I’d like to give you three reasons why I make this claim, and three ways that I need your help.

 

First, the Wabash National Study measures individual gains across a range of specific outcomes.  Instead of taking a snapshot of a group of freshmen and a snapshot of a different group of seniors and assuming that those two sets of findings represent change over time, in this study we will have actually followed the same group of students from the first year to the fourth year.  Furthermore, instead of tracking only one outcome, this study tracks 15 different outcomes, allowing us to examine how gains on one outcome might relate to gains on another outcome.

 

Second, the Wabash National Study is the first and only study that allows us to figure out which student experiences significantly impact our students’ change on each outcome measure.  In other words, from this data we can determine which experiences improve gains, which experiences inhibit gains, and which experiences seem to have little educational impact. Furthermore, this data allows us to determine whether the gains we identify on each outcome are a function of pre-college characteristics (like intellectual aptitude) or a function of an experience that happened during college (like meaningful student-faculty interaction).  This gives us the kind of information on which we can more confidently base decisions about program design, college policies, and the way we link student experiences to optimize learning.

 

Third, as we continue to try to more fully embody a college that assesses itself based on what we do rather than what we have, this data can provide a foundation as we think about clearly articulating the kind of institution we want to be in the future and how we are best able to get there.  In the past decade, we have collected bits and pieces of this kind of data from NSSE, CLA, and various Teagle-sponsored studies – all important evidence on which we have made critical decisions that have improve the quality of the education we provide.  This time around, we will have all of that data in one study, allowing us to answer many of the questions that we need to answer now; questions that have previously been exceedingly difficult answer because the applicable data was scattered across different, often incompatible, studies.

 

But just because we are going to try to collect this data from our seniors over the next two months doesn’t mean we automatically get to have our cake and eat it, too.  Our seniors have to volunteer to provide this data.  Although we have some pretty decent incentives ($25 gift cards to the book store and group incentives for some student groups), this thing could be a monumental belly flop if no one shows up to fill out our surveys.  This brings me to how you can help.

 

1)      Make it your mission to tell every senior with whom you interact to participate in the survey.  We are going to invite them by email, announce this study at various student venues, and hopefully have some articles in the Observer.  But the students need to be encouraged to participate at every turn.

2)      Tell them why they should participate!  It’s not enough to ask them to do it.  They need to know that this will fundamentally shape the way that we construct Augustana College for the next generation of students.  They can play a massive role in that effort just by showing up and filling out some surveys.  Oh, if the rest of life was so easy!

3)      Remind them to participate.  We will have four different opportunities for seniors to provide data.  We will give $25 gift cards to the first 100 students at each session – so if they all wait to participate, most of them won’t get the incentives we would like to give them.  The dates, times, and locations of these sessions are:

 

  1. Monday, March 12, 6-8:30 PM in Science 102
  2. Monday, March 26, 6-8:30 PM in Olin Auditorium
  3. Thursday, March 29, 6-8:30 PM in Science 102
  4. Thursday, April 26, 10:30 AM – 12:30 PM in John Deere Lecture Hall

 

Thank you so much for your help.  Just to let you know ahead of time, I’m not going to shut up about this data collection effort until we give away all of the gift cards or we run out of data collection dates.  Yes, it’s that important.

 

Make it a good day,

 

Mark

A positivity distraction

As you slog your way through the snow and the grading and the (hypothetical) curriculum reconstruction this week, I hope you will take a moment to wire your brain for positive thoughts.  I don’t have much to say today – I’m feeling a little beat down myself – but I watched this TED talk last night and it was just the tidbit I needed to get my head straight.

 

Make it a good day (sometimes I’m really am talking to myself),

 

Mark

Understanding the “new” learning outcomes of a college education

At the Augustana Board Retreat a couple of weeks ago, Allen Bertsche (Director of International Programs) and I hosted a discussion with members of the Board, administrators, and faculty about a fundamental shift that has occurred in higher education over the past several decades.  While a college education used to be primarily about acquiring content knowledge, today the most important outcomes of a college education are a broad range of complex cognitive, psychosocial, and interpersonal skills and dispositions. These outcomes transcend a student’s major choice and are applicable in every facet of life.  In short, although content is still necessary, it is no longer sufficient.  In recent years Augustana has identified outcomes like critical thinking, collaborative leadership, and information literacy as fundamental skills that every student should develop before graduation.

 

During our conversation at the Board Retreat, Kent Barnds (Vice President of Enrollment, Communications, and Planning) pointed out that, while some of us might grasp the ramifications of this shift, perspective students and their families are still firmly entrenched in the belief that content acquisition is the primary goal of a college education.  In their minds, a college’s value is directly related to the amount of content knowledge it can deliver to its students.  As many of you know, when prospective students and families visit, they often ask about opportunities to obtain multiple majors while participating in a host of experiences.  By comparison, they rarely ask about the exact process by which we develop critical thinking or cross-cultural skills in students.

 

I think it would do us some good to consider what the current calendar discussion looks like to those who believe that the cost of tuition primarily buys access to content knowledge.  The students quoted in the most recent Observer about the 4-1-4 calendar discussion exemplify this perspective.  Their rationale for keeping the trimester system is clearly about maximizing content acquisition – more total courses required for graduation equals more total content acquired, and shorter trimesters allow students to minimize the time spent acquiring content that they don’t need, don’t like, or don’t want.  With tuition and fees set well over $40,000 next year, it’s not hard to see their concerns.

 

Now please don’t misunderstand me – I am much more interested in what we do within the calendar we choose than whether we continue on trimesters or move to semesters.  Nor am I suggesting that student opinions should or should not influence this discussion.  But if we’re trying to have a conversation about student learning – with or without students – and we don’t share a common definition of the term, then we are likely doomed to talk right past each other and miss a real opportunity to meaningfully improve what we do regardless of whether or not the faculty votes to alter the calendar.  On the other hand, if we can more clearly spell out for students, parents, (and ourselves) what we mean when we talk about “student learning” and why our focus on complex skills and outcomes is better suited to prepare students for life after graduation, not only might it temper the tensions that seem to be bubbling up among our students, it might also allow us to help them more intentionally calibrate the relationship between their current activities and obligations and their post-graduate aspirations.

 

So no matter where you sit on the semester/trimester debate, and no matter what you think about the shift in emphasis from content acquisition to the development of skills and outcomes, I would respectfully suggest that we need to better understand the presumptions that undergird each assertion in the context of the calendar discussion. In my humble opinion, as Desi used to say to Lucy, we still “got some ‘splainin’ to do.”

 

Make it a good day,

 

Mark

Teaching, learning, and sleep

It’s that time of the term again – lots to do and not nearly enough time to do it.  Especially for students, at this time of the term the amount of time needed to meet academic and co-curricular obligations thunders past critical mass like a semi-truck blowing by a hitchhiker. Pretty soon basic health and hygiene behaviors get pushed to the side and our kids are riding a rollercoaster of super-sized energy drinks, junk food, and far too little sleep.

 

One of the outcomes that the Wabash National Study allows us to track is health behaviors.  This set of variables includes measures of exercising, binge drinking, smoking, and sleep deprivation.  Since the end of the term is often a time when students look like they are groggily stumbling toward the finish line, I thought we’d examine students’ reports of sleep deprivation over the first year and see if anything faculty and staff do might impact it one way or the other.

 

Sleep issues are deceptively complicated because there are lots of reasons why someone might not get enough sleep.  It might be too much homework all at once.  Or it might be stress about something completely unrelated to school.  Since we don’t have the breadth of variables in the Wabash data to get at all of the potentially influential stress related issues, I tried to focus this analysis on the factors that might shape students’ allocation of time and thus influence the frequency of feeling sleep-deprived.

 

First of all, we found that average amount of times during a week that students’ felt sleep deprived increased from the beginning to the end of the first year – an increase that proved to be statistically significant.  Now by itself, that isn’t much of a surprise – and many of you might say that this is as it should be.  So the next question is:  What are the factors that are uniquely influencing this change?

 

(I’m glad to send you the full list of variables we examined and the output file if anyone is interested – Regression Modeling Geeks Unite!)

 

After accounting for basic demographic characteristics and pre-college behaviors, we found that both the number of hours students reported studying per week and the number of hours students spent in co-curricular activities positively influenced an increase in sleep deprivation.  However, after adding greek membership into the mix, the impact of co-curricular involvement evaporated and was replaced by a similar sized impact of greek affiliation.

 

While that finding is interesting in its own right, I wanted to know more.  Is there anything about the way that we interact with students that might also impact this increase in sleep deprivation?  Interestingly, we found evidence that faculty teaching behaviors might mitigate this apparent increase.  As our students’ reports of experiencing instructional organization and clarity increased, the increase in sleep deprivation during the freshman year was REDUCED.  In other words, the degree to which students report faculty are clear and organized in teaching their courses appears to influence healthier sleeping behaviors in our students.  Moreover, I tested this analysis with the full Wabash data set (about 3000 students from 19 schools) and again, the impact of instructional clarity and organization was significant in reducing the increase in sleep deprivation over the first year.

 

I’m not sure I’m ready to suggest a direct causal relationship – but I think it’s worth considering the legitimate possibility that the way we teach and organize our courses might indeed play an important role in fostering a positive learning environment beyond the academic sphere.

 

zzzzzzzzz . . . (make it a good day . . . shhhh),

 

Mark

The educational benefits of reflection

If there was a magic potion that turned glum, unkempt, “I dare you to learn me some teachin,” students into captivated, self-directed, and perpetually inquisitive knowledge hounds, we’d all want to know about it, right?  Of course, student development does quite work that way.  And yet, there are specific pedagogical exercises that seem to be pretty influential in our first year students’ growth – for those students lucky enough to encounter in it.

 

One such exercise is reflective learning.  Although we often think of reflection as something that might be found in a journal assignment (or a mirror), it can happen in lots of settings and formats.  And while some criticize reflection as little more than rationalized navel gazing, there is plenty of evidence to suggest that reflection – when facilitated well – can be a powerful learning tool.  So I decided to see if reflective learning had any impact on the educational development of our first-year students who participated in the Wabash National Study in 2008.  After all, since many of the “high-impact experiences” we often talk about (e.g., study abroad, internships) are rarely accessible to freshmen, we need to know the kinds of learning experiences that can make the first year of college more than “just a year of waiting to get to the good stuff.”

 

The Wabash National Study accounted for reflective learning by combining three questions.  They asked, “During the current school year,

 

  • how often did you examine the strengths and weaknesses of your own views on a topic or issue?”
  • how often did you try to better understand someone else’s views by imagining how an issue looks from his or her perspective?”
  • how often did you learn something that changed the way you understand an issue or concept?”

 

Available responses included 1=never, 2=sometimes, 3=often, and 4=very often.

 

It turns out that the frequency of reflective learning reported by students at the end of their first year significantly influenced increases in Attitudes toward Literacy, Intellectual Curiosity, Intercultural Competence, Psychological Well Being, Socially Responsible Leadership, and Civic Engagement.  These increases continued to be true even after accounting for differences in incoming ACT score, sex, gender, socio-economic status, instructional clarity and organization, integrative learning, and higher order thinking.

 

This finding is even more interesting because the average scores on each of these outcomes didn’t change during the first year.  In other words, while there were enough students who either regressed, increased, or stayed the same on each of these outcomes to keep the overall averages static, the students who made gains on these outcomes seem to have (at least) one thing in common – increased reflective learning experiences.

Coincidentally (ok, not really), on Wednesday of this week (1/25) at 4 PM, Kristin Douglas, Rebecca Cook, and Stephanie Fuhr will host a presentation in the Treadway Library about the ways that Biology has successfully infused reflection into the major.  They’ll talk about the challenges and successes they have seen and hopefully give you some ideas of ways that reflective learning might work in your course or major.  In addition, Ryan White, Director of the Center for Vocational Reflection, is offering a one-time stipend to help faculty integrate reflection into their courses.  If I weren’t on an airplane on Wednesday, I’d be there.

 

I hope you’ll attend and consider finding ways to infuse this “magic potion” into your teaching.  Maybe it’s not really an instant elixir – think of it a time-release capsule.

 

 

Make it a good day,

 

Mark

Graduating our lower income students

We knew it was coming – despite hoping against hope that we might have avoided winter this year. But even as some of us were shoveling out and bundling up, the warm couple of days last week had already turned my thoughts to spring and all that comes with the end of the academic year.  Of course, this inevitably brings up the topic of graduation – a primary measure of our success as an institution.

 

For many years, colleges have tracked graduation rates – the proportion of students from a given incoming cohort that actually graduate from that college.  Although the national conversation about graduation rates generally references 6-year rates, for most private liberal arts colleges the 4-year graduation rate matters most because 1) the curriculum is explicitly set up to graduate students in four years, and 2) the cost of tuition at private colleges makes finishing in four years particularly preferable to students and their families.  In more recent years, many institutions have figured out that the overall graduation rate isn’t really as important as the graduation rates of student subgroups that are more likely to struggle and/or withdraw from college.

 

As the cost of higher education has increased, many have worried about the effect of this trend on college access for students from lower socio-economic status backgrounds.  But another question is also important – for the students from lower socio-economic backgrounds who acquire access to higher education, do they graduate at the same rate as students from higher socio-economic backgrounds?

 

Although the answer is probably a complicated one, we are able to examine graduation rates across federal financial aid categories and find out if there are systematic differences for students entering Augustana College.  Although socio-economic status (SES) is a complex issue, federal financial aid can roughly approximate three categories of students.  The most privileged students would be those who don’t qualify for any federal financial aid.  The students with some need qualify for a subsidized Stafford Loan, but no grant aid.  And the students for whom paying for college is the biggest challenge qualify for a Pell Grant.  Based on these categories, we can test the graduation rates for each group.

 

The most recent cohort of students to finish four years at Augustana entered in the fall of 2007.  The 4-year graduation rate for these students across these three SES groups is portrayed below.

 

Students with neither Stafford or Pell

79%
Students receiving a Stafford Loan

76%
Students receiving a Pell Grant

62%

 

Clearly, something is going on for the students who received a Pell Grant that differs from those who did not.  But what?  Maybe they initially thought they could cobble together the money to come to Augustana, but then found out they just couldn’t make it work.  Maybe they decided they weren’t getting enough out of the Augustana experience to merit the costs – especially in the context of their financial situation and economic collapse in 2008.  Or maybe the issue wasn’t so much about money as it was about a sense of belonging on campus among the much larger proportion of students who don’t come from such economically disadvantaged backgrounds.  Or maybe it was a combination of factors.

 

I don’t begin to know the answers to these questions.  But I think this data suggests that we had better find out.

 

Make it a good day,

 

Mark

What does Finland have to teach us about assessment?

Welcome back!  During the break I hope you were able to enjoy some time with loved ones and (or) recharge your intellectual batteries.  I will admit that I spent part of the break embracing my inner geek, reading about the amazing improvements in Finland’s student achievement scores since they instituted a new national education policy in the 1970s.  Previously, Finland had been decidedly average.  Today, their scores are consistently among the best in the world – particularly in reading and science.  As a result, the U.S. and the U.K. – countries with substantially lower scores – are very interested in finding out what might be driving this educational success story.

 

The point of my column this week isn’t to delve into the details of Finland’s success, but rather to consider one aspect of Finland’s approach that I think is particularly applicable to our current conversation about educational outcomes and improved student learning.  So here are a few links if you are interested in reading more about Finland educational success or about the exam that is used to measure student achievement.  Instead.

 

“Accountability is something that is left when responsibility has been subtracted.”

 

If you’ve already read the Atlantic Monthly article I hyperlinked above, you know that this statement is attributed to Pasi Sahlberg, an individual deeply involved in Finland’s educational transformation.  The principle to which he refers asserts that unless an educational endeavor is intentionally designed to produce a specific outcome, it is difficult to argue that gains on that outcome are entirely attributable to the educational endeavor in question.  However, as society has increasingly demanded that education prove its worth, it is deceptively easy to start by testing for an educational effect without ever asking whether the experience is really designed to best produce it.  To make matters worse, then we mandate improvement without addressing the systematic dysfunction that created the problem in the first place.

 

My sense of Augustana’s evolution regarding student learning outcomes is that we are in the midst of a process to make explicit what we have long valued implicitly.  We are trying to be clearer about what we want our students to learn, be more transparent about those efforts, and maximize the educational quality we provide.  In this context, Sahlberg’s comment on accountability and responsibility struck me in two ways . . .

 

First, the process of identifying outcomes and designing an educational program to meet those outcomes requires us to take full responsibility for the design of the program we are delivering.  When something is repeatedly greater than the sum of its parts, it isn’t just a happy accident.  Designing a successful educational program is more than just making pieces fit together – it’s constructing the pieces so that they fit together.

 

Second, just because an outcome idea sounds like it might be valid doesn’t make it so.  But in the absence of anything else, accountability measures that mean very little can all too easily become drivers of institutional policy – sometimes to the detriment of student learning.  However, the inverse can also be true.  An institution that takes full responsibility for the design of its educational programs and the system within which they exist will likely far exceed typical accountability standards because such an institution can make coherent, empirically-grounded, and compelling arguments for why it does what it does; arguments that quickly evaporate when a pre-packaged accountability measure is hurriedly slapped onto the back end of an educational process.

 

So I’d like to close by suggesting that we consider the statement quoted above in this way: If we take explicit responsibility for student learning and the design of the educational programs we provide, demonstrating our accountability – to our students or our accreditors – will be relatively easy by comparison.

 

Make it a good day,

 

Mark

Building on our advising success

A week or so ago, I was talking with one faculty member about the information that we now receive as a result of the IDEA student ratings of instruction reports.  During that conversation, our focus kept drifting toward the recommendations for improvement.  Although this wasn’t necessarily a bad thing, we began to notice together how evidence of success can be just as important.  Not only can it help confirm that our efforts are bearing some fruit, but it can also remind us to continue to “play to our strengths.”

With this in mind, I’d like to highlight some findings about our students’ experience with advising that I believe add to our rationale for considering ways in which we might further improve our students advising experience.

Every year we ask our graduating seniors about their satisfaction with advising overall and in advising in the primary major on a scale of 1 to 5.  Below are the average scores from last spring (2011).

 

Average Score

Standard Deviation
Overall Advising

4.102

1.176
Major Advising

3.701

1.198

 

It turns out that there are two interesting tidbits in this data.  First, the difference between satisfaction with overall advising and major advising is statistically significant – meaning that the difference between the two average scores is not attributable to chance.  Second, the difference in the standard deviation (the average gap between each student response to these questions and the overall average response) suggests that there is more variability of experiences in major advising than overall advising.

At this point you might be thinking, “Mark, that is a strange interpretation of playing to our strengths!”  To which I say – hold on for just a second.  Remember that our response scale of 1 – 5 defines “4” as satisfied . . . which means that on average both groups are relatively satisfied.  If you compare these numbers to our NSSE data on advising, it turns out that our students respond much higher than the NSSE average – both for freshmen and for seniors.

In the context of these two data points, I am most interested in asking whether we might have other data that suggests a relative strength in advising that we might expand upon to both improve our students’ average major advising score AND tighten the variability in across that experience.

I think we might have just such a data point in another section of the NSSE survey.  Students are asked earlier in the survey how often they talk about career plans with a faculty member or advisor (1=never, 2=sometimes, 3=often, 4=very often).

 

Augustana

Comparable Liberal Arts Colleges
Freshmen

2.42***

2.23
Seniors

2.92***

2.63

 

It appears that our faculty and staff advisors are already talking with advisees about their career plans substantially more often than advisors and faculty at comparable liberal arts colleges.  Since we know from our self-study of advising that this efforts makes a substantial difference in the degree to which our students feel certain about their post-graduate plans, it appears to me that this is something that we are doing very well and could build upon to strengthen our students advising experience in the major.

Have a wonderful week and a great holiday break.

Make it a good day.

Mark

One way to look at our students’ spiritual development

Last week Eboo Patel, founder of Interfaith Youth Core, spoke to many of us at either convocation or in a series of other meetings about the importance of embracing an inclusive tradition of faith – no matter the faith tradition we each might choose to follow.  His comments and questions spurred some intriguing conversation that got me wondering about the degree to which our students develop a more nuanced notion of their own spirituality during their time at Augustana so that they might be aware enough to make such a choice within their own faith tradition.

Before examining our data to see what we might have that begins to address this question, we have to accept a nagging ambiguity (and this time, it’s not all that delicious).  The term “spirituality” isn’t so easily defined.  Instead, it’s a term that tends to mean different things to different people.  For some, it’s inexorably tied to religious faith, maybe even dogma.  For others, it simply applies to an acceptance of things beyond our current understanding.  For most, it’s somewhere in between.

This makes the life of a number cruncher a little messy.  On the one hand, it turns out that we have two interesting data points on this question of spirituality.  The NSSE survey asks students:

1)     During the current school year, about how often have you participated in activities to enhance your spirituality (worship, meditation, prayer, etc.)?  The response options are 1=never, 2=sometimes, 3=often, 4=very often

 

Freshmen – 2.01; Seniors – 2.01

(both responses are significantly lower than comparable liberal arts colleges)

 

2)     To what extent has your experience at this institution helped you develop a deepened sense of spirituality?

The response options are 1=very little, 2=some, 3=quite a bit, 4=very much

 

Freshmen – 2.27; Seniors – 1.99

(both responses are significantly lower than comparable liberal arts colleges)

 

On the other hand, both questions focus on the word “spirituality,” suggesting that student responses could differ based upon their conceptualization of this term.  Nonetheless, while we might not have a precise finding from the perspective of a social scientist, we definitely have something that – from the perspective of creating optimal learning conditions and assessing student growth – begs for further inquiry.

The responses to these two questions are quite interesting to me.  In my mind they triangulate with the larger narrative we hold that sees our students as strivers.  They tend to be focused on getting a job or getting into graduate school and involving themselves in every possible activity that will help them achieve their goal.  Yet, this breakneck pace can all too often occur at the expense of our responsibility as educators to develop the whole person.

If we want our students to embrace an inclusive perspective on their own faith tradition – be it Christian, Muslim, Jewish, Hindu, Buddhist, Mormon, or Humanist – while also embracing a commitment to social justice rather (than an apathetic descent into rationalized relativism), then I suggest that we would do well to dig deeper into the following three questions.

  • Why are our students relatively less engaged in their own spiritual development than students at comparable liberal arts colleges (however students choose to define spirituality)?
  • Why do our students think that their experience at Augustana has contributed relatively less to the development of their own sense of spirituality than students at comparable liberal arts colleges?
  • Are the answers to these first two questions related?

 

Make it a good day.

Mark

One course just won’t do it

From time to time, Augustana lets me out of my little cave so that I can attend a conference related to higher education research or assessment of student learning outcomes.   A few weeks ago, a paper was presented at the Association for the Study of Higher Education (ASHE) that I found fascinating and particularly germane to many of the conversations we have at Augustana about the effects of particular curricular emphases on broader student learning outcomes.

This particular paper examined the influence of required diversity courses on students’ inclination toward civic engagement.  At many institutions the general education curricula is organized around a series of categories from which students choose one or two courses to meet the institution’s requirements.  This paper hypothesized that perhaps one course on diversity issues was not enough to influence substantive, lasting learning.  The authors examined data from about 500 students, gathered at the beginning of the first year and at the end of the fourth year.  The authors also had access to student transcripts that allowed them to identify which courses the students took to fulfill their general education requirements.

Students in this study had two options in fulfilling the diversity requirement.  They could take a domestic diversity course or a global diversity course.  In some cases, students took both – especially since some courses within the diversity category also fulfilled other requirements necessary for graduation.  Thus, the researchers could test the effect of taking one domestic diversity course, one global diversity course, or both courses on students’ gains in attitudes toward civic engagement.

The study found that the only students who made substantive gains in an inclination toward civic engagement were those who took both the domestic and global diversity courses.  Conversely, students who took only one course focused on either domestic or global diversity had not unique effect on attitudinal gains.

The take away from this paper, and the discussion that followed really honed in on the tendency for us to think that substantive learning can be accomplished by a single course – a “check the box” approach.   Of course, as we think about designing a new curriculum these findings might be useful to consider.  More broadly, however, I would suggest that this paper reinforces the idea that substantive learning is a function of a series of related experiences rather than any one experience.   We are the ones who can help our students engage in related experiences and help to point out those connections.

Make it a good day.

Mark