One claim (given as advice) that I’ve heard ever since I was a plump, pimple-faced college freshman is that taking a heavier academic load in a given term (no matter the calendar) increases the likelihood that one’s grades will suffer.  It seems intuitive:

more classes (and thus more homework) / the same number of hours in a week =          less study time to allocate to each class and therefore potentially lower grades

At Augustana we are understandably sympathetic to this concern because of the degree to which we often try to pack an extensive amount of learning into our shortened academic terms while maintaining the comparatively higher number of hours in class that we require for a credit hour.  Many of us can weave a harrowing tale of students’ swamped by the academic requirements of a four-course term, but it would be wise to wonder whether our individual anecdotes actually represent the experiences of most students.  So a few weeks ago, we decided to empirically examine this wide-spread belief.  Since this concern is often raised by faculty and administrators when discussing the merits of potential policy changes, this hypothesis seems a compelling argument to test.

So we examined our students’ term-by-term GPAs over the last three years (nine terms from the fall of 2009 to the spring of 2012), comparing the GPAs of students who attempted between 8 and 11 credits – less than four three-credit courses – with the GPAs of students who attempted 12 or more credits – four three-credit courses or more.  Moreover, we conducted this analysis in two stages.  In the first analysis we only tested whether the number of credits attempted significantly impacted students’ end-of-term GPA.  In our second analysis, we accounted for two potentially confounding factors: (1) a student’s pre-college academic ability, and (2) a student’s year in school, to make sure that any statistically significant effect we might find wasn’t a function of another plausible explanation.

Our first set of analyses surprised us.  Because we thought we’d find one of two possible outcomes – either the reigning hypothesis would hold true or we would find no significant difference between the two groups.  So we were pretty shocked when we found that in every academic term from fall of 2009 through spring of 2012, students who attempted 12 or more credits, on average, earned a HIGHER GPA (between .05 and .12 points) than those who attempted 8-11 credits.  Huh?

In the second stage of our analyses, we held constant students’ incoming ACT score and year in school.  At this point, I was sure that we’d end up with insignificant findings.  Instead, the finding from our first analyses held throughout.  Not only do students who are taking a heavier load not suffer in terms of a lower GPA for that term, but their GPAs (no matter the year in school or their incoming academic ability) were marginally higher.  Huh.

So what does this mean?  Certainly, the obligations of a heavier credit load can adversely affect a student’s stress level or sleep patterns even if they don’t necessarily impact grades.  And unfortunately, the only data we have readily accessible is term-by-term GPA and term-by term-credits attempted.  In addition, the findings might be different if we looked at each student’s term-by-term GPAs longitudinally instead of comparing all students cross-sectionally across a given term.  However, students must pay overage fees to take more than 33 credits a year, so the chances of a substantial portion of students consistently taking 12 or more credits, earning strong grades, and compromising this finding is pretty low.  In the end it seems that a heavier credit load doesn’t impact students’ grades in the way that we might have thought.

I wonder if this finding exemplifies a disconnect between the way that we tend to think students engage college and the way that they actually manage their college experience.  For years we have lamented the difference between the amount of time we think that our students should study and the amount of time our survey data suggests that they actually study.  Yet these same students graduate with an average GPA of 3.3, an increasing number of them graduate with honors, and many of them go on to successful, challenging professional lives.  And lest some might want to resurrect the allegation that this is further evidence of the corrosive effects of grade inflation, (1) we have multiple sources of evidence that suggest our students make more than respectable gains on various learning outcomes, and (2) we tested the grade inflation claim last year and found it to be explained by increases in our students’ incoming ACT scores over the past two decades.

I wonder if this is an indication that students are more capable of prioritizing their time and effort than we might give them credit sometimes.  And while I’m not suggesting that this finding should be used to require that they take a heavier academic load every term, I wonder if we might take our feet off of the academic gas pedal a little too easily sometimes – which is easy to do in the face of a roomful of scowling students to whom you have just assigned an additional assignment.  One student experience measured in the Wabash National Study that was particularly predictive of learning gains was the degree to which students were challenged to work harder than they thought they could to meet their instructor’s expectations.  Our finding regarding grades and course load suggests a similar result.  If we push our students, they might surprise us.

Make it a good day,

Mark

Listening to some faculty talk, you’d think that fraternities and sororities at Augustana are a deadly concoction of Sodom and Gomorrah, Mardi Gras, Las Vegas, and Carnival, whipped up in a blender and chugged through a fire hose from a second story beer bong.   Yet, we all know of greek organizations – at Augustana and elsewhere – that make important contributions to the local community and the development of their members.  Thankfully, we don’t have to settle for dueling anecdotes.  We have plenty of data on students in Augustana’s greek organizations that allow us to test this clash of narratives.  So, since I’m on a bit of a mythbuster’s kick lately . . . let’s see what we can find out.

When the entering class of 2008 arrived at Augustana, little did they know that they would be studied like no class before.  They provided data three times as a part of the Wabash National Study (beginning of freshman year, end of freshman year, and end of senior year).  They were also the first class to complete the new senior survey in the spring of 2012.

From the data gathered at the end of the freshmen year (spring, 2009), we found one set of troubling results among first year greek members.  Freshmen who joined greek organizations reported larger increases than their independent (non-greek member) peers on three items during the first year.

• The number of times in a week that they drank alcohol
• The number of times in a week that they had five or more alcoholic drinks
• The number of days in the week that they felt sleep deprived

In addition, greek members, on average, earned a lower spring GPA – even after accounting for students’ incoming ACT score and academic motivation.  Unsurprisingly, being male exacerbated each of these differences, while being female minimized them.  Interestingly, despite these potentially negative effects, greek membership did not decrease the likelihood of retention, probably because students don’t join greek organizations until the spring term, and the primary driver of persistence or withdrawal – academic performance – has already culled the herd during the previous winter and fall terms.

Fast-forward to the end of the senior year.  At this point, what initially seemed a more negative picture becomes more complicated.  While greek members’ average GPA still trail that of non-greek members, the gap noted in the spring of the first year has shrunk by about 25%.  Again, being female mitigates further, likely making the difference in average GPA between female greek and non-greek members insignificant.

However, in numerous cases greek students’ scores on several senior survey items suggests that this experience provided some important benefits.  On average, greek members’ responded more positively (defined by differences that proved statistically significant) to these statements:

• My co-curricular experiences provided numerous opportunities to interact with students who differed from me in race/ethnicity, sexual orientation, religious beliefs, or social/political values.
• My co-curricular involvement helped me build a network of healthy lasting friendships.
• My co-curricular involvement helped me develop a better understanding of my leadership skills.
• I felt a strong sense of belonging on campus.
• The college emphasized an atmosphere of ethnic and cross-cultural understanding.
• Augustana faculty and staff welcomed student input on institutional policy and handbook decisions.
• If you could relieve your college decision would you choose Augustana again?

Taken together, these findings spell out much of the good and the bad of greek life.  On one hand, during the first year it appears that some behaviors emerge among greeks that could – and sometimes do – negatively impact students’ success.  On the other hand, by the time this group of students graduates, at least one of those deficits has been legitimately reduced, and the educational efforts of the college – particularly on the co-curricular side – appear to have produced a series of benefits that match our own educational intentions.

Of course, one important question – and a longstanding one – is how we might eliminate the bad without losing the good.  Our student affairs staff continually works to counter the impact of pledging on student success, even in the face of stiff pushback from many greek members and alumni.  Might there be a role for faculty to play in this endeavor?  Probably.  Does that role include railing against a stereotype of greeks that actually perpetuates a stereotype of faculty among students and, in so doing undermines the very trust necessary to influence students’ behavior outside of class?  Probably not.

But the question that jumps out at me is slightly different.  While it’s great to see graduating seniors from greek organizations respond so positively to all of these questions, should we actually be celebrating this?  What is it about NOT belonging to greek organizations that produces systematically lower scores on so many important markers of the college experience we are trying to deliver?  For example, I’m not comfortable with finding that the greek members’ sense of belonging on campus score was more than half a point higher than non-greek members (4.26 vs. 3.71); not because I begrudge greek organizations, but because I’m not sure I see a compelling reason for greek membership on our campus to produce such a stark difference.

It’s easy to point to anecdotes of the college experience at its best; and we have many wonderful tales of students – greek and non-greek – who have changed fundamentally during their four years at Augustana.  But as I look at these findings, my concern tends toward the students who experience less than our best.  I’d be curious to figure out what we might do to minimize, or even eliminate, the statistically significant differences between greek and non-greek members across all of these senior survey experience questions.

Answers?  You wanted answers?  Oh, grasshopper . . .

Have a great Homecoming week – and let’s not leave anyone on the outside looking in.

Make it a good day,

Mark

As much as I try to be a kind, sensitive, and empathetic institutional researcher (group hugs every fifth Tuesday – no, not really!), I can’t resist salivating just a little bit whenever word of a new uber-explanatory claim pops up on my radar.  Part of my interest comes merely from a persistent drive to apply evidence to better understand what we do.  Sometimes, we make decisions that produce unintended consequences – and many times the impact of those decisions rises to the surface inductively, through the observations of some who, thankfully, are uniquely predisposed to see it.  However – and I fully own up to my dark side here – the chance to test a claim that has already gotten itself a bandwagon, a theme song, and the specter of pitchforks and torches storming the Bastille is an institutional researcher’s dream chance to “speak truth to power.”  It’s bratwurst to a Bear’s fan, grog to a Viking, a soy latte to an NPR member . . . you get the picture.

For many, the recent decision to merge the German and Scandinavian programs has felt like another body blow to the core values on which Augustana was founded.  Moreover, this decision all too easily feeds into a larger narrative that Augustana, like many traditional liberal arts colleges before it, has long since abandoned its commitment to the liberal arts even as it has disingenuously held on to the relative prestige of claiming to be something that it is not.

So . . . have we gutted our commitment to the liberal arts?  I purposefully choose this inflammatory language because it is exactly the wording that was used when the claim was made to me – complete with raised intonation and eyebrows.  While there are many ways to unpack this question; I’m writing a blog, not a book.  However, there are a couple of ways that we might examine our data to test this claim.  To that end, I’d like to introduce a couple of data points and one observation that might flesh out this story just a little bit.

One way that an institution might shift its commitment away from the liberal arts would be to move faculty positions away from core liberal arts disciplines like the humanities, foreign languages, and fine arts and add faculty lines to new or existing pre-professional programs.  While this by no means should be consider “smoking gun” evidence, if this were indeed the case, it would provide strong evidence to support the claim that Augustana had given up its commitment to the liberal arts.

So I decided to look for any evidence of a shift in faculty distribution over the past ten years. (Whether we should have gone back further to the late sixties or early seventies is an entirely valid critique).  Nonetheless, we started by building a baseline from 2001.  Thanks to Sarah Horowitz and Jamie Nelson in Special Collections, we tracked down a 2000-01 college directory and manually counted the number of faculty in each discipline.  As best as we can tell (it’s possible that some faculty were not listed in the directory for some reason), there were 78 faculty FTE (full time equivalent) employed by Augustana in the humanities, foreign languages, and fine arts ten years ago.  To put that in context, these 78 faculty FTE made up 49.6% of the 157 total faculty FTE.

So how does the 2000-01 distribution compare to today?  Last year, 2011-12, 114 faculty FTE were employed in humanities, foreign languages, and fine arts disciplines – 53.3% of our 214 total faculty FTE.  In the particular case of foreign languages, in 2000-01 there were 18 faculty FTE teaching in foreign language departments.  In 2011-12, there were 20.33 faculty FTE teaching in foreign language departments (we included classics in this analysis to be sure that Latin and Greek weren’t left out).

This evidence hardly supports the assertion that Augustana is gutting the liberal arts.  Just as a reminder, I am not suggesting that this is “smoking gun” evidence to dismiss the aforementioned claim. There might be evidence that other academic departments have lost positions to the pre-professional programs or that the relative distribution of full-time and part-time instructors has shift away from the core liberal arts disciplines;  although a cursory glance suggests to me that neither of these possibilities are likely.  So, at least in terms of overall faculty distribution in the traditional liberal arts, the trend over the last ten years suggests an increased investment in the most traditional liberal arts disciplines.

But this data doesn’t mean that there hasn’t been a shift in students’ academic behavior patterns that might translate into a different distribution of majors and minors.  In this context, there certainly might be some perceived winners and losers.  Our institutional data does show some changes in student academic interests over ten years, but the totality of these shifts merely complicates the story.  While the proportion of students declaring their “primary” major in the humanities has declined, the proportion of students declaring a “secondary” major or minor in the humanities has remained strong and maybe even ticked up slightly.  Some of this is due to an overall increase in the number of second majors and additional minors that students now obtain.  So even thought this data might reflect a modest shift in student priorities, its a long way from suggesting that the college is gutting the liberal arts.

So where does this leave us?  That isn’t my question to answer.  My goal here was only to test the veracity of a claim that seems to be a popular rallying cry in some circles at the moment.  Based on this evidence, and if the degree to which our investment in and distribution of faculty lines across the college represents our educational philosophy, it’s pretty hard to make the case that Augustana has abandoned its commitment to the liberal arts.

However, this evidence doesn’t address the question of whether or not our collective emphasis on an interdisciplinary, liberal arts education has waned in the face of increasingly siloed major requirements, a growing belief in the perceived value of a double major and/or a second minor, and institutional policies that waive course requirements fundamental to the liberal arts (e.g., foreign language competency).  But that conversation is a very different one – one that probably involves an examination of our espoused values, a hard look at the ramifications of our actual curricular and co-curricular policies, and a mirror.

Make it a good day,

Mark

Faculty often cringe when students ask, “what do I have to do to get an “A” on this assignment?”  For most educators, this question feels more like an unsolicited back alley proposition than a genuine expression of intellectual curiosity.

Yet from the student’s perspective, grades may represent a very different kind of negotiation.  Not only have grades dictated their access to future educational opportunities, extra-curricular experiences, and sometimes even cash(!) since elementary school, but the categories of “A” student, “B” student, and “C” student have all too often come to represent individual worth and long-term potential – not just the quality of one’s work on a particular assignment.  Sadly, we’ve done a pretty good job of validating this conception.  Remember the “My kid is an honor student at ____ school” bumper stickers that still adorn many a late model mini-van or SUV?

Luckily, disentangling the relationship between our students’ perception of grades and their motivational orientations can be approached as an empirical question.  Last year we began a four-year study of the experiences that shape our students’ intrinsic motivation.  As a part of this study, we included a measure of extrinsic motivational orientation and a question that asked students to indicate the importance they place on getting good grades.

This summer, we tested the relationship between extrinsic motivation and the importance of getting good grades at the end of the first year.  We assumed we’d find a significant relationship between these two variables.  So we were quite surprised to find no significant correlation between extrinsic motivation and importance of getting good grades.  However, we found a statistically significant positive – and moderately sized (.332) – correlation between students’ intrinsic motivational orientation and the importance of getting good grades.  Hmmm . . .

At the very least, this suggests that we might need to think more carefully about the assumptions we make when students ask how they can earn an ‘A’ from us.  One student inquiry about earning a high grade might be an indication of the degree to which we simply have not communicated our expectations for an assignment clearly.  Another inquiry might reflect the degree to which a student considers the entire educational enterprise to be about jumping through hoops and collecting credentials.  Still another inquiry might only mean that the student has too many irons in the fire and is simply triangulating their available time, the expectations they perceive that you hold, and the grade they can afford to live with.

There are two additional considerations about grading practices and their relationship to student motivation that are worth noting.  First, letter grades emerged during a time in which the learning expected of students was primarily about content knowledge.  But as content has shifted from an end to a means – with colleges now focused on developing more complex skills and dispositions in addition to content knowledge, we have done very little to think about whether the traditional metric for assessing student performance might benefit from some reconsideration.

In addition, at Augustana we don’t impose a single definition of what a grade represents.  Does an ‘A’ mean that a student has met an externally defined threshold of competence?   Or does it mean that a student has improve substantially over the course of a term?  Or is it some combination of the two that shifts as the course progresses?  Or maybe it should depend on the role of the course within the larger curriculum to determine whether grading should be about improvement or competence.

Faculty employ varying iterations of these conceptions across the array of courses that they offer, and all three approaches seem entirely appropriate for different situations.  But from the students’ perspective, unless they actually understand that there are different approaches to grading, and that these approaches can (and probably should) vary depending upon the course, they are likely to feel blindsided when the conception chosen by the instructor differs from that expected by the student.  Any one of us would likely be frustrated by such a realization, and in that moment it seems entirely reasonable to ask the question, “How DO I get an ‘A’ in this class?”  Moreover, I think we would have good reason to be offended if someone responded to our question by challenging our motives for learning.

Since a large proportion of our students understand the impact of grades on their future prospects for graduate school or the job market, it is likely that many place great importance on getting a high grade regardless of their motivational orientation.  So, it appears that maybe – just maybe – the implications of a student asking, “How do I get an ‘A’ on this paper?” are, let’s just say . . . complicated.

Make it a good day,

Mark

I suspect that almost everyone has heard the “it was better in the good ol’ days” claim …if we haven’t even used it ourselves from time to time.  I would suggest that we have an academic version of this claim at Augustana.  The claim argues that there has been substantial grade inflation over the past several decades.   Apparently, this claim has carried some weight over the years, because we have created multiple mechanisms to prevent grade inflation – or at least stem the tide.

Luckily this is a claim we can test.  But before looking at the data, let’s make sure we share an understanding of this claim.  An assertion of grade inflation boils down to two points.

1)      Grades have been creeping upward.

2)      This is because faculty have shifted expectations for performance downward.

Grade inflation doesn’t just make an observation about changes in GPA; it also attributes the change to the failure of colleagues to hold the line on academic rigor.  In the context of a small college, it’s sort of a less physically damaging version of a circular firing squad.

So, testing this claim turns into two questions.  First, have grades gone up over time? And second, can we conclusively attribute this change in GPA to faculty grading practices?

Have grades gone up over time?    

Yes.

From about 1991 to the present, the average GPA of each class went up by about .15 of a grade point, whether you look at each entering cohort’s end-of-year grades from the first year to fourth year or you look at each subsequent cohort’s end-of-year grades from 1991 to the 2010.

Can we conclusively attribute this change in GPA to faculty grading practices?

No.

First, the increase in average GPA for each cohort from first to fourth year is predominantly explained by the departure – voluntary or otherwise – of students who struggled academically.  If you slice that group off the bottom of a class at the end of each year, and you recognize the likely influence of maturity and motivation for the students who remain, we would fully expect that the average GPA of a particular cohort of students would go up over time.

Second, from 1991 until 2010 the average ACT score of our incoming students improved by a full point – from 24.5 to 25.5. Since the ACT remained constant during that period, we can test whether the increase in GPA might be explained by the increase in students’ incoming academic ability.  It turns out that this increase in average test score explains virtually all of the change in GPA over the twenty year period in question.

The Verdict:

Faculty grading behaviors may well have changed over time – maybe for worse, maybe for better.  But we have little evidence to suggest a relationship between those behaviors and an increase in overall GPA.  In addition, we have better evidence to suggest that a change in our students’ pre-college academic ability might have influenced this change in GPA.  Interestingly, if faculty grading behaviors had changed in the way that the grade inflation claim suggests, ACT scores would have likely not been as powerful a predictor as they turned out to be.

So the next time you hear someone mention the good ol’ days in the context of academic standards and grades, you might remind them that there are other – and maybe better – explanations for this phenomenon.  You might also remind them of the relative trade-offs of a circular firing squad.

Make it a good day,

Mark