So what’s this about the National Survey of Student Engagement coming to Augustana to study us?  Essentially – as Ellen described in her cover article, although many institutions have been using NSSE for over a decade to assess student experiences, far fewer have 1) used the findings from their data to construct broad changes in educational programming, and 2) documented improvement in educational quality on subsequent NSSE surveys as a result of those changes.  I thought it might be helpful to explore the areas where we have seen substantive change in NSSE scores and note the program and policy changes that we think contributed to this change.

First, some background.  NSSE is a four page survey that asks a series of demographic and college experience questions.  The experience questions are organized under five broad concepts that NSSE calls “benchmarks,” each representing something that we know makes a difference in the quality of a students’ education.  They are:

Level of Academic Challenge

Active and Collaborative Learning

Student-Faculty Interaction

Enriching Educational Experiences

Supportive Campus Environment

Augustana began using NSSE in 2002 and continued to utilize it in 2003, 2006, and 2009.  In addition, NSSE is included in the Wabash National Study, so our freshmen class of 2008 also completed NSSE.  We will administer NSSE again in the spring of 2012.

Since NSSE revised the way they calculated the benchmark scores in 2005, we can’t compare the benchmark scores since 2003, but we can see some impressive changes between 2006 and 2009.  Comparing freshman scores, we made statistically significant increases in Student-Faculty Interaction and Supportive Campus Environment benchmarks.  Comparing senior scores, we made a statistically significant increase in Enriching Educational Experiences.

There are also changes on individual items that suggest some improvement in educational quality.  Changes in NSSE scores between 2003 and 2009 suggest that on average, we have increased the extent to which our freshmen make class presentations, prepare two or more drafts of a paper before turning it in, and work with classmates outside of class to prepare assignments.  In addition, our 2009 freshmen believe that Augustana is making are larger contribution to their growth in speaking clearly and effectively, working effectively with others, understanding people of other racial and ethnic backgrounds, and contributing to the welfare of the community.

A close examination of our NSSE data indicates that, although we still can do more to improve the educational quality of an Augustana experience, 1) we do a lot of things well, and 2) we have made numerous substantive improvements during the past decade.

If you’d like to know more about any aspect of our NSSE data, please don’t hesitate to ask.

Make it a good day.

Mark

To those of you who were able to take some time away last week – welcome back!  And to those of you who never left – thanks for sticking around!

Although we are well into the conversation about improving student learning through curricular reform, the other half of the educational effectiveness equation remains a bit of a conundrum.  This “other half” to which I refer is our students’ motivation to plug in to the process of learning and growing.  We all know some students who don’t seem to care much at all about their education.  In addition, we all know of students who are tremendously motivated to get good grades but seem to care very little about learning.  So what do we know about our students’ motivation to learn and succeed in college?

Augustana has not traditionally collected much data that fully addresses student motivation.  Sometimes we have presumed that increased student satisfaction will lead to increased motivation.  Yet we know that motivation is more complicated – that there are different types of motivation that can produce vastly different results.  As a liberal arts college, we actually want our students to develop an intrinsic motivation to learn and be less concerned about extrinsically measured achievement.

Although the Wabash National Study didn’t really flesh out the idea of motivation, it did include two items that we can use to dig into the way that student motivation might change in college.  Both items are presented as agree/disagree statements with a response scale of 1 (strongly disagree) to 5 (strongly agree).  The first item states, “Getting the best grades I can is very important to me.”  The second item states, “I am willing to work hard in a course to learn the material even if it won’t lead to a higher grade.”

Our current seniors participated in the Wabash National Study as freshmen in 2008 and 2009.  So the data we have comes from the beginning and the end of their first year at Augustana.  Here are their average responses to both questions.

There are two observations I would like you to consider.  First, in both the fall and spring our students appear to rate getting the best grades they can as more important than working hard to learn regardless of that effort’s effect on grades.  Second, between fall and spring the change in the importance of getting the best possible grades is not large enough to be significant, suggesting that this value does not change on average.  However, the change in willingness to work hard regardless of grades between fall and spring is significant – suggesting that intrinsic motivation to learn may have actually dropped during the first year.

I am going to revisit this topic in a couple of weeks because it cuts to the core of our efforts to effectively prepare students to succeed in their personal and professional lives.  Moreover, it appears that there are certain types of educational experiences that may increase intrinsic motivation.   How is that for a teaser?!

Make it a good day,

Mark

One of Augustana College’s fundamental values is the importance of high quality relationships between students and faculty.  Indeed, this is one of the most compelling arguments for attending a small liberal arts college.  I would argue that this assertion has always included the quality of relationships between students and everyone who might impact our students’ educational experience – not just those traditionally considered to be faculty.  In fact, students don’t differentiate between those of us who are “faculty” and those of us who are “staff” or “administrators” in the same way that we do.  Thus, all of us should fully expect to have a significant impact on our students’ education – no matter the context of those interactions.

There are two questions on the National Survey of Student Engagement (NSSE) that ask about the quality of students’ relationships with 1) faculty members and 2) administrative personnel and offices. The response options are somewhat unique – instead of a simple 5-item agree/disagree scale, the available responses are portrayed on a spectrum from 1-7; with 1 representing “unfriendly, unhelpful, sense of alienation” and 7 representing “friendly, supportive, sense of belonging.”  This spectrum may require a little more thought, but the idea is that these choices more fully reflect the concept underlying the question.

In comparison with all of the schools that use NSSE (most of which are much larger than Augustana), both our freshmen and our seniors report a significantly higher quality of relationships with faculty.  Moreover, the quality of relationships appears to improve between the freshmen and senior year.

However, something happens when our students are asked the same question about their quality of relationships with administrative personnel and offices.  While our freshmen scores are significantly higher than the overall NSSE average, our senior scores are significantly lower than the comparable overall NSSE average.

Regardless of the possible reasons for lower ratings of relationships with administrators generally (one such reason might be that administrators tend to dispense fines more often than faculty, for example), it seems reasonable to strive for and expect an increase in the quality of those relationships that mirrors the change in student/faculty relationships.  Unfortunately, this does not appear to be case.  While the reasons for these opposing trends are probably complex, I would humbly suggest that they not beyond our control.

Usually I end my column with a simple “Make it a good day.”  This time, I’d like to end with something slightly different.

In your own way – big or small, make it a good day for a student.

Mark

A couple of weeks ago I highlighted our success in maintaining a historically high 1^st-2^nd year retention rate (87%) despite a substantial increase in the size of the freshmen class between 2009 to 2010.  Although this is something that we should indeed celebrate, we need to be willing to look inside these numbers and explore whether our overall rate accurately reflects the behaviors of various student types.  This week I want to dig a little deeper and explore those variations.  To be precise, I’ll call it persistence when talking about the students’ decision and retention when talking about the number that we track as a proxy measure for student experience and success in the first year.

As you might expect, our retention rate isn’t the same for all student types.  Pre-college academic ability plays a big role.  Students with an ACT score of 23 or above persisted at 89%, while those with an ACT score below 23 persisted at 81%.  Likewise, there are two demographic characteristics – race/ethnicity and gender – that historically influence retention rates across the country as well as at Augustana.  In the 2010 cohort, white students persisted at 89%, while multicultural students persisted at 81%.  In addition, female students persisted at 89%, while male students persisted at 85%.

Before talking about what these differential rates might mean, it is important to remember that pre-college ability, race/ethnicity, and gender don’t exist independently – an individual student is necessarily categorized along all three dimensions.  So the question also becomes whether or not there is a subset of categories that, when combined, produce a starkly lower likelihood of persistence to the second year.

Not surprisingly, we have such a troublesome combination at Augustana.  Of the three categories listed above, it is the combination of being male and multicultural that produces the lowest retention rate of any combination – 77%.  Interestingly the effect of being male also influences the retention rate of students with higher incoming ACT scores.  Females with ACT scores of 23 or above persisted at 92%, while males with a similar ACT score persisted at 86%.  By comparison, the retention rate of students with lower ACT scores (below 23) did not vary significantly by gender.

Although these differences might suggest an array of programmatic interventions, solving a retention “problem” can be a bit like the old carnival game Whack-a-Mole.  A singular focus on one subset of students can become a frustratingly reactionary exercise over time. Yet, understanding the nature of these students’ challenges can be a critical first step in addressing retention issues. What common issues might be at the core of these differences across gender and race/ethnicity?  To help us take a first step in thinking about the issues that our male students face, I’d encourage you to attend Dr. Tracy Davis’ presentation at Friday Conversation this week.  I’ll talk more about the challenges facing multicultural students in a later column.

Make it a good day,

Mark

Some of you have expressed concerns about our students’ high level of involvement in extra-curricular activities and its potential effect on academic engagement as well as mental health.  There seems to be plenty of evidence to support this concern – some of the most poignant being the writings of our students themselves in the Observer and on the Augie Blog.

Yet there is one lesser known data point regarding our students’ social behaviors that reflects well on a cultural norm within the student community.  In addition, I’d like to suggest that we might learn something from our students’ social behaviors that could be a powerful lever in deepening academic engagement.

A question on the National Survey of Student Engagement asks students how often in the last year they “attended an art exhibit, play, dance, music, theatre, or other performance.”  The response options are 1= Never, 2= Sometimes, 3= Often, 4= Very Often.  Instead of focusing on the absolute numbers here (mostly because there is no consensus ‘window’ within which we want our students’ response to sit).  Rather, I want to present this data in the context of a comparison with other comparable private liberal arts colleges and what that might suggest.

It turns out that both our freshmen and seniors attend these kinds of performances substantially more often than students at comparable private liberal arts colleges. In fact, the difference in average response between our students (freshmen – 2.70, seniors – 2.60) and students at comparable colleges (freshmen – 2.44, seniors – 2.36) is extremely statistically significant, meaning that this difference is likely attributable to something happening here.

This suggests to me that there is something in the student culture that encourages and values supporting the arts.  Our students place a relatively high value on attending and supporting friends involved in those performances.  This finding corroborates independently gathered anecdotal evidence from the Office of Student Services.

What does this have to do with deepening academic engagement? If students are in the habit of supporting their friends’ co-curricular accomplishments, I would suggest that this apparent cultural norm provides a real opportunity to increase the relative value of and interest in students’ academic accomplishments. Public presentations of student scholarship can serve as a spark to inspire informal conversations among students about intellectual ideas and their application to the world around them.  Although traditionally associated with the fine arts, there is plenty of evidence to suggest placing a greater value on public presentations or exhibits of scholarship can deepen academic engagement outside of class across many disciplines.

By the way, our NSSE data also indicates that our students don’t talk about ideas from readings or classes with others outside of class at the same rate as students at comparable institutions. Hmmm . . .

Make it a good day,

Mark

Like most colleges and universities, we believe that we should establish an educationally rigorous environment.  Unlike a lot of colleges and universities, we have a healthy body of quantitative and qualitative evidence from which we can explore, 1) whether this is in fact the case, and 2) whether appropriate academic rigor is experienced by students across the board or only in certain situations.

As you may know, for well over a decade we have been using various assessment mechanisms to measure student learning and academic rigor.  It seems that our efforts to increase our educational effectiveness and academic rigor have borne some fruit – especially on our National Survey of Student Engagement (NSSE) Academic Challenge scores among freshmen.  Those numbers have jumped markedly since we first used NSSE back in 2003.

Yet one of the hallmarks of a college that is truly focused on continual improvement is a perpetual inclination to ask questions, to compare findings with what we might already suspect or know by different means, to face what we uncover, and to take action.

With that in mind – and in light of my perpetual effort to help us all embrace a formative spirit, I’d like to present two data points from the 2006 and 2009 NSSE survey that seem especially worthy of further consideration.

In both 2006 and 2009, Augie students were asked how often they “come to class without completing readings or assignments.” (The response options are 1=Never, 2=Sometimes, 3=Often, and 4=Very Often.)  I would propose that the one thing we would not want to see is that seniors come to class unprepared more often than freshmen.  Unfortunately, this does not appear to be the case.  And if you were wondering, the difference between the average freshmen and senior response is large enough to be significant.

To add insult to injury, the change from freshmen to senior year looks worse when comparing our 2009 data to other small liberal arts colleges.  In this context, our freshmen actually come to class prepared significantly more often than freshmen at comparable institutions.  However, our seniors come to class prepared significantly less often than seniors at comparable institutions.

Does this match what we already suspect?  Are we ok with it?  How might we address this issue?

Make it a good day,

Mark

There was a time in higher education when an institution’s attrition rate was a point of pride and a supreme marker of academic rigor.  More recently, it sometimes seems as if retention and graduation rates have actually surpassed educational growth in ranking institutional quality.  In reality, these two markers are clearly intertwined.  Although educational growth is paramount, such growth seems a bit empty if an institution is also hemorrhaging students somewhere between matriculation and graduation.

So if a college could actually demonstrate substantial educational growth while simultaneously increasing retention rates, the faculty and staff at that institution would have a real reason to take great pride in their collective accomplishments.

It is becoming clear that Augustana College is one such institution.  We now have both direct and indirect evidence of educational growth.  Using the Collegiate Learning Assessment (CLA) to measure the growth of critical thinking skills between 2005 and 2009, our students improved by 28 percentile points – double the average of students participating in the Academically Adrift study.  In addition, during the last 10 years our NSSE Academic Challenge Benchmark scores have improved significantly among first year students – an accomplishment that was recently highlighted by the National Institute for Learning Outcomes Assessment.

By itself, this is well worth a hearty pat on the back.  However, it looks even better in the context of our increases in retaining students to the second year.  For a long time, Augustana’s first-to-second year retention rate hovered around 85%.  Three years ago, we retained the freshman class of 2008 at 82%, sparking some concern as the size of our incoming 2009 class also dropped.  After an increased focus on supporting struggling first year student, our retention rate among the 2009 class jumped to about 87%.  But we weren’t sure if that was an anomaly or a true reflection of our efforts.

Now that we have locked in our 10^th day enrollment data this fall, we are able to look at our first-to-second year retention rate for the incoming class of 2010.  Some of us had wondered aloud whether our retention rate with this class would take a hit, presupposing that it’s a lot easier to retain students from a class of 616 (the freshman class of 2009) than students from a class of 753 (the 2010 freshmen class).

However, our retention rate for the 2010 freshmen class remained steady at 87.5%. Thus, despite an increase in class size of 137 students, we maintained Augustana’s highest retention rate on record.  Your efforts to help students succeed in the first year are bearing fruit.  We have a lot reasons to be very proud of our community.

Make it a good day,

Mark

Over the last several years, Augustana has increasingly encouraged students to participate in an internship, study abroad, or undergraduate research.  As a result, our participation rates have increased and more of our students are gaining the benefits of these potentially powerful learning experiences. Yet while we track the participation rates for each of these educational experiences separately, we haven’t talked as much about the degree to which students who participate in any one of these experiences actually participate in multiple experiences, thereby making the total percentage of students who engage in any of these experiences substantively lower than we might like to think.   In addition, we have not yet thought as much about how to systematically ensure that each student engages in the kind of experiential learning that best fits their educational needs and goals.

To get a better handle on these questions, I looked at the senior survey data from the spring of 2008 and 2009.  For context purposes, here are the separate participation rates for study abroad, internships, and undergraduate research (just to clarify, undergraduate research is defined as working with a faculty member on a research project).

• Study abroad                                     39.5%
• Internships                                         44.1%
• Undergraduate Research                  16.6%

However, it is clear that there are quite a few students engaged in multiple activities.  So, after re-examining the data, here is the percentage of students who did or did not participate in at least one of these activities.

• Students who participated in at least one activity:                          67%
• Students who did not participate in any activities:                          33%

Thus, while a majority of our students are gaining the benefits of one of these “high impact activities,” a substantial proportion of them are not.

While there might be many varied and valid reasons for the 33% who did not participate in any of these activities, I’d like to highlight one difference between those who did and did not participate for you to ponder – the relative proportion of men and women in each group.  Again, for context, in this dataset 58.8% of the respondents were female and 41.2% were male.

Among those who participated in at least one activity, 64.7% were female and 35.3% were male.

Among those who did not participate in any of these activities, 47% were female and 53% were male.

Is this participation gap a reason for concern?  If so, what could we do about it?

Make it a good day,

Mark