No matter how solid the overall academic characteristics of a first year class, by the time spring rolls around we always seem to wish that we could be just a bit more precise in identifying students who can succeed at Augustana. Yes, there are some applicants who are almost guaranteed to succeed and some who are obviously nowhere near ready for the Augustana experience. But most applications fall somewhere in between those two poles. And even though this is an exceedingly complicated and imperfect exercise, every bit of information we can tease from our own data can help us perfect our efforts to pick out those diamonds in the rough.

Standardized test score (i.e., ACT or SAT) and high school GPA have always been the big dogs in predicting college success. In the not too distant past, these two metrics were often the only numbers that a college used to make admissions decisions. But (at least) two problems have emerged that make it critically important to test the veracity of both metrics among our own students. First, the test prep business has become an almost ubiquitous partner to the tests themselves. ACT or SAT preparation resources (be they online or in person) are often strongly encouraged and sometimes are even offered as a part of the high school curriculum. As a result, one could argue that standardized test scores increasingly predict test taking skills rather than academic preparation. (Given that the average ACT composite score has remained the same from 1997 to 2016, I’m not exactly sure what the growth in the test prep business says about the industry or the people who pay for those services . . . but that is another story). When we add to the mix the correlation between socioeconomic status and available educational resources, test score becomes an even more suspect measure of academic potential.

Second, high school GPA has become an increasingly “flexible” number as high schools have added more and more weighted courses and varying academic “tracks” for different types of students. As a result, high school GPAs sometimes appear much more tightly clustered for certain types of students, making it more difficult to claim that a moderate differences in high school GPA between two applicants represents an actual difference in college readiness. In addition, these patterns of clustering quickly become specific to an individual school or district, making it even harder to compare applicants between schools or districts. For these reasons Augustana decided a number of years back to generate for each applicant a recalculated GPA that removes much of the peculiarity of the GPA initially provided by the high school.

Given the increasing murkiness of these two metrics, it makes sense to test the predictive validity (i.e., trustworthiness) of each among our own students. In addition, since students almost always submit both a test score and a high school GPA, it would help us a lot to know more about each metric in the context of the other. For example, what if a student’s test score seems much stronger than their high school GPA, or the other way around. Should we place more value on one over the other? Should we put our trust in the more favorable of the two metrics?

To conduct this inquiry thoroughly, we tested the effect of high school GPA and standardized test score on three different measures of first year success: cumulative GPA at the end of the first year, retention to the second year, and the number of credits completed during the first year.

In short, it isn’t much of a contest. The recalculated high school GPA significantly predicts first year cumulative GPA, retention, and number of credits completed. Standardized test score only predicts first year cumulative GPA, while producing no statistically significant effect on retention or number of credits completed. In addition, when high school GPA and test score were analyzed head-to-head (i.e., both variables were included in the same statistical analysis predicting first year cumulative GPA), the size of the high school GPA effect was two and a half times larger than the effect of the standardized test score.

This would suggest that one ought to prioritize the recalculated high school GPA over the standardized test score when evaluating a prospective student. But remember the questions I posed a few paragraphs back about an applicant whose test score and high school GPA don’t seem to match up? We felt like we needed to run one more test just in case.

To test the phenomenon of a test score and a high school GPA that appear to “disagree” with each other, we created a variable that reflected the relative gap between test score and high school GPA and tested the relationship between this variable and the first year cumulative GPA (I’ll gladly explain in more detail the steps we took to build this variable off line. Suffice it to say that, “We got our stats nerd on, and it was awesome.”) Interestingly, our findings closely mirrored our prior results. As the test score exceeded the high school GPA (i.e., as the test score represented an increasingly higher academic potential than the GPA), the first year cumulative GPA tended to drop. Conversely, when the high school GPA exceeded the test score, first year cumulative GPA tended to rise. Although there is a point at which this variable is no longer useful (e.g., if the test score is 35 and the high school GPA is 1.5, one starts to suspect something more nefarious might be at play), these findings corroborate our earlier tests. For Augustana applicants, recalculated high school GPA is a more accurate predictor of first year success than the standardized test score.

So if you are advising first year students, be careful about making assumptions about your students’ ability based on their test scores. Likewise, when there appears to be a gap between what the test score and the high school GPA might suggest, which metric exceeds the other might tell very different stories about the needs of a given student.

Make it a good day,

Mark