Now that we’ve been using the Starfish tool for a couple of years to foster a network of early alerts and real-time guidance for our students, I suppose it makes sense to dig into this data and see if there are any nifty nuggets of knowledge worth knowing. Kristin Douglas (the veritable Poseidon of our local Starfish armada) and I have started combing through this data to look for useful insights. Although there is a lot more combing to be done (no balding jokes, please), I thought I’d share just a few things that seem like they might matter.
Starfish is an online tool that allows us to provide something close to real-time feedback, positive, negative, or informational, to students. In addition, this same information goes to faculty and staff who work closely with that student in an effort to provide early feedback that influences future behavior. Positive feedback should beget more the same behavior. Negative feedback hopefully spurs the student to do something differently.
In general, there are two ways to raise a Starfish flag for a student. The first is pretty simple: you see something worth noting to a student, you raise a flag. These flags can come from anyone who works with students and has access to Starfish. The second is through one of two surveys that are sent to faculty during the academic term. This data is particularly interesting because it is tied to performance in a specific class and, therefore, can be connected to the final grade the student received in that class. The data I’m going to share today comes from this survey data.
We send a Starfish survey to faculty twice per term. The first goes out in week 3 and asks faculty to raise flags on any student that has inspired one(or more) of four different concerns:
- Not engaged in class
- Unprepared for class
- Missing/late assignments
- Attendance concern
The second Starfish survey goes out in week 6 and asks faculty to raise flags that address two potential concerns:
- Performing at a D level
- Performing at an F level
We now have a dataset of almost six thousand flags from winter, 2015/16 through winter, 2017/18. Do any of these flags appear to suggest a greater likelihood of success or failure? Given that we are starting with the end in mind, let’s first look at the flags that come from the week 6 academic concerns survey.
There are 1,947 flags raised for performing at a D level and 940 flags raised for performing at an F level. What proportion of those students (represented by a single flag each) ultimately earned a passing grade in the class in which a flag was raised?
The proportion that finished with a C or higher final grade
- Performing at a D level (1059 out of 1947) – 54%
- Performing at an F level (232 out of 940) – 25%
On first glance, these findings aren’t much of a surprise. Performing at an F level is pretty hard to recover from with only three weeks left in a term. At the same time, over half of the students receiving the “D” flag finished that course with a C grade or higher. This information seems useful for those advising conversations where you need to have a frank discussion with a student about what it will take to salvage a course or drop it late in the term.
The second set of flags comes from the third week of the term and represent behaviors instead of performance. Are any of these raised flags – not engaged in class (278), unprepared for class (747), missing/late assignments (1126), and attendance concern (904) – more or less indicative of final performance?
The proportion that finished with a C or higher final grade
- Not engaged in class (202/278) – 73%
- Unprepared for class (454/747) – 61%
- Missing/late assignments (571/1126) – 51%
- Attendance concern (387/904) – 43%
There appears that these four flags vary considerably in their correlation with a final grade. Attendance concern flags appear to be the most indicative of future trouble while appearing unengaged in class seems relatively salvageable.
Without knowing exactly what happened after these flags were raised, it’s hard to know exactly what (if anything) might have spurred a change in the behavior of those students who earned a final grade of C or higher. However, at the very least these findings add support to the old adage about just showing up.
What does this data suggest to you?
Make it a good day,
Mark
Here’s a question Mark. Does having one’s flag cleared (or cleared within a week) predict higher rates of passing one’s class? That would seem to be the lowest bar to clear in order to show that the starfish system is encouraging students/professors/advisers to do something that gets the student’s performance turned around.