My colleagues and I have been talking a lot lately about student/course evaluations, as are many folks in higher ed. Discussions about course evals center on validity (“truth in measurement”), in particular threats to it that come from response biases. Many things bias responses to any kind of behavioral measurement, and the current bias in course evaluations under the spotlight is “Gender Bias” — such that students reputedly rate male professors higher (or more positively) than female professors.

• NPR reported just yesterday: Why Female Professors Get Lower Ratings
• Earlier this month in “Inside Higher Ed”: Bias Against Female Instructors

Both news stories are reporting on this study — Student Evaluations of Teaching (mostly) do not Measure Teaching Effectiveness — also published this month.

The soundbite carried throughout the reporting:

“Course evals are a better measure of students’ gender bias than they are of teaching effectiveness.”

Coming from a place where course evals are like currency, I feel strongly about improving our process, as do my colleagues. So we read these news stories with interest and with intensity. If the process is biased, we must know and we must act, right!?! If we can cut through the bias, then evaluations will be more fair, and who wouldn’t want that? Duh. Of course we want to make our process fair.

But, then I read the reports — first a once-over, then a closer look. When you read these reports, at first glance, if you are like me (a female professor who wants a fair process in place!) you may be inclined to cheer. “Finally, folks are talking about this!”

But then, if you take a closer look at the research and at the claims made from it, you might find yourself feeling let down. That’s just what happened with me. I was hoping to finally find solid evidence that we could then use to improve our practice. Instead though, I found a disappointing and biased discussion about bias (oh, the irony). The IDEA organization nicely sums up the problems with the research and the reporting of it, so I don’t have to. It’s really worth the read.

So, there’s that. Bias in reporting about bias.

I wish that rather than putting our energies towards critiquing poor research though, instead folks could turn their eyes and minds towards building solid understanding and prevention. I would so much like to see national discussion extend beyond “tweaking a measurement process to deal with bias” and include a discussion of how to reduce bias in folks’ thinking, when their thinking holds high stakes. The extension I’d like to see would encompass discussion of …

… how to actually measure/document bias. Despite the IHE & NPR remarks, course evals are not a “measure of bias.” They are measurement systems that may be invalid because of bias. This distinction matters.

… how to raise our youth so that they don’t harbor biased views of their teachers and professors to such an extent that it interferes with their ability to engage in honest healthy meta-cognitive awareness of what and how they learn. When we understand bias, we can overcome it.

Gender bias is a big issue, and one that needs to be handled delicately at many levels and from the ground up, developmentally.  We would be well served to focus our energies on raising our school-aged youth to respect diversity in all its forms and to recognize bias in themselves and in others (something I discussed here few years ago). With recognition comes the ability to not let it cloud judgement. Moving up the educational ladder, we need to continue to challenge our college/university students to move beyond their “knee-jerk” responses, to cut through their potential bias, and to focus their thinking more solidly on the process of learning.

Really, isn’t that what education is supposed to be about? It’s not supposed to be about test scores, international rankings, and bench-mark-reaching. It’s supposed to be about socializing, nurturing, informing, and preparing our youth for successful adult lives. Making them good people. I wish the conversation was more about how to raise our children right and how to encourage our students to cut through the bias, and less about how to live with the bias that’s there, as if we can’t do anything to reduce it. We can, and we should.