Jesse Singal on “The problems with implicit bias tests”

Rationally Speaking | Official Podcast of New York City Skeptics - Current Episodes - RS 192 - Jesse Singal on “The problems with implicit bias tests”

"'A solid minority of even black test takers are also implicitly biased against their own race... The most common alternative explanation, and this has been proposed in literature a lot by several different researchers is: if you're aware of negative stereotypes about black people, you might be quicker to associate words like “crime” with black faces or victim or violence with black faces. If that's the case then that could generate a biased score of IAT, even though you're not implicitly biased against black people, you're just aware of these negative stereotypes.

There was one particularly ingenious experiment in which researchers actually created a new non-existent you could call either a race or a species called Noffians, N-O-F-F-I-A-N-S. By inducing in the experiments participants the idea that Noffians are this down-trodden group that society doesn't like, they were able to generate-... When Noffians were oppressed for example, people would score a higher IAT about Noffians.'

'More biased, more -- what we would have naively assumed was bias against Noffians?... And obviously they can't actually have any reason to think Noffians are bad because they're a made up group. And all we know is they have been oppressed.'

'Right, exactly. That was what was so ingenious about the experiment, was in this case there was no other explanation but that if you view a group as down-trodden, it might boost your IAT score regarding that group'...

'We're talking about, in some cases, a difference in reaction time of a couple hundred milliseconds. If you have a pie that is 200 milliseconds, how big a slice of the pie is actual implicit bias? How big is associations? How big is problems with the way the test is designed, or with error?The basic problem is that people have assumed the whole pie, or most of it, is something that we can genuinely call implicit bias against a group. But the studies trying to connect IAT scores to actual behavior in a lab setting simply haven't really shown that...

The most recent sophisticated meta-analysis showed that IAT scores account for less than 1% of the variance in racist behavior in a lab setting... this test does not come close to the level of test / re-test reliability we would expect for any instrument professionally used to measure anxiety, depression or anything else.The reason why it blew up the way it did and became this viral sensation, despite performing so poorly in terms of its psychometric attributes, is an important question...

It fairly consistently finds that women are more implicitly biased against women than men are... it basically shows that white women have the highest level of implicit bias. And any kind of man, white men, black men, Hispanic men are less implicitly biased when it comes to gender... It's basically just reported as implicit gender bias, which we can take safely to mean just that men are better than women, or better qualified for certain positions than women...

Generally speaking all women are more biased than all men, more implicitly biased... even strongly liberal women, they're more implicitly biased against women than far, far right men, so think about that for a minute...

I think this test tells white liberals a story they want to hear, and I'm including myself in that because I'm very much white and very much a liberal. There's something about the experience of taking this test that can make white liberals feel like they are taking part in the fight against racism.

Despite -- I'll try to be gentle, but often times white liberals don't really do much but take the IAT or tweet about racial injustice. Oftentimes, because of the crappy system that we have set up, we are perpetuating many of these forms of inequality. I think the IAT might give us an easy way out, to feel like we're fighting the good fight without actually doing anything.'

'There's this sort of ritual that people seem to go through of reporting their score with this really troubled, emotional tone, almost like a confessional, like ... This wasn't your phrasing, but it reminded me of some of the circles in the early '70s, like at Esalen or something, where people were sort of confessing their sins to the group in this really emotional way, almost exercising their bigoted demons or something.'...

'They made claims that significantly ran ahead of the evidence. That's most striking in some of the text of their book, where they really say, "This test does a great job predicting behavior, better than anything else we have, better than explicit measures."Two years later, they themselves are admitting in an academic journal that no one's going to read, "This test should not be used to diagnose individual levels of implicit bias." They themselves have admitted that.Why was there all this excitement over a test that, 20 years in, the architects of the test had to say, "You shouldn't use this on individuals."'...

'There's also this thing that I've seen some of ... Not from all proponents of the IAT, but from some of them, that's like: when they encounter criticism or pushback on the claims, they use the defense, "If you're criticizing the IAT, then maybe it's because you're a racist, or you don't care about racism," or something like that.'...

'It's just weird to see a totally different set of standards apply to the IAT than would apply to basically any other scientific project.'"