Description of the video:
[Video: Koji Chavez stands in front of a wall speaking to people in the room.]
Graphic: Recorded October 25, 2019
Koji speaks: Yeah so I'm Koji Chavez. I’m in my second year here and I was asked to give this presentation so I'm it's an honor and a privilege to be here.
[Video: A graphic that says “Creating Effective Searches: Implicit Bias in Evaluation and How to Reduce It]
Koji speaks: Today I'll be talking a little bit about how bias works in selection decisions particularly in hiring decisions.
[Video: Koji stands in front of a wall speaking to people in the room.]
Koji speaks: Full disclosure I haven't been on a hiring committee here at the University yet but I have started hiring in other contexts, in corporate contexts. So hopefully some of the things that I talk about will be transferable to all of you as you go through your hiring searches. So we’ll talk a little bit about implicit bias and how social psychologists think it works and then we'll go through a number of policy solutions or best practices that have been shown to be somewhat effective in terms of combatting implicit bias. And so you know I'm not going to give you a whole list of things that I think you should do, but maybe picking up one or two to implement in your hiring searches might might be nice, okay?
[Video: A graphic says:
“Examples stereotyping in everyday life:
- Choosing a tutor
- Getting the cable fixed
- Common features:
- Occurs unconsciously, often not aware stereotypes are affecting cognition
- Even if you don’t consciously endorse/believe the stereotype
- Busy, using top-down cognition
- Stereotyping process can be interrupted, takes effort]
Koji speaks: First just some examples of what stereotypes look like, how they are very common in everyday sort of life. I was thinking through some examples in my own life here at IU and one came to mind. I remember a student came up to the after class, you know I teach a race ethnicity course, I'm teaching a work in society course. I forgot I think he was in the work in society course and after class he comes to my office hours.
[Video: Koji Chavez stands in front of a wall speaking to people in the room.]
Koji speaks:
He's like, you know the moment you walked in i knew this class is gonna be fun. And I was thinking like well how did why did he think that the moment I walked in, right?
Maybe because I'm younger than his other professors, he was a business student, all right, so maybe he had some preconceived notions about what sociology was all about, right. So he was using stereotypes to sort of affect his perceptions about who I was as a professor, right.
[Video: A graphic says:
“Examples stereotyping in everyday life:
- Choosing a tutor
- Getting the cable fixed
- Common features:
- Occurs unconsciously, often not aware stereotypes are affecting cognition
- Even if you don’t consciously endorse/believe the stereotype
- Busy, using top-down cognition
- Stereotyping process can be interrupted, takes effort]
Koji speaks: But you can think of this in all different types of situations. One of my colleagues had a student come up to him with a question about who he should choose as a tutor and he had this list of tutors out, right. And he told he tells my colleague, oh I just picked the name that was probably the best tutor.
[Video: Koji stands in front of a group of people sitting around round tables.]
Koji speaks: And it was the most, you know, Eastern European sounding name on the list. So he was using stereotypes, you know he was thinking about stereotypes, about who he thought was a good tutor, and he was using that to influence his perceptions, influence his decision-making, really, without really thinking through it. There's a bunch of these examples that we could think about that happen in our everyday life, right. Now what are some common features about how stereotypes typically influence our decision-making? So often they occur unconsciously, right. In these decisions that I had talked about a moment ago, and the one that my student was making, it's probably done unconsciously.
[Video: A graphic says:
“Examples stereotyping in everyday life:
- Choosing a tutor
- Getting the cable fixed
- Common features:
- Occurs unconsciously, often not aware stereotypes are affecting cognition
- Even if you don’t consciously endorse/believe the stereotype
- Busy, using top-down cognition
- Stereotyping process can be interrupted, takes effort]
Koji speaks:They're not consciously thinking about a stereotype and saying I'm gonna apply the stereotype to my evaluation, I'm gonna apply the stereotype to my decision. It
doesn't usually happen.
[Video: Koji stands next to the screen with the stereotype graphic on it, still speaking.]
Koji speaks: We're not often aware that the stereotypes are affecting our decisions, are affecting our perceptions and yet they often are. And social psychologists have
talked about this over and over. Another important point, just a general point about stereotypes is that you don't have to endorse the stereotypes or believe in
them for them to have an effect on your perceptions and your evaluations, right. That's often a common misconception. And you know another element of stereotypes that I think is important is that we often rely on them, again subconsciously or unconsciously, when we're busy right, when we're thinking about other things. Also when we're like
frustrated or angry, these sort of things we're not really paying attention to during the task at hand, oftentimes stereotypes will affect our evaluation, affect our perceptions.
[Video: A graphic says:
“Examples stereotyping in everyday life:
- Choosing a tutor
- Getting the cable fixed
- Common features:
- Occurs unconsciously, often not aware stereotypes are affecting cognition
- Even if you don’t consciously endorse/believe the stereotype
- Busy, using top-down cognition
- Stereotyping process can be interrupted, takes effort]
Koji speaks: This doesn't mean that this is a process that is inevitable right? So we do know that we can sort of interrupt the stereotyping process if we just put in a little bit of effort right. But it does take a little bit of effort to do so.
[Video: A graphic says:
Where do stereotypes come from:
- Two or more concepts become associated with one another through related exposure.]
Koji speaks: So where does stereotypes come from? I'm starting at the most basic, the most basic foundations of stereotypes and evaluations.
[Video: Koji stands in front of screen.]
Koji speaks: So what stereotypes come from is we typically associate two or more concepts together over repeated exposure over time, right. So one simple way to think about this is, you know you might have a favorite song and when you think about that favorite song, you're thinking about maybe what you were doing when you first heard it or who you heard it with, right. We associate those two things together in our heads. So for me, whenever I hear Drop it like it's hot by Snoop Dogg, I think of the time that I was driving from my parents house in Irvine to Santa Barbara where I went to college. And me and my college roommate we, you know, memorized all the words to that song over that two and a half hour drive. So when I hear that song, I associate with the specific experience, right. So if I said peanut butter and you all would say jelly.
[Video: A graphic says:
Where do stereotypes come from:
- Two or more concepts become associated with one another through related exposure.
- Favorite song/thing you did while listening to that song
- Peanut butter and…]
Koji speaks: if I said rum, rum and coke, right. So you associate these words together, yeah that's essentially how stereotypes work, right.
[Video: Lines are added to graphic:
- Rum and…
- Thinking of one concepts brings to mind the other]
Koji speaks: Bringing up, thinking about one of these concepts brings up the other one because they're associated in your head.
[Video: Koji stands in front of group of people:.]
Person in crowd speaks: indecipherable mumble
Koji speaks: Yeah that's true, that is true, right. So the specific content of stereotype might differ in different cultural contexts, that is true. Yeah it is sort of weird to have peanut butter and jelly in some places, right. It's very strange sort of combination of things.
[Video: A graphic says:
Where do stereotypes come from:
- Two or more concepts become associated with one another through related exposure.
- Favorite song/thing you did while listening to that song
- Peanut butter and…
- Rum and…
- Thinking of one concepts brings to mind the other
- But doesn’t mean you endorse this association
- Thinking of peanut butter might make you think of jelly, but it doesn’t mean you like peanut butter and jelly sandwiches]
Koji speaks: Here though right, it doesn't mean, so even though you're thinking about these concepts together, it doesn't mean you're endorsing them. So you could be thinking about peanut butter and jelly, but it doesn't mean that you like peanut butter and jelly sandwiches, doesn't mean that you want one right now, it's just you are associating those two things together in your head.
[Video: Koji gestures as he speaks]
Koji speaks: So how social psychologists have, sociologists and I guess psychologists think about stereotypes is often different from our common sense notion of how stereotypes work or who is actually using stereotypes to evaluate people or perceive people.
[Video: A graphic says:
“Common sense” explanation of stereotyping vs. research based explanation
- Bad prejudiced people vs. good, unprejudiced people
- True that some people do hold explicitly prejudiced beliefs
- This is called “conscious” or “explicit” stereotyping
- But stereotypes often operate implicitly:
- Everyone exposed to same cultural stereotypes
- Stereotypes we reject can still affect our behavior
- “Common sense” explanations can make it hard to talk about stereotyping]
Koji speaks:So the common misunderstanding about how how bias works in general is that there are bad people out there that are prejudiced and they use stereotypes and they you know are unfair in their evaluations and they're just like bad people, right.
[Video: Koji gestures as he talks to group.]
Koji speaks: And that's compared to good people that are unbiased, that don't use stereotypes, that are very pure, right. And we have this image in our head. You know there's a classic Bill Burr skit, not skit but a bit that he does where he talks about how racism is shown in movies where it's always this old guy with like a bang coming out of his the side of his head, like yelling at people alright. That's sort of this image that we have of how prejudice works, of how racism works in general.
[Video: A graphic says:
“Common sense” explanation of stereotyping vs. research based explanation
- Bad prejudiced people vs. good, unprejudiced people
- True that some people do hold explicitly prejudiced beliefs
- This is called “conscious” or “explicit” stereotyping
- But stereotypes often operate implicitly:
- Everyone exposed to same cultural stereotypes
- Stereotypes we reject can still affect our behavior
- “Common sense” explanations can make it hard to talk about stereotyping]
Koji speaks: But that's not really how it works in most the time in real life. And I would say that you know obviously sometimes it is true that there are people that hold prejudice beliefs and that is true but in general, right, stereotypes work at an implicit level, in an unconscious level.
Everyone has these stereotypes, everyone knows what these stereotypes are, they just, you know, become associated in our head, so it's not this image, right, of good and bad people. That is not really how it works. It's something that happens to everyone and again, even if we reject this stereotype, we don't believe it, it still can affect our behavior still can affect our perceptions, affect our evaluations, all right. That's sort of the power of stereotypes in general. They're very, very powerful. So what I'm getting at here is that you know, this common sense notion of good and bad people, either you know, the good people don't stereotype, the bad people do it, makes it hard, right, when thinking about how stereotypes actually work right. Because it's something that affects most people or everyone has been affected by stereotypes at the unconscious level.
[Video: A graphic that says:
What are stereotypes?
- A fixed set of characteristics that is attributed to all members of a group
- (Michener, Delemater, and Meyers)
- An image says “The best college professors have beards”]
Koji speaks: Okay yeah well actually that sort of relates to my my anecdote, right, where I'm not that bearded. I do have a mustache though, but I'm not that bearded professor. Someone saw that and they thought it was fun. So what are stereotypes? Stereotypes are a fixed set of characteristics that is attributed to all members of a given group, right.
[Video: A graphic that says:
What are stereotypes?
- A fixed set of characteristics that is attributed to all members of a group
- (Michener, Delemater, and Meyers)
- Stereotypes can be explicit or implicit
- Assumption that members of a group possess certain traits
An image says “The best college professors have beards”]
Koji speaks: Stereotypes can be explicit or implicit and there's just this assumption that the people within that group hold those stereotypes to a certain degree, right. And it's important here to know that you know stereotypes can be both positive and negative.
[Video: A graphic that says:
What are stereotypes?
- A fixed set of characteristics that is attributed to all members of a group
- (Michener, Delemater, and Meyers)
- Stereotypes can be explicit or implicit
- Assumption that members of a group possess certain traits
- May be positive or negative
- Or often inaccurate
- Can be difficult to change
An image says “The best college professors have beards”]
Koji speaks: So you can hold stereotypes of a certain group that are that are both positive and negative. You can hold both those things in your head at the
same time. They're not all negative. They're oftentimes inaccurate, right, and this is important. You know they, they, your stereotypical beliefs or widely held beliefs about a certain group, they can often be very difficult to change.
[Video: Koji talks to crowd]
Koji speaks: ou know one reason why this would be is something that we call subtyping, right, where you have these larger stereotypes let's say about gender, right, but you know they really don't apply to the people that you know because they're different right than the other group. All right that's sort of this, this subtyping that we do, that's one of the reasons why it's very difficult to change the overall stereotypes that we hold, right.
[Video: A graphic that says:
What are stereotypes?
- A fixed set of characteristics that is attributed to all members of a group
- (Michener, Delemater, and Meyers)
- Stereotypes can be explicit or implicit
- Assumption that members of a group possess certain traits
- May be positive or negative
- Or often inaccurate
- Can be difficult to change
- People can hold negative stereotypes of groups they belong to
An image says “The best college professors have beards”]
Koji speaks: And then another thing, and this is also a misconception I think, is that, you know, we often think of people thinking of their group as good and having a good stereotypes about their group, bad stereotypes about an out-group. That's oftentimes not really how it works. I mean you can hold negative stereotypes even about your own group. That's another sort of misconception that we have.
[Video: A graphic says:
Stereotypes are a specific case of a general process.
- We place things into categories
- This includes categorizing people into groups
- Allows faster cognition, behavioral responses
- But can lean to errors or overgeneralization
Illustrations of white male professors are on the right side of the screen.]
Koji speaks: So in general or this process of stereotyping I think, well what we're talking about today is a part of a larger process of what we do cognitively just to make sense of our world, right. So this is a larger process of us putting things into categories, you know, and when it comes to people placing people in different groups, right, and the reason why we do this at a cognitive level is because it just makes it faster for us to interpret information, right.
[Video: Koji speaks to crowd]
Koji speaks: So you could imagine a world where we didn't do any of this , right, and we just, you know, got all this information into our eyes and we had to like in that moment decipher what was going on. That'd be a lot, that would be a cognitive overload for you everyday, right. So our brain uses stereotypes and in a larger sense it makes these categories, right, to make it easier for us to understand and interpret the information that we are receiving.
[Video: A graphic says:
Stereotypes are a specific case of a general process.
- We place things into categories
- This includes categorizing people into groups
- Allows faster cognition, behavioral responses
- But can lean to errors or overgeneralization
- E.g. when cultural images of professors don’t match reality
Illustrations of white male professors are on the right side of the screen.]
Koji speaks: Now the downside to this is oftentimes it could lead to errors. It could lead to overgeneralizations right, so this is something that we just have to keep in mind, you know. So yeah, especially when the cultural images don't match the reality.
[Video: A graphic says:
Do stereotypes effect hiring?
- Audit study method: a type of field experiment
Two sample resumes are below, with only the names— Brian Miller and Karen Miller—being legible]
Koji speaks: Okay, now why are we talking about this? Okay we're talking about this because of hiring.
The big question is do stereotypes and biases on those stereotypes affect hiring decisions. And the short answer is, yes they do.
So first I'm going to just walk through a little bit of the evidence from audit studies. And I think most of you probably have heard about audit studies. If not, what it is, it's a field experiment where experimental pros send out send out equivalent resume, so tested to be equivalent, but they’re varying certain characteristics on those resumes that they want to study, right? So they might vary the race of the applicant, they might vary the gender, sexual orientation, a lot of different, I've seen one with military experience or not, right. People are testing a lot of different things to see if those characteristics will affect the eventual hiring decision.
[Video: A graphic says:
Do stereotypes effect hiring?
- Audit study method: a type of field experiment
- Apply to actual jobs
Two sample resumes are below, with only the names— Brian Miller and Karen Miller—being legible]
Koji speaks: And what's nice because it's an experiment, a field experiment, right, you could say that there's a causal relationship between the gender or race or whatever characteristic you're interested in and the final outcome. Now typically here, oh and it's also important to realize these are, they're applying to real jobs in these audit studies, right.
[Video: A graphic says:
Do stereotypes effect hiring?
- Audit study method: a type of field experiment
- Apply to actual jobs
- Use fictional resumes, randomly vary two or more characteristics of applicants
Two sample resumes are below, with only the names— Brian Miller and Karen Miller—being legible]
Koji speaks: In the old days of, you know when well they've done audit studies for a while, but Diva Pager was a sociologist that really brought them to the fore. And they would send out actual people, like she would train people to go out to job sites, like and there was, if you read the old studies like it's really interesting because they she would train, you know, they would make sure that they're, they're acting the same, talking the same, that they rated them in terms of attractiveness being the same, right.
So nowadays we don't do that as much in terms of real people, but we send out resumes, which makes it a little easier. You can send out a lot of different resumes, right, so you're varying a few characteristics again. It's nice because you can get that causal effect.
[Video: Koji gestures to the resumes on the screen]
Koji speaks: This would be an example right. Everything is the same on the resumes except the gender’s different.
I'm running an audit study right now varying gender and race. And we, you know, how do we do that, we do it essentially just by the name, right. And there's, we can find very large effects by just changing the names on their resumes.
[Video: A graphic says:
Do stereotypes effect hiring?
- Audit study method: a type of field experiment
- Apply to actual jobs
- Use fictional resumes, randomly vary two or more characteristics of applicants
- Measure callback rates
Two sample resumes are below, with only the names— Brian Miller and Karen Miller—being legible]
Koji speaks: The measurement in these audit studies are typically the callback grades right. So this is again, a caveat here is that it's nice because you get a causal effect, but the outcome variables is just that first stage of hiring so just the callback stage. So you gotta you know keep that in mind it's not looking at the final hiring decision, it's just that first callback stage, although the callback stage is important.
[Video: Koji speaks to crowd]
Koji speaks: All right, so what did we find in general, based on decades of audit studies? A very clear disadvantage for African Americans in the U.S. And this is one of the most constant findings over time. I remember there was a review article maybe a few years ago just looking at all the oddest ways when it came to race since the 1980s right, when they first started doing this. And one of the most constant findings is that there is just a penalty for being African American in hiring overall right. And it doesn't matter your education, your experience, across all sort of dimensions we see this effect.
[Video: A graphic says:
Do stereotypes effect hiring?
- Audit study method: a type of field experiment
- Apply to actual jobs
- Use fictional resumes, randomly vary two or more characteristics of applicants
- Measure callback rates
- Disadvantages for African Americans, women, skilled immigrants, gay men, among others
Two sample resumes are below, with only the names— Brian Miller and Karen Miller—being legible]
Koji speaks: There's bias against women at the at the callback stage, skilled immigrants, for example there was one in 2011 looking at Asian immigrants, it was in the Canadian context but the effects were quite large. Gay men and others, right. There's just, we just find that there's a lot of discrimination and it depends on the sort of characteristics that are in that resume.
[Video: Koji speaks to crowd]
Koji speaks: Actually one that's missing from here that I forgot to put on here is a motherhood penalty. In fact that's a huge penalty in hiring, maybe even more so than just a straight gender penalty. A motherhood penalty is quite large. Okay so those are all you know what I think are some of the best evidence of bias and hiring. Although again, there's some caveats because it's that first stage of hiring. But there's some other evidence I think is important. So there's a classic study by Golden and Rouse that looks at hiring in orchestras. All of you have probably heard about this if you've seen any lecture about bias because it’s the first thing that they talk about but it's important I think. It's a very innovative type of study.
[Video: A graphic says:
Some other evidence
- Orchestras are more likely to advance/hire women when using screens/curtains for auditions
- Among 3000 R&D scientists and engineers, men rated more positively by managers
- Net of education, publications, patents, experience, personality traits, relational skills
- Among 700 NSF/NRC postdoctoral fellowship winners, 72.8% of women and 12.9% of men report discrimination
There is an illustration of orchestra performers performing behind a curtain on a stage]
Koji speaks: They were they were studying orchestras. Orchestras are historically very male-dominated right and what they found was that, you know orchestras that started using a blind audition right? So people go up and play their instrument and they put a blind curtain in front. They saw that those orchestras were starting to, women were more likely to get advanced in the hiring process, eventually become part of the orchestra when those screens and curtains were used right. So the idea is that the the the people that were judging, they couldn't really tell the gender of the person and so that gender bias went away in the evaluations.
[Video: Koji turns to the audience to ask the crowd a question]
Koji speaks: Now what's interesting here is that sometimes there was still a gender bias even though they were using a screen right. So why do you think that was? Why do you think you could still make it, still tell there was a gender? They could still tell the gender of the person playing but why do you think that was bad? They still have the screen up. The shoes, right. They could hear the click clack of the the high heels. Oh there you go. Hey so this is a classic, right. And so this is the thing, right, then they put the carpet down and then the the gender difference went away, right.
[Video: A graphic says:
Some other evidence
- Orchestras are more likely to advance/hire women when using screens/curtains for auditions
- Among 3000 R&D scientists and engineers, men rated more positively by managers
- Net of education, publications, patents, experience, personality traits, relational skills
- Among 700 NSF/NRC postdoctoral fellowship winners, 72.8% of women and 12.9% of men report discrimination
There is an illustration of orchestra performers performing behind a curtain on a stage]
Koji speaks: So some other examples. There was a study of 3,000 R and D scientists and engineers right and net of any possible, you know, measures of productivity among these educators, this is across all different corporations, education, publications, patents, experience, even measured personality traits and relational type skills, men were still rated more positively by managers. Again, this is correlational. You can control for a lot of stuff, but it still fits this sort of pattern. And this is the perception of discrimination among 7,000 NSF postdoctoral fellowship winners where if you looked at the difference between perceived discrimination among men and women, about 75% of women, say they perceived discrimination in their tenure, right. Whereas there are only 13% of men. So that is perception, but again this is part of this larger story of a bias in these sort of selection processes.
[Video: Koji speaks to crowd]
Koji speaks: I guess this goes back to my first example. So you know stereotypes also affect because it affects perception, affects how students perceive us, right, so here you're thinking about teaching evaluation specifically, right. How students perceive us really depends on or is affected by stereotypes, right. And so I'll just talk about a few studies, right.
[Video: A graphic says:
How students perceive us
- A study at Waterloo found:
- Male and female instructors gave similar grades
- Male and female instructors’ ratings were close to equal when both gave high grades
- But female instructors’ ratings were much lower than male instructors’ ratings when both gave low grades
- Think about long-term implications, e.g. women asked to prep additional courses]
Koji speaks: So this one was done at Waterloo and what they found was that they were looking at the evaluations of men and women across different fields and what they found was that male and female instructors would give students, you know, equal grades. So essentially it didn'tmatter the gender of your instructor, you would probably have the same chance of getting an A no matter the gender of the instructor. Now the evaluations of the male and female instructors were the same if those students did well in the class, right. So they would they would evaluate the male and female instructors the same. But among the students who received the very lowest scores, so if they didn't do that well in the course, then you see this gender difference in terms of evaluations, right. So then the women about the women instructors are not very good teachers relative to the men, right.
[Video: Koji speaks to the crowd]
Koji speaks:And so one thing to think about here, you know, you know what are some of the long-term implications when it comes to like how do we use this information. You know, this a woman who might get, you know, worse evaluation from students maybe her her department might think, oh she wasn't a good fit for that that that course, maybe she should prep another course, right. So the idea is here just think about what is happening with these evaluations. There could be some bias in terms of how people are perceived.
[Video: A graphic says:
How students perceive us
- A study at Waterloo found:
- Male and female instructors gave similar grades
- Male and female instructors’ ratings were close to equal when both gave high grades
- But female instructors’ ratings were much lower than male instructors’ ratings when both gave low grades
- Think about long-term implications, e.g. women asked to prep additional courses]
Koji speaks: I mean what these studies really make me think, I think we all sort of know this, right, if you teach undergrads. I remember the first time I taught in grad school I immediately saw that there was a gender difference in how I was being perceived, right.
[Video: Koji speaks to crowd]
Koji speaks: I could and I was willing a grad school a grad student, but you know I remember I just realized that students would take me seriously. But you know I could say whatever and they were just taking me seriously, compared to my female colleagues who would say you know, the students aren't taking me seriously, we're teaching the same thing and they were probably more confident than I was, but you could just tell, right, and they would have to dress a little better, they would have to act more professional, whereas I had leeway not to and that was simply because the people were using gender stereotypes to interpret our competence levels.
[Video: A graphic says:
How recommenders write about candidates
- A study of 300 recommendation letters at a US medical school found:
- Women more likely to have very short letters, men more likely to have very long letters
- More doubts raised about female candidates
- Difference in grouping of possessives:
- ‘her teaching’, ‘her training’, and ‘her application’
- ‘his research’, ‘his skills and abilities’, and ‘his career’.
Koji speaks:Okay so where else do we find evidence of gender bias and in places that are applicable to hiring here. Well one is that we see there's very gender differences in terms of how recommenders are even talking about those people that they are recommending, right. So this is a study of 300 recommendation letters to us, med schools, right. And you know even taking into account the the productivity levels of the candidates themselves or the people that are applying to these jobs, if you just looked at the letters and how they’re worded differently, women tend to tended to have shorter letters than men, right. Men tended to have longer letters so their recommenders were willing to write more for men in those letters, the recommend the recommenders would bring up more doubts about the the candidates ability or ability to enter academia or I guess here be a doctor and then even in the style of writing they found that there was a gender difference .So for women they would often talk about her teaching, her training, her application, right. For men it was often his research, his skills and ability in his career, all right. So all this kind of stuff, even though it might seem nuanced and small I mean that sort of stuff matters when you're reading through an application very quickly right. And so we do see these gender differences here in terms of evaluations or recommendations.
[Video: a graphic says:
How hiring committees perceive us
- Qualitative case study of hiring at a private research university
- Observed candidates in social sciences, humanities, natural sciences
- Committees focused heavily on relationship status
- Men assumed to be moveable, not women
- Eran (committee chair): She seems to have the highest potential based on limited information
- Cole: Her market is good so far. Has (names top 10 R I department in another city) and (top-10 R I department) offers; (top-5 R I) liked her.
- Marco: Some people think it’s unlikely she’d come because of her boyfriend. He’s a (names the boyfriend’s occupation) and (the city where her other offer is) is really the best for that.
- Sharon: I want to put the (acceptance) probabilities on the board. (She writes a .5 probability next to Esther’s name). She told me that we are better than (her other offers). But we need to work out her husband. If it were up to her, she’d come here.]
Koji speaks: Okay so what about within the hiring committee? So just as a side track I, so part of my research looks at hiring committees although in a corporate setting, looking at hiring of software engineers and they're specifically fine how gender really does come up during but when people are evaluating people's skills right so what I find is that even among men and women at these companies that are evaluated equally in terms of technical ability when it comes to the deliberation what becomes emphasized are typically gendered ways or gendered strengths and weaknesses. Right now I haven't, again I haven't sat in a hiring committee here at the University. I'm a new professor. But other people have, other, other sociologists have sat in on hiring committees and what they have found was that what is a very salient conversation, one that differs by the gender of the candidate is whether they are movable or not and this is specifically about partners right. And so you know if, if the candidate is a female candidate and she has let's say a male partner, the conversation is about that relationship status and whether the man is willing to move or not, right. That's where the conversation is. Is he really gonna move, because the assumption is he has people, he has a really good job, he's in a great place. What what can we do to get him to move. If the candidate is a male and he has a female partner that conversation about what the partner is gonna do is not really on the table, right. This, this study was actually, didn't Harvard by the way I know this so right so the conversation about the relational, the, the partner changes based on the the gender of the candidate so this is, this is you know, I'll just read the beginning of this right. They're talking about a female candidate here in this evaluation setting. One person says she seems to have the highest potential based on limited information. Another person says her market is good so far, she has you know basically all these other offers., they really liked her. Some people think it's unlikely she'd come because of her boyfriend, he has a really good job, he's in a great city for that job, right, and so they're talking, they're really talking about the boyfriend’s job and whether they can get him to move. That conversation is quite different if it's a male candidate because the assumption is, right, that women would be more willing to move for their male partner.
[Video: A graphic says
Reducing the influence of implicit bias
- Stereotypes are an unconscious shortcut
- More likely when tired, distracted, rushed
- Training and good intentions alone unlikely to be sufficient
There is a photo with the caption “Depiction of Ulysses and the Sirens by John Williams Waterhouse]
Koji speaks: So those are just some areas in which we think stereotypes and bias can affect things are relevant to your hiring. What goes on in the hiring committee.
So what are some ways to reduce the role of implicit bias? Well first it's important to go back and just think about what we're talking about here, right. So stereotypes are just this unconscious shortcut that we use, we use them more and more, just distracted, when we're stressed, when we're rushed, right and so that's sort of the context in which all this stuff happens.
Another thing to keep in mind is that typically training doesn't work.
[Video: Koji speaks to crowd]
Koji speaks: So just listening to me talk is not going to do anything, right and this is this is a known thing.
If you remember like Starbucks did like this unbiased conscious training, I would love to see if there are any effects on that because there probably are not. And so we don't find em, you know there might be like short-term effects. No let me could talk about that afterwards, the long term, no. Right, so just training, just telling people about it doesn't really do anything.
[Video: A graphic says
Reducing the influence of implicit bias
- Stereotypes are an unconscious shortcut
- More likely when tired, distracted, rushed
- Training and good intentions alone unlikely to be sufficient
- (some evidence that intergroup contact, perspective taking help)
- What concrete steps are useful?
- Consider picking 1-2 to implement this year
There is a photo with the caption “Depiction of Ulysses and the Sirens by John Williams Waterhouse]
Koji speaks: What does do, what does something right is if you instill some practices. Now what I'm, you know not I'm not saying to like change entirely your hiring structure, but just think about, you know, one or two things that you could sort of tweak, right. So these are very doable type things.
[Video: A graphic says:
Accountability (Lerner and Tetlock 1999)
- The implicit or explicit expectation that one may be called on to justify one’s beliefs, feelings, and actions to others
- What makes you think this person is qualified or unqualified? What did you take into account when making this decision?
- Leads us to devote more cognitive effort to evaluating candidates
- More effective before final decision has been made
- Structure searches to include explicit justifications for candidate evaluations]
Koji speaks: Okay so what's the first category of practices that we know have an effect on reducing bias? One is just general accountability, right. This is, this is big and there's a number of studies that show the importance of accountability in hiring decisions and what that means is that accountability is this implicit or explicit expectation that you might have to justify your evaluation or justify your decision, right. So you know, sort of explaining what, you know what makes this person qualified or unqualified, right. What did you take into account when you made this decision or what are you thinking about, right when you're making this decision. And what this really does, it's very simple, right. It just makes you think a little bit more about how you about your decision making process right. How you're evaluating things and then making you think like is our gender stereotypes, our racial stereotypes, our immigrant stereotypes, sort of influencing this evaluation process. Just think about it a little more, right. Now what's important here is you know this obviously is more effective before you do the decision. If you do it afterwards, that's post hoc rationalization, right. That's just you make a decision and then you rationalize. So you know talking it through before, having some accountability for you know making sure that people are applying criteria equally, for example, that should come before the evaluative or the actual hiring decision right. So the recommendation here would be to structure the search to include explicit justifications for your decisions or for the candidate evaluation, right. And you might already be doing this, which is good, okay.
[Video: A graphic says:
Transparent evaluation criteria
- Standards are sometimes applied differently across candidates
- In Sweden, women needed to publish 2.5x more papers to be judged as worthy of a postdoctoral fellowship as men (Wenneras and Wold 1997)
- Raters may also shift emphasis on qualification (experience or education) to match favored candidates (Norton, Vandello, & Darley 2004)
- Clearly define search criteria in advance
- E.g., potential vs. accomplishments, how teaching is weighted, which journals are equivalent, movability of candidates
- Monitor application of criteria
- In fact, consider making this the job of a search committee member
Koji speaks: So what are some other just very basic things that we know help? Well one is just having transparent evaluation criteria like what are the criteria for hiring someone, right. Making those explicitly clear because we know that if they are not clear oftentimes the criteria can change based on the person that you're evaluating right. So you know, one study, it showed that women needed to publish two and a half times more to get a post doctoral fellowship.
So the things that I write, the the actual evaluations needed differed by gender.
Another thing that we see a lot of times is that people will switch which evaluations are more important or which criteria are more important based on the person that they want to hire, right. And so we see this a lot in experimental work where people will actually shift the criteria because they want to hire a certain person and so they're shifting what they think is important based on, you know, what can get that person in, right. So setting, making everything transparent, making the criteria transparent beforehand and understanding that these are things that happen, right, are important to reduce bias in your search.
So part of that is clearly defining the search criterion in advance. How much how much are we going to weigh potential versus accomplishment, how much is teaching weighed, you know thinking about which journal articles are more important, or which journals are more important for your department and so forth, and of course movability which you know we just talked about could have some gender dimensions, and what's, you know what we find is effective here is that you know having someone like monitoring this, like putting someone in charge of you know is our criteria transparent, are we sort of applying it equally to different people.
And this sort of overlaps at the last point of accountability with transparency.
[Video: A graphic says:
Critically analyze supporting materials
- If demographics, net of candidate quality, affect…
- Teaching evaluations
- Recommendation letters
- And other evaluation criteria
- Consider how these factors are weighted in evaluation
- E.g., lower teaching evaluations, particular combined with lower grade distributions may reflect student stereotypes rather than instructor quality]
Koji speaks: And this sort of overlaps at the last point of accountability with transparency. Okay. And then, you know this is a very simple one but just critically analyzing the materials they're using based on what we all know in terms of how bias and stereotypes can affect, you know teaching evaluations, recommendation letters, and so forth right. So just being cognizant of you know, demographics could affect or demographic characteristics could affect people's materials in ways that we know of because of research right again like you know if you're looking at someone's teaching evaluations and you notice that the worst students tend to score the female candidate worse, well you know maybe there could be something going on with bias there and so yeah considering how these factors are weighed into the evaluation knowing that bias could have affected the results.
[Video: A graphic says:
Additional suggestions:
- Attempt to avoid last-minute decision making, rushed searches
- Facilitate job talks effectively to avoid some candidates being interrupted more than others
- Measure outcomes, progress over time (e.g. applicant pool, offers, acceptances, etc.)
Koji speaks: So some final additional suggestions and this goes back to the idea that stereotypes are most used unconsciously when we are busy, when we're frustrated, when we have somewhere else to be. Try to avoid last-minute decision-making. I know this seems very basic but it does have an effect on you know, how potent stereotypes are so try to avoid those last-minute decision-making rush searches.
Another thing that we find effective is you know making sure that during job talks that you know well the culture is different in different departments in terms of how much you interrupt the job, person that's giving the talk, but just make sure it's even across all candidates right. Because we know that people are more likely to interrupt with interrupts women during job talks right so just be cognizant of that. See if there you can create some sort of equality in terms of interruptions during peoples job talks. And another good thing I think is just to measure your progress over time right. You know what do the characteristics of the applicant pool look like, what are the characteristics of the people that eventually get a callback, get an offer, get accepted right. Because then you have some data that you could use, maybe not for that search but maybe for next time to say okay these are sort of the pressure points, these are the trouble
areas that we might need to work on. So measuring is always, always good okay.
[Video: A graphic says:
Thank you!
- Stephen Benard
- Indiana University
- sbenard@indiana.edu