How to structure the interviews process to limit bias in hiring

by Tara Robertson

So, you’ve done the work to take an honest look at your work culture and you’ve started to make systemic changes to make your culture more inclusive. Yay! You’ve been telling the stories about what it’s really like to work at your company, you’ve written thoughtful job postings, and you’re getting a wider range of people applying than before. Woo hoo! 

This post will address the next step: how to design an inclusive interview process, from the top of the funnel to hiring your next employee. 

Tell candidates what to expect in your hiring process

Be up front with candidates about the whole hiring process. How many stages are there? How quickly should they expect to progress to the next stage? Will they be given feedback along the way? Will there be a coding challenge, that they’ll need to allocate time to complete? Who should they contact if they need to request a disability accommodation? Will the whole process take 2 weeks, or is it closer to 2 months? Being explicit about the hiring process helps set candidates’ expectations, puts them at ease, and removes the advantage that some might have with certain types of companies or in certain industries. 

Angela Riggs makes a compelling argument for sending the interview questions to the candidate ahead of time. There are very few jobs that involve having to think on your feet and be eloquent in high pressure situations. Maybe it’s appropriate if you’re interviewing to be a game show host, but it’s not relevant for most jobs, so why is this practice so common? 

I like how Shopify spells out their entire process and explains what they mean by The Life Story interview. 

job candidates comparison: hand with magnifying glass analyzing a shortlist

Shortlisting resumes 

The recruiter and hiring manager should have already aligned on the key criteria for the role when the job posting was written. This post has good advice for hiring managers on division of labor and building synergy with recruiters. Generally the recruiting team will shortlist the applicants that will move on in the selection process. Shortlisting applications for a recruiter screen is usually the first step in the hiring process where bias can creep in. People on recruiting teams can have someone on else on their team double check the candidates who’ve been screened out to ensure you haven’t mistakenly screened out qualified candidates.

There have been many resume studies, where the same resume is sent out to a bunch of jobs with only one detail changed. Here’s the findings of 3 resume studies: 

  • “Jamal” needed eight additional years of work experience to be considered as qualified as “Greg”. (2004 study) 
  • Jennifer was offered a starting salary of $4000 less than Greg. (2012 study) 
  • A queer woman who volunteered with a LGBTQ+ organization received 30% fewer callbacks than a straight woman who volunteered with a progressive student organization. (2016 study) 

Skills evaluation 

In many industries a skills evaluation is a common part of the interview process. For junior tech roles (where a company could get more than 1000 applicants) the technical screen will be scheduled early in the process to weed out candidates. For assessments like a writing test or presentation, these are typically scheduled later in the process to compare your top candidates’ skills. 

Without a skills assessment you are likely basing your perception of a candidate’s skills on their reputation, their degrees (or lack of degrees), the university they attended, or the prestige or name brand recognition of the places they worked. 

However you’re using a skills evaluation it should be as close as possible to “a day in the life” of someone on that job, designed to be done in a minimal amount of time, and to mitigate bias you should determine the evaluation criteria when you make the assignment. 

Technical screen

Technical screens or coding challenges are an objective way to evaluate a candidate’s skill. Like any assessment you need to be thoughtful and intentional about the design. Emily Thompson, a data science leader, shares “I really like interviews that test less for “textbook” knowledge, which biases more towards people who have had extra time to study, and test more for actual skills those people can bring to the table. The way I’ve done this in interviews has been to structure them as close as possible to “a day in the life” of someone in that role. Test for ability, not for memory.”

Engineering Director Pete Holiday, argues that there’s a better way to do this. He outlines that asking good interview questions will help you learn about the candidate’s direct experience. Having the candidate review your code will help you “identify candidates who can level-up your whole team’s code and make your office a better place to work in the process”. Finally Holiday says you should design a system together. He says that this works well across different amounts of experience. 

In this post, Terra Field, shares her frustration at what she calls “interview theater”. She states that candidates should be given the option to choose between live coding exercises or a take home assessment. Field also points out that as a senior engineer with almost 20 year of experience working on Linux and Windows based servers, do people think she’s faking knowing Python? Like Holiday, Field says it’s about asking the right questions. If you’re involved in hiring for software engineers her post is worth reading and reflecting on the candidate experience at your company. 

Interview scorecards

When you wrote the job post, you clearly stated the must-have criteria for the job. The interview questions should dig into these criteria. Use the same questions for each candidate and use a scorecard to minimize bias. Align on the differences between a decent answer, a good answer and an awesome answer. Google’s rework site has an example of scoring rubric for the imaginary role of Underwater Basket Weaver. 

Be consistent about which interviewer asks which questions across the interview process. Use a neutral opening question to put the candidate at ease, instead of commenting on points of commonality (like attending the same university, or a shared hobby). Schedule time right after each interview for the interviewer to fill out a scorecard and rank the candidate on each specific criteria. I know it’s easier in a busy environment to schedule four 30 min interviews over two hours, but recency bias means that you’ll be clearer about evaluating the 4th person than the 1st. At this stage, interviewers should not be backchanneling with each other about who is good and who isn’t–you risk groupthink, or the most powerful person in the team swaying others. 

I’m a huge fan of Dr. Joan Williams’ work on interrupting bias. Here’s a few of the bias busters she’s identified for hiring: 

  • Make sure to give everyone—or no one—the benefit of the doubt.
  • If you waive objective requirements, do so consistently and require an explanation.
  • Don’t insist on likeability, modesty, or deference from some but not others

Culture fit

Hiring for “culture fit” is something you should avoid. Culture fit is usually shorthand for people like us, or someone I’d like to have a beer with after work or hang out with on the weekend. In tech there’s usually an assumption of white maleness that excludes women, people of colour and some white men too! In libraries I observed that culture fit meant nice, white ladies like us, which excluded people of colour and women who weren’t nice or ladies. 

The reframe of “culture add” instead of culture fit can be useful–in addition to meeting the must-haves, who brings something that we don’t already have on our team?

If hiring people who are aligned with your company’s values is important then you’ll need to be explicit about those values and craft the right questions to get at those values and, like for everything else in the interview, determine the rubric for scoring answers. 

Take care with details, they matter

My name is often mispronounced in North America. Greenhouse, an applicant tracking system (ATS), has functionality where applicants can record their names. LinkedIn has similar functionality so you can record your name and I have this link in my email signature. If you’re not sure how to say someone’s name ask, “Before we get started can you tell me how to say your name? I want to make sure I’m pronouncing it correctly.”

The microaggressions I experience are small compared to BIPOC people with non-English names. I love this story from actor Uzo Aduba when she told her mom that she wanted to be called Zoe. Her mother replied “If they can learn to say Tchaikovsky and Michelangelo and Dostoyevsky, they can learn to say Uzoamaka.”

The same goes for the applicant’s pronouns. If you’re not sure, ask, or do your homework ahead of time. Being misgendered is a crappy experience, especially when you’re trying to shine in a job interview. As the interviewer you can share your pronouns with candidates, but don’t put the candidate on the spot to share theirs–they will if they’re comfortable. 

As I mentioned in the  post about job postings, if applicants requested accommodations ensure those are in place. The applicant is the expert in what they need. For example if a Deaf person requires CART transcription for a Zoom interview, booking an ASL interpreter is not adequate or helpful. It’s possible that this Deaf person doesn’t know ASL. 

All of these details demonstrate if you are paying attention, care, and a bit about what your culture is really like.

Be kind

The best experiences that I’ve had as a candidate were the ones where there was clear communication throughout the process and where it felt human. Even when Talent Acquisition teams are slammed it’s important to not take shortcuts on clear and empathetic communication throughout the process. Take the time to wrap things up with candidates who made it to the final round who didn’t get the job. It feels really impersonal to get the templated rejection email form an ATS after you’ve gone through many interviews with a company. Avoid ghosting candidates–it’s a good way to ensure that they won’t apply to work for your company again and it might impact if they want to be a customer too. 

By the time I’d made it through five rounds of interviews for my dream job I was at the stage where I was imagining relocating and what I was going to wear on my first day. When I found out I didn’t make it to the final round, I was disappointed. The recruiter shared feedback from some of the interviews about where I was strong and where they thought I needed more experience. The feedback was really valuable–I’d love to see more companies do this. 

Final post in this series

The last post in this series will be an introduction to analyzing your hiring pipeline data to understand where you need to focus your efforts to ensure an equitable hiring process. 

Thank you Emily Thompson, Angela Riggs, Celia Hodent and Annie Bélanger for feedback on this post.