This is the last post in a five part series that has looked at assessing how inclusive your culture is, employer branding, writing a job posting that attracts a broader range of applicants, and how to structure the interview process to limit bias. How can you use data to identify where your hiring process needs improvements? 

I’m assuming that your organization has a DEI strategy, that your senior leaders are bought in, aligned and leading in this area and that you have a measurable representation goal to increase diversity or a diversity hiring goal. If your company doesn’t have these things it doesn’t mean you can’t use data to audit your hiring process but your time is probably better spent doing those other things first.

This post is a long one, so you might need another coffee or a snack. First, I’ll cover why you should bring a data informed approach to hiring and how to do that. Next, we’ll look at two fictional examples where the pipeline drops off in different ways and different tactics you could use in each scenario. Then I’ll share a way to build relationship in BIPOC communities that is not extractive. Finally I’ll explain why you need to audit your referrals and the importance of disaggregating race/ethnicity data. Let’s get into it!

Auditing your hiring pipeline data

If you’re committed to diversity, hiring is an important way to shift representation. Data is an essential tool for most organizational functions. For example, sales and marketing teams rely on data to understand their conversion funnel and track progress on their goals. If a marketing team kept missing their goals and excused their misses with, “But we’re really passionate about marketing.” that wouldn’t be tolerated for very long. And yet, in some Talent Acquisition teams, missing goals for recruiting underrepresented talent is an ordinary event. To make progress on diversity, Talent Acquisition teams must start using the data from their hiring pipelines in a more effective way. The first place to start is with an audit of your pipeline data.  

 

So, how do you audit your hiring pipeline data?

In MIT Sloan Management Review Elizabeth J. Kennedy argues “racial equity strategies must be systemic, race-explicit, and outcome-oriented if they are to succeed”. In her research she found that: 

…while many employers expressed a strong desire to create equitable and inclusive workplaces, few had identified specific, measurable outcomes consistent with those goals. Even fewer had analyzed their policies and practices to determine whether they were helping or hindering their efforts. This makes it difficult, if not impossible, to measure progress.

Before you start analyzing data from your hiring funnel, you’ll need to make sure you’re surveying candidates demographic information during the recruiting process. Check your ATS (Applicant Tracking System) to understand what demographic information you’re asking of candidates, and when. At a minimum, you want to ask candidates to self-identify their gender, race, and ethnicity.  It’s important to tell candidates why you’re asking for their demographic information so that they understand why you’re asking for this info and how it will be used by the company. It’s also important to have “decline to state” as an option, as not everyone wants to share this information. If candidates decline to share their demographic data in the recruiting process, don’t guess! You can’t presume to know someone’s identity through visual ID, name, or any other cue.  As a mixed race woman with brown skin I know just how uncomfortable and awkward it is when someone makes an assumption, which is almost always incorrect, about my race. 

Analyzing your hiring funnel data with a robust approach is a key part of tracking if and where candidates from underrepresented backgrounds are dropping out of the pipeline, and where you need to strategically invest more time and energy into pipeline development and improving your hiring process.

Here are  the five steps to follow to audit  your hiring pipeline data:

1. Identify the main stages in your hiring process. The examples below have 6 stages.
2. Calculate the percentage of underrepresented people for each stage. The first example looks at women for a software engineering role. If there were 1000 applicants for this role and 400 were women, then the percentage of women at the application stage would be 40%. 
3. Look at where the percentage of underrepresented people drop off. 
4. Formulate hypotheses as to why the dropoffs are happening. You’ll need to partner with your recruiting team and the hiring manager to do this effectively. A hypothesis for Example 2 might be “Writing a job description that explicitly invites data scientists from other industries who lack tech experience to apply”. 
5. Test your hypotheses by seeing if your interventions shifted the next round of hiring. 

This is not an academic or scientific research, so if you’ve got a couple of ideas on how to shift things it’s ok to try them out together. The goal here is to leverage hiring to shift diversity, not to prove your hypothesis. This is an iterative process that you’ll want to do more than once. 

Here are two hypothetical examples to illustrate two different pipeline issues. 

Example 1: Software engineer role

Here is an example of looking at gender in the hiring funnel data for a software engineering role. 

Application

At the top of the funnel 40% of people applying for this posting are women. According to a 2020 study from AnitaB.org, the representation of women in technical roles is 28.8%. So, this is better than the overall representation of women in tech. 

Recruiter screen

At the next step, the recruiter screen, the percentage of women drops to 30%. In tech the recruiter screen is typically a 30 min phone interview where the recruiter shares a bit more context about the role and asks a few questions to see if the candidate meets specific must-have experience, and the candidate has a chance to ask a few questions too. 

Questions that go through my mind include: 

• Are objective assessment standards being used to evaluate candidates? 
• Are women being held to a different standard than men? Here’s some common biases that crop up in hiring.

Audit a sample of the people who were screened out to see if they were correctly screened out or if a mistake might have happened. 

Hiring manager 

At the third stage, where the hiring manager interviewed the candidates, representation of women dropped to 8%. 

This is the biggest drop off. I wonder:

• Are the recruiter and hiring manager aligned on the key must-have criteria? 
• Is the hiring manager slipping on the initial must-have criteria and looking to hire a unicorn?  

Team

Only 2% of the candidates who interviewed with people on the team were women. (For detail oriented folks reading this, I realize now that these percentages are improbable, as the team would need to be interviewing 50 people at this stage, but I’m trying to make a point, so be flexible on this). 

Final rounds

No women made it through to final rounds, so there was 0% chance that an offer went out to a woman. 

Observations

If I was partnering with the hiring manager and recruiting team on this I’d share the data and ask them their perspective about what they thought was going on. It’s likely that they have information and perspective that I don’t. One thing is clear in this scenario–it’s not a pipeline problem. 40% of the applicants are women, so assuming they are all qualified, the problem is with the internal processes and decision making. 

Recommendations

• Training – Did the recruiting team and the hiring manager have training on how bias plays into hiring? Joan C. Williams’ TEDx talk that describes five common patterns of bias would be a great component of a training. Adding in real life comments that we’ve heard during the hiring process that illustrate these types of bias and give the team some examples to practice evaluating candidates against the job description criteria. 
• Audit the candidates who were weeded out at each stage, and identify if one of these types of bias was present, or if the candidate was correctly weeded out. 
• Rotate the role of “bias monitor”, who has the responsibility to speak up when they hear bias  to meetings, between Talent Acquisition and the Hiring Manager. 

Example 2

Here is an example of looking at race/ethnicity in the hiring funnel data for a data scientist role. 

Application

At the top of the funnel only 3% of people applying for this posting are underrepresented minorities, an umbrella term for Black, Latinx, or Indigenous (American Indian, Alaskan Native, Native Hawaiian, Pacific Islander) people. 

According to the Harnham whitepaper on US Diversity in Data & Analytics, 49% data science and analytics professionals are white, 3% are Black, 6% are Latinx, 30% are Asian. This report doesn’t include Indigenous data scientists and analysts, so I’m assuming it’s close to 0%. So, 9% of the available labor market for data scientists are underrepresented minorities. 

Recruiter screen

At the next step, the recruiter screen, the percentage of underrepresented minorities stays at 3%. 

Hiring manager 

At the third stage, where the hiring manager interviewed the candidates, representation of underrepresented minorities remained at 3%. 

Team

At the next stage where candidates interviewed with members of the team, underrepresented minorities made up 3% of candidates at that stage. 

Final rounds

No underrepresented minorities made it through to final rounds, so there was no chance that an underrepresented minority was hired. 

Observations

In this example, the number of underrepresented minorities at the top of the funnel is lower than the labor market. So, people aren’t applying. This makes me think about the employer brand as well as the networks of the existing team. The sourcers on the Talent Acquisition team and the Hiring Manager will need to reach out to people in their networks.

If your Talent Acquisition team is all white, you have a problem. In addition to all the well researched reasons why diversity helps make us smarter, a homogeneous recruiting team will not have the breadth of networks and connections. Also: companies like Intel, Netflix and Cisco have shown that diversity on an interview panel helped mitigate in-group bias, and this logic makes sense for Talent Acquisition teams too. 

Tech companies often prefer to hire data scientists who are already working in the tech industry. Many great data scientists in tech come from academic backgrounds: biology, biostatistics, immunology, statistics, or from other industries, like finance. 

Recommendations

• Talk to Black, Latinx and Indigenous data scientists who are already on the team. What do they love about doing this work and their team? What impacts are they making? Where does this job fit in their career trajectory? Work with your comms team to tell those stories as part of your employer brand.
  If there aren’t any underrepresented minorities on your team, or if there’s only one person, take a good look at your culture. Is your culture set up for the next person to join the team and thrive, or will they have to navigate white people’s awkward discomfort and contend with having to clear a higher bar to be seen as competent? 
• Look at the job description–does it call out to Black, Latinx and Indigenous data scientists? Does it explicitly call out to data scientists outside of tech? 
• Work with sourcers to get more candidates in the top of the pipeline. Each sourcer who I’ve worked with brings their own special sauce to their work. These methods range from mining the ATS for previous applicants who weren’t hired (but already are interested in the company), boolean searching in LinkedIn, and outreach in Slack channels focused on specific communities, disciplines, or software languages. Consider building ongoing partnerships with professional organizations like SACNAS and NSBE to develop networks you’re likely missing. 
• Be intentional about building authentic relationships outside your core network. 

Build authentic relationships in community

Research shows that most of us have homogenous social networks. I had to laugh when I noticed that I’d hired another Asian woman with pink hair to help me run a series of focus group. Without a clear intention and a plan this is something that we all can easily default to. 

Dr. Dori Tunstall’s class Hiring for Decolonization, Diversity, and Inclusion in the Creative Industries was awesome (it’s being offered again in May 2022). She’s been the architect and catalyst for 3 successful cluster hires at OCAD U: two Indigenous cluster hires and one Black cluster hire. A cluster hire is when you hire new employees in groups rather than individually. Dr. Tunstall’s model of systems change has been so successful that many universities across Canada have copied it to shift representation of faculty. Even as the only Black Dean of a design program anywhere in the world, she describes how she showed up at an event that’s important to Black creative communities every week, for over a year. She listened to what the community wanted and needed, and how the community would want to use OCAD U so that the community would flourish.

Where are you spending your time? Where do you choose to speak? What conferences do you attend? What Slack channels are you active in? What organizations do you volunteer with? It’s interesting to map this on an individual level and it’s even more powerful to map this out for a team, especially a senior leadership team. 

Where should you be spending time? This will depend on your industry, geography, communities that you’re looking to connect with and your interests–your DEI team or ERG leaders might be able to offer suggestions. When you show up, spend time building authentic relationships, listen to what is important to people and the community, where can you extend your social capital or how can your company help show up and support the community? Building relationships takes time, so block time on your calendar and make an accountability plan. 

In Mapping Exclusion in the Organization, Carboni, Parker and Langowitz state

Networks are how people learn the unwritten rules of success, hear about job and promotion opportunities before they are posted, and — most critically — build a level of interpersonal trust and rapport with their contacts that translates into a willingness to pick up the phone and vouch for someone’s capabilities.

While their research is looking inside an organization, I’ve seen how this is also true within industries. By building meaningful, trusting relationships in the community you can be a node to share information and opportunities. 

Another way to level the playing field is to have an open Q&A session where candidates can learn more about the job and ask any questions they might have. Candidates who are part of the hiring manager’s existing network still have an advantage and this is a great way to try and make that access more equitable. Kudos to James Madison University Libraries for doing this:

Join @JMUlibraries & my dept. as our 2nd Business Librarian! We’re holding an info session Fri. 1/7 at 1:30pm EST on Zoom. Attendee list will be hidden from participants & questions can be asked anonymously. Any questions, ask me! https://t.co/JqydK99PEZ & https://t.co/OUvPscOULY

— Alyssa Young (@alyssahvyoung) January 4, 2022

Referrals

Many companies have referral programs where current staff vouch for candidates who then get some kind of preferred treatment–in some organizations this means candidates who were referred in skip ahead to the interview process, or that there’s a guarantee that a recruiter looks at their application. Staff often receive a bonus if the people they referred are hired. 

There’s several reasons why referral programs are important. Leveraging an employees’ network can be an advantage in a competitive talent market. Having employees refer candidates might also mean that that person sponsors that person in the future.  I’ve heard people say this can also reduce time to hire but haven’t seen any data on this. 

Again, given that most of us have fairly homogeneous social and professional networks. If your company isn’t very diverse then these types of referral programs are likely replicating the existing lack of diversity. 

It is essential to crunch the numbers to understand if your referral program is helping or hindering your diversity efforts. 

Disaggregate race/ethnicity data

In example 2 we looked at the hiring pipeline for underrepresented minorities for a data science role. Looking at non-white applicants as a group can help made some patterns visible but it also conceals differences between different racial groups. 

When people in tech talk about underrepresented minorities, I wonder what the differences between Black, Latinx and Indigenous candidates look like. 

When people talk about hiring BIPOC staff I always wonder what the hiring pipeline looks like specifically for Black candidates and for Indigenous candidates. I wonder if there’s a difference between Asian and white candidates. 

Be thoughtful about when you aggregate data and when you disaggregate it to surgically identify trends. 

Job board recommendation

This whole series digging into different parts of inclusive hiring started with the question that I’m often asked: “Which job board do I post on to reach ‘diverse’ candidates?”. I hope you now see that there’s a lot more to an effective hiring strategy than simply finding the right job board and posting job ads there. 

That said, I can recommend Diversify Tech as a great site with a variety of resources for underrepresented folks in tech, including a job board. Kudos Veni Kunche for doing great work. 

Thank you to Katrina Jones and Emily Thompson for helpful feedback on this post.