It seems that 2016 was the year when recruiters fixated on improving candidate engagement and the candidate experience. There were hundreds of articles and reports on candidate engagement, awards, and at least one conference devoted entirely to discussing it. There is nothing wrong about wanting to make the experience a candidate has friendlier or in making it easier to apply for a job. I know all too well how frustrating it can be for a candidate to navigate the typical career site, learn about and apply for a suitable position, and then find out that all his or her work went into a black hole. Most career sites and recruiting processes are designed for administrative ease or for the recruiter and are not seamless or simple for a candidate.
But, if we are going to improve the candidate experience or make them more engaged, wouldn’t it be nice to start with a clear definition of what engagement or experience are? And wouldn’t it also be useful to know if a positive experience results in any tangible improvement in the number of hires made per candidate or in the quality of hire (another ill-defined term)?
If recruiters and our profession ever hope to taken seriously, we must become more scientific and disciplined in what we focus on. We need measurable definitions, large samples of data, useful metrics, and longitudinal studies to support our claims. Today we do not have that.
To start, what does it mean when we say a candidate is engaged or that they have a good experience? Is it that they have click on more links, view more videos, or watch them longer, or is it the amount of time a candidate stays on the career site? Maybe it is how fast they get a response to their application or whether they can talk with a recruiter? We don’t know which, if any, of these are important. We have numerous adjectives to describe the characteristics we think are associated with engagement — words that define attitudes, behaviors, culture, and the interplay between these. But none are definitions. And none can be measured.
I know many recruiters say that this doesn’t matter. It is more about “I know it when I see it.” And many recruiters say they when they speak with candidates they learn that response time, having good videos about the positions, and so on made a difference. But do we know if they did or not? Maybe the candidate was just saying that because we asked a leading question or because they don’t know themselves.
There is no evidence that any measures of engagement or experience have predictive validity — evidence that if you measured them today (however you choose to do it or whatever definition you use) the results would predict that something similar would happen in the future.
The two most common ways engagement/experience are justified are:
Expert opinion and anecdotes: Stories abound about how a specific approach led to more candidates or better candidates. No one defines “more” or “better” in any meaningful way. Some people swear that videos resulted in more candidates or higher quality candidates. Others on some other approach.
None of these are supported by quantitative evidence nor are the approaches recreated and validated by others. Experts have written articles about the value of more engagement, but none produce any objective measures to prove it and, in fact, most do not even bother to define what they mean when they say engagement or experience. Statisticians consider this type of evidence the least reliable and of the lowest quality.
Survey data: Surveys of candidates are common. Awards are based on the results of surveys that ask recruiters a set of questions about their process and candidates about their experience in getting hired, what attracted them, what made them more interested, or what excited them. This is used as evidence that several factors are important — speed of response, quality of information and so on.
Surveys such as these are biased, ask leading questions, are rarely complied by qualified statisticians, and are completed by a narrow set of respondents. All they do is cloak the stories and anecdotes with a veneer of what seems to be objectivity. They are of no real scientific value.
If engagement or experience were to be objectively measured it would require a measurable definition and then either objective, third-party research with a control group or a longitudinal study involving a large sample complied over time to see what makes a difference. I have seen none of these done, and it would be tough to do it, which is most likely why it has not been done.
Does having a good experience or being engaged result to more people being hired from a given set of candidates? Where is the evidence that this is the case? Any single recruiter can make that claim, but is it verifiable? Does it apply year after year? Does it apply to more than one recruiter/firm? Can we make a generalization from it?
Does having a good experience or being engaged result in better candidates? How do we define quality? Again, there is no accepted, measurable definition of candidate quality that I have seen. There are mounds of anecdotes, but there were mounds of anecdotes and case studies proving that sanitariums and sunshine cured tuberculosis. Anecdotes are not evidence.
We are a long way from showing that a positive experience or engaged candidates are any better than those who were not particular engaged or from those who had a bad experience. Spending time and money on unproved theories is not only wasteful, it can divert us from doing things that make a bigger difference — learn about the business, anticipate future needs, build relationships, and work with hiring managers to set realistic requirements.
