Games are fun, assessments are not.
This is all you need to know to understand the strong interest in injecting games into the hiring process. But are game-based assessments really all they are cracked up to be?
Should you be betting the farm on Farmville? A quick overview of the various types of game-based assessments will help you decide for yourself.
Recruitment Games
These are games that live outside the funnel. Recruitment games are very face valid (i.e., they look like the actual job) and are used for employment branding and candidate attraction. Some great examples are the long-running L’Oreal brand storm challenge and Heineken’s slick “The Interview” experience. Talent leaders always start the conversation by requesting this type of games because they offer a superior candidate experience.
News flash: these games are not true assessments. Using the data from these games to make hiring decisions is a “no-no” because:
- They are used outside of the funnel before applicants have opted in.
- Since those playing the games are not applicants, and because not all applicants pass into the funnel having played the games, the data from these games is off limits.
- These games are also not psychometrically validated, meaning the data is not reliable for decision making.
The bottom line is that recruitment games are great, we all love them. But don’t expect them to behave like true assessments.
“Non-Contextualized” Games
These are games that function more like assessments. These games typically place the applicant in a fantasy-based environment that does not look anything like the actual job (i.e., outer space, amusement park, Middle Ages, etc). These games are enticing because they are entertaining, are often mobile first, and actually offer some predictive validity. These games also do not require a custom build, making them cheaper and more accessible. I hate to spoil the party, but replacing assessments with these games can be a mistake because:
- They send the wrong message to candidates — jobs are serious stuff for candidates. While engagement is super important, don’t throw the baby out with the bathwater. They may be fun to play, but these games may actually be hurting your employment brand because applicants do not like to think that employment decisions are being made based on how many martians they can zap in 30 seconds. Using this type of game out of fear that your current assessment will not appeal to millenials is a mistake. Candidates of all generations look for a professional, job-relevant application process.
- They are still not as proven as traditional assessments that measure the same thing — These games often use metadata (all the data points of every type, such as cursor movement, number of clicks, etc.) collected from the game to score applicants. The cornerstone of psychological measurement for predicting job success mandates the reliable and accurate measurement of human traits. Expecting game-play metadata to reliably and accurately measure human traits is ambitious at best. Someday we will get there, but we are not there yet, despite what the gaming companies tell you.
Suitable Alternatives
If the skinny on game-based assessments is not as optimistic as you had hoped, don’t fret. There are some alternatives that can scratch your itch.
Trait-based games
These games offer an interactive experience that is relatively plain looking, but they are still very effective as measurement tools. Trait-based games fall into two categories:
- Cognitive games. Leading the charge in this category are cognitive games. These games are great for several reasons: they are typically mobile first, they miminize bias when compared to traditional cognitive tests, and they can be designed to measure a variety of different facets of intelligence (i.e., logic, comprehension, multitasking, etc). Cognitive games offer the opportunity to use the best single predictor of job performance, while minimizing the stigma and risk of bias (i.e., adverse impact). While they may not be considered entertainment, cognitive games are more engaging than traditional cognitive tests which are typically about as fun as a root canal.
- Personality-based games. This type of game is evolving nicely, but is still not quite ready for prime time. Cognitive ability is much easier to measure than personality because it is much more straightforward. Personality has more models/options, more leeway for interpretation, and is more malleable. When it comes to game play, it is much harder to create games that provide reliable and accurate measures of personality. The research continues to show some progress, but when it comes to predicting job success, these games should be approached with a healthy dose of caution.
Gamification
Assessments can be gamified without being an actual game. Gamification means injecting several characteristics that make the user experience more engaging. Not all types of gamification are practical for assessments, but the relevant ones include:
- Narrative. Providing a storyline or context for the experience
- Flexibility of choice. Providing the user with multiple pathways to follow
- Rewards. Giving the user something of value for their achievement/progress (badges, greetings, points, etc.)
- Feedback. Allowing users to see data about their performance and how they compare to others or to some standard.
It is a challenge to inject many traditional assessments with gamified elements, but gamification is having an impact on how new assessments are developed, as prinicipals of gamification are adding new levels of engagement to traditional assessments.
Simulations
Simulations are a tried-and-true assessment method that have a ton of value. The truth about simulations has not changed in the past 50 years. They are winners because they provide an interactive experience that really looks like the job. There are tons of great assessment-based sims out there to choose from, but sims also present some significant limitations.
- Not all jobs are not easily simulated. Jobs that take place on a computer (call-center rep, manufacturing tech, etc.) are easy to simulate. Jobs that require face-to-face interactions with other people are much harder to simulate.
- Developing localized sims is expensive. Unleashing the real power of a sim requires that it be custom made for the job. But, most sims are generic because the expense of making a customized sim can be prohibitive.
- Scoring is hard. Scoring the complex branching interactions and open text that are required for highly realistic sims is really difficult given our present technologies. Scoring limitations make accurate scoring of fluid interpersonal interactions very difficult. While there is some progress scoring the metadata that is captured via a sim, connecting these data to reliable and accurate measures of human traits is still elusive.
- Non-mobile. The content of many jobs simply cannot be rendered on a mobile device, making a mobile simulation of that job a bad idea.
Sims and Games Announce a Merger
Despite the limitations, don’t count sims or games out just yet. Sims are the future of game-based assessments (or vice versa). The assessments of the future will be pretty much all simulation-based and will offer a gamified, realistic, and engaging experience for users. Here are the factors that will fuel this fire.
- AI will provide the ability to score more fluid and complex interactions. Text analysis, voice analysis, AI-based scoring bots, and AI-based agents will all be part of taking sims to the next level.
- VR and AR. The jury is still out on assessments that use current VR technology. There is a lot of hype around VR assessments, but there are still many limitations to be overcome. As technology evolves, there is no doubt that simulated environments will be part of the assessment experience — but I’m not sure that current technology will get us there.
- Distributed data collection. Assessments will become an experience that is distributed over time. For instance, you may interact with an agent on your phone over a period of days or weeks, and data from all these interactions will be used to score your performance based on the competencies required for a job.
So, the answer to the question “should I invest in assessment games?” is “yes,” but be sure to do do your homework and be realistic about what is possible. When it comes to assessment games in 2018, we are still playing Pong. The good news is that while the game-based assessment you are dreaming of probably doesn’t exist yet, there are still plenty of great alternatives, and things are only going to get better.