When Hiring, How Important is Emotional Intelligence?

Emotional intelligence is a hot topic in HR lately and, at face value, it seems like an attribute that every great employee should have. But how do you define and measure emotional intelligence well enough to seek it out in your job candidates?

The answer is not so simple. Much of the ambiguity stems from competing definitions of what emotional intelligence is in the first place. There are two main models of emotional intelligence (EI), one based on abilities and another based on traits.

The ability model posits that people vary in their ability to process and think about emotions, and that this ability can be measured through adaptive behaviors. These behaviors include perceiving, using, understanding, and managing emotions, which this model measures through emotion-based problem solving tasks.

In contrast, the trait-based model measures EI through people’s self-perceived emotional abilities. EI tests that use this model require individuals to self-report their personality/behaviors based on prompts, similar to the way that many established personality tests assess individuals. There’s also a third “mixed” model popularized by Daniel Goleman’s 1995 book Emotional Intelligence, which is a combination of the ability and traits model. While there are pros and cons to each model, there is no general consensus within the scientific community about which one is more accurate.

To complicate things further, the research linking emotional intelligence to job performance shows very mixed results. One meta-analysis of dozens of studies on EI and the workplace concluded that the results so far are inconsistent. Noted psychologist Adam Grant, himself a fan of the new emphasis on emotional intelligence research, recently argued that the evidence does not yet support the use of EI tests to inform hiring decisions. In comparison, tests of cognitive aptitude (or traditional intelligence) are consistently shown to be much more predictive of performance than emotional intelligence.

This is not to say that emotional intelligence isn’t valuable in the workplace. Much of what we perceive EI to be may actually overlap with other more established measures. For instance, some evidence shows that EI may be linked to some traits commonly measured in personality tests, including agreeableness and openness, although the extent of those relationships vary from study to study. What’s more, EI is shown to be positively correlated with cognitive aptitude, suggesting that some components of EI may be encompassed within traditional intelligence.

But many questions still remain: How can we measure EI in a way that is predictive of job performance? What relationship does EI have to cognitive aptitude? What relationship does EI have to personality?

Here at Criteria, we think emotional intelligence is a really exciting frontier for research. While a lot of fascinating work is being done to uncover the link between emotional intelligence and workplace performance, the current research isn’t quite strong enough for us to recommend using it as a factor for making hiring decisions.

So while for now there might not be a well-validated EI test for hiring purposes, there are ways you can approximate emotional intelligence through other more predictive factors. In the meantime, we look forward to seeing what future research has to tell us about emotional intelligence and the workplace.

Structured vs. Unstructured Interviews: The Verdict

For most employers, interviews continue to be a pivotal factor in the hiring process despite mounting evidence that interviews can be incredibly unreliable for predicting job success. One study found that impressions made in the first 10 seconds of an interview could impact the interview’s outcome; another study suggested that employers hire people that they like the most on a personal level; and research has consistently demonstrated that unstructured interviews are one of the worst predictors for job performance.

Despite all this, ditching the interview altogether is probably not a good solution. The reason is the key difference between unstructured and structured interviews. Unstructured interviews lack defined questions and unfold organically through conversation. It’s easily apparent how unstructured interviews can lead to bias when the “success” of the interview is dependent on natural chemistry or common interests.

In contrast, structured interviews consist of defined, standardized questions designed to efficiently determine if the candidate is up for the job at hand. By standardizing the interview process for all candidates, structured interviews minimize bias so that employers can focus on the factors that will have a direct impact on job performance.  At face value, structured interviews are more useful for predicting job performance, and it shows in the data. Structured interviews are almost twice as predictive of job performance as unstructured interviews.

So why aren’t more people exclusively using structured interviews? One of the biggest obstacles may be how difficult it is to actually plan and write a structured interview in the first place. Constructing a format for a structured interview can be time-consuming, requiring careful thought and a little bit of trial and error. Structured interviews can also feel awkward and stiff for candidates.

While establishing a structured interview process may be a challenge, it’s still a worthy goal. The data consistently reaffirms that unstructured interviews are significantly less predictive than structured interviews. Unstructured interviews also increase your chances of introducing more bias into the process. If your goal is to hire the candidates who are most likely to succeed on the job, then structured interviews are the way to go.

4 Tips for Improving the Candidate Experience

The hiring process is a two-way street. While candidates have to put their best foot forward to impress potential employers, companies also need to make themselves as appealing as possible to attract the best talent. Most recruiters agree that we’re now in a candidate-driven market, which means that candidates have more power to be choosy.

This places more pressure on employers to improve their “employer brand” – according to a 2016 LinkedIn survey on recruiting trends, 59% of the respondents said they were planning to invest more in their employer brand than in the previous year.

One critical aspect of employer brand is candidate experience. How a candidate is treated through the hiring process can affect the number and quality of candidates you receive in the future. Now, not only can candidates share their experiences with their acquaintances, but they can also post candid reviews about the company on sites such as Glassdoor.

Here are some tips for making sure your candidate experience is the best that it can be:

  1. Communicate clearly. Provide your candidates with all the information they need to understand your company and the role. This starts with a clear and detailed job description that signals not only what the position entails but also what sort of company culture you have. Communication is also invaluable during the process itself. Respect the time and effort that candidates put into the application process by promptly communicating where they are in the process or what is expected of them.

    Transparent communication is shockingly rare: a 2015 CareerBuilder survey found that 58% of employers don’t let candidates know how long the application or interview process will take. This lack of communication isn’t going unnoticed by candidates. Candidates in that same survey said that they received a response to their applications only 40% of the time, and only 14% of candidates felt that companies were responsive to them.
  2. Don’t make them jump through hoops. Your goal is to get enough information to make an informed hiring decision without making the process burdensome to the candidate. Endless interviews or extra requests for time-consuming work samples may deter some candidates from applying. However, the amount of time and effort put into hiring will vary based on the level of the position. The hiring process for an entry-level position is generally much less intensive than the process for hiring a manager or VP.
  3. Keep the process brief and efficient. One way to snag the top talent is by keeping the entire length of the process as brief as possible. Candidates who are actively looking for a job are probably applying and interviewing with other companies, and they may be swayed to pick an earlier offer. Dragging out the hiring process could cause you to lose your top candidates to the competition.
  4. Be flexible. A lot of people applying to jobs are already employed full-time. When scheduling interviews, try to be flexible with scheduling. If you need to request additional information or work samples, give them ample time to provide it. And of course, be clear and transparent about the entire process.

As a pre-employment testing company, we advise employers to also incorporate all of these strategies into their testing process. When you administer tests, it’s important to communicate to candidates why they are being asked to take a test and what the next step in the process will be. To keep the process burden-free for candidates, we recommend administering no more than 40-45 minutes of tests so that candidates don’t get burnt out or discouraged. And it’s also ideal to allow candidates to complete the assessments at a time and place that is most convenient for them, ideally by administering tests online.

All of these strategies make the hiring process a little less stressful for candidates and bolster your employer brand.

When Hiring, General Abilities Predict Success Better Than Specific Skills

How can you tell if your job applicants have what it takes to succeed in a particular position? There are so many factors that go into a hiring decision, and resumes can only tell you so much. Resumes are notoriously unreliable, with research suggesting that up to 78% of resumes contain misleading statements, while 46% contain actual lies. Similarly, your candidates’ work experience and educational background aren’t a guarantee that they possess critical thinking skills or problem solving ability, and these factors have been shown to be poor predictors of future job performance. Sometimes the best way to dig deeper into what your candidates can actually do is by testing their abilities.

When it comes to pre-employment tests, how do you decide which tests to choose or, more importantly, what abilities to test for in the first place? There are a lot of different types of tests, but most tests fall into one of two basic categories: general or specific.

General tests include cognitive aptitude tests and a lot of personality tests. At their core, general tests assess broad or innate abilities or characteristics that provide insight into a candidate’s potential for success. Alternatively, specific tests are, well, specific. They test specific skills that a candidate has picked up through education or work experience, such as typing speed or familiarity with Microsoft Excel.

In essence, the main difference between general and specific tests is that general tests measure potential, while specific tests measure acquired skills that candidates have already learned. It’s the classic dichotomy between aptitude and achievement.

As it turns out, general tests (cognitive aptitude tests in particular) are much better at predicting overall job success than specific skills tests. This is because general tests measure core abilities such as critical thinking, learning ability, and problem solving skills, all of which have an impact on how well an employee is able to adapt and thrive in a new position. One meta-analysis – or  summation of numerous studies done in this field – even found that cognitive aptitude tests were three times as predictive as job experience and over four times as predictive as education level.

General personality tests also have a lot of predictive value, particularly when they measure conscientiousness. Conscientiousness is a trait that is consistently correlated with job success because it indicates how goal-oriented, self-disciplined, and dependable an individual will be.

In contrast, specific tests tend to be less predictive of long-term success. While a specific test of “microskills” – such as a test on a particular programming language or a test assessing data entry skills – can help you find out if your candidate already knows how to perform a certain task, research shows that they do not tend to be great predictors of overall performance in the long-term. A general skills test that measures broader job-readiness competencies is an exception to this rule, but microskills tests typically only assess one limited part of the role. They do not assess a person’s ability to learn new skills, or adapt and grow as an organization or job evolves.  A general aptitude or personality test sheds light on that candidate’s long-term potential.

And because general tests measure broad abilities that are critical to success in many positions, they are predictive for a wide range of job roles. Both general and specific tests do have value when it comes to finding the right candidate, but using a more general test as your primary assessment, possibly in combination with a secondary skills test, is the best strategy for uncovering the candidates who are most likely to succeed.

Why is Measuring Quality of Hire So Difficult?

Amidst all the buzz over the advent of “big data,” HR departments are increasingly focused on using data to improve their talent acquisition strategies.  In our particular business—developing pre-employment assessments used by businesses to help inform their hiring decisions—we are seeing an increasing willingness on the part of employers to adopt evidence-based hiring tools.  The goal of all this is simple: better hiring results, or in other words, improvements in quality of hire (QoH).

There is widespread consensus about this: in a recent LinkedIn survey on recruiting trends in 2016, talent leaders cited quality of hire as the most important metric for tracking success in the recruiting process. Another finding, while not surprising, highlighted a central challenge that hiring managers face: only a third of the respondents felt that their methodologies for measuring quality of hire were strong.

It’s difficult to uncover what parts of the recruitment process are working without a metric for measuring job success once a person is hired. While tracking some QoH-related measures—such as retention—is relatively straightforward, getting to a unified performance metric that summarizes whether someone is a good hire or not can be very difficult.

We encounter this problem often when doing local validity studies, which are essentially a way to analyze how successful a pre-employment test is at predicting success for a particular role in a specific organization.  The typical process for doing these studies is to administer the tests to a group of employees—customer service reps, for example—and then to compare the test results to the employees’ performance metrics. By tying your employees’ pre-hire test scores to their eventual work performance, you gain insight into how predictive and effective your employee selection criteria is. This can give credence to your current tactics or help you identify ways to improve your recruitment process.

One problem that often arises with local validity studies is when companies don’t have meaningful performance metrics in place. Alternatively, they may be able to provide performance metrics, but have little confidence that the metrics reflect who top performers are, or can’t agree internally as to the appropriateness and accuracy of those metrics. This presents a huge problem: how can you predict what you don’t measure, or don’t measure accurately?  The CEO of Hogan Assessments, a competitor of ours, expressed the problem well when he wrote that using data-driven hiring techniques without tracking quality of hire is “the equivalent of investing a great deal of money in weather forecasts without subsequently paying attention to the actual weather.”

So whether quality of hire metrics come from supervisor performance ratings, tangible business metrics (such as sales volume or customer satisfaction ratings), retention rates, or some combination thereof, it is important to invest time in coming up with performance metrics that measure something meaningful and that all stakeholders agree represent something real. Absent this, there is no point in spending time trying to predict who will be a good hire if you can’t agree on a definition of success once the hire is made.

Can Aptitude Tests Be Used to Predict Bad Behavior?

We’ve previously written about the use of the Wonderlic aptitude test on NFL draft prospects, pointing out that the popular press and NFL fans as a whole have often unfairly dismissed aptitude tests as irrelevant to future gridiron success. This seems to be based on jock stereotypes about the sport and on a misunderstanding of how tests, and predictive tools in general, work.  Virtually every article about the Wonderlic test at the NFL draft mentions Dan Marino, who bombed the Wonderlic and went on to a Hall of Fame career, as evidence that the tests aren’t predictive of success in football. However, this type of anecdotal evidence clearly holds no weight when statistically determining whether or not a test works.

We’ve argued, for example, that there may be more of a correlation between Wonderlic scores of NFL quarterbacks and their future performance than is supposed. Nevertheless, it is fair to say that the evidence for the predictive power of the Wonderlic in the NFL is mixed. This is not surprising, because while the modern NFL game is quite complex and requires quick decision-making skills—especially from quarterbacks—it is clear that so many of the determinants of success in the NFL have to do with athleticism, work ethic, and other things aptitude tests can’t measure.

Recently, CBS Sports published a story about a new analysis of the links between Wonderlic scores and the subsequent fates of the NFLers who took it (and yes, it does contain the obligatory mention of Dan Marino). This one had a very different focus, however, because instead of examining on-field performance, the study looked at the relationship between Wonderlic scores and the arrest records of NFL players. The results of the study, which appeared in the American Journal of Applied Psychology, were striking; players with below average Wonderlic scores were twice as likely to be subsequently arrested as those who scored above the mean.

This is the first time we’ve seen a study that links low Wonderlic test scores to what the study calls “off-duty deviance,” or ODD, which may be our new favorite psychological term (“you down with ODD? yeah you know me.”)  Employers trying to prevent discipline-related problems in the workplace often use integrity/honesty tests or behavioral risk assessments that measure rule adherence or personality traits like conscientiousness that are linked to good behavior. Such tests have been shown to help prevent a wide variety of counterproductive work behaviors such as safety violations, absenteeism, illicit drug use, theft and fraud.  Aptitude tests, however, are more commonly used to predict overall performance, not who will constitute a behavioral risk.

But the new Wonderlic study is actually not the only sign of a possible link between intelligence and honesty.  The Washington Post recently reported on an Israeli study that seemed to link intelligence with honesty and truth-telling behavior. The study asked participants to enter a booth, roll a six-sided die, and report the number that came up to receive that amount of money instantly (if you roll a 4, you get $4, etc.). What they found was that those who scored lower on an intelligence test were far more likely to lie about rolling a six.

The implications of this study remain to be seen, so the results should be taken with a grain of salt. However, there seems to be growing evidence of a link between cognitive aptitude (intelligence) and other qualities that are typically thought to be purely behavioral or personality-driven. We expect to see a lot of future psychological research take on questions such as these, and we’re excited to see where the data lands!

Why Math Skills Are So Important in the Workplace

You’re forgiven if you didn’t know it was Math Awareness Month, but there are a lot of reasons why everyone should be more aware of the important role math plays in the workplace and in our everyday lives. With more and more evidence that Americans are falling behind in math ability compared to other developed nations, math ability is, in the United States at least, a gravely undervalued commodity.

You may think back to all the trigonometry you learned in school and point out that most jobs will never require you to find the cosine of an angle. But math skills are about much more than all the minutiae you were taught in school. Math skills – particularly numeracy and numerical problem solving – are not only fundamentally important to everyday job functions but also are a strong indicator of broader cognitive abilities. And because cognitive aptitude is one of the most predictive factors of job success, testing your candidates’ math abilities is a great way to assess their ability to succeed on the job.

Math and numerical problem solving are a part of most cognitive ability tests. This is partly because math problems aren’t simply measuring math skills; they’re also measuring critical thinking, problem solving, and logic. So even though you may be hiring for a position that doesn’t “require” math skills, measuring your candidates’ basic numeracy skills often has implications for their ability to solve problems in the workplace.

You might also think that testing math ability is unnecessary in the modern age because we have access to computers and calculators that can perform more complicated math functions for us. While we do have nearly constant access to computers, they can’t do all the work for us if we don’t fundamentally understand the math we need them to perform.

If anything, math abilities are more important than ever with the rise of big data. Companies are relying more and more on data to guide their decisions, and employees who can analyze and interpret data in ways that inspire actionable decisions are extremely valuable. Even employees who may not work directly with data are at a disadvantage if they can’t understand what the data is conveying on a basic level.

Mathematical prowess is an extremely critical, chronically overlooked ability. Math skills are associated with broader cognitive abilities, and they are reflective of a candidate’s critical thinking and problem solving ability. Yes, a lot of the math we learned in school doesn’t end up being all that relevant for the majority of us, but basic numeracy is unavoidable in everyday life, and those who do avoid it are at a fundamental disadvantage. And for employers seeking critical thinkers and problem solvers, aptitude tests that measure math skills are a great way to gain insight into your candidates’ abilities.

Introducing the UCAT, an Internationally-Friendly Aptitude Test

Today we’re excited to launch our new internationally-friendly aptitude test, the Universal Cognitive Aptitude Test, or UCAT. The UCAT measures general cognitive aptitude, one of the most predictive factors for job success.

Just like the CCAT, our most popular aptitude test, the UCAT measures critical thinking, problem-solving ability, and logic, all elements of cognitive aptitude. Because the UCAT and the CCAT are measuring the same abilities, they are highly correlated with each other.

What makes the UCAT different is that it deemphasizes verbal ability so that the test is ideal for use with non-native English speakers and international candidates. The UCAT places more of an emphasis on problem-solving, attention-to-detail, and data interpretation, making it a particularly great assessment for testing quantitative and analytical positions.

The UCAT is a 20 minute test with 40 questions. The test is written in English but can easily be translated into other languages or used as a test for non-native English speakers. Moving forward, we plan to make the UCAT available in other languages – let us know what languages you’d be most interested in using!


3 Mistakes to Avoid When Using Pre-Employment Tests

Pre-employment tests provide incredibly useful information that allows you to make more informed hiring decisions. By incorporating professionally developed pre-hire assessments into the hiring process, you gain relevant, objective data that, when combined with other factors such as interviews and work experience, can present a more comprehensive view of your candidate’s capabilities.

However, it pays to be mindful about how to use pre-hire tests in a way that provides the most value to your organization. Here are three of the biggest mistakes you could be making with pre-employment testing:

  1. Choosing the wrong tests. This is by the far the most important pitfall to avoid when using pre-employment tests. Test selection is vital because no matter how well-validated a test may be, it has little value if it isn’t measuring job-related capabilities. Test validity is often misunderstood—it does not exist in a vacuum, and even a well-validated test can be problematic if it’s being used for a purpose for which it was not validated.

    For example, you wouldn’t give a typing test to candidates applying to be maintenance workers if they won’t be expected to use a computer on the job. Their scores on such a test would prove meaningless for making a hiring decision.

    Even more importantly, as the EEOC has made clear in its Uniform Guidelines on Employee Selection Procedures (UGESP), the crucial standard in assessing compliance with respect to any criterion used in making hiring decisions—including tests—is that it must be job-related. Taking the time to evaluate the skills and abilities required for a particular position will enable you to select the tests that will provide the most valuable information while remaining legally compliant.

  2. Having unrealistic expectations. The right employment tests will help employers predict work performance. However, while pre-employment tests are predictive of success, they are not a crystal ball.

    When a pre-employment test has predictive validity this means that, on average and across a large sample of data, the test correctly predicts business outcomes. It does not mean that the test correctly predicts performance in every single individual case.  Outliers can and do happen now and then, but on average, there should be a significant correlation between test results and work performance.  Weather reporters make predictions that don’t always come true, and so do pre-employment tests. It is unrealistic to expect a pre-employment test to make the right prediction every time.

    Fortunately, pre-employment tests, and aptitude tests in particular, are some of the most predictive hiring criteria you can use. In fact, one study conducted by the National Bureau of Economic Research found that pre-employment tests are consistently better at predicting job success than are hiring managers.

  3. Ignoring the candidate experience. Let’s face it, not everyone loves taking tests. Pre-employment tests are designed to help employers find the best talent, but making the testing experience too burdensome can have the unintended consequence of turning off some candidates. Being cognizant of the candidate experience not only improves your employer brand but also minimizes the amount of drop-off you may experience from candidates who don’t feel invested enough to commit to a lengthy testing process.

    One of the most important elements of candidate experience when it comes to testing is the amount of time it takes a candidate to complete the tests. We generally recommend administering tests at the beginning of the hiring process to get the most value out of the test results. When testing candidates early in the process, a pre-employment test may be one of the first touchpoints a candidate has with the company, which means that if the company immediately assigns a 3 hour battery of testing, the candidate may quickly lose interest. We’ve done our own research on the subject and found that the level of candidate drop-off is minimal so long as the total testing time remains below 45 minutes.

    There are a lot of other ways to be mindful of the candidate experience, and a lot can be accomplished through thoughtful messaging. For instance, explaining to candidates why they’re being tested and what the tests measure can be helpful. Similarly, sending an email confirming that the candidate’s test results have been received goes a long way towards making your candidates feel that their time is respected.

The Cost of a Bad Hire and Reducing the Odds of Making One

It’s well known that hiring a bad employee can be incredibly costly. Estimates of the true cost of a bad hire vary widely depending on the type of position and amount of experience required. One estimate from the US Department of Labor places the average cost of a bad hiring decision at about 30% of the employee’s first-year salary. In another study conducted by CareerBuilder, 69% of companies surveyed were negatively impacted by a bad hire, and nearly a quarter of those employers stated that a bad hire cost them over $50,000.

Bad hires are costly in a lot of different ways, some of them less tangible than others. While the costs associated with hiring and training a new employee are obvious, bad hires can also have a negative impact on employee morale and overall productivity. As a (relatively) small business ourselves, whose customer base is made up of a lot of small and medium-sized businesses, it’s our position that the risks of a bad hire can be more dramatic for smaller companies. Smaller companies have significantly less bandwidth to put towards covering the duties of the vacant position and recruiting a replacement, and an unproductive or–even worse–a toxic hire can have a bigger impact on a small group than he/she will have on a larger organization.

Because of these costs, companies often strive to reduce the risk of hiring the wrong person as much as possible. There are a lot of reasons why a company might hire the wrong person, but 21% of the employers in a CareerBuilder survey attributed their bad hires to a failure to sufficiently assess employee skills in the pre-hire process. Employees who lack the necessary skills or abilities for the job will underperform, ultimately leading to involuntary turnover.

Testing your candidates in the pre-hire process is one of the best ways to minimize the risks posed by a potential bad hire. Resumes and interviews can only reveal so much information – one survey found that 56% of hiring managers have caught job candidates lying on their resumes, most of whom were embellishing their stated abilities. Administering pre-employment tests for vital job-related abilities is one of the few objective ways to accurately assess your candidate’s potential to fulfill the responsibilities of the job.

Despite the claims of some testing vendors, pre-employment tests can’t magically erase the chance of making another bad hire. We cringe, and potential customers should too, when we see testing providers make claims that their tests will prevent you from ever making a bad hire again. There are many things that tests cannot measure, and the best pre-employment tests arm hiring managers with predictive data that helps them make informed hiring decisions. Incorporating professionally developed pre-employment tests into your employee selection process is about reducing your hiring risk, not eliminating it.