The Accuracy of Psychometric Tests - How Much Testing is Too Much?

Recently I addressed a large group of Human Resources practitioners at a gathering of the NZ HR Institute. There was such a wide range of organisations and expertise attending. Add to this a bad case of the flu and it made for a challenging address.

During question time, an interesting question stirred up quite a debate, “When it comes to hiring, how much testing is too much?” I like to use the medical analogy that if you want to check for diabetes there is no need to drain all the blood from your body; a vile full will do.

The same analogy applies to psychometric testing. Some of my contemporaries are subjecting applicants to 3 plus hours of assessments, or, heaven forbid, an all day assessment centre process. Here applicants are subjected to many practical work simulation tests as well as a number of self report tests.

Two plus hours of testing is overkill. Collins (2003) did a meta analysis - a study of 524 individual correlations between measures of personality and cognitive ability and job performance in assessment centres. The result was a whopping .83. Their conclusion: most of the valid variance in Overall Assessment Centre Ratings can be captured with good mental ability (cognitive) and personality measures that could be completed in one, to one and half hours!
  
The only winners, when it comes to lengthy assessment procedures, are the promoters of the testing procedure, who in many cases do not have organisational psychology training. These promoters will charge a fat fee for a lot of reports that probably won’t be read by a busy executive, or even worse, will be full of so much psychobabble, they won’t understand it!

So, back to the question, “How much is too much?” This depends on the job role and expected number of applicants. If there are a large number of applicants and you are recruiting for a mid to low level role, you may want to use a quick pre-screening assessment to select out those who do not “fit”.

Here you are seeking a broad overview, for example, a look at scores on the Big 5 dimensions of personality, plus a brief cognitive test measuring verbal and numerical ability, or may be a quick attitudes test (integrity). A tool like PeopleCLUES is ideal for this purpose. It is inexpensive and takes about 25 minutes to complete.

On the other hand, if the role is a management or professional position and you have a smaller pool of applicants then I’d advise a “deeper” assessment like Rembrandt or ASSESS. I’d still like to do this testing before the main interview. This allows you to go into the main interview armed with a complete picture of the candidate. You can then use the interview and reference process to validate the information gained from the profiling.

Usually the length of testing time is governed by the depth of information required and this depth is correlated to the job level. In a nutshell testing is going to revolve around four basic dimensions – Personality Profiles, Mental Ability testing, Values and Motives inventories (the how and why the candidate will do the job ).

Do you need to conduct all of these tests? Well, if you have the budget and the time you’d certainly cover all the bases and have a complete profile of the candidate’s attributes and abilities – but, do you need all of this information to make an informed decision?

As much as I love to see all of this information, I don’t think it would add much more to the decision making process – as psychologists we refer to this as adding incremental validity. On top of this, do we really need to subject the candidates to this much intense scrutiny, or spend that much money?

This begs the question, “What would serve as the basics for profiling candidates for most job roles?” The research is pretty clear; a combination of two tests will usually give you enough incremental validity to make a confident decision. Let’s re-visit our medical analogy. One vile of blood will highlight a diabetic problem, but hey, whilst you are here let’s just take another vile to check your cholesterol levels.   

A list of validity levels of tests used in the selection process may give you a clearer picture and a useful guideline for questioning test promoters to justify a range of tests or costly assessment centre testing:

Personality, Mental Ability, Motives & Interests (matched to job) .75
Personality, Mental Ability & Motives .66
Mental Ability & Integrity .65
Mental Ability & Structured Behavioural Interview .63
Mental Ability & Work Sample Test .60
Personality & Mental Ability Test .58
Work Sample Test .54
Mental Ability Test .51
Structured Behavioural Interview .51
Job Knowledge Test .48
Integrity Test .41
Personality Test .40
Assessment Centres .38
Biodata (application forms) .35
References .26
Years of Job Experience .18
Unstructured Interview .14
Years of Education .10
Interests .10
Graphology .02

Note: Mental ability testing can also be referred to as a cognitive or intelligence test;
the correlations relate to overall job performance. When tests are validated against specific job roles, correlations can increase. As an example, a recent study we did on the Rembrandt Personality Tests, aligned to a specific sales role, gave a .70 correlation.  

In summary, the point I am trying to make is that psychometric testing, although not the total panacea to selecting the right person first time, is supported by strong scientific evidence to add incremental power to the selection process. Couple this with a multi rated, behavioural based interview and you will at least have a 75 to 80% chance of getting it right. Far better odds than most managers are getting now with their casual glance over a CV, a general chit-chat that serves as an interview, and a tertiary reference check.  

Sources for validity correlations: Mike Smith, University of Manchester, 1994, John E. Hunter and Ronda F. Hunter, 1984, Robert P. Tett, Douglas N. Jackson, and Mitchell Rothstein, 1991. Testing People at Work, Smith & Smith, 2005