The Accuracy of Psychometric Tests – How Much Testing is Too Much?

Recently I addressed a large group of Human Resources practitioners at a gathering of the NZ HR Institute. There was such a wide range of organisations and skill attending. Add to this a bad case of the flu and it made for a challenging address.

During question time, an interesting question stirred up quite a argue, “When it comes to hiring, how much testing is too much?” I like to use the medical analogy that if you want to check for diabetes there is no need to drain all the blood from your body; a vile complete will do.

The same analogy applies to psychometric testing. Some of my contemporaries are subjecting applicants to 3 plus hours of assessments, or, heaven forbid, an all day assessment centre course of action. Here applicants are placed under many functional work simulation tests in addition as a number of self report tests.

Two plus hours of testing is overkill. Collins (2003) did a meta examination – a study of 524 individual correlations between measures of personality and cognitive ability and job performance in assessment centres. The consequence was a whopping.83. Their conclusion: most of the valid variance in Overall Assessment Centre Ratings can be captured with good mental ability (cognitive) and personality measures that could be completed in one, to one and half hours!

The only winners, when it comes to lengthy assessment procedures, are the promoters of the testing procedure, who in many situations do not have organisational psychology training. These promoters will charge a fat fee for a lot of reports that probably won’t be read by a busy executive, or already worse, will be complete of so much psychobabble, they won’t understand it!

So, back to the question, “How much is too much?” This depends on the job role and expected number of applicants. If there are a large number of applicants and you are recruiting for a mid to low level role, you may want to use a quick pre-screening assessment to select out those who do not “fit”.

Here you are seeking a general overview, for example, a look at scores on the Big 5 dimensions of personality, plus a fleeting cognitive test measuring verbal and numerical ability, or may be a quick attitudes test (integrity). A tool like PeopleCLUES is ideal for this purpose. It is inexpensive and takes about 25 minutes to complete.

however, if the role is a management or specialized position and you have a smaller pool of applicants then I’d advise a “deeper” assessment like Rembrandt or estimate. I’d nevertheless like to do this testing before the main interview. This allows you to go into the main interview armed with a complete picture of the candidate. You can then use the interview and reference course of action to validate the information attained from the profiling.

Usually the length of testing time is governed by the thoroughness of information required and this thoroughness is correlated to the job level. In a nutshell testing is going to revolve around four basic dimensions – Personality Profiles, Mental Ability testing, Values and Motives inventories (the how and why the candidate will do the job).

Do you need to conduct all of these tests? Well, if you have the budget and the time you’d certainly cover all the bases and have a complete profile of the candidate’s attributes and abilities – but, do you need all of this information to make an informed decision?

As much as I love to see all of this information, I don’t think it would add much more to the decision making course of action – as psychologists we refer to this as adding incremental validity. On top of this, do we really need to subject the candidates to this much intense scrutiny, or use that much money?

This begs the question, “What would serve as the basics for profiling candidates for most job roles?” The research is pretty clear; a combination of two tests will usually give you enough incremental validity to make a confident decision. Let’s re-visit our medical analogy. One vile of blood will highlight a diabetic problem, but hey, whilst you are here let’s just take another vile to check your cholesterol levels.

A list of validity levels of tests used in the selection course of action may give you a clearer picture and a useful guideline for questioning test promoters to justify a range of tests or costly assessment centre testing:

Personality, Mental Ability, Motives & Interests (equaled to job).75 Personality, Mental Ability & Motives.66 Mental Ability & Integrity.65 Mental Ability & Structured Behavioural Interview.63 Mental Ability & Work Sample Test.60 Personality & Mental Ability Test.58 Work Sample Test.54 Mental Ability Test.51 Structured Behavioural Interview.51 Job Knowledge Test.48 Integrity Test.41 Personality Test.40 Assessment Centres.38 Biodata (application forms).35 References.26 Years of Job Experience.18 Unstructured Interview.14 Years of Education.10 Interests.10 Graphology.02

observe: Mental ability testing can also be referred to as a cognitive or intelligence test;

the correlations relate to overall job performance. When tests are validated against specific job role, correlations can increase. As an example, a recent study we did on the Rembrandt Personality Tests, aligned to a specific sales role, gave a.70 correlation.

In summary, the point I am trying to make is that psychometric testing, although not the total panacea to selecting the right person first time, is supported by strong scientific evidence to add incremental strength to the selection course of action. associate this with a multi rated, behavioural based interview and you will at the minimum have a 75 to 80% chance of getting it right. Far better odds than most managers are getting now with their casual to peek briefly over a CV, a general chit-chat that serves as an interview, and a tertiary reference check.

leave your comment