Allowing applicants to retake a test can help increase the likelihood of hiring the best person for the job
Psychometric and aptitude tests are a common tool for the biggest corporations to select the best applicants. 80 percent of Fortune 500 companies use such tests, such as Procter & Gamble’s P&G Success Drivers Assessment that serves as a personality test and its Reason Tests that assess a candidate’s verbal, numerical and abstract reasoning. Samples of such tests abound online for job applicants to practise ahead of the real thing at their selected employer(s).
But are these tests a good indicator of an applicant’s true ability and/or fit? And should they be allowed to retake a test if the initial score is inadequate?
“If you think about how organisations often make personnel decisions,” explains Jisoo Ock, Postdoctoral Fellow at Nanyang Technological University, “they’re often based on scores from a battery of employment measures given to job applicants. Applicants might be pre-screened on their resumes to see if there’s a good fit between the job and applicant. A large proportion of applicants are often screened with tests that are cheaper or less time-consuming such as cognitive ability tests or personality tests.
“However, these scores are imperfect. These imprecisions in measurement can be due to any random factors. A candidate might not be feeling particularly well, or maybe a little anxious, or it be could a long test and the candidate becomes tired towards the end. These moods or feelings may fluctuate over time or within [test] occasions. Your performance on those scores may not necessarily reflect your true ability. As a result, any top down selection on the scores that you get will not capture the best applicant.”
He adds: “If it’s a physical test, you might fail it the first time but you might pass if you go to the gym and lift some weights or do some running. Or you could have the opportunity to study information pertaining to a job. In those cases, it’s better to allow applicants to take the test again. The second test score might be more reflective of applicant’s ability. If so, it would be in the organisation’s interest to let the applicant retake a test.”
Ock was speaking at a recent SMU Social Sciences Capstone Seminar, “Retesting in Personnel Selection: The Impact of Test Reliability”, where he presented research findings of a computer simulation that showed higher rates of true acceptance (qualified applicants passing a test) and false acceptance (unqualified applicants passing a test) with retests.
“Of course, retesting is associated with increased risk of hiring unqualified applicants, but there are ways to guard against this issue,” says Ock. “For example, although many organisations allow applicants who were unsuccessful in their initial attempt to gain entry to retest, many organisations also have different rules concerning retesting (e.g., considering retest score that is too much higher than the previous score to be invalid, taking the average of test-takers' initial and retest scores).”
How then can companies determine the reliability of the result of the first and possibly only test a candidate takes?
“Reliability can be estimated in different ways depending on the relevant source of interest that one seeks to identify,” Ock says.” Based on a single test occasion, reliability can be estimated by examining how high the items correlate together (internal consistency) or arbitrarily split the test in half and correlate the scores, essentially treating the two halves as two separate tests. There are also ways to model the amount of measurement error variance present in the measurement scores using advanced quantitative methods.”
Click ‘Like’?
Suppose a candidate clears the initial test and makes it to an interview with the hiring and Human Resource managers. Does it boil down to the question: “Do I like this candidate?”
“A lot of organisations rely on tests to make personnel selection decisions, knowing at some level there is evidence for such measures predicting organisational outcomes of interest (e.g., job performance, low turnover),” Ock says. “Interviews are also considered tests, but the quality and the kind of information that are gathered in interviews depend on the questions that are asked in the interview. It is important to assess whether the applicant has positive characteristics or if the applicant fits well with the job and/or the organisation, which interviews may be used to identify.
“However, if decisions based on the interview questions become too dependent on the "do I like this applicant?" factor, this is likely to result in less accurate selection decisions and may even invite litigations against the organisation to the extent that the decisions are biased against certain demographic subgroups.”
How then should hiring managers and HR professionals best use a job interview to select the best candidate? What questions should they ask?
"It is important to derive interview questions based on empirical analyses of job requirements and standardising the interview process," explains Ock. "That is, organisations need to find out the tasks that are involved in the job, evaluate what are the knowledge, skills, and abilities (KSAs) that are needed for effective performance on that job, and to develop a common set of questions that effectively capture those KSAs (e.g., developing questions based on a job analysis, asking the same questions to every applicants and not allowing excessive follow-up questions).
"Then, the organisation also needs to develop a standardised scoring rubric that minimises interviewer disagreements or subjective judgments (e.g., using the same interviewer(s) across all applicants, using behaviourally anchored rating scales, providing training to minimize interviewer idiosyncrasies). Namely, it is important to minimise the degree of interviewer discretion in conducting the interview."
Follow us on Twitter (@sgsmuperspectiv) or like us on Facebook (https://www.facebook.com/PerspectivesAtSMU)
Last updated on 27 Oct 2017 .
