In our previous articles, we talked about how to define successful performance for a new hire and how to choose critical requirements for candidates.
Those are the first steps in any recruitment process, serving as the cornerstone for what’s to come: defining what high performance means in this specific role, and the traits & knowledge a candidate needs to achieve it.
The next step then is to evaluate the candidates that you’ve attracted to the process: do they have the right skills & traits? And to which proficiency?
Structuring the recruitment process
Not all recruitment processes should look the same. Although a general structure is encouraged, the exact assessments should change based on needs for the role. You need to define which assessment tools, and in which order, you’ll use to evaluate the different requirements. There’s tests, interview questions, practical cases, among others. Defining this beforehand, and using the same assessments for all candidates ensures fairness and consistency in the evaluation of requirements.
Starting with the basics: yes and no questions
Sometimes a role will have requirements that aren’t traits or skills, but instead mandatory qualifications for various reasons.
For example a driving license, a language, a work permit, being able to work in certain shift hours, a specific certification, etc.
Those are straightforward, they are a yes or no, either the candidate has it or not.
So you should be explicit about it in the job ad, and have screening questions in the application form so candidates can select yes/no for those requirements, saving screening time for the recruiters. If it’s not possible to have them in the application form, then it’s among the first things a recruiter should ask in a screening call to qualify a candidate.
Assessing Proficiency: the science of measuring human behavior
Ranges:
Now that we are done with the objective yes/no requirements, we move on to the more subjective ranged requirements: skills & traits.
By “ranged” we mean that the question isn’t “do they have this skill/trait?”, but rather “how proficient are they in this skill/trait?”. - For example, it’s not “does this candidate have negotiation skills?”, but rather “how good is this candidate at negotiation?”
This is a fundamental understanding assessment, because it changes the perception of skills & traits as a binary “yes/no” to a proficiency scale.
In Academic Work we use an adaptation of the Dreyfus model of skill acquisition (2021)1, following five proficiency levels that can be applied broadly to all skills. In summary:
- Beginner: Has “textbook” understanding.
- Advanced: Has a general practical understanding in key aspects.
- Competent: Has good and flexible applied knowledge of the practice.
- Proficient: Very strong expertise of practice.
- Expert: Authoritative expertise and deep tacit understanding across areas of practice.
Then for each skill a behavioral rubric is created based on the 1-5 model, with anchors on the extremes and middle. This defines the assessment criteria.
Using the same Negotiation skill example as before, this is what a simplified rubric could look like:
Negotiation
Understands the logic and structure of a negotiation process, and is able to apply it in a low-pressure and low-complexity setting with guidance.
-
Can independently and confidently handle complex negotiations with difficult stakeholders.
-
Leads extremely challenging, high-risk and aggressive negotiations. Able to come back from very unfavorable positions minimizing losses.
Objectivity:
As with everything related to human behavior, subjectivity and bias are prone to happen. Part of the objective of the assessment criteria in the rubric is to provide more objectivity and focus on what truly matters. Doing so happens by building an assessment instrument that’s both valid and reliable from a psychometric point of view.
‘Valid’ means that the item is actually assessing what it means to assess, and not something else by accident; and ‘Reliable’ means that it always assesses the same thing.
Validity and reliability are continuously improved by reviewing the predictive power of an assessment in corelation to work performance. We’ll get to that later in this series, but what matters right now is to understand that good assessment tools are ‘alive’, always improving thanks to data, and not something fixed that’s just built once.
This is where our expertise as AW in psychometrics and assessment science comes in to ensure the best, most accurate assessments for our clients.
Preparing to apply the assessments
A key responsibility of everyone involved in evaluating the candidates -be it recruiters, interviewers, case reviewers, and hiring managers- is to know and understand the assessment tools thoroughly, both in their implementation and interpretation.
Avoid defaulting to using old questions out of habit, and instead consider carefully the purpose of each question and tool, ensuring that they have a clear rubric and assessment criteria to evaluate the candidates’ answers.
You should have them prepared and know them well before going to an interview or administering an assessment, to ensure you are properly able to evaluate the answers.
This step is often skipped, but even the best tools in the world lose effectiveness if not used properly. So remember that you owe it to yourself, the candidates, recruiters and everyone involved in the process to prepare properly for the evaluation.
During the evaluation with the candidate
Once you are sitting in front of the candidate for the assessment (whether it’s an interview, a practical case, etc) remember always the main objective: to evaluate the skills & traits.
The recruiter working with you can give you detailed guidelines on how to handle the logistics of the evaluation (eg: introducing yourself, creating a comfortable atmosphere, keeping track of time, etc), so we’ll focus on how to carry the assessment.
First, inform the candidate that this is a structured assessment with predefined questions/tasks, and which specific skills/traits you seek to assess. This transparency will help the candidate to focus.
Then proceed with the questions/tasks as per the planned order. Don’t make up random questions on the spot based on “gut feeling”, as that can introduce bias, however feel free to ask follow-up questions based on the candidate’s answers to further understand what they mean, specifically when you think you are lacking information to accurately assess the candidate’s answer against a specific score in the rubric.
Speaking of the rubric, don’t feel the need to set a definitive score for each answer right on the spot. Instead, mark a “tentative” score as a placeholder to calmly revise at the end of the interview.
That being said, it’s important to take detailed notes of the candidate’s answers to serve as reference. Either write them yourself, or use an AI tool to support you. This is a best practice because it’ll (1) help you carefully consider the score for each question (improving its objectivity), (2) serve as reference and documentation for the rest of the hiring team to know more about the candidate, (3) and ultimately compare final candidates to facilitate and justify the scoring, leveling and hiring decisions.
That being said, make sure you have a proper place to keep track of the candidate’s answers and scores. Ideally that’d be the company’s ATS (Applicant Tracking System) to ensure both privacy compliance and data integrity. This is one of the logistic aspects that your recruiter should guide you in and prepare you in advance.
Once you’ve finished with the assessment and the candidate has left, take a few minutes to go over your notes and write down anything you didn’t have the time to during the interview (or adjust the notes taken by AI), and review/set the final scores for each question/task. Doing this final review, without the pressure of the timer during the interview, is crucial to ensure an objective assessment through proper consideration.
_________
And that’s it! You’ve now structured the recruitment process, gone over qualifier requirements screening, and prepared and used assessment tools based on clear evaluation criteria and rubrics.
Going over these steps has given your process a level of rigour that very few hiring managers achieve (and thus why only 19% of recruitments on average are considered a success2).
If you want to learn more about handling the end-to-end recruitment process with an evidence-based approach that enables the best fit for your role, you can check out our introductory article here.
Feel free to reach out to us at Academic Work if you have any doubts or want support in handling your full recruitment, we are leading industry experts in profile definition, candidate attraction and talent assessments.
Sources:
1. Silva Mangiante, Elaine M.; Peno, Kathy; Northup, Jane, eds. (2021). Teaching and learning for adult skill acquisition: applying the Dreyfus and Dreyfus model in different fields. Adult learning in professional, educational, and community settings. Charlotte, NC: Information Age Publishing, Inc. p. 17. ISBN 978-1-64802-500-6.
2. Leadership Q Study (2020). Why New Hires Fail, The Landmark "Hiring For Attitude". https://www.leadershipiq.com/blogs/leadershipiq/35354241-why-new-hires-fail-emotional-intelligence-vs-skills