Continuous improvement process in aptitude diagnostics

Today’s working world is in a constant state of change. Due to increasing globalization and digitalization, job profiles, tasks, work content and processes are changing and, as a result, the competencies that employees need to have to be successful in a fast-moving world of work are also changing. For example, the increasing automation of routine tasks is helping to make analytical and cross-disciplinary skills such as adaptability and willingness to learn more relevant.

Against this background, it is surprising that aptitude diagnostics has changed little over the past 50 years. Therefore, biography-centered methods alone are still used in a large part of the companies (Schuler et al., 2007). Such biography-centered procedures include, for example, examining the applicant’s background on the basis of the CV or in the context of an unstructured or a structured interview. These are so-called backward-facing procedures – past performance in the past is evaluated as indicators for future performance on the job (Schuler & Hoft, 2007). And this (so far) mostly with success. After all, it is not for nothing that a central axiom of psychological measurement states that past behavior is the best predictor of future behavior (Sutton, 1994; Norman & Conner, 1996).

‍

Aptitude diagnostics: “Aptitude diagnostics is a collective term for psychological selection procedures used to test a fit between an applicant and the workplace.” (Schuler & Hoff 2007)

‍

However, this assumption becomes problematic in a constantly changing world of work. After all, if job profiles and job content are constantly changing, experience in a previous job naturally loses predictive power. For example, let’s imagine that an innovative start-up in the financial sector is looking for a software developer to join a dynamic project team that uses the latest methods in the software field. Applicants have already gained several years of professional experience as a developer in a well-known banking institution, but worked there exclusively with a single established, but rather outdated operating system. It may come as little surprise that good performance in the previous job alone will not be a good indicator of career success in the start-up. More importantly, it will also be about hiring an employee who can learn new methods quickly, is adaptable, and has a high level of problem-solving skills. All of these are competencies and characteristics that can only be inferred to a limited extent from one’s professional career.

Consequently, the focus is shifting from a very narrow, occupation-specific aptitude assessment to a broader concept of occupational aptitude, which increasingly includes supra-disciplinary aptitude dimensions, such as adaptability and intelligence, in addition to occupation-specific competencies. This involves moving away from a static concept of suitability, which only determines the applicant’s status quo, to an analysis of potential, which also takes into account the applicant’s development potential.

Providing the instruments for such an analysis of potential is increasingly becoming the task of professional psychological aptitude diagnostics. Against this background, so-called design-oriented processes in particular are gaining in importance. This refers to psychological test procedures which serve to record stable characteristics such as general cognitive (e.g. intelligence, ability to concentrate) and non-cognitive abilities (e.g. personality traits, attitudes, interests).

‍

‍

1. Requirement analysis

At the beginning of every professional aptitude diagnostics should be a well-founded requirement analysis (DIN 33430, 2016). Within this, criteria for professional success are defined. The central guiding question here is: What does the applicant need to bring to be successful in the job? Experience-guided intuitive methods or person-based empirical methods can be used to define such job profiles (Schuler & Kanning, 2014).

Experience-guided intuitive methods usually rely on expert judgment, such as the manager’s assessment. Based on their experience, they assess which competencies and personality traits are relevant for success in the job. Person-based empirical methods, on the other hand, use statistical correlations between the characteristics of professionals and their job performance to derive job profiles. If, for example, it can be observed that financial officers with a high level of the personality trait conscientiousness work particularly error-free and successfully, conscientiousness would be included in the job profile. Empirical-personal methods have the advantage that they largely exclude distortions due to subjective perceptions of the experts and can continuously adapt to changes in the world of work. For example, managers tend to designate competencies that they themselves believe they possess and are often late in noticing changes in required competencies over time.

However, a well-founded requirement analysis that continuously adapts to the changes in a fast-moving working world is the cornerstone for the selection and design of suitable test procedures. After all, to be able to judge if a person is suitable for a specific job or not, HR managers need to know what is really relevant to the job – and this is becoming harder than ever in a complex and constantly changing world of work. However, empirically oriented requirement analysis is still rarely used. As a result, it is often possible to observe a test construction that is not in line with actual practice: So, skills and characteristics are often tested with the help of psychological tests, which are only (still) hardly related to professional success in the respective job.

‍

2. Selection and design of the test procedure

If there is agreement on the aptitude dimensions to be recorded, the next step is the selection and design of the test procedure. The used methods should always be critically examined with relation to the quality criteria of the classical test theory – especially objectivity, reliability and validity. Below you will find detailed information on the design of test procedures based on the central quality criteria.

‍

3. Selection decision

The third step is the evaluation and interpretation of the test results and finally the selection decision: A decision is made either for or against the applicant. The aim should be to minimize the proportion of applicants rejected despite their actual suitability (so-called false negatives) and the proportion of applicants accepted despite their lack of suitability (so-called false positives).

‍

4. Evaluation

In practice, aptitude diagnostics often ends at this point. Not so according to the requirements for job-related aptitude diagnostics specified in DIN 33430, which provides for a comprehensive evaluation and critical reflection of the implemented selection process. While the process of evaluation has become indispensable in other areas, such as production, it unfortunately rarely takes place in personnel selection. In this context, evaluation offers great potential for uncovering weaknesses, adjusting them, and so gradually optimizing the selection process (Nachtwei & Schermuly, 2009). For example, if it is determined that unsuitable applicants were hired based on a test procedure, the test can be adjusted accordingly to avoid this in future selection decisions. Such a process not only allows for continuous development of the procedures, but also for the perception of possible changes in the requirements for the applicants (cf. step 1).

‍

Aptitude diagnostics must not be a one-way street

The examples show: To meet the requirements of a constantly changing world of work, aptitude diagnostics must not be understood as a one-way street which ends with the selection of an applicant. Rather, we need to understand aptitude diagnostics in terms of a continuous improvement process (see figure), in the context of which we learn more and more about the connections between the applicant’s characteristics and professional success. However, the reality today is often different. Validated test procedures, for example, usually have standardization cycles of more than 15 years, and job profiles are also only sporadically checked for their continued validity.

However, the variety of data generated during the selection process provides a wide range of opportunities for optimizing personnel selection. With the help of intelligent evaluation and interpretation by combining psychological and statistical expertise, patterns can be identified that are difficult for humans to comprehend. This is achieved, for example, with the help of self-learning algorithms. For example, modern apps in the field of aptitude diagnostics make use of machine learning methods to continuously improve their test procedures. Instead of developing and standardizing a test at a specific point in time, data on the success of a selection decision is used to adjust job profiles, optimize test content, and determine the best possible statistical correlation between job success and the applicant’s characteristics. Through this iterative process, the estimate of the applicant’s suitability eventually becomes better and better and the probability of error is reduced. HR managers are less likely to get it wrong, and the company benefits through cost savings thanks to more objective and validated personnel selection.

Sounds exciting? Read here how such an interaction between psychological diagnostics and intelligent data analysis integrated in a single app can look like.

‍

‍