Dr Hui Xiong – Matching People with their Ideal Job Using Artificial Intelligence

The main responsibility of recruitment consultants is to match individuals to jobs that best suit their professional experience, skills, capabilities, dispositions, and academic background. Dr Hui Xiong at Rutgers University has been leading efforts to develop tools based on artificial neural networks that can automatically identify the right individuals for specific roles. Over the past few years, he has led efforts to design a comprehensive intelligent HR management system that could bring significant intelligence in human resource management.

Tech-fuelled Evolution of Recruitment

Practices in the field of human resources (HR) management have been continuously evolving over the past decade or so, in large part due to the emergence and proliferation of online recruitment platforms.  Platforms such as LinkedIn, Lagou, Glassdoor and Indeed have opened new possibilities for recruiters to seek out and source fresh talent, allowing HR agents to easily advertise new job opportunities to a wider range of potential applicants.  For instance, in 2017 alone, LinkedIn had 467 million users and 3 million active job listings.

In addition to allowing employers and HR agencies to conveniently access talent worldwide, online recruitment platforms have enabled the collection of vast amounts of employment-related data. This data could aid the development of tools to automate or simplify the recruitment process, such as job recommendation systems or models that match individuals to suitable jobs.

The main responsibility and objective of recruitment agents is to identify the best person-job fit. Past studies suggest that a person’s suitability for a given job depends on several factors, including obvious ones such as their practical skills, academic background, and past professional experience, as well as psychological features, such as their personality traits, attitude, abilities, and preferences.

Dr Hui Xiong, a Distinguished Professor at Rutgers, the State University of New Jersey, has been developing artificial intelligence (AI) approaches that could help recruiters to identify ideal candidates for a given job faster and more efficiently. These tools are trained on datasets containing a wide range of real employment-related data.

‘Our suite of algorithms and tools have been developed for supporting intelligent decision-making in talent management, organisation management, and culture management,’ Dr Xiong says. ‘At the talent level, this powerful set of innovative AI techniques can support talent recruiting, performance evaluation, talent retention, talent development, and person-job matching. At the organisation level, they can help to support organisation stability analysis, leadership development and management, and intelligent team development. Finally, at the culture level, they can help to identify key factors for the development of creative culture and perform organisation culture analysis.’

Tackling the Person-Job Fit Problem

In recent years, many computer scientists worldwide have been trying to develop computational tools that could speed up or simplify online recruitment, such as recommender systems. These systems, which are often trained on data collected by online recruitment platforms, can learn to recommend suitable jobs for individual candidates by analysing their employment history and background information, as well as different features of job advertisements available online.

Unfortunately, automatic job recommendation systems typically need to be trained on large amounts of data reviewed and annotated by humans. This makes their development expensive and time-consuming, which in turn prevents their large-scale and widespread implementation. Over the past few years, Dr Xiong and his team created new systems based on artificial neural networks that could significantly simplify and innovate HR management practices.

‘The big data trend has made its way to talent management,’ explains Dr Xiong. ‘Indeed, the availability of large-scale HR data provide unparalleled opportunities for business leaders to understand talent behaviours and generate useful talent knowledge, which can be used to develop intelligence for real-time decision making and effective people management at work.’

The PJFNN Model

In 2018, Dr Xiong and other researchers at Baidu’s Talent Intelligence Centre developed a new computational model called the Person-Job Fit Neural Network, or ‘PJFNN’. This model is based on a convolutional neural network, which is a machine-learning algorithm commonly used by computer scientists to tackle tasks that involve the analysis of large amounts of images or other data.

The model created by Dr Xiong and his colleagues can learn the joint representation of person-job fitness by analysing past online job applications. In other words, it can match a candidate’s qualification to the requirements for a specific role. Its unique structure allows the model to estimate whether a given candidate matches a role, while also identifying specific requirements in the job posting that are satisfied by the candidate. 

Dr Xiong and his colleagues evaluated the PJFNN model in a series of experiments, using a large-scale dataset containing employment-related information. The model performed remarkably well and could predict the person-job fit with high levels of reliability and accuracy.

‘Our suite of algorithms and tools have been developed for supporting intelligent decision-making in talent management, organisation management, and culture management.’

The TAPJFNN Model

After they published their first paper on the PJFNN model, Dr Xiong and his colleagues continued working on the model, with the aim of broadening its capabilities and further improving its performance. In 2020, they proposed a new version of the model, dubbed the Topic-based Ability-aware Person-Job Fit Neural Network (TAPJFNN) framework.

The TAPJFNN framework, which was also trained on job application data gathered in the past, utilises a recurrent neural network with a hierarchical attention mechanism to create semantic representations of requirements listed in job postings and the experiences of candidates. Subsequently, the model employs two strategies to quantify the importance of different job requirements, as well as the extent to which individual professional experiences mean that a candidate meets a specific requirement.

As part of their study, Dr Xiong and his colleagues also devised a strategy to refine the model’s ability to predict the best person-job fit based on a candidate and employer’s past recruitment records. They also showed how the TAPJFNN model could be used for two different HR-related applications, namely to source new talent and recommend suitable jobs to candidates.

Finally, the researchers evaluated their model and compared it with other machine-learning tools for sourcing new talent or recommender systems. In their experiments, the TAPJFNN model performed remarkably well, outperforming several baseline techniques for HR management.

The SSCN Model

Earlier this year, Dr Xiong and his colleagues also introduced a technique to assess the value of individual professional skills within the global job market. This technique was designed to help companies and HR agencies worldwide to identify the most suitable type of candidates for a given role.

Initially, Dr Xiong and his collaborators represented the task of assessing the value of job skills as the composition of a set of requirements associated with contextual job-related information. In their representation, a job’s salary is assumed to be dependent on the context-related value of individual professional skills.  

Subsequently, the researchers developed an enhanced neural network-based model, dubbed the Salary-Skill Composition Network (SSCN). This model can discern between different job skills and evaluate their individual value based on how much they contribute to the job salary. In a series of experiments and evaluations, the SSCN model effectively assigned value to multiple job skills, while also outperforming benchmark job salary prediction models.

New Tools to Improve Recruitment

In the future, the research carried out by Dr Xiong could help to significantly improve HR practices and speed up recruitment processes. So far, the tools his team has developed have achieved highly promising results, and they could soon be tested and employed by real recruitment agents. 

The computational tools developed by Dr Xiong and his colleagues have several advantages over traditional HR practices. ‘Traditional HR management systems are usually based on rules extracted from experts’ long-term experiences and have various disadvantages in practice,’ explains Dr Xiong. ‘For instance, traditional HR management decisions are based on fragmented information and are subjective. The predictive management of risks in HR is highly desired but difficult to implement with traditional HR systems.’

His team’s approaches can analyse vast amounts of employment-related data and identify good person-job fits much faster than human agents would. ‘My team has exploited advanced big data and AI technologies for smart HR management and designed the pioneering talent intelligent management system, which can support HR management in a transparent, objective, comprehensive, and predictive manner,’ Dr Xiong says.

The three models presented in Dr Xiong’s recent papers, namely the JFNN, TAPJFNN, and SSCN models, could be highly valuable tools for both employers and recruitment agencies, as they could help them to identify suitable candidates for specific roles faster and more reliably. In addition, the researchers developed the talent intelligent management system, a management system that could greatly enhance current HR practices.

‘The talent intelligent management system is the first comprehensive intelligent HR management system that has changed the paradigm for traditional HR management,’ concludes Dr Xiong.