Hybrid SBERT Retrieval and Random Forest Reranking for Career Recommendation with Skill Gap Analysis

Resumo

Transformer-based models like Sentence-BERT (SBERT) have improved resume-job matching through semantic similarity; however, there has been no optimization of ranking performance. Career recommendation systems that rely solely on semantic similarity will result in a sub-optimally ordered list of job roles that are relevant. To overcome this limitation, we propose a two-stage hybrid career recommendation framework that utilizes skill gap analysis. In the first stage, an SBERT retriever fits candidates with job descriptions semantically to form a candidate shortlist. A Random Forest Re-ranker that is supervised was used during phase 2 of this project to further improve on the conclusion of ordering candidates based on similarities to potential matches. The Reranker uses created features such as baseline similarity values, the rank position of the candidates and any constraints of their role (for example, Senior or Junior). The combined (or 'ensemble') of both semantic properties and the supervised property of candidates must be calibrated so that they maintain a realistic variance and decrease the potential of over fitting. The Hybrid Architecture was tested using a multisource job search dataset. The experimental results suggest that the Hybrid Architecture significantly outperformed the base SBERT model. These results show that the combination of using a supervised reranker and semantic retrieval can greatly improve the accuracy of recommending a given candidate to a particular job opportunity.