HR Analytics • Stacking Ensemble • Imbalanced Data

Understanding Why Employees Leave

A comprehensive data analysis pipeline and machine learning model designed to predict employee attrition using historical HR data from an IT consulting firm.

Problem Statement

Employee attrition leads to increased recruitment costs, productivity loss, and operational inefficiencies. This project focuses on exploratory data analysis (EDA) and machine learning modeling to identify the key factors driving attrition, enabling the HR department to take proactive, targeted retention measures.

Dataset Overview

The dataset is a modified version of the IBM HR Analytics Employee Attrition & Performance dataset, adapted to reflect realistic business ambiguity.

Approximately 16% of employees are at risk of attrition (84% No, 16% Yes). This severe class imbalance requires specialized techniques during preprocessing and evaluation to avoid biased models.

Data Preprocessing Pipeline

To prepare the raw data for modeling, the following preprocessing steps were systematically applied:

Exploratory Data Analysis

Extensive visualization revealed distinct patterns between employees who stayed vs. left:

Machine Learning Approach (Stacking Ensemble)

Rather than relying on a single classifier, a Stacking Ensemble Model was developed to combine the strengths of multiple algorithms and ensure robust generalization:

# Model Architecture
Base Models: 
    - XGBoost (captures complex, non-linear relationships)
    - LightGBM (efficient tree-based learning, handles large datasets)

Meta-Model:
    - Random Forest (combines base model predictions with robustness)

Techniques:
    - Train-Test Split: 80/20 with stratification
    - Class Imbalance: SMOTE (Synthetic Minority Over-sampling Technique) applied ONLY to training data
    - Validation: Stratified 5-Fold Cross-Validation

Model Evaluation

Because of the 84/16 class imbalance, standard accuracy is highly misleading. The model was evaluated using robust metrics:

0.9379

ROC-AUC

Excellent ability to distinguish between attrition classes.

0.92

F1-Score

High precision and recall, specifically identifying leaving employees.

Feature Importance & Statistical Validation

The model identified the most critical factors driving employee turnover. To ensure these weren't just random noise, a Mann-Whitney U Test (Wilcoxon Rank-Sum Test) was conducted to statistically validate the findings.

Top Predictors of Attrition:

Business Impact

Back to Portfolio