Problem Statement
Employee attrition leads to increased recruitment costs, productivity loss, and operational inefficiencies. This project focuses on exploratory data analysis (EDA) and machine learning modeling to identify the key factors driving attrition, enabling the HR department to take proactive, targeted retention measures.
Dataset Overview
The dataset is a modified version of the IBM HR Analytics Employee Attrition & Performance dataset, adapted to reflect realistic business ambiguity.
- Approximately 1,500 employees with a mix of numerical and categorical HR-related attributes.
- Target variable: Attrition (Yes/No).
- Features include Demographics, Job Role, Compensation, Work-Life Balance, and Performance Metrics.
Approximately 16% of employees are at risk of attrition (84% No, 16% Yes). This severe class imbalance requires specialized techniques during preprocessing and evaluation to avoid biased models.
Data Preprocessing Pipeline
To prepare the raw data for modeling, the following preprocessing steps were systematically applied:
- Removal of Zero-Variance Features: Constant columns (e.g., standard working hours = 80 for everyone) offer no predictive value and were removed.
- Removal of ID Columns: Unique identifiers like `EmployeeNumber` were dropped to prevent the model from learning arbitrary noise.
- Label Encoding: Categorical variables were converted into numerical values using Label Encoding to ensure consistency and compatibility across all selected algorithms (XGBoost, LightGBM, and Random Forest).
Exploratory Data Analysis
Extensive visualization revealed distinct patterns between employees who stayed vs. left:
- Compensation & Recognition: Departed employees earned overall less monthly income and received significantly lower incentives (65% lower on average).
- Work Flexibility & Stress: Employees who left reported higher stress ratings, lower work-life balance scores, and had significantly less access to extended leave or remote work opportunities.
- Business Travel: Attrition rate was highest among frequent travelers (23.5%) compared to non-travelers (7.3%).
Machine Learning Approach (Stacking Ensemble)
Rather than relying on a single classifier, a Stacking Ensemble Model was developed to combine the strengths of multiple algorithms and ensure robust generalization:
# Model Architecture
Base Models:
- XGBoost (captures complex, non-linear relationships)
- LightGBM (efficient tree-based learning, handles large datasets)
Meta-Model:
- Random Forest (combines base model predictions with robustness)
Techniques:
- Train-Test Split: 80/20 with stratification
- Class Imbalance: SMOTE (Synthetic Minority Over-sampling Technique) applied ONLY to training data
- Validation: Stratified 5-Fold Cross-Validation
Model Evaluation
Because of the 84/16 class imbalance, standard accuracy is highly misleading. The model was evaluated using robust metrics:
0.9379
ROC-AUC
Excellent ability to distinguish between attrition classes.
0.92
F1-Score
High precision and recall, specifically identifying leaving employees.
Feature Importance & Statistical Validation
The model identified the most critical factors driving employee turnover. To ensure these weren't just random noise, a Mann-Whitney U Test (Wilcoxon Rank-Sum Test) was conducted to statistically validate the findings.
Top Predictors of Attrition:
- Monthly Achievement & Performance Index: High-performing employees were highly likely to leave, possibly signaling burnout or unmet expectations.
- Monthly Income: Lower salary is the strongest signal (Statistically significant: p < 0.001).
- OverTime: Frequent overtime drastically increases attrition risk.
- Distance From Home: Longer commutes correlate positively with employees leaving.
Business Impact
- Targeted Retention: HR can now flag high-performing employees with lower relative income or excessive overtime for immediate retention interventions.
- Cost Reduction: Accurately predicting 93% of attrition cases helps reduce massive recruitment and onboarding costs.
- Policy Adjustment: Emphasizes the need to improve remote work policies and re-evaluate incentive structures.