Diabetes Risk Prediction

Machine learning project to predict diabetes risk from demographic and clinical features, with a recall-oriented evaluation strategy suited to medical screening.

Project overview

This project applies supervised binary classification to predict whether a patient has diabetes, using tabular demographic and clinical data.

The modeling strategy prioritizes recall: in a screening context, false negatives (missing a diabetic patient) are more costly than false positives (flagging a healthy patient for follow-up). This shapes every decision from model selection to hyperparameter tuning.

The project started as an academic competition notebook and has been refactored into a reproducible, portfolio-ready repository.

Dataset

Training rows	95,000
Test rows	5,000
Positive class prevalence (training set)	8.5%

Input features:

• age
• gender
• hypertension
• heart_disease
• smoking_history
• bmi
• HbA1c_level
• blood_glucose_level

Approach

Data cleaning:
EDA revealed two data quality issues in the training set: a concentration of records at age ≥ 80 and an artificial spike at BMI = 27.32 (accounting for ~29% of rows). These were removed from training. The test set is not filtered, as EDA-derived decisions should not influence held-out data.

Encoding:
Categorical variables (gender, smoking_history) are mapped to integers. BMI is converted to an ordinal category (Underweight / Normal / Overweight / Obese) and the raw BMI column is dropped.

Models evaluated:
Three models were trained and compared on a stratified 80/20 train/test split:

Model	Train Recall	Test Recall
Decision Tree	95.82%	95.48%
Random Forest	99.91%	70.33%
XGBoost	99.45%	99.49%

Final model

XGBoost was selected as the final model.

The Decision Tree achieves consistent train/test recall but offers a simpler decision boundary. The Random Forest shows severe overfitting (99.91% train vs. 70.33% test). XGBoost achieved the best test recall among the evaluated models, with limited evidence of overfitting.

Class imbalance is addressed via the scale_pos_weight parameter, tuned through GridSearchCV with recall as the scoring objective.

On false positives and false negatives

In this context:

• A false negative means a diabetic patient is not flagged. This is the more serious error — it delays diagnosis and treatment.
• A false positive means a healthy patient is referred for further testing. This is disruptive but recoverable.

Maximizing recall accepts a higher false positive rate. The XGBoost model reflects this trade-off explicitly.

This project intentionally prioritizes recall over precision, since false negatives are considered more costly in a diabetes-risk screening context. As a result, the final model is designed to identify as many positive cases as possible, accepting a higher number of false positives. The model should therefore be interpreted as a screening/prioritization tool rather than a definitive diagnostic system.

Repository structure

diabetes-risk-prediction/
├── data/
│   └── raw/
│       ├── train.csv
│       └── test.csv
├── models/
├── notebooks/
│   ├── 01_original_competition_notebook.ipynb
│   └── 02_project_notebook.ipynb
├── src/
│   ├── data.py
│   └── preprocessing.py
├── requirements.txt
└── README.md

Notebooks:

• 01_original_competition_notebook.ipynb — the original competition notebook, preserved for context. It was developed in Google Colab and reflects the initial competition workflow. It is not intended for local re-execution.
• 02_project_notebook.ipynb — the refactored portfolio notebook, intended for reproducible local execution. This is the primary notebook for this project.

How to run

This project requires Python 3.11 and assumes Jupyter is already installed in your environment.

pip install -r requirements.txt
jupyter notebook notebooks/02_project_notebook.ipynb

The notebook is self-contained. Run all cells from top to bottom. Final predictions are saved to data/predictions.csv.