Cost-Sensitive Abstention for Human Activity Recognition

Six classifiers built from scratch in NumPy plus a cost-sensitive abstention (reject) framework — a HAR system that knows when not to predict, refusing low-confidence cases so the answers it does give clear a target accuracy.

Overview

Human Activity Recognition from smartphone/accelerometer signals, evaluated on two datasets (UCI HAR and WISDM). Every classifier, dimensionality-reduction method, calibration routine, abstention rule, and ensemble strategy is implemented from scratch in NumPy/SciPy — scikit-learn is used only for validation metrics, never for the models.

The core contribution is a selective-prediction layer: rather than forcing a label on every window, the system calibrates its posteriors and applies a cost-sensitive reject option, so it abstains on the hardest cases and its accepted predictions meet a chosen accuracy target. That's the behavior you want in safety-critical monitoring, where a wrong call is worse than "ask a human."

Results

Test set: UCI HAR (6 activities) and WISDM (cross-dataset check). "Base" = best single classifier, no abstention.

• Base accuracy: 94.0% on UCI HAR (from-scratch models).
• Abstention reaches the 95% accuracy target by abstaining on just 2.7% of samples on UCI HAR — and the same target requires ~12% abstention on the noisier WISDM, showing that abstention behavior is dataset/context dependent, not a fixed property of a model.
• Calibrated posteriors (Platt scaling; ECE reported) make the reject option trustworthy; ensemble fusion further sharpens the risk-coverage trade-off.

Full derivations, tables, and the 5-page write-up are in report.pdf.

Implemented From Scratch

• Classifiers: Bayesian Gaussian (MLE + MAP), k-Nearest Neighbors, Parzen-window density estimation, Gaussian Mixture Model (EM), Multilayer Perceptron (backprop) — plus K-Means (unsupervised).
• Dimensionality reduction: PCA and Fisher LDA.
• Calibration: Platt scaling + Expected Calibration Error (ECE).
• Abstention: a cost-sensitive reject option over calibrated posteriors (Chow-style), swept into risk-coverage curves.
• Ensembles: three fusion strategies (majority, accuracy-weighted, entropy-weighted).

Tech Stack

• Languages: Python
• From scratch (NumPy/SciPy): classifiers, PCA/LDA, EM, backprop, calibration, abstention, ensembles
• Tooling: matplotlib/seaborn/plotly, Jupyter; scikit-learn for evaluation metrics only

Repository Layout

src/
  classifiers/  bayesian_gaussian · knn · parzen · gmm_em · mlp
  abstention.py     cost-sensitive reject option (core contribution)
  calibration.py    Platt scaling + ECE
  ensemble.py       three fusion strategies
  dimensionality.py PCA + Fisher LDA
  data_loader.py · wisdm_loader.py · unsupervised.py · visualization.py
notebooks/    01–08: EDA → classifiers → comparison → abstention → ensembles → WISDM → safety
figures/      risk-coverage + per-activity plots
report.pdf    5-page IEEE-style write-up
download_data.py  fetches the UCI HAR dataset

Running Locally

Prerequisites: Python 3.10+.

python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

python download_data.py       # UCI HAR → ./data/
# WISDM: download from https://www.cis.fordham.edu/wisdm/dataset.php → ./data/WISDM/

jupyter notebook              # run notebooks 01–08 in order (~15–20 min total)

Datasets are not included (download via the script/link above). Citations: UCI HAR — Anguita et al. (2013); WISDM — Kwapisz, Weiss & Moore (2011).

License

MIT — see LICENSE.