classifier_zoo.md

📚 Table of Contents (Classifiers Section)

• ⚙️ Training Algorithms
• 🧠 Model Architectures & Pretraining
• 🧪 Training Scripts for Different Classifiers
  • CelebA
  • MetaShift
  • NIH ChestX-ray (via SubpopBench)
  • Waterbirds
  • 🏥 Training ResNet50 on NIH-CXR
  • 🏥 Training EN-B5 on RSNA-Mammo
  • 🏥 Training EN-B5 on VinDr-Mammo

⚙️ Training Algorithms

We support a wide range of algorithms for bias mitigation and robust training.
Click here to view the implementation.

• ERM — Empirical Risk Minimization (Vapnik, 1998)
• IRM — Invariant Risk Minimization (Arjovsky et al., 2019)
• GroupDRO — Group Distributionally Robust Optimization (Sagawa et al., 2020)
• CVaRDRO — Conditional Value-at-Risk DRO (Duchi & Namkoong, 2018)
• Mixup (Zhang et al., 2018)
• JTT — Just Train Twice (Liu et al., 2021)
• LfF — Learning from Failure (Nam et al., 2020)
• LISA — Learning Invariant Selective Augmentation (Yao et al., 2022)
• DFR — Deep Feature Reweighting (Kirichenko et al., 2022)
• MMD — Maximum Mean Discrepancy (Li et al., 2018)
• CORAL — Deep Correlation Alignment (Sun & Saenko, 2016)
• ReSample — Data Re-Sampling (Japkowicz, 2000)
• ReWeight — Cost-Sensitive Re-Weighting (Japkowicz, 2000)
• SqrtReWeight — Square-Root Reweighting (Japkowicz, 2000)
• Focal — Focal Loss (Lin et al., 2017)
• CBLoss — Class-Balanced Loss (Cui et al., 2019)
• LDAM — Label-Distribution-Aware Margin Loss (Cao et al., 2019)
• BSoftmax — Balanced Softmax (Ren et al., 2020)
• CRT — Classifier Re-Training (Kang et al., 2020)

---

🧠 Model Architectures & Pretraining

Supported image model architectures are implemented here.

ResNet-50 Variants

• resnet_sup_in1k — Supervised on ImageNet-1K
• resnet_sup_in21k — Supervised on ImageNet-21K (Ridnik et al., 2021)
• resnet_simclr_in1k — SimCLR (Chen et al., 2020)
• resnet_barlow_in1k — Barlow Twins (Zbontar et al., 2021)
• resnet_dino_in1k — DINO (Caron et al., 2021)

ViT-B Variants

• vit_sup_in1k — Supervised on ImageNet-1K (Steiner et al., 2021)
• vit_sup_in21k — Supervised on ImageNet-21K (Steiner et al., 2021)
• vit_clip_oai — CLIP (OpenAI) (Radford et al., 2021)
• vit_clip_laion — OpenCLIP on LAION-2B (OpenCLIP)
• vit_sup_swag — Supervised on SWAG (Singh et al., 2022)
• vit_dino_in1k — DINO (Caron et al., 2021)

---

🧪 Training scripts for different classifiers

Training scripts for different datasets are available in the repo:

• 📷 CelebA
• 🐶 MetaShift
• 🫁 NIH ChestX-ray using subpopshift
• 🐦 Waterbirds

🏥 Training ResNet50 using ERM (resnet_sup_in1k) as classifier for NIH-CXR

We train the ERM variant for NIH-CXR as follows:

python ./src/codebase/train_classifier_CXR.py --img-size 224 --arch ResNet50 --lr 1e-5

🏥 Training EfficientNet-B5 (EN-B5) using ERM as classifier for RSNA-Mammo

We train the ERM variant for RSNA-Mammo as follows:

python ./src/codebase/train_classifier_Mammo.py \
  --data-dir '/restricted/projectnb/batmanlab/shared/Data/RSNA_Breast_Imaging/Dataset/' \
  --img-dir 'RSNA_Cancer_Detection/train_images_png' \
  --csv-file 'RSNA_Cancer_Detection/rsna_w_upmc_concepts_breast_clip.csv' --start-fold 0 --n_folds 1 \
  --dataset 'RSNA' --arch 'tf_efficientnet_b5_ns-detect' --epochs 9 --batch-size 6 --num-workers 0 \
  --print-freq 10000 --log-freq 500 --running-interactive 'n' \
  --lr 5.0e-5 --weighted-BCE 'y' --balanced-dataloader 'n' \
  --tensorboard-path="/restricted/projectnb/batmanlab/shawn24/PhD/Ladder/out/RSNA/fold0" \
  --checkpoints="/restricted/projectnb/batmanlab/shawn24/PhD/Ladder/out/RSNA/fold0" \
  --output_path="/restricted/projectnb/batmanlab/shawn24/PhD/Ladder/out/RSNA/fold0" \
  --label "cancer"

🏥 Training EfficientNet-B5 (EN-B5) as classifier for VinDr-Mammo

We train the ERM variant for VinDr-Mammo as follows:

python ./src/codebase/train_classifier_Mammo.py \
  --data-dir '/restricted/projectnb/batmanlab/shared/Data/RSNA_Breast_Imaging/Dataset' \
  --img-dir 'External/Vindr/vindr-mammo-a-large-scale-benchmark-dataset-for-computer-aided-detection-and-diagnosis-in-full-field-digital-mammography-1.0.0/images_png' \
  --csv-file 'External/Vindr/vindr-mammo-a-large-scale-benchmark-dataset-for-computer-aided-detection-and-diagnosis-in-full-field-digital-mammography-1.0.0/vindr_detection_v1_folds_abnormal.csv' \
  --dataset 'ViNDr' --arch 'tf_efficientnet_b5_ns-detect' --epochs 20 --batch-size 8 --num-workers 0 \
  --print-freq 10000 --log-freq 500 --running-interactive 'n' \
  --lr 5.0e-5 --weighted-BCE 'y' --balanced-dataloader 'n'  --n_folds 1  --label "abnormal" \
  --tensorboard-path="/restricted/projectnb/batmanlab/shawn24/PhD/Ladder/out/ViNDr/fold0" \
  --checkpoints="/restricted/projectnb/batmanlab/shawn24/PhD/Ladder/out/ViNDr/fold0" \
  --output_path="/restricted/projectnb/batmanlab/shawn24/PhD/Ladder/out/ViNDr/fold0"