sats — Multispectral Satellite Image Classification with Band Selection

Bachelor's thesis project — University of Parma, Department of Engineering and Architecture
Supervisor: Prof. Andrea Prati · Co-supervisor: Dr. Leonardo Rossi · A.Y. 2023/24

---

Overview

This project addresses a fundamental challenge in remote sensing: how to classify multispectral satellite imagery efficiently, using only the spectral bands that actually matter.

Using the EuroSAT benchmark — 27,000 Sentinel-2 patches across 10 land-use classes, each with 13 spectral bands — the work pursues two research questions:

1. Which deep learning architecture performs best on multispectral imagery? Six models are systematically compared: three CNNs (ResNet50, Res2Net50, EfficientNet-B3) and three Vision Transformers (Swin, Swin V2, CSWin).
2. Which spectral bands are most informative, and how can we find them automatically? Four original band selection algorithms are designed and evaluated, from a SHAP-based approach to a novel iterative refinement strategy.

---

Key Results

Experiment 1 — Architecture Comparison (13 bands, 200 epochs)

Model	Type	F1-micro	Notes
Swin Transformer	ViT	0.9646	Best overall; stabilizes by epoch 50
Res2Net50	CNN	0.9543	Best CNN; stabilizes by epoch 50
ResNet50	CNN	~0.9543	Comparable accuracy, slower convergence (~190 epochs)
CSWin Transformer	ViT	—	Stabilizes fastest (~30 epochs), lower final accuracy
Swin V2	ViT	—	Lower than Swin V1 in this setting
EfficientNet-B3	CNN	—	Failed to converge; architecture appears incompatible with dense multispectral input

Finding: Swin Transformer outperforms all CNN baselines on EuroSAT. Reducing model parameters by ~20% causes no significant accuracy degradation — a useful insight for resource-constrained deployments.

Experiment 2 — Band Selection (Res2Net and Swin, k = 3 and k = 6 bands)

Res2Net

Algorithm	Bands Selected	F1-micro
Bottom Up	B03-B04-B02	0.8625
Top Down V1	B11-B04-B02	0.8789
Top Down V2	B11-B04-B08	0.9050
Merge	B11-B03-B04	0.8830

3 bands — Top Down V2 wins

Algorithm	Bands Selected	F1-micro
Bottom Up	B03-B02-B04-B05-B08-B11	0.9403
Top Down	B11-B04-B02-B08-B12-B06	0.9354
Merge	B11-B03-B04-B02-B08-B05	0.9435

6 bands — Merging wins

Swin Transformer

Algorithm	Bands Selected	F1-micro
Bottom Up	B08-B03-B04	0.9296
Top Down	B11-B02-B04	0.9046
Merge (v1)	B11-B08-B02	0.9273

3 bands — Bottom Up wins

Algorithm	Bands Selected	F1-micro
Bottom Up	B08-B03-B04-B05-B02-B10	0.9531
Top Down	B11-B02-B04-B10-B08-B12	0.9486
Merge	B11-B08-B02-B03-B04-B05	0.9563
Iterative Top Down	B02-B04-B06-B07-B10-B11	0.9536

6 bands — Merging wins; Iterative Top Down outperforms Bottom Up and Top Down

Key finding: Bands B09 and B08A are consistently the least informative across models and algorithms. The Top Down V2 (SHAP-based) approach is significantly more computationally efficient than Bottom Up: it requires a single training run instead of one per band, while matching or exceeding Bottom Up accuracy.

---

Band Selection Algorithms

Four algorithms were designed and implemented to identify the most informative spectral band subsets:

1. Bottom-Up
Train the model with each of the 13 bands individually. Rank bands by their single-band F1 score and iteratively add the next best band. Accurate but expensive — requires 13 separate training runs.

2. Top-Down (V1 & V2)
Train once on all 13 bands, then compute SHAP values to measure each band's contribution to correct predictions. V2 extends V1 by incorporating SHAP values from incorrect-class predictions (sign-inverted), making the importance estimate more complete. Requires a single training run — far more efficient than Bottom-Up.

3. Merging
Combine the rankings from Bottom-Up and Top-Down by interleaving their top-ranked bands. Designed to capture complementary information that each method independently misses. Achieves the best 6-band results for both Res2Net and Swin.

4. Iterative Top-Down
An improvement over Top-Down: after computing SHAP values, remove the k least important bands (e.g. k=3), retrain, and repeat. Allows SHAP to account for inter-band correlations that change as bands are removed. More precise than Top-Down without proportionally increasing cost. Preliminary results outperform both Bottom-Up and Top-Down — identified as the most promising direction for future work.

---

Requirements

• Python 3.10
• CUDA 12.1
• PyTorch 2.1.0

---

Installation

Option A — Conda (recommended)

conda env create -f conda-environment.yaml
conda activate l1bsr3

Option B — pip

pip install -r requirements.txt

---

Usage

All phases are controlled by a YAML config file. Example configs are in cfgs/.

1. Compute dataset statistics (first-time setup)

Required before training on a new dataset to compute mean and std for normalization:

python src/main.py --phase mean_std --config cfgs/eurosat_Res2Net_v1.yml

Copy the output values into the stats section of your config file.

2. Train

python src/main.py --phase train --config cfgs/eurosat_Res2Net_v1.yml

3. Evaluate

python src/main.py --phase test --config $CONFIG_FILE --output $OUT_DIR --batch_size 32 --epoch 300

4. Visualize predictions

python src/main.py --phase vis --config $CONFIG_FILE --output $OUT_DIR --num_images 2 --epoch 200

5. Benchmark inference speed

python src/main.py --phase avg_time --config $CONFIG_FILE --repeat_times 100 --warm_times 10

---

SHAP Explainability

SHAP (SHapley Additive exPlanations) values are computed per spectral band to quantify each band's average contribution across all predictions and all spatial positions. This underpins both the Top-Down and Iterative Top-Down band selection algorithms.

SHAP analysis outputs are stored in Shap/<model>/SHAP_values/. Analysis and plotting scripts are in scripts/:

# Export per-band F1 scores to CSV
python scripts/bands_f1_micro_tocsv.py

# Plot band importance curves
python scripts/plot_multiple_bands_f1.py

# Compare model variants
python scripts/diff_f1_micro_tocsv.py

---

Configuration

Configs use a __base__ inheritance system. A minimal experiment config:

__base__: eurosat_v1.yml

cls:
  in_channels: 13
  num_classes: 10
  type: Res2Net

Band-subset experiments override dataset.kwargs.bands to restrict input channels:

dataset:
  kwargs:
    bands: ['B02', 'B04', 'B06', 'B07', 'B10', 'B11']

cls:
  in_channels: 6

---

Project Structure

sats/
├── cfgs/                  # Experiment configs (YAML)
├── src/
│   ├── cls/               # Classification models
│   │   └── models/        # Swin, CSWin, Res2Net, ResNet, EfficientNet
│   ├── datasets/          # EuroSAT, WorldStrat, Sen2Venus loaders
│   ├── scripts/           # Band importance analysis and plotting
│   └── main.py            # Entry point (train / test / vis / mean_std / avg_time)
├── Shap/                  # SHAP output per model variant
├── data/                  # Dataset files (not tracked)
├── requirements.txt
└── conda-environment.yaml

---

About

This repository contains the code developed for the bachelor's thesis:

"Multispectral Satellite Image Classification: Comparison of Deep Learning Models and Band Selection Algorithms"
Diego Terzi — University of Parma, Dept. of Engineering and Architecture
Supervisor: Prof. Andrea Prati · Co-supervisor: Dr. Leonardo Rossi
Academic Year 2023/24

---

Citation

If you use this work, please cite it using the information in CITATION.cff.

---

License

Apache License 2.0 — see LICENSE for details.