RepCLBlock: Building an Inception-like Unit for Continual Learning in Remote Sensing

Anonymous submission

Under Review

Abstract: Continual Learning (CL) aims to endow neural networks with the capability for incremental learning and adaptation, requiring a careful balance between stability and plasticity. While most research has focused on optimizing algorithms, the impact of network architecture remains underexplored, particularly at the micro-architectural level of fundamental building blocks. In this paper, we introduce an architecture-centric paradigm for CL, inspired by the biological principle of Neural Degeneracy, where the brain uses heterogeneous yet functionally similar pathways to achieve robust and flexible learning. We systematically translate this biological principle into five actionable design guidelines for developing robust and plastic network modules. Following these guidelines, we develop the Re-parameterization Continual Learning Block (RepCLBlock), a universal, plug-and-play building block that instantiates these principles using Structural Re-parameterization (SR) to maintain inference efficiency. Experiments demonstrate that simply replacing blocks in existing architectures with RepCLBlock consistently and significantly improves the performance of various CL methods across different architectures and datasets.

🚀 Key Features

• 🧠 Brain-inspired Design: Translates Neural Degeneracy principle into actionable architectural guidelines
• 🔧 Plug-and-Play: Universal building block that can replace existing blocks in any architecture
• ⚡ Inference Efficient: Uses Structural Re-parameterization to maintain efficiency during inference
• 📈 Consistent Improvements: Significant performance gains across various CL methods and datasets
• 🌍 Cross-Domain: Effective on both remote sensing and general-purpose datasets

🏗️ Key Components

• blocks/repclblock.py: Core RepCLBlock implementation.
• blocks/repconv.py: Convolution components of RepCLBlock .
• blocks/repclnet.py: RepCLNet architectures.
• utils/repclblock_verify.py: Verification script for the re-parameterization of RepCLBlock.
• utils/convert.py: Model deployment conversion utility.

📋 Requirements

• Python 3.8+
• PyTorch 1.8+
• torchvision
• numpy
• Pillow

🛠️ Installation

git clone https://github.com/anonymous/RepCLBlock.git
cd RepCLBlock
pip install -r requirements.txt

📊 Datasets

We evaluate RepCLBlock on four benchmark datasets:

Dataset Category	Dataset Name	Number of Classes	Incremental Learning Steps	Classes per Step	Download Link
Remote Sensing	UCMerced-LandUse	21	3	7	link
Remote Sensing	AID	30	5	6	link
Remote Sensing	SIRI-WHU	12	6	2	link
General Dataset	CIFAR-100	100	10	10	——

Replace the dataset paths with your own in utils/data.py:

# For UCMerced-LandUse
train_dir = "/path/to/UCMerced_LandUse_Splited/train/"
test_dir = "/path/to/UCMerced_LandUse_Splited/valid/"

# For AID
train_dir = "/path/to/AID_Splited/train/"
test_dir = "/path/to/AID_Splited/val/"

# For SIRI-WHU
train_dir = "/path/to/SIRI-WHU_Splited/train/"
test_dir = "/path/to/SIRI-WHU_Splited/val/"

🏃‍♂️ Quick Start

ResNet-18 vs RepCLNet-18

UCMerced-LandUse (UC21-inc7):

# ResNet-18 (Baseline)
python test.py --config ./exps/replay_ucmerced_resnet18.json --file_id Resnet18-replay-UC21-inc7 --gpu_id 0
python test.py --config ./exps/wa_ucmerced_resnet18.json --file_id Resnet18-wa-UC21-inc7 --gpu_id 1
python test.py --config ./exps/icarl_ucmerced_resnet18.json --file_id Resnet18-icarl-UC21-inc7 --gpu_id 2
python test.py --config ./exps/foster_ucmerced_resnet18.json --file_id Resnet18-foster-UC21-inc7 --gpu_id 3

# RepCLNet-18 (Ours)
python test.py --config ./exps/replay_ucmerced_repclnet_18.json --file_id RepCLNet_18-replay-UC21-inc7 --gpu_id 0
python test.py --config ./exps/wa_ucmerced_repclnet_18.json --file_id RepCLNet_18-wa-UC21-inc7 --gpu_id 1
python test.py --config ./exps/icarl_ucmerced_repclnet_18.json --file_id RepCLNet_18-icarl-UC21-inc7 --gpu_id 2
python test.py --config ./exps/foster_ucmerced_repclnet_18.json --file_id RepCLNet_18-foster-UC21-inc7 --gpu_id 3

AID (AID30-inc6):

# ResNet-18 (Baseline)
python test.py --config ./exps/replay_aid_resnet18.json --file_id Resnet18-replay-AID30-inc6 --gpu_id 0
python test.py --config ./exps/wa_aid_resnet18.json --file_id Resnet18-wa-AID30-inc6 --gpu_id 1
python test.py --config ./exps/icarl_aid_resnet18.json --file_id Resnet18-icarl-AID30-inc6 --gpu_id 2
python test.py --config ./exps/foster_aid_resnet18.json --file_id Resnet18-foster-AID30-inc6 --gpu_id 3

# RepCLNet-18 (Ours)
python test.py --config ./exps/replay_aid_repclnet_18.json --file_id RepCLNet_18-replay-AID30-inc6 --gpu_id 0
python test.py --config ./exps/wa_aid_repclnet_18.json --file_id RepCLNet_18-wa-AID30-inc6 --gpu_id 1
python test.py --config ./exps/icarl_aid_repclnet_18.json --file_id RepCLNet_18-icarl-AID30-inc6 --gpu_id 2
python test.py --config ./exps/foster_aid_repclnet_18.json --file_id RepCLNet_18-foster-AID30-inc6 --gpu_id 3

SIRI-WHU (WHU12-inc2):

# ResNet-18 Baseline
python test.py --config ./exps/replay_siriwhu_resnet18.json --file_id Resnet18-replay-WHU12-inc2 --gpu_id 0
python test.py --config ./exps/wa_siriwhu_resnet18.json --file_id Resnet18-wa-WHU12-inc2 --gpu_id 1
python test.py --config ./exps/icarl_siriwhu_resnet18.json --file_id Resnet18-icarl-WHU12-inc2 --gpu_id 2
python test.py --config ./exps/foster_siriwhu_resnet18.json --file_id Resnet18-foster-WHU12-inc2 --gpu_id 3

# RepCLNet-18 (Ours)
python test.py --config ./exps/replay_siriwhu_repclnet_18.json --file_id RepCLNet_18-replay-WHU12-inc2 --gpu_id 0
python test.py --config ./exps/wa_siriwhu_repclnet_18.json --file_id RepCLNet_18-wa-WHU12-inc2 --gpu_id 1
python test.py --config ./exps/icarl_siriwhu_repclnet_18.json --file_id RepCLNet_18-icarl-WHU12-inc2 --gpu_id 2
python test.py --config ./exps/foster_siriwhu_repclnet_18.json --file_id RepCLNet_18-foster-WHU12-inc2 --gpu_id 3

CIFAR-100 (C100-inc10):

# ResNet-18 (Baseline)
python test.py --config ./exps/replay_cifar100_resnet18.json --file_id Resnet18-replay-CIFAR100-inc10 --gpu_id 0
python test.py --config ./exps/wa_cifar100_resnet18.json --file_id Resnet18-wa-CIFAR100-inc10 --gpu_id 1
python test.py --config ./exps/icarl_cifar100_resnet18.json --file_id Resnet18-icarl-CIFAR100-inc10 --gpu_id 2
python test.py --config ./exps/foster_cifar100_resnet18.json --file_id Resnet18-foster-CIFAR100-inc10 --gpu_id 3

# RepCLNet-18 (Ours)
python test.py --config ./exps/replay_cifar100_repclnet_18.json --file_id RepCLNet_18-replay-CIFAR100-inc10 --gpu_id 0
python test.py --config ./exps/wa_cifar100_repclnet_18.json --file_id RepCLNet_18-wa-CIFAR100-inc10 --gpu_id 1
python test.py --config ./exps/icarl_cifar100_repclnet_18.json --file_id RepCLNet_18-icarl-CIFAR100-inc10 --gpu_id 2
python test.py --config ./exps/foster_cifar100_repclnet_18.json --file_id RepCLNet_18-foster-CIFAR100-inc10 --gpu_id 3

ResAC-A vs RepCLNet-A

UCMerced-LandUse (UC21-inc7):

# ResAC-A (Baseline)
python test.py --config ./exps/replay_ucmerced_arch_craft.json --file_id ResAC_A-replay-UC21-inc7 --gpu_id 0
python test.py --config ./exps/wa_ucmerced_arch_craft.json --file_id ResAC_A-wa-UC21-inc7 --gpu_id 1
python test.py --config ./exps/icarl_ucmerced_arch_craft.json --file_id ResAC_A-icarl-UC21-inc7 --gpu_id 2
python test.py --config ./exps/foster_ucmerced_arch_craft.json --file_id ResAC_A-foster-UC21-inc7 --gpu_id 3

# RepCLNet-A (Ours)
python test.py --config ./exps/replay_ucmerced_repclnet_a.json --file_id RepCLNet_A-replay-UC21-inc7 --gpu_id 0
python test.py --config ./exps/wa_ucmerced_repclnet_a.json --file_id RepCLNet_A-wa-UC21-inc7 --gpu_id 1
python test.py --config ./exps/icarl_ucmerced_repclnet_a.json --file_id RepCLNet_A-icarl-UC21-inc7 --gpu_id 2
python test.py --config ./exps/foster_ucmerced_repclnet_a.json --file_id RepCLNet_A-foster-UC21-inc7 --gpu_id 3

AID (AID30-inc6):

# ResAC-A (Baseline)
python test.py --config ./exps/replay_aid_arch_craft.json --file_id ResAC_A-replay-AID30-inc6 --gpu_id 0
python test.py --config ./exps/wa_aid_arch_craft.json --file_id ResAC_A-wa-AID30-inc6 --gpu_id 1
python test.py --config ./exps/icarl_aid_arch_craft.json --file_id ResAC_A-icarl-AID30-inc6 --gpu_id 2
python test.py --config ./exps/foster_aid_arch_craft.json --file_id ResAC_A-foster-AID30-inc6 --gpu_id 3

# RepCLNet-A (Ours)
python test.py --config ./exps/replay_aid_repclnet_a.json --file_id RepCLNet_A-replay-AID-inc6 --gpu_id 0
python test.py --config ./exps/wa_aid_repclnet_a.json --file_id RepCLNet_A-wa-AID-inc6 --gpu_id 1
python test.py --config ./exps/icarl_aid_repclnet_a.json --file_id RepCLNet_A-icarl-AID-inc6 --gpu_id 2
python test.py --config ./exps/foster_aid_repclnet_a.json --file_id RepCLNet_A-foster-AID-inc6 --gpu_id 3

SIRI-WHU (WHU12-inc2):

# ResAC-A (Baseline)
python test.py --config ./exps/replay_siriwhu_arch_craft.json --file_id ResAC_A-replay-WHU12-inc2 --gpu_id 0
python test.py --config ./exps/wa_siriwhu_arch_craft.json --file_id ResAC_A-wa-WHU12-inc2 --gpu_id 1
python test.py --config ./exps/icarl_siriwhu_arch_craft.json --file_id ResAC_A-icarl-WHU12-inc2 --gpu_id 2
python test.py --config ./exps/foster_siriwhu_arch_craft.json --file_id ResAC_A-foster-WHU12-inc2 --gpu_id 3

# RepCLNet-A (Ours)
python test.py --config ./exps/replay_siriwhu_repclnet_a.json --file_id RepCLNet_A-replay-WHU12-inc2 --gpu_id 0
python test.py --config ./exps/wa_siriwhu_repclnet_a.json --file_id RepCLNet_A-wa-WHU12-inc2 --gpu_id 1
python test.py --config ./exps/icarl_siriwhu_repclnet_a.json --file_id RepCLNet_A-icarl-WHU12-inc2 --gpu_id 2
python test.py --config ./exps/foster_siriwhu_repclnet_a.json --file_id RepCLNet_A-foster-WHU12-inc2 --gpu_id 3

CIFAR-100 (C100-inc10):

# ResAC-A (Baseline)
python test.py --config ./exps/replay_cifar100_arch_craft.json --file_id ResAC_A-replay-CIFAR100-inc10 --gpu_id 0
python test.py --config ./exps/icarl_cifar100_arch_craft.json --file_id ResAC_A-icarl-CIFAR100-inc10 --gpu_id 1
python test.py --config ./exps/wa_cifar100_arch_craft.json --file_id ResAC_A-wa-CIFAR100-inc10 --gpu_id 2
python test.py --config ./exps/foster_cifar100_arch_craft.json --file_id ResAC_A-foster-CIFAR100-inc10 --gpu_id 3

# RepCLNet-A (Ours)
python test.py --config ./exps/replay_cifar100_repclnet_a.json --file_id RepCLNet_A-replay-CIFAR100-inc10 --gpu_id 0
python test.py --config ./exps/icarl_cifar100_repclnet_a.json --file_id RepCLNet_A-icarl-CIFAR100-inc10 --gpu_id 1
python test.py --config ./exps/wa_cifar100_repclnet_a.json --file_id RepCLNet_A-wa-CIFAR100-inc10 --gpu_id 2
python test.py --config ./exps/foster_cifar100_repclnet_a.json --file_id RepCLNet_A-foster-CIFAR100-inc10 --gpu_id 3

🔧 Model Deployment

Verify the re-parameterization of RepCLBlock :

python utils/repclblock_verify.py

Convert trained RepCLBlock models to efficient inference format:

python utils/convert.py \
    --config exps/replay_ucmerced_repclnet_a.json \
    --load_model runs/RepCLNet_A-replay-UC21-inc7/round0/task_2.pkl \
    --save_model runs/RepCLNet_A-replay-UC21-inc7/round0/task_2_deployed.pkl
    
# For example as follows:
python utils/convert.py 
	--config ./exps/replay_ucmerced_repclnet_a.json 
	--load_model runs/RepCLNet_A-replay-UC21-inc7/round0/task_2.pkl 
	--save_model runs/RepCLNet_A-replay-UC21-inc7/round0/task_2_deployed.pkl

🙏 Acknowledgements

This work builds upon several excellent prior works: PyCIL and ArchCraft.

📧 Contact

For questions and discussions, please open an issue in this repository.

---

⭐ If you find this project helpful, please consider giving it a star! ⭐