Historical research code for optimizing simulated base-station antenna azimuth and downtilt with multi-agent reinforcement learning (MARL). The repository combines a grid-based stochastic RSRP coverage simulator, local observation preprocessing, two MADDPG variants, and a Tkinter visualization.
This is a research prototype and historical software snapshot. It is not a production radio-access-network controller, it has no interface to physical antennas or live operator systems, and the available repository is not a complete reproduction package for the thesis experiments.
- Original title: 基于强化学习的 5G 基站自适应天馈系统
- English title: A Reinforcement-Learning-Based Adaptive Antenna-Feed System for 5G Base Stations
- Author: Yuhang Shen
- Degree: Bachelor of Engineering in Internet of Things Engineering
- Institution: University of Electronic Science and Technology of China (UESTC), School of Information and Communication Engineering
- Year: 2021
- Supervisor: Professor Sheng Wang
The thesis describes a broader cloud-edge system, including virtual-environment pretraining, constrained exploration, parameter transfer, and inter-station coordination. This repository contains only the simulation and learning subset documented below. See CITATION.cff for citation metadata.
| Capability | Status in this repository |
|---|---|
| Configurable grid and saved base-station layouts | Implemented |
| Three simulated antenna sectors per base station | Implemented |
| Azimuth and downtilt adjustment in a stochastic RSRP simulator | Implemented |
| Per-sector observation sampling and linear-regression gap filling | Implemented |
| Baseline MADDPG with a centralized critic | Implemented |
| MADDPG variant whose critic receives the global RSRP grid and antenna angles | Implemented as a fully connected network |
| Coverage-constrained candidate-action screening and angle rollback | Experimental/partial |
| Tkinter coverage-map visualization | Implemented |
| Rayleigh fading model described in the thesis | Not implemented; the code uses a simplified stochastic signal model |
| Convolutional or residual global-state critic described in the thesis | Not implemented |
| AE-based inter-station scheduling protocol | Not implemented |
| Cloud-edge deployment and model-transfer workflow | Not implemented |
| Physical base-station or antenna-control interface | Not implemented |
| Real operator dataset, final checkpoints, logs, and plotting artifacts | Not included |
flowchart LR
Layout["Saved base-station layout"] --> Simulator["Stochastic grid-based RSRP simulator"]
Simulator --> Sampling["Per-sector spatial sampling"]
Sampling --> Actors["One actor per antenna sector"]
Actors --> Actions["Azimuth and downtilt candidates"]
Actions --> Gate["Coverage-based candidate screening"]
Gate --> Simulator
Simulator --> Replay["Replay memory"]
Replay --> Critics["Centralized critic training"]
The default source configuration uses five base stations over a 30 x 30 grid with a 10 m cell interval, representing a 300 m x 300 m simulated area. Each station has three sectors, giving 15 agents. For each sector, the current code samples 12 azimuth directions and 12 radial positions, then appends normalized azimuth and downtilt values for a 146-dimensional observation.
Each actor produces 42 scores. The first 21 select an azimuth change from -10 to +10 degrees, and the second 21 select a downtilt change over the same range. The historical scheduler converts each block to a discrete action with argmax.
.
|-- Scheduler.py # Historical training entry point
|-- Parameter.py # Layout, antenna, and coverage state
|-- Environment.py # Stochastic RSRP simulator and Tkinter view
|-- DataProcessing.py # Observation construction and reward helpers
|-- Algorithm.py # Baseline/improved MADDPG selection
|-- MADDPG/ # Baseline centralized-critic implementation
|-- Improved_MADDPG/ # Global-state fully connected critic variant
|-- data/ # Historical base-station layout files
|-- test.py # Historical evaluation/timing script
|-- test_critic.py # Historical model-size experiment script
|-- To Learn.md # Archived development notes (Chinese)
`-- Todo.md # Archived development notes (Chinese)
Despite their names, test.py and test_critic.py are exploratory experiment scripts, not automated tests. test_critic.py also writes checkpoint files into data/.
The thesis records Python 3.9 and PyTorch 1.8. The supplied environment.yml reconstructs that historical stack and adds the dependencies imported by this repository.
conda env create -f environment.yml
conda activate rl-antenna-feed
python -m compileall -q .
python -c "import numpy, pandas, sklearn, torch; import Algorithm, DataProcessing, Environment, Parameter"The environment file is a best-effort historical reconstruction, not a lockfile. It has not been validated across current Conda platforms, and old PyTorch builds may require adjustments to the channel or CUDA selection. The code explicitly uses CPU tensors by default.
The main historical entry point is:
python Scheduler.pyThe checkout does not run end to end with the default constants. Scheduler.py requests data/map_5_30_30_10.npz, while the repository contains maps for 4, 10, 15, and 20 base stations. It also does not include the final checkpoints or a frozen experiment configuration. Treat the entry point as implementation evidence, not as a ready-to-run reproduction command.
Configuration is currently expressed as module-level constants near the top of Scheduler.py, test.py, and test_critic.py. These scripts open a Tkinter window and assume a graphical session. No headless command-line interface is provided.
- The simulator estimates sector coverage geometrically and assigns stochastic RSRP values on the grid.
- Local actor observations are spatial samples from each sector's potential coverage region, with missing samples filled by linear regression.
- Rewards map RSRP quality bands to signed values and penalize uncovered cells in the sector's potential coverage region.
MADDPG/uses all agents' observations and actions in each centralized critic.Improved_MADDPG/replaces the concatenated local observations at the critic input with the flattened global RSRP grid and normalized antenna angles. It does not contain the convolutional/residual critic discussed in the thesis.- The scheduler repeatedly proposes actions and rejects candidates that violate a heuristic uncovered-cell threshold.
The repository preserves useful historical implementation detail, but it cannot substantiate an exact reproduction of the thesis figures or reported performance. In particular:
- no random seeds, dependency lock, automated regression tests, training logs, final checkpoints, or figure-generation pipeline are provided;
- the observation sampler calculates radii in metres but uses them as grid coordinates without dividing by the configured cell interval;
- the environment regenerates stochastic RSRP values after each individual sector action rather than once after a joint action;
- rollback restores antenna angles but not the corresponding RSRP and coverage maps after a rejected candidate;
- the coverage structure can record the same base station more than once when multiple sectors overlap a grid cell;
- the actor is trained on continuous score vectors although the scheduler executes two discrete
argmaxactions; and env.after(1, run_simulation(...))evaluates the simulation synchronously before registering the Tkinter callback.
The current repository should therefore be treated as an archival reference. A maintained release should separate the simulator from the UI, define one action contract, add deterministic tests, and regenerate a traceable result set before making performance claims.
Provenance review found that the historical MADDPG/ and Improved_MADDPG/ directories contain code matching or adapted from xuehy/pytorch-maddpg. That upstream repository does not declare a software license. Because redistribution and relicensing rights have not been established, this repository intentionally does not add an open-source license.
The absence of a license does not grant reuse rights. A future Apache-2.0 release should use an independently written implementation or a clearly licensed upstream implementation, retain all required notices, and publish with a clean, documented provenance trail.
If this research snapshot informs academic work, please cite the related thesis:
Yuhang Shen, "基于强化学习的 5G 基站自适应天馈系统"
(A Reinforcement-Learning-Based Adaptive Antenna-Feed System for 5G Base Stations),
Bachelor's thesis, University of Electronic Science and Technology of China, 2021.