I think one of the biggest shortcomings of our current optimization algorithms is that they are entirely sequential (i.e. 1 simulation per epoch). Alternative algorithms draw several samples tin an epoch to build a more accurate update of where the algorithm "believes"
Personally I am quite fond of CMA-ES (covariance matrix adaption evolution strategy). This approach shares mathematical similarities with Bayesian optimization, and in my own testing on toy optimization problems seems to be very simulation efficient.
https://en.wikipedia.org/wiki/CMA-ES
The math is not too difficult to translate into Python, but it will definitely be a project to get a proof of concept working. There are certainly existing packages that implement this, but the challenge will be translating this to work with our current Optimization backend code.
I think one of the biggest shortcomings of our current optimization algorithms is that they are entirely sequential (i.e. 1 simulation per epoch). Alternative algorithms draw several samples tin an epoch to build a more accurate update of where the algorithm "believes"
Personally I am quite fond of CMA-ES (covariance matrix adaption evolution strategy). This approach shares mathematical similarities with Bayesian optimization, and in my own testing on toy optimization problems seems to be very simulation efficient.
https://en.wikipedia.org/wiki/CMA-ES
The math is not too difficult to translate into Python, but it will definitely be a project to get a proof of concept working. There are certainly existing packages that implement this, but the challenge will be translating this to work with our current Optimization backend code.