How about different training method

[andrewpolar]

There were research before MIT paper:
https://www.sciencedirect.com/science/article/abs/pii/S0952197620303742
quick introduction here:
http://openkan.org/KANscore.html

There are few counterintuitive things:

1. Piecewise linear functions provide same accuracy as splines but training is 10 time faster.
2. Kaczmarz method needs significantly less computer operations.
3. Selection of basis methos doesn't matter, they can be replaced while training without loss of acquired accuracy, example http://openkan.org/KANPRO.html

How about to use Kaczmarz for training as an optional method in addition to what you already have.
I don't like to write code in Python. Python makes me vomit. I do Java, C#, C++. But draw your attention to different concept training.

By the way I achieved Ph.D. where you teach and my coauthor graduated it from.
Regards.

[IvanDrokin]

@andrewpolar Hi there! thanks for sharing, looks interesting, I'll add this to my repo