OPT estimated vs simulated

[IsaDio]

Hi,

I started running OPT for a problem where we want to optimise managed aquifer recharge rates (minimise MAR rates) to achieve a given flow in a creek at the end of MAR (greater than constraints).

It looks like it is working (yay!) but OPT seems to have difficulties finding a solution. The estimated constraints/objective usually look great but the actuals (simulated) are often quite different and lower (constraints not satisfied). Is this due to the non linearity of the solution, and OPT is struggling to find the answer? Or could it be something else? Is there something that can be done to help with this (initial conditions of parameters, OPT settings..)?

Thank you!

[jtwhite79]

Sure sounds like nonlinearity - thats why opt writes the estimated and simulated results. opt can only tolerate the mildest nonlinearity...are you iterating with opt at all? Also what size perturbations on those MAR rates are you using and are your model solver tolerances pretty tight? Typically the decision variable perturbations would be larger than the parameter perturbations that are used to fill a jco...

[IsaDio]

Thanks! Yes that is what I understand.
Yes I am iterating with opt. It does help but it doesn't guarantee it will find an optimal solution for each scenarios I am running.
The water balance looks good with little mass balance error, but good thinking. I might try with a smaller tolerance.
I have gathered that the decision variable perturbations are quite important. I have tested different values and also tried to constrain the bounds of the decision variables. It looks like it is helping.
I am setting up a run with MOU also, as it should solve that issue hopefully?

[jtwhite79]

Seems like you are across the issues! MOU is a different beast - its a global solver so in theory it cope with wicked nonlinearity, but that comes with a much larger computational cost. Keen to hear how this goes for you...

[IsaDio]

I have improved things for sure but it still can be a bit unpredictable. Do you have a rule of thumb or some default values you like to use for the decision variable perturbations to calculate the derivatives based on the range? Absolute or relative does it make a difference? I thought a small increment and relative would help with the non-linearity but the opposite happened. Thanks!

[jtwhite79]

I usually use a 10-15% perturbation when filling the response matrix for opt - this is because you want to make sure you get a good strong representative response at the constraints (that is in excess of any solver tolerance noise). We took a concept from GWM where you can have the perturbation decrease across iterations (there is a ++ arg for this), with idea being that as you near the optimal solution, you might want to use progressively smaller (ie more local) perturbations...I also usually start with strongly feasible solution (so in your case with high MAR rates so that the stream constraints are satisfied)

[IsaDio]

Awesome. This is around the range I have been using, maybe a bit smaller. I do start with a feasible solution, just not optimised.
An decrease in perturbation is exactly what I was thinking could help.
Thank you again for your help, greatly appreciated :)

[gmerritt123]

Hey @jtwhite79 just came across this discussion and seems to be what I'm running into. My first question is how to adjust decision variable perturbation amounts, both "initially" and between iterations. For my setup, the result seems to be bouncing around too much in between iterations. You mention here that there's a ++ arg for this (what is it? rlambda1?).

I am, similar to @IsaDio , starting with a feasible solution, but definitely not optimized. It's very slightly non-linear but pretty dang close.

I guess my follow up general question also is if I should expect to see pestpp-opt "march" slowly towards a (maybe local based on initial conditions) optimal solution, or should I see it try a bunch of crazy things (e.g. violate constraints big time etc.) and eventually bounce to a good solution? W a gradient based optimizer I expected the former but my intuition has been very wrong before...

Thanks as always!

[jtwhite79]

Applogies - the perturbation decreasing factor is not documented (IDK how that happened but it will be fixed). Its called opt_iter_derinc_fac and ranges from 1.0 (no change between iterations) to >0.0.

Beware: the outer iterative scheme in SLP (and therefore pestpp-opt) is pretty weak and cannot deal very well with nonlinearity. And the simplex solver is not so much marching toward a solution as much as it crawls along the (implied) linearized decision-variable-constraint relations to find a maximum of the objective function (assuming a linear gradient between the objective function and the decision variables). Therefore, the optimal solution should by design be on one or more constraints (including decision variable bounds), so its not like parameter estimation - here we want to find the extremes (so a bunch of crazy things). And if the response matrix/jacobian changes (alot) between iterations, then I would expect the optimal solution to change (alot). This is why its not a strong nonlinear iterative scheme. We (well, @rqmacasieb ) are working on a stronger iterative nonlinear optimizer...hopefully ready in a few months.