Generalized Cauchy direction and box constraints

[mateuszbaran]

This is a WIP PR that implements the GCP idea to L-BFGS and maybe some other solvers, so that they can be used in the presence of box constraints on the Euclidean part of the manifold. The constraints are handled similarly to L-BFGS-B (although not all Euclidean tricks are applied).

TODO:

• a full example of working with a box domain
• code coverage

The implementation was prepared in collaboration with @paprzybysz .

[codecov]

Codecov Report

❌ Patch coverage is 99.46381% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (dev-0.6@00c160b). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
src/plans/stopping_criterion.jl	94.20%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             dev-0.6     #554   +/-   ##
==========================================
  Coverage           ?   99.96%           
==========================================
  Files              ?       91           
  Lines              ?    11605           
  Branches           ?        0           
==========================================
  Hits               ?    11601           
  Misses             ?        4           
  Partials           ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mateuszbaran]

I have also added a simple direction-based stepsize limiting that can be used in solvers that don't approximate the Hessian to limit the stepsize.

[kellertuer]

Here is a set of comments.

I think most of them are merely something to work though. The major points to check is

• What is a Finder? Is that the best name we can come up with?
• Why does a Finder store a manifold – the manifold should only be stored in the problem and otherwise passed down.

[mateuszbaran]

The major points to check is

• What is a Finder? Is that the best name we can come up with?
• Why does a Finder store a manifold – the manifold should only be stored in the problem and otherwise passed down.

Probably a "subsolver" would be a better alternative to "finder", what do you think?

I can easily remove manifold from GeneralizedCauchyDirectionFinder if that's a problem, sure.

[kellertuer]

Ah it is a subsolver ;) I did not see that.
Yes, I would prefer for consistency to not “copy” (I know it is a reference) the manifold, the only thing it can lead to is inconsistencies, when someone (myself not excluded even) manages to create such a thingy /finder/subsolver) with a manifold different from the one in the problem.

To avoid that a user makes this mistake, the easiest is to not double information :)

[kellertuer]

I think all my points are clarified by now. Thanks for the productive discussions.

[kellertuer]

Since this will also be part of the next major version 0.6, I diverted the base branch (this merges into) already to dev-0.6; I can also resolve the conflicts somewhen the next few days.