Change rrule verbosity behaviour

[Jutho]

This drastically changes the behaviour of verbosity in the rrules:

• The warnings printed by the rrules are now controlled by alg_primal.verbosity rather than alg_rrule.verbosity. This way, alg_rrule is fully restricted to specifying the linear/eigen subproblem solved in the rrule. I think it is more naturally that warnings or info printed in the rrule of a certain algorithm are controlled by the verbosity setting of that algorithm
• The required verbosity level for all warnings in the rrules is increased by one, i.e. with verbosity=0 none of the warnings will be printed. I am still not sure if that is the right decision. Some of these warnings could point to errors. Whereas for the primal algorithm, it is the user's responsibility to check for convergence in the info struct, so that it makes sense that warnings about lack of convergence are only printed for verbosity >= 1, it is impossible to check this for the pullback
• Some tolerances are also increased a bit to have warnings triggered less quickly overall

Aside from this, this PR also removes MinimalVec and takes the one from VectorInterface.jl 0.5. Maybe that required a separate PR, but here we are 😸 .

[codecov]

Codecov Report

Attention: Patch coverage is 74.80620% with 65 lines in your changes missing coverage. Please review.

Project coverage is 87.19%. Comparing base (3304d77) to head (0cecee8).

Files with missing lines	Patch %	Lines
src/matrixfun/expintegrator.jl	37.73%	33 Missing ⚠️
src/eigsolve/golubye.jl	30.76%	9 Missing ⚠️
src/eigsolve/svdsolve.jl	66.66%	5 Missing ⚠️
src/eigsolve/arnoldi.jl	90.90%	4 Missing ⚠️
src/linsolve/cg.jl	71.42%	4 Missing ⚠️
src/linsolve/gmres.jl	75.00%	4 Missing ⚠️
src/linsolve/bicgstab.jl	90.00%	2 Missing ⚠️
src/lssolve/lsmr.jl	80.00%	2 Missing ⚠️
src/KrylovKit.jl	90.00%	1 Missing ⚠️
src/eigsolve/lanczos.jl	95.65%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #106      +/-   ##
==========================================
+ Coverage   86.08%   87.19%   +1.10%     
==========================================
  Files          32       32              
  Lines        3385     3381       -4     
==========================================
+ Hits         2914     2948      +34     
+ Misses        471      433      -38     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lkdvos]

What happens if you just add a -1 setting to disable all warnings, and still have it default to printing the warnings as well?

[Jutho]

Then you get the old behaviour if I understand your suggestion correctly (except then for the alg_reverse.verbosity -> alg_primal.verbosity change).

[lkdvos]

I am okay with not printing the warnings and having them on an opt-in level, it's just that I don't think I realized I could turn them off by setting the verbosity to -1, and maybe other people also didn't. What I mean then is, that it might not be necessary to change this behavior, if you could just get it to behave like that anyways with a setting.

Ultimately though, I have no strong feelings about this, I guess that if things are going wrong I would typically start debugging by setting the verbosity levels higher anyways.

[Jutho]

The verbosity = -1 level was used in the tests, since, if you compute full Jacobians, you are bound to probe gauge-dependent directions. But indeed, nobody was probably aware of this and it was a hacky solution, so I guess the current one is better. Alternatively, I could also keep at least the actual convergence warnings at level zero, but mention explicitly that level -1 could be used to disable them.

[Jutho]

Thinking some more about it, current behavior for the forward algorithms is that

• at verbosity=0 they do not print anything
• at verbosity=1 they print a warning if the number of requested solutions is obtained (with convergence up to specified tolerance), but also an info message if the number of solutions is found, with details of number of iterations / ...
• at verbosity=2 you get per iteration info

I could remove the info message at verbosity = 1. Then both the forward algorithm and the reverse rule have "verbosity = 1" being something like "warning mode". No info, only warnings. The question is then if having only the final info requires a separate level (level 2), and so the per iteration info shifts to level 3.

[XingyuZhang2018]

I think an additional level should be included:

• At verbosity=0, no output is printed.
• At verbosity=1, a warning is printed.
• At verbosity=2, you receive start and end information.
• At verbosity=3, you receive information for each iteration.

This addition is necessary because there could be numerous iterations that might overwhelm the screen.

[Jutho]

Ok, a long overdue update here:
I've completely overhauled the verbosity/printing system in both the original methods as well as the corresponding rrules. In particular, I now use the levels as proposed in the previous post by @XingyuZhang2018, with there being a level 1 for only warnings and a level 2 for an info message at the end (as well as at the beginning for linsolve and lssolve, where this makes sense, but not for eigsolve etc where it makes less sense). Level 3 is "per iterations", eigsolve and friends supports level 4 for even more information on the underlying Krylov subspace generation, but that will probably never be necessary.

I've also made level 1 the default, so that warnings will be printed if a method does not converge (this is new!). However, also the default (KrylovDefault.verbosity) is configurable, as this variable (as well as KrylovDefault.tol and KrylovDefault.maxiter) are now Refs.

Finally, for the AD rules, I've taken the following approach: the alg_rrule.verbosity is used within the Krylov method that is used to solve the subproblem as part of the rrule. However, potential problems that occur with the result of that calculation within the surrounding rrule code are still based on alg_primal.verbosity (there are only warnings, no other info statements). One exception is the gauge dependency warning of the input to the eigsolve and svdsolve rrule, which is triggered by alg_rrule.verbosity >= 1. This way, you can disable the gauge dependency warning if you are sure that this is what you want (alg_rrule.verbosity = 0), while still receiving other warnings about unexpected behaviour in the rrule computation (alg_primal.verbosity >= 1).

I realise this will not be the final solution here, and some features might be made nicer (e.g. using Preferences.jl for the defaults). But for know, I think I am happy with this (it also took much more time than anticipated to redesign and in particularly test all the logs). So if the tests turn green, this is good to go for me, unless there are pertinent comments by @lkdvos or @XingyuZhang2018 .