Add example for hyperparameter tuning #89
Conversation
Codecov Report
@@ Coverage Diff @@
## main #89 +/- ##
===========================================
- Coverage 95.97% 73.97% -22.00%
===========================================
Files 13 16 +3
Lines 497 661 +164
===========================================
+ Hits 477 489 +12
- Misses 20 172 +152
Continue to review full report at Codecov.
|
|
Nice thanks! If that's okay with you, could we merge #70 first and adapt this PR accordingly? |
|
I will be making the necessary adaptations shortly. |
|
The figure is probably not needed, but will be auto-generated by Literate, right? |
There was a problem hiding this comment.
My main issue is with DSVGP, I don't understand why it needs to be introduced.
It would be more interesting to see what is the difference between the model with and without hyperparameters optimized.
Also I would rather have the NonCentered appproach, it looks like it is slowly getting solved upstream!
examples/hyperpars_nb/Project.toml
Outdated
| [compat] | ||
| GPLikelihoods = "0.4.2" |
There was a problem hiding this comment.
CompatHelper should do it, but it would be helpful if you already add compat for all the packages
| end | ||
|
|
||
| # ## Diagonal SVGP model | ||
| # We define a standard diagonal sparse variational GP for comparison |
There was a problem hiding this comment.
I don't understand this. What is the point of having a diagonal svgp?
There was a problem hiding this comment.
Are you asking why a diagonal SVGP as opposed to a full SVGP, or is your question why have a SVGP in this example at all?
I simply added the SVGP example to compare to the AVGP, to see whether they would get roughly comparable results.
| M = min(30, length(l.y)) | ||
| z0 = rand(M) | ||
| z0 .*= xmax - xmin | ||
| z0 .+= xmin |
There was a problem hiding this comment.
It's not particularly fair to compare a sparse GP with a full GP
There was a problem hiding this comment.
The reason why I'm interested in the augmented GP framework is to make training faster. The sparse variational GP is already much slower for moderate data sizes. If we make the variational GP use all training points, it would be hopelessly slow, wouldn't it?
| end | ||
|
|
||
| # In order to make this work with Zygote, we had to change from | ||
| # `Centered` to `NonCentered` here. This deviates from the usual practice. |
There was a problem hiding this comment.
I would really rather solve the Centered approach instead of using a workaround here.
There was a problem hiding this comment.
Sure, but if we merge this before those issues get resolved, we have to have a workaround.
Are there any concerns that the NonCentered representation is incorrect or less efficient?
Co-authored-by: Théo Galy-Fajou <[email protected]>
Co-authored-by: Théo Galy-Fajou <[email protected]>
…ntedGPLikelihoods.jl into hyperpar_example
|
Unfortunately after more testing it is still not working reliably. The convergence is sometimes very fast but the solution not very good. |
This is supposed to close #76