[This article was first published on T. Moudiki's Webpage - R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.This post describes the R version of Python’s GPopt (https://docs.techtonique.net/GPopt/GPopt.html), a package for‘Bayesian’ optimization of black-box functions and machine learning hyperparameter tuning, using GaussianProcess Regression and other conformalized surrogates. The package is available on GitHub, and through the R universe.
Keep in mind that this package is for Machine Learning hyperparameter tuning: the global minimum won’t always be found, but this isn’t an issue, since it means you aren’t overfitting the training set.
It’s ported the same way as nnetsauce for R was: with uv tocreate an isolated Python virtual environment containing the Python GPopt package, and reticulate to callinto it from R. Every function in this R package is a thin wrapper that returns the underlying Python object;the general rule is: object accesses with .’s in Python are replaced by $’s in R.
See this post for the technique: Finally figured out a way to port python packages to R using uv and reticulate.
Install1. Create a Python virtual environment with uv
```
``
Keep track of wherevenv/lives – you’ll pass its path asvenv_path` to every function in this package.
install.packages("remotes")remotes::install\_github("Techtonique/GPopt\_r")
reticulate will be installed automatically as a dependency.ExamplesMinimizing the Branin functionThis is a standard test function for optimization algorithms. GPOpt is more suitable for expensive black-box functions, but this is a good example to illustrate the usage of the package.
library(GPopt)branin <- function(x) { x1 <- x[1]; x2 <- x[2] term1 <- (x2 - (5.1 * x1^2) / (4 * pi^2) + (5 * x1) / pi - 6)^2 term2 <- 10 * (1 - 1 / (8 * pi)) * cos(x1) term1 + term2 + 10}opt <- GPOpt( lower\_bound = c(-5, 0), upper\_bound = c(10, 15), objective\_func = branin, n\_init = 10, n\_iter = 40, venv\_path = "./venv")opt$optimize(verbose = 1L)print(opt$x\_min) # best parametersprint(opt$y\_min) # best objective value
Tuning a scikit-learn model’s hyperparameters
library(GPopt)sklearn <- get\_sklearn(venv\_path = "./venv")RandomForestClassifier <- sklearn$ensemble$RandomForestClassifierX <- as.matrix(iris[, 1:4])y <- as.integer(iris$Species) - 1Lmlopt <- MLOptimizer(scoring = "accuracy", cv = 5, venv\_path = "./venv")param\_config <- list( n\_estimators = list(bounds = c(10, 300), dtype = "int"), max\_depth = list(bounds = c(1, 20), dtype = "int"))mlopt$optimize( X\_train = X, y\_train = y, estimator\_class = RandomForestClassifier(), param\_config = param\_config, verbose = 1L)print(mlopt$get\_best\_parameters())print(mlopt$get\_best\_score())
Bayesian optimization with early stopping
library(GPopt)opt <- BOstopping( f = branin, bounds = rbind(c(-5, 10), c(0, 15)), venv\_path = "./venv")result <- opt$optimize(n\_iter = 100L)
Using a custom (conformalized) surrogate model
library(GPopt)sklearn <- get\_sklearn(venv\_path = "./venv")ns <- get\_nnetsauce(venv\_path = "./venv")opt <- GPOpt( lower\_bound = c(-5, 0), upper\_bound = c(10, 15), objective\_func = branin, acquisition="ucb", method="splitconformal", surrogate\_obj = ns$PredictionInterval(sklearn$ensemble$RandomForestRegressor()), venv\_path = "./venv")opt$optimize(verbose = 1L)
To leave a comment for the author, please follow the link and comment on their blog: T. Moudiki's Webpage - R.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.Continue reading: GPopt for R: Bayesian and conformal optimization of black-box functions and hyperparameter tuning