[This article was first published on Pachá, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.R/Shiny Training: Should you find this blog to be of interest, kindly note that I offer personalized and group-based trainingsessions that may be reserved through Buy me a Coffee. Additionally, I provide training services in the Spanish language and am available to discuss means by which I may contribute to your Shiny project.
MotivationA large part of my research interest requires to estimate computationally intensive models, such asthe General Equilibrium Poisson Pseudo Maximum Likelihood (GEPPML) estimator derived from the equilibrium conditionsintroduced by Anderson and Van Wincoop (2004) for estimation and inference.
The GEPPML estimator is a computationally intensive estimator that requires to solve a system of non-linear equations,and for this task we might be better-off by using a compiled language such as C++. The good news is that we can use C++ codewithin R and Python, and this blog post is about using C++ functions from R.
Also, I do not pretend to be an expert on C++ or debate if R is better than Python. I use both from Visual Studio Code.I do want to share my experience on how to use C++ code within R.
Honest disclaimerThis blog post is a summary of what worked after hours of fails for my future self. I hope it helps youtoo.
I am a Statistician and Political Scientist, not a Computer Scientist!
SetupBecause I have been already learning C++ version 11, I decided to install llvm-11 on my laptopthat has Linux Mint installed, and which is based on Ubuntu 22.04.
Ubuntu and its derived distributions use gcc as the default C++ compiler, and clang is not installed by default. Differentresources mention that clang provides more informative error messages when the compilation fails and when we debug code.
Counting on informative error messages is highly useful resource when we are learning C++ or when our code is failing in twodifferent ways, being one that it does not compile, and the other that it compiles but then when we call a function fromRStudio (or VSCode) it crashes the R session.
I installed the R packages cpp11 and usethis.
install.packages(c("cpp11", "usethis"))
I created a file ~/.Rprofile containing the following lines.
library(devtools)library(usethis)library(cpp11)
I run nano ~/.R/Makevars from bash and then saved with CTRL+O+ENTER and closed it with CTRL+X. It is the same as creating it with the text editor from Gnome or any other desktop environment.
Now forget about devtools::install(). After reopening your editor, every time you use RStudio (or VSCode) you just call install(), and the same applies to usethis::use_*() and cpp11::cpp_*() functions.
To install llvm-11 I downloaded the installation script from the official LLVM repository, and it also installed clang-11.
cd Downloadswget https://apt.llvm.org/llvm.shchmod +x llvm.shsudo ./llvm.sh 11
Up to this point I still had the following error messages when compiling C++ code.
fatal error: 'cstdio' file not foundfatal error: 'vector' file not foundcannot find -lc++abi: No such file or directory
I had to install additional packages. This took me a few hours searching on the Internet until I figured it out.
sudo apt install g++-11 libc++-11-dev libc++abi-11-dev
To be sure that the install() function in R uses the correct version of clang++ I created the ~/.R/Makevars file. The contents of the file are the following.
CLANGVER=-11CLANGLIB=-stdlib=libc++CXX=$(CCACHE) clang++$(CLANGVER) $(CLANGLIB)CXX11=$(CCACHE) clang++$(CLANGVER) $(CLANGLIB)CC=$(CCACHE) clang$(CLANGVER)SHLIB\_CXXLD=clang++$(CLANGVER) $(CLANGLIB)CXXFLAGS=-Wall -O0 -pedanticCXX11FLAGS=-Wall -O0 -pedantic
For both CXXFLAGS and CXX11FLAGS I am using -O0 to avoid optimization, which is useful for debugging. After the code is working, I can change it to -O3 to optimize the compiled code.
If later on I need to compile with gcc, I can open ~/.R/Makevars, comment all the lines, restart RStudio or VSCode, and run install() again.
If you close RStudio (or VSCode) and open it again, you can check that the changes were implemented by running pkgbuild::check_build_tools(debug = TRUE), which should return the following output.
Trying to compile a simple C fileRunning /usr/lib/R/bin/R CMD SHLIB foo.cusing C compiler: ‘Ubuntu clang version 11.1.0-6’clang-11 -I"/usr/share/R/include" -DNDEBUG -fpic -g -O2 -ffile-prefix-map=/build/r-base-JhpCKt/r-base-4.3.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D\_FORTIFY\_SOURCE=2 -c foo.c -o foo.oclang-11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -o foo.so foo.o -L/usr/lib/R/lib -lR
If I were using gcc, the output would have been as in the following lines.
Trying to compile a simple C fileRunning /usr/lib/R/bin/R CMD SHLIB foo.cusing C compiler: ‘gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0’gcc -I"/usr/share/R/include" -DNDEBUG -fpic -g -O2 -ffile-prefix-map=/build/r-base-JhpCKt/r-base-4.3.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D\_FORTIFY\_SOURCE=2 -c foo.c -o foo.ogcc -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -o foo.so foo.o -L/usr/lib/R/lib -lR
The key here is that when I use clang the lines start with clang, not with gcc.
Creating a dummy packageFrom RStudio (or VSCode) we can create a new package by running create_package("~/cpp11dummypackage"). This will create a new folder with the name cpp11dummypackage. Then I run use_cpp11() to add the required files to use C++ code within R.
Then I run use_r("cpp11dummypackage-package") to create a new R script file with the name cpp11dummypackage-package.R within the R folder, and added the following code to it.
```
``
Theusethisskeleton also created the filesrc/code.cpp` for us. I added a simple function to transpose a matrix to it, by replacing the file contents by the following lines.
```
``
In order to export the function, I added the following lines tocpp11dummypackage-package.R`.
```
``
I tested the functions after runningcpp11_register()andload_all()`.
```
set.seed(1234)> X <- matrix(rnorm(4), nrow = 2, ncol = 2)> X [,1] [,2][1,] -1.2070657 1.084441[2,] 0.2774292 -2.345698> cpp11_Xt(X) [,1] [,2][1,] -1.207066 0.2774292[2,] 1.084441 -2.3456977
`` If I would have passed1:4instead ofrnorm(4)tomatrix()`, I would have obtained the following error message.
```
cpp11_Xt(X)Error: Invalid input type, expected 'double' actual 'integer'
`` This is because I declared the function to accept adoubles_matrix<>as input, and not anintegers_matrix<>`.
To install the recently created package, I run the following lines in the R console.
clean\_dll()cpp\_register()document()install()
Debugging the packageIn order to access debugging symbols, I created a new Makevars file within the src folder, and added the following lines.
CXX\_STD = CXX11PKG\_CPPFLAGS = -UDEBUG -g
Then I reinstalled the package compiled with debugging symbols, and in bash I run R -d lldb-11. From there I could follow this excellent guide to debug R and C++ code.
A more complex exampleI created a package containing a set of simple functions to obtain the Ordinary Least Squares (OLS) estimator by calling a C++ function that calls other C++ functions. My approach was to create one function per step, which meant to create one function to obtain (X^tX), another for ((X^tX)^{-1}) which consisted in implementing the Gauss-Jordan method to invert a matrix, another for (X^tY) and then call each of those functions to obtain (\hat{\beta} = (X^tX)^{-1}(X^tY)).
This implementation is extremely naive, but it is enough to show how to use C++ code within R. Please see it from my GitHub profile.
A good challenge would be to implement the QR decomposition used by the lm() function in R and use it to obtain the OLS estimator in C++. This would require some effort, but here you can find a good starting point.
In any case, it would be extremely hard to beat the performance of the lm() function in R, which has some internals written in C, and how computationally robust lm() is means another feature that is hard to beat.
References Debugging in R with a single command * Debugging an R package with C++ * Clang++ missing C++ header? * How to I tell RStudio not to ignore the indication to use clang in Makevars? * R’s Makevars: PKG_CXXFLAGS vs. PKG_CXX11FLAGS * Debugging memory errors with valgrind and gdb * A Deep Dive Into How R Fits a Linear Model To leave a comment for the author, please follow the link and comment on their blog: Pachá*.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.Continue reading: A step by step guide to write an R package that uses C++ code (Ubuntu)