Gaussian Kernel Bandwidth Optimization with Matlab Code
In this article, I write on “Optimization of Gaussian Kernel Bandwidth” with Matlab Code.
First, I will briefly explain a methodology to optimize bandwidth values of Gaussian Kernel for regression problems. In other words, I will explain about “Cross validation Method.”
Then, I will share my Matlab code which optimizes the bandwidths of Gaussian Kernel for Gaussian Kernel Regression. For the theory and source code of the regression, read my previous posts <link for 1D input>, <link for multidimensional input>. This Matlab code can optimize bandwidths for multidimensional inputs. If you know the theory of cross validation, or if you don’t need to know the algorithm of my program, just download the zip file from the below link, then execute demo programs. Probably, you can use the program without big difficulties.
1. Bandwidth optimization by a cross validation method
The most common way to optimize a regression parameter is to use a cross validation method. If you want to know about the cross validation deeply, I want to recommend to read this article. Here I will shortly explain about the cross validation method that I am using. This is just a way of cross validation.
1. Randomly sample 75% of the data set, and put into the training data set, and put the remaining part into the test set.
2. Using the training data set, build a regression model. Based on the model, predict the outputs of the test set.
3. Compare between the predicted output, and the actual output. Then, find the best model (best bandwidth) to minimize the gap (e.g, RMSE) between the predicted and actual outputs.
2. Matlab code for the algorithm
You can download all functions and demo programs from the below link.
This program is for multidimensional inputs (of course, 1D is also OK). The most important function is Opt_Hyp_Gauss_Ker_Reg( h0,x,y ) and it requires Matlab optimization toolbox. I am attaching two demo programs and their results. I made these demo programs as much as I can. So, I believe that everybody can understand.
<Demo 1D>
<Demo 2D>
I wish this post can save your time and efforts in your work. If you have any question, please leave a reply.
-Mok-
—————————————————————————————————
I am Youngmok Yun, and writing about robotics theories and my research.
My main site is http://youngmok.com, and Korean ver. is http://yunyoungmok.tistory.com.
—————————————————————————————————