Sampling Strategies for Global Optimization

Most optimization methods are only guaranteed to converge to a local optimum. This is so for gradient descent, Newton’s method, and Quasi-Newton methods. These methods need a good starting value to converge to the global optimum and that is where sampling comes in. If not much is known about the objective function the default in this exploring phase is uniform sampling. We will consider Markov Chain Monte Carlo methods with adaptive variance and show desirable properties in finding good staring values to exploit for a rapidly converging algorithm.