Kavli Affiliate: Feng Yuan
| First 5 Authors: Xingdong Feng, Yuan Gao, Jian Huang, Yuling Jiao, Xu Liu
| Summary:
We propose a relative entropy gradient sampler (REGS) for sampling from
unnormalized distributions. REGS is a particle method that seeks a sequence of
simple nonlinear transforms iteratively pushing the initial samples from a
reference distribution into the samples from an unnormalized target
distribution. To determine the nonlinear transforms at each iteration, we
consider the Wasserstein gradient flow of relative entropy. This gradient flow
determines a path of probability distributions that interpolates the reference
distribution and the target distribution. It is characterized by an ODE system
with velocity fields depending on the density ratios of the density of evolving
particles and the unnormalized target density. To sample with REGS, we need to
estimate the density ratios and simulate the ODE system with particle
evolution. We propose a novel nonparametric approach to estimating the
logarithmic density ratio using neural networks. Extensive simulation studies
on challenging multimodal 1D and 2D mixture distributions and Bayesian logistic
regression on real datasets demonstrate that the REGS outperforms the
state-of-the-art sampling methods included in the comparison.
| Search Query: ArXiv Query: search_query=au:”Feng Yuan”&id_list=&start=0&max_results=10