Kavli Affiliate: Jia Liu
| First 5 Authors: Tianchen Zhou, FNU Hairi, Haibo Yang, Jia Liu, Tian Tong
| Summary:
Reinforcement learning with multiple, potentially conflicting objectives is
pervasive in real-world applications, while this problem remains theoretically
under-explored. This paper tackles the multi-objective reinforcement learning
(MORL) problem and introduces an innovative actor-critic algorithm named MOAC
which finds a policy by iteratively making trade-offs among conflicting reward
signals. Notably, we provide the first analysis of finite-time
Pareto-stationary convergence and corresponding sample complexity in both
discounted and average reward settings. Our approach has two salient features:
(a) MOAC mitigates the cumulative estimation bias resulting from finding an
optimal common gradient descent direction out of stochastic samples. This
enables provable convergence rate and sample complexity guarantees independent
of the number of objectives; (b) With proper momentum coefficient, MOAC
initializes the weights of individual policy gradients using samples from the
environment, instead of manual initialization. This enhances the practicality
and robustness of our algorithm. Finally, experiments conducted on a real-world
dataset validate the effectiveness of our proposed method.
| Search Query: ArXiv Query: search_query=au:”Jia Liu”&id_list=&start=0&max_results=3