Kavli Affiliate: Li Xin Li
| First 5 Authors: Bingchen Li, Xin Li, Yiting Lu, Zhibo Chen,
| Summary:
We present the first loss agent, dubbed LossAgent, for low-level image
processing tasks, e.g., image super-resolution and restoration, intending to
achieve any customized optimization objectives of low-level image processing in
different practical applications. Notably, not all optimization objectives,
such as complex hand-crafted perceptual metrics, text description, and
intricate human feedback, can be instantiated with existing low-level losses,
e.g., MSE loss. which presents a crucial challenge in optimizing image
processing networks in an end-to-end manner. To eliminate this, our LossAgent
introduces the powerful large language model (LLM) as the loss agent, where the
rich textual understanding of prior knowledge empowers the loss agent with the
potential to understand complex optimization objectives, trajectory, and state
feedback from external environments in the optimization process of the
low-level image processing networks. In particular, we establish the loss
repository by incorporating existing loss functions that support the end-to-end
optimization for low-level image processing. Then, we design the
optimization-oriented prompt engineering for the loss agent to actively and
intelligently decide the compositional weights for each loss in the repository
at each optimization interaction, thereby achieving the required optimization
trajectory for any customized optimization objectives. Extensive experiments on
three typical low-level image processing tasks and multiple optimization
objectives have shown the effectiveness and applicability of our proposed
LossAgent. Code and pre-trained models will be available at
https://github.com/lbc12345/LossAgent.
| Search Query: ArXiv Query: search_query=au:”Li Xin Li”&id_list=&start=0&max_results=3