i-Razor: A Neural Input Razor for Feature Selection and Dimension Search in Large-Scale Recommender Systems

Kavli Affiliate: Ke Wang

| First 5 Authors: Yao Yao, Bin Liu, Haoxun He, Dakui Sheng, Ke Wang

| Summary:

Input features play a crucial role in the predictive performance of DNN-based
industrial recommender systems with thousands of categorical and continuous
fields from users, items, contexts, and their interactions. Noisy features and
inappropriate embedding dimension assignments can impair the performance of
recommender systems and introduce unnecessary complexity in model training and
online serving. Optimizing the input configuration of DNN models, including
feature selection and embedding dimension assignment, has become one of the
essential topics in feature engineering. Typically, feature selection and
embedding dimension search are optimized sequentially, i.e., feature selection
is performed first, followed by embedding dimension search to determine the
optimal dimension size for each selected feature. In contrast, this paper
studies the joint optimization of feature selection and embedding dimension
search. To this end, we propose a differentiable neural textbf{i}nput
textbf{razor}, namely textbf{i-Razor}. Specifically, inspired by recent
advances in neural architecture search, we introduce an end-to-end
differentiable model to learn the relative importance between different
embedding regions of each feature. Furthermore, a flexible pruning algorithm is
proposed to simultaneously achieve feature filtering and dimension size
derivation. Extensive experiments on two large-scale public datasets in the
Click-Through-Rate (CTR) prediction task demonstrate the efficacy and
superiority of i-Razor in balancing model complexity and performance.

| Search Query: ArXiv Query: search_query=au:”Ke Wang”&id_list=&start=0&max_results=10

Read More