Kavli Affiliate: Xiang Zhang
| First 5 Authors: Yihe Wang, Taida Li, Yujun Yan, Wenzhan Song, Xiang Zhang
| Summary:
Medical time series (MedTS) play a critical role in many healthcare
applications, such as vital sign monitoring and the diagnosis of brain and
heart diseases. However, the existence of subject-specific features poses
unique challenges in MedTS evaluation. Inappropriate evaluation setups that
either exploit or overlook these features can lead to artificially inflated
classification performance (by up to 50% in accuracy on ADFTD dataset): this
concern has received little attention in current research. Here, we categorize
the existing evaluation setups into two primary categories: subject-dependent
and subject-independent. We show the subject-independent setup is more
appropriate for different datasets and tasks. Our theoretical analysis explores
the feature components of MedTS, examining how different evaluation setups
influence the features that a model learns. Through experiments on six datasets
(spanning EEG, ECG, and fNIRS modalities) using four different methods, we
demonstrate step-by-step how subject-dependent utilizes subject-specific
features as a shortcut for classification and leads to a deceptive high
performance, suggesting that the subject-independent setup is more precise and
practicable evaluation setup in real-world. This comprehensive analysis aims to
establish clearer guidelines for evaluating MedTS models in different
healthcare applications. Code to reproduce this work in
url{https://github.com/DL4mHealth/MedTS_Evaluation}.
| Search Query: ArXiv Query: search_query=au:”Xiang Zhang”&id_list=&start=0&max_results=3