Kavli Affiliate: Xiang Zhang
| First 5 Authors: Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, Marinka Zitnik,
| Summary:
Pre-training on time series poses a unique challenge due to the potential
mismatch between pre-training and target domains, such as shifts in temporal
dynamics, fast-evolving trends, and long-range and short cyclic effects, which
can lead to poor downstream performance. While domain adaptation methods can
mitigate these shifts, most methods need examples directly from the target
domain, making them suboptimal for pre-training. To address this challenge,
methods need to accommodate target domains with different temporal dynamics and
be capable of doing so without seeing any target examples during pre-training.
Relative to other modalities, in time series, we expect that time-based and
frequency-based representations of the same example are located close together
in the time-frequency space. To this end, we posit that time-frequency
consistency (TF-C) — embedding a time-based neighborhood of a particular
example close to its frequency-based neighborhood and back — is desirable for
pre-training. Motivated by TF-C, we define a decomposable pre-training model,
where the self-supervised signal is provided by the distance between time and
frequency components, each individually trained by contrastive estimation. We
evaluate the new method on eight datasets, including electrodiagnostic testing,
human activity recognition, mechanical fault detection, and physical status
monitoring. Experiments against eight state-of-the-art methods show that TF-C
outperforms baselines by 15.4% (F1 score) on average in one-to-one settings
(e.g., fine-tuning an EEG-pretrained model on EMG data) and by up to 8.4% (F1
score) in challenging one-to-many settings, reflecting the breadth of scenarios
that arise in real-world applications. The source code and datasets are
available at https: //anonymous.4open.science/r/TFC-pretraining-6B07.
| Search Query: ArXiv Query: search_query=au:”Xiang Zhang”&id_list=&start=0&max_results=10