基于路径签名的时间序列领域自适应方法
作者:
中图分类号:

TP18

基金项目:

科技创新2030—“新一代人工智能”重大项目(2021ZD0111501); 国家优秀青年科学基金 (62122022); 国家自然科学基金(61876043, 61976052, 62206064)


Time-series Domain Adaptation Based on Path Signature
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [85]
  • |
  • 相似文献
  • | | |
  • 文章评论
    摘要:

    近年来深度学习因其在各个场景下的优异性能而受到越来越多研究者的重视, 但是这些方法通常依赖独立同分布假设. 领域自适应则是为了缓解分布偏移带来的影响而提出的问题, 它利用带标签的源域数据和不带标签的目标域数据能够训练得到在目标域数据上性能较好的模型. 现有的领域自适应方法大多针对静态数据, 而时间序列数据的方法需要捕捉变量之间的依赖关系. 现有的方法虽然采用针对时间序列数据的特征提取器, 例如递归神经网络, 以学习变量间的依赖关系, 但是往往将冗余的信息也一同提取. 这些冗余信息容易和语义信息耦合, 进而影响模型的预测性能. 基于上述问题, 提出一种基于路径签名的时间序列领域自适应方法(path-signature-based time-series domain adaptation, PSDA). 该方法一方面利用路径签名变换来捕捉变量间的稀疏依赖关系, 排除冗余相关关系的同时保留语义相关关系, 从而有利于提取时序数据中具有判别力的特征; 另一方面通过约束源域和目标域之间的依赖关系一致性来保留领域之间不变的依赖关系, 排除领域变化的依赖关系, 从而有利于提取时序数据中具有泛化性的特征. 基于以上方法, 进一步提出一个距离度量标准和泛化性边界理论, 并且在多个时间序列领域自适应标准数据集上获得了最好的实验效果.

    Abstract:

    Recently, deep learning has received increasing attention from researchers due to its excellent performance in various scenarios, but these methods often rely on the independent and identically distribution assumption. Domain adaptation is a problem proposed to mitigate the impact of distribution offset, which uses labeled source domain data and unlabeled target domain data to achieve better performance on target data. Existing methods are devised for static data, while the methods for time series data need to capture the dependencies between variables. Although these methods use feature extractors for time series data, such as recurrent neural networks, to learn the dependencies between variables, they often extract redundant information. This information is easily entangled with semantic information, affecting the model performance. To solve these problems, this study proposes a path-signature-based time-series domain adaptation (PSDA). On the one hand, this method uses path signature transformation to capture sparse dependencies between variables and eliminate redundant correlations while preserving semantic dependencies, thereby facilitating the extraction of discriminative features from temporal data. On the other hand, the invariant dependency relationships are preserved by constraining the consistency of dependency relationships among different domains, and the changing dependency relationships between domains are excluded, which is conducive to extracting generalized features from temporal data. Based on the above methods, the study further proposes a distance metric and generalized boundary theory and obtains the best experimental results on multiple time series domain adaptation standard datasets.

    参考文献
    [1] 李晶晶, 孟利超, 张可, 鲁珂, 申恒涛. 领域自适应研究综述. 计算机工程, 2021, 47(6): 1–13.
    Li JJ, Meng LC, Zhang K, Lu K, Shen HT. Review of studies on domain adaptation. Computer Engineering, 2021, 47(6): 1–13 (in Chinese with English abstract).
    [2] 庄福振, 罗平, 何清, 史忠植. 迁移学习研究进展. 软件学报, 2015, 26(1): 26–39. http://www.jos.org.cn/1000-9825/4631.htm
    Zhuang FZ, Luo P, He Q, Shi ZZ. Survey on transfer learning research. Ruan Jian Xue Bao/Journal of Software, 2015, 26(1): 26–39 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4631.htm
    [3] 顾鑫. 跨领域分类学习方法及应用研究 [博士学位论文]. 无锡: 江南大学, 2014.
    Gu X. A study on cross-domain classification and its application [Ph.D. Thesis]. Wuxi: Jiangnan University, 2014 (in Chinese with English abstract).
    [4] 覃姜维. 迁移学习方法研究及其在跨领域数据分类中的应用 [博士学位论文]. 广州: 华南理工大学, 2011.
    Qin JW. Research of transfer learning and its application in classifying cross-domain data [Ph.D. Thesis]. Guangzhou: South China University of Technology, 2011 (in Chinese with English abstract).
    [5] 许敏. 领域自适应学习算法及其应用研究 [博士学位论文]. 无锡: 江南大学, 2014.
    Xu M. A study on domain adaptation algorithm and its application [Ph.D. Thesis]. Wuxi: Jiangnan University, 2014 (in Chinese with English abstract).
    [6] 唐宋, 叶茂, 李旭冬. 领域自适应目标识别综述. 中兴通讯技术, 2017, 23(4): 25–31.
    Tang S, Ye M, Li XD. Domain adaptation object recognition. ZTE Technologies, 2017, 23(4): 25–31 (in Chinese with English abstract).
    [7] 李晓. 基于迁移学习的跨域图像分类方法研究 [博士学位论文]. 西安: 西安电子科技大学, 2017.
    Li X. Research on cross domain image classification via transfer learning [Ph.D. Thesis]. Xi’an: Xidian University, 2017 (in Chinese with English abstract).
    [8] 付家慧. 深度迁移学习算法及其应用研究 [硕士学位论文]. 南京: 南京邮电大学, 2020. [doi: 10.27251/d.cnki.gnjdc.2020.000222]
    Fu JH. On the deep transfer learning and its applications [MS. Thesis]. Nanjing: Nanjing University of Posts and Telecommunications, 2020 (in Chinese with English abstract). [doi: 10.27251/d.cnki.gnjdc.2020.000222]
    [9] Gretton A, Borgwardt KM, Rasch MJ, Schölkopf BH, Smola A. A kernel two-sample test. The Journal of Machine Learning Research, 2012, 13(1): 723–773.
    [10] 蔡瑞初, 李嘉豪, 郝志峰. 基于类内最大均值差异的无监督领域自适应算法. 计算机应用研究, 2020, 37(8): 2371–2375.
    Cai RC, Li JH, Hao ZF. Unsupervised domain adaptive algorithm with intra-class maximum mean discrepancy. Application Research of Computers, 2020, 37(8): 2371–2375 (in Chinese with English abstract).
    [11] 孙俏, 凌卫新. 基于域间相似度序数的迁移学习源领域的选择. 科学技术与工程, 2020, 20(20): 8245–8251.
    Sun Q, Ling WX. Source domain selection in transfer learning based on domain similarity rank. Science, Technology and Engineering, 2020, 20(20): 8245–8251 (in Chinese with English abstract).
    [12] 臧文华. 基于生成对抗网络的迁移学习算法研究 [硕士学位论文]. 成都: 电子科技大学, 2018.
    Zang WH. Research on transfer learning based on generative adversarial networks [MS. Thesis]. Chengdu: University of Electronic Science and Technology, 2018 (in Chinese with English abstract).
    [13] 蒋玲玲, 罗娟娟, 朱玉鹏, 周东青. 基于深度学习的对抗攻击技术综述. 航天电子对抗, 2023, 39(1): 10–18, 50.
    Jiang LL, Luo JJ, Zhu YP, Zhou DQ. A survey on adversarial attacks against deep learning. Aerospace Electronic Warfare, 2023, 39(1): 10–18, 50 (in Chinese with English abstract).
    [14] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 1997, 9(8): 1735–1780.
    [15] Mikolov T, Karafiát M, Burget L, Černocký J, Khudanpur, S. Recurrent neural network based language model. In: Proc. of the 11th Annual Conf. of the Int’l Speech Communication Association. Makuhari, 2010. 1045–1048.
    [16] Da Costa PRDO, Akçay A, Zhang YQ, Kaymak U. Remaining useful lifetime prediction via deep domain adaptation. Reliability Engineering & System Safety, 2020, 195: 106682. [doi: 10.1016/j.ress.2019.106682]
    [17] Liu Q, Xue H. Adversarial spectral kernel matching for unsupervised time series domain adaptation. In: Proc. of the 30th Int’l Joint Conf. on Artificial Intelligence. Montreal: IJCAI.org, 2021. 2744–2750. [doi: 10.24963/ijcai.2021/378]
    [18] He H, Queen O, Koker T, Cuevas C, Tsiligkaridis T, Zitnik M. Domain adaptation for time series under feature and label shifts. In: Proc. of the 40th Int’l Conf. on Machine Learning. Honolulu: PMLR, 2023. 12746–12774.
    [19] Wu HX, Xu JH, Wang JM. Autoformer: Decomposition Transformers with auto-correlation for long-term series forecasting. In: Proc. of the 35th Conf. on Neural Information Processing Systems. OpenReview.net, 2021.
    [20] Li WD, Yang X, Liu WQ, Xia YC, Bian J. DDG-DA: Data distribution generation for predictable concept drift adaptation. In: Proc. of the 37th AAAI Conf. on Artificial Intelligence. Washington: AAAI Press, 2022. 4092–4100. [doi: 10.1609/aaai.v36i4.20327]
    [21] Zhang K, Schölkopf B, Muandet K, Wang ZK. Domain adaptation under target and conditional shift. In: Proc. of the 30th Int’l Conf. on Int’l Conf. on Machine Learning. Atlanta: JMLR.org, 2013. III-819–III-827.
    [22] Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T. Deep domain confusion: Maximizing for domain invariance. arXiv:1412.3474, 2014.
    [23] Sun BC, Feng JS, Saenko K. Correlation alignment for unsupervised domain adaptation. In: Csurka G, ed. Domain Adaptation in Computer Vision Applications. Cham: Springer, 2017. 153–171. [doi: 10.1007/978-3-319-58347-1_8]
    [24] Chen C, Fu ZH, Chen ZH, Jin S, Cheng ZW, Jin XY, Hua XS. HoMM: Higher-order moment matching for unsupervised domain adaptation. In: Proc. of the 37th AAAI Conf. on Artificial Intelligence. Washington: AAAI Press, 2020. 3422–3429.
    [25] Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016, 17(1): 2096–2030.
    [26] Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. In: Proc. of the 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017. 2962–2971.
    [27] Zhao H, Zhang SH, Wu GH, Costeira JP, Moura JMF, Gordon GJ. Adversarial multiple source domain adaptation. In: Proc. of the 32nd Int’l Conf. on Neural Information Processing Systems. Montréal: Curran Associates Inc., 2018. 8559–8570.
    [28] Alam F, Joty S, Imran M. Domain adaptation with adversarial training and graph embeddings. In: Proc. of the 56th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers). Melbourne: ACL, 2018. 1077–1087. [doi: 10.18653/v1/P18-1099]
    [29] Wright D, Augenstein I. Transformer based multi-source domain adaptation. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2020. 7963–7974.
    [30] Xie SA, Zheng, ZB, Chen, L, Chen C. Learning semantic representations for unsupervised domain adaptation. In: Proc. of the 35th Int’l Conf. on Machine Learning. Stockholm: PMLR, 2018. 5419–5428.
    [31] Chen XY, Wang SN, Wang JM. Representation subspace distance for domain adaptation regression. In: Proc. of the 38th Int’l Conf. on Machine Learning. PMLR, 2021. 1749–1759.
    [32] Zhu YC, Zhuang FZ, Wang JD, Ke GL, Chen JW, Bian J, Xiong H, He W. Deep subdomain adaptation network for image classification. IEEE Trans. on Neural Networks and Learning Systems, 2021, 32(4): 1713–1722.
    [33] Cai RC, Li ZJ, Wei PF, Qiao J, Zhang K, Hao ZF. Learning disentangled semantic representation for domain adaptation. In: Proc. of the 28th Int’l Joint Conf. on Artificial Intelligence. Macao: AAAI Press, 2019. 2060–2066.
    [34] Cai RC, Chen JW, Li ZJ, Chen W, Zhang KL, Ye JJ, Li ZZ, Yang XY, Zhang ZJ. Time series domain adaptation via sparse associative structure alignment. In: Proc. of the 37th AAAI Conf. on Artificial Intelligence. Washington: AAAI Press, 2021. 6859–6867. [doi: 10.1609/aaai.v35i8.16846]
    [35] Li ZH, Cai RC, Ng HW, Winslett M, Fu TZ, Xu BY, Yang XY, Zhang ZJ. Causal mechanism transfer network for time series domain adaptation in mechanical systems. ACM Trans. on Intelligent Systems and Technology, 2021, 12(2): 23.
    [36] 许宪东. 基于迁移学习的一种可见数字水印分类方法. 科技创新与应用, 2023, 13(8): 139–142.
    Xu XD. A visible digital watermarking classification method based on transfer learning. Technology Innovation and Application, 2023, 13(8): 139–142 (in Chinese with English abstract).
    [37] 汪云云, 孙顾威, 赵国祥, 薛晖. 基于自监督知识的无监督新集域适应学习. 软件学报, 2022, 33(4): 1170–1182. http://www.jos.org.cn/1000-9825/6478.htm
    Wang YY, Sun GW, Zhao GX, Xue H. Unsupervised new-set domain adaptation with self-supervised knowledge. Ruan Jian Xue Bao/Journal of Software, 2022, 33(4): 1170–1182 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6478.htm
    [38] Jin XY, Park Y, Maddix D, Wang H, Wang YY. Domain adaptation for time series forecasting via attention sharing. In: Proc. of the 39th Int’l Conf. on Machine Learning. PMLR, 2022. 10280–10297.
    [39] Ragab M, Eldele E, Chen ZH, Wu M, Kwoh CK, Li XL. Self-supervised autoregressive domain adaptation for time series data. IEEE Trans. on Neural Networks and Learning Systems, 2024, 35(1): 1341–1351.
    [40] Li ZJ, Cai RC, Fu TZJ, Hao ZF, Zhang K. Transferable time-series forecasting under causal conditional shift. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2023: 1–18. [doi: 10.1109/TPAMI.2023.3304354]
    [41] Chevyrev I, Kormilitzin A. A primer on the signature method in machine learning. arXiv:1603.03788, 2016.
    [42] Yang WX, Lyons T, Ni H, Schmid C, Jin LW. Developing the path signature methodology and its application to landmark-based human action recognition. In: Yin G, Zariphopoulou T, eds. Stochastic Analysis, Filtering, and Stochastic Optimization. Cham: Springer, 2022. 431–464. [doi: 10.1007/978-3-030-98519-6_18]
    [43] Glad W, Woolf T. Path signature area-based causal discovery in coupled time series. In: Proc. of the 2021 Causal Analysis Workshop Series. Minneapolis: PMLR, 2021. 21–38.
    [44] Morrill J, Fermanian A, Kidger P, Lyons T. A generalised signature method for multivariate time series feature extraction. arXiv:2006.00873, 2020.
    [45] Bonnier P, Kidger P, Arribas IP, Salvi C, Lyons TJ. Deep signature transforms. In: Proc. of the 33rd Int’l Conf. on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019. 279.
    [46] De Curtò J, De Zarzà I, Yan H, Calafate CT. Learning with signatures. arXiv:2204.07953, 2022.
    [47] 赵艺. 基于路径签名的改进时空图卷积网络. 计算机工程与科学, 2022, 44(12): 2213–2219.
    Zhao Y. Signature spatial improved temporal graph convolutional network. Computer Engineering and Science, 2022, 44(12): 2213–2219 (in Chinese with English abstract).
    [48] Lyons TJ, Caruana M, Lévy T. Differential Equations Driven by Rough Paths. Berlin: Springer, 2007. [doi: 10.1007/978-3-540-71285-5]
    [49] Friz PK, Victoir NB. Multidimensional Stochastic Processes as Rough Paths: Theory and Applications. Cambridge: Cambridge University Press, 2010.
    [50] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 6000–6010.
    [51] Duff IS. A survey of sparse matrix research. Proc. of the IEEE, 1977, 65(4): 500–535.
    [52] Satuluri V, Parthasarathy S, Ruan YY. Local graph sparsification for scalable clustering. In: Proc. of the 2011 ACM SIGMOD Int’l Conf. on Management of Data. Athens: ACM, 2011. 721–732. [doi: 10.1145/1989323.1989399]
    [53] Child R, Gray S, Radford A, Sutskever I. Generating long sequences with sparse Transformers. arXiv:1904.10509, 2019.
    [54] Martins AFT, Astudillo RF. From softmax to sparsemax: A sparse model of attention and multi-label classification. In: Proc. of the 33rd Int’l Conf. on Machine Learning. New York: JMLR.org, 2016. 1614–1623.
    [55] Rüschendorf L. The wasserstein distance and approximation theorems. Probability Theory and Related Fields, 1985, 70(1): 117–129.
    [56] Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC. Improved training of Wasserstein GANs. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 5767–5777.
    [57] Santambrogio F. Optimal Transport for Applied Mathematicians. Cham: Birkhäuser, 2015. [doi: 10.1007/978-3-319-20828-2]
    [58] Vapnik VN. The Nature of Statistical Learning Theory. New York: Springer, 2000. [doi: 10.1007/978-1-4757-3264-1]
    [59] Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz JL. A public domain dataset for human activity recognition using smartphones. In: Proc. of the 21st European Symp. on Artificial Neural Networks. Bruges: ESANN, 2013. 437–442.
    [60] Kwapisz JR, Weiss GM, Moore SA. Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations Newsletter, 2011, 12(2): 74–82.
    [61] Roggen D, Calatroni A, Rossi M, et al. Collecting complex activity datasets in highly rich networked sensor environments. In: Proc. of the 7th Int’l Conf. on Networked Sensing Systems (INSS). Kassel: IEEE, 2010. 233–240. [doi: 10.1109/INSS.2010.5573462]
    [62] Zhang M, Sawchuk AA. USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In: Proc. of the 2012 ACM Conf. on Ubiquitous Computing. Pittsburgh: ACM, 2012. 1036–1043. [doi: 10.1145/2370216.2370438]
    [63] PPG-DaLiA. 2019. https://archive.ics.uci.edu/ml/datasets/PPG-DaLiA
    [64] Rahman MM, Fookes C, Baktashmotlagh M, Sridharan S. On minimum discrepancy estimation for deep domain adaptation. In: Singh R, Vatsa M, Patel VM, Ratha N, eds. Domain Adaptation for Visual Understanding. Cham: Springer, 2020. 81–94. [doi: 10.1007/978-3-030-30671-7]
    [65] Long MS, Cao ZJ, Wang JM, Jordan MI. Conditional adversarial domain adaptation. In: Proc. of the 32nd Int’l Conf. on Neural Information Processing Systems. Montréal: Curran Associates Inc., 2018. 1647–1657.
    [66] Shu R, Bui H, Narui H, Ermon S. A DIRT-T approach to unsupervised domain adaptation. In: Proc. of the 6th Int’l Conf. on Learning Representations. Vancouver: OpenReview.net, 2018.
    [67] Huang WB, Zhang L, Wu H, Min FH, Song AG. Channel-Equalization-HAR: A light-weight convolutional neural network for wearable sensor based human activity recognition. IEEE Trans. on Mobile Computing, 2023, 22(9): 5064–5077.
    [68] Huang WB, Zhang L, Wang SY, Wu H, Song AG. Deep ensemble learning for human activity recognition using wearable sensors via filter activation. ACM Trans. on Embedded Computing Systems, 2022, 22(1): 15.
    [69] Kidger P, Lyons T. Signatory: Differentiable computations of the signature and logsignature transforms, on both CPU and GPU. In: Proc. of the 2019 Int’l Conf. on Learning Representations. Vienna: OpenReview.net, 2019.
    [70] Wilson G, Doppa JR, Cook DJ. Multi-source deep domain adaptation with weak supervision for time-series sensor data. In: Proc. of the 26th ACM SIGKDD Int’l Conf. on Knowledge Discovery & Data Mining. ACM, 2020. 1768–1778.
    相似文献
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

蔡瑞初,颜嘉文,陈道鑫,李梓健,郝志峰.基于路径签名的时间序列领域自适应方法.软件学报,2025,36(2):570-589

复制
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2023-06-28
  • 最后修改日期:2023-08-15
  • 在线发布日期: 2024-07-03
文章二维码
您是第19921079位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京市海淀区中关村南四街4号,邮政编码:100190
电话:010-62562563 传真:010-62562533 Email:jos@iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号