Zhang SHENGYU  

Tenure-track Assistant Professor

ZJU100 Young Professor（百人计划研究员）
School of Software Technology & Lab of Artificial Intelligence
Zhejiang University
Zhejiang, China. 310000.

Address: Room 207, Zetong Building, Yuquan Campus
Email: sy_zhang αt zju dοt edu dοt cn

Biography

I obtained my Ph.D in the College of Computer Science and Technology at Zhejiang University, advised by Prof. Fei Wu. I was so lucky to work with Prof. Zhou Zhao and Prof. Kun Kuang at Zhejiang University. From March 2021 to September 2022, I become a visiting research scholar of NExT++ Research Center, National University of Singapore, advised by Prof. Tat-seng Chua. I feel grateful to work with Prof. Fuli Feng at University of Science and Technology of China.

As an AI researcher with a specialization in machine learning, my work revolves around the cutting-edge domain of device-cloud collaborative learning, multi-media analysis, and data mining. My research endeavors are driven by a deep interest in the development and deployment of machine learning models that operate collaboratively across both edge devices and cloud servers. Specifically, my research seeks to address the unique challenges associated with the seamless integration of heterogeneous models in these diverse computational environments. Through the design and evaluation of these models, I seek to improve the scalability, reliability, and real-time performance of machine learning systems in a wide range of applications.

News

[Show more]

Research Summary

Knowledge Transfer
Collaborative Learning
Applied Research

Cloud to Device (C2D): Diverse end devices exhibit distinct task functionalities and usage scenarios, rendering the migration and deployment of cloud models to the edge a complex endeavor. This process encounters significant challenges in achieving cross-scenario/domain/task/distribution generalization.

Publications

* denotes co-first authors, ✉ denotes the corresponding author, # denotes (co-)supervised students

Highlights
ModelGPT: Unleashing LLM’s Capabilities for Tailored Model Generation
Zihao Tang, Zheqi Lv, Shengyu Zhang, Fei Wu, Kun Kuang
Arxiv, 2024
[Paper] [知乎]
Main Idea:
User description + A few data + ModelGPT  
=(Inference)   Off-the-shelf AI Model
Instruction tuning for large language models: A survey
Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Guoyin Wang, Fei Wu
Arxiv, 2023
[Paper] [GitHub]
Main Idea:
An early survey on LLM instruction tuning
2024
Semantic Codebook Learning for Dynamic Recommendation Models
Zheqi Lv, Shaoxuan He, Tianyu Zhan, Shengyu Zhang✉, Wenqiao Zhang, Jingyuan Chen, Zhou Zhao, Fei Wu
ACM MM 2024 (to appear)
Main Idea:
Dynamic Parameter generation through codebook learning.
GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian Splatting
Hongyun Yu, Zhan Qu, Qihang Yu, Jianchuan Chen, Zhonghua Jiang, Zhiwen Chen, Shengyu Zhang✉, Jimin Xu, Fei Wu, chengfei lv, Gang Yu
ACM MM 2024 (to appear)
[Presentation]
Main Idea:
Heterogeneous model collaboration for 3D & Video.
Cross-modal Observation Hypothesis Inferences
Mengze Li, Kairong Han, Jiahe Xu, Yueying Li, Tao Wu, Zhou Zhao, Jiaxu Miao, Shengyu Zhang✉, Jingyuan Chen
ACM MM 2024 (Oral), BEST PAPER Candidate
LLMCO4MS: LLMs-aided Neural Combinatorial Solver for Ancient Manuscript Restoration from Fragments
Yuqing Zhang, Hangqi Li, Shengyu Zhang✉, Runzhong Wang, Baoyi He, Huaiyong Dou, Junchi Yan, Yongquan Zhang, Fei Wu
ECCV 2024 (to appear)
Main Idea:
An early work to explore Combinatorial Solver augmented LLM
DIET: Customized Slimming for Incompatible Networks in Sequential Recommendation
Kairui Fu, Shengyu Zhang✉, Zheqi Lv, Jingyuan Chen, Jiwei Li
KDD 2024 (Research Track, to appear)
Main Idea:
Random Networks - Distribution-incompatible Params.
= Compact Device Model customized in Real time
Transferring Causal Mechanism over Meta-representations for Target-unknown Cross-domain Recommendation
Shengyu Zhang, Qiaowei Miao, Ping Nie, Mengze Li, Zhengyu Chen, Fuli Feng, Kun Kuang, Fei Wu
ACM Transactions on Information Systems (TOIS), 2024
AuG-KD: Anchor-Based Mixup Generation for Out-of-Domain Knowledge Distillation
Zihao Tang, Zheqi Lv, Shengyu Zhang✉, Yifan Zhou, Xinyu Duan, Fei Wu, Kun Kuang
The International Conference on Learning Representations (ICLR), 2024
[Paper] [GitHub]
Intelligent Model Update Strategy for Sequential Recommendation
Zheqi Lv, Wenqiao Zhang, Zhengyu Chen, Shengyu Zhang✉, Kun Kuang
The Web Conference (WWW), 2024
[Paper] [Presentation]
MPOD123: One Image to 3D Content Generation Using Mask-enhanced Progressive Outline-to-Detail Optimization
Jimin Xu, Tianbao Wang, Tao Jin, Shengyu Zhang✉, Dongjie Fu, Zhe Wang, Jiangjing Lyu, Chengfei Lv, Chaoyue Niu, Zhou Yu, Zhou Zhao, Fei Wu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
[Paper] [Demo]
CoreRec: A Counterfactual Correlation Inference for Next Set Recommendation
Kexin Li, Chengjiang Long, Shengyu Zhang, Xudong Tang, Zhichao Zhai, Kun Kuang, Jun Xiao
AAAI Conference on Artificial Intelligence (AAAI), 2024
2023
Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems
Shengyu Zhang, Ziqi Jiang, Jiangchao Yao, Fuli Feng, Kun Kuang, Zhou Zhao, Shuo Li, Hongxia Yang, Tat-seng Chua, Fei Wu
IEEE Transactions on Knowledge and Data Engineering (TKDE), 2023
SLED: Structure Learning based Denoising for Recommendation
Shengyu Zhang, Tan Jiang, Kun Kuang, Fuli Feng, Jin Yu, Jianxin Ma, Zhou Zhao, Jianke Zhu, Hongxia Yang, Tat-sen Chua, Fei Wu
ACM Transactions on Information Systems (TOIS), 2023
Reconnecting the Broken Civilization: Patchwork Integration of Fragments from Ancient Manuscripts
Yuqing Zhang, Zhou Fang, Xinyu Yang, Shengyu Zhang✉, Baoyi He, Huaiyong Dou, Junchi Yan, Yongquan Zhang, Fei Wu
ACM International Conference on Multimedia (ACM MM), 2023
Unsupervised Domain Adaptation for Video Object Grounding with Cascaded Debiasing Learning
Mengze Li, Haoyu Zhang, Juncheng Li, Zhou Zhao, Wenqiao Zhang, Shengyu Zhang✉, Shiliang Pu, Yueting Zhuang, Fei Wu
ACM International Conference on Multimedia (ACM MM), 2023
DisCover: Disentangled Music Representation Learning for Cover Song Identification
Jiahao Xun, Shengyu Zhang✉, Yanting Yang, Jieming Zhu, Liqun Deng, Zhou Zhao, Zhenhua Dong, Ruiqi Li, Lichao Zhang, Fei Wu
International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2023
WINNER: Weakly-supervised hIerarchical decompositioN and aligNment for spatio-tEmporal video gRounding
Mengze Li, Han Wang, Wenqiao Zhang, Jiaxu Miao, Wei Ji, Zhou Zhao, Shengyu Zhang✉, Fei Wu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Are Binary Annotations Sufficient? Video Moment Retrieval via Hierarchical Uncertainty-based Active Learning
Wei Ji, Renjie Liang, Zhedong Zheng, Wenqiao Zhang, Shengyu Zhang, Juncheng Li, Mengze Li, Tat-Seng Chua
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Personalized Latent Structure Learning for Recommendation
Shengyu Zhang, Fuli Feng, Kun Kuang, Wenqiao Zhang, Zhou Zhao, Hongxia Yang, Tat-Seng Chua, Fei Wu
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023
Video-Audio Domain Generalization via Confounder Disentanglement
Shengyu Zhang, Xusheng Feng, Wenyan Fan, Wenjing Fang, Fuli Feng, Wei Ji, Shuo Li, Li Wang, Shanshan Zhao, Zhou Zhao, Tat-Seng Chua, Fei Wu
AAAI Conference on Artificial Intelligence (AAAI), 2023
End-to-End Optimization of Quantization-based Structure Learning and Interventional Next-item Recommendation
Kairui Fu#, Qiaowei Miao, Shengyu Zhang✉, Kun Kuang, Fei Wu
The 3rd CAAI International Conference on Artificial Intelligence (CICAI), 2023, BEST PAPER AWARD
Multi-trends Enhanced Dynamic Micro-video Recommendation
Yujie Lu, Yingxuan Huang, Shengyu Zhang✉, Wei Han, Hui Chen, Zhou Zhao, Fei Wu
The 3rd CAAI International Conference on Artificial Intelligence (CICAI), 2023
DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization
Zheqi Lv, Wenqiao Zhang, Shengyu Zhang, Kun Kuang, Feng Wang, Yongwei Wang, Zhengyu Chen, Tao Shen, Hongxia Yang, Beng Chin Ooi, Fei Wu
The Web Conference (WWW), 2023
Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning
Ye Wang, Wang Lin, Shengyu Zhang, Tao Jin, Linjun Li, Xize Cheng, Zhou Zhao
The Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Multi-modal Action Chain Abductive Reasoning
Mengze Li, Tianbao Wang, Jiahe Xu, Kairong Han, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Shiliang Pu, Fei Wu
The Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Personalizing Intervened Network for Long-tailed Sequential User Behavior Modeling
Zheqi Lv*, Feng Wang*, Shengyu Zhang*, Kun Kuang, Hongxia Yang, Fei Wu
The 3rd CAAI International Conference on Artificial Intelligence (CICAI), 2023
2022
MIC: Model-agnostic Integrated Cross-channel Recommender
Ping Nie, Yujie Lu, Shengyu Zhang, Ming Zhao, Ruobing Xie, William Yang Wang, Yi Ren
ACM International Conference on Information and Knowledge Management (CIKM), 2022
Weakly-supervised Disentanglement Network for Video Fingerspelling Detection
Ziqi Jiang#, Shengyu Zhang✉, Siyuan Yao, Wenqiao Zhang, Sihan Zhang, Juncheng Li, Zhou Zhao, Fei Wu
ACM International Conference on Multimedia (MM), 2022
HERO: Hierarchical Spatio-temporal reasoning with Contrastive Action Correspondence for Video Object Grounding
Mengze Li#, Tianbao Wang, Haoyu Zhang, Shengyu Zhang, Zhou Zhao, Wenqiao Zhang, Jiaxu Miao, Shiliang Pu, Fei Wu
ACM International Conference on Multimedia (MM), 2022
Dilated Context Integrated Network with Cross-Modal Consensus for Temporal Emotion Localization in Videos
Juncheng Li, Junlin Xie, Linchao Zhu, Long Qian, Siliang Tang, Wenqiao Zhang, Haochen Shi, Shengyu Zhang, Longhui Wei, Qi Tian, Yueting Zhuang
ACM International Conference on Multimedia (MM), 2022
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey
Jiangchao Yao, Shengyu Zhang, Yang Yao, Feng Wang, Jianxin Ma, Jianwei Zhang, Yunfei Chu, Luo ji, Kunyang Jia, Tao Shen, Anpeng Wu, Fengda Zhang, Ziqi Tan, Kun Kuang, Chao Wu, Fei Wu
IEEE Transactions on Knowledge and Data Engineering (TKDE), 2022
Intelligent Request Strategy Design in Recommender System
Xufeng Qian, Yue Xu, Fuyu Lv, Shengyu Zhang✉, Ziwen Jiang, Qingwen Liu, Xiaoyi Zeng, Tat-Seng Chua, Fei Wu
The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022
BoostMIS: Boosting Medical Image Semi-supervised Learning with Adaptive Pseudo Labeling and Informative Active Annotation
Wenqiao Zhang, Lei Zhu, James Hallinan, Shengyu Zhang, Andrew Makmur, Qingpeng Cai, Beng Chin Ooi
IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR), 2022
[Paper] [GitHub]
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding
Mengze Li, Tianbao Wang, Haoyu Zhang, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Wenming Tan, Jin Wang, PENG WANG, Shiliang Pu, Fei Wu
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation
Shengyu Zhang, Lingxiao Yang, Dong Yao, Yujie Lu, Fuli Feng, Zhou Zhao, Tat-Seng Chua, Fei Wu
International World Wide Web Conferences (WWW), 2022
[Paper] [GitHub]
Contrastive Learning Adopting Positive-Negative Frame Mask for Music Representation
Dong Yao, Zhou Zhao, Shengyu Zhang✉, Jieming Zhu, Yudong Zhu, Rui Zhang, Xiuqiang He
International World Wide Web Conferences (WWW), 2022
[Paper] [GitHub]
MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning
Wenqiao Zhang, Haochen Shi, Jiannan Guo, Shengyu Zhang, Qingpeng Cai, Juncheng Li, Sihui Luo, Yueting Zhuang
Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI), 2022
Uncovering Causal Effects of Online Short Videos on Consumer Behaviors
Ziqi Tan✰, Shengyu Zhang, Nuanxin Hong, Kun Kuang, Yifan Yu, Zhou Zhao, Jin Yu, Hongxia Yang, Shiyuan Pan, Jingren Zhou, Fei Wu
The Fifteenth International Conference on Web Search and Data Mining (WSDM), 2022
2021
[Show]
2020 and prior
[Show]

Workshop & Short Papers

[Show]

Talks

Selected Honors Awarded

Academic Service

Last updated on July, 2024. Webpage template borrowed from Prof. Sida Peng.