Zhang SHENGYU  

Tenure-track Assistant Professor

ZJU100 Young Professor(百人计划研究员)
School of Software Technology & Lab of Artificial Intelligence
Zhejiang University
Zhejiang, China. 310000.

Address: Room 207, Zetong Building, Yuquan Campus
Email: sy_zhang αt zju dοt edu dοt cn

Biography

I obtained my Ph.D in the College of Computer Science and Technology at Zhejiang University, advised by Prof. Fei Wu. I was so lucky to work with Prof. Zhou Zhao and Prof. Kun Kuang at Zhejiang University. From March 2021 to September 2022, I become a visiting research scholar of NExT++ Research Center, National University of Singapore, advised by Prof. Tat-seng Chua. I feel grateful to work with Prof. Fuli Feng at University of Science and Technology of China.

As an AI researcher with a specialization in machine learning, my work revolves around the cutting-edge domain of device-cloud collaborative learning, multi-media analysis, and data mining. My research endeavors are driven by a deep interest in the development and deployment of machine learning models that operate collaboratively across both edge devices and cloud servers. Specifically, my research seeks to address the unique challenges associated with the seamless integration of heterogeneous models in these diverse computational environments. Through the design and evaluation of these models, I seek to improve the scalability, reliability, and real-time performance of machine learning systems in a wide range of applications.

News

[Show more]

Research Summary

Knowledge Transfer
Collaborative Learning
Applied Research

Cloud to Device (C2D): Diverse end devices exhibit distinct task functionalities and usage scenarios, rendering the migration and deployment of cloud models to the edge a complex endeavor. This process encounters significant challenges in achieving cross-scenario/domain/task/distribution generalization.

Publications

* denotes co-first authors, denotes the corresponding author, # denotes (co-)supervised students

Highlights
OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use
Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shawn Wang, Xinchen Xu, Shuofei Qiao, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang✉, Fei Wu
[Paper] [GitHub]
Main Idea:
One of the early surveys on Mobile/UI/OS/Computer MLLM/LLM Agent
Reinforcement Learning Enhanced LLMs: A Survey
Shuhe Wang, Shengyu Zhang, Jie Zhang, Runyi Hu, Xiaoya Li, Tianwei Zhang, Jiwei Li, Fei Wu, Guoyin Wang, Eduard Hovy
[Paper] [GitHub]
Main Idea:
An early survey on RL-enhanced LLMs
ModelGPT: Unleashing LLM’s Capabilities for Tailored Model Generation
Zihao Tang, Zheqi Lv, Shengyu Zhang, Fei Wu, Kun Kuang
Arxiv, 2024
[Paper] [知乎]
Main Idea:
User description + A few data + ModelGPT  
=(Inference)   Off-the-shelf AI Model
Instruction tuning for large language models: A survey
Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Guoyin Wang, Fei Wu
Arxiv, 2023
[Paper] [GitHub]
Main Idea:
An early survey on LLM instruction tuning
2025
MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities
Kunxi Li, Tianyu Zhan, Kairui Fu, Shengyu Zhang✉, Kun Kuang, Jiwei Li, Zhou Zhao, Fan Wu, Fei Wu
AAAI 2025 (to appear)
Main Idea:
An Unified framework for heterogeneous knowledge transfer
FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learning
Zhonghua Jiang, Jimin Xu, Shengyu Zhang✉, Tao Shen, Jiwei Li, Kun Kuang, Haibin Cai, Fei Wu
AAAI 2025 (to appear)
Main Idea:
Counterfactual on-device learning for debiased on-cloud aggregation
Optimize Incompatible Parameters through Compatibility-aware Knowledge Integration
Zheqi Lv, KeMing Ye, Wei Zishu, Qi Tian, Shengyu Zhang✉, Wenqiao Zhang, Wenjie Wang, Kun Kuang, Tat-Seng Chua, Fei Wu
AAAI 2025 (to appear)
Main Idea:
Optimize model over other models with parameter merging
Forward Once for All: Structural Parameterized Adaptation for Efficient Cloud-coordinated On-device Recommendation
Kairui Fu, Zheqi Lv, Shengyu Zhang✉, Fan Wu, Kun Kuang
KDD 2025 (Research Track, to appear)
Main Idea:
Compact Device Model *Architecture* customized in Real time.
Collaborative Large Language Models and Sequential Recommendation Models for Device-Cloud Recommendation
Zheqi Lv, Tianyu Zhan, Wenjie Wang, Xinyu Lin, Shengyu Zhang✉, Wenqiao Zhang, Jiwei Li, Kun Kuang, Fei Wu
KDD 2025 (Research Track, to appear)
Main Idea:
Large-small model collaboration for RecSys.
2024
PhiloGPT: A Philology-Oriented Large Language Model for Ancient Chinese Manuscripts with Dunhuang as Case Study
Yuqing Zhang, Baoyi He, Yihan Chen, Hangqi Li, Han Yue, Shengyu Zhang✉, Huaiyong Dou, Junchi Yan, Zemin Liu, Yongquan Zhang, Fei Wu
EMNLP 2024 (Main)
Semantic Codebook Learning for Dynamic Recommendation Models
Zheqi Lv, Shaoxuan He, Tianyu Zhan, Shengyu Zhang✉, Wenqiao Zhang, Jingyuan Chen, Zhou Zhao, Fei Wu
ACM MM 2024
Main Idea:
Dynamic Parameter generation through codebook learning.
GaussianTalker: Speaker-specific Talking Head Synthesis via 3D Gaussian Splatting
Hongyun Yu, Zhan Qu, Qihang Yu, Jianchuan Chen, Zhonghua Jiang, Zhiwen Chen, Shengyu Zhang✉, Jimin Xu, Fei Wu, chengfei lv, Gang Yu
ACM MM 2024
[Presentation]
Cross-modal Observation Hypothesis Inferences
Mengze Li, Kairong Han, Jiahe Xu, Yueying Li, Tao Wu, Zhou Zhao, Jiaxu Miao, Shengyu Zhang✉, Jingyuan Chen
ACM MM 2024 (Oral), BEST PAPER Candidate
LLMCO4MS: LLMs-aided Neural Combinatorial Solver for Ancient Manuscript Restoration from Fragments
Yuqing Zhang, Hangqi Li, Shengyu Zhang✉, Runzhong Wang, Baoyi He, Huaiyong Dou, Junchi Yan, Yongquan Zhang, Fei Wu
ECCV 2024 (to appear)
Main Idea:
An early work to explore Combinatorial Solver augmented LLM
DIET: Customized Slimming for Incompatible Networks in Sequential Recommendation
Kairui Fu, Shengyu Zhang✉, Zheqi Lv, Jingyuan Chen, Jiwei Li
KDD 2024 (Research Track, to appear)
Main Idea:
Random Networks - Distribution-incompatible Params.
= Compact Device Model customized in Real time
Transferring Causal Mechanism over Meta-representations for Target-unknown Cross-domain Recommendation
Shengyu Zhang, Qiaowei Miao, Ping Nie, Mengze Li, Zhengyu Chen, Fuli Feng, Kun Kuang, Fei Wu
ACM Transactions on Information Systems (TOIS), 2024
AuG-KD: Anchor-Based Mixup Generation for Out-of-Domain Knowledge Distillation
Zihao Tang, Zheqi Lv, Shengyu Zhang✉, Yifan Zhou, Xinyu Duan, Fei Wu, Kun Kuang
The International Conference on Learning Representations (ICLR), 2024
[Paper] [GitHub]
Intelligent Model Update Strategy for Sequential Recommendation
Zheqi Lv, Wenqiao Zhang, Zhengyu Chen, Shengyu Zhang✉, Kun Kuang
The Web Conference (WWW), 2024
[Paper] [Presentation]
MPOD123: One Image to 3D Content Generation Using Mask-enhanced Progressive Outline-to-Detail Optimization
Jimin Xu, Tianbao Wang, Tao Jin, Shengyu Zhang✉, Dongjie Fu, Zhe Wang, Jiangjing Lyu, Chengfei Lv, Chaoyue Niu, Zhou Yu, Zhou Zhao, Fei Wu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
[Paper] [Demo]
CoreRec: A Counterfactual Correlation Inference for Next Set Recommendation
Kexin Li, Chengjiang Long, Shengyu Zhang, Xudong Tang, Zhichao Zhai, Kun Kuang, Jun Xiao
AAAI Conference on Artificial Intelligence (AAAI), 2024
2023
Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems
Shengyu Zhang, Ziqi Jiang, Jiangchao Yao, Fuli Feng, Kun Kuang, Zhou Zhao, Shuo Li, Hongxia Yang, Tat-seng Chua, Fei Wu
IEEE Transactions on Knowledge and Data Engineering (TKDE), 2023
SLED: Structure Learning based Denoising for Recommendation
Shengyu Zhang, Tan Jiang, Kun Kuang, Fuli Feng, Jin Yu, Jianxin Ma, Zhou Zhao, Jianke Zhu, Hongxia Yang, Tat-sen Chua, Fei Wu
ACM Transactions on Information Systems (TOIS), 2023
Reconnecting the Broken Civilization: Patchwork Integration of Fragments from Ancient Manuscripts
Yuqing Zhang, Zhou Fang, Xinyu Yang, Shengyu Zhang✉, Baoyi He, Huaiyong Dou, Junchi Yan, Yongquan Zhang, Fei Wu
ACM International Conference on Multimedia (ACM MM), 2023
Unsupervised Domain Adaptation for Video Object Grounding with Cascaded Debiasing Learning
Mengze Li, Haoyu Zhang, Juncheng Li, Zhou Zhao, Wenqiao Zhang, Shengyu Zhang✉, Shiliang Pu, Yueting Zhuang, Fei Wu
ACM International Conference on Multimedia (ACM MM), 2023
DisCover: Disentangled Music Representation Learning for Cover Song Identification
Jiahao Xun, Shengyu Zhang✉, Yanting Yang, Jieming Zhu, Liqun Deng, Zhou Zhao, Zhenhua Dong, Ruiqi Li, Lichao Zhang, Fei Wu
International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2023
WINNER: Weakly-supervised hIerarchical decompositioN and aligNment for spatio-tEmporal video gRounding
Mengze Li, Han Wang, Wenqiao Zhang, Jiaxu Miao, Wei Ji, Zhou Zhao, Shengyu Zhang✉, Fei Wu
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Are Binary Annotations Sufficient? Video Moment Retrieval via Hierarchical Uncertainty-based Active Learning
Wei Ji, Renjie Liang, Zhedong Zheng, Wenqiao Zhang, Shengyu Zhang, Juncheng Li, Mengze Li, Tat-Seng Chua
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Personalized Latent Structure Learning for Recommendation
Shengyu Zhang, Fuli Feng, Kun Kuang, Wenqiao Zhang, Zhou Zhao, Hongxia Yang, Tat-Seng Chua, Fei Wu
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023
Video-Audio Domain Generalization via Confounder Disentanglement
Shengyu Zhang, Xusheng Feng, Wenyan Fan, Wenjing Fang, Fuli Feng, Wei Ji, Shuo Li, Li Wang, Shanshan Zhao, Zhou Zhao, Tat-Seng Chua, Fei Wu
AAAI Conference on Artificial Intelligence (AAAI), 2023
End-to-End Optimization of Quantization-based Structure Learning and Interventional Next-item Recommendation
Kairui Fu#, Qiaowei Miao, Shengyu Zhang✉, Kun Kuang, Fei Wu
The 3rd CAAI International Conference on Artificial Intelligence (CICAI), 2023, BEST PAPER AWARD
Multi-trends Enhanced Dynamic Micro-video Recommendation
Yujie Lu, Yingxuan Huang, Shengyu Zhang✉, Wei Han, Hui Chen, Zhou Zhao, Fei Wu
The 3rd CAAI International Conference on Artificial Intelligence (CICAI), 2023
DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization
Zheqi Lv, Wenqiao Zhang, Shengyu Zhang, Kun Kuang, Feng Wang, Yongwei Wang, Zhengyu Chen, Tao Shen, Hongxia Yang, Beng Chin Ooi, Fei Wu
The Web Conference (WWW), 2023
Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning
Ye Wang, Wang Lin, Shengyu Zhang, Tao Jin, Linjun Li, Xize Cheng, Zhou Zhao
The Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Multi-modal Action Chain Abductive Reasoning
Mengze Li, Tianbao Wang, Jiahe Xu, Kairong Han, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Shiliang Pu, Fei Wu
The Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Personalizing Intervened Network for Long-tailed Sequential User Behavior Modeling
Zheqi Lv*, Feng Wang*, Shengyu Zhang*, Kun Kuang, Hongxia Yang, Fei Wu
The 3rd CAAI International Conference on Artificial Intelligence (CICAI), 2023
2022 selected papers
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey
Jiangchao Yao, Shengyu Zhang, Yang Yao, Feng Wang, Jianxin Ma, Jianwei Zhang, Yunfei Chu, Luo ji, Kunyang Jia, Tao Shen, Anpeng Wu, Fengda Zhang, Ziqi Tan, Kun Kuang, Chao Wu, Fei Wu
IEEE Transactions on Knowledge and Data Engineering (TKDE), 2022
Intelligent Request Strategy Design in Recommender System
Xufeng Qian, Yue Xu, Fuyu Lv, Shengyu Zhang✉, Ziwen Jiang, Qingwen Liu, Xiaoyi Zeng, Tat-Seng Chua, Fei Wu
The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2022
Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation
Shengyu Zhang, Lingxiao Yang, Dong Yao, Yujie Lu, Fuli Feng, Zhou Zhao, Tat-Seng Chua, Fei Wu
International World Wide Web Conferences (WWW), 2022
[Paper] [GitHub]
[Show more papers on 2022]
2021
CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendation
Shengyu Zhang, Dong Yao, Zhou Zhao, Tat-Seng Chua, Fei Wu
ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2021
[Paper] [GitHub]
[Show more papers on 2021]
2020 and prior
DeVLBert: Learning Deconfounded Visio-Linguistic Representations
Shengyu Zhang, Tan Jiang, Tan Wang, Kun Kuang, Zhou Zhao, Jianke Zhu, Jin Yu, Hongxia Yang, Fei Wu
ACM International Conference on Multimedia (MM), 2020
[Paper] [GitHub]
[Show more papers on 2020]

Workshop & Short Papers

[Show]

Talks

Selected Honors Awarded

Academic Service

Last updated on Dec., 2024. Webpage template borrowed from Prof. Sida Peng.