Xueyao Zhang (张雪遥)

Ph.D. student,
School of Data Science,
The Chinese University of Hong Kong, Shenzhen

E-mail: xueyaozhang [AT] link.cuhk.edu.cn

Curriculum Vitae    /    Google Scholar    /    DBLP    /    GitHub    /    Zhihu

About me

I'm a second-year Ph.D. student at the Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), supervised by Professor Zhizheng Wu. My ongoing works focus on Singing Voice Conversion (SVC). To know more about SVC, you can read this tutorial and my latest work. Recently, I'm participating in and leading the development of the prototype of the open-source Amphion toolkit. Before CUHK-Shenzhen, I received my master's degree (2019-2022) at the Institute of Computing Technology Chinese Academy of Sciences (ICT, CAS), working on computational social science, especially on Fake News Detection and Fact Checking.

My research interests include:

  • Applications: AI Music & AI for Music, Audio Generation, AI for Social Good
  • Technologies: Generative Model, Representation Learning

Milestones
2023/12 My first attempt at leading the development of a large-scale open-source project.
2023/10 My first paper about singing voice processing got accepted by ML4Audio @ NeurIPS 2023.
2023/04 I was admitted by Tencent Rhino-Bird Talent Program 腾讯犀牛鸟精英人才计划 (Top 50+ of China).
2022/09 I entered the Chinese University of Hong Kong (Shenzhen) as a Ph.D. student.
2022/06 My first paper about music generation got accepted by ACM MM 2022.
2021/11 I got the National Graduate Scholarship 2021 (funded by Ministry of Education of China).
2021/01 My first paper about fake news detection got accepted by WWW 2021.
2020/01 I got the third place at Campus Singer Competition, University of Chinese Academy of Sciences.

✍   Call for Cooperations

Our team is broadly interested in Audio/Speech processing and synthesis, DeepFake detection, and AI + Music. We publish our work in the top conferences and journals, and deploy research output to products by collaborating with industry. Our team has open positions for Postdocs, PhD students, Research Assistants, and visiting research students/researchers (more information can be seen here). If you are willing to join our team, feel free to contact wuzhizheng [AT] cuhk.edu.cn. Besides, you are always welcome to discuss any ideas with me.

Representative Works
AI Music and Audio Generation
   Amphion: An Open-Source Audio, Music and Speech Generation Toolkit
Xueyao Zhang*, Liumeng Xue*, Yicheng Gu*, Yuancheng Wang*, Haorui He, Chaoren Wang, Xi Chen, Zihao Fang, Haopeng Chen, Junan Zhang, Tze Ying Tang, Lexiao Zou, Mingxuan Wang, Jun Han, Kai Chen, Haizhou Li, Zhizheng Wu (*: Equal Contribution)
Technical Report / GitHub / HuggingFace / OpenXLab
TL;DR: We develop a unified audio generation open-source toolkit.
ML4Audio @ NeurIPS 2023   Leveraging Content-based Features from Multiple Acoustic Models for Singing Voice Conversion
Xueyao Zhang, Yicheng Gu, Haopeng Chen, Zihao Fang, Lexiao Zou, Liumeng Xue, Zhizheng Wu
Machine Learning for Audio Workshop (ML4Audio) at NeurIPS 2023
Preprint / Code / Demo / Pretrained Model / HuggingFace Space / OpenXLab App
TL;DR: We propose to utilize multiple content features for singing voice conversion.
MM 2022   Structure-Enhanced Pop Music Generation via Harmony-Aware Learning
Xueyao Zhang, Jinchao Zhang, Yao Qiu, Li Wang, Jie Zhou
Proceedings of the 30th ACM International Conference on Multimedia (Acceptance Rate: 690/2473=27.9%)
PDF / Preprint / Code / Slides / Demo
TL;DR: We propose to learn harmony for generating form- and texture- enhanced pop music.
Fake News Detection
WWW 2021   Mining Dual Emotion for Fake News Detection
Xueyao Zhang, Juan Cao, Xirong Li, Qiang Sheng, Lei Zhong, and Kai Shu
Proceedings of the 30th Web Conference (Acceptance Rate: 357/1736=20.6%)
PDF / Code / Slides / Video / Chinese Video
TL;DR: We leverage both publisher emotion and social emotion for fake news detection.
CIKM 2021   Integrating Pattern- and Fact-based Fake News Detection via Model Preference Learning
Qiang Sheng*, Xueyao Zhang*, Juan Cao, and Lei Zhong (*: Equal Contribution)
Proceedings of the 30th ACM International Conference on Information and Knowledge Management (Acceptance Rate: 271/1251=21.7%)
PDF / Poster / Code / Chinese Blog
TL;DR: We propose a graph-based model preference learning framework to separately handle the pattern and fact indicators in fake news detection.
👉      Full Publications
Presentations
2024/02 I was honored to be invited to give talks, A Comprehensive Guide to Amphion's Singing Voice Conversion (Amphion的歌声转换指南) [Slides], at
  • Professor Yong Qin's Lab at Nankai University (2024/01/04),
  • Speech Home 语音之家 (2024/01/12),
  • BAAI Talk 智源社区 (2024/01/16) [Recording],
  • TechBeat 将门创投 (2024/02/07) [Recording],
  • Professor Li Liu's Lab at the Hong Kong University of Science and Technology, Guangzhou (2024/2/27).
2022/10 I got the Best Presentation Award at Huawei-CUHKSZ NLP/Speech Workshop by presenting my research work,
Structure-Enhanced Pop Music Generation via Harmony-Aware Learning. [Slides] [Award]
2022/06 I gave a talk at two labs of ICT as an outstanding graduate,
如何做有新意的研究——以“做问题”的视角. [Chinese Blog] [Chinese Slides]

The talk was also presented at SDS Early Career Colloquium of CUHK-Shenzhen,
Towards a novel research in a problem-driven way. [Slides]
2022/05 My master's theis defense,
Research on Fake News Detection Based on Emotion (基于情感的虚假新闻检测方法研究). [Chinese Slides]
2021/05 I gave a talk about composition at Pattern Recognition Center of Wechat AI,
How to create a pop song? (一首流行歌是如何创作的). [Chinese Slides]
Internship
WeChat, Tencent Research Intern, Pattern Recognition Center of Wechat AI, Beijing, China.
AI+Music Research
  • 2022/01      I participated to release the theme song of WeChat Open Class 2022 as a co-producer, Ru Wei(入微), which was composed by AI and sung by humans.
  • 2023/06      I was admitted by Tencent Rhino-Bird Talent Program 腾讯犀牛鸟精英人才计划 (Top 50+ of China), supervised by Jinchao Zhang.
Services
Reviewer Conferences:
  • ACL Rolling Review
  • CSCW 2021
  • EMNLP 2021
  • ICASSP 2023 (valuable reviewer), 2024
  • ICMC 2023
  • MM 2023
  • NCMMSC 2023
Journals:
  • EURASIP Journal on Audio, Speech, and Music Processing
  • IEEE Transactions on Audio, Speech and Language Processing (TASLP)
  • IEEE Transactions on Computational Social Systems (TCSS)
  • Information Processing and Management (IP&M)
  • Journal of Chinese Information Processing (中文信息学报)
Student Volunteer
  • IEEE Spoken Language Technology Workshop 2024
  • Teaching Assistant
  • 2017 Fall, Object-Oriented Programming (JAVA), Wuhan University
  • 2022 Fall, CSC3100 Data Structures, CUHK-Shenzhen
  • 2023 Spring, CSC3160/MDS6002 Fundamentals of Speech and Language Processing, CUHK-Shenzhen. I got the Best Teaching Assistant Award.
  • 2023 Fall, CSC4130 Introduction to Human-Computer Interaction, CUHK-Shenzhen
  • 2024 Spring, CSC4050 Computing Capstone, CUHK-Shenzhen
  • Education
    2022- Ph.D. student in Data Science, supervised by Professor Zhizheng Wu
    School of Data Science (SDS), The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)
    2019-2022 Master in Computer Application Technology (Research-based), supervised by Professor Juan Cao
    Institute of Computing Technology Chinese Academy of Sciences (ICT);
    University of Chinese Academy of Sciences (UCAS)
    2015-2019 B.Eng. in Software Engineering
    School of Computer Science, Wuhan University (WHU)
    2012-2015 Jiyuan No.1 Middle School of Henan
    Honors and Awards
    2022 Outstanding Graduate, University of Chinese Academy of Sciences & Beijing Municipal Education Commission (Top 5%)
    2021 National Graduate Scholarship, Ministry of Education of China (Top 0.2%)
    2020 Third place at Campus Singer Competition, University of Chinese Academy of Sciences (Top 3 among over 50,000)
    2019 Outstanding Graduate, Wuhan University (Top 10%)
    2019 Excellent Bachelor Thesis, Wuhan University (Top 5%)
    2016 National Undergraduate Scholarship, Ministry of Education of China (Top 0.2%)
    2014 First price in Chinese High School Mathematics League (Top 50 in Henan Province)
    Blogs
    #年终总结# 2023 《2023与兔年:开端、挑战与不经意间》
    2022 《2022年:伤痛、时间与起点》
    2021 《2021年:收获、决定与困惑 》
    2020 《去年》
    2019 《2019大事记》
    2018 《请回答2018》
    2017 《2017流水账》
    #哲学思考# 2020/02/10 《对人工智能艺术创作的哲学思考》
    2016/11/24 《为什么有有,没有没有》
    #关于大学# 2019/10/02 《后大学时代》
    2018/08/04 《的大学》
    #关于高考# 2017/10/20 《后高考时代|感情篇》
    2016/05/02 《后高考时代|思辨篇》
    2016/04/26 《后高考时代|离别篇》
    Recommended Sites
    Visualized Machine Learning Jay Alammar            Distill
    Institutions Magenta (Google AI)            Sony CSL Music            OpenAI            DeepMind
    Individual Bloggers Lil'Log            科学空间(苏剑林)