[Invited Speaker] Voice identity cloning and protection

講演者：Junichi Yamagishi
会議名：The 2nd Korea-Japan Workshop on Artificial Intelligence
主催者：National Research Foundation (NRF), Korea & Japan Science and Technology Agency (JST), Japan
開催地：Jeju Island, Korea
開催日：2024年8月3日
URL：https://www.jst.go.jp/aprc/en/event/japan-korea2024.html

Voice technology that reproduces an individual’s voice, known as voice cloning, is expected to bring new value to entertainment. However, if misused, it can cause serious problems, such as impersonation and attacks on voice-based personal authentication systems, known in society as deepfakes. We were among the first to focus on this problem and have been researching countermeasures to automatically defend against spoofing attacks using deepfakes since 2015.

In this talk, we will first introduce the correct use of speech generative models such as speech intelligibility enhancement, then explain neural waveform generation and speaker embedding techniques used in voice cloning, and show how they are used in society. Next, as countermeasures against deepfake, we introduce the construction of a large speech database for deepfake detection, the definition of metrics, evaluation in adverse environments, the use of speech foundation models, and the ASVspoof challenge to evaluate detectors on a shared database. The remaining challenges will also be presented.