Music technologies will open the future up to new ways of enjoying music both in terms of music creation and music appreciation. In this invited talk, I will introduce the frontiers of music technologies by showing some practical research examples, which have already been made into commercial products or made open to the public, to demonstrate how end users can benefit from singing synthesis technologies, music understanding technologies, and music interfaces.
From the viewpoint of music creation, I will demonstrate a singing synthesis system, VocaListener, that can synthesize natural singing voices by analyzing and imitating human singing. I will also introduce the world's first culture in which people actively enjoy songs with synthesized singing voices as the main vocals: emerging in Japan since singing synthesis software such as Hatsune Miku based on VOCALOID has been attracting attention since 2007. Singing synthesis thus breaks down the long-cherished view that listening to a non-human singing voice is worthless. This is a feat that could not have been imagined before. In the future, other long-cherished views could also be broken down.
As for music appreciation, I will demonstrate a web service for active music listening, "Songle" (http://songle.jp), that has analyzed more than 1,000,000 songs on music- or video-sharing services and facilitates deeper understanding of music. Songle is used to provide a web-based multimedia development framework, "Songle Widget" (http://widget.songle.jp), that makes it easy to develop web-based applications with rigid music synchronization by leveraging music-understanding technologies. Songle Widget enables users to control computer-graphic animation and physical devices such as lighting devices and robot dancers in synchronization with music available on the web. I will then demonstrate a web service for large-scale music browsing, "Songrium" (http://songrium.jp), that allows users to explore music while seeing and utilizing various relations among more than 740,000 music video clips on video-sharing services. Songrium has a three-dimensional visualization function that shows music-synchronized animation, which has already been used as a background movie in a live concert of Hatsune Miku.
This session will be held on November 10th.
Prime Senior Researcher
National Institute of Advanced Industrial Science and Technology (AIST), Japan
Masataka Goto received the Doctor of Engineering degree from Waseda University in 1998. He is currently a Prime Senior Researcher and the Leader of the Media Interaction Group at the National Institute of Advanced Industrial Science and Technology (AIST), Japan. In 1992 he was one of the first to start work on automatic music understanding, and has since been at the forefront of research in music technologies and music interfaces based on those technologies. Over the past 23 years, he has published more than 220 papers in refereed journals and international conferences and has received 43 awards, including several best paper awards, best presentation awards, the Tenth Japan Academy Medal, the Tenth JSPS PRIZE, and the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology (Young Scientists' Prize). He has served as a committee member of over 90 scientific societies and conferences, including the General Chair of the 10th and 15th International Society for Music Information Retrieval Conferences (ISMIR 2009 and 2014).