Shiyan Jiang
Agora's AI digital human technology leader
Jiang Shiyan, currently the technical leader of AI Digital Human, focuses on the technology development and innovative application of 2D and 3D digital human. He has led and deeply participated in the whole process of development from algorithm design to product landing of Soundnet digital human project, and has achieved remarkable results in several technical fields. In the field of 2D digital human, through the technological breakthrough of voice-driven character image, natural and smooth mouth synchronization and dynamic expression generation have been realized, greatly enhancing the expressiveness and realism of the virtual image. In the field of 3D virtual digital human, a set of real-time mapping model from audio to expression wash has been constructed, using which the virtual Avatar can be accurately driven, providing a high-quality solution for real-time interactive scenes. In addition, in the field of 3D realistic digital human, using 3D Gaussian Splatting technology, we successfully realized multi-view angle and high fidelity virtual image generation, which further promotes the application of digital human in the fields of film and television, games and virtual reality.
Topic
Analysis and innovative application of 2D and 3D digital human technology
With the rapid development of artificial intelligence and computer graphics, 2D and 3D digital human technologies are witnessing unprecedented innovations and breakthroughs. In the field of 2D digital human, Agora utilizes voice-driven technology to achieve natural and smooth mouth synchronization and dynamic expression generation, which significantly improves the expressiveness and realism of the virtual image and injects new vitality into content creation and interactive experience. In terms of 3D virtual digital human, Agora has greatly enhanced its ability to accurately drive virtual Avatars by constructing a real-time mapping model from audio to expression, providing a high-quality solution for real-time interactive scenarios, which is widely used in the fields of virtual anchor, online education, and so on. In addition, in the field of 3D realistic digital human, Agora has successfully realized the generation of multi-view and high fidelity virtual image with the help of 3D Gaussian Splatting technology, which further promotes the in-depth application of digital human in the fields of film and television production, game development and virtual reality. This session will focus on these technological breakthroughs, analyze Agora's key role in technological innovation and its far-reaching impact on the development of the industry, and look forward to the future development trend and potential application scenarios of digital human technology. Outline: 1. Overview of Digital Human Technology: Core Differences and Characteristics of 2D and 3D 2.2D Digital Human: Technical Implementation and Optimization of Speech-Driven Images 3.3D Digital Human: NeRF and 3D GS modeling multi-view generation methods 4.Practical application cases: scene landing and user experience enhancement 5.Technical Challenges and Future Development Direction