On the latency of voice conversion in an active voice cloning device

Abstract

Although significant progress has been made in voice conversion (VC), the presence of the source speaker’s original voice becomes problematic, especially in a real-time voice cloning scenario where listeners are close to the source speaker. An overlapping of the source speaker’s voice and the converted voice results in a degradation of the sense of immersion to the converted voice. In this paper, we conceptualize an active voice cloning (AVC) device, which can convert one’s voice timbre to another’s while confining the source speaker’s voice with active noise control (ANC). The VC system is realized through a low-latency deep neural network model, and the ANC system is constructed by a feedforward single-channel implementation. The mockup of the AVC device is assembled in a short open tube that can be worn on the source speaker’s mouth. Taking into consideration that the latency in the VC system introduces a phase difference between the converted voice and the residual voice of the source speaker, we further assess its effect on the intelligibility of the converted voice, as well as
the overall performance of the AVC device in ameliorating the perceptual experience.

More information

Identifiers

ORCID for Chuang Shi: orcid.org/0000-0002-1517-2775

Catalogue record

Export record