The embodiment of the invention provides a voice separation method and method and an electronic device. The voice separation method includes the steps of obtaining a mixed voice signal including voicesignals of at least two target objects; acquiring single-channel frequency spectrum characteristics and multi-channel azimuth characteristics of a full-voice frequency band corresponding to the mixedvoice signal, wherein the full-voice frequency band comprises K sub-frequency bands, and K is a positive integer greater than or equal to 2; extracting single-channel frequency spectrum characteristics and multi-channel azimuth characteristics of the K sub-frequency bands from the single-channel frequency spectrum characteristics and multi-channel azimuth characteristics of the full-voice frequency band; processing the single-channel frequency spectrum characteristics and multi-channel azimuth characteristics of the K sub-frequency bands through K first neural networks to obtain K first characteristic vectors; generating merged characteristic vectors according to the K first characteristic vectors; processing the merged characteristic vectors through a first prediction network to obtain afirst voice spectrum mask matrix of each target object in the mixed voice signal.

Voice separation method and method and electronic device

The invention discloses a speaker-independent single-channel voice separation method, which comprises the following steps: preparing a data set, and carrying out data preprocessing; establishing a single-channel voice separation model based on complex ideal floating value masking; when the single-channel voice separation model is trained, adopting statement-level replacement invariance training; and inputting the mixed voice data into the trained model for voice separation. According to the speaker-independent single-channel voice separation method based on statement-level replacement invariance training and complex ideal floating value masking, complex ideal floating value masking estimation is effectively and accurately realized through statement-level replacement invariant training; inthe method, a bidirectional long-short-term memory neural network structure is adopted to estimate complex number ideal floating value masking, and a statement-level replacement invariant training standard is further utilized to solve the problem of label blurring, so that single-channel voice separation has a good effect.

Speaker-independent single-channel voice separation method

The embodiment of the invention provides a voice separation method, a voice identification method and related equipment. The voice separation method comprises the steps of: acquiring a mixed voice signal of voice signals, which comprises at least two target objects; acquiring single-channel spectrum features and multi-channel azimuth features of a full voice frequency band corresponding to the mixed voice signal, wherein the full voice frequency band comprises K sub frequency bands, and K is a positive integer greater than or equal to 2; extracting single-channel spectrum features and multi-channel azimuth features of the K sub frequency bands from the single-channel spectrum features and the multi-channel azimuth features of the full voice frequency band; carrying out processing on the single-channel spectrum features and the multi-channel azimuth features of the K sub frequency bands by K first neural networks so as to obtain K first feature vectors; generating a combined feature vector according to the K first feature vectors; and carrying out processing on the combined feature vector by a first prediction network so as to obtain a first voice spectrum mask matrix of each targetobject in the mixed voice signal.

Voice separation method, voice identification method and related equipment

The invention discloses a voice cloning method and device based on a single-speaker voice synthesis data set, electronic equipment and a computer storage medium, and relates to the technical field ofvoice cloning, and the method comprises the following steps: training a single-speaker voice synthesis model based on the single-speaker voice synthesis data set; training a voiceprint model based ona multi-speaker voice data set; performing calculation on the multi-speaker voice data set through the trained voiceprint model to obtain a voiceprint feature data set, and training a voice conversionmodel based on the voiceprint feature data set; and calculating a target text and a target speaker voice through the trained single speaker voice synthesis model, voiceprint model and voice conversion model to obtain a target speaker text voice. According to the method, only one single-speaker voice synthesis data set is needed, cloning of the target speaker voice is realized, processing of the speech synthesis data is simple and convenient, collecting and processing of a large amount of speech synthesis data of a speaker are not needed, and various costs are greatly reduced.

Voice cloning method and device based on single-speaker voice synthesis data set

The invention discloses an LSTM voice enhancement method based on a psychoacoustic model, and the method comprises the following steps: S1, carrying out the VAD processing of an inputted PCM signal, and judging whether the PCM signal is a voice signal or not; S2, if the signal is a voice signal, performing feature processing on the voice signal to obtain a feature signal; S3, processing the characteristic signal through a psychoacoustic model to obtain an acoustic characteristic signal; S4, processing the acoustic characteristic signal through an LSTM network to obtain an enhanced voice signal; and S5, performing ASR processing on the enhanced voice signal to realize voice recognition. According to the scheme, the noisy signals are processed through the psychoacoustic model, redundant components are eliminated, and the data processing amount of the network is reduced; meanwhile, compared with a full-connection voice enhancement network, the LSTM network adopted by the scheme of the invention can be in contact with the context information, so that the voice enhancement performance is further improved, and the voice recognition rate is improved.

LSTM voice enhancement method based on psychoacoustic model

The application provides a voice enhancing method, device, intelligent voice equipment and computer equipment; the method comprises the following steps: obtaining a to-be-processed voice signal; inputting the voice signal into a voice enhancing model, removing noises and/or interference voices in the voice signal so as to obtain a processed voice signal; the voice enhancing model is obtained by training mixed voice signals; the mixed voice signals refer to signals obtained by adding noises and/or interference voices in pure wakeup word voice signals; the pure wakeup word voice signals refer towakeup word voice signals with a noise and/or interference voice ratio smaller than a proportion threshold; the voice enhancing model can effectively remove noises and/or interference voices from thevoice signal, such as voices related to non-wakeup words, thus improving the voice enhancing effect; the method and device can carry out voice identification treatment for the processed voice signal,thus improving wakeup word identification accuracy and wake up efficiency, and improving the user usage experiences.

Voice enhancing method, device, intelligent voice equipment and computer equipment

The invention discloses a voice recognition method and device, computer equipment and a storage medium, and belongs to the field of data processing. The method comprises the steps of: inputting collected audio data are into a time domain separation model, performing prediction by the time domain separation model based on the audio data to obtain time domain separation information, and using the time domain separation information for separating noise data and voice data in the audio data; performing voice separation on the audio data based on the time domain separation information to obtain time domain voice data; performing feature extraction on the time domain voice data to obtain time domain voice feature information corresponding to the time domain voice data; and performing voice recognition on the time domain voice feature information corresponding to the time domain voice data, and determining the voice content corresponding to the time domain voice data. Through the voice recognition method provided by the invention, the computer equipment can complete the whole voice recognition processes in the time domain without converting the audio information in the time domain to thefrequency domain and then carrying out voice separation, thereby improving the speed of voice recognition.

Voice recognition method and device, computer equipment and storage medium

The invention relates to an inter-channel feature extraction method of a multi-channel multi-sound-source mixed audio signal, an audio separation method and device, computing equipment, a computer readable storage medium and a multi-sound-source audio separation system. The audio separation method comprises the following steps: transforming one of a plurality of channel components of the multi-channel multi-sound-source mixed audio signal into a single-channel multi-sound-source mixed audio feature in a feature space; performing two-dimensional cavity convolution on the multi-channel multi-sound-source mixed audio signal so as to extract multiple inter-channel features; performing feature fusion on the single-channel multi-sound-source mixed audio feature and the multiple inter-channel features; estimating respective weights of a plurality of sound sources in the single-channel multi-sound-source mixed audio feature on the basis of a fused multi-channel multi-sound-source mixed audio feature; according to the single-channel multi-sound-source mixed audio feature and the respective weights, obtaining respective features of the plurality of sound sources; and transforming the respective features of the plurality of sound sources into respective audio signals of the plurality of sound sources.

Inter-channel feature extraction method, audio separation method and device, and computing equipment

The embodiment of the invention provides a voice separation method, a voice recognition method and related equipment. The voice separation method comprises the following steps: obtaining a hybrid voice signal comprising the voice signals of at least two target objects; obtaining a single-channel spectrum feature and a multi-channel orientation feature corresponding to the hybrid voice signal; carrying out processing on the single-channel spectrum feature and the multi-channel orientation feature by an overlapping judgment model, and obtaining a judgment result whether overlapping exists between the target objects in the hybrid voice signal, wherein the overlapping judgment model is used for judging whether spatial overlapping exists between the target objects; determining a target voice spectrum mask matrix for obtaining each target object in the hybrid voice signal by using a single-channel separation network or a multi-channel separation network according to the judgment result.

Su Dan

Papers

Voice separation method and method and electronic device

Voice enhancing method, device, intelligent voice equipment and computer equipment

Voice recognition method and device, computer equipment and storage medium

Inter-channel feature extraction method, audio separation method and device, and computing equipment

Voice separation method, voice recognition method and related equipment