Speaker leakage encountered in refactor-v2

[haru0l]

Hello! I am currently training a multispeaker using the refactor-v2 repo and the code that has previously given, and it seems that in the tensorboard samples, timbre leakage has occured with the datasets that has the least amount of data tend to leak more from the larger datasets. This behaviour did not happen in the refactor repo, however.

Here are the sample from tensorboard:

2023-05-05_23-18-51.mp4

[haru0l]

(For reference, the gt is a one minute dataset, and the pred seems to have leaked from a larger dataset.)
Settings are as follows:
residual_channels: 512, residual_layers: 20, use_spk_id: True, pl_trainer_precision: 16-mixed

[haru0l]

Sorry! It turns out I accidentally set residual_channels to 384, explaining the issue.