Tensorflow Implementation of WaveGlow
- add code for generate the wave on the fly during training for fast evaluate the model quality.
ATTENTION: I have verified that if you use tf.nn.conv2d() for dilated convolution with data format NHWC, the model does not convergence. I have tried at least more that 30 experiments.
Because tf.nn.conv2d() with data format NHWC does not convergence, so in master branch I changed the dilated convolution implementation to implementation in tensorflow-wavenet.
In my experiment with tf.nn.conv2d() by data format NHWC, even after 652K steps the model still did not convergence. See example in ./samples/tf_conv2d_as_dilated_conv/
I spent lots of time to investigate what's wrong with my code by using tf.nn.conv2d with data format NHWC, after the private implemented dilated convolution convergences as expected, so I doubt there maybe a bug in Tensorflow's implementation for dilated convolution.
And, I tried using tf.nn.conv2d() by data format NCHW, then the model convergences quickly as expected, see example in samples/tf_conv2d_NCHW, so there is a bug in Tensorflow's dilated convolution with data format NHWC.
- tf.nn.conv2d() with data format NHWC implementation is in branch
tf_dilated_conv - tf.nn.conv2d() with data format NCHW implementation is in branch
tf_dilated_conv_channel_first
Samples are in folder samples.
process data by preprocess_data.py, following the command:
python preprocess_data.py --wave_dir=corpus\wavs --mel_dir=corpus\mels --data_dir=corpus
python train.py --filelist=xxx --wave_dir=xxx --lc_dir=xxx
model parameters are in file params.py
In my first implementation of WaveGlow, I used tf.nn.conv2d to do dilated convolutions, the 3D Tensor(B*T*depth) is reshaped to 4D Tensor (B*1*T*depth), and then leverage tf.nn.conv2d to do dilated convolution, but after many experiments I found that tf.nn.conv2d with dilated convolution does not convergence as expected. For a long time, I have suspected that there maybe a bug in my implementation.
- with a learning_rate=1e-4, the model does not convergence even after 652K steps.
- with a learning_rate=1e-3, the model does not convergence even after 552K steps.
Example waves by tf.nn.conv2d are in samples/tf_conv2d_as_dilated_conv
In implementation b04901014/waveglow-tensorflow, the author also used tf.nn.conv2d for dilated convolution, this code convergence but very very slow. So there maybe something wrong in my usage.
VERIFIED: tf.nn.conv2d with data format NCHW convergences, but NHWC does not convergence.
tf.nn.conv2d() for dilated convolution did not convergence as expected in my experiments, so I changed the dilated convolution to implementation from tensorflow-wavenet.