Inconsistent output from onnx and jit

[jjjjohnson]

Hi @snakers4

I tried both the lang_classifier_95.onnx and lang_classifier_95.jit and found when fed with the same input, the outputs are different(with large enough margin). Based on the name, I guess they are exported from the same pytorch model. Why are the outcome different? Please help!

Thanks!
Junjie

[snakers4]

Hi,

Can you please provide a more concrete example, i.e. audio you used + code + the results?

[jjjjohnson]

Sure!

def read_audio(path: str,
               target_sr: int = 16000):

    assert torchaudio.get_audio_backend() == 'soundfile'
    wav, sr = torchaudio.load(path)

    if wav.size(0) > 1:
        wav = wav.mean(dim=0, keepdim=True)

    if sr != target_sr:
        transform = torchaudio.transforms.Resample(orig_freq=sr,
                                                   new_freq=target_sr)
        wav = transform(wav)
        sr = target_sr

    assert sr == target_sr
    return wav.squeeze(0)

def init_jit_model(model_path: str,
                   device=torch.device('cpu')):
    torch.set_grad_enabled(False)
    model = torch.jit.load(model_path, map_location=device)
    model.eval()
    return model

jit_model = init_jit_model('files/lang_classifier_95.jit')
jit_model.eval()

file = '8137_common_voice_th_23779126.wav'
wav = read_audio(file)


with torch.no_grad():
    out = jit_model(wav.unsqueeze(0))

print(out[0].sum()) # output : tensor(-1983.2550)

import onnxruntime
onnx_model = onnx.load('files/lang_classifier_95.onnx')
session = onnxruntime.InferenceSession(onnx_model)
output = session.get_outputs()[0].name
lang_out = session.run([output], {'input': wav.unsqueeze(0).numpy()})
print(lang_out[0].sum()) # output : -1783.928

Audio file can be found here

[snakers4]

PyTorch

ONNX

The results look pretty close to me
I am not the biggest expert, I have heard people and my friend speak Thai in Thailand, the audio sounds tonal and resembles Thai

[jjjjohnson]

Yeah! The language probability is similar, but the logits are different.
,which is quite wired...

[snakers4]

Logits are passed through softmax
So if the result is the same, then probably investigating the particular reason is too much hassle - nothing has to be gained from it, and probably some normalization layer just works a bit differently (models had to be tweaked a bit)