Skip to content

Latest commit

 

History

History
 
 

iir-eng

README.md

opus-2020-07-04.zip

  • dataset: opus
  • model: transformer
  • source language(s): asm awa ben bho gom guj hif_Latn hin jdt_Cyrl kur_Arab kur_Latn mai mar npi ori oss pan_Guru pes pes_Latn pes_Thaa pnb pus rom san_Deva sin snd_Arab tgk_Cyrl tly_Latn urd zza
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-07-04.zip
  • test set translations: opus-2020-07-04.test.txt
  • test set scores: opus-2020-07-04.eval.txt

Benchmarks

testset BLEU chr-F
Tatoeba-test.asm-eng.asm.eng 10.4 0.308
Tatoeba-test.awa-eng.awa.eng 5.3 0.229
Tatoeba-test.ben-eng.ben.eng 38.7 0.542
Tatoeba-test.bho-eng.bho.eng 24.4 0.431
Tatoeba-test.fas-eng.fas.eng 5.0 0.251
Tatoeba-test.guj-eng.guj.eng 14.3 0.331
Tatoeba-test.hif-eng.hif.eng 1.2 0.212
Tatoeba-test.hin-eng.hin.eng 33.5 0.516
Tatoeba-test.jdt-eng.jdt.eng 9.6 0.080
Tatoeba-test.kok-eng.kok.eng 3.4 0.230
Tatoeba-test.kur-eng.kur.eng 2.9 0.145
Tatoeba-test.lah-eng.lah.eng 18.3 0.315
Tatoeba-test.mai-eng.mai.eng 38.8 0.598
Tatoeba-test.mar-eng.mar.eng 19.5 0.435
Tatoeba-test.multi.eng 18.9 0.400
Tatoeba-test.nep-eng.nep.eng 2.1 0.166
Tatoeba-test.ori-eng.ori.eng 2.8 0.193
Tatoeba-test.oss-eng.oss.eng 1.5 0.182
Tatoeba-test.pan-eng.pan.eng 11.8 0.320
Tatoeba-test.pus-eng.pus.eng 1.5 0.203
Tatoeba-test.rom-eng.rom.eng 2.5 0.179
Tatoeba-test.san-eng.san.eng 2.4 0.168
Tatoeba-test.sin-eng.sin.eng 26.3 0.485
Tatoeba-test.snd-eng.snd.eng 10.5 0.302
Tatoeba-test.tgk-eng.tgk.eng 7.3 0.280
Tatoeba-test.tly-eng.tly.eng 0.8 0.085
Tatoeba-test.urd-eng.urd.eng 21.1 0.409
Tatoeba-test.zza-eng.zza.eng 0.8 0.102

opus-2020-07-26.zip

  • dataset: opus
  • model: transformer
  • source language(s): asm awa ben bho gom guj hif_Latn hin jdt_Cyrl kur_Arab kur_Latn mai mar npi ori oss pan_Guru pes pes_Latn pes_Thaa pnb pus rom san_Deva sin snd_Arab tgk_Cyrl tly_Latn urd zza
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-07-26.zip
  • test set translations: opus-2020-07-26.test.txt
  • test set scores: opus-2020-07-26.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2014-hineng.hin.eng 7.9 0.320
newsdev2019-engu-gujeng.guj.eng 7.3 0.296
newstest2014-hien-hineng.hin.eng 11.1 0.367
newstest2019-guen-gujeng.guj.eng 4.9 0.266
Tatoeba-test.asm-eng.asm.eng 12.8 0.314
Tatoeba-test.awa-eng.awa.eng 7.9 0.214
Tatoeba-test.ben-eng.ben.eng 40.7 0.556
Tatoeba-test.bho-eng.bho.eng 24.1 0.439
Tatoeba-test.fas-eng.fas.eng 4.9 0.254
Tatoeba-test.guj-eng.guj.eng 16.0 0.345
Tatoeba-test.hif-eng.hif.eng 1.1 0.205
Tatoeba-test.hin-eng.hin.eng 34.3 0.525
Tatoeba-test.jdt-eng.jdt.eng 11.4 0.101
Tatoeba-test.kok-eng.kok.eng 6.9 0.157
Tatoeba-test.kur-eng.kur.eng 3.5 0.151
Tatoeba-test.lah-eng.lah.eng 20.0 0.301
Tatoeba-test.mai-eng.mai.eng 66.2 0.715
Tatoeba-test.mar-eng.mar.eng 19.7 0.439
Tatoeba-test.multi.eng 19.2 0.405
Tatoeba-test.nep-eng.nep.eng 1.9 0.159
Tatoeba-test.ori-eng.ori.eng 2.3 0.192
Tatoeba-test.oss-eng.oss.eng 1.9 0.179
Tatoeba-test.pan-eng.pan.eng 12.4 0.331
Tatoeba-test.pus-eng.pus.eng 0.9 0.184
Tatoeba-test.rom-eng.rom.eng 1.4 0.173
Tatoeba-test.san-eng.san.eng 2.2 0.165
Tatoeba-test.sin-eng.sin.eng 27.4 0.497
Tatoeba-test.snd-eng.snd.eng 10.2 0.332
Tatoeba-test.tgk-eng.tgk.eng 9.5 0.270
Tatoeba-test.tly-eng.tly.eng 0.8 0.064
Tatoeba-test.urd-eng.urd.eng 22.2 0.419
Tatoeba-test.zza-eng.zza.eng 0.9 0.101

opus2m-2020-08-01.zip

  • dataset: opus2m
  • model: transformer
  • source language(s): asm awa ben bho gom guj hif_Latn hin jdt_Cyrl kur_Arab kur_Latn mai mar npi ori oss pan_Guru pes pes_Latn pes_Thaa pnb pus rom san_Deva sin snd_Arab tgk_Cyrl tly_Latn urd zza
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus2m-2020-08-01.zip
  • test set translations: opus2m-2020-08-01.test.txt
  • test set scores: opus2m-2020-08-01.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2014-hineng.hin.eng 8.1 0.324
newsdev2019-engu-gujeng.guj.eng 8.1 0.309
newstest2014-hien-hineng.hin.eng 12.1 0.380
newstest2019-guen-gujeng.guj.eng 6.0 0.280
Tatoeba-test.asm-eng.asm.eng 13.9 0.327
Tatoeba-test.awa-eng.awa.eng 7.0 0.219
Tatoeba-test.ben-eng.ben.eng 42.5 0.576
Tatoeba-test.bho-eng.bho.eng 27.3 0.452
Tatoeba-test.fas-eng.fas.eng 5.6 0.262
Tatoeba-test.guj-eng.guj.eng 15.9 0.350
Tatoeba-test.hif-eng.hif.eng 10.1 0.247
Tatoeba-test.hin-eng.hin.eng 36.5 0.544
Tatoeba-test.jdt-eng.jdt.eng 11.4 0.094
Tatoeba-test.kok-eng.kok.eng 6.6 0.256
Tatoeba-test.kur-eng.kur.eng 3.4 0.149
Tatoeba-test.lah-eng.lah.eng 17.4 0.301
Tatoeba-test.mai-eng.mai.eng 65.4 0.703
Tatoeba-test.mar-eng.mar.eng 22.5 0.468
Tatoeba-test.multi.eng 21.3 0.424
Tatoeba-test.nep-eng.nep.eng 3.4 0.185
Tatoeba-test.ori-eng.ori.eng 4.8 0.244
Tatoeba-test.oss-eng.oss.eng 1.6 0.173
Tatoeba-test.pan-eng.pan.eng 14.8 0.348
Tatoeba-test.pus-eng.pus.eng 1.1 0.182
Tatoeba-test.rom-eng.rom.eng 2.8 0.185
Tatoeba-test.san-eng.san.eng 2.8 0.185
Tatoeba-test.sin-eng.sin.eng 22.8 0.474
Tatoeba-test.snd-eng.snd.eng 8.2 0.287
Tatoeba-test.tgk-eng.tgk.eng 11.9 0.321
Tatoeba-test.tly-eng.tly.eng 0.9 0.076
Tatoeba-test.urd-eng.urd.eng 23.9 0.438
Tatoeba-test.zza-eng.zza.eng 0.6 0.098

opus4m-2020-08-12.zip

  • dataset: opus4m
  • model: transformer
  • source language(s): asm awa ben bho gom guj hif_Latn hin jdt_Cyrl kur_Arab kur_Latn mai mar npi ori oss pan_Guru pes pes_Latn pes_Thaa pnb pus rom san_Deva sin snd_Arab tgk_Cyrl tly_Latn urd zza
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus4m-2020-08-12.zip
  • test set translations: opus4m-2020-08-12.test.txt
  • test set scores: opus4m-2020-08-12.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2014-hineng.hin.eng 9.1 0.335
newsdev2019-engu-gujeng.guj.eng 9.3 0.324
newstest2014-hien-hineng.hin.eng 12.9 0.389
newstest2019-guen-gujeng.guj.eng 6.8 0.289
Tatoeba-test.asm-eng.asm.eng 17.2 0.352
Tatoeba-test.awa-eng.awa.eng 7.1 0.244
Tatoeba-test.ben-eng.ben.eng 43.4 0.578
Tatoeba-test.bho-eng.bho.eng 35.0 0.525
Tatoeba-test.fas-eng.fas.eng 6.2 0.267
Tatoeba-test.guj-eng.guj.eng 17.2 0.355
Tatoeba-test.hif-eng.hif.eng 6.0 0.255
Tatoeba-test.hin-eng.hin.eng 37.3 0.551
Tatoeba-test.jdt-eng.jdt.eng 0.0 0.064
Tatoeba-test.kok-eng.kok.eng 6.6 0.156
Tatoeba-test.kur-eng.kur.eng 2.2 0.144
Tatoeba-test.lah-eng.lah.eng 19.4 0.315
Tatoeba-test.mai-eng.mai.eng 38.7 0.622
Tatoeba-test.mar-eng.mar.eng 22.3 0.470
Tatoeba-test.multi.eng 21.4 0.427
Tatoeba-test.nep-eng.nep.eng 4.3 0.195
Tatoeba-test.ori-eng.ori.eng 2.6 0.228
Tatoeba-test.oss-eng.oss.eng 1.7 0.196
Tatoeba-test.pan-eng.pan.eng 17.6 0.370
Tatoeba-test.pus-eng.pus.eng 0.8 0.177
Tatoeba-test.rom-eng.rom.eng 2.1 0.180
Tatoeba-test.san-eng.san.eng 4.1 0.204
Tatoeba-test.sin-eng.sin.eng 26.3 0.477
Tatoeba-test.snd-eng.snd.eng 6.5 0.244
Tatoeba-test.tgk-eng.tgk.eng 9.5 0.324
Tatoeba-test.tly-eng.tly.eng 1.3 0.099
Tatoeba-test.urd-eng.urd.eng 24.7 0.445
Tatoeba-test.zza-eng.zza.eng 0.8 0.102