Skip to content

Latest commit

 

History

History
 
 

gmw-eng

README.md

opus-2020-06-27.zip

  • dataset: opus
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-06-27.zip
  • test set translations: opus-2020-06-27.test.txt
  • test set scores: opus-2020-06-27.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 26.2 0.535
news-test2008-deueng.deu.eng 25.1 0.528
newstest2009-deueng.deu.eng 24.4 0.524
newstest2010-deueng.deu.eng 27.4 0.560
newstest2011-deueng.deu.eng 24.7 0.532
newstest2012-deueng.deu.eng 26.0 0.542
newstest2013-deueng.deu.eng 28.9 0.559
newstest2014-deen-deueng.deu.eng 29.5 0.566
newstest2015-ende-deueng.deu.eng 30.6 0.570
newstest2016-ende-deueng.deu.eng 35.7 0.614
newstest2017-ende-deueng.deu.eng 31.5 0.578
newstest2018-ende-deueng.deu.eng 38.7 0.631
newstest2019-deen-deueng.deu.eng 35.1 0.602
Tatoeba-test.afr.eng 46.8 0.633
Tatoeba-test.afr-eng.afr.eng 61.1 0.744
Tatoeba-test.ang-eng.ang.eng 6.4 0.156
Tatoeba-test.deu-eng.deu.eng 47.1 0.648
Tatoeba-test.enm-eng.enm.eng 15.8 0.431
Tatoeba-test.frr-eng.frr.eng 14.1 0.224
Tatoeba-test.fry-eng.fry.eng 24.6 0.457
Tatoeba-test.gos-eng.gos.eng 13.4 0.301
Tatoeba-test.gsw-eng.gsw.eng 12.5 0.296
Tatoeba-test.ksh-eng.ksh.eng 8.2 0.242
Tatoeba-test.ltz-eng.ltz.eng 28.6 0.438
Tatoeba-test.multi.eng 46.8 0.633
Tatoeba-test.nds-eng.nds.eng 29.7 0.490
Tatoeba-test.nld-eng.nld.eng 56.9 0.720
Tatoeba-test.pdc-eng.pdc.eng 26.5 0.422
Tatoeba-test.sco-eng.sco.eng 31.3 0.514
Tatoeba-test.stq-eng.stq.eng 20.8 0.406
Tatoeba-test.swg-eng.swg.eng 12.2 0.294
Tatoeba-test.yid-eng.yid.eng 14.3 0.348

opus-2020-07-26.zip

  • dataset: opus
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-07-26.zip
  • test set translations: opus-2020-07-26.test.txt
  • test set scores: opus-2020-07-26.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 26.7 0.536
news-test2008-deueng.deu.eng 25.2 0.529
newstest2009-deueng.deu.eng 24.5 0.525
newstest2010-deueng.deu.eng 27.2 0.559
newstest2011-deueng.deu.eng 24.9 0.534
newstest2012-deueng.deu.eng 25.9 0.543
newstest2013-deueng.deu.eng 28.8 0.558
newstest2014-deen-deueng.deu.eng 29.5 0.565
newstest2015-ende-deueng.deu.eng 30.6 0.570
newstest2016-ende-deueng.deu.eng 36.1 0.617
newstest2017-ende-deueng.deu.eng 31.7 0.578
newstest2018-ende-deueng.deu.eng 38.8 0.630
newstest2019-deen-deueng.deu.eng 34.9 0.601
Tatoeba-test.afr-eng.afr.eng 61.3 0.748
Tatoeba-test.ang-eng.ang.eng 6.4 0.203
Tatoeba-test.deu-eng.deu.eng 47.5 0.649
Tatoeba-test.enm-eng.enm.eng 15.9 0.426
Tatoeba-test.frr-eng.frr.eng 6.1 0.073
Tatoeba-test.fry-eng.fry.eng 25.5 0.468
Tatoeba-test.gos-eng.gos.eng 12.5 0.308
Tatoeba-test.gsw-eng.gsw.eng 8.5 0.279
Tatoeba-test.ksh-eng.ksh.eng 8.4 0.261
Tatoeba-test.ltz-eng.ltz.eng 27.3 0.437
Tatoeba-test.multi.eng 47.5 0.637
Tatoeba-test.nds-eng.nds.eng 30.1 0.494
Tatoeba-test.nld-eng.nld.eng 57.5 0.723
Tatoeba-test.pdc-eng.pdc.eng 28.9 0.427
Tatoeba-test.sco-eng.sco.eng 45.3 0.602
Tatoeba-test.stq-eng.stq.eng 20.1 0.385
Tatoeba-test.swg-eng.swg.eng 13.8 0.284
Tatoeba-test.yid-eng.yid.eng 14.2 0.341

opus2m-2020-08-01.zip

  • dataset: opus2m
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus2m-2020-08-01.zip
  • test set translations: opus2m-2020-08-01.test.txt
  • test set scores: opus2m-2020-08-01.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 27.2 0.538
news-test2008-deueng.deu.eng 25.7 0.534
newstest2009-deueng.deu.eng 25.1 0.530
newstest2010-deueng.deu.eng 27.9 0.565
newstest2011-deueng.deu.eng 25.3 0.539
newstest2012-deueng.deu.eng 26.6 0.548
newstest2013-deueng.deu.eng 29.6 0.565
newstest2014-deen-deueng.deu.eng 30.2 0.571
newstest2015-ende-deueng.deu.eng 31.5 0.577
newstest2016-ende-deueng.deu.eng 36.7 0.622
newstest2017-ende-deueng.deu.eng 32.3 0.585
newstest2018-ende-deueng.deu.eng 39.9 0.638
newstest2019-deen-deueng.deu.eng 35.9 0.611
Tatoeba-test.afr-eng.afr.eng 61.8 0.750
Tatoeba-test.ang-eng.ang.eng 7.3 0.220
Tatoeba-test.deu-eng.deu.eng 48.3 0.657
Tatoeba-test.enm-eng.enm.eng 16.1 0.423
Tatoeba-test.frr-eng.frr.eng 7.0 0.168
Tatoeba-test.fry-eng.fry.eng 28.6 0.488
Tatoeba-test.gos-eng.gos.eng 15.5 0.326
Tatoeba-test.gsw-eng.gsw.eng 12.7 0.308
Tatoeba-test.ksh-eng.ksh.eng 8.4 0.254
Tatoeba-test.ltz-eng.ltz.eng 28.7 0.453
Tatoeba-test.multi.eng 48.5 0.646
Tatoeba-test.nds-eng.nds.eng 31.4 0.509
Tatoeba-test.nld-eng.nld.eng 58.1 0.728
Tatoeba-test.pdc-eng.pdc.eng 25.1 0.406
Tatoeba-test.sco-eng.sco.eng 40.8 0.570
Tatoeba-test.stq-eng.stq.eng 20.3 0.380
Tatoeba-test.swg-eng.swg.eng 20.5 0.315
Tatoeba-test.yid-eng.yid.eng 16.0 0.366

opus4m-2020-08-12.zip

  • dataset: opus4m
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus4m-2020-08-12.zip
  • test set translations: opus4m-2020-08-12.test.txt
  • test set scores: opus4m-2020-08-12.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 27.4 0.541
news-test2008-deueng.deu.eng 25.8 0.534
newstest2009-deueng.deu.eng 25.5 0.533
newstest2010-deueng.deu.eng 28.5 0.569
newstest2011-deueng.deu.eng 25.8 0.542
newstest2012-deueng.deu.eng 27.0 0.552
newstest2013-deueng.deu.eng 30.0 0.568
newstest2014-deen-deueng.deu.eng 30.6 0.574
newstest2015-ende-deueng.deu.eng 31.5 0.577
newstest2016-ende-deueng.deu.eng 37.0 0.626
newstest2017-ende-deueng.deu.eng 32.8 0.588
newstest2018-ende-deueng.deu.eng 40.4 0.643
newstest2019-deen-deueng.deu.eng 36.6 0.615
Tatoeba-test.afr-eng.afr.eng 61.1 0.747
Tatoeba-test.ang-eng.ang.eng 7.4 0.217
Tatoeba-test.deu-eng.deu.eng 49.0 0.661
Tatoeba-test.enm-eng.enm.eng 16.3 0.430
Tatoeba-test.frr-eng.frr.eng 16.8 0.255
Tatoeba-test.fry-eng.fry.eng 28.5 0.482
Tatoeba-test.gos-eng.gos.eng 16.1 0.329
Tatoeba-test.gsw-eng.gsw.eng 11.8 0.289
Tatoeba-test.ksh-eng.ksh.eng 7.5 0.236
Tatoeba-test.ltz-eng.ltz.eng 29.1 0.443
Tatoeba-test.multi.eng 49.4 0.653
Tatoeba-test.nds-eng.nds.eng 32.5 0.522
Tatoeba-test.nld-eng.nld.eng 59.1 0.734
Tatoeba-test.pdc-eng.pdc.eng 29.3 0.438
Tatoeba-test.sco-eng.sco.eng 39.3 0.572
Tatoeba-test.stq-eng.stq.eng 21.4 0.405
Tatoeba-test.swg-eng.swg.eng 15.8 0.321
Tatoeba-test.yid-eng.yid.eng 17.8 0.378