Skip to content

Latest commit

 

History

History
 
 

roa-eng

README.md

opus-2020-06-28.zip

  • dataset: opus
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn glg ita lad lad_Latn lij lld_Latn lmo mwl oci osp_Latn pms por roh ron scn spa vec wln
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-06-28.zip
  • test set translations: opus-2020-06-28.test.txt
  • test set scores: opus-2020-06-28.eval.txt

Benchmarks

testset BLEU chr-F
Tatoeba-test.arg-eng.arg.eng 44.4 0.532
Tatoeba-test.ast-eng.ast.eng 34.3 0.515
Tatoeba-test.cat-eng.cat.eng 54.6 0.704
Tatoeba-test.cos-eng.cos.eng 53.7 0.670
Tatoeba-test.egl-eng.egl.eng 1.7 0.148
Tatoeba-test.ext-eng.ext.eng 40.2 0.576
Tatoeba-test.fra-eng.fra.eng 54.3 0.694
Tatoeba-test.frm-eng.frm.eng 27.1 0.448
Tatoeba-test.fvr-eng.fvr.eng 37.9 0.554
Tatoeba-test.glg-eng.glg.eng 54.8 0.702
Tatoeba-test.ita-eng.ita.eng 65.6 0.773
Tatoeba-test.lad-eng.lad.eng 11.0 0.376
Tatoeba-test.lij-eng.lij.eng 10.1 0.292
Tatoeba-test.lld-eng.lld.eng 13.4 0.308
Tatoeba-test.lmo-eng.lmo.eng 14.7 0.310
Tatoeba-test.multi.eng 57.3 0.715
Tatoeba-test.mwl-eng.mwl.eng 33.1 0.597
Tatoeba-test.oci-eng.oci.eng 18.8 0.393
Tatoeba-test.osp-eng.osp.eng 51.4 0.699
Tatoeba-test.pms-eng.pms.eng 11.9 0.324
Tatoeba-test.por-eng.por.eng 59.9 0.739
Tatoeba-test.roh-eng.roh.eng 17.3 0.397
Tatoeba-test.ron-eng.ron.eng 54.8 0.701
Tatoeba-test.scn-eng.scn.eng 47.5 0.421
Tatoeba-test.spa-eng.spa.eng 56.6 0.716
Tatoeba-test.vec-eng.vec.eng 16.4 0.334
Tatoeba-test.wln-eng.wln.eng 14.9 0.296

opus-2020-07-27.zip

  • dataset: opus
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn gcf_Latn glg hat ind ita lad lad_Latn lij lld_Latn lmo max_Latn mfe min mwl oci pap pms por roh ron scn spa tmw_Latn vec wln zlm_Latn zsm_Latn
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-07-27.zip
  • test set translations: opus-2020-07-27.test.txt
  • test set scores: opus-2020-07-27.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2016-enro-roneng.ron.eng 36.3 0.626
newsdiscussdev2015-enfr-fraeng.fra.eng 30.9 0.560
newsdiscusstest2015-enfr-fraeng.fra.eng 35.5 0.589
newssyscomb2009-fraeng.fra.eng 29.2 0.561
newssyscomb2009-itaeng.ita.eng 32.5 0.583
newssyscomb2009-spaeng.spa.eng 28.9 0.560
news-test2008-fraeng.fra.eng 25.1 0.531
news-test2008-spaeng.spa.eng 26.2 0.538
newstest2009-fraeng.fra.eng 28.2 0.553
newstest2009-itaeng.ita.eng 31.5 0.578
newstest2009-spaeng.spa.eng 28.6 0.557
newstest2010-fraeng.fra.eng 29.5 0.569
newstest2010-spaeng.spa.eng 32.9 0.593
newstest2011-fraeng.fra.eng 30.8 0.582
newstest2011-spaeng.spa.eng 31.8 0.582
newstest2012-fraeng.fra.eng 31.2 0.577
newstest2012-spaeng.spa.eng 34.8 0.604
newstest2013-fraeng.fra.eng 31.5 0.571
newstest2013-spaeng.spa.eng 32.5 0.589
newstest2014-fren-fraeng.fra.eng 34.0 0.606
newstest2016-enro-roneng.ron.eng 35.0 0.609
Tatoeba-test.arg-eng.arg.eng 45.7 0.549
Tatoeba-test.ast-eng.ast.eng 33.7 0.506
Tatoeba-test.cat-eng.cat.eng 53.7 0.696
Tatoeba-test.cos-eng.cos.eng 59.9 0.641
Tatoeba-test.egl-eng.egl.eng 3.2 0.184
Tatoeba-test.ext-eng.ext.eng 32.9 0.481
Tatoeba-test.fra-eng.fra.eng 53.3 0.687
Tatoeba-test.frm-eng.frm.eng 24.0 0.405
Tatoeba-test.gcf-eng.gcf.eng 14.3 0.287
Tatoeba-test.glg-eng.glg.eng 55.4 0.711
Tatoeba-test.hat-eng.hat.eng 35.1 0.523
Tatoeba-test.ita-eng.ita.eng 64.5 0.765
Tatoeba-test.lad-eng.lad.eng 13.4 0.413
Tatoeba-test.lij-eng.lij.eng 10.2 0.288
Tatoeba-test.lld-eng.lld.eng 19.3 0.336
Tatoeba-test.lmo-eng.lmo.eng 7.4 0.305
Tatoeba-test.mfe-eng.mfe.eng 63.1 0.805
Tatoeba-test.msa-eng.msa.eng 40.5 0.588
Tatoeba-test.multi.eng 54.5 0.694
Tatoeba-test.mwl-eng.mwl.eng 24.3 0.523
Tatoeba-test.oci-eng.oci.eng 20.8 0.408
Tatoeba-test.pap-eng.pap.eng 55.7 0.675
Tatoeba-test.pms-eng.pms.eng 11.6 0.322
Tatoeba-test.por-eng.por.eng 59.3 0.735
Tatoeba-test.roh-eng.roh.eng 15.2 0.402
Tatoeba-test.ron-eng.ron.eng 54.2 0.696
Tatoeba-test.scn-eng.scn.eng 40.0 0.396
Tatoeba-test.spa-eng.spa.eng 56.1 0.713
Tatoeba-test.vec-eng.vec.eng 12.7 0.316
Tatoeba-test.wln-eng.wln.eng 13.3 0.293

opus2m-2020-08-01.zip

  • dataset: opus2m
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn gcf_Latn glg hat ind ita lad lad_Latn lij lld_Latn lmo max_Latn mfe min mwl oci pap pms por roh ron scn spa tmw_Latn vec wln zlm_Latn zsm_Latn
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus2m-2020-08-01.zip
  • test set translations: opus2m-2020-08-01.test.txt
  • test set scores: opus2m-2020-08-01.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2016-enro-roneng.ron.eng 37.1 0.631
newsdiscussdev2015-enfr-fraeng.fra.eng 31.6 0.564
newsdiscusstest2015-enfr-fraeng.fra.eng 36.1 0.592
newssyscomb2009-fraeng.fra.eng 29.3 0.563
newssyscomb2009-itaeng.ita.eng 33.1 0.589
newssyscomb2009-spaeng.spa.eng 29.2 0.562
news-test2008-fraeng.fra.eng 25.2 0.533
news-test2008-spaeng.spa.eng 26.6 0.542
newstest2009-fraeng.fra.eng 28.6 0.557
newstest2009-itaeng.ita.eng 32.0 0.580
newstest2009-spaeng.spa.eng 28.9 0.559
newstest2010-fraeng.fra.eng 29.9 0.573
newstest2010-spaeng.spa.eng 33.3 0.596
newstest2011-fraeng.fra.eng 31.2 0.585
newstest2011-spaeng.spa.eng 32.3 0.584
newstest2012-fraeng.fra.eng 31.3 0.580
newstest2012-spaeng.spa.eng 35.3 0.606
newstest2013-fraeng.fra.eng 31.9 0.575
newstest2013-spaeng.spa.eng 32.8 0.592
newstest2014-fren-fraeng.fra.eng 34.6 0.611
newstest2016-enro-roneng.ron.eng 35.8 0.614
Tatoeba-test.arg-eng.arg.eng 38.7 0.512
Tatoeba-test.ast-eng.ast.eng 35.2 0.520
Tatoeba-test.cat-eng.cat.eng 54.9 0.703
Tatoeba-test.cos-eng.cos.eng 68.1 0.666
Tatoeba-test.egl-eng.egl.eng 6.7 0.209
Tatoeba-test.ext-eng.ext.eng 24.2 0.427
Tatoeba-test.fra-eng.fra.eng 53.9 0.691
Tatoeba-test.frm-eng.frm.eng 25.7 0.423
Tatoeba-test.gcf-eng.gcf.eng 14.8 0.288
Tatoeba-test.glg-eng.glg.eng 54.6 0.703
Tatoeba-test.hat-eng.hat.eng 37.0 0.540
Tatoeba-test.ita-eng.ita.eng 64.8 0.768
Tatoeba-test.lad-eng.lad.eng 21.7 0.452
Tatoeba-test.lij-eng.lij.eng 11.2 0.299
Tatoeba-test.lld-eng.lld.eng 10.8 0.273
Tatoeba-test.lmo-eng.lmo.eng 5.8 0.260
Tatoeba-test.mfe-eng.mfe.eng 63.1 0.819
Tatoeba-test.msa-eng.msa.eng 40.9 0.592
Tatoeba-test.multi.eng 54.9 0.697
Tatoeba-test.mwl-eng.mwl.eng 44.6 0.674
Tatoeba-test.oci-eng.oci.eng 20.5 0.404
Tatoeba-test.pap-eng.pap.eng 56.2 0.669
Tatoeba-test.pms-eng.pms.eng 10.3 0.324
Tatoeba-test.por-eng.por.eng 59.7 0.738
Tatoeba-test.roh-eng.roh.eng 14.8 0.378
Tatoeba-test.ron-eng.ron.eng 55.2 0.703
Tatoeba-test.scn-eng.scn.eng 10.2 0.259
Tatoeba-test.spa-eng.spa.eng 56.2 0.714
Tatoeba-test.vec-eng.vec.eng 13.8 0.317
Tatoeba-test.wln-eng.wln.eng 17.3 0.323

opus4m-2020-08-12.zip

  • dataset: opus4m
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn gcf_Latn glg hat ind ita lad lad_Latn lij lld_Latn lmo max_Latn mfe min mwl oci pap pms por roh ron scn spa tmw_Latn vec wln zlm_Latn zsm_Latn
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus4m-2020-08-12.zip
  • test set translations: opus4m-2020-08-12.test.txt
  • test set scores: opus4m-2020-08-12.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2016-enro-roneng.ron.eng 37.2 0.632
newsdiscussdev2015-enfr-fraeng.fra.eng 31.8 0.566
newsdiscusstest2015-enfr-fraeng.fra.eng 36.2 0.595
newssyscomb2009-fraeng.fra.eng 29.5 0.564
newssyscomb2009-itaeng.ita.eng 32.1 0.584
newssyscomb2009-spaeng.spa.eng 28.6 0.560
news-test2008-fraeng.fra.eng 25.3 0.533
news-test2008-spaeng.spa.eng 26.4 0.541
newstest2009-fraeng.fra.eng 28.7 0.557
newstest2009-itaeng.ita.eng 31.9 0.581
newstest2009-spaeng.spa.eng 29.1 0.561
newstest2010-fraeng.fra.eng 30.1 0.575
newstest2010-spaeng.spa.eng 33.3 0.595
newstest2011-fraeng.fra.eng 31.2 0.585
newstest2011-spaeng.spa.eng 32.2 0.584
newstest2012-fraeng.fra.eng 31.3 0.579
newstest2012-spaeng.spa.eng 35.1 0.605
newstest2013-fraeng.fra.eng 32.0 0.576
newstest2013-spaeng.spa.eng 32.9 0.591
newstest2014-fren-fraeng.fra.eng 34.6 0.610
newstest2016-enro-roneng.ron.eng 35.9 0.614
Tatoeba-test.arg-eng.arg.eng 44.6 0.535
Tatoeba-test.ast-eng.ast.eng 34.9 0.527
Tatoeba-test.cat-eng.cat.eng 55.1 0.708
Tatoeba-test.cos-eng.cos.eng 63.2 0.651
Tatoeba-test.egl-eng.egl.eng 5.6 0.212
Tatoeba-test.ext-eng.ext.eng 29.3 0.468
Tatoeba-test.fra-eng.fra.eng 53.9 0.692
Tatoeba-test.frm-eng.frm.eng 23.6 0.405
Tatoeba-test.gcf-eng.gcf.eng 18.0 0.316
Tatoeba-test.glg-eng.glg.eng 55.4 0.712
Tatoeba-test.hat-eng.hat.eng 37.3 0.553
Tatoeba-test.ita-eng.ita.eng 65.4 0.772
Tatoeba-test.lad-eng.lad.eng 14.0 0.441
Tatoeba-test.lij-eng.lij.eng 9.2 0.282
Tatoeba-test.lld-eng.lld.eng 8.7 0.267
Tatoeba-test.lmo-eng.lmo.eng 10.8 0.287
Tatoeba-test.mfe-eng.mfe.eng 66.0 0.805
Tatoeba-test.msa-eng.msa.eng 41.1 0.595
Tatoeba-test.multi.eng 55.1 0.699
Tatoeba-test.mwl-eng.mwl.eng 22.3 0.393
Tatoeba-test.oci-eng.oci.eng 21.7 0.410
Tatoeba-test.pap-eng.pap.eng 56.6 0.672
Tatoeba-test.pms-eng.pms.eng 12.9 0.335
Tatoeba-test.por-eng.por.eng 59.8 0.738
Tatoeba-test.roh-eng.roh.eng 19.4 0.429
Tatoeba-test.ron-eng.ron.eng 55.4 0.704
Tatoeba-test.scn-eng.scn.eng 35.8 0.388
Tatoeba-test.spa-eng.spa.eng 56.4 0.716
Tatoeba-test.vec-eng.vec.eng 13.6 0.343
Tatoeba-test.wln-eng.wln.eng 13.4 0.315