Skip to content

Latest commit

 

History

History
135 lines (132 loc) · 22.3 KB

higher.md

File metadata and controls

135 lines (132 loc) · 22.3 KB

Tatoeba Challenge Data

This is the "higher" sub-set of the Tatoeba data. Download the data files from the link in the table below. There is a total of

  • 125 language pairs in this sub-set
lang-pair test dev train
Afrikaans - German afr-deu 1583 1268185
Afrikaans - English afr-eng 1374 1000 1933346
Afrikaans - Dutch afr-nld 1056 1318141
Afrikaans - Russian afr-rus 228 1016505
Afrikaans - Spanish afr-spa 448 1275543
Arabic - Japanese ara-jpn 1336 2968884
Bengali - English ben-eng 2500 2637 2681705
Bulgarian - Japanese bul-jpn 319 2519065
Bulgarian - Ukrainian bul-ukr 1020 1235705
Catalan - Portuguese cat-por 729 1164333
Catalan - Spanish cat-spa 1534 1000 7953020
Cebuano - English ceb-eng 378 1014362
Czech - Ukrainian ces-ukr 1787 1000 1637809
Welsh - English cym-eng 818 1285302
Danish - Japanese dan-jpn 896 3052507
Danish - Norwegian dan-nor 1311 1000 8638653
Danish - Russian dan-rus 1713 1000 8108664
German - Persian deu-fas 3185 1000 4150841
German - Icelandic deu-isl 968 2009703
German - Japanese deu-jpn 10000 28315 3641010
German - Korean deu-kor 1103 2788640
German - Malay (macrolanguage) deu-msa 535 6808168
German - Norwegian deu-nor 3651 1000 7603899
German - Tagalog deu-tgl 326 1188925
German - Ukrainian deu-ukr 10000 11608 1661237
German - Vietnamese deu-vie 400 3114765
German - Chinese deu-zho 2500 6732 7182427
English - Basque eng-eus 1060 1000 3584276
English - Persian eng-fas 3768 1000 8278501
English - Irish eng-gle 1924 1661054
English - Hindi eng-hin 5000 5821 1502665
English - Icelandic eng-isl 2500 6938 2744745
English - Japanese eng-jpn 10000 193553 4701376
English - Korean eng-kor 2414 1000 3792612
English - Malayalam eng-mal 802 1148395
English - Macedonian eng-mkd 10000 70284 3664467
English - Maltese eng-mlt 203 7095316
English - Albanian eng-sqi 1065 3635633
English - Swahili (macrolanguage) eng-swa 386 1674553
English - Tamil eng-tam 311 1063288
English - Tagalog eng-tgl 2500 4753 1273814
English - Thai eng-tha 1145 4434237
English - Ukrainian eng-ukr 10000 157764 2201978
English - Urdu eng-urd 1663 1138043
English - Vietnamese eng-vie 2500 2824 4574505
Estonian - Russian est-rus 598 7112388
Basque - Spanish eus-spa 1850 1000 2151765
Persian - French fas-fra 375 4600456
Finnish - Japanese fin-jpn 10000 8417 3344849
Finnish - Korean fin-kor 428 2561825
Finnish - Norwegian fin-nor 2489 1000 9258628
Finnish - Chinese fin-zho 380 8657366
French - Japanese fra-jpn 10000 28518 3762983
French - Korean fra-kor 320 2962264
French - Malay (macrolanguage) fra-msa 998 8254972
French - Norwegian fra-nor 476 8305134
French - Tagalog fra-tgl 836 1153203
French - Ukrainian fra-ukr 10000 18306 1701550
French - Vietnamese fra-vie 1299 3537187
Serbo-Croatian - Japanese hbs-jpn 400 5140833
Serbo-Croatian - Ukrainian hbs-ukr 941 2144095
Hebrew - Japanese heb-jpn 242 2831004
Hebrew - Ukrainian heb-ukr 959 1149178
Hebrew - Chinese heb-zho 708 9373060
Hungarian - Japanese hun-jpn 2500 3558 3019958
Hungarian - Korean hun-kor 271 2432159
Hungarian - Ukrainian hun-ukr 464 1503410
Icelandic - Italian isl-ita 236 1580753
Icelandic - Spanish isl-spa 238 2085931
Italian - Japanese ita-jpn 2665 1000 3618212
Italian - Lithuanian ita-lit 224 9365564
Italian - Malay (macrolanguage) ita-msa 426 7572767
Italian - Norwegian ita-nor 931 7875740
Italian - Ukrainian ita-ukr 5000 8658 1683593
Italian - Vietnamese ita-vie 250 3257326
Italian - Chinese ita-zho 2943 1000 7784323
Japanese - Korean jpn-kor 593 2701560
Japanese - Lithuanian jpn-lit 242 1043108
Japanese - Malay (macrolanguage) jpn-msa 2644 1000 2598008
Japanese - Dutch jpn-nld 3443 1000 3643442
Japanese - Norwegian jpn-nor 1046 2704583
Japanese - Polish jpn-pol 10000 14807 3295824
Japanese - Portuguese jpn-por 1943 1000 5542948
Japanese - Russian jpn-rus 10000 15301 2812096
Japanese - Spanish jpn-spa 10000 23383 4182722
Japanese - Swedish jpn-swe 898 3099109
Japanese - Turkish jpn-tur 806 2707742
Japanese - Ukrainian jpn-ukr 436 1397412
Japanese - Vietnamese jpn-vie 1098 1517217
Japanese - Chinese jpn-zho 2500 2829 1839365
Korean - Russian kor-rus 221 2207039
Korean - Spanish kor-spa 942 3306282
Latvian - Russian lav-rus 274 1091421
Lithuanian - Polish lit-pol 1787 9973745
Lithuanian - Russian lit-rus 2500 5155 1698961
Lithuanian - Spanish lit-spa 452 9773648
Lithuanian - Turkish lit-tur 1473 1975209
Macedonian - Spanish mkd-spa 217 3190726
Malay (macrolanguage) - Malay (macrolanguage) msa-msa 870 6944663
Malay (macrolanguage) - Chinese msa-zho 369 6472486
Dutch - Norwegian nld-nor 203 8986340
Dutch - Ukrainian nld-ukr 10000 5001 1676117
Dutch - Chinese nld-zho 1654 9796675
Norwegian - Norwegian nor-nor 948 1442077
Norwegian - Polish nor-pol 281 7909565
Norwegian - Russian nor-rus 1279 5447125
Norwegian - Spanish nor-spa 960 9429990
Norwegian - Swedish nor-swe 562 8144583
Norwegian - Ukrainian nor-ukr 670 1503314
Norwegian - Chinese nor-zho 200 3787972
Polish - Ukrainian pol-ukr 2500 6900 1688128
Portuguese - Tagalog por-tgl 1777 1187078
Portuguese - Ukrainian por-ukr 3372 1000 2350476
Russian - Slovenian rus-slv 378 9539016
Russian - Swedish rus-swe 1282 8978238
Russian - Ukrainian rus-ukr 10000 6815 1525582
Russian - Vietnamese rus-vie 313 3048826
Slovenian - Ukrainian slv-ukr 848 1331119
Slovenian - Chinese slv-zho 825 7586432
Spanish - Tagalog spa-tgl 629 1148187
Spanish - Ukrainian spa-ukr 10000 12924 1891614
Spanish - Vietnamese spa-vie 582 4056729
Swedish - Chinese swe-zho 1050 6995702
Turkish - Ukrainian tur-ukr 2500 6722 1194087
Vietnamese - Chinese vie-zho 438 2697352