Skip to content

Commit

Permalink
generate downloads table on release (#96)
Browse files Browse the repository at this point in the history
* downloads file

* name release with calver

* wip

* fix typo

* iterate faster

* try 2

* try 3

* try chmod

* try temp.md

* add preamble

* add downloads section link

* typo

* remove testing langauges

* remove temp test prefix
  • Loading branch information
StefanVukovic99 authored Jul 1, 2024
1 parent e522777 commit 43b882c
Show file tree
Hide file tree
Showing 6 changed files with 109 additions and 14 deletions.
31 changes: 30 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,44 @@ jobs:
run: |
# See https://docs.github.com/en/get-started/using-git/dealing-with-special-characters-in-branch-and-tag-names
TAG=v$(date -Iseconds | sed 's/[T:\+]/-/g')
CALVER=$(date +'%y.%m.%d.%H')
echo "$TAG"
echo "tag=$TAG" >> $GITHUB_OUTPUT
echo "calver=$CALVER" >> $GITHUB_OUTPUT
git tag -a $TAG -m "Published version $TAG" ${GITHUB_SHA}
git push origin $TAG
- name: Generate markdown table
id: generate_markdown
run: |
./tools/generate-downloads-table.sh > table.md
cat table.md
- name: Write to downloads.md
id: write_downloads_md
shell: bash
run: |
branch="update-downloads-md-$(date +%s)"
git checkout -b "$branch"
cat downloads-preamble.md table.md > downloads.md
git add downloads.md
git commit -m "Update downloads.md with list of .zip files"
git push -u origin "$branch"
echo "branch=$branch" >> $GITHUB_OUTPUT
- name: Create pull request
uses: peter-evans/create-pull-request@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ steps.write_downloads_md.outputs.branch }}
base: master
title: "Update downloads.md with list of .zip files"
body: "This PR updates the downloads.md file with a table listing all .zip files."

- name: Release
uses: softprops/action-gh-release@v2
with:
name: Test auto release
name: ${{ steps.tag.outputs.calver }}
tag_name: ${{ steps.tag.outputs.tag }}
prerelease: true
generate_release_notes: true
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
Converts wiktionary data from https://kaikki.org/ to yomitan-compatible dictionaries. Converted dictionaries can be found in the [Releases](https://github.com/themoeway/kaikki-to-yomitan/releases) section.
[DOWNLOADS](./downloads.md)

Converts wiktionary data from https://kaikki.org/ to yomitan-compatible dictionaries. Converted dictionaries can be found in the [Downloads](./downloads.md) section.

# Instructions

Expand Down
16 changes: 16 additions & 0 deletions downloads-preamble.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Downloads

This page contains 4 types of dictionary files:

1. Bilingual dictionaries - `kty-en-de.zip` for example has English headwords and their definitions/translations in German.
2. Monolingual dictionaries - `kty-en-en.zip` and such.
3. International Phonetic Alphabet (IPA) dictionaries from a single wiktionary edition - e.g. `kty-en-en-ipa.zip`
4. Merged IPA dictionaries from all 6 wiktionary editions - e.g. `kty-en-ipa.zip`. These have more terms covered but not all the entries might be formatted the same way.

Note that [Kaikki](https://kaikki.org/dictionary/rawdata.html) currently only supports 6 wiktionary editions (English, Chinese, French, German, Russian and Spanish), so only dictionaries with definitions in these languages are available.

If the language you want isn't here, or you would like to see an improvement to a dictionary, please [open an issue](https://github.com/themoeway/kaikki-to-yomitan/issues/new).

Some of the dictionaries listed here are small; rather than decide on a lower bound for usefulness they are all included here.

<sub><sup> Languages are referred to by their shortest [ISO code](https://en.wikipedia.org/wiki/List_of_ISO_639_language_codes) (ISO 639-1 where available, [ISO 639-3](https://en.wikipedia.org/wiki/List_of_ISO_639-3_codes) where not)</sup></sub>
Empty file added downloads.md
Empty file.
24 changes: 12 additions & 12 deletions languages.json
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
[
{"iso": "afb", "language": "Gulf Arabic", "flag": "🇦🇪"},
{"iso": "ang", "language": "Old English", "flag": "🗡️"},
{"iso": "sq", "language": "Albanian", "flag": "🇦🇱"},
{"iso": "grc", "language": "Ancient Greek", "flag": "🏺"},
{"iso": "ar", "language": "Arabic", "flag": "🇪🇬"},
{"iso": "zh", "language": "Chinese", "flag": "🇨🇳"},
{"iso": "cs", "language": "Czech", "flag": "🇨🇿"},
{"iso": "de", "language": "German", "flag": "🇩🇪"},
{"iso": "el", "language": "Greek", "flag": "🇬🇷"},
{"iso": "nl", "language": "Dutch", "flag": "🇳🇱"},
{"iso": "en", "language": "English", "flag": "🇺🇸🇬🇧"},
{"iso": "eo", "language": "Esperanto", "flag": "🌍"},
{"iso": "es", "language": "Spanish", "flag": "🇪🇸"},
{"iso": "fa", "language": "Persian", "flag": "🇮🇷"},
{"iso": "de", "language": "German", "flag": "🇩🇪"},
{"iso": "el", "language": "Greek", "flag": "🇬🇷"},
{"iso": "fi", "language": "Finnish", "flag": "🇫🇮"},
{"iso": "fr", "language": "French", "flag": "🇫🇷"},
{"iso": "grc", "language": "Ancient Greek", "flag": "🏺"},
{"iso": "afb", "language": "Gulf Arabic", "flag": "🇦🇪"},
{"iso": "hu", "language": "Hungarian", "flag": "🇭🇺"},
{"iso": "id", "language": "Indonesian", "flag": "🇮🇩"},
{"iso": "it", "language": "Italian", "flag": "🇮🇹"},
Expand All @@ -21,18 +21,18 @@
{"iso": "ko", "language": "Korean", "flag": "🇰🇷"},
{"iso": "la", "language": "Latin", "flag": "🏛"},
{"iso": "mn", "language": "Mongolian", "flag": "🇲🇳"},
{"iso": "nl", "language": "Dutch", "flag": "🇳🇱"},
{"iso": "ang", "language": "Old English", "flag": "🗡️"},
{"iso": "sga", "language": "Old Irish", "flag": "🍀"},
{"iso": "fa", "language": "Persian", "flag": "🇮🇷"},
{"iso": "pl", "language": "Polish", "flag": "🇵🇱"},
{"iso": "pt", "language": "Portuguese", "flag": "🇧🇷🇵🇹"},
{"iso": "ro", "language": "Romanian", "flag": "🇷🇴"},
{"iso": "ru", "language": "Russian", "flag": "🇷🇺"},
{"iso": "sga", "language": "Old Irish", "flag": "🍀"},
{"iso": "sh", "language": "Serbo-Croatian", "flag": "🇷🇸🇭🇷"},
{"iso": "sq", "language": "Albanian", "flag": "🇦🇱"},
{"iso": "es", "language": "Spanish", "flag": "🇪🇸"},
{"iso": "sv", "language": "Swedish", "flag": "🇸🇪"},
{"iso": "th", "language": "Thai", "flag": "🇹🇭"},
{"iso": "tr", "language": "Turkish", "flag": "🇹🇷"},
{"iso": "uk", "language": "Ukrainian", "flag": "🇺🇦"},
{"iso": "vi", "language": "Vietnamese", "flag": "🇻🇳"},
{"iso": "zh", "language": "Chinese", "flag": "🇨🇳"}
{"iso": "vi", "language": "Vietnamese", "flag": "🇻🇳"}
]
48 changes: 48 additions & 0 deletions tools/generate-downloads-table.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
find data/language -type f -name '*.zip' > zip_files.txt

check_file_exists() {
local filename=$1
while IFS= read -r file; do
# Extract the part of the path after 'kty-' and before the '.zip'
extracted_name=$(basename "$file" .zip | sed 's/^kty-//')
if [ "$extracted_name" = "$filename" ]; then
return 0
fi
done < zip_files.txt
return 1
}

declare -a languages="($(
jq -r '.[] | @json | @sh' languages.json
))"

echo "| | English (en) | Chinese (zh) | French (fr) | German (de) | Russian (ru) | Spanish (es) | Merged IPA |" > temp.md
echo "|---|---|---|---|---|---|---|---|" >> temp.md
columns=("en" "zh" "fr" "de" "ru" "es" "ipa")
for source_lang in "${languages[@]}"; do
source_iso=$(echo "${source_lang}" | jq -r '.iso')
source_language_name=$(echo "${source_lang}" | jq -r '.language')

row="| $source_language_name ($source_iso)"

for column in "${columns[@]}"; do

if [ "$column" = "ipa" ]; then
expected_filenames=("${source_iso}-ipa")
else
expected_filenames=("${source_iso}-${column}" "${source_iso}-${column}-ipa")
fi

cell=""
for expected_filename in "${expected_filenames[@]}"; do
if check_file_exists "$expected_filename"; then
cell="$cell [kty-$expected_filename.zip](https://github.com/themoeway/kaikki-to-yomitan/releases/latest/download/kty-$expected_filename.zip) </br>"
fi
done
row="$row | $cell"
done
echo "$row" >> temp.md
done

cat temp.md
rm temp.md

0 comments on commit 43b882c

Please sign in to comment.