Skip to content

Commit

Permalink
Update tokenizers
Browse files Browse the repository at this point in the history
  • Loading branch information
@as-suvorov
as-suvorov committed Apr 10, 2024
1 parent acab030 commit 9cc9ca4
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion thirdparty/openvino_tokenizers
Submodule openvino_tokenizers updated 10 files
+59 −11 .github/workflows/linux.yml
+58 −10 .github/workflows/mac.yml
+59 −10 .github/workflows/windows.yml
+6 −1 CMakeLists.txt
+1 −1 pyproject.toml
+8 −7 python/openvino_tokenizers/hf_parser.py
+8 −0 python/openvino_tokenizers/tokenizer_pipeline.py
+8 −3 src/CMakeLists.txt
+1 −1 tests/pass_rates.json
+1 −1 tests/tokenizers_test.py

0 comments on commit 9cc9ca4

Please sign in to comment.