Skip to content

Latest commit

 

History

History
43 lines (40 loc) · 780 Bytes

README.md

File metadata and controls

43 lines (40 loc) · 780 Bytes
Raw

Charabyx

WIP : Elixir NIFs for Meilisearch's Charabia crate.

Functions will be added as I need them.

iex> Charabyx.tokenize("bonjour chers amis !")
[
  %Charabyx.NifToken{
    kind: :word,
    lemma: "bonjour",
    script: :Latin,
    char_start: 0,
    char_end: 7,
    language: nil
  },
  %Charabyx.NifToken{
    kind: :separator_soft,
    lemma: " ",
    script: :Latin,
    char_start: 7,
    char_end: 8,
    language: nil
  },
  %Charabyx.NifToken{
    kind: :word,
    lemma: "chers",
    script: :Latin,
    char_start: 8,
    char_end: 13,
    language: nil
  },
  %Charabyx.NifToken{
    kind: :separator_soft,
    lemma: " ",
    script: :Latin,
    char_start: 13,
    char_end: 14,
    language: nil
  },
  ...