Remove ellipsis when tokenizing special tokens such as :m:
and :si:
#3
Labels
help wanted
Extra attention is needed
:m:
and :si:
#3
BELA v2.0.0a21.post6 tokenize() function does not process ellipsis when they are embedded in singing or mimicking segments.
E.g., applying tokenize() function on
:si:walking+in+the+jungle+walking+in+the+jung...
yields: ['walking', 'in', 'the', 'jungle', 'walking', 'in', 'the', 'jung...']Proposing to remove ellipsis if ellipsis parameter is set True. Expected result would look like this: ['walking', 'in', 'the', 'jungle', 'walking', 'in', 'the', 'jung'].
The text was updated successfully, but these errors were encountered: