GitHub - hunter/ferret_tokenizer: A string tokenizer based on Ferret::Analysis::StandardAnalyzer.

hunter / ferret_tokenizer Public

forked from feedbackmine/ferret_tokenizer

Notifications You must be signed in to change notification settings
Fork 1
Star 3

A string tokenizer based on Ferret::Analysis::StandardAnalyzer.

helpdeskontwitter.com

3 stars 4 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 8 Commits
ext		ext
test		test
Manifest.txt		Manifest.txt
README.txt		README.txt
Rakefile		Rakefile
ferret_tokenizer.gemspec		ferret_tokenizer.gemspec

Repository files navigation

== Overview
A string tokenizer based on Ferret::Analysis::StandardAnalyzer. I made some improvements for better text analysis.

== Install

sudo gem sources -a http://gems.github.com   (you only have to do this once)
sudo gem install feedbackmine-ferret_tokenizer

== Usage

require 'ferret_tokenizer'
t = FerretTokenizer.new '@feedbackmine, I LOVE this!!! :-)'
while (tok = t.next)
  p tok
end

== Follow us on twitter

http://twitter.com/feedbackmine