dzo

This package is not ready for production use!!!

Overview

Python implemented portable and easy-to-use search engine.

Installation

# just for use.
$ pip install dzo

# for developers.
$ git clone [email protected]:moriaki3193/dzo.git
$ pip install -e .[dev]  # installs extra dependencies for development.

Usage

Command line tool

Make inverted indices

# display help
$ dzo preprocess -h

# Local directory
## 1. n-gram tokenization
$ dzo preprocess <target_dir> <result_path>
### e.g.
$ dzo preprocess ./data/products ./inverted-index.pkl

## 2. mecab tokenization
$ dzo preprocess --tokenizer=mecab --dicdir=<dicdir> <target_dir> <result_path>
### e.g.
$ dzo preprocess --tokenizer=mecab --dicdir=/usr/local/lib/mecab/dic/ipadic ./data ./inverted-index.pkl

Search

$ dzo search <query> --index-path <index_path>

# e.g.
$ dzo search おにぎり --index-path ./data/inverted-index.pkl

# e.g.
$ dzo search おにぎり --index-path ./data/inverted-index.pkl --dicdir=/usr/local/lib/mecab/dic/ipadic

Python package

WIP

Development

Note

Style of all docstrings for functions and methods have to be Google Style Python Docstring. Please refer to napoleon style guide.

Commands

# Building cython extensions.
$ make build/ext

# Linting
$ make check/lint

# Type hinting
$ make check/type

# Running tests using pytest.
$ make test

# Look up dictionary directories for MeCab
$ make list/dicdir

# Other utility commands are available.
# See Makefile for more details.

Sponsors