Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerize #4

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Dockerize #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1bc2338
Add Dockerfile
Lodifice Nov 19, 2019
5fafce4
parser/fst/lazy_composition.h: fix string usage
Lodifice Nov 19, 2019
41a4309
setup.py: add /usr/local/include to eigen includes
Lodifice Nov 19, 2019
be76406
requirements.txt: add nltk dependency
Lodifice Nov 19, 2019
f8003f3
Dockerfile: build GF from source (for Python 3)
Lodifice Nov 19, 2019
e557477
INSTALL: add docker section
Lodifice Apr 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
FROM ubuntu:bionic

# Change versions of dependencies here
ARG EIGEN_REVISION=9e6bc1d
ARG OPENFST_VERSION=1.7.5
ARG PYNINI_VERSION=2.0.9
ARG GF_VERSION=3.10

# @Python: read files in UTF-8 encoding!
ENV LANG=C.UTF-8

# Install build environment and most dependencies via apt
ENV DEBIAN_FRONTEND noninteractive
RUN apt update \
&& apt -y install autoconf build-essential cmake git graphviz libncurses5-dev libre2-dev libtool mercurial python-dev python3-dev python3-numpy python3-pip wget zlib1g-dev \
&& apt -y autoremove \
&& apt -y clean \
&& rm -rf /var/lib/apt/lists/*

# Install eigen from source
RUN hg clone https://bitbucket.org/eigen/eigen/ -r "$EIGEN_REVISION"
RUN mkdir eigen/build
WORKDIR eigen/build
RUN cmake ..
RUN make install
WORKDIR /

# Install disco-dop from source
RUN git clone --recursive --branch=chart-exposure https://github.com/kilian-gebhardt/disco-dop
WORKDIR disco-dop
RUN pip3 install -r requirements.txt
RUN sed -i 's/install --user/install/' Makefile
RUN make install
WORKDIR /

# Install OpenFST from source
RUN wget "http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-$OPENFST_VERSION.tar.gz"
RUN tar -xf "openfst-$OPENFST_VERSION.tar.gz"
WORKDIR "openfst-$OPENFST_VERSION"
RUN ./configure --enable-bin --enable-compact-fsts --enable-compress --enable-const-fsts --enable-far --enable-linear-fsts --enable-lookahead-fsts --enable-mpdt --enable-ngram-fsts --enable-pdt --enable-python PYTHON=python3
RUN make
RUN make install
WORKDIR /

# Install Pynini from source
RUN wget "http://www.openfst.org/twiki/pub/GRM/PyniniDownload/pynini-$PYNINI_VERSION.tar.gz"
RUN tar -xf "pynini-$PYNINI_VERSION.tar.gz"
WORKDIR "pynini-$PYNINI_VERSION"
RUN python3 setup.py install
WORKDIR /

# Install Grammatical Framework custom package
RUN wget "https://github.com/GrammaticalFramework/gf-core/archive/GF-$GF_VERSION.tar.gz"
RUN tar -xf "GF-$GF_VERSION.tar.gz"
WORKDIR "gf-core-GF-$GF_VERSION/src/runtime/c"
RUN bash setup.sh configure
RUN bash setup.sh build
RUN bash setup.sh install
WORKDIR "/gf-core-GF-$GF_VERSION/src/runtime/python"
RUN python3 setup.py build
RUN python3 setup.py install
WORKDIR /

# Install Boost from source
ARG BOOST_VERSION=1.69.0
RUN wget "https://dl.bintray.com/boostorg/release/$BOOST_VERSION/source/boost_$(echo $BOOST_VERSION | sed 's/\./_/g').tar.gz"
RUN tar -xf "boost_$(echo $BOOST_VERSION | sed 's/\./_/g').tar.gz"
RUN cd "boost_$(echo $BOOST_VERSION | sed 's/\./_/g')" && ./bootstrap.sh --with-libraries= && ./b2 install || true

# Build Panda parser
ADD . /panda-parser
WORKDIR panda-parser
RUN pip3 install -r requirements.txt
RUN python3 setup.py build_ext --inplace
10 changes: 10 additions & 0 deletions INSTALL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# INSTALL GUIDE

## Via Docker

Build an image, the current directory is added as the top layer.

`docker build --tag=panda-parser .`

Obtain a shell in the container.

`docker run --rm -it panda-parser bash`

## General prerequisites

`python3.5` or newer, `sqlite3`, `pip3`, `re2` (`libre2-dev` on ubuntu)
Expand Down
4 changes: 2 additions & 2 deletions parser/fst/lazy_composition.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

using namespace fst;

StdVectorFst construct_fa(std::vector<string> sequence, const StdFst & fst) {
StdVectorFst construct_fa(std::vector<std::string> sequence, const StdFst & fst) {
StdVectorFst fsa;
fsa.SetInputSymbols(fst.InputSymbols());
fsa.SetOutputSymbols(fst.InputSymbols());
Expand Down Expand Up @@ -47,4 +47,4 @@ std::vector<unsigned> lazy_compose_(const StdVectorFst & fst_a, const StdFst & f
}
}
return rules;
}
}
1 change: 1 addition & 0 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
backports-abc>=0.4
Cython>=0.25.2
graphviz>=0.5.1
nltk>=3.4.5
# openfst>=1.5.1.post6
# pgf>=1.0
pickleshare>=0.7.4
Expand Down
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
# change if eigen is installed in the user-local directory
# $COMPUTE_ROOT/usr/include/eigen3,
compute_root = ""
eigen_include = [compute_root + "/usr/include/eigen3", compute_root + "/usr/include"]
eigen_include = [compute_root + "/usr/include/eigen3", compute_root + "/usr/local/include/eigen3", compute_root + "/usr/include"]
add_include = [compute_root + "/usr/local/include"]

# Schick Parser (mainly implemented by Timo Schick according to construction
Expand Down