Skip to content

Commit

Permalink
Merge branch 'development' into mod-annotate-synth
Browse files Browse the repository at this point in the history
  • Loading branch information
mtholder committed Oct 31, 2023
2 parents 11604de + d32db56 commit 35206ac
Show file tree
Hide file tree
Showing 330 changed files with 41,201 additions and 31,536 deletions.
195 changes: 195 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
name: Build and test

on:
push:
branches:
- "*"
pull_request:
branches:
- "*"

jobs:
build:
strategy:
matrix:
name: [ubuntu-gcc-10,
ubuntu-gcc-11,
# ubuntu-gcc-11-sanitize,
ubuntu-gcc-12,
ubuntu-gcc-12-debugoptimized,
ubuntu-clang-13,
# macos-xcode-12.5
]

include:
- name: ubuntu-gcc-9
os: ubuntu-latest
compiler: gcc
version: "9"
buildtype: "release"

- name: ubuntu-gcc-10
os: ubuntu-latest
compiler: gcc
version: "10"
buildtype: "release"

- name: ubuntu-gcc-11
os: ubuntu-latest
compiler: gcc
version: "11"
buildtype: "release"

- name: ubuntu-gcc-12
os: ubuntu-latest
compiler: gcc
version: "12"
buildtype: "release"

# To enable this, we'd probably need to add `export ASAN_OPTIONS=detect_odr_violation=0`.
#
# - name: ubuntu-gcc-11-sanitize
# os: ubuntu-latest
# compiler: gcc
# version: "11"
# buildtype: "release"
# sanitize: "address"
#

- name: ubuntu-gcc-12-debugoptimized
os: ubuntu-latest
compiler: gcc
version: "12"
buildtype: "debugoptimized"

- name: ubuntu-clang-13
os: ubuntu-latest
compiler: clang
version: "13"
buildtype: "release"

# Currently we get "dyld: Library not loaded: @rpath/librestbed.4.dylib"
# when attempting to run executables.
#
# - name: macos-xcode-12.5
# os: macos-latest
# compiler: xcode
# version: "12.5"
# buildtype: "release"

runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.8

- name: Install (Linux)
if: runner.os == 'Linux'
run: |
sudo apt install -y ccache libboost-all-dev
sudo rm -rf /usr/local/share/boost
if [ "${{ matrix.compiler }}" = "gcc" ]; then
sudo apt-get install -y g++-${{ matrix.version }}
echo "CC=ccache gcc-${{ matrix.version }}" >> $GITHUB_ENV
echo "CXX=ccache g++-${{ matrix.version }}" >> $GITHUB_ENV
else
sudo apt-get install -y clang-${{ matrix.version }}
echo "CC=ccache clang-${{ matrix.version }}" >> $GITHUB_ENV
echo "CXX=ccache clang++-${{ matrix.version }}" >> $GITHUB_ENV
fi
- name: Install (macOS)
if: runner.os == 'macOS'
run: |
brew install pkg-config boost ccache coreutils
echo "DEVELOPER_DIR=/Applications/Xcode_${{ matrix.version }}.app/Contents/Developer" >> $GITHUB_ENV
echo "CC=ccache clang" >> $GITHUB_ENV
echo "CXX=ccache clang++" >> $GITHUB_ENV
- name: Install meson
run: |
python3 -mpip install meson ninja
# Caches for different branches are isolated, so we don't need to put the branch name into the key.

- name: Prepare ccache timestamp
id: ccache_cache_timestamp
run: |
ccache --set-config=cache_dir=$HOME/.ccache
if [ "${{ runner.os }}" = "Linux" ]; then
stamp=$(date '+%s')
else
stamp=$(gdate '+%s')
fi
echo "${stamp}"
echo "timestamp=${stamp}" >> $GITHUB_OUTPUT
- name: ccache cache files
uses: pat-s/[email protected]
with:
path: ~/.ccache
key: ${{ matrix.name }}-ccache-${{ steps.ccache_cache_timestamp.outputs.timestamp }}
restore-keys: |
${{ matrix.name }}-ccache-
- name: build restbed
run: |
cd
git clone --recursive https://github.com/corvusoft/restbed.git
mkdir restbed/build
cd restbed/build
cmake .. -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DBUILD_TESTS=NO -DBUILD_SSL=NO -DCMAKE_INSTALL_PREFIX="${HOME}/local/" -G Ninja
ninja install
echo "LD_LIBRARY_PATH=${HOME}/local/library:$LD_LIBRARY_PATH" >> $GITHUB_ENV
- name: build g3log
run: |
cd
git clone https://github.com/KjellKod/g3log.git
mkdir g3log/build
cd g3log/build
cmake .. -DCMAKE_INSTALL_PREFIX="/usr/" -DUSE_DYNAMIC_LOGGING_LEVELS=ON -G Ninja
ninja package
sudo dpkg -i g3log*.deb
- name: Configure otcetera
run: |
cd ${GITHUB_WORKSPACE}
ARGS="${ARGS} -Drestbed_dir=${HOME}/local/"
if [ "${{ runner.os }}" = "Linux" ] && [ "${{ matrix.name }}" != "windows" ] && [ "${{ matrix.sanitize }}" != "" ] ; then
# Address sanitizer can't find its dylibs on OS X?
ARGS="${ARGS} -Db_sanitize=${{ matrix.sanitize }}"
fi
echo "CPPFLAGS=-g $CPPFLAGS" >> $GITHUB_ENV
echo meson setup build --prefix=$HOME/local --buildtype=${{ matrix.buildtype }} ${ARGS}
meson setup build --prefix=$HOME/local --buildtype=${{ matrix.buildtype }} ${ARGS}
- name: Build otcetera
run: |
ccache -p
ninja -C build install -j4
ccache -s
- name: Run otcetera tests
run: |
$HOME/local/bin/otc-version-reporter
$HOME/local/bin/otc-solve-subproblem -h
python3 -mpip install requests
ninja -C build test
- name: Upload test files if they failed
uses: actions/upload-artifact@v3
if: failure()
with:
name: test-results-${{ matrix.name }}
path: expected

- name: Upload test files if they failed
uses: actions/upload-artifact@v3
if: failure()
with:
name: test-resultsws-${{ matrix.name }}
path: expected-ws

1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
*~
#*#
.#*
build*
COPYING
INSTALL
*gmon.out
Expand Down
2 changes: 0 additions & 2 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ otcetera has been written by:
Mark T. Holder http://orcid.org/0000-0001-5575-0536
Ben D. Redelings http://ben-redelings.org

otceteraincludes: http://github.com/easylogging

otcetera depends on https://github.com/nlohmann/json

otcetera has borrowed many ideas/snippets from NCL (see author list for NCL below)
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ flag, then you can choose between a GPL or BSD license.
If you use the --with-webservices=yes configure flag, only the
GPL is permitted.

Copyright (c) 2015-2017 Mark T. Holder and Benjamin D. Redelings
Copyright (c) 2015-2020 Mark T. Holder and Benjamin D. Redelings
All rights reserved.

otcetera builds upon NCL and is distributed under the same license:
Expand Down
43 changes: 24 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,6 @@ otcetera owes a lot of code and ideas to Paul Lewis' Nexus Class Library.
See http://hydrodictyon.eeb.uconn.edu/ncl/ and
https://github.com/mtholder/ncl

It also uses easyloggingpp which is distributed under an MIT License. See
http://github.com/easylogging/ for info on that project. The file from
that project is otc/easylogging++.h

Some set comparisons (in util.h) were based on
http://stackoverflow.com/posts/1964252/revisions
by http://stackoverflow.com/users/127669/graphics-noob
Expand All @@ -26,23 +22,27 @@ The instructions below contain all of the gory detail. There are a few quirks wi

### Compiler
Otcetera requires a C++17 compiler. You can use
* g++ version 8 (or higer)
* clang++ version 7 (or higher)
* XCode version 10.1 (or higher)
* g++ version 9 (or higer)
* clang++ version 11 (or higher)
* XCode version 13 (or higher)

### Build tools: meson, cmake, and ninja
To build otcetera, we need the build tools
* [meson](http://mesonbuild.com).
* [meson](http://mesonbuild.com)
* ninja
* cmake (to build the restbed library)

### HTTP library: restbed

We are using the [Restbed framework](https://github.com/corvusoft/restbed) to implement web services for the tree of life. By default, otcetera will not compile the web services if it can't find restbed.

### Logging library: g3log

Otcetera now requires the logging library g3log.

### Testing: requests

The python requests package is need for running the `make check` target because it runs tests in the `ws` subdirectory.
The python requests package is need for running the `ninja test` target because it runs tests in the `ws` subdirectory.

## prerequisites: quick start

Expand All @@ -58,14 +58,12 @@ On recent versions of Debian or Ubuntu Linux, you can run:
sudo apt-get install meson cmake ninja-build libboost-all-dev libcurl4-openssl-dev

### Meson (alternate)
If you don't have version >= 0.49 of meson, you can install it in a virtualenv
If you don't have version >= 0.60 of meson, you can install it in a virtualenv

# Install meson in a virtualenv
python3 -m venv meson
source meson/bin/activate
pip3 install meson
# Install ninja to the virtualenv bin directory
wget https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip && unzip -q ninja-linux.zip -d meson/bin
pip3 install meson ninja

On windows, you can install meson using the MSI installer on the [releases page](https://github.com/mesonbuild/meson/releases).

Expand All @@ -82,12 +80,20 @@ After installing prerequisites, try the following commands to build `restbed` an
git clone --recursive https://github.com/corvusoft/restbed.git
mkdir -p $OPENTREE/otcetera
cd $OPENTREE/otcetera
git clone https://github.com/mtholder/otcetera.git
git clone https://github.com/OpenTreeOfLife/otcetera.git

# On Mac, check that we are using homebrew ssl in /usr/local/opt/openssl, not system ssl!
echo "CPPFLAGS=${CPPFLAGS}"
echo "LDFLAGS=${LDFLAGS}"

# Build g3log
git clone https://github.com/KjellKod/g3log.git
mkdir g3log/build
(cd g3log/build
cmake .. -G Ninja -DUSE_DYNAMIC_LOGGING_LEVELS=ON -DCMAKE_INSTALL_PREFIX=/usr -DCPACK_PACKAGE_FILE_NAME=g3log
nice -n10 ninja package
sudo dpkg -i g3log.deb)

# Build restbed
alias ninja='nice -n10 ninja'
cd $OPENTREE/restbed
Expand All @@ -98,12 +104,12 @@ After installing prerequisites, try the following commands to build `restbed` an

# Make restbed library available too.
export CPPFLAGS="-I${OPENTREE}/local/include $CPPFLAGS"
export LDFLAGS="-L${OPENTREE}/local/lib $LDFLAGS"
export LDFLAGS="-L${OPENTREE}/local/library $LDFLAGS"
echo "CPPFLAGS=${CPPFLAGS}"
echo "LDFLAGS=${LDFLAGS}"
# Mac ignores LD_LIBRARY_PATH and doesn't need it, but linux needs it.
export LD_LIBRARY_PATH=${OPENTREE}/local/lib
export LD_LIBRARY_PATH=${OPENTREE}/local/library

# Build otcetera
cd $OPENTREE/otcetera
meson build otcetera --prefix=$OPENTREE/local
Expand Down Expand Up @@ -566,8 +572,7 @@ The syntax used to describe a new test is described in [../expected/README.md](.
and the directories that describe the expected behavior are in the `expected` subdirectory.

## ACKNOWLEDGEMENTS
See comments above about usage of [easyloggingpp](https://github.com/easylogging/)
and [nlohmann::json](https://github.com/nlohmann/json)
See comments above about usage of [nlohmann::json](https://github.com/nlohmann/json)

To acknowledge the contributions of the NCL code and ideas, a snapshot of the
NCL credits taken from the version of NCL used to jump start otcetera is:
Expand Down
9 changes: 9 additions & 0 deletions data/add-hidden.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[
{
"action": "edit",
"taxon": {
"ott_id": 2,
"flags": "edited,hybrid,hidden"
}
}
]
9 changes: 9 additions & 0 deletions data/add-redirect.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[
{
"action": "add",
"forward": {
"former": 6520249,
"redirect_to": 100
}
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,6 @@
"xx_1@t2"
],
"synth_id": "<UNKNOWN>",
"taxonomy_version": "exampletaxonomy",
"taxonomy_version": "0.0draft0",
"tree_id": "<UNKNOWN>"
}
Loading

0 comments on commit 35206ac

Please sign in to comment.