Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add jdbcUrl as configuration parameter #10

Merged
merged 1 commit into from
Sep 17, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 23 additions & 114 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,129 +1,38 @@
version: 2.1

executors:
leiningen:
docker:
- image: circleci/clojure:openjdk-8-lein-2.9.3
environment:
LEIN_ROOT: nbd
JVM_OPTS: -Xmx3200m
working_directory: /home/circleci/datahike-jdbc

jobs:
setup:
executor: leiningen
steps:
- restore_cache:
keys:
- source-{{ .Branch }}-{{ .Revision }}
- source-{{ .Branch }}
- source-
- checkout
- save_cache:
key: source-{{ .Branch }}-{{ .Revision }}
paths:
- .git
- restore_cache:
keys:
- deps-{{ checksum "project.clj" }}
- deps-
- run: lein deps
- save_cache:
key: deps-{{ checksum "project.clj" }}
paths:
- /home/circleci/.m2
- /home/circleci/.lein
- persist_to_workspace:
root: /home/circleci/
paths:
- .m2
- datahike-jdbc
- bin

integrationtest:
executor: leiningen
docker:
- image: circleci/clojure:openjdk-8-lein-2.9.3
- image: circleci/postgres:11-alpine
environment:
POSTGRES_USER: alice
POSTGRES_PASSWORD: foo
POSTGRES_DB: config-test
- image: circleci/mariadb:10.4
environment:
MYSQL_USER: alice
MYSQL_PASSWORD: foo
MYSQL_DATABASE: config-test
MYSQL_RANDOM_ROOT_PASSWORD: true
steps:
- attach_workspace:
at: /home/circleci
- run:
name: Run Integrationtests
command: lein test
no_output_timeout: 5m

build:
executor: leiningen
steps:
- attach_workspace:
at: /home/circleci
- run:
name: Build
command: lein uberjar
no_output_timeout: 5m

deploy-snapshot:
executor: leiningen
steps:
- attach_workspace:
at: /home/circleci
- run:
command: |
VERSION=$(head -n 1 project.clj | awk '{print $3}' | tr -d \")
if [[ ${VERSION} =~ .*-SNAPSHOT ]]; then
lein deploy clojars
else
exit 0
fi
deploy-release:
executor: leiningen
steps:
- attach_workspace:
at: /home/circleci
- run:
name: Deploy Release to Clojars
command: |
VERSION=$(head -n 1 project.clj | awk '{print $3}' | tr -d \")
if [[ ${VERSION} =~ .*-SNAPSHOT ]]; then
exit 0
else
lein deploy clojars
fi
orbs:
lein: replikativ/clj-lein@0

workflows:
build_and_test:
build-test-and-deploy:
jobs:
- setup
- integrationtest:
- lein/setup
- lein/format:
requires:
- lein/setup
- lein/integrationtest-postgresql-mariadb:
requires:
- setup
- build:
- lein/setup
- lein/build:
requires:
- setup
- deploy-snapshot:
- lein/setup
- lein/deploy-snapshot:
context: clojars-deploy
filters:
branches:
only: development
requires:
- setup
- integrationtest
- build
- deploy-release:
- lein/setup
- lein/format
- lein/build
- lein/integrationtest-postgresql-mariadb
- lein/deploy-release:
context: clojars-deploy
filters:
branches:
only: master
requires:
- setup
- integrationtest
- build
- lein/setup
- lein/format
- lein/build
- lein/integrationtest-postgresql-mariadb
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ pom.xml.asc
.nrepl-port
.cpcache/
temp/
datahike.mv.db
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ DATAHIKE_STORE_BACKEND=jdbc
DATAHIKE_STORE_CONFIG='{:dbtype "postgresql" :user "datahike" :password "datahike" :dbname "datahike"}'
```

It is also possible to pass a configuration url via `:jdbcUrl` like it is mentioned in the underlying library [next.jdbc](https://cljdoc.org/d/seancorfield/next.jdbc/1.0.462/doc/getting-started#the-db-spec-hash-map). The Url can pass additional arguments in the query part or you can combine the url with arguments passed as key-value-pairs.

Arguments not mentioned will be passed downstream to the corresponding jdbc-driver so every configuration option available should be working.

## Prerequisites
For this backend to work you need to choose a database that is supported by JDBC. Please have a
look at the docs for [clojure.java.jdbc](https://github.com/clojure/java.jdbc/). For the sake
Expand Down
4 changes: 2 additions & 2 deletions bin/run-integration-tests → bin/run-integrationtests
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ trap teardown EXIT

function setup_PG() {
if [ -z ${CIRCLECI} ]; then
echo $(docker run --detach --publish 5432:5432 --env POSTGRES_DB=config-test --env POSTGRES_USER=alice --env POSTGRES_PASSWORD=foo postgres:11-alpine)
echo $(docker run --detach --publish 5432:5432 --env POSTGRES_DB=config-test --env POSTGRES_USER=alice --env POSTGRES_PASSWORD=foo postgres:12-alpine)
fi
}

Expand All @@ -29,4 +29,4 @@ MARIA_NAME=$(setup_MARIA)

sleep 5

TIMBRE_LEVEL=':warn' DATAHIKE_STORE_BACKEND=jdbc DATAHIKE_STORE_CONFIG='{:dbtype "h2:mem" :dbname "datahike"}' lein test
TIMBRE_LEVEL=':warn' DATAHIKE_STORE_BACKEND=jdbc DATAHIKE_STORE_CONFIG='{:dbtype "h2" :dbname "datahike"}' lein test
1 change: 1 addition & 0 deletions project.clj
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
[com.taoensso/timbre "4.10.0"]
[io.replikativ/datahike "0.3.2-SNAPSHOT" :exclusions [io.replikativ/superv.async]]
[alekcz/konserve-jdbc "0.1.0-SNAPSHOT"]]
:plugins [[lein-cljfmt "0.6.8"]]
:deploy-repositories
[["clojars"
{:url "https://clojars.org/repo"
Expand Down
16 changes: 8 additions & 8 deletions src/datahike_jdbc/core.clj
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,7 @@
[datahike.config :refer [map-from-env]]
[hitchhiker.tree.bootstrap.konserve :as kons]
[konserve-jdbc.core :as k]
[environ.core :refer [env]]
[clojure.spec.alpha :as s]
[clojure.edn :as edn]
[superv.async :refer [<?? S]]))

(defmethod empty-store :jdbc [store-config]
Expand All @@ -24,12 +22,13 @@

(defmethod default-config :jdbc [config]
(merge
(map-from-env :datahike-store-config {:dbtype "h2:mem"
:dbname "datahike"})
config))
(map-from-env :datahike-store-config {:dbtype "h2:mem"
:dbname "datahike"})
config))

(s/def :datahike.store.jdbc/backend #{:jdbc})
(s/def :datahike.store.jdbc/dbtype #{"h2" "h2:mem" "hsqldb" "jtds:sqlserver" "mysql" "oracle:oci" "oracle:thin" "postgresql" "redshift" "sqlite" "sqlserver"})
(s/def :datahike.store.jdbc/jdbcUrl string?)
(s/def :datahike.store.jdbc/dbname string?)
(s/def :datahike.store.jdbc/dbname-separator string?)
(s/def :datahike.store.jdbc/host string?)
Expand All @@ -38,9 +37,10 @@
(s/def :datahike.store.jdbc/classname string?)
(s/def :datahike.store.jdbc/user string?)
(s/def :datahike.store.jdbc/password string?)
(s/def ::jdbc (s/keys :req-un [:datahike.store.jdbc/backend
:datahike.store.jdbc/dbtype]
:opt-un [:datahike.store.jdbc/dbname
(s/def ::jdbc (s/keys :req-un [:datahike.store.jdbc/backend]
:opt-un [:datahike.store.jdbc/dbtype
:datahike.store.jdbc/jdbcUrl
:datahike.store.jdbc/dbname
:datahike.store.jdbc/dbname-separator
:datahike.store.jdbc/host
:datahike.store.jdbc/host-prefix
Expand Down
47 changes: 23 additions & 24 deletions test/datahike_jdbc/core_test.cljc
Original file line number Diff line number Diff line change
@@ -1,27 +1,26 @@
(ns datahike-jdbc.core-test
(:require
#?(:cljs [cljs.test :as t :refer-macros [is are deftest testing]]
:clj [clojure.test :as t :refer [is are deftest testing]])
[datahike.api :as d]
[datahike-jdbc.core]))
(:require
#?(:cljs [cljs.test :as t :refer-macros [is are deftest testing]]
:clj [clojure.test :as t :refer [is are deftest testing]])
[datahike.api :as d]
[datahike-jdbc.core]))

(deftest ^:integration test-postgresql
(let [config {:store {:backend :jdbc
:dbtype "postgresql"
:user "alice"
:password "foo"
:dbname "config-test"}
:jdbcUrl "jdbc:postgresql://localhost/config-test?user=alice"
:password "foo"}
:schema-flexibility :read
:keep-history? false}
_ (d/delete-database config)]
(is (not (d/database-exists? config)))
(let [_ (d/create-database config)
conn (d/connect config)]

(d/transact conn [{ :db/id 1, :name "Ivan", :age 15}
{ :db/id 2, :name "Petr", :age 37}
{ :db/id 3, :name "Ivan", :age 37}
{ :db/id 4, :age 15}])
(d/transact conn [{:db/id 1, :name "Ivan", :age 15}
{:db/id 2, :name "Petr", :age 37}
{:db/id 3, :name "Ivan", :age 37}
{:db/id 4, :age 15}])
(is (= (d/q '[:find ?e :where [?e :name]] @conn)
#{[3] [2] [1]}))

Expand All @@ -48,10 +47,10 @@
{:db/ident :age
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one}])
(d/transact conn [{ :db/id 1, :name "Ivan", :age 15}
{ :db/id 2, :name "Petr", :age 37}
{ :db/id 3, :name "Ivan", :age 37}
{ :db/id 4, :age 15}])
(d/transact conn [{:db/id 1, :name "Ivan", :age 15}
{:db/id 2, :name "Petr", :age 37}
{:db/id 3, :name "Ivan", :age 37}
{:db/id 4, :age 15}])
(is (= (d/q '[:find ?e :where [?e :name]] @conn)
#{[3] [2] [1]}))

Expand All @@ -77,10 +76,10 @@
{:db/ident :age
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one}])
(d/transact conn [{ :db/id 1, :name "Ivan", :age 15}
{ :db/id 2, :name "Petr", :age 37}
{ :db/id 3, :name "Ivan", :age 37}
{ :db/id 4, :age 15}])
(d/transact conn [{:db/id 1, :name "Ivan", :age 15}
{:db/id 2, :name "Petr", :age 37}
{:db/id 3, :name "Ivan", :age 37}
{:db/id 4, :age 15}])
(is (= (d/q '[:find ?e :where [?e :name]] @conn)
#{[3] [2] [1]}))

Expand All @@ -101,10 +100,10 @@
{:db/ident :age
:db/valueType :db.type/long
:db/cardinality :db.cardinality/one}])
(d/transact conn [{ :db/id 1, :name "Ivan", :age 15}
{ :db/id 2, :name "Petr", :age 37}
{ :db/id 3, :name "Ivan", :age 37}
{ :db/id 4, :age 15}])
(d/transact conn [{:db/id 1, :name "Ivan", :age 15}
{:db/id 2, :name "Petr", :age 37}
{:db/id 3, :name "Ivan", :age 37}
{:db/id 4, :age 15}])
(is (= (d/q '[:find ?e :where [?e :name]] @conn)
#{[3] [2] [1]}))

Expand Down