Skip to content

Getting Started

Jump to bottom
johnchronis edited this page Nov 3, 2015 · 7 revisions

Software Dependencies

Installing from tarball

  • After installing the software dependencies you can unpack the tarball by executing :
$ mkdir ~/exareme
$ tar xf exareme-*tar.gz -C ~/exareme
$ cd ~/exareme

  • Installation directory structure (under ~/exareme/ ) :

    • /bin/ - binary scripts.
    • /etc/ - configuration and properties files.
    • /var/ - logs and pids files.
    • /lib/ - exareme, madis libraries and dependencies.

  • Current release can operate in local or distributed mode. You can configure the system accordingly by set up the appropiate files in ~/exareme/etc/exareme/.

    • exareme-env.sh exports the required environment variables like java etc.
    • Local mode requires only the master ip which you can provide by set up the master file.
    • Distributed mode requires to set up the master and workers files. Workers file format requires one worker ip per line.
    • Example:
$ cat ~/exareme/etc/exareme/master
192.168.0.1
$ cat ~/exareme/etc/exareme/workers
192.168.0.2
192.168.0.3

Running Exareme

  • In ~/exareme/bin directory, you can find the exareme-admin.sh script which is responsible for the administration tasks for both local/distributed mode.
$ cd ~/exareme
$ ./bin/exareme-admin.sh --install  #You can skip it in local mode.
$ ./bin/exareme-admin.sh --start
$ ./bin/exareme-admin.sh --status
$ ./bin/exareme-admin.sh --stop
$ ./bin/exareme-admin.sh --kill
$ ./bin/exareme-admin.sh --uninstall #You can skip it in local mode.

Executing Queries

  • In order to execute exareme queries you can use either our jdbc driver or the console. By executing ./bin/exareme-admin.sh --console a madis console will be prompt along with some exa-like operators available.

    • exaquery - submit an EXADFL query.
    • exaqueryscript - submit an EXADFL queryscript.
    • exaresult - returns result table.

  • Exa-like operators requires host, port, db property.

    • you can set this global properties by executing.
      mterm> var ‘db’ ‘/data/demo/tpch’;
    • or you can set as operator property
      mterm> exaresult ‘db:/data/demo/tpch’ nation;
    • host and port are optional and can be skipped if the console is running on the same host as master node.

Example usage of exa-like operators:

Sample import
$ ./bin/exareme-admin.sh --console
mterm> exaquery ‘db:/path/to/db/’ distributed create table lineitem as external
select
 cast(c1 as int) as l_orderkey,
 cast(c2 as int) as l_partkey,
 cast(c3 as int) as l_suppkey,
 cast(c4 as int) as l_linenumber,
 cast(c5 as int) as l_quantity,
 cast(c6 as float) as l_extendedprice,
 cast(c7 as float) as l_discount,
 cast(c8 as float) as l_tax,
 cast(c9 as text) as l_returnflag,
 cast(c10 as text) as l_linestatus,
 cast(c11 as text) as l_shipdate,
 cast(c12 as text) as l_commitdate,
 cast(c13 as text) as l_receiptdate,
 cast(c14 as text) as l_shipinstruct,
 cast(c15 as text) as l_shipmode,
 cast(c16 as text) as l_comment
from (file '/path/to/datasets/lineitem.tbl.gz' delimiter:| fast:1);
Sample query
$ cat /path/to/queries/q6.sql
distributed create temporary table lineitem_partial to 1 as
select sum(l_extendedprice * l_discount) as revenue_PARTIAL
from lineitem
where l_shipdate >= '1994-01-01' 
and l_shipdate < '1995-01-01' 
and l_discount > 0.07 
and l_discount < 0.09 
and l_quantity < 24;

distributed create table q6_result as
select sum(revenue_PARTIAL) as revenue
from lineitem_partial;

$ ./bin/exareme-admin.sh --console
mterm> exaqueryscript ‘db:/path/to/db’ file ‘/path/to/queries/q6.sql’;
Sample result
$ ./bin/exareme-admin.sh --console
mterm> select * from (exaresult ‘db:/path/to/db’ q6_result) limit 2;

Development

  • Requires
  • JDK 1.7+
  • maven 3.0+
  • Building
  • CLONE dev branch.
  • RUN "mvn clean install -Pmultiarch -DskipTests=true" create a tarball (exareme/exareme-distribution/target/exareme-*.tar.gz) ready for production intallation.
Clone this wiki locally