Skip to content

Commit

Permalink
Add RocksDB
Browse files Browse the repository at this point in the history
Add scripts to run ycsb on rocksdb and update readme
  • Loading branch information
OmSaran committed Oct 27, 2020
1 parent 5ce2ae4 commit b5e8b19
Show file tree
Hide file tree
Showing 1,513 changed files with 502,275 additions and 4 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ SplitFS is under active development.
5. rsync
6. Filebench
7. LMDB
8. RocksDB (with YCSB)

## Testing
[PJD POSIX Test Suite](https://www.tuxera.com/community/posix-test-suite/) that tests primarily the metadata operations was run on SplitFS successfully. SplitFS passes all tests.
Expand Down
15 changes: 15 additions & 0 deletions dependencies/rocskdb_deps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash

sudo apt-get update

sudo apt-get install libgflags-dev

sudo apt-get install libsnappy-dev

sudo apt-get install zlib1g-dev

sudo apt-get install libbz2-dev

sudo apt-get install liblz4-dev

sudo apt-get install libzstd-dev
9 changes: 5 additions & 4 deletions experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ We evaluate and benchmark on SplitFS using different application benchmarks like
* `$ export PATH=$PATH:$JAVA_HOME/bin`
* Check installation using `java -version`
* `$ sudo apt install maven`

3. RocksDB: Upgrade your gcc to version at least 4.8 to get C++11 support. For ubuntu, please run `cd dependencies ./rocksdb_deps.sh; cd..`
If you face any dependency issues, please refer the [doc](https://github.com/utsaslab/SplitFS/blob/master/rocksdb/INSTALL.md#dependencies)
---

### Kernel Setup
Expand Down Expand Up @@ -55,7 +56,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila

### Workload Generation

1. YCSB: `cd scripts/ycsb; ./gen_workloads.sh; cd ../..` -- This will generate the YCSB workload files to be run with LevelDB, because YCSB does not natively support LevelDB, and has been added to the benchmarks of LevelDB
1. YCSB: `cd scripts/ycsb; ./gen_workloads.sh; cd ../..` -- This will generate the YCSB workload files to be run with LevelDB & RocksDB, because YCSB does not natively support LevelDB & RocksDB(C++), and has been added to the benchmarks of LevelDB & RocksDB
2. TPCC: `cd scripts/tpcc; ./gen_workload.sh; cd ../..` -- This will create an initial database on SQLite on which to run the TPCC workload
3. rsync: `cd scripts/rsync/; sudo ./rsync_gen_workload.sh; cd ../..` -- This will create the rsync workload according to the backup data distribution as mentioned in the paper
4. tar: `cd scripts/tar/; sudo ./gen_workload.sh; cd ../..` -- This will create the tar workload as mentioned in the paper
Expand All @@ -65,7 +66,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila

### Run Application Workloads

1. YCSB: `cd scripts/ycsb; ./run_ycsb.sh; cd ../..` -- This will run all the YCSB workloads on LevelDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-strict`
1. YCSB-LevelDB: `cd scripts/ycsb; ./run_ycsb.sh; cd ../..` -- This will run all the YCSB workloads on LevelDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-strict`
2. TPCC: `cd scripts/tpcc; ./run_tpcc.sh; cd ../..` -- This will run the TPCC workload on SQLite3 with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-POSIX`
3. rsync: `cd scripts/rsync; ./run_rsync.sh; cd ../..` -- This will run the rsync workload with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-sync`
4. tar: `cd scripts/tar; ./run_tar.sh; cd ../..` -- This will run the tar workload with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS-POSIX, SplitFS-sync, SplitFS-strict`
Expand All @@ -78,7 +79,7 @@ Note: The <num_threads> argument in the compilation scripts performs the compila
7. Filebench: `cd scripts/filebench; ./run_filebench.sh; cd ../..` --
This will run the varmail workload with `ext4-DAX, NOVA strict,
NOVA Relaxed, PMFS, SplitFS-POSIX`

8. YCSB-RocksDB: `cd scripts/ycsb_rocksdb; ./run_ycsb_rocksdb.sh; cd ../..` -- This will run all the YCSB workloads on RocksDB (Load A, Run A-F, Load E, Run E) with `ext4-DAX, NOVA strict, NOVA Relaxed, PMFS, SplitFS (based on the one found in splitfs/libnvp.so)`
---

### Run Software Overhead Workloads
Expand Down
12 changes: 12 additions & 0 deletions rocksdb/AUTHORS
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Facebook Inc.
Facebook Engineering Team

Google Inc.
# Initial version authors:
Jeffrey Dean <[email protected]>
Sanjay Ghemawat <[email protected]>

# Partial list of contributors:
Kevin Regan <[email protected]>
Johan Bilien <[email protected]>
Matthew Von-Maszewski <https://github.com/matthewvon> (Basho Technologies)
Loading

0 comments on commit b5e8b19

Please sign in to comment.