Skip to content

Commit

Permalink
Update the README with instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
JulienPeloton committed Feb 8, 2019
1 parent ef86df3 commit c024c70
Showing 1 changed file with 28 additions and 2 deletions.
30 changes: 28 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,28 @@
# 1807.03078
Supplementary material for arXiv:1807.03078
# Analysing billion-objects catalogue interactively: Apach Spark for physicists

This repository contains supplementary material for arXiv:[1807.03078](https://arxiv.org/abs/1807.03078).

## How to run the notebook

You must have [Apache Spark](http://spark.apache.org/) and [Jupyter notebook](https://jupyter.org/) installed on your machine or your cluster.

### On a local machine

```bash
PACK="com.github.astrolabsoftware:spark-fits_2.11:0.7.2"
PYSPARK_DRIVER_PYTHON_OPTS="jupyter-notebook" pyspark \
--master local[*] \
--packages $PACK
```

### On a cluster

Standalone mode:

```bash
PACK="com.github.astrolabsoftware:spark-fits_2.11:0.7.2"
PYSPARK_DRIVER_PYTHON_OPTS="jupyter-notebook --debug --no-browser --port=$PORT1" pyspark \
--master $SPARKURL \
--packages $PACK \
--driver-memory $MEMDRIVER --executor-memory $MEMEXEC --executor-cores $EXECCORES --total-executor-cores $TOTALCORES
```

0 comments on commit c024c70

Please sign in to comment.