Skip to content

Commit

Permalink
Update Docs to Point to Working Artifact Repos
Browse files Browse the repository at this point in the history
Looks like Spark Packages is now dead, so lets point to Maven Central
instead.
  • Loading branch information
RussellSpitzer committed Nov 21, 2019
1 parent 92be24c commit 29e31d3
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 8 deletions.
11 changes: 5 additions & 6 deletions doc/0_quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,16 @@ Install and launch a Cassandra cluster and a Spark cluster.

Configure a new Scala project with the Apache Spark and dependency.

The dependencies are easily retrieved via the spark-packages.org website. For example, if you're using `sbt`, your build.sbt should include something like this:
The dependencies are easily retrieved via Maven Central

resolvers += "Spark Packages Repo" at "https://dl.bintray.com/spark-packages/maven"
libraryDependencies += "datastax" % "spark-cassandra-connector" % "2.4.1-s_2.11"
libraryDependencies += "com.datastax.spark" % "spark-cassandra-connector" % "2.4.1-s_2.11"

The spark-packages libraries can also be used with spark-submit and spark shell, these
commands will place the connector and all of its dependencies on the path of the
Spark Driver and all Spark Executors.

$SPARK_HOME/bin/spark-shell --packages datastax:spark-cassandra-connector:2.4.1-s_2.11
$SPARK_HOME/bin/spark-submit --packages datastax:spark-cassandra-connector:2.4.1-s_2.11
$SPARK_HOME/bin/spark-shell --packages com.datastax.spark:spark-cassandra-connector_2.11:2.4.1
$SPARK_HOME/bin/spark-submit --packages com.datastax.spark:spark-cassandra-connector_2.11:2.4.1

For the list of available versions, see:
- https://spark-packages.org/package/datastax/spark-cassandra-connector
Expand Down Expand Up @@ -59,7 +58,7 @@ Run the `spark-shell` with the packages line for your version. To configure
the default Spark Configuration pass key value pairs with `--conf`

$SPARK_HOME/bin/spark-shell --conf spark.cassandra.connection.host=127.0.0.1 \
--packages datastax:spark-cassandra-connector:2.4.1-s_2.11
--packages com.datastax.spark:spark-cassandra-connector_2.11:2.4.1

This command would set the Spark Cassandra Connector parameter
`spark.cassandra.connection.host` to `127.0.0.1`. Change this
Expand Down
2 changes: 1 addition & 1 deletion doc/13_spark_shell.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Find additional versions at [Spark Packages](https://spark-packages.org/package/
```bash
cd spark/install/dir
#Include the --master if you want to run against a spark cluster and not local mode
./bin/spark-shell [--master sparkMasterAddress] --jars yourAssemblyJar --packages datastax:spark-cassandra-connector:2.4.1-s_2.11 --conf spark.cassandra.connection.host=yourCassandraClusterIp
./bin/spark-shell [--master sparkMasterAddress] --jars yourAssemblyJar --packages com.datastax.spark:spark-cassandra-connector_2.11:2.4.1 --conf spark.cassandra.connection.host=yourCassandraClusterIp
```

By default spark will log everything to the console and this may be a bit of an overload. To change this copy and modify the `log4j.properties` template file
Expand Down
2 changes: 1 addition & 1 deletion doc/15_python.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ https://spark-packages.org/package/datastax/spark-cassandra-connector

```bash
./bin/pyspark \
--packages com.datastax.spark:spark-cassandra-connector_2.11:2.3.2
--packages com.datastax.spark:spark-cassandra-connector_2.11:2.4.1
```

### Loading a DataFrame in Python
Expand Down

0 comments on commit 29e31d3

Please sign in to comment.