Skip to content

Releases: datastax/spark-cassandra-connector

Preview Release 1.2.0 alpha 2

11 Feb 10:39
Compare
Choose a tag to compare
Pre-release

Changes since 1.2.0 alpha 1:

1.2.0 alpha 2

  • All connection properties can be set on SparkConf / CassandraConnectorConf objects and
    the settings are automatically distributed to Spark Executors (SPARKC-28)
  • Report Connector metrics to Spark metrics system (SPARKC-27)
  • Upgraded to Spark 1.2.1 (SPARKC-30)
  • Add conversion from java.util.Date to java.sqlTimestamp for Spark SQL (#512)

Preview Release 1.2.0 alpha 1

16 Jan 17:25
Compare
Choose a tag to compare
Pre-release

1.2.0 alpha 1

  • Added support for TTL and timestamp in the writer (#153)
  • Added support for UDT column types (SPARKC-1)
  • Upgraded Spark to version 1.2.0 (SPARKC-15)
  • For 1.2.0 release, table name with dot is not supported for Spark SQL,
    it will be fixed in the next release
  • Added fast spanBy and spanByKey methods to RDDs useful for grouping Cassandra
    data by partition key / clustering columns. Useful for e.g. time-series data. (SPARKC-2)
  • Refactored the write path so that the writes are now token-aware (SPARKC-5, previously #442)
  • Added support for INSET predicate pushdown (patch by granturing)

Note this is a preview release and it may not be source nor binary compatible with next 1.2 releases.

Release 1.0.6

14 Jan 10:06
Compare
Choose a tag to compare

1.0.6

  • Upgraded Java Driver to 2.0.8 and added some logging in LocalNodeFirstLoadBalancingPolicy (SPARKC-18)

Release 1.1.1

29 Dec 13:50
Compare
Choose a tag to compare

1.1.1

  • Fixed NoSuchElementException in SparkSQL predicate pushdown code (SPARKC-7, #454)

Also includes a fix from 1.0.5:

  • Fixed setting output consistency level which was being set on prepared
    statements instead of being set on batches (#463)

Release 1.0.5

29 Dec 11:13
Compare
Choose a tag to compare

1.0.5

  • Fixed setting output consistency level which was being set on prepared
    statements instead of being set on batches (#463)

1.1.0 Release Candidate 3

24 Nov 11:53
Compare
Choose a tag to compare

1.1.0 rc 3

  • Fix NPE when saving CassandraRows containing null values (#446)

Release 1.1.0

24 Nov 19:11
Compare
Choose a tag to compare

Changes since 1.0.x:

1.1.0

  • Switch to java driver 2.1.3 and Guava 14.0.1 (yay!).

1.1.0 rc 3

  • Fix NPE when saving CassandraRows containing null values (#446)

1.1.0 rc 2

  • Added JavaTypeConverter to make is easy to implement custom TypeConverter in Java (#429)
  • Fix SparkSQL failures caused by presence of non-selected columns of UDT type in the table.

1.1.0 rc 1

  • Fixed problem with setting a batch size in bytes (#435)
  • Fixed handling of null column values in Java API (#429)

1.1.0 beta 2

  • Fixed bug in Java API which might cause ClassNotFoundException
  • Added stubs for UDTs. It is possible to read tables with UDTs, but
    values of UDTs will come out as java driver UDTValue objects (#374)
  • Upgraded Java driver to 2.1.2 version and fixed deprecation warnings.
    Use correct protocolVersion when serializing/deserializing Cassandra columns.
  • Don't fail with "contact points contain multiple datacenters"
    if one or more of the nodes given as contact points don't have DC information,
    because they are unreachable.
  • Removed annoying slf4j warnings when running tests (#395)
  • CassandraRDD is fully lazy now - initialization no longer fetches Cassandra
    schema (#339).

1.1.0 beta 1

  • Redesigned Java API, some refactorings (#300)
  • Simplified AuthConf - more responsibility on CassandraConnectionFactory
  • Enhanced and improved performance of the embedded Kafka framework
    • Kafka consumer and producer added that are configurable
    • Kafka shutdown cleaned up
    • Kafka server more configurable for speed and use cases
  • Added new word count demo and a new Kafka streaming word count demo
  • Modified build file to allow easier module id for usages of 'sbt project'

1.1.0 alpha 4

  • Use asynchronous prefetching of multi-page ResultSets in CassandraRDD
    to reduce waiting for Cassandra query results.
  • Make token range start and end be parameters of the query, not part of the query
    template to reduce the number of statements requiring preparation.
  • Added type converter for GregorianCalendar (#334)

1.1.0 alpha 3

  • Pluggable mechanism for obtaining connections to Cassandra
    Ability to pass custom CassandraConnector to CassandraRDDs (#192)
  • Provided a row reader which allows to create RDDs of pairs of objects as well
    as RDDs of simple objects handled by type converter directly;
    added meaningful compiler messages when invalid type was provided (#88)
  • Fixed serialization problem in CassandraSQLContext by making conf transient (#310)
  • Cleaned up the SBT assembly task and added build documentation (#315)

1.1.0 alpha 2

  • Upgraded Apache Spark to 1.1.0.
  • Upgraded to be Cassandra 2.1.0 and Cassandra 2.0 compatible.
  • Added spark.cassandra.connection.local_dc option
  • Added spark.cassandra.connection.timeout_ms option
  • Added spark.cassandra.read.timeout_ms option
  • Added support for SparkSQL (#197)
  • Fixed problems with saving DStreams to Cassandra directly (#280)

1.1.0 alpha 1

  • Add an ./sbt/sbt script (like with spark) so people don't need to install sbt
  • Replace internal spark Logging with own class (#245)
  • Accept partition key predicates in CassandraRDD#where. (#37)
  • Add indexedColumn to ColumnDef (#122)
  • Upgrade Spark to version 1.0.2
  • Removed deprecated toArray, replaced with collect.
  • Updated imports to org.apache.spark.streaming.receiver
    and import org.apache.spark.streaming.receiver.ActorHelper
  • Updated streaming demo and spec for Spark 1.0.2 behavior compatibility
  • Added new StreamingEvent types for Spark 1.0.2 Receiver readiness
  • Added the following Spark Streaming dependencies to the demos module:
    Kafka, Twitter, ZeroMQ
  • Added embedded Kafka and ZooKeeper servers for the Kafka Streaming demo
    • keeping non private for user prototyping
  • Added new Kafka Spark Streaming demo which reads from Kafka
    and writes to Cassandra (Twitter and ZeroMQ are next)
  • Added new 'embedded' module
    • Refactored the 'connector' module's IT SparkRepl, CassandraServer and
      CassandraServerRunner as well as 'demos' EmbeddedKafka
      and EmbeddedZookeeper to the 'embedded' module. This allows the 'embedded'
      module to be used as a dependency by the 'connector' IT tests, demos,
      and user local quick prototyping without requiring a Spark and Cassandra
      Cluster, local or remote, to get started.

1.1.0 Release Candidate 2

24 Nov 11:54
Compare
Choose a tag to compare

1.1.0 rc 2

  • Added JavaTypeConverter to make is easy to implement custom TypeConverter in Java (#429)
  • Fix SparkSQL failures caused by presence of non-selected columns of UDT type in the table.

1.1.0 Release Candidate 1

20 Nov 21:11
Compare
Choose a tag to compare

Changes since 1.1.0-beta2:

  • Fixed problem with setting a batch size in bytes (#435)
  • Fixed handling of null column values in Java API (#429)

v1.1.0-beta2

12 Nov 16:35
Compare
Choose a tag to compare
v1.1.0-beta2 Pre-release
Pre-release

1.1.0 beta 2

  • Fixed bug in Java API which might cause ClassNotFoundException
  • Added stubs for UDTs. It is possible to read tables with UDTs, but
    values of UDTs will come out as java driver UDTValue objects (#374)
  • Upgraded Java driver to 2.1.2 version and fixed deprecation warnings.
    Use correct protocolVersion when serializing/deserializing Cassandra columns.
  • Don't fail with "contact points contain multiple datacenters"
    if one or more of the nodes given as contact points don't have DC information,
    because they are unreachable.
  • Removed annoying slf4j warnings when running tests (#395)
  • CassandraRDD is fully lazy now - initialization no longer fetches Cassandra
    schema (#339).

Also includes changes released in 1.0.4:

1.0.4

  • Synchronized TypeConverter.forType methods to workaround some Scala 2.10
    reflection thread-safety problems (#235)
  • Synchronized computation of TypeTags in TypeConverter#targetTypeTag,.
    ColumnType#scalaTypeTag methods and other places to workaround some of.
    Scala 2.10 reflection thread-safety problems (#364)
  • Downgraded Guava to version 14.
    Upgraded Java driver to 2.0.7.
    Upgraded Cassandra to 2.0.11. (#366)
  • Made SparkContext variable transient in SparkContextFunctions (#373)
  • Fixed saving to tables with uppercase column names (#377)
  • Fixed saving collections of Tuple1 (#420)