Releases: datastax/spark-cassandra-connector
Releases · datastax/spark-cassandra-connector
Preview Release 1.2.0 alpha 2
Changes since 1.2.0 alpha 1:
1.2.0 alpha 2
- All connection properties can be set on SparkConf / CassandraConnectorConf objects and
the settings are automatically distributed to Spark Executors (SPARKC-28) - Report Connector metrics to Spark metrics system (SPARKC-27)
- Upgraded to Spark 1.2.1 (SPARKC-30)
- Add conversion from java.util.Date to java.sqlTimestamp for Spark SQL (#512)
Preview Release 1.2.0 alpha 1
1.2.0 alpha 1
- Added support for TTL and timestamp in the writer (#153)
- Added support for UDT column types (SPARKC-1)
- Upgraded Spark to version 1.2.0 (SPARKC-15)
- For 1.2.0 release, table name with dot is not supported for Spark SQL,
it will be fixed in the next release - Added fast spanBy and spanByKey methods to RDDs useful for grouping Cassandra
data by partition key / clustering columns. Useful for e.g. time-series data. (SPARKC-2) - Refactored the write path so that the writes are now token-aware (SPARKC-5, previously #442)
- Added support for INSET predicate pushdown (patch by granturing)
Note this is a preview release and it may not be source nor binary compatible with next 1.2 releases.
Release 1.0.6
1.0.6
- Upgraded Java Driver to 2.0.8 and added some logging in LocalNodeFirstLoadBalancingPolicy (SPARKC-18)
Release 1.1.1
1.1.1
- Fixed NoSuchElementException in SparkSQL predicate pushdown code (SPARKC-7, #454)
Also includes a fix from 1.0.5:
- Fixed setting output consistency level which was being set on prepared
statements instead of being set on batches (#463)
Release 1.0.5
1.0.5
- Fixed setting output consistency level which was being set on prepared
statements instead of being set on batches (#463)
1.1.0 Release Candidate 3
1.1.0 rc 3
- Fix NPE when saving CassandraRows containing null values (#446)
Release 1.1.0
Changes since 1.0.x:
1.1.0
- Switch to java driver 2.1.3 and Guava 14.0.1 (yay!).
1.1.0 rc 3
- Fix NPE when saving CassandraRows containing null values (#446)
1.1.0 rc 2
- Added JavaTypeConverter to make is easy to implement custom TypeConverter in Java (#429)
- Fix SparkSQL failures caused by presence of non-selected columns of UDT type in the table.
1.1.0 rc 1
- Fixed problem with setting a batch size in bytes (#435)
- Fixed handling of null column values in Java API (#429)
1.1.0 beta 2
- Fixed bug in Java API which might cause ClassNotFoundException
- Added stubs for UDTs. It is possible to read tables with UDTs, but
values of UDTs will come out as java driver UDTValue objects (#374) - Upgraded Java driver to 2.1.2 version and fixed deprecation warnings.
Use correct protocolVersion when serializing/deserializing Cassandra columns. - Don't fail with "contact points contain multiple datacenters"
if one or more of the nodes given as contact points don't have DC information,
because they are unreachable. - Removed annoying slf4j warnings when running tests (#395)
- CassandraRDD is fully lazy now - initialization no longer fetches Cassandra
schema (#339).
1.1.0 beta 1
- Redesigned Java API, some refactorings (#300)
- Simplified AuthConf - more responsibility on CassandraConnectionFactory
- Enhanced and improved performance of the embedded Kafka framework
- Kafka consumer and producer added that are configurable
- Kafka shutdown cleaned up
- Kafka server more configurable for speed and use cases
- Added new word count demo and a new Kafka streaming word count demo
- Modified build file to allow easier module id for usages of 'sbt project'
1.1.0 alpha 4
- Use asynchronous prefetching of multi-page ResultSets in CassandraRDD
to reduce waiting for Cassandra query results. - Make token range start and end be parameters of the query, not part of the query
template to reduce the number of statements requiring preparation. - Added type converter for GregorianCalendar (#334)
1.1.0 alpha 3
- Pluggable mechanism for obtaining connections to Cassandra
Ability to pass custom CassandraConnector to CassandraRDDs (#192) - Provided a row reader which allows to create RDDs of pairs of objects as well
as RDDs of simple objects handled by type converter directly;
added meaningful compiler messages when invalid type was provided (#88) - Fixed serialization problem in CassandraSQLContext by making conf transient (#310)
- Cleaned up the SBT assembly task and added build documentation (#315)
1.1.0 alpha 2
- Upgraded Apache Spark to 1.1.0.
- Upgraded to be Cassandra 2.1.0 and Cassandra 2.0 compatible.
- Added spark.cassandra.connection.local_dc option
- Added spark.cassandra.connection.timeout_ms option
- Added spark.cassandra.read.timeout_ms option
- Added support for SparkSQL (#197)
- Fixed problems with saving DStreams to Cassandra directly (#280)
1.1.0 alpha 1
- Add an ./sbt/sbt script (like with spark) so people don't need to install sbt
- Replace internal spark Logging with own class (#245)
- Accept partition key predicates in CassandraRDD#where. (#37)
- Add indexedColumn to ColumnDef (#122)
- Upgrade Spark to version 1.0.2
- Removed deprecated toArray, replaced with collect.
- Updated imports to org.apache.spark.streaming.receiver
and import org.apache.spark.streaming.receiver.ActorHelper - Updated streaming demo and spec for Spark 1.0.2 behavior compatibility
- Added new StreamingEvent types for Spark 1.0.2 Receiver readiness
- Added the following Spark Streaming dependencies to the demos module:
Kafka, Twitter, ZeroMQ - Added embedded Kafka and ZooKeeper servers for the Kafka Streaming demo
- keeping non private for user prototyping
- Added new Kafka Spark Streaming demo which reads from Kafka
and writes to Cassandra (Twitter and ZeroMQ are next) - Added new 'embedded' module
- Refactored the 'connector' module's IT SparkRepl, CassandraServer and
CassandraServerRunner as well as 'demos' EmbeddedKafka
and EmbeddedZookeeper to the 'embedded' module. This allows the 'embedded'
module to be used as a dependency by the 'connector' IT tests, demos,
and user local quick prototyping without requiring a Spark and Cassandra
Cluster, local or remote, to get started.
- Refactored the 'connector' module's IT SparkRepl, CassandraServer and
1.1.0 Release Candidate 2
1.1.0 rc 2
- Added JavaTypeConverter to make is easy to implement custom TypeConverter in Java (#429)
- Fix SparkSQL failures caused by presence of non-selected columns of UDT type in the table.
1.1.0 Release Candidate 1
Changes since 1.1.0-beta2:
- Fixed problem with setting a batch size in bytes (#435)
- Fixed handling of null column values in Java API (#429)
v1.1.0-beta2
1.1.0 beta 2
- Fixed bug in Java API which might cause ClassNotFoundException
- Added stubs for UDTs. It is possible to read tables with UDTs, but
values of UDTs will come out as java driver UDTValue objects (#374) - Upgraded Java driver to 2.1.2 version and fixed deprecation warnings.
Use correct protocolVersion when serializing/deserializing Cassandra columns. - Don't fail with "contact points contain multiple datacenters"
if one or more of the nodes given as contact points don't have DC information,
because they are unreachable. - Removed annoying slf4j warnings when running tests (#395)
- CassandraRDD is fully lazy now - initialization no longer fetches Cassandra
schema (#339).
Also includes changes released in 1.0.4:
1.0.4
- Synchronized TypeConverter.forType methods to workaround some Scala 2.10
reflection thread-safety problems (#235) - Synchronized computation of TypeTags in TypeConverter#targetTypeTag,.
ColumnType#scalaTypeTag methods and other places to workaround some of.
Scala 2.10 reflection thread-safety problems (#364) - Downgraded Guava to version 14.
Upgraded Java driver to 2.0.7.
Upgraded Cassandra to 2.0.11. (#366) - Made SparkContext variable transient in SparkContextFunctions (#373)
- Fixed saving to tables with uppercase column names (#377)
- Fixed saving collections of Tuple1 (#420)