CassandraJavaUtil session leak issue with RDD on with spark 3
9 views
Skip to first unread message
surya goli
Jan 16, 2025, 3:25:37 PMJan 16
to DataStax Spark Connector for Apache Cassandra
Hi Community,
we are currently upgrading the spark version from 2 to 3 along with Cassandra spark connector which was working fine on spark2, but facing issue with spark3 upgrades as session leak, we are using Spark RDD with CassandraJavaUtil to store the data using spark, but after the upgrade it is creating new connection for each batch causing connection overhead. please help solve this issue, i know dsreams solve the issue, but we don't have the flexibility to change the RDD's to Dstreams for CassandraStreamingJavaUtil. Thanks in advance.
we are currently upgrading the spark version from 2 to 3 along with Cassandra spark connector which was working fine on spark2, but facing issue with spark3 upgrades as session leak, we are using Spark RDD with CassandraJavaUtil to store the data using spark, but after the upgrade it is creating new connection for each batch causing connection overhead. please help solve this issue, i know dsreams solve the issue, but we don't have the flexibility to change the RDD's to Dstreams for CassandraStreamingJavaUtil. Thanks in advance.
Regards,
Surya Goli.
Reply all
Reply to author
Forward
0 new messages