Apache Spark JavaRDD pipe() need help

[Shashi]

Hi All,

I am trying to use the JavaRDD.pipe() API.

I have one object with me from the JavaRDD<Object> and not the complete RDD. I mean I am operating on one object inside the RDD.  In my object I have some attribute values using which I create one string like "param1 param2 param3 param4". I have one C binary file with me which does some complex mathematic calculations. Now I want to invoke the C binary using JavaRDD.pipe() API but I do not have the RDD with me. I just have a string which I want to  pass to the C binary.

How do I do this? My code is not in driver program. It is in some Java class which is part of JavaRDD.

In driver.java class
{
 //create config
 //create context
  // generate RDD using data from Cassandra
 // Do a map operation on RDD and for each object in RDD do some operation in helper Java class
 //Get the updated objects and save them back in Cassandra
}

In Helper.java class
{
//Get the data from object
//do some processing
//get some attributes string values
//Create one string out of those attributes
//Now invoke the C library and pass this one string as input parameter  <---How to do this
//Read the output and update he object
}

Let me know if you need more inputs from me.

Thanks in advance. 
Shashi

[Olivier Michallat]

Hi,

This mailing list is for the core Java driver. It looks like your question is more related to the Spark Cassandra connector, or Spark itself. I would suggest trying one of the following lists:

Spark Cassandra Connector: http://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user

Apache Spark: http://spark.apache.org/community.html

--

Olivier Michallat

Driver & tools engineer, DataStax

--
You received this message because you are subscribed to the Google Groups "DataStax Java Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-driver-user+unsubscribe@lists.datastax.com.