error in plyrmr + SparkR example

[Jong Hyun Bae]

Hi guys. I have some problem in my example codes.

In console
----
$SPARKR_PATH/sparkR


In R
----
> library(plyrmr)
> plyrmr.options(backend="spark")
> write.csv(mtcars, "mtcars.csv")
> output(where(input("mtcars.csv", "csv"), cyl==8), path="mtcars.output", output="json", input.format="R.serialize")

This is my example code that find rows about "cyl==8" conditions.

However it show error messages that

Error in FUN(part) : could not find function "rdd.list2kv"
Calls: <Anonymous> -> func -> FUN -> FUN

I work this code in my own device environment.

 - ubuntu 12.04
 - R 3.2.0
 - plyrmr 0.6.0
 - SparkR 0.1
 - spark 1.2.1

I saw almost same error in SparkR issue pages at here (https://sparkr.atlassian.net/browse/SPARKR-54) but still in my device, this error occurs.
Any advice or idea to solve this problem for me?

[Antonio Piccolboni]

I work on plyrmr and I am the reporter of the issue you linked to. Not only the SparkR maintainers are not hurrying to fix this bug, but they are retiring most of the SparkR package, including crucial calls needed by plyrmr on spark. There's a whole new API that could be used to implement a new plyrmr backend and some SparkR functionality may be brought back in Spark 1.5, but it's all very uncertain at this point. While I appreciate your interest in this package and its newest backend, at this time I am afraid it is a waste of your time to try and make it work. If it's an absolute must I can tell you how far back you have to go in SparkR to see plyrmr work again. You may have to downgrade Spark as well. It's clearly not a viable path for most users and a dead end for the future. Plyrmr continues to work on the Mapreduce and local backends. Sorry I didn't make an announcement earlier, I was hoping to have more clarity before writing.

--
post: rha...@googlegroups.com ||
unsubscribe: rhadoop+u...@googlegroups.com ||
web: https://groups.google.com/d/forum/rhadoop?hl=en-US
---
You received this message because you are subscribed to the Google Groups "RHadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhadoop+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jong Hyun Bae]

Thanks for your fast comment!!

[Antonio Piccolboni]

You bet. I don't normally mark my own answers as best answers, but since nobody does and I need to make sure I've taken care of a thread, from now on I will do some of the marking when there is some evidence the request was satisfied -- as in the case of your kind thank you note. But I'd prefer if the original poster did the marking by themselves, that would be less  error prone.