question about uploading objects to worker nodes
30 views
Skip to first unread message
stewart
May 16, 2012, 10:28:24 AM5/16/12
to Segue for R
Hi James -
I am new user of Segue and I have a question about rObjectsOnNodes. As
I understand it, these are objects passed into the clusters when you
first create them. But is there a way to pass more objects after the
creation of the clusters? For example, suppose I want to do the
following:
result1 <- emrlapply(myCluster, seedList, myfun1)
bigObject <- somefun(result1,...)
result2 <- emrlapply(myCluster, seeList2, myfun2, bigObject)
In this case bigObject is generated after myCluster was created and is
used by myfun2. Is this possible?
I am new user of Segue and I have a question about rObjectsOnNodes. As
I understand it, these are objects passed into the clusters when you
first create them. But is there a way to pass more objects after the
creation of the clusters? For example, suppose I want to do the
following:
result1 <- emrlapply(myCluster, seedList, myfun1)
bigObject <- somefun(result1,...)
result2 <- emrlapply(myCluster, seeList2, myfun2, bigObject)
In this case bigObject is generated after myCluster was created and is
used by myfun2. Is this possible?
James Long
May 17, 2012, 8:06:17 AM5/17/12
to seg...@googlegroups.com
Hey Stewart. The use case you lay out is exactly what I wish I could
do with Segue. The problem is, I have not committed the time to figure
out how to upload files to existing nodes using the Amazon API. So, in
short, that isn't possible right now. However if you or someone else
hacked up a solution, I'd love to include it in Segue!
-J
do with Segue. The problem is, I have not committed the time to figure
out how to upload files to existing nodes using the Amazon API. So, in
short, that isn't possible right now. However if you or someone else
hacked up a solution, I'd love to include it in Segue!
-J
Reply all
Reply to author
Forward
0 new messages