[JIRA] (SPARKR-236) Can not import SparkR

73 views
Skip to first unread message

Antonio Piccolboni (JIRA)

unread,
Mar 19, 2015, 2:38:01 PM3/19/15
to spark...@googlegroups.com
Antonio Piccolboni created an issue
 
SparkR / Bug SPARKR-236
Can not import SparkR
Issue Type: Bug Bug
Assignee: Unassigned
Created: 19/Mar/15 11:37 AM
Priority: Major Major
Reporter: Antonio Piccolboni

Hi
it appears that in the latest master something necessary to the working of SparkR happens when attaching it. If someone imports the package or uses is without attaching it, for instance with the :: syntax, some classes are not found. There is a message during package load that I think may give away where the problem is.

> sc = SparkR::sparkR.init()
Launching java with command /Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java -Xmx512m -cp '' edu.berkeley.cs.amplab.sparkr.SparkRBackend /var/folders/_p/1gx4vy311_x4syn2xq6f2xtc0000gr/T//RtmpbjJinh/backend_port132d07b8f819d
Error: Could not find or load main class edu.berkeley.cs.amplab.sparkr.SparkRBackend
Error in SparkR::sparkR.init() : JVM is not ready after 10 seconds
> library(SparkR)
[SparkR] Initializing with classpath /Users/antonio/Library/R/3.1/library/SparkR/sparkr-assembly-0.1.jar

Attaching package: ‘SparkR’

The following object is masked from ‘package:plyrmr’:

count

The following objects are masked from ‘package:rmr2’:

keys, values

The following objects are masked from ‘package:dplyr’:

collect, count, distinct, first

> sc = SparkR::sparkR.init()
Launching java with command /Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java -Xmx512m -cp '/Users/antonio/Library/R/3.1/library/SparkR/sparkr-assembly-0.1.jar:' edu.berkeley.cs.amplab.sparkr.SparkRBackend /var/folders/_p/1gx4vy311_x4syn2xq6f2xtc0000gr/T//RtmpbjJinh/backend_port132d0a86a9cb
15/03/19 11:22:47 INFO Slf4jLogger: Slf4jLogger started

Works swimmingly from here

Built yesterday from e24b1ad6

Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v6.4-OD-15-055#64014-sha1:797c660)
Atlassian logo

Shivaram Venkataraman (JIRA)

unread,
Mar 19, 2015, 2:51:01 PM3/19/15
to spark...@googlegroups.com
Shivaram Venkataraman commented on Bug SPARKR-236
 
Re: Can not import SparkR

So we need to pick up the install path of the SparkR package to figure out where the JAR file is – This happens right now in sparkR.onLoad https://github.com/amplab-extras/SparkR-pkg/blob/master/pkg/R/sparkR.R#L5 which is called by .onAttach https://github.com/amplab-extras/SparkR-pkg/blob/master/pkg/R/zzz.R#L1

If we change this to be called .onLoad , will that solve your use case ?

Antonio Piccolboni (JIRA)

unread,
Mar 19, 2015, 3:14:02 PM3/19/15
to spark...@googlegroups.com

Thanks for your quick reply. The answer is yes in theory and I also just tested it.

Shivaram Venkataraman (JIRA)

unread,
Mar 19, 2015, 3:18:01 PM3/19/15
to spark...@googlegroups.com

Cool - Feel free to send a PR if you get a chance. Or I'll fix this later tonight.

Antonio Piccolboni (JIRA)

unread,
Mar 19, 2015, 5:14:02 PM3/19/15
to spark...@googlegroups.com

Shivaram Venkataraman (JIRA)

unread,
Mar 19, 2015, 5:51:02 PM3/19/15
to spark...@googlegroups.com
Change By: Shivaram Venkataraman
Assignee: Antonio Piccolboni

Shivaram Venkataraman (JIRA)

unread,
Mar 19, 2015, 5:52:01 PM3/19/15
to spark...@googlegroups.com
Shivaram Venkataraman resolved as Fixed
Change By: Shivaram Venkataraman
Resolution: Fixed
Status: Open Resolved

Shivaram Venkataraman (JIRA)

unread,
Mar 19, 2015, 5:52:01 PM3/19/15
to spark...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages