Help needed: Install Sparkling-water(H20) on Redhat failed

125 views
Skip to first unread message

Xiuxia Zhang

unread,
Apr 15, 2015, 10:34:36 AM4/15/15
to h2os...@googlegroups.com
I use git clone to download code from https://github.com/h2oai/sparkling-water 
My operate system is Red Hat Enterprise Linux Server release 6.5 (Santiago)
I use spark 1.2 

I followed the instructions to install, but I got the following errors.
[Galaxy@blade12 sparkling-water]$ ./gradlew build
> Configuring > 0/5 projects > root project > Resolving dependencies ':classpath'
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script

FAILURE: Build failed with an exception.

* Where:
Build file '/home/Galaxy/zxx/sparkling-water/core/build.gradle' line: 84

* What went wrong:
A problem occurred evaluating project ':sparkling-water-core'.
> Could not download artifact 'h2o-algos.jar (ai.h2o:h2o-algos:0.3.0.1109)'
      > Connection to http://h2o-release.s3.amazonaws.com refused

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 1 mins 22.15 secs

I googled on the internet, but I couldn't any information.  I am very worried.


Parag Sanghavi

unread,
Apr 15, 2015, 1:28:21 PM4/15/15
to Xiuxia Zhang, h2os...@googlegroups.com
Xiuxia,

Are you able to connect to s3 from your network. The error indicates that you might have some firewall issues that prevent you to get to network
Can you do a wget on this URL from a command line and see if the jar downloads ?

Parag


--
You received this message because you are subscribed to the Google Groups "H2O & Open Source Scalable Machine Learning - h2ostream" group.
To unsubscribe from this group and stop receiving emails from it, send an email to h2ostream+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Parag Sanghavi
Head of Customer Success
H2O.ai
(650) 303-4069

Xiuxia Zhang

unread,
Apr 15, 2015, 9:44:34 PM4/15/15
to Parag Sanghavi, h2os...@googlegroups.com
Yes, I could use wget to download  'http://h2o-release.s3.amazonaws.com/h2o-dev/master/1109/maven/repo/ai/h2o/h2o-algos/0.3.0.1109/h2o-algos-0.3.0.1109.jar'.  I don't know why it complains about this. 
--
Zhang Xiuxia
Institute of Computing Techonology  Chinese Academy of Sciences


Michal Malohlava

unread,
Apr 16, 2015, 2:22:22 AM4/16/15
to h2os...@googlegroups.com
Hi Xiuxia,

thanks for trying wget. It seems like kind of misconfiguration in gradle or OS.

May you please to try to run ./gradlew --debug and send us the output?

Thank you!
michal

Dne 4/15/15 v 6:44 PM Xiuxia Zhang napsal(a):

Xiuxia Zhang

unread,
Apr 16, 2015, 4:52:04 AM4/16/15
to mic...@h2o.ai, h2os...@googlegroups.com
In fact, I made a little progress in installing sparkling-water, I close iptables, and tried several times with command: ./gradlew build. I could pass the following error. 

[root@blade12 sparkling-water]# ./gradlew build
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script
Cannot find Spark master - please specify MASTER env variable or pass -PsparkMaster property to gradle script
:sparkling-water-core:compileJava UP-TO-DATE
:sparkling-water-core:compileScala
Error occurred during initialization of VM
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Could not reserve enough space for object heap
Could not write standard input into: Gradle Compiler Daemon 1.
java.io.IOException: Broken pipe
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:50)
at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
:sparkling-water-core:compileScala FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sparkling-water-core:compileScala'.
> Process 'Gradle Compiler Daemon 1' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 21.187 secs

I also tried ./gradlew --debug
There are lots of output. Here are the last few lines of the output.
16:47:23.773 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache artifact-at-repository.bin (/root/.gradle/caches/modules-2/metadata-2.14/artifact-at-repository.bin)
16:47:23.773 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache module-metadata.bin (/root/.gradle/caches/modules-2/metadata-2.14/module-metadata.bin)
16:47:23.773 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Closing cache module-artifacts.bin (/root/.gradle/caches/modules-2/metadata-2.14/module-artifacts.bin)
16:47:23.773 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on artifact cache (/root/.gradle/caches/modules-2).
16:47:23.782 [DEBUG] [org.gradle.cache.internal.DefaultCacheAccess] Cache task history cache (/home/Galaxy/zxx/sparkling-water/.gradle/2.2.1/taskArtifacts) was closed 0 times.
16:47:23.782 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.memcache.InMemoryCachedRepositoryFactory                                                                                           ]                                                                                                        In-memory dependency metadata cache closed. Repos cached: 11, cache instances: 6, modules served from cache: 224, artifacts: 43
16:47:23.784 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.CachedStoreFactor]                                                                                                        Resolution result cache closed. Cache reads: 0, disk reads: 0 (avg: 0.0 secs, total: 0.0 secs)
16:47:23.784 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.CachedStoreFactor]                                                                                                        Resolved configuration cache closed. Cache reads: 0, disk reads: 2 (avg: 0.052 secs, total: 0.105 secs)
16:47:23.784 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.ResolutionResultsStoreFactory                                                                                             ]                                                                                                        Deleted 2 resolution results binary files in 0.0 secs

Thanks very much for you kind help.


--
You received this message because you are subscribed to a topic in the Google Groups "H2O & Open Source Scalable Machine Learning - h2ostream" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/h2ostream/DOwIyiyWf6A/unsubscribe.
To unsubscribe from this group and all its topics, send an email to h2ostream+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

Michal Malohlava

unread,
Apr 21, 2015, 4:31:00 PM4/21/15
to Xiuxia Zhang, h2os...@googlegroups.com
Hi Xiuxia,

the problem is in the following lines;
Dne 4/16/15 v 1:52 AM Xiuxia Zhang napsal(a):
Error occurred during initialization of VM
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Could not reserve enough space for object heap
Could not write standard input into: Gradle Co

What version of Java are you using? If java6, please use java7

What is the size of memory on your machine?
It seems that you do not have enough memory to even invoke build process?

michal

Xiuxia Zhang

unread,
Apr 21, 2015, 11:21:33 PM4/21/15
to mic...@h2o.ai, h2os...@googlegroups.com
I am 
[Galaxy@blade12 ~]$ java -version
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

the total memory on the server is 47GB.

--
Zhang Xiuxia
Institute of Computing Technology  Chinese Academy of Sciences


Michal Malohlava

unread,
Apr 22, 2015, 1:55:10 PM4/22/15
to Xiuxia Zhang, h2os...@googlegroups.com
Dne 4/21/15 v 8:21 PM Xiuxia Zhang napsal(a):
I am 
[Galaxy@blade12 ~]$ java -version
java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)
Looks good. In your environment build should be passing. So something is suspicious there...

Can you please rerun gradle build again with --debug flag and send us the output.

Thank you!
michal
Reply all
Reply to author
Forward
0 new messages