hivecleaner is stopped (Single Machine EC2 Installtion) 0.10

[vikra...@gmail.com]

I tried installing hopsworks 0.10 AMI.I am not able to login.I see hivecleaner is not running, below is the log for reference.

2019-07-16T18:50:05 <info> (Notifier.cpp:73:connect_to_cluster) Connection string 127.0.0.1
2019-07-16T18:50:06 <info> (Notifier.cpp:35:start) hiveCleaner starting...
2019-07-16T18:50:06 <error> (TableTailer.cpp:83:createListenerEvent) Event creation failed, event exists, dropping Event...
2019-07-16T18:50:06 <info> (TableTailer.cpp:101:waitForEvents) create EventOperation for [tail-SDS]
2019-07-16T18:50:06 <info> (TableTailer.cpp:114:waitForEvents) Execute
2019-07-16T18:50:06 <error> (TableTailer.cpp:83:createListenerEvent) Event creation failed, event exists, dropping Event...
2019-07-16T18:50:06 <info> (TableTailer.cpp:101:waitForEvents) create EventOperation for [tail-SKEWED_VALUES]
2019-07-16T18:50:06 <info> (TableTailer.cpp:114:waitForEvents) Execute
2019-07-16T18:50:06 <error> (TableTailer.cpp:83:createListenerEvent) Event creation failed, event exists, dropping Event...
2019-07-16T18:50:06 <info> (TableTailer.cpp:101:waitForEvents) create EventOperation for [tail-SKEWED_COL_VALUE_LOC_MAP]
2019-07-16T18:50:06 <info> (TableTailer.cpp:114:waitForEvents) Execute
2019-07-16T18:50:06 <error> (TableTailer.cpp:83:createListenerEvent) Event creation failed, event exists, dropping Event...
2019-07-16T18:50:06 <fatal> (TableTailer.cpp:84:createListenerEvent) code:4710, msg: Event not found.

[Theofilos Kakantousis]

Hi,

Which credentials are you using to log in? Credentials for 0.10 have changed to "ad...@hopsworks.ai" - "admin".

https://hopsworks.readthedocs.io/en/0.10/getting_started/setups/hopsworks_ami.html

For related error you can tail the glassfish log at "/srv/hops/domains/domain1/logs/server.log"

[vikra...@gmail.com]

Thanks for Reply

I am not able to reach till login page. I see obama page only.
Why hivecleaner is not running ?
please see the log below ..looks like some file missing and ssl error

tail -f /srv/hops/domains/domain1/logs/server.log

[#|2019-07-16T11:38:50.960+0000|WARNING|Payara 4.1|javax.enterprise.network.config|_ThreadID=28;_ThreadName=http-thread-pool::http-listener-2(5);_TimeMillis=1563277130960;_LevelValue=900;|GRIZZLY0053: Unrecognized cipher [TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA].|#]

[#|2019-07-16T11:38:50.961+0000|WARNING|Payara 4.1|org.glassfish.grizzly.filterchain.DefaultFilterChain|_ThreadID=28;_ThreadName=http-thread-pool::http-listener-2(5);_TimeMillis=1563277130961;_LevelValue=900;|GRIZZLY0013: Exception during FilterChain execution
java.lang.NullPointerException
at org.glassfish.grizzly.ssl.SSLEngineConfigurator.createSSLEngine(SSLEngineConfigurator.java:214)
at org.glassfish.grizzly.ssl.SSLEngineConfigurator.createSSLEngine(SSLEngineConfigurator.java:186)

[#|2019-07-16T11:38:50.988+0000|WARNING|Payara 4.1|javax.enterprise.network.config|_ThreadID=24;_ThreadName=http-thread-pool::http-listener-2(1);_TimeMillis=1563277130988;_LevelValue=900;|GRIZZLY0050: SSL support could not be configured!
java.io.IOException: A MultiException has 4 exceptions. They are:
1. java.lang.IllegalStateException: java.io.FileNotFoundException: /srv/hops/domains/domain1/config/keystore.jks (No such file or directory)
2. java.lang.IllegalStateException: Unable to perform operation: create on com.sun.enterprise.security.ssl.impl.SecuritySupportImpl
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of com.sun.enterprise.security.ssl.SSLUtils errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on com.sun.enterprise.security.ssl.SSLUtils

Vikram

[Theofilos Kakantousis]

Looks like you did not run the "./start-services.sh" or the script failed. Can you try running it again? Instructions are at the bottom of the page

https://hopsworks.readthedocs.io/en/0.10/getting_started/setups/hopsworks_ami.html

[Kumar R]

I started so many times.I am planning to relaunch now.

is there in any specific requirement while selecting ec2 instance ?

I am using 50 GB,t2.xlarge 

Vikram

--
You received this message because you are subscribed to a topic in the Google Groups "Hops" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hopshadoop/tyBs4JFAFR4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to hopshadoop+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hopshadoop/289a6efd-6404-4030-a61a-beace245f5cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kumar R]

While running 

./start-services.sh 

I see below warning.

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /srv/hops/domains/domain1/config/keystore.jks -destkeystore /srv/hops/domains/domain1/config/keystore.jks -deststoretype pkcs12".
Certificate stored in file </tmp/cert.pem>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /srv/hops/domains/domain1/config/keystore.jks -destkeystore /srv/hops/domains/domain1/config/keystore.jks -deststoretype pkcs12".
Certificate was added to keystore

[Kumar R]

looks llike 0.10 AMI is broken.Is 0.10 AMI working fine ? 

I tried manually starting hivecleaner but  no success.

ubuntu@ip-172-31-8-191:~$ sudo systemctl start hivecleaner
ubuntu@ip-172-31-8-191:~$ systemctl status hivecleaner
● hivecleaner.service - Hive cleaner
   Loaded: loaded (/lib/systemd/system/hivecleaner.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2019-07-17 11:11:26 UTC; 18s ago
  Process: 16457 ExecStart=/srv/hops/hivecleaner/start-hivecleaner.sh (code=exited, status=0/SUCCESS)
 Main PID: 16459 (code=exited, status=1/FAILURE)

Jul 17 11:11:25 ip-172-31-8-191 systemd[1]: Starting Hive cleaner...
Jul 17 11:11:25 ip-172-31-8-191 systemd[1]: Started Hive cleaner.
Jul 17 11:11:26 ip-172-31-8-191 systemd[1]: hivecleaner.service: Main process exited, code=exited, status=1/FAILURE
Jul 17 11:11:26 ip-172-31-8-191 systemd[1]: hivecleaner.service: Unit entered failed state.
Jul 17 11:11:26 ip-172-31-8-191 systemd[1]: hivecleaner.service: Failed with result 'exit-code'.
ubuntu@ip-172-31-8-191:~$ date
Wed Jul 17 11:12:00 UTC 2019

[Theofilos Kakantousis]

Hi Vikram,

The warning you get for jks is expected behavior, doesn't affect anything. Same as the hive-cleaner, which is an known issue and will be fixed in the next version. Still, hive-cleaner does not affect the rest of the services.

I just launched again the 0.10 in the London region and worked fine. Other users have successfully used the 0.10 AMI. Which region are you trying on?

Have you tried clicking on the link of the Obama page? The link text is "Spark up again, by going back to the login page!". I have attached a screenshot as well.

If that doesn't work, can you send me by email the IP of your instance? 

Thanks,

Theo

 

On Wednesday, July 17, 2019 at 1:15:01 PM UTC+2, Kumar R wrote:

looks llike 0.10 AMI is broken.Is 0.10 AMI working fine ? 

I tried manually starting hivecleaner but  no success.

ubuntu@ip-172-31-8-191:~$ sudo systemctl start hivecleaner
ubuntu@ip-172-31-8-191:~$ systemctl status hivecleaner
● hivecleaner.service - Hive cleaner
   Loaded: loaded (/lib/systemd/system/hivecleaner.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2019-07-17 11:11:26 UTC; 18s ago
  Process: 16457 ExecStart=/srv/hops/hivecleaner/start-hivecleaner.sh (code=exited, status=0/SUCCESS)
 Main PID: 16459 (code=exited, status=1/FAILURE)

Jul 17 11:11:25 ip-172-31-8-191 systemd[1]: Starting Hive cleaner...
Jul 17 11:11:25 ip-172-31-8-191 systemd[1]: Started Hive cleaner.
Jul 17 11:11:26 ip-172-31-8-191 systemd[1]: hivecleaner.service: Main process exited, code=exited, status=1/FAILURE
Jul 17 11:11:26 ip-172-31-8-191 systemd[1]: hivecleaner.service: Unit entered failed state.
Jul 17 11:11:26 ip-172-31-8-191 systemd[1]: hivecleaner.service: Failed with result 'exit-code'.
ubuntu@ip-172-31-8-191:~$ date
Wed Jul 17 11:12:00 UTC 2019

On Wed, Jul 17, 2019 at 4:39 PM Kumar R <vikramvilli> wrote:

While running 

./start-services.sh 

I see below warning.

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /srv/hops/domains/domain1/config/keystore.jks -destkeystore /srv/hops/domains/domain1/config/keystore.jks -deststoretype pkcs12".
Certificate stored in file </tmp/cert.pem>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /srv/hops/domains/domain1/config/keystore.jks -destkeystore /srv/hops/domains/domain1/config/keystore.jks -deststoretype pkcs12".
Certificate was added to keystore

On Wed, Jul 17, 2019 at 4:20 PM Kumar R <vikra...@gmail.com> wrote:

I started so many times.I am planning to relaunch now.

is there in any specific requirement while selecting ec2 instance ?

I am using 50 GB,t2.xlarge 

Vikram

On Wed, Jul 17, 2019, 2:06 PM Theofilos Kakantousis <t.kakantousis wrote:

Looks like you did not run the "./start-services.sh" or the script failed. Can you try running it again? Instructions are at the bottom of the page

https://hopsworks.readthedocs.io/en/0.10/getting_started/setups/hopsworks_ami.html

On Wednesday, July 17, 2019 at 4:45:16 AM UTC+2, vikramvilli wrote:

Thanks for Reply

I am not able to reach till login page. I see obama page only.
Why hivecleaner is not running ?
please see the log below ..looks like some file missing and ssl error

tail -f /srv/hops/domains/domain1/logs/server.log

[#|2019-07-16T11:38:50.960+0000|WARNING|Payara 4.1|javax.enterprise.network.config|_ThreadID=28;_ThreadName=http-thread-pool::http-listener-2(5);_TimeMillis=1563277130960;_LevelValue=900;|GRIZZLY0053: Unrecognized cipher [TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA].|#]

[#|2019-07-16T11:38:50.961+0000|WARNING|Payara 4.1|org.glassfish.grizzly.filterchain.DefaultFilterChain|_ThreadID=28;_ThreadName=http-thread-pool::http-listener-2(5);_TimeMillis=1563277130961;_LevelValue=900;|GRIZZLY0013: Exception during FilterChain execution
java.lang.NullPointerException
        at org.glassfish.grizzly.ssl.SSLEngineConfigurator.createSSLEngine(SSLEngineConfigurator.java:214)
        at org.glassfish.grizzly.ssl.SSLEngineConfigurator.createSSLEngine(SSLEngineConfigurator.java:186)

[#|2019-07-16T11:38:50.988+0000|WARNING|Payara 4.1|javax.enterprise.network.config|_ThreadID=24;_ThreadName=http-thread-pool::http-listener-2(1);_TimeMillis=1563277130988;_LevelValue=900;|GRIZZLY0050: SSL support could not be configured!
java.io.IOException: A MultiException has 4 exceptions.  They are:
1. java.lang.IllegalStateException: java.io.FileNotFoundException: /srv/hops/domains/domain1/config/keystore.jks (No such file or directory)
2. java.lang.IllegalStateException: Unable to perform operation: create on com.sun.enterprise.security.ssl.impl.SecuritySupportImpl
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of com.sun.enterprise.security.ssl.SSLUtils errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on com.sun.enterprise.security.ssl.SSLUtils

Vikram

--
You received this message because you are subscribed to a topic in the Google Groups "Hops" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/hopshadoop/tyBs4JFAFR4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to hopshadoop+unsubscribe@googlegroups.com.