unable to start embedded Tomcat

[Sonny Heer]

Hi users,

While creating a transformation feed - during step 4.  i get the following error in spark shell:

2017-08-11 02:16:04 INFO  launcher-proc-2:SparkShellApp:61 -  diagnostics: User class threw exception: org.springframework.context.ApplicationContextException: Unable to start embedded container; nested exception is org.springframework.boot.context.embedded.EmbeddedServletContainerException: Unable to start embedded Tomcat

Using 0.8.2 on HDP 2.4

Thanks

[Sonny Heer]

Here is what is in the yarn logs:

java.lang.ClassNotFoundException: org.datanucleus.api.jdo.JDOPersistenceManagerFactory

classpath issue.  where does that class need to be loaded from?

[Sonny Heer]

In case someone else is running into this.  Had to add --jars in spark.properties.  There might be an issue elsewhere still though...

spark.shell.sparkArgs=--jars /usr/hdp/current/hive-server2/lib/datanucleus-api-jdo-3.2.6.jar,/usr/hdp/current/hive-server2/lib/datanucleus-core-3.2.10.jar,/usr/hdp/current/hive-server2/lib/datanucleus-rdbms-3.2.9.jar

Now the spark job cannot find db/table.  although when I go to "tables" I'm able to view tables and preview.  My guess is its a different type of connection there vs spark job w/kerberos.  Any ideas?

[Greg Hart]

Hi Sonny,

Could you try setting this property in spark.properties?

spark.shell.files=/etc/hive/conf/hive-site.xml

[Sonny Heer]

Hi Greg,

I was just about to respond.  I added: 

spark.shell.sparkArgs=--files /usr/hdp/current/spark-client/conf/hive-site.xml

Got past that issue with adding the hive site file.

I'm not sure how to debug when my transformation continues to run.  e.g. A simple transform on a single Hive table without doing any spark functions.  Timer is set to run every 20 minutes.  It starts the feed, and it just keeps running.  I would expect this feed to take less than a couple minutes.

There is a yarn application started and running with my user name (AD).  Nothing else is wrong in the logs...just random updates.  

any ideas?

[Sonny Heer]

I've opened another thread for the issue with transformation jobs not finishing.