Genie 3.0.1 - No bean named 'file.system.s3' is defined

[Jelez Raditchkov]

I have installed genie from jar: genie-app-3.0.1.jar

Starting genie as:

$ export JAVA_OPTS="-Djava.io.tmpdir=/opt/genie/tmp -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true"

$ exec java ${JAVA_OPTS} -jar /opt/genie/lib/genie-app-3.0.1.jar --spring.config.location=/opt/genie/config/

In /opt/genie/config/ I have application-dev.yml  application-prod.yml  application-s3.yml  application.yml

Cluster YAML config looks like:

name: spark cluster

user: hadoop

version: 2.7.3

status: UP

tags:

  - type:yarn

  - ver:2.7.3

  - emr.cluster.release:emr-5.1.0

configs:

  - s3://mybucket/genie/clusters/j-XXXXXXXXXX/conf/hadoop/core-site.xml

  - s3://mybucket/genie/clusters/j-XXXXXXXXXX/conf/hadoop/mapred-site.xml

  - s3://mybucket/genie/clusters/j-XXXXXXXXXX/conf/hadoop/yarn-site.xml

When i run a job I get exception. Could you point me to what I could be doing wrong? Do you have an example config that works?

Exception details:

2017-03-04 02:57:02.176 ERROR 1459 --- [taskScheduler-1] com.netflix.genie.core.jobs.JobLauncher  : Unable to submit job due to exception: Failed getting the file s3://mybucket/genie/clusters/j-XXXXXXXXXX/conf/hadoop/yarn-site.xml

com.netflix.genie.common.exceptions.GenieServerException: Failed getting the file s3://mybucket/genie/clusters/j-XXXXXXXXXX/conf/hadoop/yarn-site.xml

        at com.netflix.genie.core.services.impl.CacheGenieFileTransferService.getFile(CacheGenieFileTransferService.java:117) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.netflix.genie.core.jobs.workflow.impl.ClusterTask.executeTask(ClusterTask.java:131) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.netflix.genie.core.services.impl.LocalJobRunner.executeJob(LocalJobRunner.java:285) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.netflix.genie.core.services.impl.LocalJobRunner.submitJob(LocalJobRunner.java:151) ~[genie-core-3.0.1.jar!/:3.0.1]

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121]

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_121]

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_121]

        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_121]

        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.validation.beanvalidation.MethodValidationInterceptor.invoke(MethodValidationInterceptor.java:139) ~[spring-context-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at com.sun.proxy.$Proxy138.submitJob(Unknown Source) ~[na:na]

        at com.netflix.genie.core.jobs.JobLauncher.lambda$run$0(JobLauncher.java:99) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.netflix.spectator.api.AbstractTimer.record(AbstractTimer.java:48) ~[spectator-api-0.30.0.jar!/:0.30.0]

        at com.netflix.genie.core.jobs.JobLauncher.run(JobLauncher.java:96) ~[genie-core-3.0.1.jar!/:3.0.1]

        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_121]

        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_121]

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_121]

        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_121]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_121]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_121]

        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]

Caused by: com.google.common.util.concurrent.UncheckedExecutionException: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'file.system.s3' is defined

        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201) ~[guava-17.0.jar!/:na]

        at com.google.common.cache.LocalCache.get(LocalCache.java:3934) ~[guava-17.0.jar!/:na]

        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938) ~[guava-17.0.jar!/:na]

        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821) ~[guava-17.0.jar!/:na]

        at com.netflix.genie.core.services.impl.CacheGenieFileTransferService.getFile(CacheGenieFileTransferService.java:100) ~[genie-core-3.0.1.jar!/:3.0.1]

        ... 25 common frames omitted

Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'file.system.s3' is defined

        at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:704) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:1175) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:284) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:275) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.beans.factory.config.ServiceLocatorFactoryBean$ServiceLocatorInvocationHandler.invokeServiceLocatorMethod(ServiceLocatorFactoryBean.java:373) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at org.springframework.beans.factory.config.ServiceLocatorFactoryBean$ServiceLocatorInvocationHandler.invoke(ServiceLocatorFactoryBean.java:363) ~[spring-beans-4.2.8.RELEASE.jar!/:4.2.8.RELEASE]

        at com.sun.proxy.$Proxy136.get(Unknown Source) ~[na:na]

        at com.netflix.genie.core.services.impl.GenieFileTransferService.getFileTransfer(GenieFileTransferService.java:96) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.netflix.genie.core.services.impl.CacheGenieFileTransferService.loadFile(CacheGenieFileTransferService.java:153) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.netflix.genie.core.services.impl.CacheGenieFileTransferService$1.load(CacheGenieFileTransferService.java:55) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.netflix.genie.core.services.impl.CacheGenieFileTransferService$1.load(CacheGenieFileTransferService.java:51) ~[genie-core-3.0.1.jar!/:3.0.1]

        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524) ~[guava-17.0.jar!/:na]

        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317) ~[guava-17.0.jar!/:na]

        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280) ~[guava-17.0.jar!/:na]

        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195) ~[guava-17.0.jar!/:na]

Thank you!

[Tom Gianos]

Hey Jelez,

I've responded to the Github issue you opened here. We can continue the discussion there to reduce redundancy.

Thanks,

Tom