Setting drelephant.analysis.fetch.initial.windowMillis causes java.lang.NullPointerException on startup - halts processing

16 views
Skip to first unread message

Justin Miller

unread,
Aug 9, 2017, 1:38:52 PM8/9/17
to dr-elephant-users
When using the v2.0.13 (https://github.com/linkedin/dr-elephant/releases) release uncommenting the below property from dr-elephant/app-conf/GeneralConf.xml causes a null pointer exception on startup - and analysis never starts. 

<property>
<name>drelephant.analysis.fetch.initial.windowMillis</name>
<value>3600000</value>
</property>

When this property is commented back out everything starts up fine and it starts working through the entire JobHistory (9600 or so queue taking forever, it is my understanding that this parameter is supposed to prompt Dr Elephant to only look at the last hour). 

I traced the code down to the highlighted operation below. This operation is triggering a NullPointerException for some unknown reason to me. 


@Override
public void configure(Configuration configuration)
throws IOException {
this.configuration = configuration;
String initialFetchWindowString = configuration.get(FETCH_INITIAL_WINDOW_MS);
if (initialFetchWindowString != null) {
long initialFetchWindow = Long.getLong(initialFetchWindowString);
_lastTime = System.currentTimeMillis() - FETCH_DELAY - initialFetchWindow;
_fetchStartTime = _lastTime;
}
updateResourceManagerAddresses();
}

08-09-2017 12:31:47 ERROR [Thread-6] com.linkedin.drelephant.ElephantRunner : Error occurred when configuring the analysis provider.
java.lang.NullPointerException
        at com.linkedin.drelephant.analysis.AnalyticJobGeneratorHadoop2.configure(AnalyticJobGeneratorHadoop2.java:117)
        at com.linkedin.drelephant.ElephantRunner.loadAnalyticJobGenerator(ElephantRunner.java:86)
        at com.linkedin.drelephant.ElephantRunner.access$100(ElephantRunner.java:50)
        at com.linkedin.drelephant.ElephantRunner$1.run(ElephantRunner.java:103)
        at com.linkedin.drelephant.ElephantRunner$1.run(ElephantRunner.java:98)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1900)
        at com.linkedin.drelephant.security.HadoopSecurity.doAs(HadoopSecurity.java:108)
        at com.linkedin.drelephant.ElephantRunner.run(ElephantRunner.java:98)
        at com.linkedin.drelephant.DrElephant.run(DrElephant.java:34)
08-09-2017 12:31:47 ERROR [Thread-6] com.linkedin.drelephant.ElephantRunner : java.lang.NullPointerException
08-09-2017 12:31:47 ERROR [Thread-6] com.linkedin.drelephant.ElephantRunner : java.lang.RuntimeException: java.lang.NullPointerException
        at com.linkedin.drelephant.ElephantRunner.loadAnalyticJobGenerator(ElephantRunner.java:89)
        at com.linkedin.drelephant.ElephantRunner.access$100(ElephantRunner.java:50)
        at com.linkedin.drelephant.ElephantRunner$1.run(ElephantRunner.java:103)
        at com.linkedin.drelephant.ElephantRunner$1.run(ElephantRunner.java:98)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1900)
        at com.linkedin.drelephant.security.HadoopSecurity.doAs(HadoopSecurity.java:108)
        at com.linkedin.drelephant.ElephantRunner.run(ElephantRunner.java:98)
        at com.linkedin.drelephant.DrElephant.run(DrElephant.java:34)
Caused by: java.lang.NullPointerException
        at com.linkedin.drelephant.analysis.AnalyticJobGeneratorHadoop2.configure(AnalyticJobGeneratorHadoop2.java:117)
        at com.linkedin.drelephant.ElephantRunner.loadAnalyticJobGenerator(ElephantRunner.java:86)
        ... 9 more

08-09-2017 12:32:26 INFO  [Thread-6] com.linkedin.drelephant.ElephantRunner : Dr.elephant has started

Justin Miller

unread,
Aug 15, 2017, 10:21:05 AM8/15/17
to dr-elephant-users
This has been addressed and acknowledged on GitHub - https://github.com/linkedin/dr-elephant/issues/273
Reply all
Reply to author
Forward
0 new messages