Getting EMR error "Timeout occurred during bootstrap"

[Raul Reynoso]

I'm new to Snowplow and EMR.  I'm having some problems getting the EmrEtlRunner to work.  It hangs when it kicks off the EMR jobflow.  Looking at EMR in the console I see that the Master Instance is running but the 2 Core instances timeout during the bootstrap process.  EMR attempts to relaunch and they timeout again.  This cycle continues until eventually there are too many errors and the Cluster is terminated.  

The configuration is as follows: 

- EMR Cluster runs in the public subnet of a VPC

- Region is US-East-1

- ami_version = 3.6.0  - That is what is set in the config.yml.sample with a comment not to change, though amazon has deprecated this version.

- hbase "0.92.0"

- instance types m1.xlarge

Looking at the system logs of the terminated instances, I saw the following:

/dev/fd/11: line 1: /sbin/plymouthd: No such file or directory
initctl: Event failed

That's the only error I saw in the log and the shutdown process began immediately afterward.

Any help fixing this issue would be greatly appreciated.

Thanks,

Raul

[Alex Dean]

Hi Raul,

Do you need HBase running - can you try without?

Thanks,

Alex

--
You received this message because you are subscribed to the Google Groups "Snowplow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snowplow-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Co-founder
Snowplow Analytics
The Roma Building, 32-38 Scrutton Street, London EC2A 4RQ, United Kingdom
+44 (0)203 589 6116
+44 7881 622 925
@alexcrdean

[Raul Reynoso]

Alex,

Thanks you for your help.  Your suggestion yielded some more debug info in the logs.  That led me to discover that my issue was related to this: https://forums.aws.amazon.com/thread.jspa?threadID=158426

I fixed the VPC DNS hostname setting and my cluster now starts up successfully.

Raul

[Alex Dean]

Thanks for posting back with the fix - very helpful!

Alex

[Peter Vandenberk]

Raul,

We encountered exactly the same issue when we first set up Snowplow's EMR ages ago, and fixed it in exactly the same way... maybe this can be added to the Snowplow docs (hint, hint... :-)

Peter

[Alex Dean]

Added to troubleshooting guide!