Coreos at Metamarkets

[Charles Allen]

A post just went live about how we use Container OS at Metamarkets.

https://metamarkets.com/2017/druid-and-spark-together-mixing-analytics-workflows/

[Alex Crawford]

On 09/15, Charles Allen wrote:
> A post just went live about how we use Container OS at Metamarkets.
>
> https://metamarkets.com/2017/druid-and-spark-together-mixing-analytics-workflows/

Wow, this is a great write-up! Very interesting to see the kernel
debugging and tuning.

I saw that you are maintaining your own custom builds of Container Linux
(which is encouraging to see people further customizing the OS) and was
curious if there was something that prevented you from using vanilla
builds. We introduced Ignition a while ago to help address the
short-comings of coreos-cloudinit and, in my mind, if you are unable to
accomplish certain tasks, it's a bug in Ignition.

-Alex

[Charles Allen]

Thanks, Alex!

Since we're using Mesos instead of Kubernetes in order to be able to launch with 0 external blob storage dependencies we bake the Mesos executables into the build process. We bake the ignition config into the instance definition itself and do not pull the ignition config from blob storage. The alternative, and probably more sustainable approach would be to bake some sort of package (either a docker/rkt container image or statically compiled binaries) into the image to allow it to boot uninhibited. But I didn't find great docs or examples on how to do this compared with how to simply add new EBUILDs in an overlay.

We also add ssh related configs to the image to make sure the right PAMs get linked up so we can get in and debug the instances, which are too big and too common to put in every ignition config.

So its not a question of ignition, its a question of how to best pre-package more stuff on the vanilla images.

Cheers,

Charles Allen