delivery pod, deployer and indexing

[bitsofinfo]

I'm looking at this:

https://docs.craftercms.org/en/3.1/system-administrators/activities/kubernetes/simple-kubernetes-deployment.html?highlight=kubernetes

From what I understand the deployer is responsible for determining changes to be applied to engines and then also manages indexing that content in ES. 

If I have N replicas of this delivery pod, when changes are inbound and get re-indexed, does every single deployer container across the N pod replicas submit the same change for indexing? or is this somehow coordinated so only one deployer is responsible for the indexing. Amongst indexing via searchIndexingProcessor I see other things in the pipeline, is only one deployer responsible for all of this?

thanks!

[Sumer Jabri]

Single Delivery POD per geographic region. Remember, Delivery can be peppered globally (as is the case in large global deployments where they need local delivery of content (think secure and dynamic content that can't be in a CDN)).

--sumer

[BitsOfInfo]

Then why is the deployer container inside the same pod as the delivery container? If I scale this up I’m going to have multiple deployers repeating the same work, no?

https://docs.craftercms.org/en/3.1/system-administrators/activities/kubernetes/simple-kubernetes-deployment.html?highlight=kubernetes

On Dec 9, 2019, at 1:08 PM, Sumer Jabri <sumer...@craftersoftware.com> wrote:



--
You received this message because you are subscribed to the Google Groups "CrafterCMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to craftercms+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/craftercms/4de94b44-c2c4-4e89-8986-20d383de0200%40googlegroups.com.

[Sumer Jabri]

You're probably right. You can either file a ticket to fix the docs or send a PR to fix the docs.

(If not on K8s, it's very possible (and some do this) to deploy every Engine along with a small Elasticsearch and Deployer on the same node to have a single-self-contained delivery node that needs nothing else. This use-case is real for cruise-ships and other disconnected or low-connectivity environments (including kiosks, where Crafter simply renders kiosk screens/or video reels). I would imagine if someone built a box with Engine in it for airlines and it syncs up from Studio when docked, and delivers everything disconnected while in the air it would work well. Along the same lines, see the deployment in Snowball Edge: https://craftersoftware.com/blog/2019/07/aws-snowball-edge-low-latency-and-remote-content-delivery)

--sumer

On Monday, December 9, 2019 at 4:06:50 PM UTC-5, bitsofinfo wrote:

Then why is the deployer container inside the same pod as the delivery container? If I scale this up I’m going to have multiple deployers repeating the same work, no?

https://docs.craftercms.org/en/3.1/system-administrators/activities/kubernetes/simple-kubernetes-deployment.html?highlight=kubernetes

On Dec 9, 2019, at 1:08 PM, Sumer Jabri 



[bitsofinfo]

So, in short; the ratio of "deployers" to an elasticsearch for any topology would be a 1-to-1, correct? How can you have an HA setup for the "deployer" so more than one instance running but in a failover capacity or so that you don't have 2 of them doing the same work?

[Sumer Jabri]

Yes, re the ratio.

Kuber should keep one up at any time per region. Failure doesn't bring down the delivery nor authoring systems, simply delays publishing until the new Deployer is back up. You can probably think up a clever way to keep n-nodes up but only one active within a single region, and we may pursue something like that in the future.

--sumer