Running microservices with Deis Workflow

[Martin Angers]

Hello,

I'm trying to see how I could run an app with the following architecture:


- Internet-facing website (web server, e.g. https://www.domain.com)
- Internet-facing API (web server, e.g. https://api.domain.com)
- Various internal microservices (HTTP-based) called by the API
- Various stateful backing services (DBs), off-cluster, so not an issue here


And I'd want scalable, load-balanced, health-checked processes for the web, api and each microservice.


From what I can see, a Deis app == a k8s service, and a k8s service is load-balanced and supports service discovery, so I guess the microservices would be separate Deis apps, so I'd have an app for each msvc + an app for the API and the Website? And the docs state that each app is reachable based on its name, but I want the microservices to be reachable only from within the cluster (via the API), how would I do that?


There's also the information about the app's processes, where it states that a non-cmd, non-web process type doesn't receive traffic from the deis router, so maybe that's what's needed for the msvc processes, but how would the callers be able to reach them and have them load-balanced/health-checked? This must be done at the raw k8s level?


I think an example github repo for such an architecture would be great (apologies if there's already one, I did take a quick look at the example-* repos under deis).

Thanks,
Martin

[Matthew Fisher]

Hey Martin,

You're right about your assumptions. The internet-facing apps would be separate apps. The internal microservices would be deployed in the same namespace as the internal API server (assuming they are a non-web process), so the API server would be able to communicate to the non-web processes through DNS. Even though they are not load-balanced by the router, we still deploy a service for that process type so they can discover eachother.

Let me boot up a k8s cluster and see if I can deploy example-dockerfile-procfile-http to give you an example.

Cheers,

Matt

--
You received this message because you are subscribed to the Google Groups "Deis Users List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to deis-users+...@googlegroups.com.
To post to this group, send email to deis-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/deis-users/e39d0e1d-af43-47ed-8d1e-e78e02b272e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Martin Angers]

Thanks Matt, that'd be awesome!

[Martin Angers]

Ok gotcha, I'll play with it a bit and see what works best. I think that would be a very useful scenario to support eventually though, maybe a "svc" proctype that is registered as a discoverable, load-balanced k8s service, but not exposed to the internet.

Thanks for your help!
Martin

On Thu, Jul 14, 2016 at 17:30 Matthew Fisher <m...@bacongobbler.com> wrote:

Turns out I was incorrect in that assumption. The controller only creates a service for the `web` process, so yes you would have to find a way to discover the internal API server (perhaps deploy your own service in the namespace that has a selector of `worker`, but turn the `router.deis.io/routable` annotation to false) or expose it to the public as another app, then use something like `deis config:set MY_INTERNAL_API_URL=http://internal-api.deis.rocks` for your API server.

Matt

[Matthew Fisher]

No problem! For reference I created https://github.com/deis/controller/issues/883 as a result of this conversation. :)

Cheers!

Matt

[Martin Angers]

I tried a few things out, and really, for a microservice architecture, you want to be able to deploy independently (and rollback, and have isolated sets of configs, etc.) so having separate deis apps makes a lot of sense. The only issue is that the internal microservice is internet-facing by default.

I implemented a prototype with a web server deistest that calls an internal microservice deistest2 via the cluster-internal http://deistest2.deistest2 URL (that is, http://<service>.<namespace>) and set the deistest2 service's routable label to false, and it works like a charm. I can scale the microservice up and it load balances between all processes. Only issue I've seen in my totally not exhaustive tests is that when I redeploy the microservice, the routable label is set back to true automatically.

Any way I could make that persistent? And any other downsides to this approach that I may be overlooking?

Thanks,

Martin

[Ekrem Aksoy]

Currently (in Deis v1), I'm doing this by using Multi-buildpack where my Backend microservices is Java, and I've created dummy API's for each of them (just heartbeating might be useful by the way), and custom Fleet services. In K8S I've found that is relatively easy to make these services discoverable. However, a big +1 for #883

Regards,

Ekrem

CTO, Hiddenslate

[Matthew Fisher]

Turns out I was incorrect in that assumption. The controller only creates a service for the `web` process, so yes you would have to find a way to discover the internal API server (perhaps deploy your own service in the namespace that has a selector of `worker`, but turn the `router.deis.io/routable` annotation to false) or expose it to the public as another app, then use something like `deis config:set MY_INTERNAL_API_URL=http://internal-api.deis.rocks` for your API server.

Matt