Config to route to multiple grpc services

[Rama Rao]

I am configuring envoy to route to multiple grpc services. I have defined a listener as follows and defined two routes

{

      "address": "tcp://0.0.0.0:7070",

      "bind_to_port": true,

      "filters": [

        {

          "type": "read",

          "name": "http_connection_manager",

          "config": {

            "codec_type": "auto",

          "generate_request_id": true,

            "stat_prefix": "grpc",

            "route_config": {

              "virtual_hosts": [

                {

                  "name": "backend",

                  "domains": ["*"],

                  "routes": [

                    {

                      "timeout_ms": 0,

                      "prefix": "/grpc.sample.GrpcSampleService/",

                      "cluster": "service2"

                    },

                    {

                      "timeout_ms": 0,

                      "prefix": "/time.TimeService/",

                      "cluster": "service3"

                    }

                  ]

                }

              ]

            }

and correspondingly, I have defined the Clusters as follows

 "clusters": [

      {

        "name": "service1",

        "connect_timeout_ms": 5000,

        "type": "strict_dns",

        "lb_type": "round_robin",

        "hosts": [

          {

            "url": "tcp://localhost:8080"

          }

        ]

      },

      {

        "name": "service2",

        "connect_timeout_ms": 5000,

        "type": "sds",

        "features": "http2",

        "lb_type": "round_robin",

        "service_name": "samplegrpc"

      },

      {

        "name": "service3",

        "connect_timeout_ms": 5000,

        "type": "sds",

        "features": "http2",

        "lb_type": "round_robin",

        "service_name": "time"

      }

If I understand correctly, if I want to route to any grpc service, I need to repeat the "route" and "cluster" for each of them. Is that the recommended approach or is there a more simplified configuration than this?

Thanks,

Rama

[Harvey Tuch]

Each distinct service (with its own set of upstream endpoints) will need its own cluster definition and one or more routes pointing at this cluster.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To post to this group, send email to envoy...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CAFfmCFFbYx16HD%3DxVBsMhuzJouQdbxTxAkjSf8KmVUF-%3D3oEUg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Rama Rao]

Is it not complicated if two services have different services to talk and they have to have completely different configs for their envoys? Is it possible to make it little generic  templated routes and clusters like

             "routes": [

          

                    {

                      "timeout_ms": 0,

                      "prefix": "/$service_name/",

                      "cluster": "$service_name"

                    }

                  ]

                }

and 

   {

        "name": "$service_name",

        "connect_timeout_ms": 5000,

        "type": "sds",

        "features": "http2",

        "lb_type": "round_robin",

        "service_name": "$service_name"

      },

so that these routes are evaluated based on service name as an example. This route can be last in the list of routes so that if non the top routes match, this generic route is used.

What do you think?

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

[Harvey Tuch]

I think you are asking for the path equivalent of cluster_header (https://www.envoyproxy.io/envoy/configuration/http_conn_man/route_config/route.html#config-http-conn-man-route-table-route-headers). I'm not sure in your use case if this is really justified unless you have a massive number of gRPC services. Keep in mind that the Envoy configs are not intended to be human crafted except for trivial examples, they are intended to be machine generated, so extra verbosity is not a concern.

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.

[Rama Rao]

I was also thinking that they will be machine generated.  Even in machine generated case,  the input has to be provided and that is manual. Or is there any other way?  Also with this type of generic configuration,  we can possibly avoid verbose configuration and mostly will be same for service-service flows. 

To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users+unsubscribe@googlegroups.com.

[Matt Klein]

Adding extra templating complexity to Envoy IMO is not worth it. In general we are targeting machine generation for complex deployments so all of this can be easily done by the management server. At Lyft, we actually still use Jinja templating for some uses cases to expand the configs (an approximation is here: https://github.com/envoyproxy/envoy/tree/master/configs).

The guiding principle that we have been using is to only add features in Envoy that cannot be accomplished any other way by a management server.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CAFfmCFERu7M-%3DQmiS44mGxFNogw6TYdVeA%3D5Woa%2B7U4PGrKV0g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Matt Klein

Software Engineer

mkl...@lyft.com

https://calendly.com/mattklein123

[Rama Rao]

Thinking further on this, I think it would be OK to generate "same" config for all envoy's in the mesh, irrespective of whether a particular service/envoy uses all the services or not. So if we make that assumption that having extra config, does not harm, we can just generate "routes" and "clusters" for all the services in the mesh and the same config can be used by all envoys. This will simplify things a lot rather than worrying about which services a given envoy should talk and generate only that config. Only thing we need to worry about in this case is adding router and cluster when a new service is part of the mesh.

Is that how you generate at Lyft? or do you generate service/envoy specific configuration for only services it talks to?

Thanks,

Rama

[Matt Klein]

Is that how you generate at Lyft? or do you generate service/envoy specific configuration for only services it talks to?

That is what we did when we first rolled out Envoy in 2015. At this point it's substantially more complicated and involves custom configuration for each service, a new "envoy manager" service, etc.

You might find this talk useful: https://www.microservices.com/talks/mechanics-deploying-envoy-lyft-matt-klein/

(I'm also giving an updated version of this talk at Kubecon). 

[Rama Rao]

Thanks Matt. The talk is really useful and I was actually thinking of some thing similar where in the manifests would have that additional information to generate the config - looks like it works well :-)

[Rama Rao]

Matt,

Thinking further on simplifying this - Can we introduce a "service_name_header" that tells the service this request should be routed? If that is acceptable, then we can simplify the configuration as follows :

We have a route definition that refers a generic "grpc_services" cluster.

"route_config": {

              "virtual_hosts": [

                {

                  "name": "backend",

                  "domains": ["*"],

                  "routes": [

                    {

                      "timeout_ms": 0,

                      "prefix": "/",

                      "cluster": "grpc_services"

                     }

                  ]

                }

              ]

}

And in the Clusters, we can define the grpc_services clusters whose type is "sds". In the cluster definition, I am proposing a new "service_name_header"  attribute. I can either specify service_name or service_name_header. If I specify a "service_name_header", it uses the header to derive the service_name and calls sds. The only thing, we need to handle here is since service_name_header is dynamic, sds needs to maintain this list for talking to sds service - Not sure how it is done today. 

 "cluster_manager": {

    "sds": {

      "cluster": {

        "name": "sds",

        "connect_timeout_ms": 250,

        "type": "strict_dns",

        "lb_type": "round_robin",

        "hosts": [{"url": "tcp://localhost:8080"}]

      },

      "refresh_delay_ms": 30000

    },

    "clusters": [

      {

        "name": "grpc_services",

        "connect_timeout_ms": 5000,

        "type": "sds",

        "features": "http2",

        "lb_type": "round_robin",

        "service_name_header": "grpc_service"

      }

    ]

  }

Do you think it is worth going this route? Or do you still think it is better we generate configs by manifests?

Thanks,

Rama

[Matt Klein]

Sorry I don't quite follow. It sounds like you are asking for a dynamic SDS lookup to happen in the data path flow? If so I don't think this is something we are likely to support.

To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/CAFfmCFGgab3cdduPwxwzuDHuSaKhzZZ9aWX2ACD_RaiEvw9sHQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Rama Rao]

I think I am not suggesting for a dynamic SDS lookup in the data path flow.

What I am proposing is - to sds gets the service name to query from cluster definition (service_name attribute in the cluster). I am suggesting to make this service_name dynamic by reading from service_name_header in the request. So retrieve the service_name from that header property and use it to query against sds.

Thanks,

Rama

[Matt Klein]

Sorry I'm still confused. How is what you want different from the "cluster_header" option? https://www.envoyproxy.io/envoy/configuration/http_conn_man/route_config/route

[Rama Rao]

With cluster_header option, you still need to define clusters for all the services to which cluster_header attribute resolves to. For example the evaluation of cluster_header resolves to foo and bar, we need two clusters defined with names "foo" and "bar" otherwise it would return 404. With this option, you define only cluster and use it in route definition - but the hosts that this cluster evaluates to if it is "sds" type to would be based on service_header_name attribute. So if the service_header_name evaluates to "foo", it would give you "foo" related hosts from service discovery and if it evaluates to "bar" , it would give you "bar" related hosts from service discovery., That way we are able to resolve to different "hosts" based on service using the same cluster definition.

Hope I am able to make this clear now :-)

[Matt Klein]

But when will Envoy fetch the hosts? I don't understand how this would work. For performance reasons you do not want Envoy fetching hosts in the data path. Therefore, you need to predefine the clusters.

[Rama Rao]

Correct. In the first request to a new service we can get the hosts and add this service_name to the cluster list that sds tracks. From then on sds refreshes the list of hosts the way it does today..

[Matt Klein]

This is a very large and complicated feature that I doubt we will support any time soon. It is somewhat (but not quite) related to https://github.com/envoyproxy/envoy/issues/1606. Feel free to open an issue but I would recommend not relying on anything like this existing for the foreseable future.

[Rama Rao]

Matt,

I have created the issue https://github.com/envoyproxy/envoy/issues/2022. One follow-up question as I was thinking more on this - Would this approach aggregate stats of all services? Because we do stats collection by Cluster if I understand it correctly?

Thanks,

Rama

[Matt Klein]

TBH I haven't really thought about it much yet. This approach is generally against the design philosophy of Envoy and a considerable amount of effort. With that said, I'm happy to track in an issue and see if we get other user demand for it. 

[Rama Rao]

Ok. Thanks. As I see the current design of Envoy, I am pretty sure that this "generic" cluster approach, would aggregate stats.. Let us see if we get other user demands.

Thanks,

Rama

[Rama Rao]

Or probably we should dynamically create a new cluster if we see a new service_name if we want stats for each service. I agree with you - it looks complicated :-)

Thanks

Rama