Grpc file_based_metadata Credentials in 1.14.1

[pba...@hashicorp.com]

Hi All,

I'm trying to get https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/grpc_credential/v2alpha/file_based_metadata.proto working for my xDS control plane.

I've gone through the integration test code to figure out how it expects to be configured, and even found an old issue where a working configuration is provided (for SDS though) and not had any luck replicating it.

I've tried many permutations of config so far but I'm not getting very far.

First up this section of config is working for me to rule out any of this stuff:

dynamic_resources:

ads_config:

api_type: DELTA_GRPC

transport_api_version: V3

grpc_services:

- google_grpc:

target_uri: "localhost:8080"

stat_prefix: xds_cluster

If I run that with a local instance of my test xDS server, it connects and fetches clusters correctly.

But as soon as I add any of the credentials configs it breaks (defined below). My best guess at correct based on the test code and issue is:

dynamic_resources:

ads_config:

api_type: DELTA_GRPC

transport_api_version: V3

grpc_services:

- google_grpc:

target_uri: "localhost:8080"

stat_prefix: xds_cluster

credentials_factory_name: "envoy.grpc_credentials.file_based_metadata"

call_credentials:

- from_plugin:

name: "envoy.grpc_credentials.file_based_metadata"

config:

secret_data:

inline_string: "foo"

header_name: "bar"

header_prefix: "baz"

But adding any of those fields causes Envoy to just fail to send any gRPC messages. I eventually want to use the file data source but just trying to rule out any file system issues for now. The output is not super helpful either even in `trace` level:

-------

[2020-05-01 15:54:31.976][5391459][debug][config] [bazel-out/darwin-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:47] Establishing new gRPC bidi stream for rpc DeltaAggregatedResources(stream .envoy.service.discovery.v3.DeltaDiscoveryRequest) returns (stream .envoy.service.discovery.v3.DeltaDiscoveryResponse);

[2020-05-01 15:54:31.976][5391465][trace][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:52] completionThread CQ event 0 false

[2020-05-01 15:54:31.976][5391459][trace][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:213] Queued message to write (8079 bytes)

[2020-05-01 15:54:31.976][5391459][trace][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:263] handleOpCompletion op=0 ok=false inflight=1

[2020-05-01 15:54:31.976][5391459][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:203] notifyRemoteClose 13

[2020-05-01 15:54:31.976][5391459][warning][config] [bazel-out/darwin-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:92] DeltaAggregatedResources gRPC config stream closed: 13,

[2020-05-01 15:54:31.976][5391459][debug][config] [external/envoy/source/common/config/grpc_subscription_impl.cc:88] gRPC update for type.googleapis.com/envoy.config.cluster.v3.Cluster failed

[2020-05-01 15:54:31.976][5391459][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:224] resetStream

[2020-05-01 15:54:31.976][5391459][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:382] Stream cleanup with 0 in-flight tags

[2020-05-01 15:54:31.976][5391459][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:371] Deferred delete

[2020-05-01 15:54:31.976][5391459][trace][main] [external/envoy/source/common/event/dispatcher_impl.cc:163] item added to deferred deletion list (size=1)

[2020-05-01 15:54:31.976][5391459][trace][main] [external/envoy/source/common/event/dispatcher_impl.cc:79] clearing deferred deletion list (size=1)

[2020-05-01 15:54:31.976][5391459][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:146] GoogleAsyncStreamImpl destruct

-------

The above lines get repeated on each retry, no other relevant errors. Despite the error looking like xDS server hangup "notifyRemoteClose" I'm pretty sure this isn't the case - if I sniff traffic with WireShark I can see the HTTP2 requests in the first example but I see nothing, not even an open TCP connection with the second. 

Does anyone have this working or have any idea what I could be doing wrong?

Thanks!

[Michael Wozniak]

When you add call credentials, I think the channel defaults to using SSL channel credentials. The issue you linked has channel credentials set, so it would avoid that issue.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/envoy-users/7f42504a-7a93-4019-91a5-373e7ba83496%40googlegroups.com.