Iterating over grpc_metadata_batch

[John Ousterhout]

In the Homa module for gRPC there are a few places where the code needs to iterate over all of the values in a grpc_metadata_batch, but there are currently cases where my code isn't seeing all of the values. Here is an example from my code that attempts to log all of the values in a batch. The code invokes the Encode method, passing it a MetataLogger object:

grpc_metadata_batch *batch;

MetadataLogger logger(separator);
batch->Encode(&logger);

Here is the definition of the logger object:

class MetadataLogger {
public:
    MetadataLogger(const char *separator) : separator(separator) {}

    void Encode(const grpc_core::Slice &key, const grpc_core::Slice &value)
    {
        uint32_t keyLength = key.length();
        uint32_t valueLength = value.length();
        Mock::logPrintf(separator, "metadata %.*s: %.*s", keyLength,
                key.data(), valueLength, value.data());
    }

    template <typename MetadataTrait>
    void Encode(MetadataTrait, const grpc_core::Slice &value)
    {
        absl::string_view key = MetadataTrait::key();
        uint32_t keyLength = key.length();
        uint32_t valueLength = value.length();
        Mock::logPrintf(separator, "metadata %.*s: %.*s", keyLength,
                key.data(), valueLength, value.data());
    }

    template <typename MetadataTrait>
    void Encode(MetadataTrait, const typename MetadataTrait::ValueType& value)
    {
        absl::string_view key = MetadataTrait::key();
        uint32_t keyLength = key.length();
        const grpc_core::Slice& slice =
                grpc_core::MetadataValueAsSlice<MetadataTrait>(value);
        uint32_t valueLength = slice.length();
        Mock::logPrintf(separator, "metadata %.*s: %.*s", keyLength,
                key.data(), valueLength, slice.data());
    }

    const char *separator;
};

The class has 3 different Encode methods, intended to catch all of the different variants of metadata. However, it doesn't seem to be catching absolutely all of them. In particular, if I set the PeerString value, where the key is grpc_core::PeerString(), none of these methods gets invoked. Are there other variants of Encode that I need to define to catch this key? What do I need to do to be sure I'm catching absolutely all of the metadata entries? Is there a better way I should be doing this?

Thanks in advance for any help you can provide.

-John-

[Craig Tiller]

`PeerString` (and some others) are Non-encodable metadata: they're getting carried around there because they make sense at the same times as metadata, but they don't go out on the wire.

So... being non-encodable Encode() skips them.

There's some logging helpers that will catch them, so they show up as debug text, but they're not directly iterable right now.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/f14fe33f-5f76-4f6a-9066-b9c920ed9835n%40googlegroups.com.

[Craig Tiller]

(take a look at MetdataMap<>::Log() for an API to get a textual representation out of *all* the things)

[John Ousterhout]

Thanks for the response. I tried MetadataMap<>::Log(), but it didn't iterate over the peer string either (and I verified that the peer string is present by calling batch.get_pointer(grpc_core::PeerString()). I then thought perhaps MetadataMap<>::ForEach would do the trick, since its comment says "Like Encode, but also visit the non-encodable fields". But, then I noticed that Log uses ForEach, so the comment for ForEach must be stale.

Any other suggestions?

-John-

[Craig Tiller]

I just changed grpc_core::Call::ProcessIncomingInitialMetadata on my enlistment to add the line:

gpr_log(GPR_ERROR, "%s", md.DebugString().c_str());

(noting that DebugString calls Log to do its work)

when I ran (the test I just happened to be debugging five minutes prior):

bazel run test/core/end2end:disappearing_server_fuzzer

I got:

E0101 00:00:05.005000000  691479 call.cc:448]                          PeerString: ipv4:127.0.0.1:2, user-agent: grpc-c/36.0.0 (linux; chttp2), :authority: localhost:1, grpc-timeout: @30005ms, GrpcRegisteredMethod: (nil), grpc-accept-encoding: identity, deflate, gzip, GrpcStatusFromWire: true, :method: POST

So I think the docs there are up to date and there's something else at play.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/68277d44-9d1f-48df-b973-fff314e4feaen%40googlegroups.com.

[John Ousterhout]

Oops, you're right. I made a mistake in my test. I had two pieces of code that iterate over metadata_batches and I only changed one of them to use Log, but it was the other one that was actually getting used in the test. Things are now working fine with Log; this solves all of my problems.

Thanks for your help, and sorry for my false alarm.

-John-