Logging request payload - request mirroring vs tap vs external processor

[Sanjay Awatramani]

Hi,

I am using envoy as a forward proxy and want to log the request payload for auditing purpose. The request payload is protobuf with size ranging from 200-1000 kb. This audit logging would be turned on temporarily and I would like to log a fraction of requests as the throughput may be high.

I ruled out lua and webassembly becuase they seem to be an overkill as their primary purpose is to process and modify the requests e.g. modify request headers, etc...

I found three other functionalities to log request payload.

1) request mirroring to a sidecar webserver that deserializes the protobuf and logs to a file

2) request tapping based on admin /tap endpoint which deserializes protobuf and logs to a file

3) external processor in observability_mode that calls external process in a Fire and Forget mode. The external gRPC handler can deserialize the protobuf and log to a file.

Regarding (2), the envoy threading architecture seems to indicate that admin client runs in the main thread and /tap can result in performance issues as all worker threads payload is processed in the main thread's /tap. Is this understanding correct?

(1) offers a a fractional request mirroring functionality that can be implemented in the (3)'s gRPC handler as well.

What is the difference between (1) and (3) for logging use case specially in terms of performance? Is there anything else that I am missing in comparing (1) vs (3)

Regards,

Sanjay

[Yan Avlasov]

In terms of performance, about the same. However for approach 3 you will need to implement ext_proc service. For approach 1 you just need HTTP server or gRPC server if the request is gRPC.

--
You received this message because you are subscribed to the Google Groups "envoy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to envoy-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/envoy-users/5ffffc92-f774-420e-811e-80802f5daa7an%40googlegroups.com.