Size of compiled C++ binary

[Xuejie Xiao]

Hey guys!

We are evaluating gRPC internally, one thing we've found out, is that the compile C++ binary size is quite big, this is even the case for client binaries. For example, if we compile the C++ helloworld example from grpc-common repo, greeter_client will be 9.4M, while greeter_server is 9.5M.

So our question here is: is it true that the client-side binary requires almost the same binary size as the server-side binary? Is it possible to strip the client binary size here?

Or maybe we are actually using a wrong linking argument here?

Many thanks!

[Craig Tiller]

Hey!

We haven't spent a huge amount of time optimizing for size, but I expect to do some work in that direction in the coming quarter (or so).

That said, most of our code base is relatively symmetric, so I don't expect to see massive differences in client and servers for the greeter example.

Most of the existing cost is OpenSSL I expect. If you can rely on the system library (and therefore don't mind using npn instead of alpn) - or if you don't need crypto and auth - then that cost goes way down.

If you do need alpn (its required by http2, so as soon as you need ssl and don't control all intermediaries you may), then that cost is going to be there. You might be able to amortize that cost if any other library you're considering also needs to link a recentish OpenSSL.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/47b360e7-8332-449d-b444-060ec98c45ab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nicolas Noble]

Also, it seems that you'd be compiling / linking against the static version of the library ? The grpc shared libraries, once installed, should be considerably smaller, and wouldn't have an impact on your binary size.

Right now if I compile grpc, I get the following:

$ ls -lah *.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel  51K Jul 22 07:11 libgpr.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 2.5M Jul 22 07:11 libgrpc.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 229K Jul 22 07:11 libgrpc++.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 287K Jul 22 07:11 libgrpc_unsecure.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 205K Jul 22 07:11 libgrpc++_unsecure.so.0.10.0.0

And this is with OpenSSL embedded.

If I remove the embedded OpenSSL, and fallback on NPN, hence using the system's OpenSSL, as Craig mentioned, I get only this:

$ ls -lah *.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel  51K Jul 22 07:12 libgpr.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 405K Jul 22 07:12 libgrpc.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 229K Jul 22 07:12 libgrpc++.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 287K Jul 22 07:12 libgrpc_unsecure.so.0.10.0.0

-rwxrwxr-x 1 pixel pixel 205K Jul 22 07:12 libgrpc++_unsecure.so.0.10.0.0

As expected, only the grpc library shrunk down in size, from 2.5M to 405K.

So something would seem off with your numbers in all cases.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAAvp3oNkLi9iAuajBcrogJddk_AoTLUetDncs857w55tG8qhHw%40mail.gmail.com.

[Xuejie "Rafael" Xiao]

Craig & Nicolas:

Thank you both for your reply! I will use one post to reply to you both to save everyone from one extra email :)

I understand that dynamic linking might be able to solve the problem. However, we are now thinking of using gRPC as an *external* RPC protocol instead of an *internal* one. This means that the client side binary will be installed on end user's machine. In this case, I think every byte we save will count, and dynamic linking is really not an option for us here(user will still have to install gRPC as well as OpenSSL).

Craig: I know this might be too early to tell, but just curious enough to ask: how much of the codebase do you think only relate to server hence can be stripped from the client? In other words, is there any estimation you have on how small the client binary can be, assuming the fact you guys have the time for this next quarter?

Or maybe we are just using gRPC in the wrong place :P

--

Best Regards,

Xuejie "Rafael" Xiao

[Xuejie "Rafael" Xiao]

Oops, I must admin that I overlooked the OpenSSL size part in my previous email, so I *have* to send another email. I just wish we have an "undo" button for mailing lists :)

From Nicolas' numbers, it would seem OpenSSL indeed bring a lot of size. I'm not so familiar with gRPC internals as well as HTTP/2 encryption, so please forgive me if this sounds silly: what features are required by gRPC in a SSL library? You mentioned ALPN support here, so do you think it will be an easy task to switch to a different SSL library that also has ALPN support, such as MatrixSSL or mbed TLS? Considering those SSL library is smaller than OpenSSL, we should expect a smaller binary size, shouldn't we?

[Nicolas Noble]

For the server part, the linker will already do what you want, really. If you're not using any of the server function in your binary, with the way gRPC works and is setup, while statically linking, the linker will remove any unused function and the server code will disappear. But the gain should really be negligible. The "server-specific" code I I'm talking about here is but a few lines of code to set up a listener socket, and accepting connections on it.

One of the problems of OpenSSL is its registration system, meaning that we're dragging with us an insane amount of unwanted features. It may be possible to shrink down the size of the library by tweaking the compilation of OpenSSL to cull all of the unnecessary parts that the linker can't remove due to the registration. But the current way we're embedding OpenSSL into gRPC is more of a convenience than something that should be used in production. What you really want is to rely on the presence of the OpenSSL dynamic library on your client's machines, and make usage of our NPN fallback mechanism.

Basically, remove or rename openssl from third_party (or don't recursively clone the repository). You'll have to make sure you have openssl-dev installed, and you'll have to link with openssl yourself, but then you'll be using the system-wide, well known OpenSSL, hence reducing the size of your binary.

As for using another library than OpenSSL, contributions are welcome :-) The relevant code is going to be in src/core/security and src/core/tsi. But this is probably going to be the same issue. In fact, our choice of OpenSSL is only because it's a usually widely installed SSL library, and we don't have to embed it. As I said, we're only embedding it for convenience and test reproducibility. If we were to use another SSL library that we know we'd always embed instead of relying on system-wide libraries, we'd use Google's BoringSSL otherwise :-)

Last but not least, if you statically linked, remember to strip your binary. The libraries are compiled with debug symbols in by default, and stripped only during installation, just like any other package. Meaning that if you skip the installation phase, your libraries won't be stripped out of their debugging information, and will drastically bloat your final binary.

[Xuejie "Rafael" Xiao]

Nicolas:

Thanks for the detailed explanation! Actually I was being silly here, it is indeed the problem of debug symbols. With a single `-s` flag, I can strip the binary from around ~ 9M down to ~800K. This is quite reasonable for us.

Thanks again for all the help!

[uttav.a...@gmail.com]

Hi Xuejie Xiao,

Could you please tell how did you reduce the binary size to 800KB.

I have tried to build with strip-static option, and then compiling client with g++ -s . I am trying to compile the helloworld example.

But the size I am getting is 4.9 MB minimum which is still bigger than what we require.

Could you please share some info how you have built your client ?

[Dmitry Gorelov]

Dear Sirs, please share with the community ))) How could you strip binaries to size of 800kb?

вторник, 19 сентября 2017 г. в 12:09:40 UTC+3, uttav.a...@gmail.com:

[Dmitry Gorelov]

ah.Got it.

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -s") 

does the job.

среда, 15 марта 2023 г. в 20:22:59 UTC+3, Dmitry Gorelov: