grpc++ as a Windows DLL for creating plugins for hosted app environments.

[San Mehat]

Howdy,

I've been doing some legwork to add grpc++ support to UnrealEngine on 64 bit Windows builds and have a few questions.

First off, I understand that because of the various oddities associated with Windows DLLs that the recommended approach is to consume these libraries in their static forms. However for use-cases of grpc which are plugins to other applications, it is often the case that an /MT DLL is the only mechanism for having 'your stuff' be loaded. In this particular case, UnrealEngine has a 'rich' plugin / module system which can only consume /MT DLLs.

Note: Prior to this I had success building libproto as a dll (with a bit of tweaking) and was able to successfully integrate that into UnrealEngine, however I'd really like to leverage grpc instead of having to build my own HttpRpcChannel and HttpRpcController implementations.

At this point I've gotten static versions of the targets to build in 64 bit (with more tweaking), and am at a slight cross-roads in terms of approach (which I'd appreciate some feedback on):

1. I can try to muck with the existing project files to create /mt DLL linkable versions of this stuff - possibly by introducing a new target.
   
• It is unclear to me whether you folks would be interested in taking this support given the note one of the proto readme files on Windows static vs dynamic linking.
2. I can treat grpc++ as a completely black box, and wrap it with my own /mt DLL bindings and maintain that as a separate project on github.
• This strikes me as obviously the 'cleanest' in terms of separating maintenence burden if you guys are to maintain your current stance on Windows DLLs, however again it strikes me that for many 'interesting' Windows use-cases it would behoove the project to add at least skelaton support for using grpc in windows hosted plugin scenarios.
  

Note - I'm not by any means a Visual Studio / Windows expert (most of my experience is from Unix/Linux land), so there may be other approaches that make sense here in which case I'm 'all ears'.

Respectfully,

-San.

[Jan Tattermusch]

Hi San,

please see my reponses inline.

On Mon, Nov 9, 2015 at 5:24 PM, San Mehat <san....@gmail.com> wrote:

Howdy,

I've been doing some legwork to add grpc++ support to UnrealEngine on 64 bit Windows builds and have a few questions.

First off, I understand that because of the various oddities associated with Windows DLLs that the recommended approach is to consume these libraries in their static forms. However for use-cases of grpc which are plugins to other applications, it is often the case that an /MT DLL is the only mechanism for having 'your stuff' be loaded. In this particular case, UnrealEngine has a 'rich' plugin / module system which can only consume /MT DLLs.

Note: Prior to this I had success building libproto as a dll (with a bit of tweaking) and was able to successfully integrate that into UnrealEngine, however I'd really like to leverage grpc instead of having to build my own HttpRpcChannel and HttpRpcController implementations.

At this point I've gotten static versions of the targets to build in 64 bit (with more tweaking), and am at a slight cross-roads in terms of approach (which I'd appreciate some feedback on):

1. I can try to muck with the existing project files to create /mt DLL linkable versions of this stuff - possibly by introducing a new target.
   
• It is unclear to me whether you folks would be interested in taking this support given the note one of the proto readme files on Windows static vs dynamic linking.

Yes, if you need /MT DLLs, the easiest way how to get them would be to patch the project files. Note that the .vsxproj file are generated using mako templates (https://github.com/grpc/grpc/tree/master/templates/vsprojects). Switching the templates to /MT  and regenerating the project files should be easy enough and give you the results you need. I don't think we would pull your change into the upstream directly (at least at this point), but please share your patch anyway - other users might run into the same issue and it's good to have the solution in the back pocket.

Regarding the 64 bit support - can you publish the changes you've made and perhaps create a pull request? We are planning to look into 64bit support on Windows in the near future and contributions are very welcome.

 

1. I can treat grpc++ as a completely black box, and wrap it with my own /mt DLL bindings and maintain that as a separate project on github.
• This strikes me as obviously the 'cleanest' in terms of separating maintenence burden if you guys are to maintain your current stance on Windows DLLs, however again it strikes me that for many 'interesting' Windows use-cases it would behoove the project to add at least skelaton support for using grpc in windows hosted plugin scenarios. 

Note - I'm not by any means a Visual Studio / Windows expert (most of my experience is from Unix/Linux land), so there may be other approaches that make sense here in which case I'm 'all ears'.

Respectfully,

-San.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d786f764-a8c1-4627-b2f3-25b87b484c86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[San Mehat]

On Tue, Nov 10, 2015 at 7:34 AM, Jan Tattermusch <jtatte...@google.com> wrote:

Hi San,

please see my reponses inline.

On Mon, Nov 9, 2015 at 5:24 PM, San Mehat <san....@gmail.com> wrote:

Howdy,

I've been doing some legwork to add grpc++ support to UnrealEngine on 64 bit Windows builds and have a few questions.

First off, I understand that because of the various oddities associated with Windows DLLs that the recommended approach is to consume these libraries in their static forms. However for use-cases of grpc which are plugins to other applications, it is often the case that an /MT DLL is the only mechanism for having 'your stuff' be loaded. In this particular case, UnrealEngine has a 'rich' plugin / module system which can only consume /MT DLLs.

Note: Prior to this I had success building libproto as a dll (with a bit of tweaking) and was able to successfully integrate that into UnrealEngine, however I'd really like to leverage grpc instead of having to build my own HttpRpcChannel and HttpRpcController implementations.

At this point I've gotten static versions of the targets to build in 64 bit (with more tweaking), and am at a slight cross-roads in terms of approach (which I'd appreciate some feedback on):

1. I can try to muck with the existing project files to create /mt DLL linkable versions of this stuff - possibly by introducing a new target.
   
• It is unclear to me whether you folks would be interested in taking this support given the note one of the proto readme files on Windows static vs dynamic linking.

Yes, if you need /MT DLLs, the easiest way how to get them would be to patch the project files. Note that the .vsxproj file are generated using mako templates (https://github.com/grpc/grpc/tree/master/templates/vsprojects). Switching the templates to /MT  and regenerating the project files should be easy enough and give you the results you need. I don't think we would pull your change into the upstream directly (at least at this point), but please share your patch anyway - other users might run into the same issue and it's good to have the solution in the back pocket.

AHA - I had missed the template magic - thanks :).

 

Regarding the 64 bit support - can you publish the changes you've made and perhaps create a pull request? We are planning to look into 64bit support on Windows in the near future and contributions are very welcome.

Once I have something I know works I will absolutely send up a pull request. 

Thanks again Jan, lunch is on me ;)

-san

 

 

1. I can treat grpc++ as a completely black box, and wrap it with my own /mt DLL bindings and maintain that as a separate project on github.
• This strikes me as obviously the 'cleanest' in terms of separating maintenence burden if you guys are to maintain your current stance on Windows DLLs, however again it strikes me that for many 'interesting' Windows use-cases it would behoove the project to add at least skelaton support for using grpc in windows hosted plugin scenarios. 

Note - I'm not by any means a Visual Studio / Windows expert (most of my experience is from Unix/Linux land), so there may be other approaches that make sense here in which case I'm 'all ears'.

Respectfully,

-San.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

[Nicolas Noble]

In my opinion, the ideal solution would be to create a separate solution configuration probably called "Release-DLL" and "Debug-DLL". It shouldn't be the default, for all of the reasons described in the document you're linking, but we could theoretically provide it this way.

I'll empathize a bit however on the binary compatibility bit: shipping C++ in a DLL is usually a Bad Idea, especially if the various DLLs are trying to use each other. Basically, if you're trying to build a plugin for UE4, you should still build everything statically, and link everything into a stand alone, monolithic DLL which only exports the C API that UE4 requires, and which represents your plugin.

We can chat more directly if you want more details.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d786f764-a8c1-4627-b2f3-25b87b484c86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAHKA-Lm7VjdxmLwpG5TLbsPpfKeBZiBvEwRiAXnzqAGDHyJfzw%40mail.gmail.com.

[San Mehat]

On Tue, Nov 10, 2015 at 9:21 AM, Nicolas Noble <nno...@google.com> wrote:

In my opinion, the ideal solution would be to create a separate solution configuration probably called "Release-DLL" and "Debug-DLL". It shouldn't be the default, for all of the reasons described in the document you're linking, but we could theoretically provide it this way.

Heh - funny you should mention that - I've spent much of the morning clowning through getting the buildgen stuff to run on Windows (I suppose it's time I invest in a full cygwin install), I'm going to try a little hackery to explore the solution you mentioned. Of course this is likely to get ugly as i have to add the necessary DLL exports but at this point I'd like to try to get something 'working' and then spend some good quality time understanding how to clean it up for the more general case.

 

I'll empathize a bit however on the binary compatibility bit: shipping C++ in a DLL is usually a Bad Idea, especially if the various DLLs are trying to use each other. Basically, if you're trying to build a plugin for UE4, you should still build everything statically, and link everything into a stand alone, monolithic DLL which only exports the C API that UE4 requires, and which represents your plugin.

I agree completely here. One potential disaster mitigater here is that the DLL thing is really more about packaging for UE4 consumption. In reality this stuff is all going to get built at the same time.

UE4 seems to have gone to extensive lengths to prevent static linking. One additional approach I have been considering is potentially porting grpc into the UE4 build-system as a ThirdParty component which *may* allow me to add static linkages (since at that point the engine would have 'native' support for grpc instead of having to pull it in via a 'module').

 

We can chat more directly if you want more details.

Thanks - appreciate your opinions and quick response.

Best,

-san

 

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d786f764-a8c1-4627-b2f3-25b87b484c86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

[Nicolas Noble]

Careful with cygwin. If you're going to compile grpc++ as a DLL that exports things, it really won't work if the thing that loads it is compiled using Visual Studio. cygwin drags its own libstdc++, which is incompatible with Visual Studio's, and the symbol mangling and ABI is incompatible anyway.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d786f764-a8c1-4627-b2f3-25b87b484c86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

[San Mehat]

On Tue, Nov 10, 2015 at 9:54 AM, Nicolas Noble <nno...@google.com> wrote:

Careful with cygwin. If you're going to compile grpc++ as a DLL that exports things, it really won't work if the thing that loads it is compiled using Visual Studio. cygwin drags its own libstdc++, which is incompatible with Visual Studio's, and the symbol mangling and ABI is incompatible anyway.

Understood - I was only going to use cygwins environment so I get a working bash + python27 combination in order to regenerate project files from templates - because I'm just not going to sink to the level of writing windows .bat files... (yet). I'll be using VS for the actual builds.

Thanks ;)

-San

 

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d786f764-a8c1-4627-b2f3-25b87b484c86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

[Jan Tattermusch]

On Tue, Nov 10, 2015 at 5:58 PM, San Mehat <san....@gmail.com> wrote:

On Tue, Nov 10, 2015 at 9:54 AM, Nicolas Noble <nno...@google.com> wrote:

Careful with cygwin. If you're going to compile grpc++ as a DLL that exports things, it really won't work if the thing that loads it is compiled using Visual Studio. cygwin drags its own libstdc++, which is incompatible with Visual Studio's, and the symbol mangling and ABI is incompatible anyway.

Understood - I was only going to use cygwins environment so I get a working bash + python27 combination in order to regenerate project files from templates - because I'm just not going to sink to the level of writing windows .bat files... (yet). I'll be using VS for the actual builds.

I usually generate the project files using a linux workstation and then sync to windows machine to try out the changes, because getting the whole template generator working on windows is a pain. This approach works fine if you are reasonably confident about the changes, you are doing. 

 

Thanks ;)

-San

 

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d786f764-a8c1-4627-b2f3-25b87b484c86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+unsubscribe@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

[Nicolas Noble]

I usually do it under windows using msys2 myself.

 

Thanks ;)

-San

 

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/d786f764-a8c1-4627-b2f3-25b87b484c86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "grpc.io" group.

To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.

To post to this group, send email to grp...@googlegroups.com.

[Kurt]

Hi San,

I'm facing a similar issue, having to use gRPC in a DLL which is a plugin for another software.

I see there is a Release-DLL in the official release-0_14_1. Is this the correct way to build a DLL including grpc as a blackbox? Can you please share some info on how this is meant to be used?

Thanks!