Does gmock_gen.py work on COM interfaces?

[Randy Schott]

I've been trying to mock out some COM interfaces lately. Some of them
get pretty lengthy, and short of writing some really fun regular
expressions, I've had to copy the function declarations and modify
them by hand. It's very time consuming, so I was wondering if anybody
has had any luck using the script in this manner? For reference, my
most recent attempt was the IDirectSound8 interface, which is defined
in dsound.h like this:
DECLARE_INTERFACE_(IDirectSound8, IDirectSound)
{
// IUnknown methods
STDMETHOD(QueryInterface) (THIS_ REFIID, LPVOID *) PURE;
STDMETHOD_(ULONG,AddRef) (THIS) PURE;
STDMETHOD_(ULONG,Release) (THIS) PURE;

// IDirectSound methods
STDMETHOD(CreateSoundBuffer) (THIS_ LPCDSBUFFERDESC
pcDSBufferDesc, LPDIRECTSOUNDBUFFER *ppDSBuffer, LPUNKNOWN pUnkOuter)
PURE;
STDMETHOD(GetCaps) (THIS_ LPDSCAPS pDSCaps) PURE;
STDMETHOD(DuplicateSoundBuffer) (THIS_ LPDIRECTSOUNDBUFFER
pDSBufferOriginal, LPDIRECTSOUNDBUFFER *ppDSBufferDuplicate) PURE;
STDMETHOD(SetCooperativeLevel) (THIS_ HWND hwnd, DWORD dwLevel)
PURE;
STDMETHOD(Compact) (THIS) PURE;
STDMETHOD(GetSpeakerConfig) (THIS_ LPDWORD pdwSpeakerConfig)
PURE;
STDMETHOD(SetSpeakerConfig) (THIS_ DWORD dwSpeakerConfig)
PURE;
STDMETHOD(Initialize) (THIS_ LPCGUID pcGuidDevice) PURE;

// IDirectSound8 methods
STDMETHOD(VerifyCertification) (THIS_ LPDWORD pdwCertified) PURE;
};

My guess is that all of the macros are making it difficult for
cppclean to parse out the structure. All I get out of gmock_gen.py is
"Class IDirectSound8 not found".

Thanks for your help!
Randy

[Zhanyong Wan (λx.x x)]

Hi, Neal,

Is it easy to extend the generator to support this? Thanks,

--
Zhanyong

[Zhanyong Wan (λx.x x)]

Randy,

While we are waiting to hear from Neal, the creator of the mock
generator script, you can try to run the header through the
preprocessor and then feed the output to the generator. The
preprocessor will expand the COM macros to native C++, making it
possible for the mock generator to parse. To do that, just add /E to
your normal compiler command line. Thanks,

2009/7/9 Zhanyong Wan (λx.x x) <w...@google.com>:

--
Zhanyong

[Zhanyong Wan (λx.x x)]

2009/7/9 Neal Norwitz <nnor...@google.com>:
> Hi Randy,
>
> Thanks for using gmock_gen.  Unfortunately, I don't have a better
> answer that Zhanyong's suggestion.  Right now the parser doesn't
> handle macros at all.  At some point in the future I hope to add this
> support.  If you are interested, I can help you implement this in the
> generator (parser).

I'm thinking that since the COM interface definition is highly
structured, it may not be hard to write a parser for it from scratch
(i.e. not using cppclean at all). This parser may not need to do a
perfect job, but can still be more convenient than going through the
preprocessor. What do you think?

> Zhanyong, it would probably be good to post your recommendation to a
> FAQ or some documentation.

I will.

>
> Cheers,
> n

--
Zhanyong

[Randy Schott]

Thanks for your responses.  I figured the parser was struggling with the macros, since I can't get it to generate mocks for any functions that have macros anywhere near them.  I was resorting to commenting all macros, but I might try Zhanyong's suggestion for now.  It shouldn't be too hard to write a parser that can handle COM interfaces, and since we'll be mocking a lot of them, it makes sense for us if the generator can't handle them.  If I succeed, I'll post it to this group so others can use it. 
  If I can find some spare time, I might also be interested in helping you add macro support to the cppclean parser.  We have a lot of interfaces/classes that use macros in the headers and I'd like to be able to run the generator on those as well.

My last remaining question:  Does the parser support the "interface" keyword?  If I remember correctly, that's not a standard keyword, but specific to VC++.  However, all COM interfaces( and most of our pure virtual interfaces) are technically "interfaces", and not "classes", and the parser doesn't seem to recognize anything that isn't marked "class".  That would be very useful for windows users generating mocks from pure virtual interfaces.  In fact, even with macro support, the parser would still need to support "interface" to be able to generate mocks for COM objects.

Randy

2009/7/9 Neal Norwitz <nnor...@google.com>

2009/7/9 Zhanyong Wan (λx.x x) <w...@google.com>:

> 2009/7/9 Neal Norwitz <nnor...@google.com>:
>> Hi Randy,
>>
>> Thanks for using gmock_gen.  Unfortunately, I don't have a better
>> answer that Zhanyong's suggestion.  Right now the parser doesn't
>> handle macros at all.  At some point in the future I hope to add this
>> support.  If you are interested, I can help you implement this in the
>> generator (parser).
>
> I'm thinking that since the COM interface definition is highly
> structured, it may not be hard to write a parser for it from scratch
> (i.e. not using cppclean at all).  This parser may not need to do a
> perfect job, but can still be more convenient than going through the
> preprocessor.  What do you think?

It's worth a try.   It sounds reasonable, but I know nothing about COM.  :-)

n

[Randy Schott]

2009/7/9 Neal Norwitz <nnor...@google.com>

It doesn't support interface, since I've never heard of it (I don't
program on Windows).  Does it parse the same as class or struct?  If
so, it would probably be trivial to add to the parser.  Can you point
me to documentation or even better some examples?

For the purpose of the parser, "interface" is identical to "class".  Here's the MSDN documentation on the keyword:
http://msdn.microsoft.com/en-us/library/50h7kwtb(VS.71).aspx.  Note that "interface" and "__interface" are equivalent, so both would have to be supported.  The basic idea is that the keyword enforces certain constraints on the class to ensure that it is purely abstract, the important one being the __declspec(novtable) option, which causes the compiler to skip generation of the vtable for the class being defined.  So, if you interpret "interface" as "class", you should be able to parse like normal, since the definitions look exactly the same.

One thing I've thought of is to provide a way for users to their
define macros  for the parser.  That would add the expansion logic to
the parser which should be relatively easy.  Are there many macros
necessary to support COM?

I would be fine with defining my own macros for the parser.  COM isn't too bad on the use of macros.  There are only a handful that I can think of that would need to be supported to handle the majority of cases.  The main annoyance will probably be things like the distinction between "THIS" and "THIS_", "STDMETHOD" and "STDMETHOD_" and the various other defines they use inside their argument lists. 

n

Randy

2009/7/9 Randy Schott <randy...@gmail.com>:

[Vlad Losev]

Hi Scott,

2009/7/9 Randy Schott <randy...@gmail.com>

2009/7/9 Neal Norwitz <nnor...@google.com>

It doesn't support interface, since I've never heard of it (I don't
program on Windows).  Does it parse the same as class or struct?  If
so, it would probably be trivial to add to the parser.  Can you point
me to documentation or even better some examples?

For the purpose of the parser, "interface" is identical to "class".  Here's the MSDN documentation on the keyword:
http://msdn.microsoft.com/en-us/library/50h7kwtb(VS.71).aspx.  Note that "interface" and "__interface" are equivalent, so both would have to be supported.  The basic idea is that the keyword enforces certain constraints on the class to ensure that it is purely abstract, the important one being the __declspec(novtable) option, which causes the compiler to skip generation of the vtable for the class being defined.  So, if you interpret "interface" as "class", you should be able to parse like normal, since the definitions look exactly the same.

One thing I've thought of is to provide a way for users to their
define macros  for the parser.  That would add the expansion logic to
the parser which should be relatively easy.  Are there many macros
necessary to support COM?

I would be fine with defining my own macros for the parser.  COM isn't too bad on the use of macros.  There are only a handful that I can think of that would need to be supported to handle the majority of cases.  The main annoyance will probably be things like the distinction between "THIS" and "THIS_", "STDMETHOD" and "STDMETHOD_" and the various other defines they use inside their argument lists. 

I have thrown together a short Python script that uses regexes to convert MS's definition syntax to C++. The result of running it on your snippet seems to satisfy the compiler.

<START OF SCRIPT>
#!/usr/bin/python2.4

"""Converts COM-style interface definitions to C++ abstract base classes."""

import sys
import re

for line in sys.stdin:
  line = re.sub(r'\bDECLARE_INTERFACE_\s*\(\s*(\w+)\s*,\s*(\w+)\s*\)',
                r'class \1 : public \2', line)
  line = re.sub(r'\s*\bPURE\s*;', ' = 0;', line)
  line = re.sub(r'\s*\bTHIS\b\s*', '', line)
  line = re.sub(r'\s*\bTHIS_\b\s*', '', line)
  line = re.sub(r'\STDMETHOD\s*\(\s*(\w+)\s*\)\s*', r'HRESULT \1', line)
  line = re.sub(r'\STDMETHOD_\s*\(\s*(\w+)\s*,\s*(\w+)\s*\)\s*', r'\1 \2', line)
  sys.stdout.write(line)
<END OF SCRIPT>

HTH,
Vlad

[Randy Schott]

Thanks for the script Vlad.  I'll admit I have zero Python experience, so I had to do some reading.  I modified your script to take in the same parameters as gmock_gen, write the expanded output to a temp file, and then feed that into gmock_gen.  It worked great on the code snippet I provided, but when I tried it on more complex headers, the cppclean parser got confused about some of the other keywords and macros that were used.
  I modified the script to replace "interface" with "class" and that took care of some of the errors.  However, I don't quite understand the error output from cppclean, so I can't tell what some of the other errors mean.  Maybe Neal can shed some light on how to interpret the bad token errors?  It would help to know what the two numbers at the end of each error mean.  They seemed to be character positions?

Randy

2009/7/10 Vlad Losev <vl...@google.com>

[Vlad Losev]

Hi Randy,

Can you provide  pieces of code that are giving you the trouble with gmock_gen.py?

Thanks,
Vlad

2009/7/10 Randy Schott <randy...@gmail.com>

[Randy Schott]

Hi Vlad,

  I tried running it against the Direct3D header in the DirectX SDK ("d3d.h"), and asked it to generate a mock for IDirect3DDevice9.  I can attach a copy if you'd like, but I seem to remember you responding to one of my previous posts about that file, so I'm guessing you already have it?  I already have a mock generated for that class, so it was more of a test case to see if the "un-COMing" script would work against any COM interface headers I could throw at it.  We'll be mocking a lot more of them in the near future so I'm trying to make the workflow a little easier to encourage members on our team to write tests :-).

Randy

2009/7/15 Vlad Losev <vl...@google.com>

[Vlad Losev]

Hi Randy,

2009/7/15 Randy Schott <randy...@gmail.com>

Hi Vlad,

  I tried running it against the Direct3D header in the DirectX SDK ("d3d.h"), and asked it to generate a mock for IDirect3DDevice9.  I can attach a copy if you'd like, but I seem to remember you responding to one of my previous posts about that file, so I'm guessing you already have it?  I already have a mock generated for that class, so it was more of a test case to see if the "un-COMing" script would work against any COM interface headers I could throw at it.  We'll be mocking a lot more of them in the near future so I'm trying to make the workflow a little easier to encourage members on our team to write tests :-).

Here is a modified version of the script that strips a bit more from those COM headers:

#!/usr/bin/python2.4

"""Converts COM-style interface definitions to C++ abstract base classes."""

import sys
import re

for line in sys.stdin:
  line = re.sub(r'\bDECLARE_INTERFACE_\s*\(\s*(\w+)\s*,\s*(\w+)\s*\)',
                r'class \1 : public \2', line)
  line = re.sub(r'\s*\bPURE\s*;', ' = 0;', line)
  line = re.sub(r'\s*\bTHIS\b\s*', '', line)
  line = re.sub(r'\s*\bTHIS_\b\s*', '', line)

  line = re.sub(r'\STDMETHOD\s*\(\s*(\w+)\s*\)\s*', r'virtual HRESULT \1', line)

  line = re.sub(r'\STDMETHOD_\s*\(\s*(\w+)\s*,\s*(\w+)\s*\)\s*',

                r'virtual \1 \2', line)
  line = re.sub(r'\binterface\b', 'class', line)
  line = re.sub(r'\bclass\s+DECLSPEC_UUID\(\s*".*?"\s*\)\s*(\w+)\s*;',
                r'class \1;', line)
  line = re.sub(r'^\s*\#\s*define\s+\w+\(.*\n', '', line)
  line = re.sub(r'\btypedef\s+struct\s+\w+\s*\*\s*\w+\s*,\s*\*\s*\w+\s*;',
                '', line)
  sys.stdout.write (line)
<END OF SCRIPT>

After applying it to d3d9.h Neal's script doesn't choke on it anymore.

Neal - one of the standard C++ constructs in that file that the script doesn't seem to like is

typedef stuct InterfaceName *LPINTERFACENAME, *PINTERFACENAME;

It also chokes on some macro defines.

Regards,
Vlad

[Neal Norwitz]

2009/7/16 Vlad Losev <vl...@google.com>:
> Hi Randy,
>

Hmmm, I just tried that with my version of the parser and it seemed to
do fine. I think it's the same version in gmock now. Can you send me
the d3d9.h file or point me to it?

> It also chokes on some macro defines.

The easiest way for me to deal with these will be to reproduce the
errors. If you want to look into it further, you can try running the
ast.py program on the inputs and see what it gets. That can aid
debugging.

n