[PySide] Binding for a C library using shiboken

[anatoly techtonik]

Hello,

Is it possible to generate bindings for C libraries with shiboken?

I understand that C is a subset of C++, but I can't generate anything from C header file.

For example, I want to create binding for avbin_get_version() function available from

https://github.com/AVbin/AVbin/blob/master/include/avbin.h

I do:

$ shiboken lib/AVbin/include/avbin.h typesystem.xml

Where typesystem.xml is the following:

<?xml version="1.0"?>

<!-- the name of the module as it will be imported from Python-->

<typesystem package='AVbin'>

  <function signature='int avbin_get_version()' rename='version'/>

</typesystem>

First of all it always complains about "No C++ classes found!"

Then I see the message:

Global function 'int avbin_get_version()' is specified in typesystem, but not defined. This could potentially lead to compilation errors.

If I modify the <function> with signature='avbin_get_version()' I get another error:

skipping function '::avbin_get_version', unmatched return type 'int'

So, obviously, shiboken is able to parse avbin.h and find avbin_get_version() function with 'int' return type, but why it complains that the function is not defined when the typesystem is fixed?

-- 

anatoly t.

[Hugo Parente Lima]

On Wednesday 14 December 2011 12:05:57 anatoly techtonik wrote:
> Hello,
>
> Is it possible to generate bindings for C libraries with shiboken?

Yes you can, maybe shiboken isn't the better tool to bind C code, but it can
do the job, and if you want a nicer and more OO API for your bindings you can
do what Marcelo did here:

http://www.setantas.net/blog/2011/03/08/python-bindings-for-libepub-using-
shiboken/

You also need to care care of putting all your includes inside a "extern C",
because shiboken generates C++ code.

> I understand that C is a subset of C++, but I can't generate anything from
> C header file.
> For example, I want to create binding for avbin_get_version() function
> available from
> https://github.com/AVbin/AVbin/blob/master/include/avbin.h
>
> I do:
> $ shiboken lib/AVbin/include/avbin.h typesystem.xml
>
> Where typesystem.xml is the following:
>
> <?xml version="1.0"?>
>
> <!-- the name of the module as it will be imported from Python-->
> <typesystem package='AVbin'>
> <function signature='int avbin_get_version()' rename='version'/>
> </typesystem>
>
>
> First of all it always complains about "No C++ classes found!"

You can ignore this message.

> Then I see the message:
> Global function 'int avbin_get_version()' is specified in typesystem, but
> not defined. This could potentially lead to compilation errors.
> If I modify the <function> with signature='avbin_get_version()' I get
> another error:
> skipping function '::avbin_get_version', unmatched return type 'int'

Probably this function wasn't found in your global header, the header file
read by shiboken to find what classes/function can be bound, look at the log
files generated by the generator.

> So, obviously, shiboken is able to parse avbin.h and find
> avbin_get_version() function with 'int' return type, but why it complains
> that the function is not defined when the typesystem is fixed?

--
Hugo Parente Lima
INdT - Instituto Nokia de Tecnologia

[anatoly techtonik]

On Wed, Dec 14, 2011 at 7:09 PM, Hugo Parente Lima <hugo...@openbossa.org> wrote:

On Wednesday 14 December 2011 12:05:57 anatoly techtonik wrote:
> Hello,
>
> Is it possible to generate bindings for C libraries with shiboken?

Yes you can, maybe shiboken isn't the better tool to bind C code, but it can
do the job, and if you want a nicer and more OO API for your bindings you can
do what Marcelo did here:

http://www.setantas.net/blog/2011/03/08/python-bindings-for-libepub-using-
shiboken/

This tutorial requires writing C++ wrapper around C code. Is it possible to avoid that?

 

You also need to care care of putting all your includes inside a "extern C",
because shiboken generates C++ code.

Could you, please, expand this a bit? Do I need to do this in avbin.h file, or in generated .cpp/.h files?

(If I was comfortable with C/C++, I would probably already created Python binding by hand).

> I understand that C is a subset of C++, but I can't generate anything from
> C header file.
> For example, I want to create binding for avbin_get_version() function
> available from
> https://github.com/AVbin/AVbin/blob/master/include/avbin.h
>
> I do:
> $ shiboken lib/AVbin/include/avbin.h typesystem.xml
>
> Where typesystem.xml is the following:
>
> <?xml version="1.0"?>
>
> <!-- the name of the module as it will be imported from Python-->
> <typesystem package='AVbin'>
>   <function signature='int avbin_get_version()' rename='version'/>
> </typesystem>
>
>
> First of all it always complains about "No C++ classes found!"

You can ignore this message.

> Then I see the message:
> Global function 'int avbin_get_version()' is specified in typesystem, but
> not defined. This could potentially lead to compilation errors.
>  If I modify the <function> with signature='avbin_get_version()' I get
> another error:
> skipping function '::avbin_get_version', unmatched return type 'int'

Probably this function wasn't found in your global header, the header file
read by shiboken to find what classes/function can be bound, look at the log
files generated by the generator.

They are empty. 1032 bytes each.

Console output: http://pastebin.com/fU7rzmXi

Generated files:

- out/AVbin/avbin_module_wrapper.cpp - http://pastebin.com/GUkTiksN

- out/AVbin/avbin_python.h - http://pastebin.com/KmGDW8xi

I've uploaded the project to:

https://bitbucket.org/techtonik/shiboken-avbin

[Hugo Parente Lima]

On Wednesday 14 December 2011 19:15:22 anatoly techtonik wrote:
> On Wed, Dec 14, 2011 at 7:09 PM, Hugo Parente Lima
>
> <hugo...@openbossa.org>wrote:
> > On Wednesday 14 December 2011 12:05:57 anatoly techtonik wrote:
> > > Hello,
> > >
> > > Is it possible to generate bindings for C libraries with shiboken?
> >
> > Yes you can, maybe shiboken isn't the better tool to bind C code, but it
> > can
> > do the job, and if you want a nicer and more OO API for your bindings you
> > can
> > do what Marcelo did here:
> >
> > http://www.setantas.net/blog/2011/03/08/python-bindings-for-libepub-using

> > - shiboken/

>
> This tutorial requires writing C++ wrapper around C code. Is it possible to
> avoid that?

Yes it is, the C++ wrapper was done just to be able to use libepub in the OO
way in Python, but you can avoid that.

> > You also need to care care of putting all your includes inside a "extern
> > C",
> > because shiboken generates C++ code.
> Could you, please, expand this a bit? Do I need to do this in avbin.h file,
> or in generated .cpp/.h files?

When linking C code using a C++ compiler you must tell the C++ compiler that
the function is a C function, not a C++ function, you do this putting the
function declaration inside a block like:

extern "C" {
}

Some C libraries already do that in their headers, but if your library doesn't
do that you need to find a way to have all your C includes inside a extern "C"
block.

The problem is that Shiboken was meant to be used to wrap C++ code, not C, so
the code generator doesn't write the "extern C" before including the library
headers and you need to find a way to do that. Thinking a bit I don't know if
there's a clean way to do that with the current version of Shiboken.

No, I meant the log files created by the generator, something like
mjb_rejected_functions.log, the log must explain why the functions were
rejected.

> I've uploaded the project to:
> https://bitbucket.org/techtonik/shiboken-avbin

--

[anatoly techtonik]

On Wed, Dec 14, 2011 at 11:38 PM, Hugo Parente Lima <hugo...@openbossa.org> wrote:

> > You also need to care care of putting all your includes inside a "extern

> > C",
> > because shiboken generates C++ code.
> Could you, please, expand this a bit? Do I need to do this in avbin.h file,
> or in generated .cpp/.h files?

When linking C code using a C++ compiler you must tell the C++ compiler that
the function is a C function, not a C++ function, you do this putting the
function declaration inside a block like:

extern "C" {
}

Some C libraries already do that in their headers, but if your library doesn't
do that you need to find a way to have all your C includes inside a extern "C"
block.

I've wrapped code inside avbin.h into extern "C" block following the excellent explanation from http://www.parashift.com/c++-faq-lite/mixing-c-and-cpp.html but it still doesn't help.

The problem is that Shiboken was meant to be used to wrap C++ code, not C, so
the code generator doesn't write the "extern C" before including the library
headers and you need to find a way to do that. Thinking a bit I don't know if
there's a clean way to do that with the current version of Shiboken.

Do you mean that if I execute `shiboken lib/AVbin/include/avbin.h typesystem.xml` shiboken can not detect if avbin.h is C or C++ header?

All logs look the same: http://paste.kde.org/159104/

I've specified --debug-level=full but it added only one line to console output that doesn't explain anything:

DEBUG :: Parsed: 'typesystem.xml', 2 new entries 

[anatoly techtonik]

Can anybody help me? I am really stuck.

I can't understand why C++ was chosen to write the tool like Shiboken. If it was in Python - it was much more easier to troubleshoot the issues like this one.
--
anatoly t.

[João Ventura]

Hi,

have you considered Cython? I've used it already for some simple c to python modules, and it is very easy to use..

Regards,
João Ventura

_______________________________________________
PySide mailing list
PyS...@lists.pyside.org
http://lists.pyside.org/listinfo/pyside

[Hugo Parente Lima]

Yes, nobody can, even a human being.

--

[Hugo Parente Lima]

On Thursday 15 December 2011 19:06:31 anatoly techtonik wrote:
> Can anybody help me? I am really stuck.
>
> I can't understand why C++ was chosen to write the tool like Shiboken. If
> it was in Python - it was much more easier to troubleshoot the issues like
> this one.

Historic reasons, besides the fact that the C++ parser we have was written in
C++. I already tried to create bindings for ApiExtractor (where the C++ parser
lives) using the current Shiboken and rewrite the generator in Python, but
it's a lot of work and I had no free time to do that so I forgot this idea.

As I said Shiboken was intended to be used to wrap C++ libraries, not C. It
can be used to wrap C libraries but for sure there are better and simpler
tools to do that.

--

[anatoly techtonik]

On Fri, Dec 16, 2011 at 8:20 PM, Hugo Parente Lima <hugo...@openbossa.org> wrote:

On Thursday 15 December 2011 19:06:31 anatoly techtonik wrote:
> Can anybody help me? I am really stuck.
>
> I can't understand why C++ was chosen to write the tool like Shiboken. If
> it was in Python - it was much more easier to troubleshoot the issues like
> this one.

Historic reasons, besides the fact that the C++ parser we have was written in
C++. I already tried to create bindings for ApiExtractor (where the C++ parser
lives) using the current Shiboken and rewrite the generator in Python, but
it's a lot of work and I had no free time to do that so I forgot this idea.

I see.

As I said Shiboken was intended to be used to wrap C++ libraries, not C. It
can be used to wrap C libraries but for sure there are better and simpler
tools to do that.

Ok. So, what should I do to generate binding for a C library?

Here is the include file: avbin.h - http://paste.kde.org/175928/

Here is the typesystem.xml - http://paste.kde.org/175934/

I execute `shiboken avbin.h typesystem.xml` and get nothing exported to Python. You can repeat it yourself.

-- 
anatoly t.

[Marcelo Lira]

> _______________________________________________
> PySide mailing list
> PyS...@lists.pyside.org
> http://lists.pyside.org/listinfo/pyside
>

Anatoly,

if you ran Shiboken with your type system file it will issue a warning
telling that it couldn't find your function:

Global function 'int avbin_get_version()' is specified in typesystem,
but not defined. This could potentially lead to compilation errors.

That's because you should not write the return value in the signature,
so this will do:

<function signature='avbin_get_version(void)' />

It will still not work if you don't add a line to declare that you
will be using an integer as primitive type, so add this too:

<primitive-type name='int'/>

It will generate the file "out/AVbin/avbin_module_wrapper.cpp" with
the wrapper for your function.

Cheers,
Marcelo

[anatoly techtonik]

Thanks, Marcelo. That was the missing part: the format of function signature is not documented anywhere, so I have big problems trying to figure out the proper syntax. I created a report to make Shiboken produce more informative message if signature is invalid. I also didn't know that standard types like 'int' need explicit conversion - it would be helpful is this could be explained in more detail somewhere.

P.S. Trying to find out the proper command to compile my code and check that all dependencies are here.
-- 
anatoly t.