1.1.0 - simple_example.py with outbound ABI crashes on Windows

[anatoly techtonik]

I copy/paste the code from

https://cffi.readthedocs.org/en/latest/overview.html#out-of-line-example-abi-level-out-of-line

and it gives the following error on Windows

>py simple_example.py

Traceback (most recent call last):

  File "simple_example.py", line 3, in <module>

    lib = ffi.dlopen(None)         # or path to a library

OSError: cannot load library '<None>': error 0x7e

[Ryan Gonzalez]

Does Windows have a libc that can be dynamically loaded?

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[anatoly techtonik]

Inline ABI example works ok.

--
anatoly t.

[Armin Rigo]

Hi Ryan,

On 2 June 2015 at 23:37, Ryan Gonzalez <rym...@gmail.com> wrote:
> Does Windows have a libc that can be dynamically loaded?
>
> On June 2, 2015 4:25:44 PM CDT, anatoly techtonik <tech...@gmail.com>
> wrote:
>> lib = ffi.dlopen(None) # or path to a library
>> OSError: cannot load library '<None>': error 0x7e

Thanks for noticing. Indeed, in out-of-line mode, ffi.dlopen() no
longer looks around for what you meant (it used to do so using
ctypes.util.find_library()). This is documented in
https://cffi.readthedocs.org/en/latest/cdef.html#ffi-dlopen-loading-libraries-in-abi-mode
, last paragraph of the section. Windows has a libc that can be
dynamically loaded, but you need to jump through hoops to find it; the
easiest is to use ctypes.util.find_library('c'). I'll fix the
example.


A bientôt,

Armin.

[anatoly techtonik]

lib = ffi.dlopen(None) already looks like a non-obvious hack to me.
What about making an explicit API to enable access to C stdlib?

lib = ffi.cstdlib()

[Armin Rigo]

Hi Anatoly,

a) ffi.dlopen(None) matches what you get by typing "man dlopen".

b) It's just an example. In general the library you want to access is
not the C stdlib (a lot of its functions are already exposed via
Python).


Armin

[anatoly techtonik]

On Wed, Jun 3, 2015 at 11:54 PM, Armin Rigo <ar...@tunes.org> wrote:
> Hi Anatoly,
>
> On 3 June 2015 at 12:50, anatoly techtonik <tech...@gmail.com> wrote:
>> lib = ffi.dlopen(None) already looks like a non-obvious hack to me.
>> What about making an explicit API to enable access to C stdlib?
>>
>> lib = ffi.cstdlib()
>
> a) ffi.dlopen(None) matches what you get by typing "man dlopen".

>man dlopen
'man' is not recognized as an internal or external command,
operable program or batch file.

> b) It's just an example. In general the library you want to access is
> not the C stdlib (a lot of its functions are already exposed via
> Python).

Ok. But what if C stdlib is exactly what I want to access? Why use
unix hack for it, if it is possible to get sane API?

[Ryan Gonzalez]

On Wed, Jun 3, 2015 at 4:53 PM, anatoly techtonik <tech...@gmail.com> wrote:

On Wed, Jun 3, 2015 at 11:54 PM, Armin Rigo <ar...@tunes.org> wrote:
> Hi Anatoly,
>
> On 3 June 2015 at 12:50, anatoly techtonik <tech...@gmail.com> wrote:
>> lib = ffi.dlopen(None) already looks like a non-obvious hack to me.
>> What about making an explicit API to enable access to C stdlib?
>>
>> lib = ffi.cstdlib()
>
> a) ffi.dlopen(None) matches what you get by typing "man dlopen".

>man dlopen
'man' is not recognized as an internal or external command,
operable program or batch file.

`man` is a UNIX program. Try it in MinGW or Cygwin.

 

> b) It's just an example.  In general the library you want to access is
> not the C stdlib (a lot of its functions are already exposed via
> Python).

Ok. But what if C stdlib is exactly what I want to access? Why use
unix hack for it, if it is possible to get sane API?

Doesn't ctypes already provide the exact same thing (access the libc) with the exact same API?

 

--
-- python-cffi: To unsubscribe from this group, send email to python-cffi...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/python-cffi?hl=en
---
You received this message because you are subscribed to the Google Groups "python-cffi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-cffi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Ryan

[ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong.

http://kirbyfan64.github.io/

 

[anatoly techtonik]

On Thu, Jun 4, 2015 at 1:00 AM, Ryan Gonzalez <rym...@gmail.com> wrote:
> On Wed, Jun 3, 2015 at 4:53 PM, anatoly techtonik <tech...@gmail.com>
> wrote:
>>
>> On Wed, Jun 3, 2015 at 11:54 PM, Armin Rigo <ar...@tunes.org> wrote:
>> > Hi Anatoly,
>> >
>> > On 3 June 2015 at 12:50, anatoly techtonik <tech...@gmail.com> wrote:
>> >> lib = ffi.dlopen(None) already looks like a non-obvious hack to me.
>> >> What about making an explicit API to enable access to C stdlib?
>> >>
>> >> lib = ffi.cstdlib()
>> >
>> > a) ffi.dlopen(None) matches what you get by typing "man dlopen".
>>
>> >man dlopen
>> 'man' is not recognized as an internal or external command,
>> operable program or batch file.
>>
>
> `man` is a UNIX program. Try it in MinGW or Cygwin.

Null pointer reference. No such thing in MinGW.

http://linux.die.net/man/3/dlopen
...If filename is NULL, then the returned handle is for the main program...

???

>>
>> > b) It's just an example. In general the library you want to access is
>> > not the C stdlib (a lot of its functions are already exposed via
>> > Python).
>>
>> Ok. But what if C stdlib is exactly what I want to access? Why use
>> unix hack for it, if it is possible to get sane API?
>>
>
> Doesn't ctypes already provide the exact same thing (access the libc) with
> the exact same API?

I don't know. Probably not.
--
anatoly t.

[Armin Rigo]

Hi,

Please check the updated page at
http://cffi.readthedocs.org/en/latest/overview.html#out-of-line-example-abi-level-out-of-line
.

You need lib = ffi.dlopen(ctypes.util.find_library("c")) .


Armin

[anatoly techtonik]

Is it possible to go without ctypes import? =/

That's.. important.
--
anatoly t.

[Armin Rigo]

Hi Anatoly,

On 4 June 2015 at 20:37, anatoly techtonik <tech...@gmail.com> wrote:
> Is it possible to go without ctypes import? =/
> That's.. important.

Copy-paste the logic from ctypes/util.py...

Sorry, I don't have anything better to suggest. The point of CFFI's
out-of-line mode is that it only needs to import one C extension
module (in ABI mode) or two (in API mode) but not the whole Python
logic from the cffi package. If you really need to get access
Microsoft's version of the standard C library in ABI mode, but you
don't want to import the whole of ctypes either, then copy the
functions _get_build_version() and find_msvcrt().

As usual, I'd also recommend looking at the API mode, which doesn't
have all these problems but at the cost of building a C extension for
which you need a compiler --- but just like a regular project with a C
extension you can redistribute binary wheels that contain the
precompiled stuff, and so on.


A bientôt,

Armin.

[anatoly techtonik]

On Fri, Jun 5, 2015 at 10:45 AM, Armin Rigo <ar...@tunes.org> wrote:
> Hi Anatoly,
>
> On 4 June 2015 at 20:37, anatoly techtonik <tech...@gmail.com> wrote:
>> Is it possible to go without ctypes import? =/
>> That's.. important.
>
> Copy-paste the logic from ctypes/util.py...
>
> Sorry, I don't have anything better to suggest.

Patch cffi.api.dlopen

diff -r f9bbb11363fc cffi/api.py
--- a/cffi/api.py Fri Nov 21 20:45:12 2014 +0100
+++ b/cffi/api.py Fri Jun 05 16:17:07 2015 +0300
@@ -114,6 +114,10 @@
library we only look for the actual (untyped) symbols.
"""
assert isinstance(name, basestring) or name is None
+ if sys.platform == "win32" and name is None:
+ # port Unix behavior - load C stdlib
+ import ctypes.util
+ name = ctypes.util.find_library("c")
with self._lock:
lib, function_cache = _make_ffi_library(self, name, flags)
self._function_caches.append(function_cache)

No?

> The point of CFFI's
> out-of-line mode is that it only needs to import one C extension
> module (in ABI mode) or two (in API mode) but not the whole Python
> logic from the cffi package.

What is the difference in the logic. Just the loader rewritten in C?

It also feel awkward to split the function definition from a library
name. A step back from future. It would be nice if definitions were
linked to the libraries they are supposed to come from. All this C
library lookup stuff for every platform is a huge mess. If CFFI could
set some standard for library names and placement, at least for
the stuff that can be shipped together with Python packages, and
provided a manual way to override that for those who need the
flexibility - that would provide a value for those of us who are lost
in building cross-platform extensions for Python.

That means that definitions will came as:
"libc": {"""
int printf( .... );
"""}
and then ffi will try to find and load that `libc` automatically. There
could be service like PyPI to resolve ambiguity with C package
names, turning names into identifiers, like "import libc". But that
may not be required, because it looks like these symbols from
external libraries are isolated in module namespace, and the
only thing that is relevant is where from CFFI finds and loads the
library, and if symbols match.

> If you really need to get access
> Microsoft's version of the standard C library in ABI mode, but you
> don't want to import the whole of ctypes either, then copy the
> functions _get_build_version() and find_msvcrt().

Again, it looks like the best place for that code is the generated
.py module, but it doesn't load the lib and don't know the lib name
at all. It looks awkward that I can not query this generated module
about the properties of the library that I need to load for it. Like
checking name, version, comments etc.

> As usual, I'd also recommend looking at the API mode, which doesn't
> have all these problems but at the cost of building a C extension for
> which you need a compiler --- but just like a regular project with a C
> extension you can redistribute binary wheels that contain the
> precompiled stuff, and so on.

I am evaluating the possibility to replace ctypes with CFFI, so that
people could checkout and run things right away from repository
without additional steps for compilation and installation.
--
anatoly t.

[Armin Rigo]

Hi,

On 5 June 2015 at 15:45, anatoly techtonik <tech...@gmail.com> wrote:
>>> Is it possible to go without ctypes import? =/
>>> That's.. important.
>>
>> Copy-paste the logic from ctypes/util.py...
>>
>> Sorry, I don't have anything better to suggest.
>
> Patch cffi.api.dlopen
>

> + if sys.platform == "win32" and name is None:
> + # port Unix behavior - load C stdlib
> + import ctypes.util

By "Is it possible to go without ctypes import", did you mean "it is
essential that I don't have type `import ctypes` myself for some
reason, but that's fine if the import is hidden somewhere else"?

If that's what you meant, then no. The point of CFFI is not to bury
C-isms below layers of Python wrappers that add various special cases.


A bientôt,

Armin.