CFFI_DLLEXPORT macro causes issues & struct's void* function attributes are not recognized in python

[M P]

2 Questions:

1. CFFI_DLLEXPORT macro causes compiler issues. The compiler says it defaults to int.

#ifndef CFFI_DLLEXPORT
#define CFFI_DLLEXPORT __declspec(dllimport)
#endif

CFFI_DLLEXPORT void runStrategyBasic(struct _AlgoServer *);

I gave up and used

 extern struct _AlgoServer AlgoServer;

Compiled just fine. How can I use CFF_DLLEXPORT macro for the windows port?

2. Python does not recognize the void pointers as attributes from my struct.

Struct header:

struct _AlgoServer {
  void (*init)(void);
  void (*destroy)(void);
  void (*getAccountInfo)(char*);
};

extern struct _AlgoServer AlgoServer;

Struct source:

#include "AlgoServer.h"

void init() {

}

void destroy() {

}

void getAccountInfo(char *result) {
  result = "SUCCESSFUL ACCOUNT INFO RETURNED\0";
}

struct _AlgoServer AlgoServer = {
  .init = init,
  .destroy = destroy,
  .getAccountInfo = getAccountInfo
};

Python Build:

# file BuildStrategyBasic.py
import cffi
ffibuilder = cffi.FFI()

with open('../PythonAlgorithms.h') as f:
    # read plugin.h and pass it to embedding_api(), manually
    # removing the '#' directives and the CFFI_DLLEXPORT
    data = ''.join([line for line in f if not line.startswith('#')])
    data = data.replace('CFFI_DLLEXPORT', '')
    ffibuilder.embedding_api(data)

ffibuilder.set_source("StrategyBasic", r'''
    #include "../PythonAlgorithms.h"
''')

ffibuilder.embedding_init_code("""
    from StrategyBasic import ffi

    @ffi.def_extern()
    def runStrategyBasic(server):
        accountResult = ""
        server.init()
        #server.getAccountInfo(accountResult)
        print("account info: %d" % (accountResult))
        print("finished strategy")
""")

ffibuilder.compile(target="libStrategyBasic.*", verbose=True)

It's absolutely vital for my project to call functions from within the python script on live data from the C runtime. So I pass the struct from C to python and call functions from within.

Any assistance would be tremendous.

[M P]

Correction: I changed #1 to

extern void runStrategyBasic(struct _AlgoServer *);

[Armin Rigo]

Hi,

On 23 January 2017 at 05:29, M P <bmpe...@gmail.com> wrote:
> 1. CFFI_DLLEXPORT macro causes compiler issues. The compiler says it
> defaults to int.
>
> #ifndef CFFI_DLLEXPORT
> #define CFFI_DLLEXPORT __declspec(dllimport)
> #endif
>
> CFFI_DLLEXPORT void runStrategyBasic(struct _AlgoServer *);

Ah, I see that you copied these four lines from the docs. That part
of the docs was supposed to be Windows-friendly but was actually
Windows-*only* by mistake. It doesn't compile with gcc. I've fixed
the docs to put a version that compiles on both, see here:

/* file plugin.h, Windows-friendly version */
typedef struct { int x, y; } point_t;

/* When including this file from ffibuilder.set_source(), the
following macro is defined to '__declspec(dllexport)'. When
including this file directly from your C program, we define
it to 'extern __declspec(dllimport)' instead.

With non-MSVC compilers we simply define it to 'extern'.
(The 'extern' is needed for sharing global variables;
functions would be fine without it. The macros always
include 'extern': you must not repeat it when using the
macros later.)
*/
#ifndef CFFI_DLLEXPORT
# if defined(_MSC_VER)
# define CFFI_DLLEXPORT extern __declspec(dllimport)
# else
# define CFFI_DLLEXPORT extern
# endif
#endif

CFFI_DLLEXPORT int do_stuff(point_t *);

(Please tell me if there are still problems.)

> 2. Python does not recognize the void pointers as attributes from my struct.

> (...)
> @ffi.def_extern()
> def runStrategyBasic(server):

It is not very clear to me what you're trying to achieve. From this
def_extern() with name "runStrategyBasic" I guess you're also trying
to use the code found in your question #1, but it's not clear how.
Can you give a complete example that I can reproduce? Your question
is close, but not quite there: please give all files with their names,
and tell what you're trying to run and what the output of these
commands is.

(Note a purely C issue:

void getAccountInfo(char *result) {
result = "SUCCESSFUL ACCOUNT INFO RETURNED\0";
}

This doesn't return the string to the caller. The code is like "void
foo(int x) { x = 42; }", where the assignment to x has no effect at
all.)


A bientôt,

Armin.

[M P]

Thanks for the macro tip. That works 100%.

I am creating a program that needs to have access to the functions defined in python scripts. CFFI builds the scripts into a dylib that I can link so the C program can invoke those implemented functions. For instance:

int main(void) {
   AlgoServer.init();
   AlgoServer.beginTime();
   runBasicStrategy(AlgoServer);
   runOptimizedStrategy(AlgoServer);
   AlgoServer.endTime(); // print time to run 2 strategies
   AlgoServer.destroy(); // destroy allocated objects and free networking instances
   return 0;
}
   

So in the dylib for BasicStrategy, I followed the docs for embedding.

@ffi.def_extern()
def runStrategyBasic(server):
   # do some math
   var totalCosts = server.getTotalCosts() # Fetch data from db. This function is implemented in C in the struct
   # do some more math
   server.storeResults('StrategyBasic', finalScore) # Store data into db. This function is implemented in C and takes a string and an integer
   print("finished running strategy")

[Armin Rigo]

Hi,

On 23 January 2017 at 19:03, M P <bmpe...@gmail.com> wrote:
> I am creating a program that needs to have access to the functions defined
> in python scripts. CFFI builds the scripts into a dylib that I can link so
> the C program can invoke those implemented functions. For instance:

You said "Python does not recognize the void pointers as attributes
from my struct". I don't really understand what you mean by that.
That's why I suggest you give a detailed, step-by-step guide that I
can follow on my side, complete with file names and contents and
commands to execute; something that would give me the same problem as
the one you are seeing.


A bientôt,

Armin.

[M P]

AlgoServer struct definition:

 

struct _AlgoServer {
  void (*init)(void);
  void (*destroy)(void);
  void (*getAccountInfo)(char*);
};

extern struct _AlgoServer AlgoServer;

Later in the C file, I define the function pointers:

Enter code here.

#include "AlgoServer.h"

void init() {

}

void destroy() {

}

void getAccountInfo(char *result) {
  result = "SUCCESSFUL ACCOUNT INFO RETURNED\0";
}

struct _AlgoServer AlgoServer = {
  .init = init,
  .destroy = destroy,
  .getAccountInfo = getAccountInfo
};

..

See the attributes init, destroy, getAccountInfo

In the python script:

def runStrategyBasic(server):
        accountResult = ""
        server.init()
        #server.getAccountInfo(accountResult)
        print("account info: %d" % (accountResult))
        print("finished strategy")

I invoke the attribute init using server.init()

The interpreter says init is not an attribute when it clearly is.

[Armin Rigo]

Hi again,

Sorry, I think I get what you are saying, but there must be something
else going on because I tried to write a small example and it works as
expected. So I need to see everything to figure out where the problem
really is.

As an example of what I need, I have uploaded my reconstructed
version, which works fine:
https://gist.github.com/arigo/147a09e81c6b5a3580496a5100b5675e . You
can see that if you download all files and run the commands in the
README, you should get the same result. So your problem comes from
somewhere else, and to figure it out, I need to see a similar paste
containing your version of the code. You can use
https://gist.github.com/ or copy-paste into your mail, but it needs to
be complete: I must be able to run it.


Thanks!

Armin

[M P]

This is exciting! I knew it would be possible. However I am still having the same error. I have attached my code as a zip to this reply.

Mavericks-iMac:tests maverick$ ./TestAlgoServer
From cffi callback <function runStrategyBasic at 0x10db93c80>:
Traceback (most recent call last):
  File "<init code for 'StrategyBasic'>", line 6, in runStrategyBasic
AttributeError: '_cffi_backend.CData' object has no attribute 'init'

I am using latest CFFI libraries

[Armin Rigo]

Hi,

On 24 January 2017 at 21:40, M P <bmpe...@gmail.com> wrote:
> This is exciting! I knew it would be possible. However I am still having the
> same error. I have attached my code as a zip to this reply.

Ah, the problem is that ``struct _AlgoServer`` is never declared in
BuildStrategyBasic or any file it reads. So the attempt to read
``server.init`` fails, because it's just an opaque type at this point.

You get the same error if you take the example I did and move the
``struct _AlgoServer { /* stuff */ };`` part from ``algoserver.h`` to
``main.c``.

We could improve the error message when trying to read a field out a
struct that happens to be opaque (as opposed to, e.g., just doing a
typo in the field name, or confusing one struct type with another).


A bientôt,

Armin.

[M P]

What improvements can be made? init is a real type that is pointing to null until run-time. That shouldn't be an issue.

Until then, I'll merge the implementations into the same file but this isn't a concrete solution.

--
-- python-cffi: To unsubscribe from this group, send email to python-cffi+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/python-cffi?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "python-cffi" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-cffi/nVl6xN-LrvY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to python-cffi+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Armin Rigo]

Hi,

On 25 January 2017 at 00:19, M P <bmpe...@gmail.com> wrote:
> What improvements can be made? init is a real type that is pointing to null
> until run-time. That shouldn't be an issue.

You need to move

struct _AlgoServer {
void (*init)(void);
/* etc., more */
};

into the file PythonAlgorithms.h. You have a #include but it is
ignored on line 8 in BuildStrategyBasic.py (which is correct, #include
cannot be processed here). I would recommend that you end up with
only one .h file instead of two.


A bientôt,

Armin.

[M P]

That's not what I'm asking. I have resolved the issues. I'm discussing CFFI as a library. There has to be a way that original code is acceptable. During run-time the AlgoStruct instance has its pointers set to defined functions. init is a real attribute.

Secondly, how would you write this code:

void getAccountInfo(char *result) {
  result = malloc(strlen("SUCCESSFUL ACCOUNT INFO RETURNED\0")*sizeof(char));
  strcpy(result,"SUCCESSFUL ACCOUNT INFO RETURNED\0");
  // printf("result is %s", result);
}

So that passing a string (or any structure) from python is changed in the C environment? Like so:

def runStrategyBasic(server):
        accountResult = ""
        server.init()

        server.getAccountInfo(accountResult)

I plan on setting structures for bigger scripts this way:

void getWallet(struct Wallet* wallet) {
    wallet = new Wallet();
    wallet.cash = AlgoServer.getCash();
    wallet.profit = AlgoServer.getProfit();
    wallet.losses = AlgoServer.getLosses();
}

and in the script:

def runStrategyBasic(server):
        accountResult = ""
        server.init()

        var wallet = None
        server.getWallet(wallet)

        if wallet.losses > 100:
                printf("not investing")
        else
                server.buyBitCoin()

[M P]

How can I achieve the desired functions from above?

[Armin Rigo]

Hi,

On 25 January 2017 at 00:52, M P <bmpe...@gmail.com> wrote:
> That's not what I'm asking. I have resolved the issues. I'm discussing CFFI
> as a library. There has to be a way that original code is acceptable. During
> run-time the AlgoStruct instance has its pointers set to defined functions.
> init is a real attribute.

I think you may be missing the point of why there was a problem. It's
the same reason for why this code doesn't compile as C:

int foo(struct random_undeclared_name *p)
{
return p->x; /* <= the error is here */

}


> So that passing a string (or any structure) from python is changed in the C
> environment?

It's not possible to change a Python string: in Python, the strings
are immutable. But more importantly, the C code you're suggesting
shows that you are not familiar with C. I would recommend finding
regular C tutorials or other non-cffi-related forums to discuss these
questions. You might learn, for example, that the pattern you're
trying to code could look like that:

void getinfo(char *result, size_t result_size)
{
strncpy(result, "Hello world", result_size);
result[result_size - 1] = 0;
}

i.e. pass a buffer and a buffer size to the function; then, you'd call
it like that in C:

char buf[200];
getinfo(buf, sizeof(buf));
/* and use the string in 'buf' */

So, the question that has its place in this mailing list is how to
turn these two lines to Python code, and the answer is:

buf = ffi.new("char[]", 200)
server.getinfo(buf, len(buf))
python_string = ffi.string(buf)


A bientôt,

Armin.

[M P]

I see. The strings in C are not per memory but 'baked' into the program. I would need to use special functions to allocate, deallocate, and change the string. It's too bad python strings are immutable... and I should used char lists instead (as an alternative to the solution you have provided).

Thank you for your assistance.