Hoping to find an integration example

[Stephen Nuchia]

Hello, I'm new to PyKE and this group, have a long history of mixed-
language development including use of Prolog in production data
processing problems ...

I am not all that experienced with .NET but I am looking at a possible
performance win from bolting PyKE onto an existing "legacy" MFC
(non-.NET) application using IronPython 2.7.

I can find examples for each step of the chain: hosting .NET in an MFC
application, hosting IronPython on .NET, and hosting PyKE on
IronPython. Would it be too much to hope for an example that
incorporates all the elements?

I may have to build it myself but I thought I'd ask first.

TIA,
-swn

[A.T.Hofkamp]

On 05/18/2011 07:08 PM, Stephen Nuchia wrote:
> I can find examples for each step of the chain: hosting .NET in an MFC
> application, hosting IronPython on .NET, and hosting PyKE on
> IronPython. Would it be too much to hope for an example that
> incorporates all the elements?

Sounds like too much to ask imho :)

Can I give you an alternative way to connect MFC to Pyke?

As a Unix person, I don't believe in big executables.

I'd write a plain Python program with Pyke that solves your problem after reading a file or pipe
with data. It produces the problem again in some format.
At the .NET end, run that program as a child process, feeding it the problem data, and reading the
solution afterwards (either use files or pipes to communiate with it).

In my opinion, the advantage of that approach is that you have a more modular system, where you can
replace one part without having the other part drop on the floor too.
Also, you can use the child program independently of the MFC, good for eg fdebugging, but perhaps
also in other contexts useful.

The disadvantage is of course that you have to communicate using text streams.
Either devise a simple human-readable file format, or use something standard, eg JSON or an XML
stream (although I am not a big fan of XML tbh).

Greetings,
Albert

[Stephen Nuchia]

As a Unix person (and an early fan of TCL, expect, etc) I agree with
you. But in the world of Windows commercial applications that
approach is foreign. In part because the OS does not support IPC very
well (though that is mostly fixed if you can assume a modern version
of Windows) but mostly because the owner of the program doesn't want
to make its workings so obvious.

So, wearing my day job hat, I am going to plug ahead on a single
executable. In Windows the compiled code interoperability goal gets
addressed by packaging reusable units in DLLs (dynamic link
libraries, like .so) with some kind of late binding support. There
are decades of layers and generations of those binding conventions to
choose from, of course. Some of them offer a choice of in-process and
out-of-process instantiation of the functionality being imported.

It all makes me want to run screaming back to school, frankly. But
it's where most of the money is, sadly.

More fundamentally, though -- I need the resulting plan compiled into
object code inside my process, where the data it is to be applied to
live. Sending the data across a process boundary would defeat the
purpose, which is to get high performance by eliminating branches in
the inner loop body. I can take a lot of branch prediction misses for
the cost of a context switch.

I could still have PyKE create the plan in a separate process but I
will need to host the DLR in my process anyway, may as well host PyKE
too.

At least, that's how I'm seeing things right now. As always,
performance guesswork is subject to +/- 10,000% revision once I get a
chance to test it.

Thank you for responding, you've got me mulling over the out-of-
process option. The PyKE process could use the .NET version of the
library that understands the on-disk format of the data and I wouldn't
have to serve it up from the main process's memory ... hmm ....

[Bill Janssen]

Why drag .NET into it at all?

Just use Python and PyKE on Windows with your legacy MFC app. Python
can load DLLs and can also be embedded as a DLL in other programs.

[Stephen Nuchia]

> Why drag .NET into it at all?

Because that's where the just-in-time compiler lives. Byte-code
interpreters take a lot of branch prediction stalls too, I may as well
stay with the existing code (switching on column descriptors in the
inner loop) without the jitter.

[Bill Janssen]

Have you tried psyco? It's a JIT compiler for Python.

Bill

[Stephen Nuchia]

> Have you tried psyco?  It's a JIT compiler for Python.

Looks promising, but it's 32-bit only and I see no indication that
will be changing soon. For my "day job" applications it's going to
have to be .NET most likely. The product has acre-feet of MFC and
Win32 API code, it's not like we're going to be porting it to another
OS any time soon.

On the research side, hacking in x64 and GPGPU code generation would
be a hoot. I need the address space for some of the algorithms I'm
playing with. The main optimization trick psyco seem to employ,
specialization on the parameter types, is not completely orthogonal
with what PyKE can do but having psyco in the back end would permit
generation of more structured plans without compromising performance.

For my primary application I was going to generate a functions
specific to the input parameter types anyway so psyco and the .NET
jitter should be comparable from a performance point of view.

I've been too far from open source too long, I'd forgotten how much
like a game of rogue it is. Once you get into it far enough the
challenge isn't finding enough weapons, it's figuring out which ones
to keep. And not picking up the cursed ones, of course :-)

Thanks for the tip; if nothing else it's a cool piece of compiler tech
to play with.

[Bruce Frederiksen]

I haven't, but would be interested to hear if anybody else has tried it.

I also haven't tried Pyke on pypy yet.  I would think that pypy would beat psyco by quite a bit, but ???  If anybody has tried pypy, I'd like to hear about that too!

Thanks Bill!

-Bruce

--
You received this message because you are subscribed to the Google Groups "PyKE" group.
To post to this group, send email to py...@googlegroups.com.
To unsubscribe from this group, send email to pyke+uns...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pyke?hl=en.

[Bill Janssen]

Turns out that PyKE works fine on IronPython, with one minor tweak.
You have to remove the import of "subprocess" in pyke/special.py,
because IronPython 2.7 doesn't have subprocess.

Or at least, it works fine for my application :-).

Bill

On May 20, 6:37 pm, Stephen Nuchia <snuc...@gmail.com> wrote:

[Stephen Nuchia]

In a recent thread Bruce invited anyone "still out there" to post to the PyKE google group.  I was able to accomplish my immediate goals without resorting to dynamic languages and dynamic meta-programming.  But I've softened up my management for the concept and I am still forecasting adoption of PyKE (or identification of an insurmountable obstacle) in the not-too-distant future.

-swn