Ephestos : A new Blender GUI and a visual programming language and environment for cpython 3.2 and Blender

[kilon]

Hello there, I am a blender developer, that recently had the crazy
idea of taking a ambitious project that would bring visual code and
live code to python . Does it remind you something ? Imagine my
surprise when a friend recommended me your project. The surprise
turned to shock as even my idea of using supercollider has been
similar to your idea of interfacing with MAX/MSP.

Anyway the project is located here with a detail description for those
not sensitive to boring details --> http://kilon.github.com/Ephestos/

But I am not here to shamelessly promote my project, I am here to
discuss how field can be integrated with blender. Because its clear to
me that field and Ephestos have a lot common goals and I put a very
big emphasis on "a lot" so it would be insane for me not to exploit
your project in any way. Don't get me wrong there plenty of visual
code and live coding frameworks out there, but your project genially
stand on the same ideological ground as mine.

The only obstacle here is cpython , and blender embeds cpython 3.2 . I
think this is a double gain here, I will gain immensely from using
your amazing project as the basic for my project and you will gain
immensely by approaching a very live , popular , friendly and
extremely active community , Blender community.

As you may have guessed by now I am a looking for a way to make a
bridge between Blender and field. What that means to field, that it
will no longer has at disposal only its own graphic engine but also
the super powerful Blender engine both in user level and coder level.
I am not new to Blender development I have already an existing blender
add-on that had some quite good reviews and its fairly powerful and
deep, GYES (random material and texture generator) -->
http://wiki.blender.org/index.php/Extensions:2.6/Py/Scripts/System/Gyes.
I am not a good coder per se, but I think I can manage something like
this. Ephestos already implements a new GUI for blender and it works
as an add-on as you can see from visiting my link to my repo that is
included in the link I posted.

I could port field to blender , move java and jython source to blender
cpython , but that would be a tiny bit stupid and a waste of time. So
I think the ideal is to bridge the two.

Now bridging jython with cpython is possible and my first choice so
far is execnet ---> http://codespeak.net/execnet/

But I wanted your opinion on this as well, you are the creators of
field so you are the ones that know field inside out. I have not used
field yet though I am about to use it ASAP , I have however downloaded
your source and studying it , but I also needed your expert opinion.
If you feel that I am abit too excited about field, you are absolutely
right , field saves me years of Ephestos development and I can now
concentrate on things I could not even previously dream of. I would
love to port this with your help and advice to Blender.

[Eric Ameres]

Very interesting idea. I'm a blender fan as well as a field fan. For years I've been meaning to get my hands more into the guts of both, so I'd love to at least hear more of your project and at this point at least offer my moral (morale?) support !

Eric.

> --
> You received this message because you are subscribed to the Google Groups "Field-development" group.
> To post to this group, send email to field-de...@googlegroups.com.
> To unsubscribe from this group, send email to field-developm...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/field-development?hl=en.
>

[kilon]

The initial plan was to divide the project , Ephestos, to 3 parts, 3
ages as I call them cause they are made one after the other as the one
depends on another.

The first is Morpheas Age which is the first and is the GUI.
"Morpheas" name is not accidental because more or less is a port of
Morphic GUI. Morphic for those are not aware of is the first effort of
visual coding around 40 years ago from the first programming language
to implement object orientation the programming language self. Morphic
is a GUI that is not just focused on some widgets that the user
operates , what separates it from the rest is that anything can be a
morph including language atoms (elements of the language syntax) . The
morph class is the big super class that all other morphs inherit from,
even code can be represented using morphs. For morphic its really easy
to do so because self and later smalltalk understand only objects. For
example in smalltalk an "if" , and even a "for" or "while" loop is an
object. Because a morph is of course an instance object , then a morph
can map to any other object. Smalltalk took the idea of morphic even
further and of course extend it with the peak being Squeak a modern
implementation of smalltalk and Pharo that run currently on all
platforms. Morpheas is an ancient greek god (aka Morpheus) that could
take any shape, which is also the prime goal of Morphic , to represent
everything and anything.

So the idea of Morpheas is to map morphic and bring it to python.

The second age is Proteas that maps morphs to code , creating visual
code. Every single language atom as I already explained can be a
morph , and morph can embed other morph , so a morph can a a simple
"if" and even a complete application with its gui. I have not touched
Proteas but because it deeply depend on Morpheas the foundations of
Proteas are already ready.

The third age is Orpheas , which is basically nothing more than a GUI
and visual code front end for Supercollider.

At the moment I have implemented the foundation of Morpheas, meaning
the Morph class (Morph is monolithic so it even handles its own
events) , the World morph which is the morph that contains all other
morphs (there was the idea and its currently possible to have multiple
worlds) , the node which handles all the parenting of morph (a morph
can be a child of another morph and one can ask a morph for its
children and parents and it will reply ... deeply influnced by
smalltalk messages , see also objective c that borrows this concept
from smalltalk ) then Hand morph that represent the mouse cursor and
it handles Blender events like mouse keyboard and others as long as
the cursor (hand morph) is on top of a morph thus making possible to
use Morpheas together with the existing beldner gui without the one
blocking the other (If the the hand is not on top of a morph it passes
all events back to blender and of course blender responds to them
appropriately ) and of course the hand trigger the proper events of a
Morph that are nothing more than Morph class methods. Currently its
possible to move morph by mouse. There is also text morph both single
line and multiline , input text morph ( called stringfield morph, does
not work as well I would like it to work but I am improving it right
now) , a basic menu morph functionality , wip button morph
functionality and others. At the moment it looks ugly as hell , but
cosmetic was not my prime concern for such early stage. Most of the
code is a port from pymorphic.py by another author so I dont claim
that I made all that myself , I just edit, test, modify and extend
appropriately. Another coder and friend has helped me trememdously
with the code.

So to conclude I have already an addon that make some extremely
basic , limited GUIs and it uses Blender bgl for rendering. Blender
bgl is tapping into the drawing of blender windows, its nothing more
than opengl wrapping so there are no big surprises there. So morpheas
drawing is all pure opengl 1 /2 calls.

Field already is doing what I wanted to do with Proteas extremely
well. The visual code and the live coding part and this is the one
thing that interests me deeply. My only problem is how to bring this
to blender or should I just port Ephestos to field and call it a day.
Porting Moprheas is fairly easy since I use only pure python code (if
you exclude the fact that is python 3 code so it may need some extra
care) and even the draw calls which are a small part of the code is
just opengl.

I am thinking of a Ephestos as a field plugin if you want or just
something that works inside field that will be able to generate
blender addons. As I said its still early and I am just brainstorming.
I am happy to inform you that I have download field, it works so far
ok on Macos Lion and I am in the process of reading documentation and
understanding its mechanics.

Sorry for the long post but Morphic takes some explaining because
though extremely old , is very different to existing GUIs and
surprisingly monolithic.

If you want to take a look at modern version of Morphic give Pharo a
try , which is the most modern incarnation ---> http://www.pharo-project.org/home.
Many of the tricks that field can do , pharo can do already (and both
self and smalltalk could do already back in the 70s as it was one of
the first software to implement GUIs and mouse support) , the only
problem is that is based on smalltalk and not python.

So I am really trying to figure out how to bring blender closer to
field , and not so much field to Ephestos , cause since Ephestos aims
at pure python, porting it is not a big issue. I dont expect you to
do the coding for me, if you can point me to the right direction and
advice me (from the field side of course) thats all the help I will
ever need.

Are there any plans to support cpython 3 as well ?

[kilon]

Answering to myself, yes it seems Field offers some form of
communicating with 3d apps (Maya) and especially cpython as explained
here so its a matter of testing how appropriates these solutions are
for blender

http://openendedgroup.com/field/wiki/MayaIntegration

I am also very happy that osc can work with Field , that makes my
communication with supercollider for the Orpheas age so much easier.

http://openendedgroup.com/field/wiki/OscPlugin

I have buried myself inside the wiki , so I think its a matter of days
before I have fairly good grasp of how field works and what can and
cannot do already and how Ephestos fits inside all this.

[Marc Downie]

Kilon, 

Thank you for the fascinating set of links.

I think the real issue is to figure out where the computation goes. If we can come up with a very thin interface to Blender's embedded CPython runtime that fits across, say, Execnet then much of what you are planning to do can be done in Field. I'm no Blender expert, so I don't really know how wide the API is, or what you intend to do with it, but getting Field executing code in a Blender process, and getting autocomplete working should be straightforward. I'd also be happy to put some time into whatever needed to be done to make Execnet "fit" into Field as seamlessly as possible if that seems like a better route to general "remote python" support. 

As for Supercollider in Field, OSC is one option, but don't miss Overtone (Clojure bindings to Supercollider), which is happily hosted directly in Field. 

best of luck, and let me know your thoughts about how this should be architected.

Marc.

[kilon]

And I thank you for this amazing tool, you made my dreams (for visual
coding ) come true and still cant believe it .

Blender API is huge, and I am not exaggerating, here is a taste

http://www.blender.org/documentation/blender_python_api_2_62_release/genindex.html

no worries, I can manage the blender side , its no marvel of code
design , and is certainly not pythonic enough for my tastes but its
very flexible and extremely powerful. Its not bad in terms of
simplicity just awkward at times. With its fair share of issues of
course. But we at #blenderpython and #blendercoders (freenode) stick
to each other and we have an extremely helpful and active community of
python developers and c coders as well so I am pretty much covered on
blender side. By the way , it would be nice if you opened an irc
channel in freenode as well and create a more real time community. It
wont be big at first , but I can make it grow and bring some blender
heads ;)

Yesterday I have taken a deep dive in your documentation it seems the
one thing that would work best for me is the XMLRPC server, I have not
tried it yet, but I will tonight and keep you posted. From what I have
read so far, you wont have to do anything to make this work from the
field side and hopefully I wont have to do much from the blender
python side. Autocompletion should not be a problem as I have made
eclispe and pydev import all the blender python modules and
autocompletion worked fine.

Execnet remains also an option , I have to confess I have not used it
so far . I was aware of its existence for quite some time now but I
had no use for it . But now is the perfect time to do my tests with it
and keep you posted on that one as well.

Regarding what I want to do, I start with the obvious goal of making
blender cpython talk to field jython , if the two can exchange data
easily and call function mission accomplished and there is nothing
more that I need. From there on I will continue to develop Morpheas
and use it as a Field front end. By front end I do not mean
replacement of your field gui, I love your gui and there is no need
for me to reinvent the wheel here, so the user will still use Field
and its gui and use my gui just for more blenderish interactions so he
wont have to jump between field window and blender window all the
time. I still think however that all visual coding should happen in
the field window as I dont even dream of porting your functionality to
blender. The big diffirence between Morpheas and Blender standard GUI
will be that besides the fact that user will be able to design his own
Blender guis , it will talk directly to field thus making it easy for
all blender python developer and blender users to use field from
inside blender. This is my only goal so far and if I can accomplish
it , it will be a dream come true.

I will like to also offer myself as a windows maintainer, I know java
and jython , never used ant, and not very experienced with windows
building , but i build regularly on macos and ubuntu , blender and
other software and I am , I like to think, very experienced with
computers and willing to learn. So I am probably not the best choice
for windows maintainer, but I have macs at home and windows at work
and I can try my best if you want a helping hand.

Nice to meet you

On 21 Μάϊος, 06:42, Marc Downie <m...@openendedgroup.com> wrote:
> Kilon,
>
> Thank you for the fascinating set of links.
>

> I think the real issue is to figure out *where* the computation goes. If we

[kilon]

hi I tried the xmlrpc option it does not work either for blender or
field . Blender complained of course because your example was python
2 , so I converted , I hope , to correct python 3 code and it seems to
run with no issues

here is the code

import threading
from xmlrpc.server import *



class mainThreadHandler(SimpleXMLRPCRequestHandler):
def _dispatch(self, method, params):
x = getattr(cmds,method)
kw = {}
p = []
for n in params:
if (isinstance(n, type({}))):
kw = n
else:
p.append(n)
p = tuple(p)
y = executeInMainThreadWithResult(x, *p, **kw)

return y
def log_message(self, format, *args):
print(format % args)

def dispatchForever(x):
server = SimpleXMLRPCServer(("localhost", 8999),
mainThreadHandler, allow_none=1)
print("registering")
for n in dir(cmds):
xx= getattr(cmds, n)
if (callable(xx)):
server.register_function(getattr(cmds, n))
print("registering complete, serving")
server.serve_forever()

Unfortunately I could not test if it really works because issuing the
code

import DictXMLRPC
cmds = _self.cmds_ = _self.cmds_ or
DictXMLRPC.dictXMLRPCClient("http://localhost:8999")

inside a graphic element and hitting the cmd+ / it complains with this
error

print cmds = _self.cmds_ = _self.cmds_ or
DictXMLRPC.dictXMLRPCClient("http://localhost:8999")
python exception <("mismatched input '=' expecting NEWLINE",
('server[__5843149:1376fe0c981:-7ff5', 1, 11, 'print cmds =
_self.cmds_ = _self.cmds_ or DictXMLRPC.dictXMLRPCClient("http://
localhost:8999")\n'))>

I can't say I understand the field syntax up there, looks like very
weird python to me , or at least I am not used to this kind of coding.
Am I doing something wrong here ?

I could try to use execnet now but I have no idea where I should put
it inside field , any ideas ?

[Marc Downie]

[snip]

 

import DictXMLRPC
cmds = _self.cmds_ = _self.cmds_ or
DictXMLRPC.dictXMLRPCClient("http://localhost:8999")

inside a graphic element and hitting the cmd+ / it complains with this
error

You definitely want command-return, not command / . I'll assemble a working Blender install tonight to see if I can follow along. This definitely ought to work.

Marc.

[kilon]

Well blender is not my problem currently my code snippet I used for
client in Blender seems to work with no errors. Field is the only one
that gives me errors.

I tried cmd - return , now it give me this error

cmds = _self.cmds_ = _self.cmds_ or
DictXMLRPC.dictXMLRPCClient("http://localhost:8999")

python exception <name 'DictXMLRPC' is not defined>
on line 1 server / <module>
###errorLink###server[__5843149:1376fe0c981:-7ff5###1###

it seems the import fails for some reason.

but I have confirmed that the DictXMLRPC.py is in field->contents-
>macos->Resources->Java->python. And in the same location is the
class file as well.

> Blender install tonight to see if I can follow along. This definitely *ought
> * to work.
>
> Marc.

[Marc Downie]

Kilon,

Can you verify that you are using the latest build of Field (available here: http://openendedgroup.com/uploads/field_14.app.zip ). DictXMLRPC is instantiating just fine here for me.

Marc.

[kilon]

no I am using field_12 , this is what the download link gave me. I
downloaded field_14 , removed the old and installed this new one. The
new one solved my previous error

And I also simplified my blender script now looks like this



from xmlrpc.server import *


class main_handler:
pass

server = SimpleXMLRPCServer(("localhost", 8999), main_handler)


print("registering")

def call_bpy(cmd):
return execl(cmd)

server.register_function(call_bpy)

print("registering complete, serving")

and for the client script in Field side i use this :

import DictXMLRPC

cmds = _self.cmds_ = _self.cmds_ or
DictXMLRPC.dictXMLRPCClient("http://localhost:8999")

cmds.bpy_call("import bpy")


blender works ok , no error and complains, but when I run field, again
no error but its changes to the rainbow spin wheel and its like its
caught inside an infinite loop so I have to force quit field. Field
gives me no errors.

On the subject of blender python and threading, python inside blender
is not in a separate thread and python blocks any blender execution
because python is given priority , the scripts themselves of course
can run in a non blocking manner all the time but each line of code
when executed blocks the execution of blender. So if my blender
scripts waits forever for the user something to type blender will
freeze until user types something. The irony here is that Field
freezes and not blender so I suspect something is wrong with field.
import bpy, imports the main blender python api module which is
responsible for most functionality inside blender and it is usually
the fist line of code of every blender script out there. The question
is , does importing modules like this with an eval creates any weird
conflicts for field. That question only you can answer

On May 22, 5:58 am, Marc Downie <m...@openendedgroup.com> wrote:
> Kilon,
>
> Can you verify that you are using the latest build of Field (available

> here:http://openendedgroup.com/uploads/field_14.app.zip). DictXMLRPC is

[kilon]

please ignore the execl(cmd) , the script I am using is using the
correct exec(cmd) and yes now I use exec instead of eval, as eval did
not work with import in blender. The problem remains of the infinite
loop spin rainbow, i have tested the exec("import boy") in blender it
works with no issues so most likely this is a field issue. Excuse my
ignorance on XMLRPC I hope I am not making any obvious mistake here ,
its the first time I use this.

I forgot to add that execnet fails with blender, don't know why,
probably does not like the fact that the interpreter is embedded or
maybe blender blocks it in some way. I hope the same does not happen
here with my problem with field, I really want to bring field to
blender developers , I think visual and live coding with 3d objects
will be a very welcome addition for blender. Thanks for all your help
so far.

[Marc Downie]

from xmlrpc.server import *


class main_handler:
   pass

server = SimpleXMLRPCServer(("localhost", 8999), main_handler)


print("registering")

def call_bpy(cmd):
   return execl(cmd)

server.register_function(call_bpy)

print("registering complete, serving")

Are you actually calling (from a separate thread) server.serve_forever() ? If not then Field is connecting and waiting forever for a response from Blender that never comes. Hence the beachball.

I think we could take a step back, and try to get a working RPC of any kind into Blender, then connect Field to it.

Marc.

[kilon]

ok now that is what happens when you do loads of tests and paste the
wrong test :D

sorry about that

please ignore that line too

yes i tried without that and that is what gives me the infinite loop
in field

I use no serve_forever , yet field freezes

I try to follow the example here

http://docs.python.org/release/3.1.5/library/xmlrpc.server.html

If its my fault I am more than welcome any guide . I don't see blender
complain with the code in anyway or have an abnormal behavior so far.

[Marc Downie]

I use no serve_forever , yet field freezes

No. You need to actually run the server with serve_forever inside Blender, otherwise connections will start, but will never get served. This way any application, including Field, will appear to hang when talking to your server.

The example code in openendedgroup.com/field/MayaIntegration.html has a serve_forever call (inside a new thread, so it doesn't hang Maya). If you like you can strip all this away and try to get two interactive CPython environments talking to each other and then build it back up again.

Marc.

[kilon]

oh thats a problem, I removed serve_forever because it made blender
freeze obviously because it was waiting for a message from field, so
that means I will have to create a thread for this so blender can
continue to work . OK thanks for clarifying this by the way I also
tried that as part of my tests and field gives me an error (another
reason why I removed serve_forever)

blender script :
import bpy

from xmlrpc.server import *


class main_handler:
pass

server = SimpleXMLRPCServer(("localhost", 8999),

requestHandler=main_handler)

print("registering")

def call_bpy(cmd):

exec(cmd)

server.register_function(call_bpy)
server.register_instance(bpy)

print("registering complete, serving")

server.serve_forever()

field script:

import DictXMLRPC

cmds = _self.cmds_ = _self.cmds_ or
DictXMLRPC.dictXMLRPCClient("http://localhost:8999")

cmds.call_bpy("import bpy")

field error:


python exception <error(54, 'Software caused connection abort')>
on line 4 client / <module>
from line 10 /Applications/field.app/Contents/MacOS/../python/
DictXMLRPC.py / __call__
from line 1147 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / __call__
from line 1433 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / _ServerProxy__request
from line 1183 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / request
from line 1299 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / send_content
from line 711 /Applications/field.app/Contents/MacOS/../lib/python/
httplib.py / send
from line 1 <string> / sendall
from line 1179 /Applications/field.app/Contents/MacOS/../lib/python/
socket.py / send

I have not coded threads before but I guess thats the best time to
start. I ignored the serve forever because it was not used in the
python 3 example so I thought it was more related to the thread you
create in maya . But no problem I am willing to learn .

[kilon]

Ok I did what you advised , I sat down , read the documentation
carefully , tried to understand as best I could the mechanics behind
this and I tried with cpython and blender before moving to field. I
end up using this code

http://pastebin.com/TzYWbpPT

This code works with blender as a server and cpython 3.2 intepreter as
client , and blender as a server and another blender as client. Works
like a charm. Zero problems.

Field gives me an error, it complains that one more argument is passed
to the function I call. And I have tried both of my server functions ,
call_bpy and stop_server , both complaint the same. Why field passes
my single argument functions that I call as two arguments, and what is
the extra argument ?

for example this code

import DictXMLRPC

#cmds = _self.cmds_ = _self.cmds_ or

DictXMLRPC.dictXMLRPCClient("http://localhost:8999")

#cmds = DictXMLRPC.dictXMLRPCClient("http://localhost:8999")
cmds=DictXMLRPC.dictXMLRPCClient("http://localhost:8999")
cmds.stop_server()
#cmds.call_bpy(" bpy.data.objects[0].name= \" Jim \" ")

excuse the comments, i just try to test any alternative and all those
comments are alternative code I tested. The result is always the
same , this error more or less

python exception <<Fault 1: "<class 'TypeError'>:stop_server() takes
no arguments (1 given)">>
on line 6 client / <module>

from line 10 /Applications/field.app/Contents/MacOS/../python/
DictXMLRPC.py / __call__
from line 1147 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / __call__
from line 1433 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / _ServerProxy__request

from line 1201 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / request
from line 1340 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / _parse_response
from line 787 /Applications/field.app/Contents/MacOS/../lib/python/
xmlrpclib.py / close

I am sorry If I am getting a bit annoying but I am enjoying the
experience of XMLRPC and even if field does not manage to work I can
use it for all kinda of communication with any kind of software. But I
would love to get field working. As always thank you for all your help
and patience.