Eva Impasse

[Mark Wigzell]

Hey guys, well I have spent a couple of weeks re-building opencog/docker to Ubuntu 20.04 and ROS "noetic" release, concentrating on re-vamping the "eva-ros" container initially. 

I ran into various issues with old python code, easily upgraded. Also some issues with conflicting configuration parameters, I think those are all solved or solvable.

The issue that has become insurmountable is the change in the Blender implementation from 2.79b to 2.80rc3. The key change was they dropped support for the Internal Renderer in favor of Cycles and later Eevee. 

I have tried to convert the textures using the Materials add-on they supply, but it failed to convert the eyes, and was less than perfect on the skin tones as well. If this one issue could be handled by a skilled Blender user, we might be able to "save & load" up to the current Blender release. 

The current Blender release has the distinct advantage that it runs in an Ubuntu 20.04 docker container "out of the box".  The older blenders, including 2.79b do not.  There has been a lot of churn with X11 acceleration, 32bit to 64bit, and the new Wayland support, and the default Ubuntu 64bit setup confounds the older blender versions. 

With a properly running Blender, the Eva project could be re-vamped. Is it worth it? All I can convey is my enthusiasm for the project when I trailed it back from the "Sophia the robot" interviews. Whilst I understand that basic research into AI learning does not need the heavy load of supporting robots or human like interaction, I strongly believe that having such a teaching example fully functional would do wonders for the entire field: many people are inspired by the hype around AGI and androids, 

Providing fledgelings with a setup such as was developed for Eva would be a powerful kick-off for anyone wanting to dig into the field, especially young people, who would definitely get a kick out of running their prototype AGI hookups in the Eva rig. There IS something inspirational when she comes to life, breathes, smiles etc. Granted the rig is NOT itself a demonstration of AGI, but the hookups back to the AtomSpace should allow people to make all sorts of experiments that point in that direction.

I think the Eva project is a great landing place for enthused talent. It just needs to get over this current hump, which has defeated me at the moment.

Perhaps someone in this group has access to some Blender people we can engage to remedy this situation?

Regards,

--Mark

[Reslav Hollos]

Hi Mark,

Is your current progress available somewhere? I can look into Blender materials.

Best,

Reslav

--
You received this message because you are subscribed to the Google Groups "opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opencog+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CA%2Ba9A7CcAcywy0MqqzjG7uf-zi%2B6OsPvt%2BWiWR2YbH3poarJxg%40mail.gmail.com.

[Mark Wigzell]

Hi Reslav,

Hey great! Thanks for your help:

In order to see issue:

git clone https://github.com/opencog/blender_api.git

cd blender_api

blender -y Sophia.blend -P autostart.py

You can download different blender versions from https://download.blender.org/release/

If you use 2.79b it seems to work best. If you use the next version, 2.80rc3 you'll see a lot of load errors. 

My actual work on Eva (Sophia.blend), is around getting the existing solution to build and run. You can see that here: https://github.com/opencog/docker/tree/master/noetic

Try building ros-base, ros-blender and ros-eva containers. Finally run the ros-eva container, it comes up using "tmux". There are run errors in some of the nodes, the webcam image comes up, and the current blender (3.?) comes up with Sophia.blend. Of course it looks terrible because the materials haven't been converted.

I attached an image of the 2.79b running. (If you click the "start animation" button on the right, she goes into action.)

Apparently Eva should not have been renamed Sophia.blend, that was a mistake 7 years ago or whenever. I'll fix that.

So running Eva in blender with the "blender_api" in the current working directory is essential to having the ROS node work properly. And having that work properly is essential to debugging the "perception" repo, which won't work without blender ROS 

• There is an issue of python namespace when starting up Eva in blender.  
• There is also one "cyclic dependency": Dependency cycle detected: deform depends on control through Child Of. control depends on deform through Locked Track.

[Reslav Hollos]

Hi Mark,

I've added "Principled BSDF" node materials and approximated the values to match the look, except the eye's displacement (if you zoom in really close) which I've not figured out yet.

Shadows/lighting are not tested, I can continue on this in a few days, but hopefully this would be sufficient for you to start getting some visuals.

The blend file is here https://github.com/Radivarig/blender_api/blob/update-materials/Eva_nodes.blend

Best,

Reslav

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/ccc3368d-10bf-4313-9f3f-d724c5632b44n%40googlegroups.com.

--

Kind regards,

Reslav Hollos

[Mark Wigzell]

Hi Reslav, thanks a million, I'll take a look.

--Mark

You received this message because you are subscribed to a topic in the Google Groups "opencog" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/opencog/2fq2zuBEJTs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to opencog+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CANHDs8ncAzQgtkJE%3D633XKUNstPAv41A7r%2B-AXVuQ13OEVeNZA%40mail.gmail.com.

[Mark Wigzell]

Hi Reslav, unfortunately I am unable to read the file format of that blend file: 

blender -y Eva_nodes.blend -P autostart.py
Read prefs: /home/mark/.config/blender/3.0/config/userpref.blend
Error: File format is not supported in file '/home/mark/Dev/blender_api/Eva_nodes.blend'

I have tried with 2.79b, 2.80rc3 and my latest blender on ARCH: 3.0.1 with same result. I'm afraid I have too little experience to know why this is happening or what to try next.

--Mark

On Thu, Mar 3, 2022 at 6:26 AM Reslav Hollos <reslav...@gmail.com> wrote:

You received this message because you are subscribed to a topic in the Google Groups "opencog" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/opencog/2fq2zuBEJTs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to opencog+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CANHDs8ncAzQgtkJE%3D633XKUNstPAv41A7r%2B-AXVuQ13OEVeNZA%40mail.gmail.com.

[Linas Vepstas]

Hi Reslav,

On Thu, Mar 3, 2022 at 8:26 AM Reslav Hollos <reslav...@gmail.com> wrote:

Hi Mark,

I've added "Principled BSDF" node materials and approximated the values to match the look,

It should not require any "approximation", the skin texture is there, as well as several different eyeball textures.

There's the scary-looking https://github.com/opencog/blender_api/blob/master/textures/eva.COL.png  and then a "subdermal" evaSubdermal.COL.png and then some skin gloss texture eva.GLOSS.png and perhaps a skin-transparency (so the subdermal shows through??? guessing wildly) in eva.DISP.png .. there are textures for the inside of her mouth, the eye.png but also eyeCornea.png  and finally, long long eye lashes.png

I spent multiple sessions earlier this week, trying to figure out how to wire it all up and was completely stumped.

I can see that original blender file access these textures.  I can also see many dozens of animations, but again, was stumped as to how to make them run.   I think its important to port these, because they all looked pretty good (to my untrained eye. Certain cough cough sculptors in the house were less than happy.)  The only "major" flaw was that if you used the various viseme animations (visual phonemes) while she's talking, and then made her smile, the mouth would go weird.

--linas

 

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CANHDs8ncAzQgtkJE%3D633XKUNstPAv41A7r%2B-AXVuQ13OEVeNZA%40mail.gmail.com.

--

Patrick: Are they laughing at us?

Sponge Bob: No, Patrick, they are laughing next to us.

 

[Linas Vepstas]

Also there are snapshots of what she's supposed to look like, when things are working correctly, here: here's one for example; https://github.com/opencog/blender_api/blob/master/docs/eva2b.png I  think the current textures are for the one on the right side.

I feel as if I'm bragging and showing off, but exactly 0% of this is my work; it's more like a "see, this is what it looked like 7 years ago" and to me, it looks pretty darned good.

--linas

[Mark Wigzell]

If I may re-iterate: you can see her looking and working very well if you run current blender_api repo with the 279b blender. 

When I tested Reslav's changes, I only took the blend file to try. I was under the assumption everything that needed conversion is in that file,

for some reason it won't load though. 

Regards,

--Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CAHrUA36Ln8NUCOtjSvNJtDfH2L%3DJgnTs6CpdL3o9%3D42yNn%3DJXg%40mail.gmail.com.

[Linas Vepstas]

On Sat, Mar 5, 2022 at 9:34 PM Mark Wigzell <markw...@gmail.com> wrote:

If I may re-iterate: you can see her looking and working very well if you run current blender_api repo with the 279b blender. 

When I look at the blender file (using `Blender 2.82 (sub 7)`) I can see that there are file-paths in there, pointing to the right place in the filesystem. (except for one bad path to /user/Mark/something)  (single-slash is an absolute path, double-slash is a relative path) If I altered the file-paths, I would get a spew of errors. So I think the textures load from the external files.

--linas 

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CA%2Ba9A7BTegoxpa%2BG_AaAAwyhsAFom4FQHp7kE00_0Vd5OOHLtw%40mail.gmail.com.

[Mark Wigzell]

ah, I had an error on my side, sorry for confusion, correction: I CAN read the new file from Reslav, but it gets a lot of errors. Linas that "/userMark/" path, its not me I don't think. (I have no commits in "blender_api" yet.. 

Reslav, I'm able to load your changed file, in situ with the rest of the folder, using 2.80rc3. The shading and eyes are there, which IS a considerable improvement. I wonder though, did you convert all those other textures? Also, it complains that the file was written by a later blender, so we are not aligned on the blender version. Please see log file attached. First let me say, that the Python in blender_api is not upgraded to Python 3 or whatever, so it won't work properly for sure.. 

With 2.80rc3 I see all those warnings and errors in the log.txt file attached. It seems there is still some texture fixing to be done. I wonder, did you utilize the "add-on materials" conversion helper to convert them?  This link shows what to do: https://developer.blender.org/T58420 (there is a built in add-on for materials conversion to Cycles.It however needs to be done before 2.80 is loaded,  since 2.80 doesn't support the Internal Render as I understand. (I hope I'm making sense, I don't understand blender very well).

If I am not being clear, lets zoom or something and straighten things out. I'm on PST time.

[Mark Wigzell]

One more thing Linas, I discovered that there is a "sophia" branch in the blender_api repo. It has similar issues when loading, also no eyes.so it is suffering from the texture issue.

[Reslav Hollos]

Approximation refers to the values of the new material nodes, because I've not used any convert tools, just created new material nodes and mapped the textures.

Blender 2.79 materials are quite confusing to me and the workflow differs from that of 2.80 so I skipped some parts related to the eye which can only be seen when fully zoomed in.

For the shadows/lighting I was thinking mostly of the teeth being completely white when previewing in material mode instead of the render mode.

In short I've set:

- eyes to have a diffuse texture and specular reflection

- eyelashes to have a transparent diffuse texture

- face to have diffuse, specular, displacement and subsurface textures with subsurface color

(image attached)

I've not touched the script errors on start, but yeah Blender 2.80+ uses python 3.7.

What texture fixing do you mean? I've just pointed to them from materials nodes so no texture changes were done.

Is the file format issue resolved, what was it?

Because I'm in the middle of installing Arch from scratch to reproduce it.. If needed I would open a new Blender ticket for the cross system compatibility.

I've not run any animations but there are some blend shapes, if you want I can also look into that.

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/fdf57d3c-07b7-40e8-8a77-02cf738c4794n%40googlegroups.com.

[Mark Wigzell]

Hi Reslav, first let me thank you again for helping out! 

I think your changes are fine to get us going BUT we need to do them in the right order, on the right version etc. (I think we got our wires crossed slightly because the repo commit I gave you was the untouched AS IS Eva Impasse commit. No fixes whatsoever are done.

The first thing to do is get synchronized with each other about how to approach this fix.

Let me outline the steps as I see it, keep in mind, we don't want any errors resulting from using wrong blender version. Also, we MUST start with blender 2.79b, do all the fixes and save it out. ONLY then open with 2.80. I'm attaching a video which I just created, it can be played with VLC. 

• Install blender 2.79b : For my 64bit ARCH I used: https://download.blender.org/release/Blender2.79/blender-2.79b-linux-glibc219-x86_64.tar.bz2 
• clone the blender_api from: https://github.com/opencog/blender_api.git
• Navigate to the installed blender and run: "cd <path to blender>\2.79\python\bin; ./python3.5m -m ensurepip" (this installs the setuptools needed to run the python commands)
• Navigate to the "blender_api" folder and run the correct blender "<path to blender>/blender -y Sophia.blend -P autostart.py". (you should see something like the video I attached)
• Close out and run again without the autostart. (I don't know how to stop the animation once started)
• Click on the drop down on the top menu bar which is currently displaying "Blender Render" and choose  "Cycles Render". 
• See this thread: https://blender.chat/channel/support/thread/7hx7CinRjKa9NzbrT (Its me chatting with Nicap. See the views he sent me. 
• Click bottom left view menu and choose User Preferences/Add Ons, and enable the "Materials: Materials Utils Specials 
• Follow along in my conversation with Nicap to convert the nodes having textures. Note the eyes don't convert. 
• Now please add in your eye materials. I have no idea what you do there! 
• If all is well then save your blend file. 
• Install blender 2.80 and try loading the blend file. (don't forget the "ensurepip" command above to install the setuptools
• Run the 2.80, I have my fingers crossed: If all went well it should load up your converted file and play it. (only if the animation pane on the right side comes up, and the animation can be started, is it a success)

Good luck, please don't hesitate to reach out to me if I am not being clear somewhere. It will be so great to be able to run this with the newer blender apps!

[Linas Vepstas]

On Sat, Mar 5, 2022 at 11:20 PM Mark Wigzell <markw...@gmail.com> wrote:

 Linas that "/userMark/" path, its not me I don't think.

It's a different Mark, its Mark Thielen (mdthielen in github).  The hard-coded path was to some animation startup script that's checked into github, but the path to it was wrong.

--linas

[Linas Vepstas]

On Sun, Mar 6, 2022 at 3:57 AM Reslav Hollos <reslav...@gmail.com> wrote:

Awesome!

-- linas

[Mark Wigzell]

Yes, that DOES look good!

I followed my own prescription to you Reslav, and found out that the "BONE" failures are still occurring (in 2.80). So I forked the "opencog/blender_api" to https://github.com/mwigzell/blender_api, and uploaded a blend file (Eva_2.79b.blend)  that maybe you can update with your textures?  This file is set to use the "Cycles Renderer".

The texture updates must be done with the 2.79b blender I think, until I can get one problem fixed:

- The cyclic dependency between "deform" and "control"

If that is fixed there are no errors at all, then we can try again to load into a 2.80 blender. I'm thinking, with the cyclic dependency eliminated, the bones may load. If not, then maybe they too need converting, and I will need to get even more involved with blender!

Pls. let me know what you are thinking, Reslav and Linas. I have very little experience with Blender and I'm learning as I go.I think we need to proceed (to next blender version) only when there are no load errors, and the gesture animations still work. I like the work you did so far Reslav, but unfortunately the animations don't work in 2.80: we need to take a step back.

--Mark

[Reslav Hollos]

From your file Mark, the teeth are sticking out of one of the cheeks, is it due to errors with bones? Do you also have that?

Regardless, I've added the same material nodes for 2.79b, done slightly different (node changes between versions), approximated to visually match the looks.

File here https://github.com/Radivarig/blender_api/blob/update-materials/Eva_2.79b_PrincipledBSDF.blend

I've also figured out the ultra white teeth and updated the normal map node on both versions.

Blender consistently crashed on adding image texture node, even just when file picker was closed, but since you used some converter tool it had already added image texture nodes so I was able to copy them around!

Higher versions have more bugfixes and general optimizations including render speed and quality. Maybe 2.80 can be used as a next large update and see if it can be forward compatible with higher versions.

For the cyclic deps I can take a look as well, then we could see what is the next issue.

--

You received this message because you are subscribed to the Google Groups "opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opencog+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/3592555c-33ca-4c09-bf3b-09fe2db386ddn%40googlegroups.com.

[Mark Wigzell]

Hi Relav,

you write:

From your file Mark, the teeth are sticking out of one of the cheeks, is it due to errors with bones? Do you also have that?

So, if you are referring to this:  https://github.com/mwigzell/blender_api/blob/master/Eva_2.79b.blend then it works as described earlier: she comes up with no errors except the one cyclic dependency issue, she has skin textures, but the lighting and her eyes are bad. I can run the gestures. (all using blender 2.79b app). 

The teeth out of the cheek: yes, I've seen that, it seems when the python fails to load properly, it sometimes does manage to start an animation which then fails.  

So it doesn't load for you? Are you using the blender directly from the blender website? (https://download.blender.org/release/Blender2.79/blender-2.79b-linux-glibc219-x86_64.tar.bz2) and you have run the "python -m ensurepip" as mentioned previously? (I don't think that installing blender with the distro's package manager will work, you must install it from the tar file to some local place, or maybe to "/opt/blender/..." .

https://github.com/Radivarig/blender_api/blob/update-materials/Eva_2.79b_PrincipledBSDF.blend

So that looks a lot better than mine, you fixed the eyes and many textures, great! And she is "breathing" when I load her. But I do 

have an issue: where did the control panel go? I am missing the gesture controls entirely.See the snapshot attached.

Maybe there is something I don't understand about blender, is there a way to make the gesture panel come up?

Otherwise, its a great job. We still have the usual errors while loading into 2.80 blender, I don't know how to fix the dependency cycles, I just know, it should be a clean load in 2.79 before we load into 2.80. 

Regards

--Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/CANHDs8kDy4dRQXQdONxG8a7uoAEfa_OTCmFAU0gtDWvqgg2%3D8g%40mail.gmail.com.