Blockly with Unity

[Jacq Train]

Hi everyone,

I'm working on building a basic Unity program that I'm hoping to integrate Blockly with as a native plugin. I've read some documentation on importing .framework files to Plugins of a Unity project but I'm having some compilation trouble with the .js files the Blockly build contains. Has anyone tried anything like this before and have any tips on moving forward?

Thanks!

Jacq

[Warunyoo Moomoo]

Please someone answer. I need to know too
How can i command my charactor with blockly?.

[Cory Diers]

Heya,

I have done a bit of work integrating Blockly into Unity. It was a proof of concept, so I didn't do a lot of work with it, but it's definitely possible! I definitely had a bit of trouble finding the proper place to include the js files, as well. I used the web version of Blockly, so if you're using Android or iOS, you'll need to do some more legwork to get this working, but I found that Unity has a real problem with javascript files. So I put them in a "WebPlayerTemplates" folder. When you use that specific name, Unity has less of a problem with loading them into the build. You'll still need to do some work to create native WebViews to load the workspaces into, but I did eventually get something working. Best of luck!

On Fri, Apr 28, 2017 at 8:56 AM, Warunyoo Moomoo <mook...@gmail.com> wrote:

Please someone answer. I need to know too
How can i command my charactor with blockly?.

--

You received this message because you are subscribed to the Google Groups "Blockly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blockly+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Cory Diers | Software Engineer | cory...@google.com | 

[Jacq Train]

Hi Corey,

Thanks so much for your help here, it gave us a good solid start on getting our application running. Apologies for the delay in this, we only work on this project a few hours a week and there have been a fair few bumps in the road! It's been very much an exercise in trial and error, learning as we go. We've finally got it working though, so I wanted to let you know that we were successful :)

Unfortunately, the WebPlayerTemplates and native views approach didn't work for us but we used it as grounding to decide how to proceed. Instead we have built an iOS app (and will build an Android one in the future) so that we can support the native Blockly framework, and built in the Unity app as a webview. From here, the communication between the two was actually pretty simple to set up - although not particularly efficient right now, we still need to do a lot of work! I actually wrote an article with one of the driving team members of the project in case anyone found this interesting: https://medium.com/ocadotechnology/unity-and-blockly-a-match-almost-made-in-heaven-ff2eafcdd220. Hopefully it can offer a bit of help to others attempting similar.

Thank you again for your suggestions!

Jacq

On Monday, 1 May 2017 18:18:51 UTC+1, Cory Diers wrote:

Heya,

I have done a bit of work integrating Blockly into Unity. It was a proof of concept, so I didn't do a lot of work with it, but it's definitely possible! I definitely had a bit of trouble finding the proper place to include the js files, as well. I used the web version of Blockly, so if you're using Android or iOS, you'll need to do some more legwork to get this working, but I found that Unity has a real problem with javascript files. So I put them in a "WebPlayerTemplates" folder. When you use that specific name, Unity has less of a problem with loading them into the build. You'll still need to do some work to create native WebViews to load the workspaces into, but I did eventually get something working. Best of luck!

On Fri, Apr 28, 2017 at 8:56 AM, Warunyoo Moomoo <mook...@gmail.com> wrote:

Please someone answer. I need to know too
How can i command my charactor with blockly?.

--

You received this message because you are subscribed to the Google Groups "Blockly" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blockly+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nesh]

Hey everyone,

I just noticed this thread and I wanted to add in my experience with Blockly and Unity.

After much research and testing I found that running Blockly within a webview in the Unity scene seems to be the most straightforward way to tackle the problem.

My goal was to find a solution which worked on desktop and mobile with acceptable performance (smooth rendering of the blocks and unnoticeable delay between the calling the function and its execution).

Visualization 

I found the following projects to work pretty well:

- Unity WebView: https://github.com/gree/unity-webview

- Embedded Browser: https://www.assetstore.unity3d.com/en/#!/content/55459

Unity WebView

This open source project was the first one I tested and it proved the feasibility of this method. It is still small and the support is also quite limited but it does the job.

Pros:

- Supports OSX, Android and iOS (and indirectly Web builds)

- Small codebase

- Open Source

- Currently maintained

Cons:

- No support for Windows or Linux

- Not great smoothness (A tweak can make improve the rendering speed)

- Can only render as a 2D canvas

- Small team working on it

The limited platform support is due to the use of native libraries instead of a cross-platform solution.

Embedded Browser

I found this asset after several weeks as I somehow missed it. Very surprised with what it could achieve. It makes use of CEF so it supports pretty much anything Chrome/Chromium can.

Pros:

- Supports Windows and OSX

- Supports rendering on materials as a texture

- Can render some complex HTML5 pages (even 3D)

- Very smooth

- Company working on it (i.e. more people)

Cons:

- No support for Linux or mobile (might come in the future)

- Quite expensive

- Closed source

Here are some screenshots:

Babylon.js Playground (Volumetric Light Scattering)

Blocky-games on a 2D canvas

Blockly-games as material on 3D plane

Pretty neat, uh?

Communication

 

Next step is to be able to communicate between the JS on those pages and Unity and viceversa. That is available on both the projects I mentioned earlier via Messages; the syntax differs slightly on the two projects.

A call generally looks like this `SendMessage('FunctionOnTheOtherSide', 'firstArg', 2, 'lastArg');` and works just like a message (basically just sending information without expecting anything back).

The main problem with this implementation is the lack of return values. This means that these calls can only be one-way and you cannot ask for anything in return (e.g. the position of the player). But actually that is not true, with some ugliness that can be achieved with something like:

JS:

`SendMessage('MovePlayerAndReturnValue', 0, 0, 1, 'CallbackMovePlayerAndReturnValue');`

Unity:

public void MovePlayerAndReturnValue(float x, float y, float z, string callbackFunction) {

      this.player.position = new Vector3(x, y, z);

      SendMessageToJS(callbackFunction, this.player.position.x, this.player.position.y, this.player.position.z);

}

It's not pretty and it requires a wait on the other side (possibly with a timeout in case of lost messages) but that works and does the job. (For more info on this JS -> Unity and a way to improve its efficiency, check this out: https://forum.unity3d.com/threads/super-fast-javascript-interaction-on-webgl.382734/)

Next Steps

The next steps of my research are to evaluate the performance on low-end devices (mainly low-end phones) and the reliability of Unity messages. I want to see if there is a way to avoid having to send loads of ACK messages back and forth in order to be sure that communication is always reliable, otherwise that would slightly suck :P

Hopefully this will be useful to someone out there :)

/Nesh