web tech

[Geoff Evans]

Matt Stine and I were talking tonight about the future of Editor and our tooling in general.  Conventional desktop widgeting is done so frequenty for game tools that it kind of seems old hat to keep grinding on that design.  We brainstormed a bit about how to mix in a bit of web UI (HTML/CSS/JS), but perhaps not go so far as to just dump all the tools into a web browser windows.  From what I have heard about others doing that, its a pretty mixed bag (having to put your renderer into a custom plugin that is hosted in some browser process).  Things get weird really fast.  This got me thinking about some alternatives.

I think it would be conceivable to use http://docs.wxwidgets.org/trunk/classwx_web_view.html to build a new hybrid app around the core engine and renderer.  Using a custom helium:// URI hooked into the new wxwidgets web view control, we could conceivably keep a really skinny MainFrame window for Editor, that is principally just a render view (there could be multiple of these windows as well).  The custom URI would allow for an in-process REST API, no sockets needed (though I suppose you could route it over a socket if you really wanted to).  The actual editor GUI elements would then be hosted in MiniFrame tool windows that each just have a wxWebView control that occupy them.  The would use the helium:// URI to load whatever "page" they wanted in the positioned window (there could just be a drop down or combo box on the top of each mini frame (or context menu) that would select which client page to put into that window.  Many tools already end up here just in a conventional widgeting tool kit (I am pretty sure Blender does this).  Since each window is ostensibly controlled by our C++ code and wxWidgets, we can cache their size and location on the desktop between runs.  This effectively makes Editor work like Photoshop on the Mac, where each render window is a document, and the tool windows are free floating on whatever monitor the user wants.  There would be not master MainFrame that is a parent of every window.

We also talked about the concept of a WebGL renderer and how that might work.  Right now asset resources call into native OpenGL or D3D routines to allocate GPU resources for incoming asset data from disk, but a WebGL renderer could instead just allocate DOM elements that stand in for VBO/Texture buffers, which in turn just get delivered a la page load or ajax call.  Basically the Renderer interfaces just manages page elements instead of GPU memory allocations.  Its neat to think about.  The only bits we would probably need to upload back via the client page (into the engine proper) would be the input data, which would could handle the same way we would handle input of any other window.  The game update would still completely happen in C++ via Components.  At that point you could just host a game on real or virtual server hardware and have people play with a locally rendered but remotely updating game engine.  It would need to be low latency for some styles of games, but not all games requires lots of bandwidth or a quick roundtrip.  Especially games that people could run in their mobile browser without any app install.  Also, running in a mobile browser has low buy in for linking from social networks, and it doesn't have any restriction about paying the platform provider for in app transactions.

This all came up as ideas to potentially zag where other engine zig... try and mix things up to target platforms or styles of games that aren't as easy for other platforms that require complete integration to the OS (a native app).

[Philip Degarmo]

I don't mean to be a wet blanket.. But what does this give us other than more languages, more technology, more complexity, and more abstraction where we don't need abstraction? Being different is cool I guess but it comes at a cost of ease of use. I feel like as much as possible if there isn't a significantly BETTER way of doing something, being similar to other engines is actually a good thing and makes the whole toolset easier to learn and use. It also mitigates risk (that it might not work) - I'm down for risk taking but I wouldn't want to do it for no payoff. I think reflect and the component systems are good examples of risks (in scalability/efficiency) worth taking. I'd rather innovate in the space of game engine design, not UI design, and I think we're more likely to succeed if we focus our risks where we want to innovate.

I do think getting the main window "thin" and treating the other windows as tool palettes is a good way to go in the long run. It would make pulling tools into separate processes (like, a visual script editor or material editor) easier and more visually consistent. That said, I really disliked the separate windows up until the window docking/snapping functionalty got sophisticated. By then the products doing that were several versions in. I think we need to focus on getting what we have working before biting off that project though.

I'm definitely not against leveraging web technologies btw. I think it's a great fit for certain "headless" tools like a tray tool that builds/distributes assets to local tools/game instances or dedicated servers, or even just reflecting data from a local instance of the game. 

Philip

--
You received this message because you are subscribed to the Google Groups "Helium Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to heliumprojec...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Nicholas "Indy" Ray]

I've experimented a lot with this sort of things, So I'll get my thoughts out there.

I'll start by disagreeing with Phillip in that I think this could be a huge advantage. Having worked with web technologies, and in particular worked with people who are super proficient with them. It's staggering how quickly a good, powerful, and rich UI can be put together with modern web tools. As far as getting something working quickly, wxWidgets, and QT can't even get close to comparing. Then you additionally draw upon a huge set of existing skills, the amount of designers that can work with CSS and HTML is also vast, which significantly lowers the barrier for interface designers, a much underutilized asset in engine tools.

> (having to put your renderer into a custom plugin that is hosted in some browser process).

I agree that putting your render into a custom plugin is a huge pain. Additionally, most of the plugin APIs are on the shortlist to get dropped shortly, and the remaining ones are often quite sandboxed.

> I think it would be conceivable to use http://docs.wxwidgets.org/trunk/classwx_web_view.html to build a new hybrid app around the core engine and renderer. ....

I think this sort of thing can work well, it's actually almost exactly what the guys at wolfire games (http://www.wolfire.com/) are doing, and I've heard great things from their direction. The big problem becomes the embedding layer at that point. wx webview is big and a pain, and requires pulling in the entire wx dependencies. and a similar thing with QT. The wolfire guys use awesomium (http://www.awesomium.com/) But that's not open source, perhaps the chromium embedded project may be stable? (http://code.google.com/p/chromiumembedded/)

I may also suggest going ever farther off the reservation. I was working on a project a bit ago where we pulled the entire editor UI out of the game client, such that you'd load the renderer up, and then you could point a browser at a local URL, and all communication would run over a simple REST like api over Http (I used webby) This took a little getting used to at first, since it's not as well integrated. But I ended up really liking it since multi-window/monitor support is free, and works super well. And what ended up being a suprisingly nice side-effect was being able to load up an editor window from a remote computer. This made editing a running game (even while on device) super easy and effective.

> We also talked about the concept of a WebGL renderer and how that might work. 

This is actually a super clever idea for WebGL rendering, and I think something worth investigating. But do keep in mind, if you are already programming for the WebGL API, you are like 80% of the way to compiling through emscriptn, and running on the client directly. Perhaps an approach that allows both would be best, since it'd allow you to run high-latency games better on mobile devices, while still allowing low-latency WebGL games to work in the same engine.

Anyways, in all I think there are some good ideas here. They do overlap significantly with my particular experience however, so I'm glad to share any of my thoughts on this.

Indy

[Geoff Evans]

I don't mean to be a wet blanket.. But what does this give us other than more languages, more technology, more complexity, and more abstraction where we don't need abstraction?

I too am skeptical about this, and I am sympathetic to this viewpoint.  I am definitely not saying we should change everything now.  I just wrote this up because I saw this design in my head and wanted to share it with the team.  No doubt its more work since we have to invent the actual REST API that tool windows would use to communicate with the engine (and we have to write the web client), but I do think it provides interesting trade-offs.  To a certain extent wxWidgets promises something that is nearly impossible to deliver: a cross platform C++ wrapper around OS-specific widgets.  However, this is pretty much what a browser does, but in a completely data-driven way.  Plus, there are way more people invested in web tech being built well than there are people focusing on making wxWidgets work well.  To great extent all I am saying is that we should consider switching windowing toolkits, but to something that has a different programming model (REST instead of OO).  Being able to reflect our objects easy into other languages makes this very feasible in my mind.  I mean, there is a lot of cool stuff we could take off-the-shelf: http://www.jeasyui.com/documentation/propertygrid.php, http://designmodo.github.io/Flat-UI/, etc.

 

I feel like as much as possible if there isn't a significantly BETTER way of doing something, being similar to other engines is actually a good thing and makes the whole toolset easier to learn and use. It also mitigates risk (that it might not work) - I'm down for risk taking but I wouldn't want to do it for no payoff. I think reflect and the component systems are good examples of risks (in scalability/efficiency) worth taking. I'd rather innovate in the space of game engine design, not UI design, and I think we're more likely to succeed if we focus our risks where we want to innovate.

I guess I see the UI design problem as something that other game engine tools haven't solved well, and that challenges me to come up with a better solution.  I guess I just have a high bar for what good UI design is.

 

I do think getting the main window "thin" and treating the other windows as tool palettes is a good way to go in the long run. It would make pulling tools into separate processes (like, a visual script editor or material editor) easier and more visually consistent. That said, I really disliked the separate windows up until the window docking/snapping functionalty got sophisticated. By then the products doing that were several versions in. I think we need to focus on getting what we have working before biting off that project though.

Thinking further, we can still actually use wxAUI for the main view and REST web tech windows, and they remain just as dockable and useful as what we have now.  Really we could experiement with the design I am talking about with a single view inside the editor and see how it goes.  Its not really an all or nothing proposition.  We could, if we wanted, decide per-tool-window what is web based vs. native widget based.

[Geoff Evans]

It's staggering how quickly a good, powerful, and rich UI can be put together with modern web tools. As far as getting something working quickly, wxWidgets, and QT can't even get close to comparing. Then you additionally draw upon a huge set of existing skills, the amount of designers that can work with CSS and HTML is also vast, which significantly lowers the barrier for interface designers, a much underutilized asset in engine tools.

I guess I was hoping this to be the case, its relieving to hear that others have found this to be true in practice.

 

wx webview is big and a pain, and requires pulling in the entire wx dependencies

We already build from-scratch and use wxWidgets for the editor.  This is zero-cost for us right now.

 

perhaps the chromium embedded project may be stable? (http://code.google.com/p/chromiumembedded/)

 

https://github.com/steve-lamerton/wxWebViewChromium might be interesting, though we can for sure use wxWebView for prototyping.  I do like the idea of using one web backend across all platforms instead of wxWebView's IE on windows and webkit on OSX and Linux.  Plus I would expect that you get v8 everywhere with the Chromium embedded approach.

 

I may also suggest going ever farther off the reservation. I was working on a project a bit ago where we pulled the entire editor UI out of the game client, such that you'd load the renderer up, and then you could point a browser at a local URL, and all communication would run over a simple REST like api over Http (I used webby) This took a little getting used to at first, since it's not as well integrated. But I ended up really liking it since multi-window/monitor support is free, and works super well. And what ended up being a suprisingly nice side-effect was being able to load up an editor window from a remote computer. This made editing a running game (even while on device) super easy and effective

This crossed my mind as well, but seems more risky, but it could be a cool next-step if what I am proposing works out.  Great to hear that folks have made it work well in other engines.

> We also talked about the concept of a WebGL renderer and how that might work. 

This is actually a super clever idea for WebGL rendering, and I think something worth investigating. But do keep in mind, if you are already programming for the WebGL API, you are like 80% of the way to compiling through emscriptn, and running on the client directly. Perhaps an approach that allows both would be best, since it'd allow you to run high-latency games better on mobile devices, while still allowing low-latency WebGL games to work in the same engine.

Enscripten breaks my brain when trying to think how it would fit into all this!

Anyways, in all I think there are some good ideas here. They do overlap significantly with my particular experience however, so I'm glad to share any of my thoughts on this.

Cool, if we experiment with this count on my bugging you a bit about REST APIs and web client stuff.  I am also looking in aburke's direction.  :P

[Andy Burke]

So I have not read the entire thread, but I have tried to skim it, and I pretty much fully agree with Phillip: this is not a good idea to do with Helium. I just don't see a big upside for all the crap you would have to deal with.

On the flip side, if you were interested in writing a new WebGL-focused toolset, I think that would be something perfect to do in the browser. In-browser tools for in-browser games. I think in the next few years we'll see some really nice in-browser stuff happening and there is currently a vacuum for good toolsets.

I briefly advised Artillery Games (https://artillery.com/) a couple of years ago to go in this direction and it looks like they've made some progress. (I actually encouraged them to go 2D first, but they really wanted to go 3D and it's taken them over 2 years to get there.) I don't know what you'd want to do to differentiate from these guys (maybe try to be a bit more 'open source' and not necessarily have people run on your service?), but I really don't see a reason to try to cross over between 'native' games and their toolsets and the web. There's just so much headache that way.

Andy

--

[Nicholas "Indy" Ray]

As a bit of context since it was brought up, I was actually the primary engineer on Artillery Games' rendering engine, and much of their editor tech.

Much of that 2 year time was originally spent on 2D prototypes, and now more on game related functionality. When they started however Emscripten, ASM.js, and V8's related optimizations where nowhere near where they are now, however.

Indy

[Andy Burke]

Well, good to hear they took my advice. (I met with them when it was just 3 guys.)

I still would love it if there were a really nice 2D toolset for making in-browser games. I feel like if you nailed that niche, you'd be in good shape to go 3D, etc. Platformers to isometric RPGs, there's a lot of ground in there that if you wrote a nice little engine for people, they'd use the shit out of it.

Andy

[Nicholas "Indy" Ray]

I haven't had the chance to mess with it much, but the guys at turbulenz (https://turbulenz.com/) Where working on that to an effect. They had some impressive demos, and just open sourced their engine, which is in typescript, something that'll probably prove advantageous over pure JS, or other super dynamic web languages.

Indy