Live code editing with Haxe ?

[Malek]

Hey guys,


When you develop with Haxe many of you are likely to be using FlashDevelop as an IDE, because it's the one that integrates Haxe best as of today.

I was wondering, how do you do live debugging ? Many of you are experienced programmers and may have used at some point tools that allow to modify code while the game/app is running and have changes occur instantly, without resetting the game/app session. You could have used Eclipse to leverage Java's hot code swapping feature or to use GDB to achieve the same in C++.

Can you do something similar in Haxe ? Note, I am primarily interested in the Flash target.


Thanks a ton for your help, this would be a life savior ;)


(Hopefully I didn't post this thread several times. Sorry if I did! Got a google groups problem of thread disappearing right after being successfully published)

[Raoul Duke]

instead of relying on tools, you could think about it all
architecturally. more work, not as straight-forward, but can exist
now, and can work across setups. you basically store all your data in
a db that can be munged at runtime, and then you can load from that,
and support a reload command. (or, you can just query the db for
values at the start of every frame :-). like, if you really used an
external db, you'd interface to the db from the command line and
modify stuff on the fly. big dumb hammer time.

[Malek]

That's an interesting suggestion. It sounds worth looking into more closely if there are no more straightforward way. Is it what you do professionally ?

Anyone else does it that way or differently ? I'm taking all ideas and workflows :)

Since Evoland has just been released, do you know whether Nicolas use live code editing to, for instance, modify a method on the fly and fine tune a rendering engine in live ?

[Raoul Duke]

has anybody used Tora to make an on-device, local, app engine?

"- code hot-swapping : you can deploy an updated version of your .n
bytecode without having to restart the server, actually clients don't
even get disconnected !"

[BP]

You could consider scripting part of your app using hscript, which you  could replace at runtime:

https://code.google.com/p/hscript/

Note that there's also an hscript to actionscript converter, so you possibly consider for actually compiling  your scripts as well (haven't used it) 

https://code.google.com/p/fscript-finterp/

and for an in app "console", you could use GConsole:

http://lib.haxe.org/p/gconsole

[Malek]

Sounds like a close match !

It also replies to my question about Evoland, since hscript is made by Nicolas he could have been using that.

So to be 100% clear, how do YOU use it in your workflow ? Let's keep the example of a rendering engine. We gotta make sure our math is correct and the various arbitrary factors are chosen adequately, so we'll often tweak formulas, add or remove a couples line of code or fine tune various easing factors. How are you using GConsole and hscript ?

I can imagine coding a draw function normally to benefit from code completion, and then turning the content of that function into a string. Then adding the necessary HScript interpreter code and compiling, and from there modifying the draw string from GConsole. That's fine but not that streamlined when you gotta debug many different places in the code instead of just one function.
Do you know a smoother approach with less friction ?

[BP]

Well I'm still on the fence about it a bit.... someone in another thread a bit ago mentioned that hscript seemed fast for desktop, but a bit slower on android.... but the way I'm currently using it is:

- have my scripts in a "scripts" folder.

- embed in my app a loader to load a script on a timed interval if I choose, so my hscripts automatically get reloaded and reparsed, but not executed. 

- I then update the hscript code, and a couple of seconds later, the new code is in my app. 

- I currently serve the scripts from my scripts folder using "python -m SimpleHTTPServer" to be  a simple web server, so it works for both desktop and Flash.

I'm not currently using GConsole, I just knew about it and thought it'd possibly be a fit for you ;-)

[BP]

[Malek]

Yeah, reloading from a file is better than updating the string through GConsole. That should also add some grease to the process and make the source cleaner. I'll do that, thanksalot :)

As for your speed concern, why aren't you using the fscript-finterp as you suggested me ? It seems to take care of it, if I understand its use properly. But then I have to wonder, if I do understand, why didn't Nicolas update hscript so it produces bytecode at runtime as well ? (But maybe I didn't understand properly and fscript-finterp doesn't turn the string we input into bytecode at runtime. Gotta test that later I guess :) )


Of course if anyone has other suggestions I'm still eager to hear them :)

[Brad Parks]

My main concern as a platform is for the mobile platforms, where there
was supposedly a bit of a speed issue.... I'm pretty much where you
are, and still looking at ideas for it.... but the hscript approach
seems to be the cleanest and simplest approach cross platform, *and*
supported by a browser based target like Flash or HTML5, which I think
is super important.....

I'll probably end up using the fscript-finterp at some point.... i
think just having it working right now is good enough for me.... i'll
optimize later on ;-)

> --
> To post to this group haxe...@googlegroups.com
> http://groups.google.com/group/haxelang?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Haxe" group.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Malek]

Okay, thanks for your advices :)

I'm yet to find anything better for the Flash target so I'll use that. There's fine stuff like De Monster Debugger, but you can't edit whole methods with it.

[Malek]

It seems that fscript-finterp hasn't been updated since 2009... So it might not be too compatible with Haxe 3.

If anyone is good enough and willing to check and update it if necessary, that would be superawesome. Unfortunately this is too low level for me to help.

[AS3Boyan]

Take a look at my project:

http://www.youtube.com/watch?v=reNmLB9lslQ

Maybe this will be enough for you???

It's currently wip, it's working fine, but I think to make it work with SWC I will need to make some changes.

понедельник, 15 апреля 2013 г., 14:09:47 UTC+3 пользователь Malek написал:

[BP]

that looks super cool... is your project opensource or available somewhere?

[AS3Boyan]

I am thinking about making it open source... Currently it's isnt available. But anyway I need to make few changes, to make it easier to use. 

- Installer.

- pack Flash Develop template somehow(for easy install)

- Test it with another haxelibs(actuate works fine)

- SWC support(currently Starling SWC isn't working)

- convert Neko to C++

- make a way to stop this process

I will release it, maybe it will be open source, I don't know yet. Anyway I will notify you when it's ready.

суббота, 27 апреля 2013 г., 20:51:01 UTC+3 пользователь BP написал:

[AS3Boyan]

I am glad you liked!

суббота, 27 апреля 2013 г., 20:51:01 UTC+3 пользователь BP написал:

that looks super cool... is your project opensource or available somewhere?

[Tom]

It would be nice, if You decide to make this project open source.

Until it here is a working Starling 1.3 for Haxe 3. Maybe the externs are not so good documented, but working from the lib\staling folder.
Try it with Your project and see if it works.

[AS3Boyan]

I can't get it to work with Starling. Maybe there is something about disposing Stage3D context3D.

I created installer that should install this tool automatically(it should ask you about installing FlashDevelop templates). Please test.

https://github.com/as3boyan/HaxeBuilder

Licensed under GPL3

[Cambiata]

Wow, this is amazing! Thank you, Denis!
It will be a tremendous help in my coding! :-)

What's the status of NME Live?
There's a NME Liver Reloading project in the HaxeBuilder installer, but I can't see that it starts HB on normal project run.
Any special command to invoke?

Once again - Thanks!

J

[Malek]

Sounds like a good initiative, I'm not sure it's enough yet though! Hope I'm wrong. Let me explain how I understand it.

What you have is a SWF that is being recompiled ENTIRELY each time code is modified, safe from Haxe compiler caching features, just like if I hit CTRL+ENTER myself except more convenient.

The problems are as follows:
- The app or game state disappears. It's this feature that actually makes live coding so incredibly awesome. Without it I have to make a sad face. :(
- There is a noticeable delay that might add up with bigger projects, though I'm not sure how bad this can get since Haxe compiler server caches stuff.
- The tool might be a little intrusive ? If it could be added in a project as a lib, that would be best, but I haven't given a single thought to studying the feasibility of this. Just throwing it up in the air.

Maybe the first 2 issues could be fixed by compiling a tiny SWF containing the diff code, and have it loaded into the main SWF that is waiting, already compiled, and has received interaction already (i.e. has a non default state). Then the main SWF could do whatever is needed to start using the diff code without losing current state, such as cloning objects, swapping references and whatever else.


Also I'm only talking about SWF but in the end it should work on all platforms that Haxe targets... But one step at a time I guess ^^

[Philippe Elsass]

I think it would be worth doing something like COLT:

https://github.com/code-orchestra/documentation/wiki/Documentation

It is fairly elaborate and works quite well. Of course it requires to be able to load code at runtime.

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---

You received this message because you are subscribed to the Google Groups "Haxe" group.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Philippe

[Malek]

Nice ! That's the tool I was talking about here but I didn't find this page. It seems that it has everything we could be looking for, except it's AS3-only at the moment.

Java's hot code swapping feature (i.e. live code editing) leveraged by Eclipse gives amazing results. COLT is awesome, but the least intrusive solution is the best one IMHO.

We have 4 approaches so far:
- AS3Boyan's solution works only on a special kind of project (downside), but doesn't need to add libs or extra code in the source for it to work (upside). Though it's not real live code editing just yet.
- COLT possibly works in any AS3 project but is apparently a SWC and a program running along side FlashDevelop, and it requires adding extra code. Although the [Live] syntax is just neat enough for this to be acceptable.
- hscript. I've been using it since BP told me about it. It's pretty good, but it's limited the way I use it. Maybe hscript and fscript-finterp can be reused to make a full blown live code editing solution for Haxe, I don't know.
- Eclipse. I think it doesn't require anything but a Java compiler flag, which is ideal. Maybe it can be replicated with Flash and other targets by integrating something into Flash develop that would work like COLT, but skipping the SWC and separate program parts. The downside is that FlashDevelop might not be able to support all Haxe targets, that's a lot of work.

[David Peek]

Dammit, I had other stuff I was supposed to do today! :P

https://dl.dropboxusercontent.com/u/6286350/hxlive.mov

(top is html, middle is flash, bottom is native)

Just a proof of concept for now, project is here:

https://github.com/dpeek/hxlive.git

build.hxml generates hscript, use nme to build html/flash/native.

Basically functions marked as @live are compiled to hscript and replaced with calls to a proxy object. The proxy reloads the hscript periodically, and parses if it has changed. There are all sorts of hoops you'd have to jump through to make it work with a wide range of use cases, (ie. I'm replacing all field access with Reflect.getProperty/Reflect.setProperty) but at least it works, in principle.

Built with svn/master of all the things.

David

[Nicolas Cannasse]

Le 09/04/2013 21:49, Malek a �crit :

> Hey guys,
>
>
> When you develop with Haxe many of you are likely to be using
> FlashDevelop as an IDE, because it's the one that integrates Haxe best
> as of today.
>
> I was wondering, how do you do live debugging ?

I had a thought for it some months ago and here's what I came up :

- use JS output (or Neko) which are both dynamic enough to allow runtime
prototype changes.

- when you want to update the code, go through all the classes and mute
the prototype with the new version

For instance that would be the following JS code :

var MyClass = MyClass || function(args) { /* constructor_code */ };
MyClass.prototype.myMethod = function() {
/* method code */
}

This way everytime you recompile + reload the JS all the method code
gets updated.

Some things can be improved in order to deal with new variables :

- you can create some additional code that will store all instances
created (as weakmap keys for instance) and when you update the code,
will run some parts of the class constructor an all instances in order
to define default values for new member variables

So for instance if you have the following constructor :

function new() {
this.x = 0;
this.y = 1;
this.name = "Test";
doSomethingElse();
}

You will generate the following method :

function MyClass_fix(inst) {
if( typeof(inst.x) == "undefined" ) inst.x = 0;
if( typeof(inst.y) == "undefined" ) inst.y = 1;
if( typeof(inst.name) == "undefined" ) inst.name = "Test";
}

And apply it on all live instances of the class after rebinding the
prototypes.

That should work quite well and does not require many changes in the
Haxe compiler.

Best,
Nicolas

[Malek]

Amazing! Sounds like it's maintaining state and all. I think it was worth not doing whatever it is you had to do instead :P

Reading the source, it seems you hardcoded that it will be the update function that will be called in your Macro. Didn't using field.name work ?

Must it really depend on NME to get compiled code ? That's quite a big dependency that isn't useful for all projects. Maybe relevant NME code could be reused and wrapped into a lightweight set of classes, similar to fscript-finterp ?

Do you see a similarly straightforward way to update assets like COLT does ? Graphics, sounds, models, whatever binary streams someone is loading, it sounds tricky to make that work automatically with asset managers that keep data in memory, or write decompressed data to disk. They'd have to be updated too.


Anyway, very interesting PoC :)

[David Peek]

Whoops, field.name would work, just forgot to update. The dependency on NME is only so I didn't have to write a cross platform display object for testing, should work with anything.

Was also reading about the v8 tools for patching functions directly in the VM, as seen in this light table demo:

http://www.youtube.com/watch?feature=player_embedded&v=gtXpOD6jFls

Which seems like a great approach for js targets (just compile normally, diff and update changed methods)

[Malek]

@Nicolas:

Thanks, that's sounds good. A macro would work no ? Why would the compiler need changes ? Unless you want to have live code editing enabled through a compiler flag and have no macro and no extra lib necessary, just like Java's hot code swapping feature, in which case I have to say yay and clap hands a little.

Two issues would remain though:
- Sounds like it won't work on C++, Flash 9 and Stage3D
- Remains the issue of live *asset* editing, as I said in my previous post. A pretty neat feature that is, but sounds trickier to implement.

[David Peek]

I can imagine how you might do asset reloading, in Flash and JS at least (as long as your bitmaps don't change size). Not sure about native targets, would need to dive into the asset code.

Will give it some thought.

On Tuesday, 7 May 2013 17:28:17 UTC+10, Malek wrote:

[Alexander Kuzmenko]

I really doubt live asset reloading can be smooth. E.g. i have a tilesheet in assets, now i copy some tile from tilesheet into another BitmapData and do some fitlers/color changes/copypixels into this new BitmapData instance. how to update this instance on tilesheet update?

вторник, 7 мая 2013 г., 11:42:10 UTC+4 пользователь Malek написал:

[Malek]

Man, Light Table is some serious stuff, too bad it's JS only. Seems like everyone has been rushing towards implementing live code editing recently. If that is so it's probably quite urgent, strategically speaking, that either Haxe or FlashDevelop gets it "out of the box" too.


Live asset reloading is very interesting too, but probably less crucial as it can be implemented a bit more easily by the user while on the other hand, it sounds tricky to implement as a generic solution like COLT seems to be doing.

[AS3Boyan]

Cambiata 

I thought, that it would be better to write answer to your question and place it to HaxeBuilder blog.

You can find more info here:

http://haxebuilder.blogspot.com/2013/05/haxebuilder-101-faq.html

[Jonas Nyström]

Thanks you! Great!

2013/5/7 AS3Boyan <denis.f...@gmail.com>

--

To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "Haxe" group.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Med vänlig hälsning!

Jonas Nyström
jona...@gmail.com
070-670 36 50

[AS3Boyan]

Nicolas, what should I use if I want to use Neko? Can format.neko be usefull for this? or it's better to use neko.vm.Loader?

- when you want to update the code, go through all the classes and mute 
the prototype with the new version 

So I need to generate Neko .n file on every change and use something like Reflect and maybe Type to update classes with functions. 

- you can create some additional code that will store all instances 
created (as weakmap keys for instance) and when you update the code, 
will run some parts of the class constructor an all instances in order 
to define default values for new member variables 

Looks like to use this, I will need to modify source code, to add every new instance to weakmap. 

On Tuesday, May 7, 2013 10:22:30 AM UTC+3, Nicolas Cannasse wrote:

Le 09/04/2013 21:49, Malek a �crit :

[AS3Boyan]

Alexander Kuzmenko

That is one of the reasons, why I am working on implementing live reloading.

I tried COLT, and I don't know to how write game from scratch with this tool(if anyone got idea how to do this, please let me know), looks like it's impossible to do this with this tool. Looks like it's only usefull when you need to tweak something.

Because currently COLT doesn't update constructor code.

I saw Loom engine videos, looks like it's possible to write games from scratch using that tool. But I haven't tested this.

Nicolas, thank you for great ideas on how to implement live coding. In order to update prototypes, we will need some access to global scope in JS, or to keep all prototypes in some map and update them on every reload.

[Максим Зубков]

Hi all,

just passing by ;)

On Wednesday, May 8, 2013 12:06:24 PM UTC+2, AS3Boyan wrote:

I tried COLT, and I don't know to how write game from scratch with this tool(if anyone got idea how to do this, please let me know), looks like it's impossible to do this with this tool. Looks like it's only usefull when you need to tweak something.

 

Indeed, the thing is we did not want to wait too long and went with "one step at a time" release. So atm adding new stuff is not supported, but this restriction must be lifted at some point.

 

Because currently COLT doesn't update constructor code.

this is only impossible thanks to flash allowing things like if (param) super (a); else super (b); - we can't figure how to proxy this code. if you don't have this kind of shit in constructor, just move it to private function init() and it will work.

  so

P.s. personally I think it's great that you guys exploring this concept in haxe, and I'm surprised it took haxe community so long to even try to attempt it. The more technologies will adopt this idea, the easier will it be to sell the idea ;)

[AS3Boyan]

just move it to private function init() and it will work. 

So currently it will not work, right? 

[Makc]

On Wed, May 8, 2013 at 3:56 PM, AS3Boyan <denis.f...@gmail.com> wrote:
>> just move it to private function init() and it will work.
>
> So currently it will not work, right?

it will if you move the code by hand. it's just that colt does not do
it automagically.

[AS3Boyan]

package
{
 import flash.display.Sprite;
 import flash.events.Event;
 import flash.text.TextField;
 
 public class Main extends Sprite
 {
 
 public function Main():void
 {
 init();
 }
 
 private function init():void
 {
 var tf:TextField = new TextField();
 tf.text = "test9";
 addChild(tf);
 
 graphics.beginFill(0x68D9C2);
 graphics.drawRect(0, 100, 200, 200);
 }
 
 }
 
}

When I change init function nothing happens. When I change text property, or graphics methods nothing happends... It's only working when I change something in onEnterFrame function...

Am I missing something?

[Makc]

> Am I missing something?
>

Yes, documentation :) There is no magic, and nothing in the code can
happen on its own, you have to call the code for it to do work. In
case of enterFrame, flash calls your code for you. In case of
mouseXxx, user can click the button to call it. Same applies to whole
method chain called from there. But init()-like stuff that works once,
still works once even if you change it. It is only effective for new
instances of the class.

If you're interested in COLT I suggest you to join the skype chat,
before somebody installs forums, and not hijack this thread :)

[AS3Boyan]

I am too lazy to call code=). Like you know, when I want to change properties of particle system, I will need to

write it in onEnterFrame... like (particlesystem.scale = 2).

I just want to write code as usual and see result.

I think Loom engine can do this, I haven't tested it, but Loom doesn't not support export to Flash. And limited to 2D.

Maybe COLT can be usefull for tuning game balance.

[AS3Boyan]

I made a new version of HaxeBuilder, rewritten from scratch. Main target is Haxe/OpenFL(flash, html5 still needs work).

Please take a look at this video, to see how it works.

http://www.youtube.com/watch?v=uBtXKxHBAac

And this video is about StablexUI, how to use it with HaxeBuilder:

http://www.youtube.com/watch?v=qA1w5iuR4-k

HaxeBuilder just recompiles code on any file change(just like you do in console "openfl build flash" + compilation server). So basically it's not live coding nor live reloading I think.

Possible use cases: 

• you can use it to adjust positions
• create UI
• tune some things
• and maybe even learn some lib like HaxeFlixel or StableUI(because you may need to recompile project to see changes very often to understand how to do something with those libs).
  

Github page:

https://github.com/as3boyan/HaxeBuilderOpenFL

[Cambiata]

Thank you, Boyan!

[AS3Boyan]

You are welcome!

Thank you, Boyan!

[Philippe Elsass]

Looks interesting - one thing that would be nice in OpenFL would be to have live assets reloading which could be more granular to handle than a full recompile/reload.

On Sat, Aug 24, 2013 at 10:25 AM, AS3Boyan <denis.f...@gmail.com> wrote:

You are welcome!

Thank you, Boyan!

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---

You received this message because you are subscribed to the Google Groups "Haxe" group.

For more options, visit https://groups.google.com/groups/opt_out.

--
Philippe

[AS3Boyan]

Thank you, Philippe.

I am sure you saw this:

https://github.com/dionjwa/catapult

And seems like this project can do many things.

I haven't tried it yet, I'm not sure how it can handle update of spritesheet(TileLayer).

[Philippe Elsass]

Oh I had forgotten about catapult - must check that.

[Joshua Granick]

Do you think we could do the same by hosting this in the OpenFL tools, rather than needing npm and hosting a separate server?

[Philippe Elsass]

The server is written in Haxe but based on node.js:

https://github.com/dionjwa/catapult/blob/master/src/catapult/Server.hx

Maybe it could be ported to Neko API, then it would run directly as an OpenFL tool, but I'm not sure everything exists on Neko's side: for instance networkInterfaces, file watching.

[AS3Boyan]

Catapult workd on only html5?