SwfLoader

[maxim.porges]

Hey guys,

Just a quick note to let you know that I was officially back in the
saddle this week with Loom, working on the SwfLoader. I've got it
parsing out the tags in a SWF file, but when I try to read the ABC
bytecode block in each ABC tag there seems to be an RDF XML block
ahead of it that's not documented in the spec, so I am working my way
through that issue.

I've got two example SWF-parsing apps to work from as references so I
expect I will have this working later in the week.

Thanks,

- max

[Dirk Eismann]

cool - this definitely raises my excitement :)

Btw, one use case I have in my mind is the ad-hoc creation of Proxy
classes for classes that are compiled into the SWF already.

So instead of loading something external with the SWFLoader I'd like
to grab the ABC byte code directly from the running SWF by using the
loaderInfo.bytes ByteArray of the stage object (not sure if this bytes
property is also available on child nodes, I guess not).

At runtime, this ByteArray actually is the representation of the SWF
file that is currently loaded so it should be possible - shouldn't it?
So while you're implementing a SWFLoader could you please also add a
loadBytes method to it so we can just pass over *any* ByteArray that
resembles a SWF file? Thanks :)

Dirk.

[Maxim Porges]

Dirk,

If the property you are describing does what you say it does, then this is something I did not think existed. Being able to access the SWF bytes at runtime will offer another nice way to pull the ABC files. Might be way to do it without async loadng, too.

There's no reason why the code behind the SwfLoader shouldn't support this too. I will check it out - thanks for the tip!

- max

-----Original Message-----
From: Dirk Eismann <wuppe...@googlemail.com>
Sent: Tuesday, May 19, 2009 2:55 AM
To: loom-as3 <loom...@googlegroups.com>
Subject: Re: SwfLoader

cool - this definitely raises my excitement :)

Btw, one use case I have in my mind is the ad-hoc creation of Proxy
classes for classes that are compiled into the SWF already.

So instead of loading something external with the SWFLoader I'd like
to grab the ABC byte code directly from the running SWF by using the
loaderInfo.bytes ByteArray of the stage object (not sure if this bytes
property is also available on child nodes, I guess not).

At runtime, this ByteArray actually is the representation of the SWF
file that is currently loaded so it should be possible - shouldn't it?
So while you're implementing a SWFLoader could you please also add a
loadBytes method to it so we can just pass over *any* ByteArray that
resembles a SWF file? Thanks :)

Dirk.

On 18 Mai, 03:06, "maxim.porges" <maxim.por...@gmail.com> wrote:
> Hey guys,
>
> Just a quick note to let you know that I was officially back in the
> saddle this week with Loom, working on the SwfLoader. I've got it
> parsing out the tags in a SWF file, but when I try to read the ABC
> bytecode block in each ABC tag there seems to be an RDF XML block

[The entire original message is not included]

[Josh McDonald]

If that's possible, I'll be over the moon! It'll probably let me do exactly what I want :)

2009/5/19 Maxim Porges <maxim....@gmail.com>

--
"Therefore, send not to know For whom the bell tolls. It tolls for thee."

Josh 'G-Funk' McDonald
  -  jo...@joshmcdonald.info
  -  http://twitter.com/sophistifunk
  -  http://flex.joshmcdonald.info/

[Dirk Eismann]

here's a quick example on how to access the SWF's own bytes:

package
{
import flash.display.Loader;
import flash.display.Sprite;
import flash.events.Event;
import flash.utils.ByteArray;

public class LoadYourself extends Sprite
{
public function LoadYourself()
{
super();
addEventListener(Event.ADDED_TO_STAGE, addedToStage);
}

private function addedToStage(event:Event):void
{
var bytes:ByteArray = this.loaderInfo.bytes;
var loader:Loader = new Loader();
loader.loadBytes(bytes);
}
}
}

when you debug this you should see that the SWF gets loaded twice :)

[Josh McDonald]

"Bart, I could kiss you!.. If the bigger brothers hadn't made me sign a form promising I wouldn't."

This should make Loom *much* easier to use! Not sure it'll kill the need for it to be asynchronous, but I suppose it might, it depends on whether Player defers adding to the VM until it gets control back.

Nice one! :)

-Josh

2009/5/20 Dirk Eismann <wuppe...@googlemail.com>

[Maxim Porges]

I echo Josh's sentiments! :)  I'll use accessing the stage's bytes as the default approach so long as I can get it working, and will make a SwfLoader second fiddle. I'm interested to see how RSLs will play in to the mix, but we'll get the basic use cases covered before we start getting too adventurous.

FYI, I figured out the issue I was having with RDF metadata - classic faceplant. The AbcDeserializer in Loom always started at byte 0 of the byte array given to it, rather than continuing on from the current position of the byte array. This worked fine for ABC files, but not so much for SWFs... I've modified AbcDeserializer to accept an optional parameter for the start byte, defaulted to 0.

My last stopping point was on a subtle detail of the AVM2 spec that I had missed earlier, which is that private namespaces are always considered unique. Should be an easy fix. Short of any additional hiccups, I'm feeling good about having the SWF-parsing code working in the next seven days.

Thanks,

- max

[Josh McDonald]

That makes me wonder! Could we extract the protected and private namespaces of objects by inspecting the abc? Something like:

var buttonPrivate:Namespace = Loom.getPrivateNamespace(Button);
var secret:QName = new QName(buttonPrivate, "skinMeasuredHeight");
var skinHeight:Number = myButton[secret]; //<clarkson>Powerrrrr</clarkson>

Now *that* could be useful for all sorts of nefarious and custom purposes. I could build private-member injection into SmartyPants-IOC (and Christophe could do the same for Prana, etc). And we could use it for some *nice* serialization support, too.

-Josh

2009/5/20 Maxim Porges <maxim....@gmail.com>

[Maxim Porges]

All the ABC namespaces are available in the AbcFile object. There is a ConstantPool that holds references to all the LNamespaces (i.e. Loom namespace objects, since Namespace was already taken this was easier to autocomplete) in a property called namespaces.

I'm not sure how this would work based upon the code sample you sent, but I'd say give it a try and see what happens. My guess is that the AVM will enforce the private nature of the namespace, since each namespace still has a type within the AVM (one of Namespace, PackageNamespace, PackageInternalNamespace, ProtectedNamespace, ExplicitNamespace, StaticProtectedNamespace, and PrivateNamespace). Also, it is hard to differentiate between the individual private namespaces since many of them are called private::*.

Private member injection on the other hand is totally do-able, all you need to do is modify the instance traits of the class (or class traits if you want to add static fields). As far as serialization support, you would hope that private members intended to be persisted were at least accessible with public properties, but I suppose it depends on the use case.

I guess this will be something for us to dick around with once Loom is parsing more complex code.

- max

[maxim.porges]

Hmm... I may have to spoil the party here with regard to async
loading.

Unfortunately, I don't have a synchronous way that I have found to
load proxies in to the AVM. The SwfLoader was going to read the SWF,
catch the loaded event, generate the proxies, load them in to the AVM,
catch the loaded event, and then announce that the SWF had loaded. To
the end developer, it would appear as only one loaded event, and if
this was the loaded event for the whole app, to the end user it would
appear as if the app just loaded as normal.

So, even if we can pull the SWF bytes from a property, the act of
generating proxies and loading them in to the AVM will still create a
second async event. Unless this event is masked within the SwfLoader,
the end developer will have to deal with it prior to using any code
that relies on proxies.

If you guys have any ideas for getting around this os see flaws in my
logic, please let me know.

- max

> > 2009/5/20 Maxim Porges <maxim.por...@gmail.com>

> >> 2009/5/20 Dirk Eismann <wuppert...@googlemail.com>

>
> >> here's a quick example on how to access the SWF's own bytes:
>
> >> package
> >> {
> >>        import flash.display.Loader;
> >>        import flash.display.Sprite;
> >>        import flash.events.Event;
> >>        import flash.utils.ByteArray;
>
> >>        public class LoadYourself extends Sprite
> >>        {
> >>                public function LoadYourself()
> >>                {
> >>                        super();
> >>                        addEventListener(Event.ADDED_TO_STAGE,  
> >> addedToStage);
> >>                }
>
> >>                private function addedToStage(event:Event):void
> >>                {
> >>                        var bytes:ByteArray = this.loaderInfo.bytes;
> >>                        var loader:Loader = new Loader();
> >>                        loader.loadBytes(bytes);
> >>                }
> >>        }
> >> }
>
> >> when you debug this you should see that the SWF gets loaded twice :)
>
> >> --
> >> "Therefore, send not to know For whom the bell tolls. It tolls for  
> >> thee."
>
> >> Josh 'G-Funk' McDonald

> >>   -  j...@joshmcdonald.info

> >>   -  http://twitter.com/sophistifunk
> >>   -  http://flex.joshmcdonald.info/
>
> > --
> > "Therefore, send not to know For whom the bell tolls. It tolls for  
> > thee."
>
> > Josh 'G-Funk' McDonald

> >   -  j...@joshmcdonald.info

[Josh McDonald]

No, you're right. Even if you're simply calling loadBytes on existing code it doesn't make it into AVM until either the next frame, or at the very least when the execution thread makes it back to the AVM housekeeping code in Player. I was reading something that confirmed this just yesterday arvo. Lucky for us FlexUnit 4 is in alpha and has an async setup :)

-Josh

2009/5/22 maxim.porges <maxim....@gmail.com>

  -  jo...@joshmcdonald.info

[Dirk Eismann]

I can live with it pretty well - being a Flex developer since version
1.0 I'm so used to asynchronous coding it just doesn't matter
anymore :)

Dirk.

[maxim.porges]

Time for some good news and bad news.

Good news first: I've got the code written to parse a SWF and extract
the ABC bytecode. I implemented another 70 or so opcodes to get the
parsing to complete, and then (as usual :) ) I hit a few unexpected
items.

So, here's the bad news.

1) The code in the SWFs is 100% compliant with the AVM2 spec, except
for multinames on class instance data and traits. The spec states very
clearly that the names for these two items should always be
QualifiedNames (a name with a single namespace), but I'm getting names
with multiple namespaces (names with multiple namespaces to be
searched to resolve their reference). Since this behavior is non-
compliant, I have a question out to the Tamarin list about this
because (a) I'm not sure if this is "right" and (b) changing it in the
domain will require some moderate refactoring.

2) My expectation was that the ABC code in the SWFs was one full ABC
class definition after another; this is unfortunately not the case. It
seems that when the compiler makes a SWF, it turns all your nice neat
class definitions in to one gigantic file. All the classes are defined
within this file, with every method and property required by them
listed in sequence. The AVM2 spec does indicate that multiple classes
can exist in a file, but I always thought this was for class
definitions with inner classes. My guess is that everything is smashed
together to save space on duplicate constant pool entries.

Clearly, neither of these items are insurmountable, but they mean a
little more work. Once I get an answer on #1 I'll be able to get them
both addressed. #2 means the existing proxy generation code will need
to be changed, since currently proxies are created by copying an ABC
class file and re-generating it as a dynamic proxy; instead, I'll need
to start fresh with a new ABC file and fill in the blanks. Again, not
a big deal, since the hardest part of generating the proxies was
grocking the appropriate opcodes for the overridden subclass methods;
generating new ABC files should be pretty straightforward.

Finally, parsing large ABC code blocks is somewhat slower than I
expected, so there will need to be some performance optimization done
before this is useful in production apps. I think it will be
acceptable as it is for testing and creating mocks.

Thanks,

- max

[maxim.porges]

Hey guys,

Just wanted to post an update on the SwfLoader progress.

I got an answer back from the Tamarin list. Apparently the issue I am
seeing with multinames would cause an error in the runtime, so that
indicates that the Loom parser has issues indexing the multinames in
the constant pool in larger files This is not entirely surprising
considering that the size of the ABC files in the SWF I am parsing are
much larger than anything I've tested against so far.

I have a lead on the problem. I deliberately programmed the
ConstantPool to de-duplicate items as they are added, since (a) all
the entries are supposed to be unique and (b) automatic de-duplication
makes the API for generating an ABC file structure easier, since you
can just keep throwing the same String in over and over again and it
will only ever be referenced by a single entry in the pool. De-
duplication has not presented any issues in my smaller test files, but
there are definite collisions taking place in the larger data set
causing the multiname indices to shift (and thus incorrect multinames
being resolved when parsing instance_info blocks); I know this is the
case because there are 5885 multinames in the ABC file but only 4811
in the Loom ConstantPool by the time the parsing is complete.

Incidentally, I also think the de-duplication is the cause of the
parsing performance hangup. As the number of entries goes up, so does
the search time for matches.

So, I'm going to modify the ConstantPool to operate in dedup and non-
dedup modes. Loom will assume that the compiler knows what it is doing
and will not dedup ABC code generated by the compiler; it will just
dedup entries for its own proxies as they are generated. I'll also add
a verification step to make sure that the expected number of entries
ends up in Loom after each parsing stage.

I'll keep you posted!

Thanks,

- max

[Roland Zwaga]

Hey Max,

its been pretty quiet on the Loom list lately (how's that for a nice
roll-off-the-tongue hehe).
I'm just wondering how you've been progressing. Have you run into any
major issues?
Or perhaps you just don't have time to do any Loom work right, that's
very possible too :)

cheers,

Roland

[Maxim Porges]

Hey Roland,

Thanks for pinging the list. I've been slacking tremendously on Loom;
work's been busy which has resulted in me spending my weekends in a
"not programming" state. I do still want to get it finished, and
should be back on it in a week or two.

There's actually nothing major on the "to do" list at the moment. I
need to make a ConstantPool variant that doesn't de-duplicate for
parsing purposes, and then modify the dynamic class generator to make
a single consolidated ABC block for all dynamic proxies instead of on
per class, and that should be it. I also expect that the non-de-duping
pool will solve the parsing performance issues I was seeing. If not,
I'll need to add a basic hashing algorithm so we can do object
equality with a hashtable, but once again, that should be pretty minor
stuff.

Stay tuned for updates.

Thanks,

- max

[Roland Zwaga]

Hey Max,

good to hear you're still alive :) I can totally understand being in a
'not programming state'
so take it easy :) We're keeping our eyes peeled for your results
however :)

greets,

Roland