Question

[Josh McDonald]

Hey Max,

I don't really want to force users to re-load various .SWFs to use the advanced features I plan on building with Loom, so this is what I was wondering if this sort of thing will be possible:

1) Include .abc bytecode for an  empty class with the Loom methods for interception etc
2) Clone said bytecode
3) Give the class a new name
4) Change its superclass or add implemented interfaces (and dummy "return null" methods for those interfaces of course)
5) Load the new class, and get a reference to it so it doesn't get collected.

What are your thoughts?

Cheers,

-Josh

--
"Therefore, send not to know For whom the bell tolls. It tolls for thee."

Josh 'G-Funk' McDonald
  -  jo...@joshmcdonald.info
  -  http://twitter.com/sophistifunk
  -  http://flex.joshmcdonald.info/

[Josh McDonald]

[Maxim Porges]

Hey Josh,

If I understand you correctly, you're essentially describing the process Loom goes through to create proxies, only you are not overriding the superclass methods other than the ones for the implemented interfaces. Note also that step 5 is not required since Classes do not get garbage collected, only instances of Classes.

There shouldn't be any need to reload SWFs. If you build a custom SWF loader like Loom will have, the user just needs to use that to load their app, and then you can keep a reference to all the ABC files that make up the classes in the SWF. You can then re-clone/re-subclass/re-whatever dynamic ClassReferences as many times as you like, so long as you generate new names for each unique Class type.

What did you have in mind, and what is the drawback that you are trying to circumvent?

- max

[Josh McDonald]

Basically I don't want to force my users to use Loom to load their SWF, nor do I want to have to keep all the .abc files around taking up memory in Player. I still want to override the methods and use the AOP-style interception code you showed, but I want to be able to do it without inspecting the .abc file.

If I can change the parent class of a ABC, add implemented interfaces, and still override the methods, then I only need to embed the ABC of a stub class, which can be added-to at runtime to intercept the public methods of any object.

Requiring loom to load the application .swf opens up a whole can of worms - what about preloaders? modules?

-Josh

2009/4/3 Maxim Porges <maxim....@gmail.com>

[Maxim Porges]

Basically I don't want to force my users to use Loom to load their SWF, nor do I want to have to keep all the .abc files around taking up memory in Player. I still want to override the methods and use the AOP-style interception code you showed, but I want to be able to do it without inspecting the .abc file.

Based upon my knowledge of the Flash Player and AVM, this is serious "have your cake and eat it too" territory. :)

The decisions I have made so far for Loom have been shaped by (a) limited options for accessing the ABC code at runtime and (b) the desire to not have to entirely recreate the Tamarin compiler in AS3. I'll present the basis for the decisions I've made, with the hope that you have some alternate approaches in mind that will make things closer to what you are looking for. God knows I'd like to obviate the Loom SWFLoader; at present it is a means to an end (short of writing an Eclipse plugin or Ant task to enhance the SWF at compile time, which I plan on doing with AIR).

You said you don't want .abc files around taking up memory. My understanding of the Flash Player is that it loads a SWF and parses the ABC code, creating in-memory representations of ClassDefinitions which can be new'd in to instances. I believe that the ABC code is discarded after being parsed. Even if we assume for a second that the ABC code is not discarded, then you're never going to use up less memory than is required for (a) the ABC code and/or (b) the parsed ClassDefinition structures in memory. Loom is going to add ABC files to the end of a SWF's original ABC block, so it will take up no more memory than if a programmer had hand-coded the proxies.

You want to override methods and add AOP-style code interception without inspecting the ABC file. There is no way I know of to ascertain the namespace, namespace set, and multiname pool information required by the AVM spec without inspecting the ABC file, short of parsing the source for the original AS3 classes and re-implementing the Tamarin compiler. This information is not available from describeType() or any other runtime mechanism I am aware of, and without it the runtime will be unable to resolve qualified type names either statically or dynamically.

You don't want to force users to use a custom SWF loader. Since my prior paragraph assumes you can only get the ns/nsset/multiname pools from the ABC code, I don't know of any other way to access ABC code at runtime since the Flash Player provides no mechanism I have found for doing so. Solutions I am aware of include either (a) controlling the original loading of the SWF (as Loom will do) before handing it off to the Flash Player's SWF parser, or (b) re-reading the SWF off the server after it has been loaded. Option B obviously will take longer and will appear asynchronous to the application doing the re-loading, plus it will consume double the SWF memory since the SWF has to be loaded and then re-read before being discarded.

If I can change the parent class of a ABC, add implemented interfaces, and still override the methods, then I only need to embed the ABC of a stub class, which can be added-to at runtime to intercept the public methods of any object.

If you start with a base template and try to just change its type, you will have to fill in the template ABC file with all the namespace, namespace set, and multiname references for the runtime to find any and all types used by the ClassDefinition (i.e. method return types, superclass types, dynamic types, etc.). This information is required by the MethodInfo and InstanceInfo structures to describe method arguments and superclass names. Again, I have found no way to determine this stuff without either parsing the original ABC code or re-parsing the source class file. To be frank, from my reading so far the AVM spec doesn't really tell you how the scopes for the namespace sets are ascertained - only how the AVM interprets them and uses them at runtime.

Requiring loom to load the application .swf opens up a whole can of worms - what about preloaders? modules?

The Loom SWFLoader is a preloader from my understanding - it will load your application SWF before the Flash Player gets it, enhance the ABC block with the proxy ABC files, and then hand off the enhanced SWF to the Flash Player to load the whole thing as if the proxy ABC files had already been baked in by the compiler. As far as modules are concerned, ModuleLoader is described as follows in the Flex 3 docs:

"ModuleLoader is a component that behaves much like a SWFLoader except that it follows a contract with the loaded content. This contract dictates that the child SWF file implements IFlexModuleFactory and that the factory implemented can be used to create multiple instances of the child class as needed."

My goal is for Loom to be AS3-compatible with no dependance on the Flex framework, so that rules out a ModuleLoader. To my knowledge, a ModuleLoader provides no benefit over a SWFLoader for Loom's purposes, so a SWFLoader is what I plan on supporting.

Hopefully this explains my decisions so far. Like I said, I'd love to avoid a Loom SWFLoader dependance, so if you see any way out of this please let me know.

Thanks,

- max

[Josh McDonald]

Hi Max,

I didn't mean to sound like one of those "I demand to have my cake and eat it too" users of a system that's not even out yet! I don't understand the ABC format, even having read the spec a couple of times. I think when Loom escapes, I'll be able to use it to build an AIR-based util to inspect and mess with existing SWFs, poke around inside them and learn more about what's required.

What I was talking about wasn't what I'm expecting Loom to be able to do, just what I'd like to be able to do, and am hoping to use Loom to get me 80% there - standing on the heads of giants and such. My other alternative was to dissassemble a lot of ABCs, do some hardcore learning, and build up ABCs using a SWC-targeted compile of HxASM, which is a very big project for somebody with a 9-5, a company to grow, and a few OSS projects to run / help out with already. So as you can imagine I've been putting the whole thing off for quite some time now ;-)

I'm also planning to add the compiler jars into SmartyPants-J, but that won't help anybody who's just using SmartyPants-IOC against some other backend software.

I need to know a lot more about how one goes about subclassing Foo or implementing IFoo at the VM level, and I'll go read the spec again I think.

But a new quick question:

Assuming for now that I don't want to add anything at all to the class, would it be possible, assuming Foo is currently loaded, to construct the equivalent of:

public class MySubFoo extends Foo {}

Without inspecting the .abc for Foo, using Loom as you're planning it?

Cheers,
-Josh

2009/4/4 Maxim Porges <maxim....@gmail.com>

[Maxim Porges]

Hi Josh,

No worries, I was just trying to explain the challenges as I saw them with what you have in mind. I'm sure there will be a way to get there, but my grasp of the ABC spec only extends as far as I have needed it to in order to get Loom to do what I want it to.

Assuming for now that I don't want to add anything at all to the class, would it be possible, assuming Foo is currently loaded, to construct the equivalent of: public class MySubFoo extends Foo {} Without inspecting the .abc for Foo, using Loom as you're planning it?

I don't think so, for several reasons.

1) The class and superclass qualified name references are stored in the InstanceInfo block. Since the ABC spec is built for terseness, there is no way to predetermine the location of any block without reading the whole file up to that block. For example, they have some pretty neat tricks in the ABC spec for making a number only as large as it needs to be. Rather than declaring an int to always be four bytes, they say "check out the first byte, and if the last bit in the byte is 1, use that as a flag to determine that there are more bits in this value. Continue looking for final bit flags until you have read the whole number." This means that in the worst case scenario, a number that could have been represented in four bytes ends up requiring four and a half due to the "flag" bits, but in most cases you will save space. Unfortunately, this also means that you have to read every byte one at a time to parse the ABC file, since you never know how big each value will be, and the spec uses ints as indices in to the constant pool in almost every structure to save space (i.e. they have multiple references to the same values in the pool instead of explicitly repeating the values everywhere). Another example is that optional flags on items (such as traits) are only written to the ABC file if prior values hold certain flags - such as optional method arguments, which are only written to method traits if they exist on the method signature; otherwise they are omitted.

2) Technically, you could read all the way up to the InstanceInfo and flip the indices in to the constant pool to change the class qualified name and superclass qualified name, but you'd end up with an invalid ABC file. This is because in order to make a new class name, you'd have to make an entry in to the multiname pool representing it, and an entry in to the string pool for the string portions of the class name and namespace. Without maintaining the ABC file structure, you'd be unable to do this because...

3) ...of the third reason, which is that there are methods (namely the constructor and script initializer) which require references to the qualified name and multiname pool indices in order to instance a class. You have to push the class and superclass indices on to the stack during class init, so if you change the class name and superclass name you would need to modify the referenced indices in to the multiname pool in the opcode. Again, there's no way I know of to predetermine the location of these values without parsing the entire ABC file structure. I've pasted in the opcodes for the script initializer for one of my template classes in Loom below. For reference, the class hierarchy for this class is Object <= loom.template::BaseClass <= loom.template::DynamicSubClass.

function script0$init():* /* disp_id 0*/

{

  // local_count=1 max_scope=3 max_stack=2 code_len=25

  0         getlocal0    

  1         pushscope    

  2         getscopeobject 0

  4         findpropstrict Object

  6         getproperty   Object

  8         pushscope    

  9         findpropstrict loom.template::BaseClass

  11        getproperty   loom.template::BaseClass

  13        pushscope    

  14        findpropstrict BaseClass

  16        getproperty   BaseClass

  18        newclass       loom.template::DynamicSubClass

  20        popscope      

  21        popscope      

  22        initproperty   loom.template::DynamicSubClass

  24        returnvoid    

}

Again, this is just based upon what I know of the spec. I encourage you to look at what I have after I release the source to you guys and see if you can find a way to get this done.

Thanks,

- max