Using an ugly hack to enforce macro execution order, is there a better way?

[Drakim]

Hi fellow Haxers,

I'm working with new experimental web technology involving multithreading with shared memory. This means forgoing lots of the higher level language features such as objects in favor of simple primitives. Fortunately Haxe is awesome for this, since I can use Abstracts and Macros to create syntax sugar on top of these primitives.

However, there is one issue I keep running into when using project-wide macros in Haxe: The execution order of macros. I often need one macro to not execute before all other instances of another macro have safely executed. Enforcing this is tricky!

Imagine the following scenario: All across our project we have various macro calls defining letters of the alphabet. Once all of these have run, we want to run a macro that generates an alphabet lookup table using all the letters that have been defined.

// We have these shattered across various files
Alphabet.defineLetter('A'); // macro
Alphabet.defineLetter('B'); // macro
Alphabet.defineLetter('C'); // macro

// This macro should run only after all defineLetter calls have run
// It generates a runtime lookup table filled with all the letters that have been defined
Alphabet.generateAlphabet(); // macro

// And lastly we use the alphabet lookup table to do neat stuff
AlphabetTools.fetchRandomLetter(); // non-macro runtime function



Now, I actually managed to achieve the proper macro execution order with a neat trick, I simply imported every file that calls defineLetter() at the top of the Alphabet class file. This way, every defineLetter() macro call is run before the generateAlphabet() macro is called. Problem solved!

...or so I thought. As I continued coding, I came across a situation where I needed to use the generated alphabet in one of the files that calls the defineLetter() macro. Something like this:

// file1.hx
Alphabet.defineLetter('B');
var foobar = AlphabetTools.fetchRandomLetter();

// file2.hx
Alphabet.defineLetter('C');

So what happens is, fetchRandomLetter() will obviously need to refer to the lookup table of all the letters. So, I have Alphabet.generateAlphabet() invoked inside of it, and thus it ruins my entire neat little arranged execution order. Now we miss out on all the defineLetter() calls that happens after 'B'. What appears to happen is that in file1.hx, when the compiler comes across the line containing AlphabetTools.fetchRandomLetter(), the compiler looks into the function and resolves the macro calls waiting inside it. This causes the macro Alphabet.generateAlphabet() to be called prematurely.

Again, I actually managed to find a workaround to this, but my workaround is horrible and ugly, and I'm hoping you guys might know of a better way.

What I do is this:

class AlphabetTools {
  static dynamic public function fetchRandomLetter() {} //Empty function that does nothing

  static public function unusedFetchRandomLetter() {
     var lookuptable = Alphabet.generateAlphabet(); // macro call that returns a static lookup table
     return lookuptable[Std.random(lookuptable.length)];
  }

  static var temp:Int = tempRun();
  static public function tempRun() {
    AlphabetTools.fetchRandomLetter = AlphabetTools.unusedFetchRandomLetter;
    return 0;
  }
}

What happens here is that I leave a empty dummy function fetchRandomLetter() for the compiler to find, which does nothing. But I mark it as "dynamic" so that it can be changed during runtime. During the startup of runtime, the tempRun() function swaps in the unusedFetchRandomLetter() function on top of the fetchRandomLetter, so that all the calls to fetchRandomLetter() ends up running unusedFetchRandomLetter() instead. But, in the eyes of the compiler, no file actually invokes unusedFetchRandomLetter() directly, and thus there is no need to resolve it's inner macro prematurely. The macro still gets resolved in the end but it doesn't happen the instant some other file refers to fetchRandomLetter();

So now the macro execution order is correct, but this solution is very ugly, and uses features in way they aren't really meant to be used. Is there a "clean" way to do it instead?

(And note that I'm not actually building the alphabet with a macro, it's just a metaphor for something else. The real data involves lots of non-primitives such as objects and function callbacks, and thus cannot be stored as class @metadata)

[Alexander Kuzmenko]

https://github.com/haxetink/tink_syntaxhub

понедельник, 1 августа 2016 г., 15:05:50 UTC+3 пользователь Drakim написал:

[Marc Weber]

1) idea parse source (looking at all .hx files in include files)
hacky, but might work

2) Have a look at: https://haxe.org/manual/macro.html
there might be alternative phases you can either dump a file you can
load at runtime (resource) or change the ast.
(I never used it)

3) Or create a file dumping A B C to it.
If you find that it contains a char too much or one is missing show an
error only (this could be at later compilation phase eventually).

Then order doesn't matter but you still have a "check".

I guess all you cane that the alphabet is sound, but soundness check
could be done after using it.

Marc Weber

[Justin Donaldson]

(Off topic) Hey Marc, good to see you post again

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---
You received this message because you are subscribed to the Google Groups "Haxe" group.
For more options, visit https://groups.google.com/d/optout.

[Drakim]

@Alexander Kuzmenko

I'm not familiar with haxetink. but tink_syntaxhub seems like a plugin for organizing other think plugins that runs macros? Am I understanding this right?

@MarcWeber

1) I'm not sure I understand, parse source?

2) I've been trying various things, but unfortunately it seems like all the various macro entry points and callbacks run at the wrong time, too early or too late.

3) Yup, I do something similar. Instead of dumping it in a file, I simply have a macro class that keeps track of what's been "collected" already, so that if anything is run twice, or too late, an error gets thrown. However, doesn't help the issue of missing pieces. The lookup table itself needs to be built at some point, and unlike an alphabet, I can't really know if some pieces are missing or not. And if the lookup table has been built, it doesn't help if my macro catches that something was "collected" post-building, it can't force the lookup table building macro to rerun (as far as I know).

The fundamental problem for me seems to be that there are two ways macros are "run" by the compiler. Either just the predictable "top to bottom" order of each file, but if the control flow of the code jumps around due to function calls, this can also trigger macros in those function definitions to be "run" earlier than they normally would be otherwise.

[Alexander Kuzmenko]

tink_syntaxhub can order your @:build macros. But it looks like you are trying to organize ordinary "function" macros. I don't think it's possible. More over i think it's a bad idea to rely on macro execution order since it's undefined.

What about defining your alphabet in some sort of external config file and then just load all of it in one go when it's requested for the first time?

вторник, 2 августа 2016 г., 10:48:50 UTC+3 пользователь Drakim написал:

[Drakim]

I could do that, but the whole point of this syntax sugar macro is to allow the declarations of the "alphabet" to be in the files they are used, as opposed to having to manually append them to a file by hand. It's so neat and orderly when you can just open a .hx file and read what it does, and not have to jump to a special static lookup table file and scroll down x pages to find your relevant entry. :/

[Skial Bainn]

You can kind of do what you're asking, see https://gist.github.com/skial/9f6b0f0a53f67d6fce4752a8010b7e2d for experimental code. I'm using a nightly build of Haxe, but it should work with the current 3.3.0-rc.1 release as well.

On Tue, Aug 2, 2016 at 10:45 AM, Drakim <kimrobin...@gmail.com> wrote:

I could do that, but the whole point of this syntax sugar macro is to allow the declarations of the "alphabet" to be in the files they are used, as opposed to having to manually append them to a file by hand. It's so neat and orderly when you can just open a .hx file and read what it does, and not have to jump to a special static lookup table file and scroll down x pages to find your relevant entry. :/

--

[Drakim]

Thanks Skial, I'll try to adapt this to my usage. It looks promising.