Re: [haxe] Alternative Haxe

[Raoul Duke]

If you are crazy enough to do such an experiment, good, more power to
you! I don't see why it should cause anybody to freak out. :-)

[Juraj Kirchheim]

I think you are making things unnecessarily complicated for yourself.

You start by doing only one of these two things:

1. write Haxe in Haxe

2. change the language design

Doing both at the same time is insanely hard to get right. If you first write Haxe in Haxe, then you have a rather big test suite to help you get things right.

Also writing Haxe in Haxe could be done gradually with either compiling Haxe with js_of_ocaml and then incrementally replacing it with haxe/js output, or by creating a haxe/ocaml backend, allowing to incrementally replace the source ocaml code.

In any case, I wish you good luck! ;)

Best,

Juraj

On Sun, Jul 26, 2015 at 9:14 AM, PeyTy <aleg...@gmail.com> wrote:

Thanks! Yeah, a bit crazy.

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---
You received this message because you are subscribed to the Google Groups "Haxe" group.
For more options, visit https://groups.google.com/d/optout.

[Itzik Arzoni]

Haxe-tng!
It IS a creazy idea. After all, haxe has 10 years of Testing and bullet proofing. But I like the fact that haxe is so versatile that compilers can be written with it.

I do have a suggestion, even crazier. If you can, make it 100% haxe compatible, such that classes written in haxe will be compiled along with classes written in tng (or whatever you call it). This way you will not loose all the libraries written in hx syntax. That would make the haxe language GROW instead of Split.

Anyway, I would be happy to hear updates.

[Juraj Kirchheim]

On Sun, Jul 26, 2015 at 10:30 AM, PeyTy <aleg...@gmail.com> wrote:

I already spent half a year. The hardest part is in past. Writing compiler (from scratch) in Haxe IS simple. I really was very surprised. Ocaml was mindblowing for me. Lexer and parser a fully handwrittend and its totally okay. Yeah, of course I'm using Ocaml sources as references :) The complete testing suite makes me happy too!

To be honest, it's not the parser I am worried about. By looking at hscript it become apparent, that parsing Haxe is not THAT much work. It's everything that happens between the frontend and backend, i.e. typing, macro execution, pattern matching, abstract handling, optimization and what not.

I'm not really sure OCaml is all that hard. It's just that the compiler team did not really exactly go out of their way to make it readable. Sparse comments and frequent abbreviations don't make exactly make code written with OCaml's rather exotic and terse syntax easy to understand. But I'm sure that could be improved.

 

I really don't see the point to use Ocaml (for writing compilers) at all.

It is fast. 

 

Haxe's ADTs and matching system makes such task stupidly simple.
Simn even has a lexer-parser framework for Haxe written in Haxe + some macro magic. But I want the most optimal way of doing things here, expessially because of new lexeme-level macro system ;)

That sounds really quite interesting. However I think you are fighting too many wars at once and risk suffering from the second system effect (https://en.wikipedia.org/wiki/The_Mythical_Man-Month#The_second-system_effect).

Best,

Juraj 

[David Elahee]

First : Good luck ! 

Second : Haxe was written with actual projects relying on it, it put both good pressure for it to actually work and kept the code base sane of non production used features. So i think if you want to be serious about having something running, you could put up an open source side project to "test" it.

Third : Good luck again !

--

To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---
You received this message because you are subscribed to the Google Groups "Haxe" group.
For more options, visit https://groups.google.com/d/optout.

--

David Elahee

[Cristian Baluta]

So much excitement that your post is intelligible. What exactly have you done? i can't see anything.

Băluță Cristian
http://ralcr.com
http://imagin.ro

[Mark Knol]

Hey PeyTy,

Woaw It sounds very ambigious what you are trying to accomplice; building and maintaining compilers for me looks quite "hardcore" :) I feel your enthousiasm! What is your drive when you are working on this already for so long!?

I already asked something on twitter, but I have some questions:
- why not bundle your enthousiasm+code powersss with the current team and just join them? I think we can use all help!?!
- what does "more productivity-ready" exactly means? If i can work faster Im very interested (!!) :)

Enjoy

[Hugh]

Hi,

Interesting project - I'm keen to see how far you get.

You already have 2 huge benefits over haxe - it is written in haxe, and it's not GPL.  These two things alone should help development and integration respectively.

The current haxe compiler architecture goes a bit like this:

1 parser .hx -> untyped AST

2. untyped macros

3 untyped AST -> typed AST

4. typed macros

5 backend

typed macros may also include built-in optimizer stuff.

I think you can make your project about 10 times easier for yourself by just re-implementing 1 and 2 yourself in haxe.  This gives you 100% control over the language, as long as you can fit your model into the haxe untyped (or fully typed if you are keen) AST.

Then, just use haxe for the rest.  If you do this right, you might also be able to mix .hx files with your own files, which will bootstrap you with all the existing libraries.

I think this can be done today in haxe 3.2 by using haxe.macro.context.onTypeNotFound to open a socket to your compiler and get it to generate the AST, then serialize this back over the socket and insert it into haxe.

This gives you all the existing backends.

Now you can also replace 5 using a similar trick with  haxe.macro.context.onGenerate.  I am seriously thinking about moving or duplicating the hxcpp backend  in haxe using a technique like this.

Lastly, duplicating the type-inference might be tricky to do if you must exactly match haxe, but getting something useful might be easy enough.  With the other components written in haxe, and an adequate typer running sufficiently fast, you might be able to persuade haxe to be self-hosting.

The full stack is a big task, and effort probably goes with some kind of exponential rule.  So if you can break the project into 3 bits, it might be 10 or 20 times easier to get a fully useful result.

Hugh

On Monday, 27 July 2015 06:55:27 UTC+8, PeyTy wrote:

Started to write about language differences. The problems are partially marked too, with samples.
https://github.com/PeyTy/Alternative-Haxe/blob/master/doc/Language_Differences.md

Hello, Mark! I was to emotional that day. So, more unrolled answer:

| What is your drive when you are working on this already for so long!?

This is just a small part of big plan :D
/* Haxe will become a SkyNet and kill humans...hah...joking! */
Just enjoy improving things, and need a solid fundament for future.

| just join them?

To early to say. This project is very different from what HF are doing.

| what does "more productivity-ready" exactly means?

I wrote "production-ready". Not loosing productivity of course :)

- backwards compatibility, no breaking changes (after 2015, I think). Like in Java and AS3
- more updates, continuously, just-in-time
- mainstream simpler syntax+features, so more recruiters will be in future
- more solid, highly maintained built-in targets, and less built-it targets overall
- better flexibility
- 100% enforced documentation
- compiler written in Haxe is easy to maintain for other companies
- complete toolkit (re-factoring as an in-compiler tool, so IDE agnostic)
- etc etc etc

You can read Daniel Glazman`s messages in WWX 2015 Videos. His words made my vision of problem complete.
(Its not meant, that I fully agree with him, and depend on him - I started project long before all these community misunderstandings)

[Marc Weber]

Excerpts from PeyTy's message of Sun Jul 26 22:55:26 +0000 2015:

> Started to write about language differences. The problems are partially marked too, with samples.
> https://github.com/PeyTy/Alternative-Haxe/blob/master/doc/Language_Differences.md

There was a time me and Stephane Le Dorze and someone else thought about
rewriting Haxe. I was missing short lambdas and functional features -
because I also know Haskell - and type classes. There is also disciple
and impredicative.com/ur which introduce different kinds of typing /
abstractions or optimizations (such as memory regions ..)

Thus when rewriting Haxe you must think about your goals.

"Language comparisons" do already exist to some degree
One example is: http://rosettacode.org/wiki/Rosetta_Code

Providing one example: disciple allows typing like this:

fun return_dict():<region x: result of type {a:Int, b:String} which will be
stored in region r>{
retun {a: 2, b: "abc"};
}

this means you can run some code without garbage collector by just by
passing a memory block ..

Example Haskell:

foo: Int
foo = 2

=> The type Int does tell you its not going to have side effects.
somthing like

function foo():Int{
println "somethnig" + sys.readFromFile("foo.txt");
return 2;
}

would be invalid.

Another very important thing is continuation passing style
transformation. To learn about one example have a look at coffescript or
iced script (from node). Eg you can do:

await read_from_file("foo.txt", defer x)
println(x)

where read_from_file takes a callback function - thus it avoids callback
hell to some degree - caring about exceptions and the like is something
I never tested for coffee implementations.

I wrote a patch for short lambdas such as
list.map($1 + 2 ) or hash.map_with_key($1+$2)
which translates to function($1) return $1 + 2 or function($1, $2) return $1 + $2

=> Apple has chosen such for his new language.. My patch that time was
rejected.

Thus eventually its worth thinking about whether introducing some new
features would make sense.

Some features such as "array / list maninpulation" cannot be represented
efficiently in PHP / neko because for neko lists are fastest, for PHP
arrays are fastest - thus lists should be represented as arrays which
might or might not break code in corner cases such as iterating over a
list without finishing the iteration while manipulating it before
iteration has finished.

Thus for "best results" on all targets there might be the need to
introduce new core types ..

Also "Exception iterators" have been shown to be somewhat faster for 5+
elements on most targets (exception PHP again) - thus introducing
typeclasses for iterators could be fun in corner cases..

Thus the question is: In which way to improve and what is the goal.

No doubt - Haxe is cool - but for some cases I prefer dedicated
languages - such as "nokogiri" (ruby) - because it feels so simple.

Thus "loving haxe and its community" is fine to me - depending on the
task to be solved :)

I want to say: Looking at other compilers could lead to wanting more
features - more than just rewriting a lexer or changing the syntax a
little bit :)

Your examples omitting ; is fine - but you may want to learn about
python/haml which both "auto-close" groups of commands/html tags by
reducing indent, eg:
for i in [1,2]:
print i
print j # loop ends, because indentation is equal the start of the for statement

Eg scala allows writing natve xml tags, such as return <foo></foo>.
While I wrote haml like syntax for haxe - the implementation is not
perfect.

So there are lots of "nice things" which could be added to a new
implementation - which are much more complex and cannot be represented
by a simple comparison - this somewhat works for very similar languages
such as "OO" or "scripting".

My advice: Get a Haskell tutorial - try to understand at least some
concepts of impredicative.com/ur tutorial to get an idea that also
things exist which are fundamentally different in nature.

Eg Scala has the concept of "lazy values" which can be useful sometimes
and very short object code such as

class Foo(x:Int){
}

x will be accessible within the class without additionial declaration or
such.

I was missing a lot of such small things in the past. And that's why
"loving Haxe" is fine - but it doesn't hurt to know that there are some
cool stuff outside of haxe to be found IMHO.

If I was wrong about some pieces forgive me, I cannot keep up with all
changes everywhere.

No doubt Haxe is to be loved - and its an extraordinary tool and
exceptional software has been written using it.

Yours
Marc Weber

[Hugh]

Ok, I understand all you are saying.  You are looking at a completely different language, but with similar goals as haxe, not some kind of haxe-only-with-better-syntax.

Hugh 

On Monday, 27 July 2015 11:31:12 UTC+8, PeyTy wrote:

Hello, Hugh! Thank you for sharing!

| it is written in haxe, and it's not GPL

;)

| The current haxe compiler architecture goes a bit like this

My compiler is already different. Macro can work at lexemes level. Parser generates expr and types directly.

| I am seriously thinking about moving or duplicating the hxcpp backend in haxe using a technique like this.

In Alt Haxe generators are macro plug-ins by design (like JSGenAPI). I must mention, because you understand the thing, about JIT optimizing of macro (similar to cppia). So no performance drop here.

| duplicating the type-inference ... exactly match haxe

I dont like some parts of the inference behaviour of Haxe.

| self-hosting

It will be

| The full stack is a big task

I wrote a big answer to you, but when got to this sentence I can finally say:

I AM WRITING EXACTLY A FULL STACK

Writing a compiler doesn’t mean to be an impossible task.

About HXCPP. It may sound to you like there is a competitive tech :) but own C++/LLVM back-end is a core part of new compiler stack.
I dont want to say that your work is bad or something. But Alternative Haxe will be more adopted for native execution by itself and contain LLVM bakend inside, for macro execution and generating binaries. Yeah, not about all those cool c++/objc interop stuff.

But HXCPP CAN be, and it will be cool, one of the backend plug-ins to generate interopable code without FFI!

[Hugh]

For me, the key difference is whether it compiles (a meaningful subset of) existing haxe code or not.

Hugh

On Monday, 27 July 2015 13:22:21 UTC+8, PeyTy wrote:

Hugh, what is haxe-only-with-better-syntax? :)

I am making a haxe-only-with-better-syntax and some features on top.
If not so, I`d named it like "Alternative Yet Another HTML-but-for-programming Language"
:Ъ

You can read my diff - https://github.com/PeyTy/Alternative-Haxe/blob/master/doc/Language_Differences.md
Less quirks. Mostly everything is same.

[Juraj Kirchheim]

This sounds very promising, although too good to be true :D

I'd be very interested to see a first working example, particularly with the lexeme-based macros!

Best,

Juraj

On Mon, Jul 27, 2015 at 7:30 AM, PeyTy <aleg...@gmail.com> wrote:

Hugh
Okay I will write a macro for that, no prob.

(You are right)

[Cristian Baluta]

if you took so many things from swift why not redirecting your efforts there and bring it to haxe? i'm not trying to stop evolution, i just can't see it's usage, i don't understand what is the output of your compiler.

On Mon, Jul 27, 2015 at 8:30 AM, PeyTy <aleg...@gmail.com> wrote:

Hugh
Okay I will write a macro for that, no prob.

(You are right)

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---
You received this message because you are subscribed to the Google Groups "Haxe" group.
For more options, visit https://groups.google.com/d/optout.

[Juraj Kirchheim]

Well, Mahomet did not build his own hill though. A better analogy would be the Tower of Babel :D

I think it's awesome - so don't let us distract you! If you get this to fly, it will speak for itself :)

On Mon, Jul 27, 2015 at 8:18 AM, PeyTy <aleg...@gmail.com> wrote:

Community wants short lambdas for last 5 years. I am not a part of Haxe Foundation. I am a part of this community.

| If the hill will not come to Mahomet, Mahomet will go to the hill.

[Baluta Cristian]

Exactly, people want short lambdas not a new language, you can change haxe to do that and hope it will be merged some day.

Null and string interpolations are exactly like in swift, other features i didn't see

Sent from my iPhone

[Juraj Kirchheim]

Well, I think you've missed the actual point of the endeavor: to have a language where adding features is straight forward. 

Where short lambda and interpolation is just a corollary of the more powerful qualities underpinning the language. 

Power to the people and all that :D

It's a powerful idea. I'm not sure minor syntax differences illustrate it well, though.

[Brennan Kinney]

@Cristian Have you seen Patricks fork of Haxe? It supports short lambda's and much more, if you're not familiar with the features they're covered now and then via Haxe Roundup(Haxe.io). 

@PeyTy With the no semi-colon support, I take it you can run into the Automatic Semicolon Insertion(ASI) issues that you might see in JS/AS3?

I'm having slight trouble understanding what you are doing. 

• You are altering Haxe to work with different syntax/features?
  
• Language/compiler will be in AltHaxe instead of Ocaml?
• Targets such as PHP, JS, C++ are handled via macro?
• Language features/syntax can be added with macro but require '@' such as await becomes @await?
• Core syntax of AltHaxe will be strictly chosen as is the case with Haxe, community can extend via the macro method(but this is better than current macro method with Haxe?)

I'd really like to have a solid language with smart people behind it, but for the community to still be able to extend it, we partially have that working with Haxe macro but is not always able to achieve this. Functionality is fine but cannot introduce syntax/operators such as '=>'. '?.' '??'. Does AltHaxe make it possible to configure language like this? I know normally you'd want a more controlled language as this could lead to fragmentation or different implementations perhaps when handled via community. If you take same stance/view as Nicholas, then when community wants to add to the language, what can be done? We can implement through this different macro approach? Or are there other ways? See the features Patrick demo's with his fork of Haxe, will AltHaxe be flexible to support these additions or would the language need to be modified in a way that my project source code will not run for another user unless they modify(or use fork) AltHaxe too.

I'm not sure if there is something like ESLint/ESFormatter in Haxe. These JS tools can pick up on coding style/conventions and change code formatting, they are very extendable. I believe Go also has a code format tool. Is there anything like this for Haxe or perhaps AltHaxe could support this better?

[Marc Weber]

> To be honest, current macro method is _very_ limited. Saying, for
> example, "you can just use macro to implement short lambdas!" is not
> true.
Its not about "being possible" - its also about "compilation speed".

I had an implementation long time ago, but AFAIK I ran into some
problems or in the end it was too cumbersome:
https://bitbucket.org/haxecommunity/haxe-mw-extensions

The haml like HTML templates can be found here:
https://github.com/MarcWeber/haxe-macro-html-templates

I also found it to get slow - also because I also added more macro code
to derive SPOD class stuff from a "data model" which can be found here:
https://bitbucket.org/haxecommunity/haxe-db-scheme

With all those experiments I found that targeting all targets
(node/neko/...) is cumbersome due to lack of abstraction over
continuations. Same about "trying to write a cross platform API
to handle requests".
https://bitbucket.org/haxecommunity/haxe-cross-request-response
(proof of concept only .. - not ready for production)

=> Thus to truly advance some more abstract way to represent code must
be found so that you can compile down to fast PHP ore node, java or
python code.
Eg the Java backend caused little problem because you have to "catch
Exceptions" ("throws .." syntax)

The node caused "trouble" due to callback -> you have to pass the final
"send stuff to browser methods' by passing a context using callbacks.
That's not a problem - but that's slow when compiling down to PHP ..

When using callbacks how to take care about exceptions?

Using neko/php you can do:

try {
...
} catchall ...

and you can at least tell the end user that something went wrong by
sending internal error message or such.

Using calbacks do you have to wrap each function by try .. catch?
If yes then it might be useful to find a way to do such automatically?

Those are thoughts/constraints/ideas I hit when thinking about "how haxe
could be improved".

I'm unsure I have solutions to all things I mentioned.

Marc Weber

[Brennan Kinney]

Thank you for clarification, your work sounds very exciting :)

In regards to formatting/style guides, I like the ESLint/ESFormatter tools(node packages not IDE). These tools have definition file to override defaults and can extend with 'plugin' rules(each new rule is separate file). With Go format tool I think this is strict style, no user choice. Example ';' rule can be turn on/off with boolean in the tool definition/config file. With good config for tool like this, can prevent consistency issues or problems like ASI which can happen with new developer or outsourced developer. This is just a bonus that would be awesome.

On Monday, 27 July 2015 23:47:32 UTC+12, PeyTy wrote:

@Brennan

| With the no semi-colon support, I take it you can run into the Automatic Semicolon Insertion(ASI) issues that you might see in JS/AS3?

Yeah, I fully understand - it will cause problems. This may be solved by promoting a one style global guide, like here https://github.com/bengourley/js-style-guide
it covers semicolon problem.

| You are altering Haxe to work with different syntax/features?

| Language/compiler will be in AltHaxe instead of Ocaml?

Yes, full selfhosting

| Targets such as PHP, JS, C++ are handled via macro?

JS and CXX (C++ & LLVM) will be built-in. Other are via macro. Like JSGenAPI, but much better and flexible.

| Language features/syntax can be added with macro but require '@' such as await becomes @await?

There is many types of macro.
1. Lexeme preprocessor, for extending syntax itself (without any requirement, only limited to predefined set of tokens)
2. Classic macro functions. I think it must be replaced to @meta
3. @meta macroses. They works like attributes and can call macro-functions, depending on context
4. Macro classes. Its from original Haxe. Have not yet investigated. Never used

This is really not fully finished. The core concept of new macro system to make it "obvious". YellowAfterLife wrote recently that "@metas fail silently, while calling macro function will work always" -> I want to not allow such situations. @metas must be defined somewhere, to be used.

| Core syntax of AltHaxe will be strictly chosen as is the case with Haxe, community can extend via the macro method(but this is better than current macro method with Haxe?)

To be honest, current macro method is _very_ limited. Saying, for example, "you can just use macro to implement short lambdas!" is not true.
In AltHaxe you limited only on token level, not AST. So anything is possible.
Core syntax _of course_ will be strictly chosen. It must be very-very simple, understandable, eye candy, and powerful. But power comes from type system, theorem-solvers (static analyzing), ... and macro.

Dont think that I want to simplify and change syntax _only_ because of these reasons. It will be a really gift for community, maintainers and newcomers.

| '=>'. '?.' '??'. Does AltHaxe make it possible to configure language like this?

Yes. Definitely. New tokens can be proposed, but its more like "just add more ASCII chars to token list" :) And... most of them already here.

| this could lead to fragmentation

Macro system works only on-demand on a per-file or even per-expression basis. Language cannot be fragmented by design.
Someone could write his lib in one dialect, and it can be safely used in a lib with another dialect.

Think of Haxe as of Intermediate form. There is Tardis Go. Believe it or not, it will be possible even to use subset/kindof of Go just like a "yet another .hx file".
I already wrote Hugh about such macro to use Haxe 3.2 subset some messages above.

| I know normally you'd want a more controlled language as this could lead to fragmentation or different implementations perhaps when handled via community. If you take same stance/view as Nicholas, then when community wants to add to the language, what can be done? We can implement through this different macro approach? Or are there other ways? See the features Patrick demo's with his fork of Haxe, will AltHaxe be flexible to support these additions or would the language need to be modified in a way that my project source code will not run for another user unless they modify(or use fork) AltHaxe too.

"Language cannot be fragmented by design."
"Macro system works only on-demand on a per-file or even per-expression basis"
"extending syntax itself (without any requirement, only limited to predefined set of tokens)"

Example: http://hacking-haxe-dev.atouchofcode.com/#0de59
```
import PatrickDialect.apply;   // Syntax will be changed. Maybe `using` fits better, or new keyword... anyway - import a macro first
// The macro is defined inside a some class, like always you do

#apply // enable preprocessor on a whole file basis
// maybe enabling via `using PatrickDialect.apply;` without a #this will be considered. #this syntax is more about "just apply this macro HERE and NOT any other place"
// but really only macro developer decides how his/her macro works

object EMath { // we enabled macro because there is not standard "object" keyword. so we need a lexer-level macro to support it.
    @tco def fact(n, acc=1) return if (n<1) acc else fact(n-1, n * acc); // I just removed `:` so `@:` to `@`. Old meta-informational (non-macro) metas syntax is WIP
}

// def is also implemented in same macro. but say, it is NOT (just for example):

object EMath {
    @tco #def fact(n, acc=1) return if (n<1) acc else fact(n-1, n * acc);
   // def became #def.
   // I `def` is just a shorthand for `function` its fun :)
   // it will just replace current token
   // and CONSIDER -> #macro CAN be nested like `var a = #do #other #another 42;` and WILL NOT conflict (pure magic!)
}

class Test {
    static function main() {
        trace(EMath.fact(12));
        @tco function sum(n, acc=0) {
        // @tco works at AST level.
        // it takes args like @tco(a,b,c) and only second expression after `@tco this_expr;` (whole function node in this case)
            if (n < 0) throw "n can't be negative";
            if (n==0) return acc;
            return sum(n-1, n+acc);
        }
        trace(sum(40));
    }
}
```

As you can see - the solution is solid. I wont tell you all those marketing /***/ if it will not be possible. I was shocked when the solution (implementation) came to my mind!

| I believe Go also has a code format tool

Its already considered to be built-in. IDE must not support all this crazy stuff. Also, I am using Sublime, and it is not even an IDE :)

About original Haxe you may ask HF. I dont know why they didnt do this already.

And...

| stance/view as Nicholas, then when community wants to add to the language, what can be done

There is PLENTY of things to be done. Syntax is just nothing (hah, you can extend it by yourself!).
- Generating native binaries from compiler directly
- Making macro as fast as possible
- Making more unit tests
- Making generated code as optimized as possible
- Making smart error-detectors
- Extending STD
- Writing macro to support externs from another langs (like TypeScript`s .d.ts to be used in Js target)
- Extending toolkit
- etc etc etc

Community must understand this amount of work, and learn some macro magic. And no "Nicholas" problems will be involved.