neos - the universal compiler that can compile any programming language

[Mr Flibble]

Hi!

neos progress report time!

neos now fully parses my "neoscript" fibonacci example/test program!
Semantic concepts are emitted now all that needs to be done is to add
semantic concept folding and semantic concept bytecode emission so I can
run the program!

Compilation session (scroll to bottom to see test program source code):

neos 1.0.0.0 ED-209
Loading schema 'neoscript.neos'...
Language: Default neoGFX scripting language
Version: 1.0.0
Copyright (C) 2019 Leigh Johnston
emit: <3> language.keyword ()
emit: <4> string.utf8.character.alpha (n)
emit: <5> string.utf8.character.alpha (e)
emit: <5> string.utf8.character.alpha (o)
emit: <5> string.utf8.character.alpha (s)
emit: <5> string.utf8.character.period (.)
emit: <6> string.utf8.character.alpha (s)
emit: <7> string.utf8.character.alpha (t)
emit: <7> string.utf8.character.alpha (r)
emit: <7> string.utf8.character.alpha (i)
emit: <7> string.utf8.character.alpha (n)
emit: <7> string.utf8.character.alpha (g)
emit: <4> module.package.name ()
emit: <3> module.package.import ()
emit: <3> module.package.instantiate ()
emit: <3> language.keyword ()
emit: <4> string.utf8.character.alpha (n)
emit: <5> string.utf8.character.alpha (e)
emit: <5> string.utf8.character.alpha (o)
emit: <5> string.utf8.character.alpha (s)
emit: <5> string.utf8.character.period (.)
emit: <6> string.utf8.character.alpha (s)
emit: <7> string.utf8.character.alpha (t)
emit: <7> string.utf8.character.alpha (r)
emit: <7> string.utf8.character.alpha (e)
emit: <7> string.utf8.character.alpha (a)
emit: <7> string.utf8.character.alpha (m)
emit: <4> module.package.name ()
emit: <3> module.package.import ()
emit: <3> module.package.instantiate ()
emit: <3> language.keyword ()
emit: <4> language.keyword ()
emit: <5> string.utf8.character.alpha (t)
emit: <6> string.utf8.character.alpha (o)
emit: <6> string.utf8.character.underscore (_)
emit: <6> string.utf8.character.alpha (s)
emit: <6> string.utf8.character.alpha (t)
emit: <6> string.utf8.character.alpha (r)
emit: <6> string.utf8.character.alpha (i)
emit: <6> string.utf8.character.alpha (n)
emit: <6> string.utf8.character.alpha (g)
emit: <5> language.identifier ()
emit: <10> string.utf8.character.alpha (x)
emit: <10> language.identifier ()
emit: <9> language.function.parameter ()
emit: <11> language.type.i32 ()
emit: <7> language.function.parameters ()
emit: <10> language.type.string ()
emit: <9> language.function.return ()
emit: <4> language.function.signature ()
emit: <4> language.function.import ()
emit: <3> language.keyword ()
emit: <4> language.keyword ()
emit: <5> string.utf8.character.alpha (t)
emit: <6> string.utf8.character.alpha (o)
emit: <6> string.utf8.character.underscore (_)
emit: <6> string.utf8.character.alpha (i)
emit: <6> string.utf8.character.alpha (n)
emit: <6> string.utf8.character.alpha (t)
emit: <6> string.utf8.character.alpha (e)
emit: <6> string.utf8.character.alpha (g)
emit: <6> string.utf8.character.alpha (e)
emit: <6> string.utf8.character.alpha (r)
emit: <5> language.identifier ()
emit: <10> string.utf8.character.alpha (s)
emit: <10> language.identifier ()
emit: <9> language.function.parameter ()
emit: <11> language.type.string ()
emit: <7> language.function.parameters ()
emit: <10> language.type.i32 ()
emit: <9> language.function.return ()
emit: <4> language.function.signature ()
emit: <4> language.function.import ()
emit: <3> language.keyword ()
emit: <4> language.keyword ()
emit: <5> string.utf8.character.alpha (i)
emit: <6> string.utf8.character.alpha (n)
emit: <6> string.utf8.character.alpha (p)
emit: <6> string.utf8.character.alpha (u)
emit: <6> string.utf8.character.alpha (t)
emit: <5> language.identifier ()
emit: <10> string.utf8.character.alpha (s)
emit: <10> language.identifier ()
emit: <9> language.function.parameter ()
emit: <11> language.keyword ()
emit: <10> language.function.parameter.direction.out ()
emit: <12> language.type.string ()
emit: <7> language.function.parameters ()
emit: <4> language.function.signature ()
emit: <4> language.function.import ()
emit: <3> language.keyword ()
emit: <4> language.keyword ()
emit: <5> string.utf8.character.alpha (p)
emit: <6> string.utf8.character.alpha (r)
emit: <6> string.utf8.character.alpha (i)
emit: <6> string.utf8.character.alpha (n)
emit: <6> string.utf8.character.alpha (t)
emit: <5> language.identifier ()
emit: <10> string.utf8.character.alpha (s)
emit: <10> language.identifier ()
emit: <9> language.function.parameter ()
emit: <11> language.keyword ()
emit: <10> language.function.parameter.direction.in ()
emit: <12> language.type.string ()
emit: <7> language.function.parameters ()
emit: <4> language.function.signature ()
emit: <4> language.function.import ()
emit: <3> language.keyword ()
emit: <4> language.keyword ()
emit: <5> string.utf8.character.alpha (a)
emit: <6> string.utf8.character.alpha (d)
emit: <6> string.utf8.character.alpha (d)
emit: <5> language.identifier ()
emit: <10> string.utf8.character.alpha (x)
emit: <10> language.identifier ()
emit: <9> language.function.parameter ()
emit: <12> string.utf8.character.alpha (y)
emit: <12> language.identifier ()
emit: <11> language.function.parameter ()
emit: <13> language.type.i32 ()
emit: <7> language.function.parameters ()
emit: <10> language.type.i32 ()
emit: <9> language.function.return ()
emit: <4> language.function.signature ()
emit: <8> language.keyword ()
emit: <13> string.utf8.character.alpha (x)
emit: <13> language.identifier ()
emit: <12> math.expression.operand ()
emit: <15> string.utf8.character.alpha (y)
emit: <15> language.identifier ()
emit: <14> math.expression.operand ()
emit: <10> math.operator.add ()
emit: <9> math.expression ()
emit: <9> language.function.return ()
emit: <7> language.statement ()
emit: <5> language.function.scope ()
emit: <4> language.function ()
emit: <3> language.keyword ()
emit: <4> language.keyword ()
emit: <5> string.utf8.character.alpha (f)
emit: <6> string.utf8.character.alpha (i)
emit: <6> string.utf8.character.alpha (b)
emit: <5> language.identifier ()
emit: <10> string.utf8.character.alpha (x)
emit: <10> language.identifier ()
emit: <9> language.function.parameter ()
emit: <11> language.type.i32 ()
emit: <7> language.function.parameters ()
emit: <10> language.type.i32 ()
emit: <9> language.function.return ()
emit: <4> language.function.signature ()
emit: <8> language.keyword ()
emit: <17> string.utf8.character.alpha (x)
emit: <17> language.identifier ()
emit: <16> math.expression.operand ()
emit: <13> math.expression ()
emit: <12> boolean.expression.operand ()
emit: <18> math.universal.number.digit (1)
emit: <18> math.universal.number ()
emit: <18> math.expression.operand ()
emit: <15> math.expression ()
emit: <14> boolean.expression.operand ()
emit: <12> boolean.operator.relational.equal ()
emit: <10> boolean.expression ()
emit: <13> language.keyword ()
emit: <17> math.universal.number.digit (1)
emit: <17> math.universal.number ()
emit: <17> math.expression.operand ()
emit: <14> math.expression ()
emit: <14> language.function.return ()
emit: <12> language.statement ()
emit: <11> logic.operator.if ()
emit: <14> language.keyword ()
emit: <16> language.keyword ()
emit: <21> string.utf8.character.alpha (a)
emit: <22> string.utf8.character.alpha (d)
emit: <22> string.utf8.character.alpha (d)
emit: <21> language.identifier ()
emit: <27> string.utf8.character.alpha (f)
emit: <28> string.utf8.character.alpha (i)
emit: <28> string.utf8.character.alpha (b)
emit: <27> language.identifier ()
emit: <33> string.utf8.character.alpha (x)
emit: <33> language.identifier ()
emit: <32> math.expression.operand ()
emit: <34> math.universal.number.digit (1)
emit: <34> math.universal.number ()
emit: <34> math.expression.operand ()
emit: <30> math.operator.subtract ()
emit: <29> math.expression ()
emit: <28> language.function.argument ()
emit: <29> language.function.call ()
emit: <26> math.expression.operand ()
emit: <23> math.expression ()
emit: <22> language.function.argument ()
emit: <29> string.utf8.character.alpha (f)
emit: <30> string.utf8.character.alpha (i)
emit: <30> string.utf8.character.alpha (b)
emit: <29> language.identifier ()
emit: <35> string.utf8.character.alpha (x)
emit: <35> language.identifier ()
emit: <34> math.expression.operand ()
emit: <36> math.universal.number.digit (2)
emit: <36> math.universal.number ()
emit: <36> math.expression.operand ()
emit: <32> math.operator.subtract ()
emit: <31> math.expression ()
emit: <30> language.function.argument ()
emit: <31> language.function.call ()
emit: <28> math.expression.operand ()
emit: <25> math.expression ()
emit: <24> language.function.argument ()
emit: <25> language.function.call ()
emit: <20> math.expression.operand ()
emit: <17> math.expression ()
emit: <17> language.function.return ()
emit: <15> language.statement ()
emit: <14> logic.operator.else ()
emit: <7> language.statement ()
emit: <5> language.function.scope ()
emit: <4> language.function ()
emit: <3> language.keyword ()
emit: <4> language.keyword ()
emit: <5> string.utf8.character.alpha (m)
emit: <6> string.utf8.character.alpha (a)
emit: <6> string.utf8.character.alpha (i)
emit: <6> string.utf8.character.alpha (n)
emit: <5> language.identifier ()
emit: <4> language.function.signature ()
emit: <8> string.utf8.character.alpha (s)
emit: <8> language.identifier ()
emit: <7> language.function.local ()
emit: <9> language.type.string ()
emit: <5> language.function.locals ()
emit: <5> language.function.locals ()
emit: <13> string.utf8.character.alpha (i)
emit: <14> string.utf8.character.alpha (n)
emit: <14> string.utf8.character.alpha (p)
emit: <14> string.utf8.character.alpha (u)
emit: <14> string.utf8.character.alpha (t)
emit: <13> language.identifier ()
emit: <19> string.utf8.character.alpha (s)
emit: <19> language.identifier ()
emit: <18> math.expression.operand ()
emit: <15> math.expression ()
emit: <14> language.function.argument ()
emit: <15> language.function.call ()
emit: <12> math.expression.operand ()
emit: <9> math.expression ()
emit: <8> language.statement ()
emit: <9> language.function.return ()
emit: <8> language.statement ()
emit: <13> string.utf8.character.alpha (p)
emit: <14> string.utf8.character.alpha (r)
emit: <14> string.utf8.character.alpha (i)
emit: <14> string.utf8.character.alpha (n)
emit: <14> string.utf8.character.alpha (t)
emit: <13> language.identifier ()
emit: <19> string.utf8.character.alpha (s)
emit: <19> language.identifier ()
emit: <18> math.expression.operand ()
emit: <22> string.utf8.character (:)
emit: <22> string.utf8.character ( )
emit: <21> string.utf8 ()
emit: <20> math.expression.operand ()
emit: <16> math.operator.add ()
emit: <23> string.utf8.character.alpha (t)
emit: <24> string.utf8.character.alpha (o)
emit: <24> string.utf8.character.underscore (_)
emit: <24> string.utf8.character.alpha (s)
emit: <24> string.utf8.character.alpha (t)
emit: <24> string.utf8.character.alpha (r)
emit: <24> string.utf8.character.alpha (i)
emit: <24> string.utf8.character.alpha (n)
emit: <24> string.utf8.character.alpha (g)
emit: <23> language.identifier ()
emit: <29> string.utf8.character.alpha (f)
emit: <30> string.utf8.character.alpha (i)
emit: <30> string.utf8.character.alpha (b)
emit: <29> language.identifier ()
emit: <35> string.utf8.character.alpha (t)
emit: <36> string.utf8.character.alpha (o)
emit: <36> string.utf8.character.underscore (_)
emit: <36> string.utf8.character.alpha (i)
emit: <36> string.utf8.character.alpha (n)
emit: <36> string.utf8.character.alpha (t)
emit: <36> string.utf8.character.alpha (e)
emit: <36> string.utf8.character.alpha (g)
emit: <36> string.utf8.character.alpha (e)
emit: <36> string.utf8.character.alpha (r)
emit: <35> language.identifier ()
emit: <41> string.utf8.character.alpha (s)
emit: <41> language.identifier ()
emit: <40> math.expression.operand ()
emit: <37> math.expression ()
emit: <36> language.function.argument ()
emit: <37> language.function.call ()
emit: <34> math.expression.operand ()
emit: <31> math.expression ()
emit: <30> language.function.argument ()
emit: <31> language.function.call ()
emit: <28> math.expression.operand ()
emit: <25> math.expression ()
emit: <24> language.function.argument ()
emit: <25> language.function.call ()
emit: <22> math.expression.operand ()
emit: <18> math.operator.add ()
emit: <15> math.expression ()
emit: <14> language.function.argument ()
emit: <15> language.function.call ()
emit: <12> math.expression.operand ()
emit: <9> math.expression ()
emit: <8> language.statement ()
emit: <9> language.function.return ()
emit: <8> language.statement ()
emit: <6> language.function.scope ()
emit: <5> language.function ()
Compilation time: 326.358ms
neoscript]cte 0
neoscript]l examples/neoscript/fibonacci.neo
Compilation time: 31.001ms
neoscript]list
-- neoscript example: Fibonacci

using neos.string;
using neos.stream;

import fn to_string(x : i32) -> string;
import fn to_integer(s : string) -> i32;
import proc input(s : out string);
import proc print(s : in string);

-- functions are pure
def fn add(x, y : i32) -> i32
{
return x + y;
}
def fn fib(x : i32) -> i32
{
if (x = 1)
return 1;
else
return add(fib(x-1), fib(x-2));
}

-- procedures are impure
def proc main()
s : string;
{
input(s);
print(s + ": " + to_string(fib(to_integer(s))));
}
neoscript]

I am quite happy with a compilation speed of 30ms for this but there are
no doubt further optimizations I can make.

/Flibble

--
“You won’t burn in hell. But be nice anyway.” – Ricky Gervais

“I see Atheists are fighting and killing each other again, over who
doesn’t believe in any God the most. Oh, no..wait.. that never happens.” –
Ricky Gervais

"Suppose it's all true, and you walk up to the pearly gates, and are
confronted by God," Bryne asked on his show The Meaning of Life. "What
will Stephen Fry say to him, her, or it?"
"I'd say, bone cancer in children? What's that about?" Fry replied.
"How dare you? How dare you create a world to which there is such misery
that is not our fault. It's not right, it's utterly, utterly evil."
"Why should I respect a capricious, mean-minded, stupid God who creates a
world that is so full of injustice and pain. That's what I would say."

[Bart]

On 21/05/2019 23:46, Mr Flibble wrote:
> Hi!
>
> neos progress report time!
>
> neos now fully parses my "neoscript" fibonacci example/test program!
> Semantic concepts are emitted now all that needs to be done is to add
> semantic concept folding and semantic concept bytecode emission so I can
> run the program!
>
> Compilation session (scroll to bottom to see test program source code):
>
> neos 1.0.0.0 ED-209
> Loading schema 'neoscript.neos'...
> Language: Default neoGFX scripting language
> Version: 1.0.0
> Copyright (C) 2019 Leigh Johnston
> emit: <3> language.keyword ()
> emit: <4> string.utf8.character.alpha (n)

This program is too small to accurately measure compilation speed (and
most of it is likely to be all that verbose output if that is part of
the process).

I have a simple test which can give an idea of how a compiler copes with
lots of lines containing the same thing:

https://github.com/sal55/qx/blob/master/compilertest.txt

You need to provide your own input, which at minimum will be 20,000
lines of 'a=b+c+d;' within a suitable framework, eg. a function with
declarations for a,b,c,d.

That's if you wish to compare with my own figures for other compilers,
although those are specific to my machine. But you'll have to disable
that logging output.

If 20K lines is too much, then start much smaller. (One highly recursive
'parser-combinator' someone posted on c.l.c, just about managed 5 lines;
with 6, it exhausted the 8GB memory in my machine.)

With some compilers, this test is unsuitable because having that much
code in one function gives problems, because of the way it's implemented.

[Mr Flibble]

The 326 ms is the time to compile the same program but with displaying all
those emit lines; the 30 ms is without.

>
> I have a simple test which can give an idea of how a compiler copes with
> lots of lines containing the same thing:
>
>   https://github.com/sal55/qx/blob/master/compilertest.txt
>
> You need to provide your own input, which at minimum will be 20,000 lines
> of 'a=b+c+d;' within a suitable framework, eg. a function with
> declarations for a,b,c,d.
>
> That's if you wish to compare with my own figures for other compilers,
> although those are specific to my machine. But you'll have to disable that
> logging output.
>
> If 20K lines is too much, then start much smaller. (One highly recursive
> 'parser-combinator' someone posted on c.l.c, just about managed 5 lines;
> with 6, it exhausted the 8GB memory in my machine.)
>
> With some compilers, this test is unsuitable because having that much code
> in one function gives problems, because of the way it's implemented.

Thanks for the info but I will be putting neos through several
optimization rounds and will include a proper test suite with test
programs are various sizes/complexities.

You might be interested to know that neos parses using backtracking with
memoization.

[Ike Naar]

On 2019-05-21, Mr Flibble <flibbleREM...@i42.co.uk> wrote:
> def fn fib(x : i32) -> i32
> {
> if (x = 1)
> return 1;
> else
> return add(fib(x-1), fib(x-2));
> }

What will fib(2) do?

[Mr Flibble]

I could lie and say that this is a deliberate error to test the neos VM
stack fault handler but I won't. :D

Corrected:

-- neoscript example: Fibonacci

using neos.string;
using neos.stream;

import fn to_string(x : i32) -> string;
import fn to_integer(s : string) -> i32;
import proc input(s : out string);
import proc print(s : in string);

-- functions are pure
def fn add(x, y : i32) -> i32
{
return x + y;
}

def fn fib(x : i32) -> i32
{

if (x < 2)

return 1;
else
return add(fib(x-1), fib(x-2));
}

-- procedures are impure
def proc main()
s : string;
{

print("Enter a positive integer: ");
input(s);
print("Fibonacci(" + s + "): " + to_string(fib(to_integer(s))) + "\n");