olmec interpreter coming along slowly

[luserdroog]

As of last night's codeathon the REPL runs and correctly
assigns and retrieves variables and eliminates parens.
While not actually functional yet (no functions), I'm
very pleased with the simplicity and power of the code
I have.
https://github.com/luser-dr00g/inca/tree/master/olmec

Various parts are available for review, all in (not too
obfuscated) C:
http://codereview.stackexchange.com/questions/98838/utf-8-encoding-decoding
http://codereview.stackexchange.com/questions/114243/unicode-capable-symbol-table-n-way-search-tree-with-hash-buckets
http://codereview.stackexchange.com/questions/115748/termios-xterm-line-editor-for-apl-interpreter

scalar encoding (ie. boxing)
clc: "code review: int-code module"
https://groups.google.com/d/topic/comp.lang.c/7NeTMCbl9zM/discussion

the correct algorithms described in the literature
clc: "code review: APL scanner and parser"
https://groups.google.com/d/topic/comp.lang.c/X7BzyIKpogg/discussion

[luserdroog]

Posted the array functions for review.
http://codereview.stackexchange.com/questions/122038/ndarrays-in-c-slicing-transposing-polynomials

That's right. Meat And Potatoes.

--
Can you spot the bug?

[luserdroog]

Changed the keyboard. A small handful of functions
and operators are implemented.

josh@LAPTOP-ILO10OOF ~/inca/olmec
$ !.
./olmec
⎕a
~ ! @ # $ % ^ & * ( ) _ +
` 1 2 3 4 5 6 7 8 9 0 - =
Q W E R T Y U I O P { } |
q w e r t y u i o p [ ] \
A S D F G H J K L : "
a s d f g h j k l ; '
Z X C V B N M < > ?
z x c v b n m , . /

⎕k
· ⌶ ⍫ ⍋ ⍒ ⌽ ⍉ ⊖ ⍟ ⍱ ⍲ ⌸ ⌹
⋄ ¨ ¯ < ≤ = ≥ > ≠ ∨ ∧ × ÷
¿ ⍹ ⍷ ℝ ⍭ ¥ ⌀ ⍸ ⍥ £ ⊣ ⊢ ⍙
? ⍵ ∈ ⍴ ⌾ ↑ ↓ ⍳ ○ ⋆ ← → ⍀
⍶ ⌷ ⍄ ≡ ⍋ ⍒ ⍤ ⍃ ⍞ ⍂ ⌻
⍺ ⌈ ⌊ _ ∇ ∆ ∘ ' ⎕ ⍁ ´
⍧ ⌺ ⍝ ⍦ ⍎ ⍕ ⍔ « » ¶
⊂ ⊃ ∩ ∪ ⊥ ⊤ ⍍ ⍪ ∵ ⌿

'+-×÷⍴⍳/\⍀←,()123abc'
+ - × ÷ ⍴ ⍳ / \ ⍀ ← , ( ) 1 2 3 a b c

[luserdroog]

On Saturday, March 12, 2016 at 12:48:04 AM UTC-6, luserdroog wrote:
> On Sunday, March 6, 2016 at 3:02:42 AM UTC-6, luserdroog wrote:
> > On Friday, February 12, 2016 at 3:55:37 PM UTC-6, luserdroog wrote:
> > > As of last night's codeathon the REPL runs and correctly
> > > assigns and retrieves variables and eliminates parens.
> > > While not actually functional yet (no functions), I'm
> > > very pleased with the simplicity and power of the code
> > > I have.
> > > https://github.com/luser-dr00g/inca/tree/master/olmec
> > >

<snip>

> Changed the keyboard. A small handful of functions
> and operators are implemented.
>
> josh@LAPTOP-ILO10OOF ~/inca/olmec
> $ !.
> ./olmec
> ⎕a
> ~ ! @ # $ % ^ & * ( ) _ +
> ` 1 2 3 4 5 6 7 8 9 0 - =
> Q W E R T Y U I O P { } |
> q w e r t y u i o p [ ] \
> A S D F G H J K L : "
> a s d f g h j k l ; '
> Z X C V B N M < > ?
> z x c v b n m , . /
>
> ⎕k
> · ⌶ ⍫ ⍋ ⍒ ⌽ ⍉ ⊖ ⍟ ⍱ ⍲ ⌸ ⌹
> ⋄ ¨ ¯ < ≤ = ≥ > ≠ ∨ ∧ × ÷
> ¿ ⍹ ⍷ ℝ ⍭ ¥ ⌀ ⍸ ⍥ £ ⊣ ⊢ ⍙
> ? ⍵ ∈ ⍴ ⌾ ↑ ↓ ⍳ ○ ⋆ ← → ⍀
> ⍶ ⌷ ⍄ ≡ ⍋ ⍒ ⍤ ⍃ ⍞ ⍂ ⌻
> ⍺ ⌈ ⌊ _ ∇ ∆ ∘ ' ⎕ ⍁ ´
> ⍧ ⌺ ⍝ ⍦ ⍎ ⍕ ⍔ « » ¶
> ⊂ ⊃ ∩ ∪ ⊥ ⊤ ⍍ ⍪ ∵ ⌿
>
> '+-×÷⍴⍳/\⍀←,()123abc'
> + - × ÷ ⍴ ⍳ / \ ⍀ ← , ( ) 1 2 3 a b c

I've made further changes, but this is mostly
what it still looks like. Of course only the
smaller line at the bottom are actually implemented,
now including conceal and reveal (aka enclose/disclose).

I think this new example is fun, but it made the lexical
analyzer more complicated.

Added 'quad-minus' variable which flip-flops the semantics of high-minus and regular minus.
./olmec
⎕-←1
¯-2-3-4
2 3 4
⎕-←0
-¯2¯3¯4
2 3 4

One infelicity it causes is if a minus is trailing
a sequence of non-numeric *identifier* characters,
and is followed by a period, then a number, the minus
falls off the scanner memory and is left part of
the identifier sequence and not part of the following
intended negative decimal.

Somewhat moot, since fractional values are not yet
implemented at all. But it'll need fixing and futzing
soon.

[luserdroog]

On Friday, February 12, 2016 at 3:55:37 PM UTC-6, luserdroog wrote:

After a few very insightful reviews by Tim Rentsch,
I've reworked most of the scanner and parser code.
And while doing so, I figured out how to define
my parser table so it looks almost exactly like
Laura Palmer, -er. um like the example in IOJ,
http://sblom.github.io/openj-core/iojSent.htm#Parsing

My current version:
https://github.com/luser-dr00g/inca/blob/cad46be90bfcdeff937f688d816924b469444a9d/olmec/exec_private.h#L41

caution. usenet word-wrapping may spoil this. it's a little
wide.

// the Parse Table defines the grammar of the language
// At each stack move, the top four elements of the right stack
// are checked against each of these patterns. A matching pattern
// causes the designated span of elements to be passed to the
// indicated function and the result interleaved back to the
// same position (shifting higher elements down and adjusting
// the top-of-stack pointer).
//
// The table itself is transformed via macro-expansion into
// branches of an if-else chain.
#define PARSE_PRODUCTIONS_FOREACH(_) \
/* p[0] p[1] p[2] p[3] */ \
/*-->items[3] items[2] items[1] items[0] */ \
/* items[start..finish] => func(items[start..finish]) */\
/* func start finish hack */\
_(L0, EDGE, MON, NOUN, ANY, monadic, 2, 1, 0) \
_(L1, EDGE+AVN, VRB, MON, NOUN, monadic, 1, 0, 0) \
_(L2, ANY, NOUN, DEX, ANY, monadic, 1, 2, 0) \
_(L3, EDGE+AVN, NOUN, DYA, NOUN, dyadic, 2, 0, 0) \
_(L4, EDGE+AVN, NOUN+VRB, ADV, ANY, adv, 2, 1, 0) \
_(L5, ANY, LEV, NOUN+VRB, ANY, adv, 1, 2, 0) \
_(L6, EDGE+AVN, NOUN+VRB, CONJ, NOUN+VRB, conj_, 2, 0, 0) \
_(L7, VAR, ASSN, AVN, ANY, spec, 3, 1, 0) \
_(L8, LPAR, ANY, RPAR, ANY, punc, 3, 1, 0) \
_(L9, MARK, ANY, RPAR, ANY, punc, 1, 2, \
stack_push(left,stack_pop(right)) ) \
_(L10,ANY, LPAR, ANY, NUL, punc, 2, 1, 0) \
/**/

[Lobachevsky]

You might want to have a look at the Pratt parser, which was relatively recently popularised by Douglas Crockford of JSON fame. I first saw this in the late 1970s and remember racking up a lot of CPU units on my account.

Crockford's site is http://javascript.crockford.com/.

In my meager opinion, one of the best Pratt parser articles is http://effbot.org/zone/simple-top-down-parsing.htm. The article is written for a Python audience, but the material is applicable for almost any programming language. there are plenty of other good articles, google "pratt parser".

Here's what I learned trying to get the Pratt parser to work with APL like languages:

The Pratt parser is function, not rule, oriented, and the application code should assign precedence levels for functions. For APL, the precedence level of an individual function is its relative position in the line of code, thus functions further right on a line have higher precedence.

The companion lexical analyser could assemble and deliver numeric vector literals i.e. a := 13 42 7 together.

Similarly, the lexical analyser could assemble functions, i.e. deliver "+/", "+.x" etc. as the parser only knows about monadic and dyadic functions. I haven't done user defined operators yet, the lexical analyser would need to have access to rudimentary symbol table information. I haven't tried any of the (ugh) APL2 binding strength examples where the last element of a numeric literal is consumed by the operator immediately to its right.

Bracket indexing x[i;j;k], axis operator +/[42]x, and other syntactic anomalies can be handled with techniques described in the article, not unlike handling formal function arguments in other verbose languages i.e. foo(x, y)

I have not attempted anything like strand notation.

The Pratt parser is very simple and handles expression languages very well. In the end, the code to invoke all the primitives could read as two switch statements, one for all monadic functions ("nud" in the article, null denotation), the other ("led", left denotation) for dyadic functions.

Personally, I prefer "detonation" to "denotation" any day.

[luserdroog]

Apropos of parsing, search just turned up a
relevanr paper. pdf no pay/reg- wall

https://www.researchgate.net/publication/234776477_APL_two_by_two-syntax_analysis_by_pairwise_reduction