[erlang-questions] Frames proposal

[Richard O'Keefe]

There is a new version of
http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf
up. This one has some encouraging experimental results.

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[José Valim]

Thanks Richard for your work and the article. It was a good read!

Given the current good results, what is the likelihood we will see this feature in R16?

José Valim

www.plataformatec.com.br

Founder and Lead Developer

[Tim Watson]

Some assistance with converting the proposal to markdown format perhaps....?

[Max Lapshin]

Maybe it possible to add JSON notation to frame handling?

It would be very convenient, because {id : 4135, title : "title2"} is
mapped to [{id, 4135}, {title, "title2"}].
The only difference is that in all external format handlers binaries
are used to represent external strings.

[Richard O'Keefe]

On 1/05/2012, at 5:13 AM, Max Lapshin wrote:

> Maybe it possible to add JSON notation to frame handling?
>
> It would be very convenient, because {id : 4135, title : "title2"} is
> mapped to [{id, 4135}, {title, "title2"}].
> The only difference is that in all external format handlers binaries
> are used to represent external strings.

I don't quite understand this question.
There is a section in the frames proposal titled "Frames and JSON"
which mainly just presents a table

JSON Erlang
null null
false false
true true
numbers numbers, but JSON has no integers
strings binaries
arrays lists or tuples
dictionaries frames

The implied mapping takes JSON {id: 4135, title: "title2"}
to <{id ~ 4135, title ~ <<"title2">>}>.

The bounded size of the Erlang atom table is a vulnerability
but there is an EEP to address that; that in itself is a much
more urgent issue than frames.

[Max Lapshin]

I mean using JSON directly inside Erlang:


function({type : <<"article">>, title : Title, id : Id} = Object) ->
Object{id : make_permalink(Id, Title)}.

or

Article = {id : 523, title : proplists:get_value(<<"title">>, Params)}

I mean this. Your syntax may have some historical roots, but they are
too ancient. Nowadays such syntax <{key ~ value>} look like inventing
bicycle with square wheels.

[Richard O'Keefe]

On 1/05/2012, at 3:38 PM, Max Lapshin wrote:

> I mean using JSON directly inside Erlang:
>
>
> function({type : <<"article">>, title : Title, id : Id} = Object) ->
> Object{id : make_permalink(Id, Title)}.
>
> or
>
> Article = {id : 523, title : proplists:get_value(<<"title">>, Params)}
>
> I mean this. Your syntax may have some historical roots, but they are
> too ancient. Nowadays such syntax <{key ~ value>} look like inventing
> bicycle with square wheels.

JSON syntax is ***Javascript*** syntax.

The frames proposal has always made it very clear why we cannot
copy JSON syntax.

{} is already an empty TUPLE,
it cannot also be an empty 'dictionary'.
We cannot reasonably use unadorned curly
braces for frames.

{a:f()} already means a tuple whose one element
is the value of a call to the f() function
in module a. We *CANNOT* even unreasonably
use a colon in maplets; it is just not going
to work.

Have a lollipop.

[Max Lapshin]

On Tue, May 1, 2012 at 9:27 AM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
>
> {}      is already an empty TUPLE,

> {a:f()} already means a tuple whose one element
>        is the value of a call to the f() function

Yes, I see. It is a pity =(

[Michael Turner]

Max Lapshin: "Your syntax may have some historical roots, but they are
too ancient."

"Ancient" always trumps "unworkable."

Max again: "Nowadays such syntax <{key ~ value>} look like inventing
bicycle with square wheels."

Bzzt. Analogy fail. Whatever a bicycle with square wheels might *look* like

http://3.bp.blogspot.com/_uhH7HeUdvmw/TB7l0F7aGCI/AAAAAAAAACU/lx6z8b6g-XM/s1600/square_wheels.jpg

we can agree that such a bicycle couldn't work -- well, except perhaps
in a special environment

http://www.math.hmc.edu/funfacts/figures/10001.2-3-8.1.gif

in which square wheels are actually ideal.

If there's any parallel to the ideal-square-wheel special environment,
here, it's existing Erlang syntax, which is so weird and annoying that
even some ardent fans of the language can't refrain from open
criticism. Richard is looking for what would be the "smoothest ride"
given what can't be changed. I think he's made the best of some
not-very-satisfactory choices. And it's not like he and I exactly
agree on syntax issues (but let's not go there again. Ever.)

Richard: "Have a lollipop."

Curmudgeon points for that one.

This proposal is almost 10 years old. Is it really so radical that we
couldn't have it in beta within three months?

-michael turner

[José Valim]

The implied mapping takes JSON {id: 4135, title: "title2"}
to <{id ~ 4135, title ~ <<"title2">>}>.

The bounded size of the Erlang atom table is a vulnerability
but there is an EEP to address that; that in itself is a much
more urgent issue than frames.

Exactly. In Ruby, for example, since atoms aren't garbage collected, converting a JSON from an external source to a hash using atoms as keys represents a security vulnerability in a web service, as someone could force the "atom table" to fill in completely, so we simply don't.

So until the atom limitation is fixed, we would be better on handling JSONs as a dict or something else.

I have read the proposal completely and I think everything is well explained and defined. Even though I am not a huge fan of the syntax, I think it fits Erlang well. One option that I haven't seen considered is still using curly brackets as delimiters and use `{~}` to specify an empty frame. This would make the common case (a frame with at least one element) easier on the eyes by sacrificing a bit the not so common case (empty frame). But again, we are in good hands with whatever the Erlang and OTP team decide.

[Max Bourinov]

Hi Richard,

Thank you very much for posting your great work!

But the title scared me. "Getting rid of records" - what should I do with all my code that uses records?

I like the idea of frames a lot. It would significantly simplify my code and I love this idea.

Best regards,

Max

[Michael Turner]

> But the title scared me. "Getting rid of records" - what should I do with
> all my code that uses records?

EEP stands for Erlang Extension Proposal, not Erlang Excision Proposal.

-michael turner

[Juan Jose Comellas]

And what about borrowing the syntax that Anton Lavrik used for erlson [1]? That syntax looks similar to the one used for records and is much easier to read/write.

[1] https://github.com/alavrik/erlson

[Max Lapshin]

I really dream about plain and simple 0 = D1.baz.fum.i, =)

[Björn-Egil Dahlberg]

2012/5/1 Juan Jose Comellas <jua...@comellas.org>

And what about borrowing the syntax that Anton Lavrik used for erlson [1]? That syntax looks similar to the one used for records and is much easier to read/write.

[1] https://github.com/alavrik/erlson

Syntax is more than just finding which characters our parser can identify, like in the frames proposals where ~ is our separator. When I read <{ k ~ value | Is }>  it says cons "k is like a value" to Is. *brr*

At first I didn't like the syntax at all, but after playing around with it for a couple of hours it kind of grows on you. #{ } vs <{ }>, the second one only uses a single character more. I think we could live with that one. I still don't like ~ .. and I definitely don't like lollipops.

Lavriks syntax proposal is very clean and seems like an obvious fit to the current Erlang syntax. However, if I read #{ k = value }, and I know how R#{k = value } works, it feels like the two should have some sort of connection. They would't have one. It is entirely possible we are fooling ourselves with a similar syntax and would be better off with a completely different syntax.

Syntax is perhaps the most fun to discuss since it is our interface. It is important to get it right since it hard to change it once implemented. But, It is of minor concern to me.

What guarantees should we leave? The underlying implementation of a Hashmap has enormous consequences on the guarantees we can promise. Speed, memory consumption and functionality are all trade offs. From a memory / speed perspective I really like a HAMT-implementation. Very compact, simple garbage collector, persistent and all around nice properties (except ordering). Doing something more fancy complicates the gc which is not unsolvable but irritating.

Richard, is the "beam"-implementation for frames around to look at? 

// Björn-Egil

[Ulf Wiger]

On 1 May 2012, at 19:19, Björn-Egil Dahlberg wrote:

It is entirely possible we are fooling ourselves with a similar syntax and would be better off with a completely different syntax.

I'm in the camp of

- similar syntax indicates similar semantics

- different syntax indicates different semantics

…or something to that effect. Most importantly, I definitely agree that borrowing a syntax just because it's well-established, is a very bad idea if your intention is to give it different semantics from what one would expect.

BR,

Ulf W

Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc.

http://feuerlabs.com

[Richard O'Keefe]

On 1/05/2012, at 7:37 PM, José Valim wrote:
> I have read the proposal completely and I think everything is well explained and defined. Even though I am not a huge fan of the syntax, I think it fits Erlang well. One option that I haven't seen considered is still using curly brackets as delimiters and use `{~}` to specify an empty frame. This would make the common case (a frame with at least one element) easier on the eyes by sacrificing a bit the not so common case (empty frame). But again, we are in good hands with whatever the Erlang and OTP team decide.

The relevant concept here is *uniformity*: are all cases of a construct
generated by the *same* rule(s)? {a~1, b~2}, {a~1}, and {~} are *not*
generated by the same rule. The bare ~ in {~} has no counterpart in the
others. As I understand it, Joe Armstrong's current proposed syntax is

'#{' [maplet {, maplet}...] '}'

so you would have #{a~1, b~2}, #{a~1}, and #{} (using ~ for maplets
where Joe would actually use =). I prefer symmetric fences, not least
because it helps tools to help me get my brackets to match properly.
But I could live with Joe's syntax.

I have been toying with the idea of a revision to my proposal, namely
that
<open> <maplet> {',' <maplet>}... '|' <expression> <close>
should only be allowed to replace slots that already exist in the
base frame, and not to add new slots. Functional support for adding
new slots would remain, but this way we would *know* that the new
frame could always share the descriptor of the old one, and it would
provide a "safety" assurance to the programmer: if you *intend* to
write 'a copy of F except that the ~count slot is zero'
then <{ count ~ 0 | F }> would faithfully check your assumption that
F _has_ a count slot. This is just like the way records are now, after
all.

[Richard O'Keefe]

On 2/05/2012, at 2:28 AM, Max Bourinov wrote:

> Hi Richard,
>
> Thank you very much for posting your great work!
>
> But the title scared me. "Getting rid of records" - what should I do with all my code that uses records?
>
> I like the idea of frames a lot. It would significantly simplify my code and I love this idea.
>

There are several answers to 'what should I do'.

(1) If it ain't broke, don't fix it.
Records are NOT going to go away.

There's an analogy from Fortran. Fortran 66 had so-called
'Hollerith literals', where you would write e.g., 4HFRED
to get a string "FRED". That's right, the length as a
decimal integer, the letter capital H, and then that many
characters. With the advent of proper character data in
Fortran 77, Hollerith literals were not needed any more.
They had always been clumsy and error-prone. But they
stayed. I can't remember if they were still there in
Fortran 90 or not, but by Fortran 95 they were gone.
Well, I just tried a Hollerith literal in a tiny program
and while the Fortran 95 compiler *complained*, the
program *worked*.

The Erlang community isn't quite _that_ committed to
backwards compatibility, but it's not too far off...

(2) Frames are almost drop-in replacements for records.
The major exception is default initial values for fields,
which have problems anyway.

Years ago I wrote a paper 'Delenda est preprocessor'.
Abstract patterns and frames are all part of a long-time
project to make the preprocessor unnecessary.


But, sigh, it will probably never go away.

[Richard O'Keefe]

On 2/05/2012, at 3:48 AM, Juan Jose Comellas wrote:

> And what about borrowing the syntax that Anton Lavrik used for erlson [1]? That syntax looks similar to the one used for records and is much easier to read/write.

The frames proposal was written long before erlson was a twinkle in its author's
eye. Indeed, it was originally drafted in the year that JSON was invented (2001),
so JSON was not an influence.

The frames proposal explains why using '=' in maplets is a bad idea.
Basically, it is HARDER to read because it makes '=' locally ambiguous.
You go searching through a file and hit the line

foo = bar(),

and you have NO IDEA whether this is a maplet or a matching. BAD!

As for #{...} vs <{...}>, it's in one way harder to write
(to get <{, press SHIFT with left hand, and then < and { are close
for two-fingered right hand typing; the same is true of }>,
but #{ requires me to hold down the shift key with my left hand
AND press the # key with my left hand at the same time, and while
there is a shift key on the right, { requires the use of the right
hand). Brackets really ought to be mirror symmetric, but I could
live with #{...}.

There's a section in the proposal titled
"The Colour of the paint on the Ark."
why do we find it easier to argue about the colour
of the paint on the Ark than to climb aboard?
That is the section that provides the reasons for the syntax.

Now Jan Burse and I have been arguing with each other about frames,
but actually, that was a *lot* more useful, because he focussed on
semantics.

But here's the most important thing, which appears in the next draft
of the frames proposal:

This would have been a workable syntax for frames
before erlson used it. Now, it is important that the syntax
should be distinct from erlson syntax, so that frames can be
added to Erlang without breaking erlson. There is also an
important pragmatic difference between frames and erlson.
Frames are new built in data type with a good deal of thought
put into making them economical in memory and time.
``At runtime, Erlson dictionaries are represented as a list
of {Name, Value} tuples ordered by Name. This way, each Erlson
dictionary is a valid proplist and orddict in terms of the
correspond[ing] stdlib modules.''
An entry in an Erlson dictionary thus takes 4 words for the
{Name,Value} tuple plus 2 words for the list cell pointing to
it, or 6 words compared with a frame's 1 or 2.
It therefore matters which one you are using: a programmer
needs to be able to tell at a glance whether an expression
deals with a frame or with an erlson dictionary.

That is, we shouldn't copy Erlson syntax, BECAUSE Erlson IS ALREADY
USING IT and we must not break Erlson.

[Loïc Hoguin]

On 05/02/2012 02:22 AM, Richard O'Keefe wrote:
> As for #{...} vs<{...}>, it's in one way harder to write
> (to get<{, press SHIFT with left hand, and then< and { are close
> for two-fingered right hand typing; the same is true of }>,
> but #{ requires me to hold down the shift key with my left hand
> AND press the # key with my left hand at the same time, and while
> there is a shift key on the right, { requires the use of the right
> hand). Brackets really ought to be mirror symmetric, but I could
> live with #{...}.

Only true for a range of QWERTY keyboards. There's a lot more layouts in
the world. Incidentaly on my keyboard writing <{}> is insanely difficult
(takes a minimum of 2 seconds when trying to go fast), while #{} is not
perfect but a lot easier (maybe half a second, hard to measure at this
point). This is because the former requires me to press the sequence "<,
ALT GR {, ALT GR }, SHIFT >", while the latter only requires "ALT GR #,
ALT GR {, ALT GR }". The keys for # and { are also right next to the
other, which isn't true for < and {.

So in my case at least it's not only 1 key less, it's also a lot simpler
to type.

It's also a lot simpler to type = instead of ALT GR ~, regardless of
other considerations.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines

[Richard O'Keefe]

It would be *so* nice to discuss semantics...

> Only true for a range of QWERTY keyboards. There's a lot more layouts in the world.

Indeed. It is impossible to optimise for all of them.

> Incidentaly on my keyboard writing <{}> is insanely difficult (takes a minimum of 2 seconds when trying to go fast), while #{} is not perfect but a lot easier (maybe half a second, hard to measure at this point). This is because the former requires me to press the sequence "<, ALT GR {, ALT GR }, SHIFT >", while the latter only requires "ALT GR #, ALT GR {, ALT GR }". The keys for # and { are also right next to the other, which isn't true for < and {.

You don't say which keyboard. The frames proposal document notes a similar
issue for Swedish keyboards, and suggests allowing *both* <{...}> *and* <(...)>
as a way of coping with the problem.

However, since the problem with <{...}> appears to be due to the use of
the curly braces, surely this must make tuples and records already 'insanely
difficult'.

As for the difficulty of typing ~, code is READ more often than it is WRITTEN,
so the readability advantage much outweighs the typing disadvantage.

Modern text editors have abbreviation features which mean that you *should* be
able to set your text editor up so that
(, expands to <{}> with the cursor in the middle
=, expands to ~ with the cursor after it
so that typing "<{a ~ 1, b ~ 2}>" requires the keystrokes
"(,a=,1, b=,2<fwd><fwd>"

It took me five minutes to write this code for my home-brew text editor:

void comma(void) {
if (argvalue != implicit) {
putin();
} else
switch (fetch(here)) {
case ')': /* For Erlang expressions */
case ']': /* For Erlang lists */
here++;
break;
case '}': /* For Erlang tuples and frames */
case '>': /* For Erlang binaries */
here += fetch(here+1) == '>' ? 2 : 1;
break;
default:
switch (fprev(here)) {
case '(': /* For Erlang frames */
delete(NORM, here-1);
strinsert(1, 4, "<{}>");
here -= 2;
break;
case '[': /* For Erlang tuples */
delete(NORM, here-1);
strinsert(1, 2, "{}");
here--;
break;
case ':': /* For Erlang records */
delete(NORM, here-1);
insert(1, '#');
break;
case '=': /* For Erlang frames */
delete(NORM, here-1);
strinsert(1, 3, " ~ ");
break;
case ' ':
here--;
insert(-1, ',');
here++;
break;
default:
putin();
}
}
}

After which it's just "(,a=,1, b=,2," which can't be _that_ hard to type.

[Anton Lavrik]

On Tue, May 1, 2012 at 7:22 PM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
>
> That is, we shouldn't copy Erlson syntax, BECAUSE Erlson IS ALREADY
> USING IT and we must not break Erlson.
>

I'm not sure if you are serious, but here's my comment to that anyway.

To me, the benefits of using Erlson syntax for a native implementation
of name-value dictionaries in Erlang outweigh the broken compatibility
with Erlson runtime data representation.

One of the reasons why I created Erlson was to prove that it is
possible to have a nice syntax for Erlang dictionaries that fits with
the language (*). While Erlson syntax is similar to records, it is
still different enough to avoid confusion and it is compatible with
the rest of Erlang's syntax.

So, for those who actually consider implementing a native dictionary
data type in Erlang and using Erlson syntax for that, please feel free
to break Erlson.

Anton

(*) I realize that this is a highly subjective statement, but it is no
worse than any of the purely theoretic arguments about syntax I read
in this thread. Language design is not a science, for better or worse.

[Andrzej Sliwa]

come on, we can't start inventing new syntax only from 'editor' perspective, otherwise we could go to "brainfuck" syntax, which is so compact and easy to write ;), but in 100% is not readable at all

 

Regards,

Andrzej

[Jan Burse]

Richard O'Keefe schrieb:

> Now Jan Burse and I have been arguing with each other about frames,

> but actually, that was a*lot* more useful, because he focussed on
> semantics.

No I didn't care about semantics. The semantics is trivial since
as I interpret Erlang it does do some flow analysis of the given
code and sees to it that everything is ground, compared to normal
Prolog.

The flow analysis happens mostly implicit by having turned the
relation semantics of Prolog into a functional semantics. With
the sad result that one way to implement relation semantics in
Erlang is via continuation.

I was more concerned with usage and complexity. Your frames have
been invented many many times. I remember also some proposals
that retain the relational character, so that a pattern <{k1~v1,
k2~vn|F}> with a tail frame would become a first class citizen
of the language. These are then not called frames but rather
feature structures.

I guess one way to implement real feature structures in a logic
programming language would be to extend the unification so
that it works on feature terms with tail features. Feature
structures are very fashion in certain attribute grammar
formalism. Besides unification there can be further operations
between feature structures.

The simple syntax there for feature structures is [k1:v1,
k2:vn|F]. And it seems to be rather the rule than the exception
that a feature structure is open. This stems from the relation
ship of feature structures to equations, so the following
equations

G:k1 === v1,
G:k2 === v2

would correspond to the following feature structure relation:

G===[k1:v1,k2:v2|_]

The (===)/2 in the two equations can be viewed as a
declarative combined setter/getter. So setter/getter
are by no way alien to logic programming and it is not
mandatory to go with a feature structure on the
surface of a language.

What seems to be a problem in the grammar domain is that
then practically highly recursive feature structures are
used. But some compilation techniques are able to flatten
them straight the way to records and removing all open
parts that are never closed. So that a feature
structure (*):

[head:[cat:[n:A,v:B|_],agr:[per:C,num:D|_],
case:E,nform:F|_],bar:G|_]

Turns into:

dag0(A,B,C,D,F,G)

In certain circumstances it can be also benefitial to turn
open parts into record positions. Maybe the grammar application
domain can provide further ideas for representing and optimizing
frame terms in Erlang, and vice versa. Most probably the
referenced paper below is already hopelessly outdated, and
people do even more clever things today. But anyway.

(*)
Compiling Feature Structures into Terms:
an Empirical Study in Prolog (1993)
by Andreas Schöter , Buccleuch Place
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.8407

[Jon Watte]

The implied mapping takes JSON {id: 4135, title: "title2"}

I'd just like to chime in that this is not actually valid JSON. In valid JSON, all keys are quoted strings. Thus, this would be:

{"id": 4135, "title": "title2"} 

Which is not at all as good a match with Erlang idioms, unless you want more magic to turn strings into atoms, only when used as keys...

Sincerely,

jw

Sincerely,

Jon Watte


--
"I pledge allegiance to the flag of the United States of America, and to the republic for which it stands, one nation indivisible, with liberty and justice for all."
~ Adopted by U.S. Congress, June 22, 1942

[Richard O'Keefe]

There is a new version of
http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf

- a couple of diagrams
- some new material, including a description of what
is wrong with using X.y to access a field
- a new section 2 apologising for the rewrite this
needs and hasn't had yet
- a new section 15 (starting on page 41)
which describes in some detail how it works
with new BEAM(-ish) instructions
put_frame allocate a frame
is_frame give true/false result
test_frame used in clause heads
check_frame used in expressions
load_slot load a slot (with inline cache)
copy_frame copy a frame for updating
store_slot store a slot (with inline cache)

On 2/05/2012, at 5:19 AM, Björn-Egil Dahlberg wrote:

> Syntax is more than just finding which characters our parser can identify, like in the frames proposals where ~ is our separator. When I read <{ k ~ value | Is }> it says cons "k is like a value" to Is. *brr*

In fact the semantics very closely related to the semantics of
`((k ,@value) ,@Is) considered as an association list in Scheme,
which really does involve a cons. Let's face it, Is#{k=value}
says that k *equals* value, which it doesn't. There is a uniquely
right character for the job, and that's ↦ .

> What guarantees should we leave?

Any data structure associated with Phil Bagwell is likely to repay study.
I do hope there is no patent on "Cache-Aware Lock-Free Concurrent Hash Tries"
by Aleksandar Prokopec, Phil Bagwell, and Martin Odersky, because that's
_exactly_ what I want for storing frame descriptors. However, the
frame implementation described in the frames proposal is meant as a
replacement for records, and however good HAMTs are at other things,
frames should leave them in the dust *when used as record replacements*.

[Richard O'Keefe]

On 2/05/2012, at 8:50 PM, Jan Burse wrote:
> I was more concerned with usage and complexity. Your frames have
> been invented many many times.

I know that. The frames proposal SAYS that with specifics.
I haven't seen this particular implementation used for this
particular purpose, but it's not novel either.

> I remember also some proposals
> that retain the relational character, so that a pattern <{k1~v1,
> k2~vn|F}> with a tail frame would become a first class citizen
> of the language. These are then not called frames but rather
> feature structures.

Actually, they're called ψ-terms.

>
> I guess one way to implement real feature structures in a logic
> programming language

It has been done in at least two different ways.
LIFE (Logic, Inheritance, Functions, and Equations)
was described by Hassan Aït-Kaci and Patrick Lincoln
in 1989. It was based on LOGIN, described in
3. A ̈ıt-Kaci, H. and Nasr, R., “LOGIN: A Logic Programming Language
with Built-in Inheritance.” Journal of Logic Programming 3(3), pp. 187–215. 1986.
(Bob Carpenter's "The Logic of Typed Feature Structures:
With Applications to Unification Grammars, Logic Programs And Constraint Resolution)
was published nearly 20 years later.
There was an interpreter for LIFE written in C called Wild LIFE
which can still be downloaded if you search for it (the link on
Peter van Roy's page appears to be dead). I found it at
http://rapidlibrary.com/files/life1-01-tar-z_ulzttmxtv8i89on.html
I was very unhappy when I learned that LIFE was dead.

The Wild LIFE system can't be compiled by an ANSI C compiler. One
can figure out by reading the source code that it uses binary trees
to represent ψ-terms.

The other way that I know of is what Michael Covington did in his
book about natural language processing in Prolog, which is what I
did independently, and which I *suspect* but don't know that
Pl-PATR did as well, which is to use equations on paths rather than
displaying feature structures as ψ-terms, and precompiling the
grammar so that the paths work on plain on Prolog terms. >>Feature
structures<< may be open, but the set of >>feature names<< is closed.

>
> The simple syntax there for feature structures is [k1:v1,
> k2:vn|F].

The syntax for ψ-terms looks like label(a => U, b => V) with
the 'tail' implied.

> And it seems to be rather the rule than the exception
> that a feature structure is open.

Yes. But this is >>Erlang<< we are talking about, not Prolog,
and not LIFE, and not linguistics. Erlang does not use unification.

> (*)
> Compiling Feature Structures into Terms:
> an Empirical Study in Prolog (1993)
> by Andreas Schöter , Buccleuch Place
> http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.8407

You will note that this comes two years _after_ WildLIFE was released
and at least seven years after Aït-Kaci's work.
"Compiling to flat terms" is the technique I mentioned using above;
it is so obvious, so easy, and so effective that it's a wonder to me
that anyone ever used anything else. In fact it was SO obvious and
SO easy that it never occurred to me that it was worth a publication.
(Erlang readers: this is actually very very close in spirit to the
way -record declarations are implemented now. Savour the irony!)

Anyway, the *POINT* of the frames proposal is not to innovate but to
take a well established idea and adapt the details to make it suitable
for >>>replacing records<<< in a distributed functional language.
It's like a mouse's nest in a cat's ear: only the location is new.

[Jan Burse]

Richard O'Keefe schrieb:

> There is a new version of
> http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf

Pitty the sections do not have a @date, now
I don't know what I have already read.

BTW: Inline cache, when your lookup would return
a pointer to an immutable {func,class}, than you
can cache this in one STORE op, making it amenable
to concurrent use. See to it that you first use
one LOAD op to retrieve this pointer into a local
register, before checking class and then possibly
using func. Works with both 64-bit and 32-bit,
no need to pack two 32-bit into one 64-bit.

(I am using the same for {fun,arity} for a cache
inside an atom, works like a charm)

> It's like a mouse's nest in a cat's ear

LoL, poor kitty

[Jan Burse]

Jan Burse schrieb:

>
> (I am using the same for {fun,arity} for a cache
> inside an atom, works like a charm)

Actually I am using {fun,arity,abolish_flag}, and
abolish_flag might change. But I guess such a
flag is not needed for frames, no hot swap problem?

[Jan Burse]

Jan Burse schrieb:

> (I am using the same for {fun,arity} for a cache
> inside an atom, works like a charm)

Speeds up =../2, call/n, predicate_property (
also when predicate indicator based (sic!)),
etc.. in Prolog.

But requires one more field in an atom, and
to avoid cache trashing when the same arity
is overloaded, I don't make atoms unique.

Could be also helpful for Erlang apply, or
maybe you already usa a cache there in some
instruction.

[Tim McNamara]

On 3 May 2012 16:22, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
> There is a new version of
> http://www.cs.otago.ac.nz/staffpriv/ok/frames.pdf

Thank you for this impressive piece of work. Here are some notes that
I made that may be of use to people who are intimidated by the size of
the article. (Please correct errors!)

- Frames are a distinct type. That is, they are not syntactic sugar
over tuples and/or lists
- They are not declared. They
- They have no default value
- Only atoms may be keys
- No frame identifier, e.g. impossible to label a frame as "person"
which represents people

I've only been able to make it through the first 8 sections in any detail.

The only thing that genuinely worries is relying on atoms as keys. I
think this would mean that frames could not be used to deserialise
untrustworthy strings. That is, programmers could expose themselves to
having their atom table overrun.

I disapprove of the tilde though. The association with approximate is
too strong for me to overlook. The argument of overloading the equals
sign is noted. However, when I am programming, I don't see the equals
sign as equality. When I see it, it represents an association, usually
from right to left on the page.

However, I'm wary of bikeshedding over the syntax. However, I don't
see why the Erlang grammar can not support the Python syntax (I'm not
calling this JSON, because JSON requires double quoted keys).

{a: "0", b: "1"}

The comment was made earlier that it's useful to use different syntax
for different conceptual things. However, I think that this syntax is
sufficiently different from a tuple to be distinguished, while looking
unified. We are referring to lots of data which has been unified
together into a single frame.

I would also make the point that there are very few single-language
programmers. Context switching is not free for humans either. It would
be great to be able to translate between JS/Python/other and Erlang
with as few hurdles as possible. Using familiar syntax for a fairly
similar task seems very useful to me.

I don't understand this example [p16]

foo:bar(Ugh),

How is this ambiguous? Are you trying to set the bar element from the
foo frame? Your next line is unclear.

If, following JSON, you use colon for maplets, your comprehension comes
to a juddering halt while you try to gure out if this is a maplet or a
remote function call. BAD!

JSON is just a serialisation format. It is not JavaScript. JavaScript
would require you something closer to: foo["bar"] = Ugh. Is that what
you intended?

I think this is a strong argument, but I am not persuaded by it:

Erlang uses curly braces for sequences, not sets. It would be
very confusing for people if {a ~ 1, b ~ 2} = {b ~ 2, a ~ 1}
but {{a, 1}, {b, 2}} !=
{{b, 2}, {a, 1}}.

I think that this would certainly be considered a trip for new Erlang
programmers. However, this risk is warranted.

[José Valim]

I don't understand this example [p16]

   foo:bar(Ugh),

How is this ambiguous? Are you trying to set the bar element from the
foo frame? Your next line is unclear.

It is ambiguous because:

    { foo:bar(Ugh) }

Could be a tuple with one element which is the result of foo:bar(Ugh) or

a frame with key foo and value bar(Ugh) (according to your proposal).

There is also the "ambiguity" in {}. Is it an empty frame or an empty tuple?

<{ foo: bar(Ugh) }> could work because there would be no ambiguity for

the compiler, but it would be still ambiguous for humans (damn you, humans).

I would shrug if I read a code like <{ foo:bar(Ugh) }> that actually means

<{ foo: bar(Ugh) }>.

[Max Lapshin]

>  - They have no default value
>  - Only atoms may be keys

These two poins are very interesting to discuss.

Problem is with handling external data. After frames will be added,
webservers will change their API to represent user query string as a
frame.

There have been done a lot in Ruby world, because Ruby also have atoms
and it has the same problem with their non-GC behaviour. So appeared
extension to Hash class, called HashWithIndifferentAccess

http_params[:key] is translated to http_params["key"] and strings are
used internally to store keys.
This is safe in terms of atom table overload and convenient.

We will not be able to use frames for handling user data if only atoms
can be keys, but if we allow to store atoms, binaries and strings as
keys, we will get a mess with different ways to introduce keys.

[Tim McNamara]

On 3 May 2012 22:50, José Valim <jose....@gmail.com> wrote:
>> I don't understand this example [p16]
>>
>>    foo:bar(Ugh),
>>
>> How is this ambiguous? Are you trying to set the bar element from the
>> foo frame? Your next line is unclear.
>
>
> It is ambiguous because:
>
>     { foo:bar(Ugh) }
>
> Could be a tuple with one element which is the result of foo:bar(Ugh) or
> a frame with key foo and value bar(Ugh) (according to your proposal).
> There is also the "ambiguity" in {}. Is it an empty frame or an empty tuple?

Ah, understood. There is certainly an opportunity for error here.
Could {:} represent the empty frame?

>
> <{ foo: bar(Ugh) }> could work because there would be no ambiguity for
> the compiler, but it would be still ambiguous for humans (damn you, humans).
> I would shrug if I read a code like <{ foo:bar(Ugh) }> that actually means
> <{ foo: bar(Ugh) }>.

FWIW, I think this is probably acceptable. If <{ }> was accepted, it
implies an intent on the programmer's behalf that they wanted to
create a frame, which requires a key and a value.

[Tim Watson]

I'm not sure that I would (shrug) at it - #{ foo->bar, baz->2 } seems much less ambiguous to me, and whilst I recognise that #{ 1 -> 2 } appears a bit odd, I would've thought that it's fairly obvious what it's doing. There is no clash with module:function syntax, no clash with #record_name{} syntax and no chance of accidental matching taking place AFAICT. And as the author of erlson mentioned, there is no real problem with trouncing erlson's #{...} syntax as erlson users will immediately migrate to frames.

[Hynek Vychodil]

If I understand ROK right, the main problem which worries him is code like this

{
quix
% some comment or some more code
, <{ a:1
% some comment or some more code
, foo:bar(Ugh)
, baz:quux(Eh)
% some comment or some more code
}>
% some comment or some more code
, [ ...]
% some comment or some more code
}

When you see code snippet/line with foo:bar(Ugh) you have to very
carefully scan surrounding code when you are willing to know what the
hell is going on and above example is relatively pretty formatted and
indented. It can be far worse in real world application and with
little bit more levels as

{ok, <{ foo:{bar, baz, <{foo:bar(Ugh), baz:quux(Eh)}>, foo:bar(Och)}, a:2 }>}

Good luck with this syntax.

--
Hynek Vychodil
BI consultant

GoodData
náměstí 28. října 1104/17, 602 00, Brno - Černá Pole
Office:   +420 530 50 7704
E-mail:  hy...@gooddata.com
Web:     www.gooddata.com

[Richard O'Keefe]

On 3/05/2012, at 8:56 PM, Jan Burse wrote:
> Pitty the sections do not have a @date, now
> I don't know what I have already read.

That's why my message identified the sections that
were new.

>
> BTW: Inline cache, when your lookup would return
> a pointer to an immutable {func,class},

Er, in frames.pdf, the pointer *doesn't* point to
{func,class}. As section 15 makes clear with a
diagram, the cache item in a load_slot or store_slot
instruction points to something which is logically
{descriptor,offset} but can be stored more compactly.

> then you

> can cache this in one STORE op, making it amenable
> to concurrent use.

As indeed section 15 already says.

>
> > It's like a mouse's nest in a cat's ear
> LoL, poor kitty

It's from "Rite of Passage" by Alexei Panshin.
While in transit between The Ship and Grainau (?)
Mia Havero is told a story, and in the store the
good guy wins a fortune by asking the ogre this
riddle:
What is not, never was, and never will be?

The answer is a mouse's nest in a cat's ear.
Rite is one of the *great* SF stories, and it has
dated very well.

[Richard O'Keefe]

On 3/05/2012, at 10:39 PM, Tim McNamara wrote:
> The only thing that genuinely worries is relying on atoms as keys. I
> think this would mean that frames could not be used to deserialise
> untrustworthy strings. That is, programmers could expose themselves to
> having their atom table overrun.

This is not a problem at all for frames that are created by patterns
that the compiler can see, only for dynamically created frames. And
there is an EEP showing in sufficient detail how we can fix *that*
problem once and for all. The frames proposal presupposes that's
been done.

> I disapprove of the tilde though. The association with approximate is
> too strong for me to overlook.

This is a classic "what is good for everybody is what *I* am used to"
argument. For a lot of people reading this, "~" is not associated with
approximation at all, but with regular expression matching.

> However, I'm wary of bikeshedding over the syntax. However, I don't
> see why the Erlang grammar can not support the Python syntax (I'm not
> calling this JSON, because JSON requires double quoted keys).
>
> {a: "0", b: "1"}

Because {a:f(), b:g()} is already legal Erlang syntax for
a tuple with two function calls to compute its elements.

> I would also make the point that there are very few single-language
> programmers. Context switching is not free for humans either. It would
> be great to be able to translate between JS/Python/other and Erlang
> with as few hurdles as possible. Using familiar syntax for a fairly
> similar task seems very useful to me.
>
> I don't understand this example [p16]
>
> foo:bar(Ugh),
>
> How is this ambiguous? Are you trying to set the bar element from the
> foo frame? Your next line is unclear.
>
> If, following JSON, you use colon for maplets, your comprehension comes
> to a juddering halt while you try to gure out if this is a maplet or a
> remote function call. BAD!
>
> JSON is just a serialisation format. It is not JavaScript. JavaScript
> would require you something closer to: foo["bar"] = Ugh. Is that what
> you intended?

Actually, no. {foo:bar(Ugh)} is 100% leval JavaScript.
s/JSON/JavasScript in the version I just put up.

The whole point here is that foo:bar(Ugh) is a legal Erlang function
call. We don't want the same sequence of tokens *also* meaning
"bind field foo to value of bar(Ugh)".

I'm painfully aware of the multilanguage programmer issue.
The last time I counted, I'd used over 200 programming languages
and could remember more than I wanted to about each of them.
Just today I wrote

g: Integer * Integer -> Integer

in a Haskell Program, where I should have written

g :: Integer -> Integer -> Integer

I'd used ML syntax...

I get far too confused by R code doing f(x=y=g()) where the
first = means "bind argument by name" and the
second = means "assign to variable" to be happy about
two different meanings for = in Erlang.
At least using "~" means (a) ***NOT*** innovating, other
people have used ~ for this purposebefore; and (b) *NOT*
using tokens ambiguously.

[Richard O'Keefe]

On 3/05/2012, at 10:55 PM, Max Lapshin wrote:

>> - They have no default value
>> - Only atoms may be keys
>
> These two poins are very interesting to discuss.
>
>
> Problem is with handling external data.

Response 1:

Frames are very explicitly a replacement for RECORDS.
You cannot currently use records in external data.


Response 2:

EEP 20 (http://www.erlang.org/eeps/eep-0020.html)
has now been around for nearly four years. It offers
a permanent fix to the limited size of the atom table,
a fix which has been used before in another concurrent
programming language.

That's not the only way to fix the atom table problem.
SWI Prolog offers true multicore concurrency AND a
garbage collected atom table. That's been around for
a few years now too.

By any engineering standard I can think of, fixing this
vulnerability in Erlang should have very high priority.
The frames proposal presupposed that this flaw has been
fixed somehow. We *cannot* let this flaw warp Erlang
programming for the rest of our lives.

>
> We will not be able to use frames for handling user data if only atoms
> can be keys, but if we allow to store atoms, binaries and strings as
> keys, we will get a mess with different ways to introduce keys.

We *WILL* be able to use frames for handling user data
WHEN THE ATOM TABLE BUG IS FIXED.

And it is LONG past time for that to happen.

[Tim Watson]

Anyone on the OTP team wish to comment on whether or not this is part of the roadmap for R16/17?