On 18/05/2012, at 12:21 PM, Tom Parker wrote:

It's that simple.  Back when Eiffel was undergoing major changes,
I proposed something I called the "diarthognathus principle".
When you want to evolve from supporting technique X to supporting
technique Y, you had *better* evolve through a stage where you
support *both* techniques.

And that means that there has to be a period when frames have begun
to work and record syntax still works *EXACTLY* the way it does now.

Frames and records are not inter-operable (records are just a
stylised use of tuples, while frames are a new completely distinct
type).

But there is one thing more.
Suppose you go around mechanically translating

        #id0{id1=E1, ..., idn=En}
to
        <id0{id1~E1, ..., idn~En}>

which is a fairly straightforward task.

What happens?  The code immediately stops working.  Because the two forms
of construction DO NOT HAVE THE SAME SEMANTICS.

A frame construction says "here is ALL the information about this frame."

A record construction says "here is SOME information about this record,
pick up anything I've left out from the -record declaration."

Because these things have different semantics they MUST have different
syntax.

THERE IS NO STANDARD ORDER!!!

It may still be true that there are more lines of COBOL than any other language.
In COBOL, it's <field> of <record>.
In Algol 68, it's <field> of <record>.

        predecessor count OF node -:= 1

seems to me a perfectly natural way to decrement the number of predecessors of
some node.

For a functional language, note that
a field selector is semantically just an overloaded function.

In the Pop family of languages, record selectors are just functions (well,
technically doublets) and are called exactly like functions.
In Lisp, selectors are called exactly like functions.
In ML-family languages, if f is a field, #f is the function that extracts
that field (and yes, that's where I got the syntax).

Yes, in Smalltalk the analogue of field selection is "object field",
but that's how Smalltalk writes unary functions: "-42 abs".

The syntax is neither novel nor unnatural unless you take an
extremely parochial view of programming languages.

One of the things that frames are meant for is to communicate
structured values from one context to another context WITHOUT
there being any declaration shared by both.

Records are *not* going to go away.  They have their virtues
and their vices.  Frames are *different* virtues and vices.
Trying to make one look like the other is about as kind to
programmers as trying to make numbers and strings look the same.
It is seductively attractive, but causes more trouble than it's
worth in practice.  (Speaking as someone who maintains an AWK
implementation and tried to write an AWK compiler.)

We know what is the right way to catch type errors and typos.
That's what the Dialyzer does, and does very well.

Speaking as someone who has tried to maintain other people's
code, if I see #point{x = 1, y = 2} I want to *KNOW* that
that is a record, not anything else at all.  I don't want
it to maybe be a record or maybe something else, possibly
depending on a file that isn't open in my editor.

You are asking for a "feature" that I regard as active
malevolence towards the programmer.

As for major to minor, I have never perceived anything major or minor
in field access.  Some languages put the field first, some put it second.
BIG DEAL.  Languages are different.  Get over it.  They don't all have
to look like Java.

I skipped the rest of the message.

The frame proposal exists precisely in order to have something that
can do the same job that records do but in a DIFFERENT way.  Because
the semantics is different, programmers need to know what they are
dealing with, so it is an incredibly bad idea to use the same syntax
for both.  It doesn't matter so very much what the syntax _is_ as
long as it is clearly _different_.  Frames "fit" into Erlang data
structures in a way that records never did; _all_ the information
that exists about a frame is right there in it without consulting
any declaration.  In particular, changing a declaration in one
field cannot change the meaning of a frame construction in another.
Adding -frame declarations isn't "raising" the proposal but "razing" it.

But does anyone know where the Frames proposal stands vis-a-vis the
OTP team's work on hashes?

My reference point is Kenneth's presentation last year:

http://www.erlang-factory.com/upload/presentations/468/EUC_Hashes2011...

Records are not without problems, but IMO this is not one of them. This
is a problem of using .hrl files and choosing to program in a way which
is not disciplined and thus error prone. There is a very simple way of
avoiding this problem that IMO is a very nice way of programming: Use
records as abstract data types and have them in a single module that
exports appropriate getters and setters for manipulating fields of the
record. If, for whatever reason, you do not like this way of using
records and want the ability to perform pattern matching and field
extraction in more than one module as you do today with records, better
make sure you use an appropriate Makefile (or equivalent) for compiling
your application. It's not the fault of record syntax if you do not use
such a mechanism! (Makefiles are technology of the 70's after all...)

You can put a local function definition in a .hrl file as well, and it
has been done too. Changes to a record usually trigger badmatch
or function_clause exceptions if not all modules are recompiled
and replaced in a coordinated way. This is a problem, for sure,
but if you make subtle semantic changes to a function in a .hrl
file, or to a macro (which can expand to pretty much anything),
then *anything* can happen - including a large set of extremely
subtle errors that are hard to achieve with records.

So putting a function definition in a .hrl file can cause some
really subtle problems. This obviously doesn't mean that functions
are fundamentally flawed, but yet an example that reliance on
preprocessors is a dangerous thing.

The worst thing you can do with records - and it's indeed a bad
thing - is to rename attributes and use the same positions for
entirely different data (no, worst would be the *same* data type
but with a different interpretation). Luckily any half-decent
programmer will understand enough not to do this.

Richard O'Keefe has many times stated that a key goal is to
remove the reliance on the pre-processor.

If you think about it, Erlang has no facility for sharing structured
data types between modules, except for code - as Kostis mentioned
- and pattern matching on the primitive elements of the structure.

You can provide accessor functions that are exported from the
same place where the data structure is managed, or you can
use data structures that are as shallow as to be easy to inspect
through pattern-matching. If this is not enough, you can pass along
property lists, which are simple, yet flexible enough to convey just
about anything - however cannot be quickly inspected with
pattern-matching. So people trade safety for speed.

…or rely on convention, which actually works very well.

Remember that records are little more than syntactic sugar.
They do not introduce a data type that would offer anything more
than the above. You could get a similar effect by introducing a
series of home-cooked macros, expanding to operations and
patterns on a tuple structure, and then let multiple modules
rely on them through a .hrl file.

I even have some old code lying around from the time before
records. It had a lot of code like this:

person(name,      {person,X,_,_}) -> X;
person(age,         {person,_,X,_}) -> X;
person(address, {person,_,_,X}) -> X.

person(name, X, {person,A,B,C}) -> {person,X,B,C};
person(age,    X, {person,A,B,C}) -> {person,A,X,C};
person(addr,  X, {person,A,B,C}) -> {person,A,B,X}.

A bit fiddly to create, but easy to verify through visual inspection.
Of course, you could put that in a .hrl file and use as a 'shrared
data type'. It would suffer from roughly the same problems as
records.

The obvious limitations with records were the subject of animated
discussion even before they were added to the language. You
wouldn't recall this unless you were a member of a pretty small
group of people debating this on a mailing list inside Ericsson, long
before the Open Source release.

Even then it was agreed that the proper way to do it was to introduce
a new data type. As my memory serves, the problem with that was
that JAM didn't have any more type tags to use. A redesign of the
memory management structure in the VM would be required in order
to add new data types. In light of this, records were seen as a
cost-efficient way to solve a common problem of copy-paste
programming.

Later on, I believe it was the HiPE team that proposed a new tagging
scheme and wrote some working code to demonstrate. By the time
it was integrated into Erlang, records had become ubiquitous, and
could not be easily replaced.

If my recollection is faulty, the old-timers can correct me. I recall this
playing out right after I joined Ericsson in 1996.

(The new tagging scheme was introduced in R7B in 2000, and is described
in Mikael Petterson's PhD thesis: http://www.it.uu.se/research/publications/reports/2000-029/)

One can argue that it's taken a long time to correct this issue, and
add a structured data type with all the goodness of records and
none of their drawbacks. The reasons:

- Erlang, even taking this into account, is a darn good language for
  writing complex, robust and scalable systems.
- Convention actually works pretty well in practice.
- The OTP team wanted to get it right the second time.
- Backwards compatibility is considered extremely important in the
  Erlang world (exactly because we don't want to break the
  complex, robust and scalable systems already written in Erlang)

BR,
Ulf W

Basically, that was it for me. I worked in erlang 1+ year building it,
I had a great time doing it but ultimately changed my mind despite the
fact it'll take me 1 year to rewrite the erlang bits in C#. You could
easily build everything you need in erlang, there is no doubt there
(and fairly quickly too). Whether you can reach cpu budgets and memory
budgets you set yourself will largely depend on how motivated you are
with erlang and how much you are willing to hybridize your system
(either with drivers or other processes). Using it only for message
processing seems reasonable but a bit strange if your solution is in a
different language. In C# for example, NServiceBus would probably be
an equally good solution. I'm sure java must have a stable solution
for messaging as well. Rolling out your own isn't all that hard
either, the Disruptor guys sorta did it.

That's my story and my 2 cents. Note that it wasn't a professional
project, only a hobby.
Cheers,
Nicholas

On Fri, May 18, 2012 at 2:05 AM, Ilyushonak Barys

On 19/05/2012, at 8:57 AM, Garrett Smith wrote:

If I remember well, your proposal mentions some benchmarks
with monomorphic caching. You also explained the SmallTalk
history of caching methods. So basically I suppose someone
is doing the following:

    if (cache == null || cache.base != base)
        cache = lookup(base,key);  /* costly */
    return cache;

Since last week I have been experimenting with polymorphic
caching. And it seems that is practically same speed as
monomorphic caching, but has the additional advantage of
not trashing the cache.

Trashing of monomorphic cache happens when the base
is not constant for a call site. When trashing happens
time spent goes drastically up. I observed for some
test case of mine:

     No Trashing: 1400 ms
     Trashing: 2200 ms

So I did a very simple polymorphic cache consisting of a
a single linked list. Its adding new entries at the end
of the list if it didn't find an entry with matching
base. Its not that bad:

     Monomorphic: 1400 ms
     Polymoprhic: 1450 ms

I am not using some locking. Its just overwriting the
the last link with a new entry, so concurrently it might
do double work sporadically, but the result should never
be wrong. But it requires GC of the eventually lost record.

Something done for Erlang records?

Bye

That's a PDF of presentation slides from 2011 User Conf-- it lists
"frames" in the title and as a input to their process.

I read the document as a status update on their work, listing some
performance results from various data structures.

The "Conclusion and way forward" slide seems open ended.

In any event, is not relevant to any discussion here what the OTP team
is actually working on? Is this is a blip of interest in a decade old
proposal that has no shot of getting into the language?

Now I can as a fact tell you that we are addressing both.

We are prototyping an implementation of "hashes", or whatever we will call
them, and we are also prototyping a reference implementation of the frames
proposal.

We think that the frames proposal is interesting and think we need an
implementation so we can evaluate performance and compare it with records.
After the evaluation we will come to a decision and will also give more
details about possible changes/restrictions/additions we want to make
compared with the current frames proposal. What we find most interesting
with the frames proposal is the well thought semantics.

/Kenneth , Erlang/OTP, Ericsson

 _______________________________________________

On 22/05/2012, at 12:15 AM, Garrett Smith wrote:

Slide  0: title
Slide  1: there's always room for improvements RECORDS Hash maps
Slide  2: outline of records (few named fields, many instances, matchable)
Slide  3: not a distinct data type, not suitable for many fields
Slide  4: outline of hash maps (proplists, dict, gb_trees, gb_sets)
Slide  5: what's wrong with proplists
Slide  6: what's wrong with dict
Slide  7: Requirements/Goals ***FOR HASHMAPS***

        And it is quite clear from content slide 7 that they are
        talking about something that can do the job of dict and
        gb_tree; suitability for type checking is NOT listed.

        More precisely, content slide 7 gives the impression of
        a vague desire for something that's a floor polish AND
        a desert topping, a bulk hashed container AND a possible
        replacement for records.

Slide  8: mentions me, Joe Armstrong, Anton Lavrik, Phil Bagwell.
Slide  9: Erlson syntax.
Slide 10: >>> We have implemented prototypes and performed measurements
          using the Vlists and HAMT datastructures internally in the
          Erlang VM <<<

        It is very very clear from this that the experiments
        concerned something suitable for *LARGE* data structures.
Slide 11: Diagram of records as tuples.
Slide 12: Diagram of VLists / vhash lists.
Slide 13: Diagram of Hash Array Mapped Tries.

        Popcount is one of those things that *ought* to be fast but
        that often get left out of actual chips.  POPCNT wasn't on
        PCs until Core i7, I believe.  It's in the SPARC V9
        architecture, but is not in all SPARC V9 _hardware_, like
        the machine on my desk.

Slide 14: Diagram of hash table with exception list
Slide 15: Another such diagram

        This appears to be an adaptation of a 1980s technique for
        O(1) array update; the latest version is compact and in
        place while old versions have a list of deltas.

Slide 16: Graph of time in microseconds against number of elements
        inserted for dict, gb_trees, vlist_hash, and hamt (up to ~ 30)
Slide 17: Another such (up to ~ 65 000)
Slide 18: Graph of lookup time (up to ~ 30)
Slide 19: Another such (up to ~ 65 000)
Slide 20: Graph of memory (up to ~ 30 elements)

        These data structures take from 4 to 8 words per element.
        Frames, in contrast, take 1 to 2 words.

Slide 21: Another such graph (up to ~ 65 000 elements)
Slide 22: Table of requirements by data structures.

        Frames are conspicuous by their absence.

Slide 23: "new better records" "and Hashmaps" "ARE TWO DIFFERENT THINGS."

        I agree with this distinction and made it a decade ago.

        This talk was about Hashmaps, NOT about anything credible as
        new better records.

It will be really good for Erlang to have a better implementation of
hashmaps.  This was good work.  It owes nothing to me; I'd never even
heard of Hash Array Mapped Tries before. I look forward to seeing
something coming of it.

But there is NOTHING reported in that presentation that evaluates
alternatives for "new better records".

To be fair, there's no reason why a single logical data structure
has to have a *uniform* representation at run time (as I was saying
to a data base class yesterday).  One could use something like frames
up to a certain size and something else above that size.