[erlang-questions] Pmods, packages, Unicode source code and column numbers in compiler - what will happen in R16?

[Patrik Nyblom]

Hi all!

The OTP Technical board decisions from last Thursday are now published
on the erlang.org website, which means that the answers to some
questions about changes in R16 are finally officially answered.

Cheers,
/Patrik
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Loïc Hoguin]

Awesome news! Happy with everything!

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Fred Hebert]

Happy with all measures. I think the balance you strike for parametrized
modules is also pretty good.

Quick question -- I remember older posts relative to tuple calls about
how their implementation somehow limited future optimizations that could
have been possible. Does it mean their support for Pmod utilization
means further optimizations will be cancelled?

Otherwise, it's good to be stripping stuff from Erlang. The less
complex, the better.

Congrats on the good decisions (unless someone proves otherwise! ;) )

Regards,
Fred.

[Mahesh Paolini-Subramanya]

pmods and packages - very nicely (and alliteratively) done :-)

cheers

Mahesh Paolini-Subramanya

That Tall Bald Indian Guy...

Google+  | Blog   | Twitter

[Tuncer Ayaz]

On Tue, Oct 16, 2012 at 3:51 PM, Patrik Nyblom wrote:
> Hi all!
>
> The OTP Technical board decisions from last Thursday are
> now published on the erlang.org website, which means that
> the answers to some questions about changes in R16 are
> finally officially answered.

I am not certain how to interpret the following:
"The default file encoding will be ISO-Latin-1 in R16,
but will be changed to UTF-8 in R17.

Source code will need no change in R16, but adding a
comment denoting ISO-Latin-1 encoding will ensure that
the code can be compiled with the R17 compiler."

Will we have to add an encoding comment for unmodified
source code in R17 (that is pure 7-bit ASCII)?

[Raimo Niskanen]

On Tue, Oct 16, 2012 at 04:15:18PM +0200, Tuncer Ayaz wrote:
> On Tue, Oct 16, 2012 at 3:51 PM, Patrik Nyblom wrote:
> > Hi all!
> >
> > The OTP Technical board decisions from last Thursday are
> > now published on the erlang.org website, which means that
> > the answers to some questions about changes in R16 are
> > finally officially answered.
>
> I am not certain how to interpret the following:
> "The default file encoding will be ISO-Latin-1 in R16,
> but will be changed to UTF-8 in R17.
>
> Source code will need no change in R16, but adding a
> comment denoting ISO-Latin-1 encoding will ensure that
> the code can be compiled with the R17 compiler."
>
> Will we have to add an encoding comment for unmodified
> source code in R17 (that is pure 7-bit ASCII)?

7-bit ASCII is valid UTF-8 and therefore will need no encoding comment.

If you have Latin-1 (8-bit ASCII) source code with codes > 255 it is safest to
add an encoding comment.

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB

[Ian]

On 16/10/2012 15:25, Raimo Niskanen wrote:
> If you have Latin-1 (8-bit ASCII) source code with codes > 255 it is safest to
> add an encoding comment.

I think you meant > 127 :)

Getting a byte > 255 would be a fine trick if you could do it.

Ian

[Fred Hebert]

Additional question regarding column numbers and future UTF8 support:
will the column number be based on byte length, grapheme clusters (what
I would usually use when typing text in a module), include combining
characters?

I think we've had many discussions regarding how difficult the idea of a
length or position in a string can be to figure out -- and I know the
unicode consortium makes its own recommendation there. I'm just curious
which is going to be followed.

[Matthew Evans]

Thank you....

I was going to use a parameterized module in a upcoming project....I guess I won't do that now :)

When will we get ihints as to other features going into the R16 release? Frames? New NIF features? JIT etc?

Thanks

Matt

> Date: Tue, 16 Oct 2012 15:51:01 +0200
> From: p...@erlang.org
> To: erlang-q...@erlang.org
> Subject: [erlang-questions] Pmods, packages, Unicode source code and column numbers in compiler - what will happen in R16?

[Michael Truog]

On 10/16/2012 06:51 AM, Patrik Nyblom wrote:
> Hi all!
>
> The OTP Technical board decisions from last Thursday are now published on the erlang.org website, which means that the answers to some questions about changes in R16 are finally officially answered.
>
> Cheers,
> /Patrik
>

Very cool! These changes should help developers avoid problems during development.

Could you please consider making UDP multicast support official, so it is no-longer undocumented/unsupported?

[Robert Virding]

Doesn't this mean that now the syntax for parametrised modules is still there but becomes meaningless? Or rather it will mean whatever the writer of a module chooses it to mean. That really won't encourage clarity. Or what am I missing?

Robert

----- Original Message -----
> From: "Patrik Nyblom" <p...@erlang.org>
> To: erlang-q...@erlang.org
> Sent: Tuesday, 16 October, 2012 3:51:01 PM
> Subject: [erlang-questions] Pmods, packages, Unicode source code and column numbers in compiler - what will happen in
> R16?
>

[Richard Carlsson]

On 2012-10-16 23:29 , Robert Virding wrote:
> Doesn't this mean that now the syntax for parametrised modules is
> still there but becomes meaningless? Or rather it will mean whatever
> the writer of a module chooses it to mean. That really won't
> encourage clarity. Or what am I missing?

It also seems ass-backwards to me that the syntax will be dropped, but
the hack that we used for the proof-of-concept implementation will be
immortalized as "tuple modules". Drop the parameterized modules if you
like, but don't make the apply-hack a documented feature; the old {M,F}
was bad enough.

/Richard

[Robert Virding]

I agree. Do it properly or get rid of it.

Robert

----- Original Message -----
> From: "Richard Carlsson" <carlsson...@gmail.com>
> To: erlang-q...@erlang.org
> Sent: Tuesday, 16 October, 2012 11:45:21 PM
> Subject: Re: [erlang-questions] Pmods, packages, Unicode source code and column numbers in compiler - what will
> happen in R16?
>

[Richard Carlsson]

On 2012-10-17 00:09 , Robert Virding wrote:
> I agree. Do it properly or get rid of it.

On second thought, there is one advantage to documenting this
representation: there can be no doubt that when you make a call through
a tuple {m,...}:f(...), you'll call the latest version of the module. We
had originally planned to make pmods an opaque datatype, like funs, but
unlike funs we didn't want to be bound to the module version used to
create the instance. Keeping the representation open should make it
clear what it is you're getting. But then don't remove the language
support - make it work throughout the toolchain instead.

[Richard Carlsson]

On 2012-10-17 00:29 , Richard Carlsson wrote:
> On 2012-10-17 00:09 , Robert Virding wrote:
>> I agree. Do it properly or get rid of it.
>
> On second thought, there is one advantage to documenting this
> representation: there can be no doubt that when you make a call through
> a tuple {m,...}:f(...), you'll call the latest version of the module. We
> had originally planned to make pmods an opaque datatype, like funs, but
> unlike funs we didn't want to be bound to the module version used to
> create the instance. Keeping the representation open should make it
> clear what it is you're getting. But then don't remove the language
> support - make it work throughout the toolchain instead.

For the record, the bad part about making a representation like this
public is that it takes implementation details that happened to make
sense for the underlying platform at the time (BEAM), and makes it
impossible to change them later on. In this case, such implementation
details are that the tuple contains exactly the module name followed by
the hidden parameters, in order. No additional fields can be added in
future versions. Also, the calling convention becomes fixed so that the
module tuple itself must always be passed as the last parameter.

[Richard O'Keefe]

On 17/10/2012, at 2:51 AM, Patrik Nyblom wrote:

> Hi all!
>
> The OTP Technical board decisions from last Thursday are now published on the erlang.org website, which means that the answers to some questions about changes in R16 are finally officially answered.

"UTF-8 BOM's will not be handled due to their limited use."

As I understand it, byte order marks are not *supposed*
to be used with UTF-8, because it has no byte order issues.

"Variable names will continue to be limited to Latin characters."

I hope that means "for this release."

An end to Java envy (otherwise known as removal of 'packages'):

Good news. But let's keep thinking about ways to solve
the problem that packages were meant to solve but didn't.

[Vlad Dumitrescu]

> On 17/10/2012, at 2:51 AM, Patrik Nyblom wrote:
>>
>> The OTP Technical board decisions from last Thursday are now published on the erlang.org website, which means that the answers to some questions about changes in R16 are finally officially answered.

Great initiative to publish these decisions and explain them!

On Wed, Oct 17, 2012 at 1:22 AM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
>
> "Variable names will continue to be limited to Latin characters."
>
> I hope that means "for this release."

That's an interesting problem. Variable names are defined as starting
with an upper case letter, but the only scripts that I know of that
have those are roman, greek, cyrillic and armenian. So would variables
with names in other scripts be forced to start with an uppercase latin
letter? We might just as well have them start with '§' or something,
and drop the capitalization rule.

My guess is that atoms will be allowed to contain unicode just so that
atom_to_list and list_to_atom can still be used, but usage of such
atoms in source code will be discouraged because these will have to be
written as quoted.

best regards,
Vlad

[Raimo Niskanen]

On Tue, Oct 16, 2012 at 03:33:54PM +0100, Ian wrote:
> On 16/10/2012 15:25, Raimo Niskanen wrote:
> >If you have Latin-1 (8-bit ASCII) source code with codes > 255 it is safest to
> >add an encoding comment.
> I think you meant > 127 :)

I was off by one. Thank you very much.

>
> Getting a byte > 255 would be a fine trick if you could do it.
>

All I need is some weird system with 9-bit bytes...

> Ian
> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB

[Marc Worrell]

On 17 okt. 2012, at 11:36, Raimo Niskanen wrote:

>> Getting a byte > 255 would be a fine trick if you could do it.
>

> All I need is some weird system with 9-bit bytes…

I remember making a C/assembler toolchain for such a cpu :)

- Marc

[Robert Virding]

----- Original Message -----
> From: "Raimo Niskanen" <raimo+erlan...@erix.ericsson.se>
> Sent: Wednesday, 17 October, 2012 11:36:19 AM
>
> On Tue, Oct 16, 2012 at 03:33:54PM +0100, Ian wrote:
> > On 16/10/2012 15:25, Raimo Niskanen wrote:
> > >If you have Latin-1 (8-bit ASCII) source code with codes > 255 it
> > >is safest to
> > >add an encoding comment.
> > I think you meant > 127 :)
>
> I was off by one. Thank you very much.
>
> >
> > Getting a byte > 255 would be a fine trick if you could do it.
> >
>
> All I need is some weird system with 9-bit bytes...

The old PDP 10 had 36-bit words and in some uses (emacs) had 9-bit bytes, 4 to a word. IIRC the standard was 5 7-bit bytes but I can't remember how the last bit was used. I think this was continued into the Lisp-machines, at least the space-cadet kbd had lots of extra qualifiers: the "normal" ctrl, the META which set the 8th bit, another CTRL which set the 9th bit, and a HYPER and SUPER which I can't remember what they set.

Robert

[Loïc Hoguin]

On 10/17/2012 01:00 AM, Richard Carlsson wrote:
> On 2012-10-17 00:29 , Richard Carlsson wrote:
>> On 2012-10-17 00:09 , Robert Virding wrote:
>>> I agree. Do it properly or get rid of it.
>>
>> On second thought, there is one advantage to documenting this
>> representation: there can be no doubt that when you make a call through
>> a tuple {m,...}:f(...), you'll call the latest version of the module. We
>> had originally planned to make pmods an opaque datatype, like funs, but
>> unlike funs we didn't want to be bound to the module version used to
>> create the instance. Keeping the representation open should make it
>> clear what it is you're getting. But then don't remove the language
>> support - make it work throughout the toolchain instead.
>
> For the record, the bad part about making a representation like this
> public is that it takes implementation details that happened to make
> sense for the underlying platform at the time (BEAM), and makes it
> impossible to change them later on. In this case, such implementation
> details are that the tuple contains exactly the module name followed by
> the hidden parameters, in order. No additional fields can be added in
> future versions. Also, the calling convention becomes fixed so that the
> module tuple itself must always be passed as the last parameter.

Now what if the tuple call support was removed and was simply replaced
in the parse transform by code similar to the following?

if is_tuple(Var) -> M = element(1, Var), M:fun();
true -> Var:fun()
end

It would still work, it would cleanup OTP completely, and I don't think
the performance loss would be too bad. After all, people caring *this
much* about performance wouldn't use tuple calls to begin with, right?

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Yang Liu]

column numbers, i like

[Håkan Huss]

On Wed, Oct 17, 2012 at 12:53 PM, Robert Virding
<robert....@erlang-solutions.com> wrote:
>
> ----- Original Message -----
>> From: "Raimo Niskanen" <raimo+erlan...@erix.ericsson.se>
>> Sent: Wednesday, 17 October, 2012 11:36:19 AM
>>
>> On Tue, Oct 16, 2012 at 03:33:54PM +0100, Ian wrote:
>> > On 16/10/2012 15:25, Raimo Niskanen wrote:
>> > >If you have Latin-1 (8-bit ASCII) source code with codes > 255 it
>> > >is safest to
>> > >add an encoding comment.
>> > I think you meant > 127 :)
>>
>> I was off by one. Thank you very much.
>>
>> >
>> > Getting a byte > 255 would be a fine trick if you could do it.
>> >
>>
>> All I need is some weird system with 9-bit bytes...
>
> The old PDP 10 had 36-bit words and in some uses (emacs) had 9-bit bytes, 4 to a word. IIRC the standard was 5 7-bit bytes but I can't remember how the last bit was used. I think this was continued into the Lisp-machines, at least the space-cadet kbd had lots of extra qualifiers: the "normal" ctrl, the META which set the 8th bit, another CTRL which set the 9th bit, and a HYPER and SUPER which I can't remember what they set.
>

And 6-bit bytes (SIXBIT) were used for e.g., file names.

Ah, the memories of trying to get a grip on what the ADJPB (adjust
byte pointer) instruction really did... :-)

/Håkan

[Richard Carlsson]

On 2012-10-17 13:01 , Loïc Hoguin wrote:
>> For the record, the bad part about making a representation like this
>> public is that it takes implementation details that happened to make
>> sense for the underlying platform at the time (BEAM), and makes it
>> impossible to change them later on. In this case, such implementation
>> details are that the tuple contains exactly the module name followed by
>> the hidden parameters, in order. No additional fields can be added in
>> future versions. Also, the calling convention becomes fixed so that the
>> module tuple itself must always be passed as the last parameter.
>
> Now what if the tuple call support was removed and was simply replaced
> in the parse transform by code similar to the following?
>
> if is_tuple(Var) -> M = element(1, Var), M:fun();
> true -> Var:fun()
> end

The above code is not equivalent with the calling convention for
parameterised modules. You need to do something like this:

Mod:Func(A1, ... AN)

becomes

if is_tuple(Mod), tuple_size(Mod) > 0 ->
(element(1, Mod))(A1, ... AN, Mod)
true ->
Mod:Func(A1, ... AN)
end

Apart from that, your point that it would still work is correct, but the
low level implementation in BEAM also caches the last looked up module
name, so that you don't have to do the full atom -> module mapping each
time you get to the same call point. Our original use case was pluggable
implementations of various things in the middle of passes in the HiPE
compiler, so performance was definitely an issue.

/Richard

[Yurii Rashkovskii]

I suspect that with your proposal the performance drop will be significant as tuple module call handling is done in C these days and according to my measurements, it's actually pretty damn fast (I think it's about in the neighbourhood of 1% of the time it takes to do the actual call)

So the loss might be significant. Also, removing or having this feature doesn't affect you — you don't use it. Why do you want to get rid of something you don't use but others do?

[Tuncer Ayaz]

On 2012-10-17 Robert Virding wrote:

> I agree. Do it properly or get rid of it.

For what it's worth I second Richard's and Robert's opinion to do

it properly or get rid of it.

> Robert
>
> ----- Original Message -----
> > From: "Richard Carlsson" <>
> > To:
> > Sent: Tuesday, 16 October, 2012 11:45:21 PM
> > Subject: Re: [erlang-questions] Pmods, packages, Unicode source
> > code and column numbers in compiler - what will happen in R16?
> >
> > On 2012-10-16 23:29 , Robert Virding wrote: > Doesn't this mean
> > that now the syntax for parametrised modules is > still there but
> > becomes meaningless? Or rather it will mean > whatever the writer
> > of a module chooses it to mean. That really > won't encourage
> > clarity. Or what am I missing?
> >
> > It also seems ass-backwards to me that the syntax will be dropped,
> > but the hack that we used for the proof-of-concept implementation
> > will be immortalized as "tuple modules". Drop the parameterized
> > modules if you like, but don't make the apply-hack a documented
> > feature; the old {M,F} was bad enough.

[Eric Merrit]

I assumed removing pmod syntax was the first step in a complete deprecation process. That is that the tuple syntax was only being retained to give folks a few releases to transition off of it. If this is the case, great! but please be explicit about it. If this is not the case, then this half way measure is probably a bad idea.

[José Valim]

I assumed removing pmod syntax  was the first step in a complete deprecation process.

The way I see it, the OTP team is removing pmod from OTP but it is still *supported* as a third-party component. So removing tuple modules would effectively disable support for the feature, since the performance costs would become prohibitive if one tries to emulate the feature using only Erlang, based on Richard Carlsson previous e-mail.

This is a long shot but if we could somehow express beam's call-site cache in Erlang Abstract Format so one could completely emulate the feature using pure Erlang, it would be easier to remove it.

Anyway, it is just my interpretation of the announcements. :)

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[José Valim]

And since we are talking about possibilities, would it be possible to support some flag or configuration that sets the error_handler per node? This way, we could remove the inheritance hack from the error_handler and developers that need it could achieve the same functionality with their own error_handler.

Today the error_handler can only be set via process_flag (afaik) which would be hard to wrap properly (since you'd need to remember to set it for each new process). This would align well with the general decision of removing pmod from OTP but still supporting it as extension.

Regardless, I am pleased with the OTP team decisions and appreciate your hard work!

[Vlad Dumitrescu]

On Wed, Oct 17, 2012 at 11:23 PM, José Valim
<jose....@plataformatec.com.br> wrote:
> And since we are talking about possibilities, would it be possible to
> support some flag or configuration that sets the error_handler per node?
> This way, we could remove the inheritance hack from the error_handler and
> developers that need it could achieve the same functionality with their own
> error_handler.
>
> Today the error_handler can only be set via process_flag (afaik) which would
> be hard to wrap properly (since you'd need to remember to set it for each
> new process). This would align well with the general decision of removing
> pmod from OTP but still supporting it as extension.

That's a great suggestion! An alternative to a global setting could be
to let the error_handler setting get inherited by child processes from
their parents. This way just a few roots need to get configured
manually and it also leaves open the possibility to have different
values for different processes.

best regards,
Vlad

[Loïc Hoguin]

On 10/17/2012 11:21 PM, José Valim wrote:
> I assumed removing pmod syntax was the first step in a complete
> deprecation process.
>
>
> The way I see it, the OTP team is removing pmod from OTP but it is still
> *supported* as a third-party component. So removing tuple modules would

I don't think it means it's supported. I think it means you can use this
and your applications will still work, but you should probably
transition to not using pmods anymore.

I assumed initially the tuple calls were going to be removed later on to
give time to people to transition.

> effectively disable support for the feature, since the performance costs
> would become prohibitive if one tries to emulate the feature using only
> Erlang, based on Richard Carlsson previous e-mail.

On benchmarks probably, on a full system I doubt it'd be that
significant. Most Erlang code doesn't use it. Of course I do know we
simple developers care more about small benchmarks than real
applications (even when we don't need them), so it'd still be prohibitive.

But that wouldn't be a bad thing. As Robert and Richard say, remove it
or do it right. Easier to remove with an alternative available (even if
a little slower), then use the clean codebase to start something new and
do it right this time, using the previous experience obtained up to this
day.

I'm sure you guys use it a lot in Elixir, and that'd surely slow Elixir
programs down, but that'd also probably make it a good test case for
doing them right.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Michael Truog]

On 10/16/2012 11:30 AM, Michael Truog wrote:
> On 10/16/2012 06:51 AM, Patrik Nyblom wrote:
>> Hi all!
>>
>> The OTP Technical board decisions from last Thursday are now published on the erlang.org website, which means that the answers to some questions about changes in R16 are finally officially answered.
>>
>> Cheers,
>> /Patrik
>>
> Very cool! These changes should help developers avoid problems during development.
>
> Could you please consider making UDP multicast support official, so it is no-longer undocumented/unsupported?

My mistake, now UDP multicast is documented, so the support must be supported/ok now. The only other thing that seems to be a nagging problem is the fact many people use prim_inet:async_accept/2 directly, which is undocumented (e.g., http://stackoverflow.com/questions/5446012/erlang-accept-incoming-tcp-connections-dynamically). It would help make development simpler if there was a documented inet function call for this.

The drama with the Pmods seems unwarranted, since it was experimental. I agree that it would be best to take it out completely, so if tuple calling will work in R16, just deprecate it in R17:
"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."
- Antoine de Saint-Exupery

[Loïc Hoguin]

On 10/18/2012 12:39 AM, Michael Truog wrote:
> The only other thing that seems to be a nagging problem is the fact many people use prim_inet:async_accept/2 directly, which is undocumented (e.g., http://stackoverflow.com/questions/5446012/erlang-accept-incoming-tcp-connections-dynamically). It would help make development simpler if there was a documented inet function call for this.

I would love to have a function to do async_accept without resorting to
using a hack or an undocumented function. It'd make my life so much
easier. Who do I need to stalk to have this? :)

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Richard O'Keefe]

On 17/10/2012, at 8:11 PM, Vlad Dumitrescu wrote:

>> On 17/10/2012, at 2:51 AM, Patrik Nyblom wrote:
>>>
>>> The OTP Technical board decisions from last Thursday are now published on the erlang.org website, which means that the answers to some questions about changes in R16 are finally officially answered.
>
> Great initiative to publish these decisions and explain them!
>
> On Wed, Oct 17, 2012 at 1:22 AM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
>>
>> "Variable names will continue to be limited to Latin characters."
>>
>> I hope that means "for this release."
>
> That's an interesting problem. Variable names are defined as starting
> with an upper case letter, but the only scripts that I know of that
> have those are roman, greek, cyrillic and armenian.

"Variables are defined as starting with an upper case letter"
isn't exactly true, unless you do what Quintus did back in the
80s and redefine "_" from being a 'punctuation connector' to
an 'upper case letter'. Quintus did that for CJK, so that 日付
was an unquoted atom and _日付 was a Prolog variable.
This was apparently acceptable, and the same practice is followed in
other Prologs. I see no reason why it would not work for Erlang,
where _1 is a perfectly good variable.

> So would variables
> with names in other scripts be forced to start with an uppercase latin
> letter? We might just as well have them start with '§' or something,
> and drop the capitalization rule.

It is _already_ the case that Erlang variables are not forced to
start with an uppercase latin letter, but may start with "_".
(The section sign is not allowed in Unicode identifiers.)
Having Latin, Greek, Cryllic, and Armenian scripts already covers a
lot of languages.

Here I am in New Zealand. There are two official languages in this country.
English isn't actually one of them, although it is in practice the language
of government, commerce, and practically everything. One of the two
official languages is New Zealand Sign Language. The other is Māori. Note
the little bar over the "a"? It's called a macron. And a-with-macron is
not a Latin-1 character. The city I'm living in has the name Dunedin in
English and Ōtepoti in Māori. (Note the macron on the "O".) The organisation
I work for is the University of Otago/Te Whare Wānanga o Otāgo. Notice a
pattern? Do I look forward to being able to tell those of my students who
are Māori that it is now possible for them to use words of their own
language as Erlang? You bet. Did I mention that although the language of
_instruction_ in this University is English, by official decree students may
submit assignments and answers to examination questions in Māori? If I ask
them to write programs in Erlang (which I did last year and will again next
year), am I actually _allowed_ to do this if they cannot use Māori words as
freely as English ones? I'd rather not find out, thanks. I'd definitely
rather not be told to require C, C++, Java, or Ada, which _do_ allow
non-Latin-1 letters in identifiers.

Unicode Standard Annex #31 (UAX 31),
'Unicode identifier and Pattern Syntax',
http://www.unicode.org/reports/tr31/
says how to handle identifiers in Unicode.
In particular, Coptic, Deseret, and Glagolitic are in table 4:
"candidate characters for exclusion from identifiers".
Section 5 recommends NFC for case-sensitive identifiers.
_ is not an ID_Start character, but if C can have that extension, so can we.

As for '§', SECTION SIGN is _not_ allowed in identifiers.

Ada 2012 identifier syntax (section 2.3) is closely based on Unicode
(technically, on ISO 10646). Let's see what they say:

identifier ::= identifier_start {identifier_start | identifier_extend}

identifier_start ::= letter_uppercase | letter_lowercase |
letter_titlecase | letter_modifier |
letter_other| number_letter

identifier_extend ::= mark_non_spacing | mark_spacing_combining |
number_decimal | punctuation_connector

An identifier shall not contain two consecutive characters in
category punctuation_connector or end with a character in that category.

Ada's restrictions on underscores (the one Latin-1 character that is a
punctuation_connector) have always been idiosyncratic. This is otherwise
pretty close to what UAX31 says.

Ada is case-insensitive, so they don't greatly care about which letters
are upper+title-case and which are not. So adapt the rules like this:

unquoted_atom ::= atom_start identifier_continuation*
atom_start ::= letter_lowercase | letter_modifier | letter_other |
number_letter | "." % the last is Erlang-specific

variable ::= variable_start identifier_continuation*
variable_start ::= letter_uppercase | letter_titlecase |
punctuation_connector % this includes "_" and some others

identifier_continuation ::= atom_start | variable_start |
number_decimal | mark_non_spacing | mark_spacing_combining |
| "@" % this is Erlang-specific

> My guess is that atoms will be allowed to contain unicode just so that
> atom_to_list and list_to_atom can still be used, but usage of such
> atoms in source code will be discouraged because these will have to be
> written as quoted.

Why _should_ Unicode identifiers that would be legal identifiers in Ada
but do not begin with an upper case letter, title case letter, or
connector punctuation mark require quoting? You might as well require
atoms containing the letter 'z' to be quoted.

[Richard O'Keefe]

On 17/10/2012, at 11:53 PM, Robert Virding wrote:
> The old PDP 10 had 36-bit words and in some uses (emacs) had 9-bit bytes, 4 to a word. IIRC the standard was 5 7-bit bytes but I can't remember how the last bit was used.

It wasn't.

[Devin Torres]

I'm very glad the OTP team has the necessary respect and foresight to
strike the balance they have regarding pmods. Rather than bowing to
pressure and uncharacteristically removing support for an oft-used
feature, I'll be heartened to see the OTP keeping a conservative and
practical approach to this.

On Wed, Oct 17, 2012 at 5:23 PM, Loïc Hoguin <es...@ninenines.eu> wrote:
> I'm sure you guys use it a lot in Elixir, and that'd surely slow Elixir
> programs down, but that'd also probably make it a good test case for doing
> them right.

And maybe one day a feature you use in Cowboy to great effect will be
decried by a small minority otherwise unbothered by such feature -- a
feature whose existence has no bearing on their everyday use -- on
which day I will make sure not to suggest you to instead do it the
"right" way.

-Devin

[Vlad Dumitrescu]

Hi Richard,

On Thu, Oct 18, 2012 at 1:34 AM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
> On 17/10/2012, at 8:11 PM, Vlad Dumitrescu wrote:
>> On Wed, Oct 17, 2012 at 1:22 AM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
>>> "Variable names will continue to be limited to Latin characters."
>>>
>>> I hope that means "for this release."
>>
>> That's an interesting problem. Variable names are defined as starting
>> with an upper case letter, but the only scripts that I know of that
>> have those are roman, greek, cyrillic and armenian.
>
> "Variables are defined as starting with an upper case letter"
> isn't exactly true, unless you do what Quintus did back in the
> 80s and redefine "_" from being a 'punctuation connector' to
> an 'upper case letter'. Quintus did that for CJK, so that 日付
> was an unquoted atom and _日付 was a Prolog variable.
> This was apparently acceptable, and the same practice is followed in
> other Prologs. I see no reason why it would not work for Erlang,
> where _1 is a perfectly good variable.

Thank you for the Unicode treatise, but sometimes one has to read
between the lines and see what the author meant to say, even if he
didn't express himself so that it couldn't be used against himself in
a court of law.

The underscore as a variable prefix is special for the compiler, as
warnings for that variable being unused are not emitted. That's why I
took a character I knew it was unused, just as an example.

Anyway, I find here (http://www.unicode.org/reports/tr31/) that "Each
programming language can define its identifier syntax as relative to
the Unicode identifier syntax, such as saying that identifiers are
defined by the Unicode properties, with the addition of “$”. By
addition or subtraction of a small set of language specific
characters, a programming language standard can easily track a growing
repertoire of Unicode characters in a compatible way.", so I see no
problems with '§'. Besides that, I would see this initial character
not as part of the identifier, but as a marker for "here comes a
variable name", just as '?' is used for macros and '#' for records.

The actual point of my note is this: there must be a way to make a
difference between atoms and variables. Some languages add a marker
before atoms, some before variables, and Erlang uses the
capitalization of the first letter. With full unicode, there is no
clear way to use that rule anymore, so I observed that an alternative
could be to do like other languages do. The exact form is mostly
irrelevant.

best regards,
Vlad

[Vlad Dumitrescu]

Hi,

On Tue, Oct 16, 2012 at 3:51 PM, Patrik Nyblom <p...@erlang.org> wrote:
> The OTP Technical board decisions from last Thursday are now published on
> the erlang.org website, which means that the answers to some questions about
> changes in R16 are finally officially answered.

If the package feature is dropped, does this mean that the associated
lexical rule for atom names is also dropped? I would like to have it
removed too, as the dot character is already overloaded and makes
lexers harder to implement.

[Loïc Hoguin]

On 10/18/2012 05:03 AM, Devin Torres wrote:
> On Wed, Oct 17, 2012 at 5:23 PM, Loïc Hoguin <es...@ninenines.eu> wrote:
>> I'm sure you guys use it a lot in Elixir, and that'd surely slow Elixir
>> programs down, but that'd also probably make it a good test case for doing
>> them right.
>
> And maybe one day a feature you use in Cowboy to great effect will be
> decried by a small minority otherwise unbothered by such feature -- a
> feature whose existence has no bearing on their everyday use -- on
> which day I will make sure not to suggest you to instead do it the
> "right" way.

Cowboy doesn't use undocumented features. That's one of the biggest
Cowboy claim. It protects both the project and its users from bad surprises.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Joe Armstrong]

Actually I rather like the "apply-hack" - Here's why.

I'll implement a counter module in two different ways:

Here's the code that uses the counter(s)

-module(test).
-compile(export_all).

test1() ->
N = counter1:make(),
N1 = counter1:bump(N, 2),
2 = counter1:read(N1).

test2() ->
N = counter2:make(),
N1 = N:bump(2),
2 = N1:read().

test1() is traditional Erlang. test2() uses the apply-hack.

test2() is shorter than test1() and there are fewer arguments to bump
all of which is good. We can't at a glance see what N and N1 (in
test2()) are but the same is true
of funs. We often have no idea what F is before we call F(...) it's
just an argument that has come from somewhere.
At least with "tuple-module" we can print them to see what they are.

The corresponding counter code is:

-module(counter1).
...
make() -> 0.
bump(N, K) -> N+K.
read(N) -> N.

and

-module(counter2).
...
make() -> {counter2, 0}.
bump(K, {counter2,N}) -> {counter2, N+K}.
read({_, N}) -> N.

It seems to me that tuple modules make for shorter client code (a good thing)
and more verbose implementations.

In certain circumstances - they are very useful.

I don't think they should be overused - but I rather like them

Cheers

/Joe


On Tue, Oct 16, 2012 at 11:45 PM, Richard Carlsson

<carlsson...@gmail.com> wrote:
> On 2012-10-16 23:29 , Robert Virding wrote:
>>
>> Doesn't this mean that now the syntax for parametrised modules is
>> still there but becomes meaningless? Or rather it will mean whatever
>> the writer of a module chooses it to mean. That really won't
>> encourage clarity. Or what am I missing?
>
>
> It also seems ass-backwards to me that the syntax will be dropped, but the
> hack that we used for the proof-of-concept implementation will be
> immortalized as "tuple modules". Drop the parameterized modules if you like,
> but don't make the apply-hack a documented feature; the old {M,F} was bad
> enough.
>
> /Richard
>

[Patrik Nyblom]

How about submitting a patch? :)

On 10/18/2012 12:42 AM, Loïc Hoguin wrote:
> On 10/18/2012 12:39 AM, Michael Truog wrote:
>> The only other thing that seems to be a nagging problem is the fact
>> many people use prim_inet:async_accept/2 directly, which is
>> undocumented (e.g.,
>> http://stackoverflow.com/questions/5446012/erlang-accept-incoming-tcp-connections-dynamically).
>> It would help make development simpler if there was a documented inet
>> function call for this.
>
> I would love to have a function to do async_accept without resorting
> to using a hack or an undocumented function. It'd make my life so much
> easier. Who do I need to stalk to have this? :)
>

[Loïc Hoguin]

It is on the todo list but not (yet) high priority. :)

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Vlad Dumitrescu]

Hi,

On Thu, Oct 18, 2012 at 1:48 PM, Joe Armstrong <erl...@gmail.com> wrote:
> Actually I rather like the "apply-hack" - Here's why.
>
> I'll implement a counter module in two different ways:
>

> test2() ->
> N = counter2:make(),
> N1 = N:bump(2),
> 2 = N1:read().
>

> -module(counter2).
> ...
> make() -> {counter2, 0}.
> bump(K, {counter2,N}) -> {counter2, N+K}.
> read({_, N}) -> N.

I would say that these tuples are actually an implementation of
passive objects - you could read them as a reference to the object's
class and its state.

They are nice for the reasons Joe and others enumerated and at least one more:
- the values are immutable

... but I see some drawbacks too:
- since the values are immutable, one has to keep track of the
"latest" value
- how would the -spec for such functions look like? How much work
is needed for the type system, dialyzer and other tools to understand
and support this?
- personally, I like more the "active objects" that processes are;
I would like to be able to use either of them interchangeably. This
would mean that N above could be a pid and we could have "N:bump(2)"
mean "N ! {bump, 2}" and "N:read()" would be "N !! read"... Is the
world ready for that? :-) There was an idea a while ago to have
messages handled in the sender's process, this way the performance
difference becomes much smaller.
- there is no way to restrict the returned value, so one could
write "bump(K, {counter2,N}) -> {not_a_counter3, N+K}." and change the
object's API [sooner or later someone will come up with a problem
where this solution will make clean, readable and super-nice code].
How would tools (and developers) keep track of which API to apply
when? This is related to the -spec question above.
- going further with solutions looking for problems, why not use
{[m1, m2, m3], State}, where the call resolution searches the modules
m1, m2 and m3 in this order, so that module "inheritance" needs not
use the error_handler hack?

My personal and humble opinion is that the most rewarding way to
explore is the one started by Joe with UBF (gosh, is it already been
10 years!?): devise a way to specify, control and handle the mesh of
processes and their interactions, both at runtime and statically. This
is what Erlang excels at and I don't see why it should try to emulate
alien features in a feeble way instead of keeping to innovate and
improve on its strenghts. I can even glimpse material for a PhD or two
:-)

On second thought, the object of my outburst above is largely
orthogonal with the original issue. One is about the language, the
other is more about the runtime. But it feels that if the larger
problem is solved, then it is probable that the other won't be
relevant anymore because there will be other, more erlangish, ways to
solve the same problem.

best regards,
Vlad

[Björn-Egil Dahlberg]

On 2012-10-18 15:11, Vlad Dumitrescu wrote:
> My personal and humble opinion is that the most rewarding way to
> explore is the one started by Joe with UBF (gosh, is it already been
> 10 years!?): devise a way to specify, control and handle the mesh of
> processes and their interactions, both at runtime and statically. This
> is what Erlang excels at and I don't see why it should try to emulate
> alien features in a feeble way instead of keeping to innovate and
> improve on its strenghts. I can even glimpse material for a PhD or two
> :-)

I strongly agree with this.

It would be nice to see further work, by many people not just a few
people at some universities, on
- on how to intuitively interact with large amount of processes,
- which paradigms/behaviours should we use to interact with clusters
of processes or groups of clusters,
- how do we hide details and illuminate control flow in our programs,
- do we really need to turn our code into OO-code to get the feeling
of abstraction or is it merely an entrance point from java, ruby, python
and c++ people,
- etc.

I say: go crazy! Do wacky stuff and present them at conferences so we
can debate them. Many things will surely be too crazy or have some
seriously flaws so it cannot be adopted. But, every once in a while we
get something really beautiful.

// Björn-Egil

[José Valim]

That's a great suggestion! An alternative to a global setting could be
to let the error_handler setting get inherited by child processes from
their parents. This way just a few roots need to get configured
manually and it also leaves open the possibility to have different
values for different processes.

best regards,
Vlad

Nice, now I prefer your alternative to mine. :)

This sounds even simpler, allows people to control the flag in a sane way and it would probably be good enough for those who need an error_handler with inheritance.

[Patrik Nyblom]

Hi!

On 10/17/2012 01:22 AM, Richard O'Keefe wrote:
> On 17/10/2012, at 2:51 AM, Patrik Nyblom wrote:
>

>> Hi all!

>>
>> The OTP Technical board decisions from last Thursday are now published on the erlang.org website, which means that the answers to some questions about changes in R16 are finally officially answered.

> "UTF-8 BOM's will not be handled due to their limited use."
>
> As I understand it, byte order marks are not *supposed*
> to be used with UTF-8, because it has no byte order issues.
Well, it's not strictly forbidden. I suppose the use is to differentiate
between different encodings, an UTF-8 BOM will tell you that it's UTF-8
and not e.g. UTF-16. But as you say there's no byte order and I have
personally never seen an editor actually writing them out, so instead of
handling a more or less theoretical BOM presence, we simply don't.

>
> "Variable names will continue to be limited to Latin characters."
>
> I hope that means "for this release."

The concept of "capital as initial character" is not applicable to all
character systems. The alternative would be "Big initial in those
languages where such exists and then any character" or something. So we
went for keeping the latin variables until someone comes up with a
*good* suggestion for this.

So it means "for now", but hopefully not "for all eternity". Sorry for
the disambiguity.
Suggestions for a good definition of a variable name in *any* script
will be greatly appreciated!
>
> An end to Java envy (otherwise known as removal of 'packages'):
>
> Good news. But let's keep thinking about ways to solve
> the problem that packages were meant to solve but didn't.
Yes, I totally agree.
>
>
>
>
Cheers,
/Patrik

[Vlad Dumitrescu]

On Thu, Oct 18, 2012 at 3:39 PM, Patrik Nyblom <p...@erlang.org> wrote:
> Suggestions for a good definition of a variable name in *any* script will be
> greatly appreciated!

Is a symbol prefix an acceptable solution? Like '?' for macros and '#'
for records. Now that we're talking unicode, there are plenty of
symbols to choose from, but of course it would be better to have one
that most people have on their keyboards :-)

'_' would be perfect, if it didn't have special meaning to the compiler...

regards,
Vlad

[Thomas Allen]

On Thu, October 18, 2012 9:46 am, Vlad Dumitrescu wrote:
> On Thu, Oct 18, 2012 at 3:39 PM, Patrik Nyblom <p...@erlang.org> wrote:
>> Suggestions for a good definition of a variable name in *any* script
>> will be
>> greatly appreciated!
>
> Is a symbol prefix an acceptable solution? Like '?' for macros and '#'
> for records. Now that we're talking unicode, there are plenty of
> symbols to choose from, but of course it would be better to have one
> that most people have on their keyboards :-)

"\" seems like a suitable prefix ;^)

In all seriousness, "@" wouldn't be too bad for a prefix -- I'd hope
people only use it when [A-Z_] are not an option, but it goes without
saying that an @ prefix will appeal to certain developers, and we begin
seeing signatures foo(@name, @size) -> ... and thus continues the rabbit
hole (@@bar à la Ruby though mutability is not a concern, etc.).

Thomas Allen

[Patrik Nyblom]

Hi!

On 10/18/2012 03:46 PM, Vlad Dumitrescu wrote:
> On Thu, Oct 18, 2012 at 3:39 PM, Patrik Nyblom<p...@erlang.org> wrote:
>> Suggestions for a good definition of a variable name in *any* script will be
>> greatly appreciated!
> Is a symbol prefix an acceptable solution? Like '?' for macros and '#'
> for records. Now that we're talking unicode, there are plenty of
> symbols to choose from, but of course it would be better to have one
> that most people have on their keyboards :-)
>
> '_' would be perfect, if it didn't have special meaning to the compiler...

You mean something like 'capital latin letter, '_' or '§', then anything
except whitespace'? Well, that could work.

I would however like something that could include e.g. cyrillic scripts
*without* needing special prefixes that we would do not need in latin
character sets. The Uppercase unicode property might be it, but then
again, languages without uppercase letters are not handled fairly.

>
> regards,
> Vlad
Cheers,
/Patrik

[Patrik Nyblom]

Hi Fred!
On 10/16/2012 04:13 PM, Fred Hebert wrote:
> Happy with all measures. I think the balance you strike for
> parametrized modules is also pretty good.
>
> Quick question -- I remember older posts relative to tuple calls about
> how their implementation somehow limited future optimizations that
> could have been possible. Does it mean their support for Pmod
> utilization means further optimizations will be cancelled?
Well, the apply instruction will need to handle it - there is an extra
branch that cannot be removed :(
>
> Otherwise, it's good to be stripping stuff from Erlang. The less
> complex, the better.
>
> Congrats on the good decisions (unless someone proves otherwise! ;) )
>
> Regards,
> Fred.

Cheers,
/Patrik

>
>
> On 12-10-16 9:51 AM, Patrik Nyblom wrote:
>> Hi all!
>>
>> The OTP Technical board decisions from last Thursday are now
>> published on the erlang.org website, which means that the answers to
>> some questions about changes in R16 are finally officially answered.
>>

[Patrik Nyblom]

Hi (again) Fred!

On 10/16/2012 04:36 PM, Fred Hebert wrote:
> Additional question regarding column numbers and future UTF8 support:
> will the column number be based on byte length, grapheme clusters
> (what I would usually use when typing text in a module), include
> combining characters?
The scanner uses the utf8 encoding of files as it looks now, so it has
no concept of bytes. Which means grapheme clusters. I think that's the
only sensible approach.

The combining characters... Hmmm... We could translate them to their
proper code points too, but we only do that for file names now... Let me
get back to you on that one :)
>
> I think we've had many discussions regarding how difficult the idea of
> a length or position in a string can be to figure out -- and I know
> the unicode consortium makes its own recommendation there. I'm just
> curious which is going to be followed.

[Yurii Rashkovskii]

Patrik,

 

> how their implementation somehow limited future optimizations that
> could have been possible. Does it mean their support for Pmod
> utilization means further optimizations will be cancelled?
Well, the apply instruction will need to handle it - there is an extra
branch that cannot be removed :(

 

According to my interpretation of my (totally unscientific) performance test this branch accounts for ~1.35% of the total call time. In my books, that's fairly insignificant.

Yurii.

[Anthony Ramine]

Le 18 oct. 2012 à 16:39, Patrik Nyblom a écrit :

> The combining characters... Hmmm... We could translate them to their proper code points too, but we only do that for file names now... Let me get back to you on that one :)

Well, not all combining characters sequences have an equivalent code point so what would be the point of that?

--
Anthony Ramine

[Devin Torres]

On Thu, Oct 18, 2012 at 3:45 AM, Loïc Hoguin <es...@ninenines.eu> wrote:
> Cowboy doesn't use undocumented features. That's one of the biggest Cowboy
> claim. It protects both the project and its users from bad surprises.

Missing the point, but okay. The point is: not removing the feature
affects nobody different today. You can continue to ignore it forever
and it will never make a difference to your daily life. Removing the
feature adversely affects some.

Instead of removing features, why don't we let the OTP team focus on
adding features that will make the lives of everyday Erlang
programmers better? Like frames. :)

-Devin

[Loïc Hoguin]

On 10/18/2012 05:40 PM, Devin Torres wrote:
> On Thu, Oct 18, 2012 at 3:45 AM, Loïc Hoguin <es...@ninenines.eu> wrote:
>> Cowboy doesn't use undocumented features. That's one of the biggest Cowboy
>> claim. It protects both the project and its users from bad surprises.
>
> Missing the point, but okay. The point is: not removing the feature
> affects nobody different today. You can continue to ignore it forever
> and it will never make a difference to your daily life. Removing the
> feature adversely affects some.

Didn't miss it, just ignored it. The point makes no sense.

Why would I want something that I consider bad practice to stay in a
language I want to improve?

> Instead of removing features, why don't we let the OTP team focus on
> adding features that will make the lives of everyday Erlang
> programmers better? Like frames. :)

Or, why not remove the bad features to allow them to spend more time on
adding good features instead of forever maintaining bad ones?

Plus that's not removing a feature, that's removing an experiment gone bad.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Richard Carlsson]

On 2012-10-18 17:48 , Loïc Hoguin wrote:
> Or, why not remove the bad features to allow them to spend more time on
> adding good features instead of forever maintaining bad ones?
>
> Plus that's not removing a feature, that's removing an experiment gone bad.

I must say, I'm still wondering why you (and others) consider
parameterized modules an "experiment gone bad". Most of the criticism
I've seen has been about the implementation (which was just intended as
a proof of concept), i.e., tuples instead of a separate datatype, no
additional support in tracing makes debugging hard, and so on. Those
things could be fixed with a proper implementation. But the current OTP
proposal is to keep only the crappy part of it, which makes very little
sense to me. And if you dislike parameterized modules, do you dislike
funs as well? They're both just closures.

/Richard

[Loïc Hoguin]

On 10/18/2012 06:09 PM, Richard Carlsson wrote:
> On 2012-10-18 17:48 , Loïc Hoguin wrote:
>> Or, why not remove the bad features to allow them to spend more time on
>> adding good features instead of forever maintaining bad ones?
>>
>> Plus that's not removing a feature, that's removing an experiment gone
>> bad.
>
> I must say, I'm still wondering why you (and others) consider
> parameterized modules an "experiment gone bad". Most of the criticism
> I've seen has been about the implementation

Yes, we dislike the implementation.

I said "gone bad" as in "got used in real products", it evolved from an
honest experimentation into something that we have to deal with
depending on what dependencies we need.

I think a proper implementation would be great, I am especially
intrigued by functors which from what I understand would be made
possible with a proper pmod implementation.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Yurii Rashkovskii]

Loïc,

> I must say, I'm still wondering why you (and others) consider
> parameterized modules an "experiment gone bad". Most of the criticism
> I've seen has been about the implementation

Yes, we dislike the implementation.

Can you please refrain from speaking for everybody?

Thanks,

Yurii. 

[Loïc Hoguin]

I'm speaking for "you (and others)", the people who consider the current
parameterized modules to be wrong.

You weren't included, obviously.

[Yurii Rashkovskii]

Loïc,

>> Yes, we dislike the implementation.
>> Can you please refrain from speaking for everybody?
>
> I'm speaking for "you (and others)", the people who consider the current
> parameterized modules to be wrong.
>
> You weren't included, obviously.

I did not mean I consider myself included into that particular "we".
What I meant is "let everybody speak for themselves", including people
from the anti-tuple module calls camp.

Those who need, will make their own observations and conclusions.

Yurii.

[Loïc Hoguin]

Those who feel misrepresented can interject. There's a pretty good
consensus in that camp AFAIK, though.

[Richard Carlsson]

On 2012-10-18 18:18 , Loïc Hoguin wrote:
> I think a proper implementation would be great, I am especially
> intrigued by functors which from what I understand would be made
> possible with a proper pmod implementation.

When I say "proper implementation", I simply mean a separate opaque
datatype (much like funs) for module instances, and support throughout
the ecosystem for tracing and debugging. Apart from that, I think the
existing syntax and semantics of parameterized modules is not lacking
anything (beyond some simple additions like static-declared functions).
Could you be more exact with what you refer to by "functors", because
that's a quite fuzzy concept. ML functors, for example, are very static
in nature and are more akin to C++ templates in the way they are
expanded at compile time. I certainly don't think that is desirable in a
language like Erlang.

/Richard

[Loïc Hoguin]

On 10/18/2012 08:42 PM, Richard Carlsson wrote:
> On 2012-10-18 18:18 , Loïc Hoguin wrote:
>> I think a proper implementation would be great, I am especially
>> intrigued by functors which from what I understand would be made
>> possible with a proper pmod implementation.
>
> When I say "proper implementation", I simply mean a separate opaque
> datatype (much like funs) for module instances, and support throughout
> the ecosystem for tracing and debugging. Apart from that, I think the
> existing syntax and semantics of parameterized modules is not lacking
> anything (beyond some simple additions like static-declared functions).

Opaque, no discrepencies between function arities, etc. Fred listed
quite well the issues, although the two I just cited are my bigger concerns.

I believe we are on the same tracks there.

> Could you be more exact with what you refer to by "functors", because
> that's a quite fuzzy concept. ML functors, for example, are very static
> in nature and are more akin to C++ templates in the way they are
> expanded at compile time. I certainly don't think that is desirable in a
> language like Erlang.

I'll have to get back to you on that after I get re-explained how it
would work, it's been a while and things have gotten vague about it,
only the good feeling remained. :)

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Richard O'Keefe]

On 18/10/2012, at 8:52 PM, Vlad Dumitrescu wrote:
> Thank you for the Unicode treatise, but sometimes one has to read
> between the lines and see what the author meant to say, even if he
> didn't express himself so that it couldn't be used against himself in
> a court of law.
>
> The underscore as a variable prefix is special for the compiler, as
> warnings for that variable being unused are not emitted. That's why I
> took a character I knew it was unused, just as an example.

The underscore as a variable prefix is not _that_ special for the
compiler. Again, this is no new thing. Prolog did the same thing.
You could perfectly well have a policy that

_<Latin 1 letter or digit>...
compiler does not warn about singleton use
_<extended letter>...
compiler DOES warn about singleton use
__<extended letter>...
compiler does NOT warn about singleton use

In any case, if you adopt Unicode identifier syntax, a whole
bunch of extra characters become available:
U+203F ‿
U+FE34 ︴
U+FE4D ﹍
U+FE4E ﹎
U+FE4F ﹏
among them. We could perfectly well say that an identifier beginning
with a Pc character is a variable, which would generalise the current
"_" rule, and the compiler would _not_ be treating those specially.

>
> Anyway, I find here (http://www.unicode.org/reports/tr31/) that "Each
> programming language can define its identifier syntax as relative to
> the Unicode identifier syntax,

Right. Already noted, which is why we can continue to allow '@' and '.'

> The actual point of my note is this: there must be a way to make a
> difference between atoms and variables.

We have one right now:
A variable begins with a character that is in
(Lu | Lt | Pc) & Latin1.
All we do is remove the restriction to Latin1. Done!

> Some languages add a marker
> before atoms, some before variables, and Erlang uses the
> capitalization of the first letter. With full unicode, there is no
> clear way to use that rule anymore,

Yes there is. Keep the existing rule exactly as it stands except
for removing the restriction to Latin1. Using a script with case?
(*LOTS* of languages.) You can use an Lu or Lt character. Using
a script without? You can use any Pc character you like. "‿" is
kind of cute.

> so I observed that an alternative could be to do like other languages do.

We don't _need_ an alternative. Once we break the Latin1 boundary
we have more underscore-like characters to work with, and if you
want to think of them as funny prefixes, good luck to you and
‿日付 will work, but if you want to think of them as differently
shaped underscores, then ‿日付‿今日 will work too (if anyone _wants_ it
to, which is of course another matter).

if you want

[Richard O'Keefe]

On 18/10/2012, at 8:55 PM, Vlad Dumitrescu wrote:
>
> If the package feature is dropped, does this mean that the associated
> lexical rule for atom names is also dropped? I would like to have it
> removed too, as the dot character is already overloaded and makes
> lexers harder to implement.

Dots were allowed in atom names long before packages were dreamed of.
The idea is that main...@frege.otago.ac.nz can be an atom without
needing quotes.

I've written a couple of lexical analysers for Erlang, and it's not
_that_ hard. Leaving @ out of the discussion,
variable = [_[:upper:]][_[:alnum:]]*
atom = [.]?[[:lower:]]([[:digit:]]|[_[:upper:]]|[.]?[[:lower:]])*
with the added weirdness that a leading "." is discarded.

[Michael Richter]

On 18 October 2012 23:40, Devin Torres <de...@devintorr.es> wrote:

Instead of removing features, why don't we let the OTP team focus on
adding features that will make the lives of everyday Erlang
programmers better? Like frames. :)

Because this is how you get kitchen sink language ecosystems like C++, Perl, Java, C#, etc.

--
"Perhaps people don't believe this, but throughout all of the discussions of entering China our focus has really been what's best for the Chinese people. It's not been about our revenue or profit or whatnot."
--Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra.

[Michael Richter]

On 19 October 2012 12:29, Devin Torres <de...@devintorr.es> wrote:

On Thu, Oct 18, 2012 at 6:40 PM, Michael Richter <ttmri...@gmail.com> wrote:
> Because this is how you get kitchen sink language ecosystems like C++, Perl,
> Java, C#, etc.

All wildly successful ecosystems and languages way more popular than Erlang.

-Devin

Lord preserve us from Java's style of "success".

[Richard O'Keefe]

If it were still possible to submit EEPs in plain text,
this would be an EEP. If someone else would like to
package this up as an EEP and submit it (under their
name, mine, or both), feel free.

Forces:
(1) Support for Unicode continues to increase, with
minimal source code support about to arrive.
(2) Unicode variable names and unquoted atoms are not
here yet, so now is the time to settle on a design.
(3) They will need to come. There may be legal or
institutional reasons why unicode-capable languages
are required. Some people just want to use their
own language and script. Erlang's strength in
network applications means that being able to
represent Internationalized Domain Names as unquoted
atoms would be just as much of a convenience as
being able to represent ASCII domain names like
www.example.com (which needs no quotes in Erlang) is.
(4) There is a framework for Unicode identifiers in
Unicode standard annex 31 (UAX#31), and several
programming languages, including Ada, Java,
C++, C, C#, Javascript, and Python (section 2.3 of
http://docs.python.org/release/3.1.5/reference/lexical_analysis.html
and see also http://www.python.org/dev/peps/pep-3131/
(5) Existing Erlang identifiers should remain valid,
including ones containing "@" and ".".
(6) Existing Erlang support features, such as ignoring
names of the form [_][a-zA-Z0-9_]* when reporting
singleton variables, should not be broken.
(7) We should not "steal" any characters to use as "magic
markers" for variables because they might be needed for
other purposes. A good (bad) example of this is "?", which
could be used for several things if it were not used for macros.

Reference

Names of sets of characters, XID_Start, XID_Continue, Lu, Lt, Lo, Pc,
Other_Id_Start, are drawn from Unicode and UAX#31.

Lu = upper case letters
Lt = title case letters
Pc = connector punctuators, including the low line (_) and
a number of other characters like undertie (‿).
Other_Id_Start = script capital p, estimated symbol,
katakana-hiragana voiced sound mark, and
katakana-hiragana semi-voiced sound mark.

Variables

variable ::= var_start var_continue*

var_start ::= XID_Start ∩ (Lu ∪ Lt ∪ Pc ∪ Other_Id_Start)

var_continue ::= XID_Continue U "@"

The choice of XID here follows Python. It ensures that the normalisation
of a variable is still a variable. In fact Unicode variables should be
normalised. Unicode has enough look-alike characters that we cannot hope
for "look the same <=> are the same" to be true, but we should go _some_
way in that direction.

Variables in scripts that do not distinguish letter case have to
begin with _some_ special character to ensure that they are not
mistaken for unquoted atoms. There are 10 Pc characters in the Basic
Multilingual Plane. The Erlang parser treats a variable beginning
with an underscore specially: there will be no complaint if it is a
singleton. There are 9 other Pc characters for which this special
treatment is not applied. Of course, someone might be using fonts
that do include say Arabic letters but not say the undertie. We can
deal with that by revising the underscore rule.

Variable does not begin with a Pc character =>
should not be a singleton.

Variable is just a Pc character and nothing else =>
is a wild card.

Variable begins with a Pc character followed by a
Latin-1 character =>
may be a singleton.

Variable begins with a Pc character following by
a character outside the Latin-1 range =>
should not be a singleton.

Thus ‿ is a wild-card, 隠者 is an atom, _隠者 should not be
a singleton, but __隠者 _may_ be a singleton. This rule is a
consistent generalisation of the existing rule.

Unquoted atoms

unquoted_atom ::= atom_start atom_continue

atom_start ::= XID_Start \ (Lu ∪ Lt ∪ Lo ∪ Pc)
| "." (Ll ∪ Lo)

atom_continue ::= XID_Continue U "@"
| "." (Ll ∪ Lo)

Again the choice of XID follows Python, and ensures that the
normalisation of an unquoted atom is still an unquoted atom.
Unquoted atoms should be normalised.

The details of Erlang unquoted atoms are somewhat subtle; I have
checked my understanding experimentally.

Keywords

Keywords have the form of unquoted atoms. No new keywords are
introduced.

Specifics

- Any Python identifier or keyword is
an Erlang variable or unquoted atom or keyword.

- @ signs may occur freely in variables and unquoted atoms except as the
first character, as now.

- dots may not be followed by capital letters, digits, or underscores,
as now.

- I am not sure whether modifier letters should be allowed after a dot.

- I am not sure what to do with the Other_ID_Start characters.
Script capital p _looks_ like a capital p and even has "capital" in
its name. All other "* SCRIPT CAPITAL *" characters are upper case
letters. Surely it should be allowed to start a variable.
The estimated sign looks like an enlarged lower case e; other symbols
that look like letters are classified as letters. You'd expect this
to begin an atom. As for the Katakana-Hiragana voicing marks, I have
no intuition whatever. Assigning the whole group to atoms seems
safest.

- All existing variable names and unquoted atoms remain legal, and no
new variable or atom forms using only Latin-1 characters have been
introduced.

Trouble spot

[Vlad Dumitrescu]

On Thu, Oct 18, 2012 at 11:50 PM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
>
> On 18/10/2012, at 8:55 PM, Vlad Dumitrescu wrote:
>>
>> If the package feature is dropped, does this mean that the associated
>> lexical rule for atom names is also dropped? I would like to have it
>> removed too, as the dot character is already overloaded and makes
>> lexers harder to implement.
>
> Dots were allowed in atom names long before packages were dreamed of.
> The idea is that main...@frege.otago.ac.nz can be an atom without
> needing quotes.

Right, I forgot about that. A problem is that this solution only goes
half way, because as per http://tools.ietf.org/html/rfc1123#page-13
labels in host names can start with digits and foo.3net.com as an atom
has to be quoted, as have IP addresses, but I guess we have to live
with it.

> I've written a couple of lexical analysers for Erlang, and it's not
> _that_ hard. Leaving @ out of the discussion,
> variable = [_[:upper:]][_[:alnum:]]*
> atom = [.]?[[:lower:]]([[:digit:]]|[_[:upper:]]|[.]?[[:lower:]])*
> with the added weirdness that a leading "." is discarded.

True, but if the "dot within atom" rule would have become unnecessary,
there would be no reason to have lexing rules more complicated than
needed. The dot can have now three lexical meanings, float decimal
point, atom "separator" and full stop, and upon seeing "2." we can't
decide which case it is without further input.

best regards,
Vlad

[Jesper Louis Andersen]

On Oct 18, 2012, at 8:42 PM, Richard Carlsson <carlsson...@gmail.com> wrote:

> When I say "proper implementation", I simply mean a separate opaque datatype (much like funs) for module instances, and support throughout the ecosystem for tracing and debugging. Apart from that, I think the existing syntax and semantics of parameterized modules is not lacking anything (beyond some simple additions like static-declared functions). Could you be more exact with what you refer to by "functors", because that's a quite fuzzy concept. ML functors, for example, are very static in nature and are more akin to C++ templates in the way they are expanded at compile time. I certainly don't think that is desirable in a language like Erlang.
>

I would absolutely *love* having a way to do ML-style functors in Erlang. I would like to have them as a static system - but my history is probably haunting here. But, to do this you probably need first-class modules as well,
which is nice for a dynamic language.

Jesper Louis Andersen
Erlang Solutions Ltd., Copenhagen

[Robert Virding]

----- Original Message -----
> From: "Vlad Dumitrescu" <vlad...@gmail.com>
> Sent: Thursday, 18 October, 2012 3:46:16 PM
>
> On Thu, Oct 18, 2012 at 3:39 PM, Patrik Nyblom <p...@erlang.org>
> wrote:
> > Suggestions for a good definition of a variable name in *any*
> > script will be
> > greatly appreciated!
>
> Is a symbol prefix an acceptable solution? Like '?' for macros and
> '#'
> for records. Now that we're talking unicode, there are plenty of
> symbols to choose from, but of course it would be better to have one
> that most people have on their keyboards :-)
>
> '_' would be perfect, if it didn't have special meaning to the
> compiler...

Variables starting with '_' are completely normal variables and can be used as such. It is the compiler which assumes a special use *for handling warnings* and only for this. You can quite happily begin all your variables with '_' and everything will work just fine. And it will keep the compiler quiet. The variable in the language which has special meaning is the variable _, just _ .

Robert

[Robert Virding]

----- Original Message -----
> From: "Devin Torres" <de...@devintorr.es>
> Sent: Thursday, 18 October, 2012 5:40:15 PM
>
> On Thu, Oct 18, 2012 at 3:45 AM, Loïc Hoguin <es...@ninenines.eu>
> wrote:
> > Cowboy doesn't use undocumented features. That's one of the biggest
> > Cowboy
> > claim. It protects both the project and its users from bad
> > surprises.
>
> Missing the point, but okay. The point is: not removing the feature
> affects nobody different today. You can continue to ignore it forever
> and it will never make a difference to your daily life. Removing the
> feature adversely affects some.
>
> Instead of removing features, why don't we let the OTP team focus on
> adding features that will make the lives of everyday Erlang
> programmers better? Like frames. :)

If you don't remove unused/experimental features then your language will slowly accumulate crud and eventually become a right mess of "features". I haven't got quite as far as removing something every time you add something but close. I think the problem is that people seem to misunderstand the meaning of "experimental" and expect them to remain even if the experiment fails.

Robert

[Dmitry Belyaev]

What is the point in the experiment if not to allow people use it?

I see pmods as a way to handle whole abstraction as one object in the language.

For example, in cowboy it is required to store socket and the appropriate protocol module. 2 objects that cannot work with some another protocol.
Other libraries create its own absractions to handle the socket as some abstract socket that encapsulates underlying protocol like lhttpc_sock, ejabberd_socket and I suppose a lot more.

And pmods might become a standard way to handle such cases.

--
Dmitry Belyaev

[Vlad Dumitrescu]

On Fri, Oct 19, 2012 at 8:06 AM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
> If it were still possible to submit EEPs in plain text,
> this would be an EEP.

Great initiative, Richard! A formal definition of what is to be
expected is the best way to eliminate misunderstandings. Even if it's
not going to be an EEP, it's a solid base for discussion.

However, I have some second thoughts about this... Does anyone has an
example of a project that is not a toy or private, in any language
that allows unicode identifiers (Java, Ruby, etc), that actually uses
any of those? There are issues with interoperability in a multi-script
environment. Today, Erlang identifiers are Latin-1, but the only times
I've seen using non-ASCII characters they had been entered by mistake.

It would be wasteful to spend a lot of work and time [*] for features
that very people might use, and then they won't be able to cooperate
with most of the other Erlang developers. Erlang is known as a
pragmatic language, maybe there is a pragmatic line to be drawn here
too. If it shows that only few people will use unicode outside
comments, strings and binary literals, then maybe it's not such a big
deal to have a rule to always enclose unicode atoms in simple quotes
and have unicode variables start with an underscore...

Or maybe it is, and then I say to those that are affected: please let us know.

best regards,
Vlad

[Robert Virding]

Yes, I quite agree. You DO need to experiment with a language, especially with a language like Erlang which has always had a very pragmatic point of view and was developed through evolution. No creation here.

I see that there are two (at least) problems here:

1. People do not realise, or want to realise, that experiments are just experiments and they might be deemed to have failed and be removed. Even if I love them enough other people might not so they disappear. Going hard-line here I would say "tough, that's YOUR problem, we told you it was an experiment".

2. In this case I think they did the wrong thing. However you look at it pmods were the real "thing" being tested and the tuple modules were were just a quick hack to a allow you to test the idea; they weren't, or shouldn't have been considered to have been, a separate thing in their own right. So the choice should have been: pmods yes or no? If no then throw them AND their hack test implementation away. If yes, then keep them and do them properly by creating an opaque data type with a set of well defined ways of interacting with it, and throw away the hack test implementation.

This is how it was done with funs, they originated as a tuple with undefined "stuff" in them and later they were changed to an opaque data type. I don't know if anyone ever seriously tried to use the internal structure of the tuple but if they did they learned the hard way. IMAO this is what should have been with pmods.

I will admit that I would probably not be popular if I did this. But I do believe that this is what you have to do in the long run. Some people on the bleeding edge will get burned, but that is the price you pay for being there. Otherwise you do accumulate crud in your language. And I do see tuple modules as crud while pmods done properly wouldn't have been.

Is there anyone I have managed NOT to rile?

Robert

[Henning Diedrich]

Hi Richard,

I don't concur with the motion but here you are, with compliments, my attempt at mark-downing your proposal.

I don't second it because it always ended in tears somewhere else in the compile chain when I dared using Umlauts where it was allowed.

I suspect that you are not burned by those earlier attempts to shake ASCII, in those cases for the valid reason to leave crippled names (replacing Umlauts with ASCII characters) behind.

I did not do this: "Send your EEP submission to the EEP editors <ee...@erlang.org>"

I left a note on markdown and on-top EEP requirements in the text below, for your convenience.

See attachment for a HTML version that I get using markdown.lua ( http://www.frykholm.se/files/markdown.lua ). As you pointed out before, markdown is a trial and error thing and your results, and those of the OTP team, may vary. But it's a start I guess.

Best,
Henning

Begin EEP (this line is not part of it):
Author: Richard O'Keefe <ok(at)cs(dot)otago(dot)ac(dot)nz>
Status: Draft
Type: Standards Track
Created: 19-Oct-2012
Erlang-Version: R16B
Post-History: 19-Oct-2012
Replaces:
****
EEP XXX: Unicode Variable And Atom Names
----



Abstract
========

Variable and atom names should be allowed to use any Unicode characters
instead of only Latin-1 characters.


Forces
------

1. Support for Unicode continues to increase, with

minimal source code support about to arrive.

2. Unicode variable names and unquoted atoms are not

here yet, so now is the time to settle on a design.

3. They will need to come. There may be legal or

institutional reasons why unicode-capable languages
are required. Some people just want to use their
own language and script. Erlang's strength in
network applications means that being able to
represent Internationalized Domain Names as unquoted
atoms would be just as much of a convenience as
being able to represent ASCII domain names like
www.example.com (which needs no quotes in Erlang) is.

4. There is a framework for Unicode identifiers in

Unicode standard annex 31 (UAX#31), and several
programming languages, including Ada, Java,
C++, C, C#, Javascript, and Python (section 2.3 of

[Python's Lexical Analysis][PyLex] and see also
[PEP 3131][]).

5. Existing Erlang identifiers should remain valid,

including ones containing "@" and ".".

6. Existing Erlang support features, such as ignoring
names of the form \[\_][a-zA-Z0-9\_]* when reporting

singleton variables, should not be broken.

7. We should not "steal" any characters to use as "magic

markers" for variables because they might be needed for
other purposes. A good (bad) example of this is "?", which
could be used for several things if it were not used for macros.

Rationale
=========

Names of sets of characters, XID\_Start, XID\_Continue, Lu, Lt, Lo, Pc,
Other\_Id\_Start, are drawn from Unicode and UAX#31.

Lu = upper case letters
Lt = title case letters
Pc = connector punctuators, including the low line (_) and
a number of other characters like undertie (‿).
Other_Id_Start = script capital p, estimated symbol,
katakana-hiragana voiced sound mark, and
katakana-hiragana semi-voiced sound mark.


Variables

---------

a singleton, but \_\_隠者 \_may\_ be a singleton. This rule is a

consistent generalisation of the existing rule.

Unquoted Atoms
--------------

unquoted_atom ::= atom_start atom_continue

atom_start ::= XID_Start \ (Lu ∪ Lt ∪ Lo ∪ Pc)
| "." (Ll ∪ Lo)

atom_continue ::= XID_Continue U "@"
| "." (Ll ∪ Lo)

Again the choice of XID follows Python, and ensures that the
normalisation of an unquoted atom is still an unquoted atom.
Unquoted atoms should be normalised.

The details of Erlang unquoted atoms are somewhat subtle; I have
checked my understanding experimentally.


Keywords

--------

Keywords have the form of unquoted atoms. No new keywords are
introduced.


Specifics

---------

- Any Python identifier or keyword is
an Erlang variable or unquoted atom or keyword.

- @ signs may occur freely in variables and unquoted atoms except as the
first character, as now.

- dots may not be followed by capital letters, digits, or underscores,
as now.

- I am not sure whether modifier letters should be allowed after a dot.

- I am not sure what to do with the Other\_ID\_Start characters.

Script capital p _looks_ like a capital p and even has "capital" in
its name. All other "* SCRIPT CAPITAL *" characters are upper case
letters. Surely it should be allowed to start a variable.
The estimated sign looks like an enlarged lower case e; other symbols
that look like letters are classified as letters. You'd expect this
to begin an atom. As for the Katakana-Hiragana voicing marks, I have
no intuition whatever. Assigning the whole group to atoms seems
safest.

- All existing variable names and unquoted atoms remain legal, and no
new variable or atom forms using only Latin-1 characters have been
introduced.

Edit Notes
==========

For convenience, quoting [EEP 33], the EEP markdown template:

See the [Markdown][] Syntax for general formatting syntax. *On top* of
this Markdown EEPs has these requirements:

You must adhere to the Emacs convention of adding two spaces at the
end of every sentence. You should fill your paragraphs to column 70,
but under no circumstances should your lines extend past column 79.
If your code samples spill over column 79, you should rewrite them.

Tab characters must never appear in the document at all.

When referencing an external web page in the body of an EEP, you
should include the title of the page in the text, with a footnote
reference to the URL. Do not include the URL in the body text of the
EEP. E.g:

Refer to the [Erlang Language web site][1] for more details.

:

[1]: http://www.erlang.org
"Erlang Programming Language"

Footnote references ... are invisible in the [Markdown][]
generated HTML.

[PyLex]: http://docs.python.org/release/3.1.5/reference/lexical_analysis.html
"2. Lexical Analysis - Python 3.1.5 Documentation"

[PEP 3131]: http://www.python.org/dev/peps/pep-3131/
"PEP 3131 -- Supporting Non-ASCII Identifiers"

[EEP 33]: eep-0033.md
"Sample Markdown EEP Template"

[Markdown]: http://daringfireball.net/projects/markdown/
"Markdown Home Page"



Copyright
=========

*Pending the author's acknowledgement:*

This document has been placed in the public domain.



[EmacsVar]: <> "Local Variables:"
[EmacsVar]: <> "mode: indented-text"
[EmacsVar]: <> "indent-tabs-mode: nil"
[EmacsVar]: <> "sentence-end-double-space: t"
[EmacsVar]: <> "fill-column: 70"
[EmacsVar]: <> "coding: utf-8"
[EmacsVar]: <> "End:"

:End of EEP (this line is not part of it)

[Raimo Niskanen]

I have converted the text below into EEP 40 and enlisted it.

http://www.erlang.org/eeps/eep-0040.html

It would be nice if Richard could check it and see if anything
got lost in translation. Especially the odd line "Trouble Spot"
is not supposed to be there, I guess, but I kept it.

/ Raimo Niskanen, as EEP editor

--

/ Raimo Niskanen, Erlang/OTP, Ericsson AB

[fr...@circlewave.net]

On Thu, Oct 18, 2012 at 06:09:39PM +0200, Richard Carlsson wrote:
> Those things could be fixed with a proper implementation. But the
> current OTP proposal is to keep only the crappy part of it, which
> makes very little sense to me.

Yeah, that "tuple module" thing is really bizarre... would be interesting
to hear the justification for keeping it (I mean, anything besides "provide
low-level hack that can be used to support pmods"?).

> And if you dislike parameterized modules, do you dislike funs as well?
> They're both just closures.

Not to sound needlessly confrontational, but this strikes me as a bit far
fetched.

First class functions have syntactic support in the language and cooperate
seemlessly with lexical scoping, all nice and no surprises. Parametrized
modules on the other hand have their environment filled in from outside by
an act of magic, distort function arities invisibly by introducing quite
weird calling convention, and you can't even call them closures because,
module being the toplevel scope, there just isn't anything to close over.
Oh, and module environments don't really support variables, just functions,
that's another strangeness.

If you argue these are unfortunate aspects of the implementation then I'd
be honestly curious to see what design is this implementation implementing
then, just so we're on the same page. Presumably the idea was to turn
modules into proper first class objects? That sounds like worthwhile
research effort (despite all the atrocities people managed to inflict
with pmods); it could be nice to be able to write small supervisors and
gen_servers "inline".

But can this be done without basically introducing syntax for an expression
that evaluates to module-thing, and which conforms to the usual lexical
scoping rules, similar I think to what 'object' stuff works like in OCaml?
Maybe I'm just lacking imagination to see other sensible approaches?

BR,
-- Jachym

[Yurii Rashkovskii]

Richard,

Please excuse my ignorance, but can you name a single good reason for non-latin atoms and variable names? From my personal point of view, this is a sure road to hell.

How would you read these pieces of code:

Довж1 = length(Сп1)

[Г|Х]

?

Isn't it a blessing that we all are using a fairly short and commonly known alphabet and are able to communicate with each other, collaborate on open source projects, etc.?

Also, with regards to Unicode support, isn't the most important problem in handling external strings — i.e. data your system receives from outside?

Thanks,

Yurii.

[Anthony Ramine]

A good reason is mathematical symbols. If I read a functional pearl
which defines a function δ, I do want to name it as such.

Also non-latin users who don't know English should be able to use
atoms and variables they understand.

On Mon, Oct 22, 2012 at 7:08 AM, Yurii Rashkovskii <yra...@gmail.com> wrote:
> Richard,
>
> Please excuse my ignorance, but can you name a single good reason for
> non-latin atoms and variable names? From my personal point of view, this is
> a sure road to hell.

[Yurii Rashkovskii]

If you need mathematical symbols, I suppose you're more in a Haskell
land. Besides, I don't even know how to type δ in (besides
copy-pasting).

I mean, realistically, do we need unfamiliar characters to start
appearing in somebody's code who thinks it's a smart idea to do so?
What happened to the principle of the least astonishment?

Non-latin users who don't know English somehow manage to use english
function names and keywords. It's a slippery slope to suggest we need
to cater to everybody's wish to program in their native language.
Isn't English a lingua franca of software development that allows us
to communicate better? If it wasn't, think about how emptier the space
would have been.

[Michael Richter]

On 22 October 2012 13:08, Yurii Rashkovskii <yra...@gmail.com> wrote:

Please excuse my ignorance, but can you name a single good reason for non-latin atoms and variable names? From my personal point of view, this is a sure road to hell.

Because people frequently like to work in their own language instead of a foreign language they ill understand?

Jesus!  How can so many smart people be so god-damned dumb over this issue?

[Rustom Mody]

On Fri, Oct 19, 2012 at 11:36 AM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:

If it were still possible to submit EEPs in plain text,
this would be an EEP.  If someone else would like to
package this up as an EEP and submit it (under their
name, mine, or both), feel free.

Forces:
 (1) Support for Unicode continues to increase, with
     minimal source code support about to arrive.
 (2) Unicode variable names and unquoted atoms are not
     here yet, so now is the time to settle on a design.

On Mon, Oct 22, 2012 at 10:38 AM, Yurii Rashkovskii <yra...@gmail.com> wrote:

Richard,

Please excuse my ignorance, but can you name a single good reason for non-latin atoms and variable names? From my personal point of view, this is a sure road to hell.

As an Erlang noob maybe I should wait a bit before I open my mouth.
Please forgive the impertinence!

1.
Python made a choice to embrace unicode more thoroughly in going from python 2 to python 3.  This seems to have caused some grief in that 'ASCII' code that used to work in python 2 now often does not in python 3. Maybe this has nothing to do with Richard's EEP because that is about the string data structure this is about variable names. Still just mentioning.

2.
I know at least one person who is working on a programming language for Indian languages.  The idea is to go beyond mere use of Indic scripts to using keywords and language constructs that respect typical forms of conjugation used in Indian languages.

3.
There is one small subset of unicode that is more universal than the languages/scripts part, viz the math symbols.
Think of replacing
fun with λ
=:= with ≑
<- with ←
-> with →
<0.51.0> with ⟦0.51.0⟧

and much else

Am I being serious about all this?

Not quite :-)

All I am saying is that we have got so used to the imprisonment of Ascii that we find it hard to imagine a little more freedom.
[Just a historical note: When Unix first came out, some of the old-schoolers turned up their noses: WHO NEEDS LOWER CASE? ]

In all fairness (for Yurii's points) I should mention:
1. I was typing this on a windows box and could not see the characters until I switched to linux
2. Our computers may become completely, effortlessly unicode-capable someday, our keyboards will never. So to the extent that code is meant to be written, ASCII will always trump.  To the extent that it is to be read, a richer (within limits) character set has its attractions.
3. I learnt Apl in my youth.  That may explain some prejudices/tendencies :-)

Rusi
--

http://blog.languager.org

[Yurii Rashkovskii]

Michael,

So you're recommending them to use function names they don't understand but name variables in a way they will understand and nobody else will?

I always thought naming things in your own language is the first sign of an amateur programmer. Programming isn't about code. Programming is about communication. You're fostering global miscommunication. 

(For cases when you absolutely can't pick an English word, use transliteration — requires no change to the parser & the language and is easily consumable [albeit still poorly understood] by others).

Michael, you're a native English speaker. I am, however, not. I've been programming since I was seven and rest assured my command of English back then was very poor. Even more, my native alphabet isn't latin. Yet I am somehow managed to get through this.

Could it be possible that I am able to communicate with you because of my passion for programming, which indirectly made me learn English so that I can understand it better?

Yurii.

[Valentin Micic]

Why stop there? We have been enslaved by mathematical symbols for even longer.

Why can't we use colors to express equations? 

Am I being serious with this?

Not quite. But sure as hell would make our lives more colorful :-)

V/

[Rapsey]

On Mon, Oct 22, 2012 at 8:25 AM, Michael Richter <ttmri...@gmail.com> wrote:

On 22 October 2012 13:08, Yurii Rashkovskii <yra...@gmail.com> wrote:

Please excuse my ignorance, but can you name a single good reason for non-latin atoms and variable names? From my personal point of view, this is a sure road to hell.

Because people frequently like to work in their own language instead of a foreign language they ill understand?

Jesus!  How can so many smart people be so god-damned dumb over this issue?

I'm not from an english speaking country. I've worked for two development companies and all had a strict english only rule on code and all documentation. I am against this idea, simply because there are so many more worthwhile things the OTP team could be spending time on. If we're talking about unicode, a to_lower and to_upper function is a pretty big missing feature.

Sergej

[Max Bourinov]

+1

Best regards,

Max

[Yurii Rashkovskii]

Rapsey,

Couldn't agree more. There are so many other more important issues that could have been addressed by the OTP team and contributors.

By the way, we just go this (lower and upper) implemented in Elixir, but it could be very well separated out of Elixir to be used by plain Erlang developers: http://coderwall.com/p/pehkba 

It compiles the necessary part of the unicode database right into the beam file, making it fairly efficient. Also, pending merge, I have a patch that makes upcase and downcase work with graphemes as opposed to codepoints (https://github.com/elixir-lang/elixir/pull/566), which we hopefully will merge in shortly.

Yurii.

[Michael Richter]

On 22 October 2012 14:54, Yurii Rashkovskii <yra...@gmail.com> wrote:

So you're recommending them to use function names they don't understand but name variables in a way they will understand and nobody else will?

I guess you could read that as my recommendation (if you can't read, that is).
 

Could it be possible that I am able to communicate with you because of my passion for programming, which indirectly made me learn English so that I can understand it better?

If this is what you consider "communication" to be, Yurii, then I'd say that your 7-year-old's passion for programming didn't help you one iota.

[Yurii Rashkovskii]

On Monday, October 22, 2012 12:12:22 AM UTC-7, Michael Richter wrote:

On 22 October 2012 14:54, Yurii Rashkovskii <yra...@gmail.com> wrote:

So you're recommending them to use function names they don't understand but name variables in a way they will understand and nobody else will?

I guess you could read that as my recommendation (if you can't read, that is).

This was a question. How are they supposed to be "using their own language" if 100% of Erlang itself is in English? 

 

 

Could it be possible that I am able to communicate with you because of my passion for programming, which indirectly made me learn English so that I can understand it better?

If this is what you consider "communication" to be, Yurii, then I'd say that your 7-year-old's passion for programming didn't help you one iota.

This is rather inappropriate, although expectable from somebody who starts off calling his interlocutor stupid ;) 

[Michael Uvarov]

It is yet another implementation without SpecialCasing.
Here is a right algorithm.
http://www.unicode.org/reports/tr21/tr21-5.html

[Yurii Rashkovskii]

Michael,

Thanks — will look into this. We only spent one day on this so covered
only a small territory to validate the approach.

[Michael Richter]

On 22 October 2012 15:15, Yurii Rashkovskii <yra...@gmail.com> wrote:

So you're recommending them to use function names they don't understand but name variables in a way they will understand and nobody else will?

 

I guess you could read that as my recommendation (if you can't read, that is).

 

This was a question. How are they supposed to be "using their own language" if 100% of Erlang itself is in English? 

Here's an experiment for you: go back over what I originally said and look for the word "only" or "sole" or any other such term.  (Hint: this is not possible.)  Extrapolate from those results to why I think you're beginning to look like a strident ass.
 

If this is what you consider "communication" to be, Yurii, then I'd say that your 7-year-old's passion for programming didn't help you one iota.

 

This is rather inappropriate, although expectable from somebody who starts off calling his interlocutor stupid ;) 

This would be another example of your inability to communicate.  Let's go back to my original statement.  I'll bold some key words for you:

How can so many smart people be so god-damned dumb over this issue?

If you can read this as me calling you stupid, you lack any and all language skills required to critique the use of language in others.   Your misunderstandings include (but are not limited to):

1. Thinking this was directed solely at you.  (The construct "so many … people" indicates plurality.  Hell just the word "people" does that all by itself!)
2. Skipping over the word "smart".
3. Missing the link between "dumb" and "this issue".
   

Now were I mean-spirited I would, at this point, start agreeing with my original assessment as it was interpreted by you.  I'm not, however, and instead chalk this up to the martyr complex you've been nursing ever since the "elders of Erlang" (as you refer to them) didn't come flocking to your sudden fascination with Elixir.

After all it is very easy to find offence if that is all you look for.

Now consider this carefully before commenting on language and work again (anywhere, I mean): Given that you've been speaking English since you were 7 (by your own statement, recall), and given that you managed to misunderstand as simple a sentence as the above, are you absolutely positive you can't see some value in people working in their native language?

[Michael Uvarov]

What is the problem about unicode variables is that some characters
are not equal: Х != X, but they look the same.
Other problem about unicode is that a lot of algorithms are
locale-based and difficult (a lot of rules and exceptions).

Even non-locale based (unified and simple version of to_lower) contains this:
- Contains additional case mappings that map to more than one
character, such as "ß" to "SS".
But this case is save for variable names. The next case is more
interesting. It is from the locale-based version:
- Characters may have case mappings that depend on the locale.
For example, in Turkish the letter U+0049 "I" capital letter i
lowercases to U+0131 "ı" small dotless i.

For example, the full version of the toLower function from ICU and its
dataset is described somewhere here:
http://www.unicode.org/reports/tr35/

[Michael Uvarov]

There is no such thing as "language" in Unicode.
Unicode is just a set of symbols.
"language" is a locale. Locale-based algorithms are difficult and each
character can have different meaning for each locale.
There are a lot of cases, when I even cannot say which case a variable
is in. How I will detect is it a variable or an atom?

Here is an example:
I want to write a module in Turkish, then the "length" id will be a
variable, not a function.

Using code, written in few languages will be a hell.

[HH Veldstra]

+1. Outwith very specific circumstances allowing non-English code is
dumb if for no other reason that it will drastically reduce the pool
of programmers that can be hired to work on your system.

And regardless of where one falls on this issue, shouldn't it be
rather low on the priority list anyway? I'm thinking way below fixing
the stdlib, or fixing records, or perhaps improving text handling.

[Henning Diedrich]

There is something to that, seriously.

[Loïc Hoguin]

On 10/22/2012 08:00 AM, Anthony Ramine wrote:
> Also non-latin users who don't know English should be able to use
> atoms and variables they understand.

Can't stress this enough.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Yurii Rashkovskii]

Imagine you don't know any English. Would writing code this way help you in any way: mwi34frrc:ehfeiurvhsdsdcd(VariableNameIUnderstand) ?

Yurii.

[Bengt Kleberg]

Greetings,

It would help slightly if it was written Variable_name_I_understand.
Camel case does not help me.


bengt