The shape of things to come - Elixir v0.8.3 and v0.9.0

[José Valim]

Hello everyone,

Short version

1) This wednesday: release v0.8.3

* Give master a try now!

2) In a week from wednesday: release v0.9.0. v0.9.0 will contain two major breaking changes:

* R15 will no longer be supported

* The module String in Elixir will translate to 'Elixir.String' in Erlang and no longer 'Elixir-String' (and so forth)

* All the deprecation warnings introduced in v0.8.3 will still be around for v0.9.0

Long version

The latest month has been full of work with great contributions! This means it is about time for a new release!

The goal is to release v0.8.3 this wednesday.

Please take a look at the CHANGELOG for more information and give master a try.

If you find any regressions, please let us know!

Right after v0.8.3 is out, we will start cooking v0.9.0 and it should be released one week (or less) from wednesday.

The main goal of v0.9.0 is to translate the module String to 'Elixir.String' in Erlang (no longer 'Elixir-String').

This change also means we can no longer support R15 and for all those reasons we are doing a minor version bump.

To easy transition, all deprecations introduced in v0.8.3 will be kept for v0.9.

We also plan to have one extra surprise for v0.9.0 that you will get to know about soon. :)

Finally, I have set up Milestones for both releases on Github.

Happy coding!

PS: After v0.9 comes v0.10 and not v1.0 (yet!)

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Alexei Sholik]

PS: After v0.9 comes v0.10 and not v1.0 (yet!)

v0.A ;)

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Best regards
Alexei Sholik

[José Valim]

Elixir v0.8.3 was released. In master we have already started the road to v0.9.

I have said that Elixir v0.9 is about removing support for Erlang R15 and about naming Elixir modules Elixir.String instead of Elixir-String. And there was one surprise...

https://github.com/elixir-lang/elixir/pull/1102

The work done by Eric in the pull request above will migrate Elixir to use reducing functions instead of iterators. This will allow us to massively clean up Enum source code, it ensures tail recursion on lists operations and ensure better performance for other types like ranges and hash dicts.

Furthermore, the pull request allows iterators to be self-contained (for example, the iterator can open up a file, iterate it and close it at the end, even if an error occurs) and they are easily composable. Which means we can easily support lazy functions and parallel folding!

I will write a longer blog post announcing v0.9 (which is coming soon) with more details about reducers and benchmarks!

Happy coding!

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Dave Thomas]

I will write a longer blog post announcing v0.9 (which is coming soon) with more details about reducers and benchmarks!

This is fantastic stuff!

Just-a-wondering. While we're all iterating, is there any chance you'd accept a pull req for Kernel.seq(enum) -> list?

Dave 

[José Valim]

Just-a-wondering. While we're all iterating, is there any chance you'd accept a pull req for Kernel.seq(enum) -> list?

Kernel.seq(enum) would return an enumerable as list? Basically the same as Enum.to_list/1? Or something else.

I would like to wait before adding a new term like "seq" to Elixir's vocabulary. There are some discussions in the issues tracker about it and I would like to get it right before continuing. :)

[Dave Thomas]

On Wed, May 22, 2013 at 11:33 AM, José Valim <jose....@plataformatec.com.br> wrote:

Just-a-wondering. While we're all iterating, is there any chance you'd accept a pull req for Kernel.seq(enum) -> list?

Kernel.seq(enum) would return an enumerable as list? Basically the same as Enum.to_list/1? Or something else.

Just enum to list. It would just be a convenience in things like

   lc x inlist seq(1..100) ,...

 

[Alexei Sholik]

There is an ongoing discussion[1] about simplifying the task of building lists of indices.

For the particular use case of `lc`, we could consider adding `inrange` to Elixir since ranges are already builtins.

  [1]: https://github.com/elixir-lang/elixir/issues/1039#issuecomment-18122295

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Saša Jurić]

Perhaps it would be better if 

lc inlist 1..100, ... and

lc inlist anything_enumerable, ... could work?

[José Valim]

Perhaps it would be better if 

lc inlist 1..100, ... and

lc inlist anything_enumerable, ... could work?

We can't make that work natively. Which means we would have to first convert the enumerable to a list and then do the comprehension. That is a very hidden overhead/complexity. It is for this reason the operator is called exactly "inlist" instead of "inenum".

There is definitely room for improvement but I don't believe that can happen by wrapping lc.

[Sasa Juric]

On May 22, 2013, at 6:55 PM, José Valim wrote:

Perhaps it would be better if 

lc inlist 1..100, ... and

lc inlist anything_enumerable, ... could work?

We can't make that work natively. Which means we would have to first convert the enumerable to a list and then do the comprehension. That is a very hidden overhead/complexity. It is for this reason the operator is called exactly "inlist" instead of "inenum".

Without looking at the source: why wouldn't this work? All Enums are based on Enum.iterator which returns only the next value. So all it takes is to call iterator one by one, execute the filter and if it is satisfied call the inner code. Am I missing something?

[José Valim]

Sorry, I did not make it clear. lc maps directly to some erlang abstract format which is optimized by the VM, it does not pass through Enum.Iterator at all. The VM does not allow us to pass anything other than a list there.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Sasa Juric]

Is it based on the same principles as erlang list comprehension, or something else is involved?

Is there a significant performance gain with this?

And if so, why not create inenum in addition?

You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/PmKCwdq2jmA/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

[José Valim]

Is it based on the same principles as erlang list comprehension, or something else is involved?

Yes, It compiles exactly to the same as an erlang list comprehension.

Is there a significant performance gain with this?

Last time I benchmarked, yes.

 

And if so, why not create inenum in addition?

The Erlang VM wouldn't know how to handle inenum. This means we need to convert the enum, at runtime, to a list. This means that collections given to inenum would be iterated twice, one to convert it to a list and another one as a list. I'd consider this to be a surprising behaviour as for any moderately large collection a difference in performance would be noticeable. If a conversion has to happen, it should happen explicitly.

[Alexei Sholik]

It's not necessary to stick to Erlang's comprehensions in the case of inenum since we're already forfeiting the performance. In fact, `lc in <enumerable>` could accept any enumerable collection and compile to Enum.map or something similar. `lc inlist <list>` will then be a recommended special case to use with lists because of its better performance.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Best regards
Alexei Sholik

[Yurii Rashkovskii]

I would disagree here. If you look at core erlang generated by the erlang compiler from lc/bc, it is just anonymous functions. I think we can definitely try heading into that direction (generating the same kind of anonymous functions but with extended functionality).

Here's core erlang for Erlang's [ X + 1 || X <- [1,2,3] ], for example:

```erlang

letrec

   'lc$^0'/1 =

fun (_cor2) ->

   case _cor2 of

     <[X|_cor1]> when 'true' ->

 let <_cor3> =

     call 'erlang':'+'

 (X, 1)

 in  let <_cor4> =

 apply 'lc$^0'/1

     (_cor1)

     in  ( [_cor3|_cor4]

   -| ['compiler_generated'] )

     <[]> when 'true' ->

 []

     ( <_cor2> when 'true' ->

   ( primop 'match_fail'

 ({'function_clause',_cor2})

     -| [{'function_name',{'lc$^0',1}}] )

-| ['compiler_generated'] )

   end

in  apply 'lc$^0'/1

([1|[2|[3]]])

```

--

[José Valim]

Thanks Yurii! If all the compiler is doing is generating anonymous functions and there is no special VM work, I wonder why I saw a considerable difference when I last benchmarked? Maybe the difference I got was because I was doing remote calls into Enum.filter instead of writing local functions? After all, using inenum would still require dispatching to a protocol.

Anyway, this opens up interesting possibilities! I will play a bit with this and come back with my findings. If anyone is interested on the subject, here is an interesting paper about comprehensions in haskell with support to order by and group by.

Any Haskell user in the house would be able to tell us if this paper is actually implemented in latest haskell versions?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Sasa Juric]

On May 22, 2013, at 7:35 PM, José Valim wrote:

Is it based on the same principles as erlang list comprehension, or something else is involved?

Yes, It compiles exactly to the same as an erlang list comprehension.

Is there a significant performance gain with this?

Last time I benchmarked, yes.

That's strange. This link: http://www.erlang.org/doc/efficiency_guide/myths.html states:

"Nowadays the compiler rewrites list comprehensions into an ordinary recursive function.Of course, using a tail-recursive function with a reverse at the end would be still faster. Or would it? That leads us to the next myth."

 

And if so, why not create inenum in addition?

The Erlang VM wouldn't know how to handle inenum. This means we need to convert the enum, at runtime, to a list. This means that collections given to inenum would be iterated twice, one to convert it to a list and another one as a list. I'd consider this to be a surprising behaviour as for any moderately large collection a difference in performance would be noticeable. If a conversion has to happen, it should happen explicitly.

Here's the ugly untested sketch of what I meant:

https://gist.github.com/sasa1977/5629646#file-mylc-ex-L10-L12

It doesn't involve filtering, but that should be no problem to include.

Notice how in the highlighted part the result is collected one by one, based on the Enum.Iterator implementation. So the extra pass does not occur. The infix recursion follows the above description from efficiency guide.

[José Valim]

Thanks Yurii and Sasa! I have good news. I was able to get good results by generating a code very similar to the one outlined in Core Erlang:

https://gist.github.com/josevalim/89926e41ef0f2c0c42be

You can copy to your machine and run the benchmark above. There is still a performance difference but that happens because I am using anonymous functions for the recursion. If we compile it down to a compiled function, we can achieve similar results to erlang comprehension while supporting enumerables (i.e. we can have inenum).

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[José Valim]

Another update, we have been able to finish everything we wanted for Elixir v0.9.0. So it should be getting out tomorrow. :) A blog post has been written and is available here.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[José Valim]

The announcement:

http://elixir-lang.org/blog/2013/05/23/elixir-v0-9-0-released/

For those who want to install it, keep in mind you need Erlang R16B.

The getting started guide was updated with recent install instructions:

http://elixir-lang.org/getting_started/1.html

Thanks!

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Sasa Juric]

On May 24, 2013, at 8:44 AM, José Valim wrote:

For those who want to install it, keep in mind you need Erlang R16B.

Probably everybody knows about this, but I'll mention it just in case.

If you need to have parallel versions of Erlang and Elixir installed (which is what I need, because I don't have the time to migrate my closed source project to R16 just now), you can use excellent kerl and exenv tools:

https://github.com/spawngrid/kerl

https://github.com/mururu/exenv

I tried them out yesterday and it worked like a charm.