Explicit Modules in protocol/implementation ?

[Arnaud Wetzel]

Hello all, 
Explicit is better than implicit and in the last Elixir adjustments of defstruct, defrecord, defexception : defmodule is not hidden in the macro anymore and so the creation of the module is explicit which is I think a good thing for the user. I think Protocol/Implementation interface (defprotocol/defimpl) are a little bit too magic currently. So I did try a small proposal for a new interface. What do you think ? The goal is that Protocol should be seen as simple Macros with a predictable behaviour and not as a specific elixir thing you have to learn when you learn elixir.

https://gist.github.com/awetzel/f882f83e90b562932273

[José Valim]

This is a neat idea!

The issue though is that you should not define other functions along side your protocol as that would limit the optimizations we could perform in the protocol. For example, today we have something called protocol consolidation that rewrites the protocol module into fast dispatches and that could get much harder if we allowed users to mix protocol dispatches with regular functions. So we do want to give users the idea it is not a regular module where they can define whatever functions they want in it (which is the opposite for defstruct).

On the implementation side, we do have less constraints, so your suggestion would work fine, but given defprotocol will likely stay as is, defimpl should probably not change too.

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Arnaud Wetzel]

But can we just put it in the documentation or "Protocol" : "You cannot write other functions is you want fast dispatch" ?

and a Log when compiling (in the Protocol.__using__) to tell : "the module contain other functions and so cannot be consolidated".

My point is that "I think" currently the fact that the defprotocol defines a dispatch module is not explicit (the word "dispatch" is nowhere) and that this module is not either a regular module because of consolidation is not explicit neither, this make this functionality very binded to the language, it makes elixir more than just "module/function/macros/stdlib", which is I think a bad thing because it makes Elixir more "complicated" to learn and use. 

You have to learn protocol internals (to know the modules) in order to correctly update modules in your OTP release upgrade for instance.

[Arnaud Wetzel]

Currently a user sees the protocol as a module when he uses it : "Blank.blank?". So the problem still exists : the user can be mistaken and use macro defining functions in protocol : defprotocol Toto, do: defstruct(tata: toto)

And no warning appears which is a problem. We can make a compilation error when the is 

"use Protocol"

and other functions than "defdispatch", the error saying that we can add an option which will slow down dispatch but allowing you to use it as a regular module :

"use Protocol, allow_slow_dispatch: true"

or 

"use Protocol, no_consolidation: true"

[José Valim]

 

this make this functionality very binded to the language, it makes elixir more than just "module/function/macros/stdlib", which is I think a bad thing because it makes Elixir more "complicated" to learn and use. 

Changing "defprotocol" to "use Protocol" doesn't make Elixir less or more hard to learn. You still need to learn protocols, how to use them and how to implement them.

I agree with your point that providing a defdispatch helps with understanding how protocols work behind the scenes but it also sends the wrong message that they are regular modules which isn't true as there specific actions and recommendations for protocols.

We had the same thing with the now defunct defrecord/3. defrecord/3 defined a module on its own instead of how defstruct/2 works because we didn't want developers to see records as a regular modules, where they can define functions/macros/etc, even if that was how it was implemented behind the scenes.

It is not a technical issue. It is about "how do we want developers to perceive this". 

You have to learn protocol internals (to know the modules) in order to correctly update modules in your OTP release upgrade for instance.

This is actually a great reason to keep protocols as is. Once release tools start to show up for Elixir, those build tools can automatically identify protocols for you and handle them automatically and that only works as long as protocols are a black box. The more options, combinations and features we allow, the harder it is to release, upgrade and consolidate protocols.

[José Valim]

the user can be mistaken and use macro defining functions in protocol : defprotocol Toto, do: defstruct(tata: toto)

And no warning appears which is a problem. We can make a compilation error when the is 

"use Protocol"

Well, this is a bug which is orthogonal to whatever strategy we pick for defining protocols. Regardless if we use defprotocol or use Protocol, we could have more extensive checks forbidding the user of defining such functions.

[Arnaud Wetzel]

Exactly the "regardless" was my point, the black box does not solve
anything more. "the 'use Protocol' doesn't help for anything compared
to "defprotocol", but 'defmodule' and 'defdispatch' does. I understand
your point and I don't agree (the same about record modules) : black
box is never good, the force of Elixir is that it as the flexibility
(with macros and syntaxic sugar) and the simplicity : in the end we
have modules to manage functions to manage immutable data,
light-linked-autogc-processes to run them and message passing to
synchronise.
The thing that Elixir allows macros gives responsability to the
developer, he could do some things very nasty but it is fine because
it is just a powerful tool. So, thanks to macros, there are many many
cases where you can put somthing in the wrong place in Elixir (many
more than defprotocol/defstruct), hiding things does not help to avoid
that, but understand things does help.

I would like to see protocol as a useful macro, part of the standard
library, that defines dispatch functions for me (and also that
optimizes dispatch for me, which add constraints to my module). The
blackbox thing makes "protocols" an Elixir specific paradigm for
polymorphism.

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elixir-lang-core" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elixir-lang-core/_SYGvgPp8Mo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[José Valim]

Exactly! Technically you can always bypass everything. That's why having explicit defprotocol helps with specifying the behaviour we want semantically.

We will likely have to agree on disagreeing. :) It seems both of us understand the trade-offs but we give them different weights (which is fine).

[Arnaud Wetzel]

I understand your point :). I am scared with the "language handles
polymorphism, I cannot know how, and am limited by this, language does
not help me to specify my problem but force me in a way to do it"
thing because of the OO nightmare as many :) . I know this is not the
case at all in Elixir (which is GREAT by the way) and so it is fine.
But anything which looks like a black box scares me. The thing is that
your users will need to know which module they have created, because
of release management but also for there own need. Maybe detail
systematically "modules and functions" generated by macros (all) in
there doc would help even more than my proposal. I use Elixir in
production for one year and it makes me sad that I had to read Elixir
code directly many times in order to make things works.