[erlang-questions] Spec is diabolical IMHO

[Steve Davis]

<rant>
Can someone who knows better explain how this evil child of -spec:

%% @doc The gen_server code_change/3 callback, called when performing
%% a hot code upgrade on the server. Currently unused.
-spec code_change(OldVsn, State, Extra) -> {ok, State} | {error, Reason}
                                               when
      OldVsn :: Vsn | {down, Vsn},
      Vsn :: term(),
      State :: #state{},
      Extra :: term(),
      Reason :: term().
code_change(_OldVsn, State, _Extra) -> {ok, State}.

...is not a total waste of too many talented people's time over the more transparent:

code_change(_OldVsn, State = #state{}, _Extra) -> {ok, State}.

...or is spec just a diabolical case of over-engineering that is at once both incredibly onerous and bug-prone justified in the cause of the weaknesses of some ill-designed test tool?

My gut reaction is that if you feel that you NEED TO DO TDD style testing, you should take the time parse the source and not bloat it beyond readability by inflicting conventions on the development of the source that will simply distract and so *cause* the bugs you think that you are testing for.

A better use of developer time would be to enforce documentation of the intention of the function.

</rant>
/s

[David Mercer]

Since no-one else stepped into the breach, I’ll make a case.

 

In my view, the –spec is a contract, which (1) is usually written before the implementation as part of your design; and (2) as such, it serves as documentation regarding your intent and usage.

 

In this case, programmers calling the function should be prepared for both an ok return value and an error.  Without the –spec, the programmer does not know this and may just do a match on {_, State} and then not notice any errors when the function changes.  On the other side, when a maintainer comes along later wishing to modify this function, he understands (1) the function’s intent, and (2) how to signal errors.  He can make whatever changes he wishes to effect new functionality so long as he obeys that contract.

 

Cheers,

 

DBM

[Steve Davis]

Hi David,

I definitely understand and appreciate that rationale. I would pick a bit at the expectation of coding: {_, State} and not expecting the ignored data to be relevant :).

My counter to that would be that if a function was too hard to understand with regard to return values, then the function itself is likely too big and complex -- and counter to any guideline I have seen. 

Rather than double (or more than double) the size of the code,  and make no mistake, -spec IS CODE (although only used by test tools), it may be more effective to devise boundary contracts similar to JA's intention for UBF-C rather than plastering bug-prone "comments" into the raw source.

Best regards,

/s

On Sep 26, 2012, at 3:27 PM, "David Mercer" <dme...@gmail.com> wrote:

Since no-one else stepped into the breach, I’ll make a case.

 

In my view, the –spec is a contract, which (1) is usually written before the implementation as part of your design; and (2) as such, it serves as documentation regarding your intent and usage.

 

In this case, programmers calling the function should be prepared for both an ok return value and an error.  Without the –spec, the programmer does not know this and may just do a match on {_, State} and then not notice any errors when the function changes.  On the other side, when a maintainer comes along later wishing to modify this function, he understands (1) the function’s intent, and (2) how to signal errors.  He can make whatever changes he wishes to effect new functionality so long as he obeys that contract.

 

Cheers,

 

DBM

 

[Daniel Eliasson]

In my experience, a lot of functions perform tail calls in such a way
that it's not immediately clear which types will be returned. Without
specs, it's frequently necessary to start following the code many
levels deep in a branching tree. Exported functions should have specs;
at the very least those intended as an API to other applications
should.

If I can't look at an API function and very quickly understand
i) what type of values I am to pass as arguments, and
ii) what type of values I get in return,
then I have resort to archeology or empirical studies, both of which
are unpleasant.

/Daniel

> _______________________________________________
> erlang-questions mailing list
> erlang-q...@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
>
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Raoul Duke]

the real technical solution would be for somebody to port the ideas of
Typed Racket over to Lisp Flavoured Erlang. :-)

[fr...@circlewave.net]

On Thu, Sep 27, 2012 at 10:29:24AM -0700, Raoul Duke wrote:
> the real technical solution would be for somebody to port the ideas of
> Typed Racket over to Lisp Flavoured Erlang. :-)

It would perhaps help to move all typespecs to single section somewhere
at the top of source file (this is probably possible even today?), or
even better move them to a per-application .sig file that has all the
annotations for all modules in that app... (I share the perception that
they're distracting when scattered all around the place).

Just my 2p.

BR,
-- Jachym

[Raoul Duke]

On Thu, Sep 27, 2012 at 11:11 AM, <fr...@circlewave.net> wrote:
> even better move them to a per-application .sig file that has all the
> annotations for all modules in that app... (I share the perception that

ah, yes, bring back c/++ style .h files and all their glory! ;-)

[fr...@circlewave.net]

On Thu, Sep 27, 2012 at 11:34:30AM -0700, Raoul Duke wrote:
> On Thu, Sep 27, 2012 at 11:11 AM, <fr...@circlewave.net> wrote:
> > even better move them to a per-application .sig file that has all the
> > annotations for all modules in that app... (I share the perception that
>
> ah, yes, bring back c/++ style .h files and all their glory! ;-)

And I thought ".sig" would be recognized as obvious reference to
Standard ML... :-)

BR,
-- Jachym

[Loïc Hoguin]

On 09/27/2012 08:11 PM, fr...@circlewave.net wrote:
> On Thu, Sep 27, 2012 at 10:29:24AM -0700, Raoul Duke wrote:
>> the real technical solution would be for somebody to port the ideas of
>> Typed Racket over to Lisp Flavoured Erlang. :-)
>
> It would perhaps help to move all typespecs to single section somewhere
> at the top of source file (this is probably possible even today?), or
> even better move them to a per-application .sig file that has all the
> annotations for all modules in that app... (I share the perception that
> they're distracting when scattered all around the place).

I'm of the opposite opinion. It's annoying enough when you add a public
function that you have to go to the top of the file to export it, I
wouldn't want to have to open another file to add the specs too.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Raoul Duke]

On Thu, Sep 27, 2012 at 11:24 AM, <fr...@circlewave.net> wrote:
>> ah, yes, bring back c/++ style .h files and all their glory! ;-)
> And I thought ".sig" would be recognized as obvious reference to
> Standard ML... :-)

i missed that, since i haven't had the good fortune of using SML for
nigh 20 years now, and have only rarely done anything related like
ocaml or f# or whatever else. more is the pity, sucks to be me.

but look at it this way: some people, when faced with the problem of
having a single source file that has embedded metadata (be it user
docs or type annotations or whatever), say, "i know! i'll split it
into 2 files!" of course, now they are faced with /two/ problems.
that's why e.g. java tried to get away from that, and the go language
was invented.

i'm not saying 1 vs. 2 files has a clear best answer for everybody,
just that both choices have problems. in the end, this feels to me
like yet another concrete example of why 99.five9s% of our development
ides are stupid crap that don't really help with the real problems. we
should not have to worry what files they are in or if we do/not have
to see them, the ide should magically take care of all that. but
that's a pipe dream of course these days.

[Mike Oxford]

I'm in the "I like C/C++ .h files," just like I like the concept of IDL files.

I can very quickly scan through the file (just like Erlang -export areas) and see what's available and the parameters.

Otherwise you're either bogged down going through the code or relying on a secondary toolpass to generate documentation which then has to be hosted somewhere.

"What was the exact parameter list?"

"Hold on, let me fire up a browser to get to the file/site."

"Nevermind, I just opened up the header and I got it."

Headers have downsides, to be sure, but they have a very nice usage pattern for figuring out WTF this object is, what it appears to do and maybe how it plays into the overall architecture of the system.

Also, I can grep a header dir and not get slammed with a wall of implementation crap.

-mox

[Richard O'Keefe]

On 28/09/2012, at 7:20 AM, Mike Oxford wrote:

> I'm in the "I like C/C++ .h files," just like I like the concept of IDL files.
>
> I can very quickly scan through the file (just like Erlang -export areas) and see what's available and the parameters.
> Otherwise you're either bogged down going through the code or relying on a secondary toolpass to generate documentation which then has to be hosted somewhere.
>
> "What was the exact parameter list?"
> "Hold on, let me fire up a browser to get to the file/site."
> "Nevermind, I just opened up the header and I got it."
>
> Headers have downsides, to be sure, but they have a very nice usage pattern for figuring out WTF this object is, what it appears to do and maybe how it plays into the overall architecture of the system.
>
> Also, I can grep a header dir and not get slammed with a wall of implementation crap.

Let me commend WYSINEYH.

What You See is Not Everything You Have.

I don't agree with Bertrand Meyer about everything, but there are a
lot of really good ideas behind Eiffel. One of them is the tools
that come with it. Recall that Eiffel has multiple inheritance and
_lightweight_ embedded documentation.

Eiffel comes with two helper programs:

flat
Takes an Eiffel class and gives you a listing with all
the inheritance folded out, so you see *every* method
in the class, not just the added ones.

short
Takes an Eiffel class and strips out the method bodies;
it gives you a pretty-printed listing of the method
interface and documentation.

The combination

short flat
Gives you the interfaces and documentation for everything
exported from a class.

Come to think of it, JavaDoc, pldoc, Haddock, and erldoc try to
do something a bit similar, but heavier weight.

What I'm thinking of is an editor mode that (possibly temporarily)
hides everything but the -specs and selected comments.

That way you get the benefit of HAVING everything in the one file
but not the cost of having to READ everything.

[Raoul Duke]

On Thu, Sep 27, 2012 at 4:03 PM, Richard O'Keefe <o...@cs.otago.ac.nz> wrote:
> That way you get the benefit of HAVING everything in the one file
> but not the cost of having to READ everything.

everybody here knows this already exists for every main-stream
language+ide, yes? :-) even though i'm often an emacs user, i also use
eclipse and jetbrains (and visual studio before). somebody oughtta
write an erlang plug-in for one of those that people can contribute
to.

[Steve Davis]

Since it's not been picked up on, I hope that everyone noticed that in the original example that I posted, even that blatantly simple spec was... wrong.

There is NO WAY that function could ever return {error, Reason}, and further wastes your valuable time in trying to convince you that it could.

[Raoul Duke]

hi,

On Thu, Sep 27, 2012 at 4:48 PM, Steve Davis
<steven.cha...@gmail.com> wrote:
> There is NO WAY that function could ever return {error, Reason}, and further wastes your valuable time in trying to convince you that it could.

sounds pretty funny, maybe somebody was trying to leave it as an
exercise for the reader, or two different people worked on the spec
vs. code. if you are complaining about getting that particular one
fixed, that makes sense, and presumably you could open a bug or send a
patch in for it somewhere?

but if you are talking about the general problem of mismatches
happening between spec+code, well, then you need to either move back
to something w/out any specs at all, or you need to use stuff like
quickcheck*, or move over to agda or epigram or ats or something crazy
like that.

(* and even then the proving-a-negative-thing is philosophically
unsolvable in the general sense, no?)

sincerely.

[Raoul Duke]

On Thu, Sep 27, 2012 at 4:57 PM, Raoul Duke <rao...@gmail.com> wrote:
> (* and even then the proving-a-negative-thing is philosophically
> unsolvable in the general sense, no?)

p.s. i'm not saying code coverage tools can't look into things like
your specific example, just that in the widest sense of 'proving a
negative' it sounds to me like the philosophical induction black swan
issue.

[Richard O'Keefe]

On 28/09/2012, at 11:59 AM, Raoul Duke wrote:
> p.s. i'm not saying code coverage tools can't look into things like
> your specific example, just that in the widest sense of 'proving a
> negative' it sounds to me like the philosophical induction black swan
> issue.

It's a bit off topic, but I am really getting tired of people talking
about black swans. I hold dual Australian/New Zealand citizenship,
and in *both* my countries black swans are the only kind of swans you
will see in the wild. Black swans are NORMAL. White swans are weird.

(I leave you to draw the obvious parallel for yourselves...)

[Steve Davis]

Hi Michael,

I suspect that if your code is beta quality then it would be fair to say that the contract is beta quality also, right?

When you/the author get around to implementing it (if you/they ever do) then you/they may definitely find that the contract changes. I think it hubris to believe that you can predict that future without implementing it first. Meantime, this "api idea" in a spec is implicitly expected to be supported by all users - even though it is bogus. I'd say to that - just be honest about what _is_, not what you think things may be.

WRT to dialyzer -> If dialyzer can see that, then what is the purpose of the spec (given that is it provably wrong). Should not dialyzer strive to look deeper rather than requesting/requiring that highly fault-prone specs should be written for every function?

You'll perhaps also note that the comment on the example was also wrong. It stated "Currently unused", and that isn't true either - the function just doesn't do anything on a code change; But the function IS USED. *Everything* preceding the actual function is misleading.

I'm saying we should all be aware of that, and dare to consider better solutions than spec.

I would post-script the above with the following thought for those that have experienced it in other languages: The drive to clear definition of static typing leads to highly repetitive and confusing code that leads to high distraction and extremely suspect verbosity. => One of erlang's core syntax strengths is a low requirement on developer time to get an error tolerant, working system, and that aggressive approach is "saved" by coding to "let it crash".

/s


On Sep 27, 2012, at 7:43 PM, Michael Radford <mr...@blorf.com> wrote:

> At least the dialyzer won't let you get away with incorrect specs, if
> that's what you're worried about. (I find this quite irritating when I
> want to define an interface to support some future expansion that
> callers should handle, but isn't fully implemented yet!)
>
> $ cat >> foo.erl
> -module (foo).
> -export ([foo/0]).
> -spec foo () -> {ok, any()} | {error, any()}.
> foo () -> {ok, now ()}.
>
> $ dialyzer --src -c foo.erl
> Checking whether the PLT /home/notroot/.dialyzer_plt is up-to-date... yes
> Proceeding with analysis...
> foo.erl:3: The specification for foo:foo/0 states that the function
> might also return {'error',_} but the inferred return is
> {'ok',{non_neg_integer(),non_neg_integer(),non_neg_integer()}}
> done in 0m0.34s
> done (warnings were emitted)
>
> Mike

>
> On Thu, Sep 27, 2012 at 4:48 PM, Steve Davis
> <steven.cha...@gmail.com> wrote:

[Loïc Hoguin]

On 09/28/2012 03:11 AM, Steve Davis wrote:
> Hi Michael,
>
> I suspect that if your code is beta quality then it would be fair to say that the contract is beta quality also, right?
>
> When you/the author get around to implementing it (if you/they ever do) then you/they may definitely find that the contract changes. I think it hubris to believe that you can predict that future without implementing it first. Meantime, this "api idea" in a spec is implicitly expected to be supported by all users - even though it is bogus. I'd say to that - just be honest about what _is_, not what you think things may be.
>
> WRT to dialyzer -> If dialyzer can see that, then what is the purpose of the spec (given that is it provably wrong). Should not dialyzer strive to look deeper rather than requesting/requiring that highly fault-prone specs should be written for every function?
>
> You'll perhaps also note that the comment on the example was also wrong. It stated "Currently unused", and that isn't true either - the function just doesn't do anything on a code change; But the function IS USED. *Everything* preceding the actual function is misleading.
>
> I'm saying we should all be aware of that, and dare to consider better solutions than spec.
>
> I would post-script the above with the following thought for those that have experienced it in other languages: The drive to clear definition of static typing leads to highly repetitive and confusing code that leads to high distraction and extremely suspect verbosity. => One of erlang's core syntax strengths is a low requirement on developer time to get an error tolerant, working system, and that aggressive approach is "saved" by coding to "let it crash".

Specs are insanely useful to me by ensuring that my code changes do not
have an unexpected impact, like unknowingly changing an interface, or
creating some dead code by not seeing a match can't happen. Having them
allows Dialyzer to help you a lot more than if it has to guess everything.

Without them I'd probably introduce bugs all the time. They're an
additional guarantee alongside tests. Specs ensure the intent is right,
tests ensure the code is right.

Plus specs are also documentation that I'd have to write somewhere anyway.

--
Loïc Hoguin
Erlang Cowboy
Nine Nines
http://ninenines.eu

[Raoul Duke]

p.s. somebody who knows erlang please port shen-qi-lisp.

[Max Bourinov]

Can you please point me to the good dialyzed docs where I can find detailed description of it syntax?

Best regards,

Max

[Raoul Duke]

On Sat, Sep 29, 2012 at 2:54 AM, Max Bourinov <bour...@gmail.com> wrote:
> Can you please point me to the good dialyzed docs where I can find detailed
> description of it syntax?

if you mean lfe, please talk to the honorable Mr. Virding, i think.

if you mean typed racket, look at http://docs.racket-lang.org/ts-guide/

if you mean shen, see http://www.shenlanguage.org/