Runtime typespecs assertion

[Sergiy Kukunin]

Hello there. This is my first message to the elixir group. Thanks for the great language.

While I'm writing my code, I want to make functions to be safer. It's bad practice if a function accepts unexpected input and pass it further, and it blows in a completely different part of a system.

At first glance, I have pattern matching, but it's pretty limited. It becomes really powerful in conjunction with guards, so I can write a signature to match literally everything. 

But they hard to re-use, If I have multiple functions operating with the same object. Yes, I can define a custom guard, but can I use pattern matching there? `Kernel.match?/2` doesn't work, so I'm limited with only guards in my custom guards.

Another thing that we have typespecs. It seems exactly what I'm looking for: you have a wide set of built-in types, and I can easily compose and reuse my own types. The problem with it, that it doesn't affect runtime. I know about static analyzer `dialyzer`, but I'm not sure it will catch all cases since it's a static check, not a runtime.

Let's assume a simple function, that wraps a value into a list:

  @spec same(number()) :: [number()]

  def same(number) do

    [number]

  end

I'm sure the `dialyzer` won't complain since a signature is valid. But what if I do: `same("abc")` ? What will prevent Elixir from returning a wrong type? I guess, nothing. 

An example from a real life: I have a function, that accepts a custom shaped value (using tuples) and feeds it to a queue. Then, in the totally different part of the system, a consumer gets values from the queue. And when a wrong value was fed on the producer side, it blows on the consumer side. So I decided to put some constraints on the producer side to fail fast.

Yes, I could define a guard, but again, if I have a pretty complex type instead of the simple `number`, I had to duplicate the type defining: one for typespec, another is for a custom guard (which is limited, since I can't use pattern matching there).

Wouldn't it be cool, If we had a mechanism to assert a value to its type, in runtime? To avoid performance penalty we could enable it only for runtime. Is there a way right now to check whether a value corresponds to a type in runtime? Can I implement a custom macro to provide a good DSL for this? Is it helpful at all?

P.S. You may say, use structs and pattern matching would work in this case. But what if my type is better represented by a tuple: {atom(), pos_integer(), string()}. Converting it to a struct might complicate a way to work with the value.

[Sergiy Kukunin]

I mistook, sorry:  To avoid performance penalty we could enable the typespec runtime asserts only for development and testing environments

[Ivan Yurov]

If you want type-safety why not to just pick a strongly typed language, like Ocaml for example? Elixir is bound to Erlang VM and will never provide any features like you're describing that are not supported by Erlang. And I don't think type-checking ever happens at runtime in any language.

[Louis Pilfold]

Hi all

The desire for more safety in Elixir is reasonable, both at compile time and at runtime.

The core team have previously experimented with introducting a compile time type checking system, and we also have the dialyser and gradualizer tools that can be used with Elixir.

Checking at runtime is something we already do in Elixir and Erlang through the use of pattern matching and guards such as `is_binary/1`.

A library of macros that automates these checks could be an interesting project, perhaps an area worth exploring for members of the community.

Cheers,

Louis

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/8c4d9dac-134d-471c-a402-e9696bf5aecf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sergiy Kukunin]

Thanks for the answers. Just want to note, that I don't want to invent type system such as in statically typed languages. I mean more about defining schemas we can check different values with. All pattern matching, guards and typespec might work for this. Furthermore, it would be cool to make it composable and reusable (such as defguards and typespecs right now).

Just to conclude, I would suggest that either of these would improve the safety and convenience of the language:

- allow pattern matching in custom guards (either via the built-in guard such as `Kernel.match?/2` or by extending the defguard syntax)

- having a macro to check whether a value corresponds to a defined @type

What's about such syntax?

 defguard is_mytype({x, y}) when is_atom(x) and is_number(y)

 def test({:ok, value}) when is_mytype(value), do: true

 def test(_), do: false

 test({:ok, {:hello, 5}}) # should be true

 test({:ok, {2, 5}})  # should be false

There are a couple of reasons I've raised this question:

- do I miss something? don't I try to solve the problem in a wrong way?

- to estimate how hard is it to implement in a 3rd-party library or does it require changes to core Elixir/ErlangVM

Thanks

[Louis Pilfold]

Hi Sergiy

The functionality you've described can be implemented with macros, no need to modify Elixir or Erlang.

To start it could be as simple as defining guards that assert nothing in the production environment.

defmodule Test do

  if Mix.env() == :prod do

    defguard is_my_type(x) when true

  else

    defguard is_my_type(x) when is_atom(x)

  end

  def go(x) when is_my_type(x) do

    x

  end

end

This could be a little error prone though as unless you remember to apply the guard to every clause of the function your logic may change when they are removed. Even if you apply them to every clause if you use exceptions as flow control you may run into problems as values that previously would result in a FunctionClauseError would be passed though.

Plenty to think about! Perhaps experiment with a little proof of concept library and see what happens :)

Cheers,

Louis

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/c7e602a5-a694-46f9-99a5-983b4d50eea0%40googlegroups.com.

[Sergiy Kukunin]

I afraid you missed my point, I might have expressed it poorly. Let's assume I have a simple type: {is_atom(), is_number(), is_binary()}. I want to define a guard to match it. Without reusing I can write a function accepting it:

func({x, y, z}) when is_atom(x) and is_number(y) and is_binary(z), do: true

but then I want to define another function which expects the same tuple:

another({x, y, z}) when is_atom(x) and is_number(y) and is_binary(z), do: true

I don't have a way to define a custom guard to match tuple elements since there is no pattern matching in defguard nor there is `elem` in guards. So both options don't work:

defguard is_mytype({x, y, z}) when is_atom(x) and is_number(y) and is_binary(z)

nor

defguard is_mytype(x) when is_atom(elem(x, 0)) and is_number(elem(x, 1)) and is_binary(elem(x, 2))

Furthermore, I would want to define a function that receives a value of my type inside of complex structure: 

function({:ok, {x, y, z}}) when is_atom(x) and is_number(y) and is_binary(z), do: true

it would be cool to have it defined as

function({:ok, x}) when is_mytype(x), do: true

P.S. Actually, I've found that `elem` works in guards, so I can define my guard without pattern matching. That's good for now, but

func({x, y, z}) when is_atom(x) and is_number(y) and is_binary(z), do: true

sounds cooler, IMHO =)

[Louis Pilfold]

Hi Sergiy

I'm afraid I don't follow. From what I understand of your proposal the current defguard system meets your needs- what are you looking to add?

Cheers,

Louis

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/f6a0f326-ffa4-4b69-998d-6f60a91abe87%40googlegroups.com.

[Sergiy Kukunin]

Actually, that what I understood only in my last message - I can implement it right now. I'm pretty new to Elixir, so that wasn't obvious to me. 

Currently, it seems it's resolved, there are only suggestions to improve syntax, that are too minor.

Thank everyone for assistance

You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/fvn29FjvSks/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFBC%3DM6s0p9po2CsoWXQ-j0gRRiyNdGms13YBUt4-sC%2BMg%40mail.gmail.com.

[Sergiy Kukunin]

Found another problem: can't express "list of strings" in guards nor pattern matching. It's an easy task for typespecs `[String.t(), ...]`, but I can't check typespecs in runtime. Found a very dirty hack, that works for lists up to 5 strings, enjoy:

  defguardp is_list_of_strings(x)

    when (length(x) == 1 and is_binary(hd(x)))

    or (length(x) == 2 and is_binary(hd(x)) and is_binary(hd(tl(x))))

    or (length(x) == 3 and is_binary(hd(x)) and is_binary(hd(tl(x))) and is_binary(hd(tl(tl(x)))))

    or (length(x) == 4 and is_binary(hd(x)) and is_binary(hd(tl(x))) and is_binary(hd(tl(tl(x))))

                        and is_binary(hd(tl(tl(tl(x))))))

    or (length(x) == 5 and is_binary(hd(x)) and is_binary(hd(tl(x))) and is_binary(hd(tl(tl(x))))

                        and is_binary(hd(tl(tl(tl(x))))) and is_binary(hd(tl(tl(tl(tl(x)))))))

Wouldn't it be cool to be able to write something like

defguard is_list_of_strings(x) match_type([String.t(), ...])

Again, I'm pretty new, and I know nothing about the implementation and where Elixir ends and Erlang starts, and how feasible it is. Just an idea =) 

[Louis Pilfold]

Hey

The implementation you've given there is expensive and only works for lists up to a certain length.

To solve this one you'll need to step outside of guard clauses as they only support a limited subset of Elixir/Erlang. The idea is that all operations in guards are very fast and run in constant time, so iterating over a list or arbitrary length is not supported.

Another option would be to write a macro that prepends a type checking statement to a function body, asserting that the arguements are of the correct type.

Cheers,

Louis

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CADp0H2jzEf38pTd9E8bxXxK%2BG5tGeZRrj0PjNoC5S7FePpDA5g%40mail.gmail.com.

[Sergiy Kukunin]

Thanks for the explanation about guards, makes sense. And I totally agree with you, that can be implemented as an assertion at the very begin of function's body.

Again, the question - is there a way to leverage typespecs? Is there a way to implement something like this "hey elixir, is this value corresponds to this type?" 

I'd like to avoid a custom way to define specs for that assertions macros.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CABu8xFBN_wh-pFWt9XJhmUmm7xgJPQWfLCkT9BGP2A%3DBqYfnKw%40mail.gmail.com.

[Louis Pilfold]

Hey

I don't believe that exists, at least not in the standard distribution. Perhaps there is a library that provides this functionality.

Cheers,

Louis

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CADp0H2h2sEadOftQMHfDXFsOYzEH5DM81JF1Wcfdtvq9M4SUyA%40mail.gmail.com.