Dialyzer warning when returning a prolist

[Paulo Zulato]

Hello,

I'm trying to figure out why dialyzer is complaining about a function when I don't explicitly declare its return as a proplist.

I have this small module which returns a proplist in the form {atom(), [atom()]}:

-module(test).

-export([test/0]).
test() ->   
   [    
       {one_test, [a,b,c]},
       {another_test, [d,e,f]}
   ].

For function above, I made the following spec, but dialyzer has complained (as stated below) when I enabled its underspecs warnings.

-spec test() -> [{atom(), [atom()]}].
% src/test.erl
%    Type specification test:test() -> [{atom(),[atom()]}] is a supertype of the success typing: test:test() -> [{'another_test',['d' | 'e' | 'f',...]} | {'one_test',['a' | 'b' | 'c',...]},...]

Well... both lists are non-empty lists as stated by dialyzer, so I modified the spec, without success:

-spec test() -> [{atom(), [atom(), ...]}, ...].
%src/test.erl
%   Type specification test:test() -> [{atom(),[atom(),...]},...] is a supertype of the success typing: test:test() -> [{'another_test',['d' | 'e' | 'f',...]} | {'one_test',['a' | 'b' | 'c',...]},...]

Then I tried to explicitly declare it as a proplist, and dialyzer has accepted it:

-spec test() -> proplists:proplist().

% OK, no warnings.

As it worked, I tried to do the same as proplists module does and created two types, which I used on my new spec. And, surprisingly, dialyzer has complained about it!

% From <https://erlang.org/doc/man/proplists.html>

-type property() :: atom() | tuple().
-type proplist() :: [property()].

-spec test() -> proplist().
%src/test.erl

%   Type specification test:test() -> proplist() is a supertype of the success typing: test:test() -> [{'another_test',['d' | 'e' | 'f',...]} | {'one_test',['a' | 'b' | 'c',...]},...]

Therefore, I could not understand why declaring the return as "proplists:proplist" is OK while declaring it as lists, nonempty lists or creating the same types as proplists are not OK. Why is dialyzer complaining about the other specs? Is dialyzer handling proplists in a different way? Could someone help me to understand this behaviour?

Best Regards,

Paulo Zulato

--
«Quis custodiet ipsos custodes?»

[Fernando Benavides]

Probably because proplists:proplist() is not in your PLT.

Have you tried enabling the unknown warning for dialyzer?

It will likely tell you that it doesn't know the proplists:proplist/0 type.

--

Sent from Gmail Mobile by Brujo Benavides

[Paulo Zulato]

Hi, Fernando,

Thank you for your reply. I tried enabling other warnings as well, but the result was the same:

rebar.config

{erl_opts, [debug_info]}.
{deps, []}.
{dialyzer, [{warnings, [error_handling, underspecs, unknown, unmatched_returns]}]}.

test.erl

-module(test).

-export([test/0]).

%% From <https://erlang.org/doc/man/proplists.html>

-type property() :: atom() | tuple().
-type proplist() :: [property()].
-spec test() -> proplist().

test() ->
   [
       {one_test, [a,b,c]},
       {another_test, [d,e,f]}
   ].

$ rebar3 dialyzer

===> Verifying dependencies...
===> Analyzing applications...
===> Compiling test
===> Dialyzer starting, this may take a while...
===> Updating plt...
===> Resolving files...
===> Checking 202 files in _build/default/rebar3_23.2.6_plt...
===> Doing success typing analysis...
===> Resolving files...
===> Analyzing 1 files with _build/default/rebar3_23.2.6_plt...

src/test.erl

  Type specification test:test() -> proplist() is a supertype of the success typing: test:test() -> [{'another_test',['d' | 'e' | 'f',...]} | {'one_test',['a' | 'b' | 'c',...]},...]

===> Warnings written to _build/default/23.2.6.dialyzer_warnings
===> Warnings occurred running dialyzer: 1

But:

test.erl

-module(test).

-export([test/0]).

-spec test() -> proplists:proplist().

test() ->
   [
       {one_test, [a,b,c]},
       {another_test, [d,e,f]}
   ].

$ rm -rf _build

$ rebar3 dialyzer

===> Verifying dependencies...
===> Analyzing applications...
===> Compiling test
===> Dialyzer starting, this may take a while...
===> Updating plt...
===> Resolving files...
===> Updating base plt...
===> Resolving files...
===> Checking 202 files in ../../home/pzulato/.cache/rebar3/rebar3_23.2.6_plt...
===> Copying ../../home/pzulato/.cache/rebar3/rebar3_23.2.6_plt to _build/default/rebar3_23.2.6_plt...
===> Checking 202 files in _build/default/rebar3_23.2.6_plt...
===> Doing success typing analysis...
===> Resolving files...
===> Analyzing 1 files with _build/default/rebar3_23.2.6_plt...

So I believe it's something else.

Best regards,

Paulo Zulato

--
«Quis custodiet ipsos custodes?»

[Fred Hebert]

The issue is that you have the underspecs option and it will complain every time a more narrow type can define what you are returning. Since you’re returning a static value, your type should be that static value to not be considered underspecified.

[Stanislav Ledenev]

1. As I understand underspec and overspec requires precise indication of types. 

So for a list of a,b,c atoms you _must_ indicate [a,b,c] but not just a list of atoms. 

Because list of atoms implies a list of any size with any atoms not a list of 3 specific atoms a,b,c.

2. The only possible reason why proplists:proplist "works" is that the dialyzer 

doesn't know about this type and it is kinda silent in such cases. Try -Wunknown option.

пн, 17 мая 2021 г. в 23:25, Paulo Zulato <paulo...@gmail.com>:

[Paulo Zulato]

Hi, Fred and Stanislav,

Thank you for your replies.

Now I understood why dialyzer is complaining about the underspec return. It really makes sense.

But I still didn't get why proplists is "working". I've enabled the unknown warning as stated below and I'm using OTP24 and the latest rebar3. The behaviour is the same with or without the unknown warning:

rebar.config:

{erl_opts, [debug_info]}.
{deps, []}.
{dialyzer, [

   {get_warnings, true},

   {warnings, [error_handling, underspecs, unknown, unmatched_returns]}

]}.

Source with proplist emulation:

-module(test).

-export([test/0]).

%% From <https://erlang.org/doc/man/proplists.html>

-type property() :: atom() | tuple().
-type proplist() :: [property()].
-spec test() -> proplist().

test() ->
   [
       {one_test, [a,b,c]},
       {another_test, [d,e,f]}
   ].

Result:

===> Verifying dependencies...
===> Analyzing applications...
===> Compiling test
===> Dialyzer starting, this may take a while...
===> Updating plt...
===> Resolving files...
===> Updating base plt...
===> Resolving files...

===> Checking 204 files in ../../home/pzulato/.cache/rebar3/rebar3_24.0_plt...
===> Copying ../../home/pzulato/.cache/rebar3/rebar3_24.0_plt to _build/default/rebar3_24.0_plt...
===> Checking 204 files in _build/default/rebar3_24.0_plt...

===> Doing success typing analysis...
===> Resolving files...

===> Analyzing 1 files with _build/default/rebar3_24.0_plt...

src/test.erl
Line 8 Column 2: Type specification test:test() -> proplist() is a supertype of the success typing: test:test() -> [{'another_test',['d' | 'e' | 'f',...]} | {'one_test',['a' | 'b' | 'c',...]},...]
===> Warnings written to _build/default/24.0.dialyzer_warnings

===> Warnings occurred running dialyzer: 1

Source with real proplist:

-module(test).

-export([test/0]).

-spec test() -> proplists:proplist().

test() ->
   [
       {one_test, [a,b,c]},
       {another_test, [d,e,f]}
   ].

Result:

===> Verifying dependencies...
===> Analyzing applications...
===> Compiling test
===> Dialyzer starting, this may take a while...
===> Updating plt...
===> Resolving files...

===> Checking 204 files in _build/default/rebar3_24.0_plt...

===> Doing success typing analysis...
===> Resolving files...

===> Analyzing 1 files with _build/default/rebar3_24.0_plt...

Do I need to do something else than adding the unknown warning into rebar.config or is it enough?

Best regards,

Paulo Zulato

--
«Quis custodiet ipsos custodes?»

[Stanislav Ledenev]

I think all of the special with proplist simply cuts down to the fact that proplist's types are exported. 

If you made your own module with types you'll get same "working" dialyzer.

myprops.erl
--------------
-module(myprops).

-type property() :: atom() | tuple().
-type proplist() :: [property()].

-export_type([proplist/0, property/0]).

test.erl
-----------
-spec test() -> myprops:proplist().

Perhaps it is somehow connected with the "dynamic" nature of exported types in the eyes of the dialyzer.
Sorry I can't say more than this. I've never dug so deep with the dialyzer.

[Kostis Sagonas]

On 5/18/21 3:36 PM, Paulo Zulato wrote:
> Hi, Fred and Stanislav,
>
> Thank you for your replies.
> Now I understood why dialyzer is complaining about the underspec return.
> It really makes sense.
> But I still didn't get why proplists is "working".

This is interesting and should be investigated by opening an issue on
GitHub with the two test programs (one with local type declarations and
one using proplists:proplist()).

Enabling unknown (or not) has nothing to do with this issue.

My (educated) guess is that since -Wunderspecs are not on by default,
the case of Wunderspec-ing specs involving remote types has not really
been thoroughly tested.

And there may be an opportunity for dialyzer's improvement here.

My .02

Kostis

[Paulo Zulato]

Hi all,

I have opened an issue as recommended by Kostis. Follow the link: https://github.com/erlang/otp/issues/4851.

P.S.: I don't know why but his email has not reached my inbox (neither spam folder), but I've seem his response on digest mail. Because of this, I'm replying to Stanislav's response.

Best regards,
Paulo Zulato

--
«Quis custodiet ipsos custodes?»