rebar3 as test compile not fetching test dependencies

[Anthony Howe]

Has anyone seen this happening?

Using rebar3 3.16.1, I remove the whole `_build` directory, my `rebar.config`
has a `test` profile with `deps` (gun and meck) specified. I do `rebar3 as test
compile` and it fetches all the default dependencies, but not the test ones,
which of course means the test suites fail.

I also tried rebar 3.18.0.

--
Anthony C Howe SnertSoft
ach...@snert.com Twitter: SirWumpus BarricadeMX & Milters
http://snert.com/ http://nanozen.snert.com/ http://snertsoft.com/

[Fred Hebert]

Here's a sample run with 3.17.0, which I had already pre-installed:

λ /tmp → rebar3 new lib chk

===> Writing chk/src/chk.erl
===> Writing chk/src/chk.app.src
===> Writing chk/rebar.config
===> Writing chk/.gitignore
===> Writing chk/LICENSE
===> Writing chk/README.md
λ /tmp → cd chk

λ chk → vim rebar.config

λ chk → cat rebar.config
{profiles, [
  {test, [{deps, [gun, meck]}]}
]}.

λ chk → rebar3 as test compile
===> Verifying dependencies...
===> Fetching gun v1.3.3
===> Fetching meck v0.9.2
===> Fetching cowlib v2.7.3
===> Analyzing applications...
===> Compiling meck

===> Compiling cowlib

===> Compiling gun

_build/test/lib/gun/src/gun.erl:654:29: Warning: erlang:get_stacktrace/0 is removed; use the new try/catch syntax for retrieving the stack backtrace

===> Analyzing applications...

===> Compiling chk

I would probably advise to double-check the configuration format (see the sample one I have included and displayed). This is pretty core functionality and thoroughly tested, so I'd be very surprised to see it break.

[Anthony Howe]

So I tried your test example:

```
docky$ rebar3 new lib chk

===> Writing chk/src/chk.erl
===> Writing chk/src/chk.app.src
===> Writing chk/rebar.config
===> Writing chk/.gitignore
===> Writing chk/LICENSE
===> Writing chk/README.md

docky$ cd chk
docky$ ls
LICENSE README.md rebar.config src
docky$ vi rear.config
docky$ vi rebar.config
docky$ ls
LICENSE README.md rebar.config src
docky$ rebar3 as test compile
===> Verifying dependencies...

===> Analyzing applications...
===> Compiling chk

docky$ cat rebar.config
{erl_opts, [debug_info]}.
{deps, []}.

{profiles, [
{test, [{deps, [gun, meck]}]}
]}.

docky$ rebar3 -v
rebar 3.18.0 on Erlang/OTP 21 Erts 10.3.5.6
docky$
```

Also there is no gun or merk related files in ~/.cache/rebar3/hex/hexpm/packages.

> ach...@snert.com <mailto:ach...@snert.com>     Twitter: SirWumpus           
> BarricadeMX & Milters
> http://snert.com/ <http://snert.com/>    http://nanozen.snert.com/
> <http://nanozen.snert.com/>     http://snertsoft.com/ <http://snertsoft.com/>

[Tristan Sloughter]

Really weird. I just copy/pasted the commands and config you have here and it fetches the test deps as expected:

$ rebar3 as test compile

===> Verifying dependencies...
===> Fetching gun v1.3.3
===> Fetching meck v0.9.2
===> Fetching cowlib v2.7.3
===> Analyzing applications...
===> Compiling meck
===> Compiling cowlib
===> Compiling gun
_build/test/lib/gun/src/gun.erl:654:29: Warning: erlang:get_stacktrace/0 is removed; use the new try/catch syntax for retrieving the stack backtrace

===> Analyzing applications...
===> Compiling chk

$ rebar3 -v
rebar 3.18.0 on Erlang/OTP 24 Erts 12.0.1

How do you install rebar3? Are you downloading the escript or building from source? (Just throwing out random thoughts of what could be different).

[Anthony Howe]

On 2022-01-13 16:52, Tristan Sloughter wrote:
> Really weird. I just copy/pasted the commands and config you have here and it fetches the test deps as expected:

I tried this on another laptop running a different VM with different Erlang and
it works fine. I think this might be peculiar to that VM. Bad install of
Erlang and/or rebar3?

> $ rebar3 -v
> rebar 3.18.0 on Erlang/OTP 24 Erts 12.0.1
>
> How do you install rebar3? Are you downloading the escript or building from source? (Just throwing out random thoughts of what could be different).
>

I build rebar3 from the cloned GitHub repository.

[Anthony Howe]

On 2022-01-13 17:01, Anthony Howe wrote:
> On 2022-01-13 16:52, Tristan Sloughter wrote:
>> Really weird. I just copy/pasted the commands and config you have here and it fetches the test deps as expected:
>
> I tried this on another laptop running a different VM with different Erlang and
> it works fine. I think this might be peculiar to that VM. Bad install of
> Erlang and/or rebar3?
>

So I just did a build and install of Erlang OTP-23.3.4.9 (--without-javac
--without-odbc) from a clone of the GitHub repo. (pre-built Ubuntu packages are
an older release and install a tonne the X-windows libraries just for `wx`
thppt) and also rebuilt rebar3 from source using this version of Erlang and
repeated the `chk` test. It still fails to fetch test dependencies.

I must be missing something in the build/install process despite following the
instructions.

Other VMs (NetBSD, FreeBSD, pre-built Erlang) behave. Would like to fix this
Ubuntu server VM since its important for work.

[Fred Hebert]

you can generally get more details of what is going on by running things with DIAGNOSTIC=1 or DEBUG=1, which will expand output to show internal details. DEBUG=1 is intended for users and will try to show which configs effect what, whereas DIAGNOSTIC=1 is for maintainers and goes into even more details about the results of intermediary commands.

You may also want to look at the environment variables and global configuration files. It's possible for some behavior to changed based on that:

https://rebar3.org/docs/configuration/configuration/#environment-configuration

Another thing that could exist is some mess with the local cache (~/.cache/rebar3) . I have no idea how, and wouldn't imagine that failure mode, but it's the one other bit of state hanging around that could be significant aside from proxies and whatnot.

[Anthony Howe]

On 2022-01-14 08:52, Fred Hebert wrote:
> you can generally get more details of what is going on by running things with
> DIAGNOSTIC=1 or DEBUG=1, which will expand output to show internal details.
> DEBUG=1 is intended for users and will try to show which configs effect what,
> whereas DIAGNOSTIC=1 is for maintainers and goes into even more details about
> the results of intermediary commands.
>
> You may also want to look at the environment variables and global configuration
> files. It's possible for some behavior to changed based on that:
> https://rebar3.org/docs/configuration/configuration/#environment-configuration
> <https://rebar3.org/docs/configuration/configuration/#environment-configuration>
>
> Another thing that could exist is some mess with the local cache
> (~/.cache/rebar3) . I have no idea how, and wouldn't imagine that failure mode,
> but it's the one other bit of state hanging around that could be significant
> aside from proxies and whatnot.

Thank you all for your help in trying to resolve this issue.

My solution in the end was clone the VM from a snapshot shortly after it was
built, apply package upgrades, then build/install Erlang and Rebar3 from GitHub
repos. Not ideal, but its a VM and most of my work-in-progress was pushed to
external repos. Starting from a "clean" base appears to have resolved the
issue, though still no idea how the problem came about.