Troubleshooting: ArchLinux install seems to be stuck, logs attached

[Chrome]

Currently on step 6 of https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md

I've tried to run make template as sudo and without (initially without). I think my build is stuck, what should I do?

If you're in the matrix chat, you've seen me talking about it. Cat logs below

terminal log after running make template

done
cat: qubes-src/app-linux-usb-proxy/pkgs/vm-archlinux/archlinux.list: No such file or directory
mkdir -p /home/user/qubes-builder/qubes-src/linux-template-builder/pkgs-for-template/archlinux/pkgs; \
for package in archlinux; do\
    pkgnames=`cat qubes-src/meta-packages/pkgs/vm-archlinux/$package.list`;\
    for pkgname in $pkgnames; do\
        ln -f qubes-src/meta-packages/$pkgname /home/user/qubes-builder/qubes-src/linux-template-builder/pkgs-for-template/archlinux/pkgs/ || exit 1;\
        ln -f qubes-src/meta-packages/$pkgname.sig /home/user/qubes-builder/qubes-src/linux-template-builder/pkgs-for-template/archlinux/pkgs/ 2>/dev/null;\
        if [ -n "" ]; then \
            echo $pkgname >> ""; \
        fi; \
    done; \
done
cat: qubes-src/meta-packages/pkgs/vm-archlinux/archlinux.list: No such file or directory
-> Building template archlinux (logfile: build-logs/template-archlinux.log)...

cat logs

error: failed retrieving file 'qubes.db' from disk : Couldn't open file /tmp/qubes-packages-mirror-repo/pkgs/qubes.db error: failed to synchronize all databases (download library error) :: Synchronizing package databases... core downloading... extra downloading... community downloading... multilib downloading... qubes downloading... error: failed retrieving file 'qubes.db' from disk : Couldn't open file /tmp/qubes-packages-mirror-repo/pkgs/qubes.db error: failed to synchronize all databases (download library error)

[unman]

I'm not familiar with the document you are referring to.
I regularly build the Arch templates, and dont recognise these errors.
It looks as if a) you haven't built some of the packages, and b) the
package database isn't being attached to the template.
Are you using the setup script?
The Arch template is simply built using standard setup:
Run setup.
make get-sources
make qubes-vm
make template.

I'm not sure why that would warrant a dedicated document.
Have you tried that?

[Chrome]

https://github.com/Qubes-Community/Contents/blob/master/docs/os/pentesting/blackarch.md

I'm really following this step and step 1 is to install arch linux minimal via the previously posted guide. I'm going to try to follow your instructions this time but yeah, that's why I'm following that documentation; its what I find when I go to the official qubes documentation site under "External Documentation."

[unman]

It's far too wordy for my taste, and image heavy.
Still, the basic is *exactly* what I suggested, and that works for
building a template.
See if you can build the arch template: not minimal.

[Chrome]

Doesn't seem to do any better. Running cat build-logs/template-archlinux.log shows the following repeated for the majority of the current log. Not too sure what to do from here.

[unman]

On Wed, Jun 16, 2021 at 10:02:34AM -0700, Chrome wrote:
> qubes downloading...
> error: failed retrieving file 'qubes.db' from disk : Couldn't open file
> /tmp/qubes-packages-mirror-repo/pkgs/qubes.db
> error: failed to synchronize all databases (download library error)
>
>

Strange - as I say, works for me.
What template are you using for the build machine?
With debug on do you see any issues with the initial mount of the local
repository?

[Chrome]

How do I turn debug on? What commands should I try running?

As for the template, I'm using fedora 32 minimal. I understood the following steps of the guide to be executed as follows:

https://github.com/Qubes-Community/Contents/blob/master/docs/building/building-archlinux-template.md

Step 0: Dom 0 terminal for all commands

Step 1: open non root xterm in templateVM Fedora32minimal, execute commands

Step 2: Setup appvm build-archlinux2 and set it up per guide and pictures

Step 3-6: All run in appvm terminal per instructions.

If I did something wrong, I don't know what it is. I think the instructions given are quite sparse for the issues I'm encountering. But that can't be helped huh?

[unman]

I've tried with a Fedora-33-minimal base for the builder (I usually use
Debian), and the Archlinux template builds fine.
As I said, there are two problems : first, you're just not building all
the required packages. I don't see how this is happening - if you run
setup, and then `make qubes-vm` you *will* get all the required
packages, or will see feedback on where a package build has failed..

You can turn on DEBUG and increase verbosity by editing builder.conf.
BUT, you should also have build-logs which you can inspect.
Try `make clean-rpms`, delete builder.conf, run setup.sh, increase
verbosity, `make get-sources`, `make qubes-vm`.
Watch for any errors.
If all is well, then `make template`

[Chrome]

I gave this a go with the fedora-32 template and still had issues. I think doing the debian thing would probably be better. Should I just clone my debian-10 template and try the steps again?

[unman]

On Sun, Jun 20, 2021 at 08:29:14AM -0700, Chrome wrote:
>
> > I've tried with a Fedora-33-minimal base for the builder (I usually use
> > Debian), and the Archlinux template builds fine.
> > As I said, there are two problems : first, you're just not building all
> > the required packages. I don't see how this is happening - if you run
> > setup, and then `make qubes-vm` you *will* get all the required
> > packages, or will see feedback on where a package build has failed..
> >
> > You can turn on DEBUG and increase verbosity by editing builder.conf.
> > BUT, you should also have build-logs which you can inspect.
> > Try `make clean-rpms`, delete builder.conf, run setup.sh, increase
> > verbosity, `make get-sources`, `make qubes-vm`.
> > Watch for any errors.
> > If all is well, then `make template`
> >
> > I gave this a go with the fedora-32 template and still had issues. I think
> doing the debian thing would probably be better. Should I just clone my
> debian-10 template and try the steps again?
>

No, because you will have to find the Debian equivalents to the Fedora
packages, and frankly, if you cant get it workign on Fedora (which is
supported), then you are not likely to get it working on Debian (which
isnt).

What issues did you have?
What did it show in the logs?

[Chrome]

Doesn't matter because I give up on this approach or the debian approach. I nuked the fedora 32 minimal template and will give it a fresh start.  I can tell you for sure I followed your instructions and the documentation and got no further than I was when I started this post.

Starting fresh: how would you install ArchLinux in preparation for installing blackarch on top of it?

[unman]

On Sun, Jun 20, 2021 at 09:14:47AM -0700, Chrome wrote:
>
> Doesn't matter because I give up on this approach or the debian approach. I
> nuked the fedora 32 minimal template and will give it a fresh start. I can
> tell you for sure I followed your instructions and the documentation and
> got no further than I was when I started this post.
>
> Starting fresh: how would you install ArchLinux in preparation for
> installing blackarch on top of it?
>

Good luck with the build - use a 33 template, turn up logging and post
your results.
If you want a prebuilt template, I have recent builds at
https://qubes.3isec.org

[Haisam Khalil]

thank you unman, I’ll run man on the commands to figure out the logging. I’ll post up when done. Appreciate all the help to date.

--

Best,

Haisam Khalil
C: 347-397-0124

[unman]

On Mon, Jun 21, 2021 at 08:57:07AM -0400, Haisam Khalil wrote:
> On Mon, Jun 21, 2021 at 6:52 AM unman <un...@thirdeyesecurity.org> wrote:
>
> > On Sun, Jun 20, 2021 at 09:14:47AM -0700, Chrome wrote:
> > >
> > > Doesn't matter because I give up on this approach or the debian
> > approach. I
> > > nuked the fedora 32 minimal template and will give it a fresh start. I
> > can
> > > tell you for sure I followed your instructions and the documentation and
> > > got no further than I was when I started this post.
> > >
> > > Starting fresh: how would you install ArchLinux in preparation for
> > > installing blackarch on top of it?
> > >
> >
> > Good luck with the build - use a 33 template, turn up logging and post
> > your results.
> > If you want a prebuilt template, I have recent builds at
> > https://qubes.3isec.org
>

> thank you unman, I???ll run man on the commands to figure out the logging.
> I???ll post up when done. Appreciate all the help to date.
>
> > <https://qubes.3isec.org>
> >
> --
> Best,

There are "DEBUG" and "VERBOSE" settings in builder.conf, explained in
doc/Configuration.md
Those parameters will impact on the log files in build-logs (in
particular I think having them set will not create a log file for the
template build), but they will produce a far more verbose output.

This should help you to identify where the problem lies.
Best of luck