Anyone Have Luck Running Gitian in Qubes Debian?

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I've been trying to get Gitian to run inside a Qubes Debian Jessie
AppVM so that I can build Bitcoin. Unfortunately, even after
tinkering with the standard Gitian instructions for days, I've had no
luck. I'm happy to post my partial results here in the hope that
someone can give me pointers -- but before I write up my broken
process, I figure I'll just ask -- has anyone successfully run a
Gitian build inside a Qubes AppVM? If someone's already done it, then
hopefully I don't need to expend large amounts of effort figuring out
why my method is broken. If I'm actually venturing into new
territory, then I'll write up what I did.

Cheers,
- -Jeremy Rand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJV/mwYAAoJEAHN/EbZ1y06uTsQAJBjTRA3gwb5b3RERC5JiKGg
IdbGgWk9fYqfWB9BOqF0SQ+l+X7bbAelomVa7DU8nQw8Lk7Z7JBYQAqVRSMtBc0a
o6xxz8YVvbYMtRAvsbS8o/Vr3Xlai3NanE8NMjNcrKvdnVTFo+N17Ts9MAKEQkKl
4glDL0FXN9NOiF1ANS6O6FgL4FqgeXA+mE2uh4bp9tcdvXRGxniXB0HbxBeowK3t
UZCY5eBMUhq0QOAzYyRP4T3trfT8p7bJ+Wjdew/lTRWWplBoKb5rLm/5kv6XNW3D
MgV3HzQbkxR3zdcmlTc/XiiIrUM61Pcf7kva5fjnMZZLVHsC8Hi+szjarZo5IXcP
nDqZzgjqOiy+rxZRvtqCRGp+L9g/YZ50GdF8N7hOV8K+jlkuImptCK8r+9M8luXl
PI8RtH60EbqHIOIp77MS92Y3GSkjBXBVeFeg87NQHj5LXSjdVUcZqXWQvbXGuIeE
m/za1QDmWb+Xj2DT9i/pPv2GRNwkIHhwqk7o4wrJqvj0KVJsdX0DWNjC4Im2RLbP
l3LTuWwpUIO6/SXV2MTjXwug6gHRHpvVtu7j7WsXM5ECAdjHY/MULlQrUeFAg7VR
XUNO/zNvElJld9OP7aYz2GzXgyaIrsf2uKowHZ3H3x8AXQP5RKmkMZlP2rRbBQiB
T7oVBPBAzX6/eX59oF/T
=yO7T
-----END PGP SIGNATURE-----

[Alex]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 09/20/2015 10:20 AM, Jeremy Rand wrote:
> [...] has anyone successfully run a Gitian build inside a Qubes
> AppVM?
No, I did not. But I just read the readme on gitian-builder, and
stumbled across the need for virtualization.

The readme says it can be done with KVM, LXC or VirtualBox (even if
the latter is not still usable yet), and the default seems to be KVM.
Since the appVM is already virtualized itself, this may lead to
problems in another virtualization environment (KVM requiring
processor hardware virtualization support).

Given that LXC does not have those strong requirements, and that
gitian's use of virtualization is to solve problems on reproducibility
of builds (not for security), I suggest you to try with the LXC
gitian-build process.

Or, just let us know the process you tried!

- --
Alex

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJV/m4QAAoJENNOJZnNP8uDcIsP/1QyvhdcbgaaH8DOtxQ5k6hE
3TeEt+Rin4lNQIRlisoSMGaeQz1niMeq5fhGEKf4IBd0AKtJZsAyvI/WrRp158fn
CZE7H8zreS8mk8eCePj4uMqBVWi2CjUiH1xvtGAuXJgsCJkEOOm+JIGNnGUWpVPh
qra0zAckEAeZ1C99XPWwph0sUiWwOlFy2VT6BFrx/OkKDAK2Ra/ZvXuYXfclenxg
7i4DtdfaFPgo9B/SmztdbQjpLCJ8DDrPIMo6IaYzZIYnnB4jXQxSRYDtwcCZyUof
SHhLwU93drOhcZzPIVfdWcVKEBN/D40R6U9kVsIAOC0pt+57xOSdayomuMe7ipjp
fkDrmuMIaLbr8fZ916fiTCGsHV2FCJd1KMw+v6LTNT8nr4zewSfiDbtJiVDMggqk
/BBBzNRdViLhrYxcSOH1QNcJWT3h8NJnvEJCb4vA1LpZlQVT934NxYrGD+HYJbXe
PDD/gTn8+EbzamW7RJNRYVWPm/XGb9Wu7ZzsLoFkufUJJys79nUdJ3o2A/2FcBb6
8xw07ZALQYcY018vO2T9Rw4L1aTUEIua8vXEgZgQNXVVWbW4Cos7pSoRrH2e21Fy
P6KFvSuzaVf3BgzBpvdGTbFRXagHZ7AHIjIYrXDlvdu5XZ9lsCwAjDDR+U3G+F+c
B6H4eQtEhAMEmW+vPvBm
=vxXB
-----END PGP SIGNATURE-----

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 09/20/2015 08:28 AM, Alex wrote:
> On 09/20/2015 10:20 AM, Jeremy Rand wrote:
>> [...] has anyone successfully run a Gitian build inside a Qubes
>> AppVM?
> No, I did not. But I just read the readme on gitian-builder, and
> stumbled across the need for virtualization.
>
> The readme says it can be done with KVM, LXC or VirtualBox (even
> if the latter is not still usable yet), and the default seems to be
> KVM. Since the appVM is already virtualized itself, this may lead
> to problems in another virtualization environment (KVM requiring
> processor hardware virtualization support).
>
> Given that LXC does not have those strong requirements, and that
> gitian's use of virtualization is to solve problems on
> reproducibility of builds (not for security), I suggest you to try
> with the LXC gitian-build process.
>
> Or, just let us know the process you tried!

Yep, I was using the LXC mode for exactly that reason. I will write
up the exact process I used sometime in the next couple days and post
it here, assuming no one else posts successful instructions before then.

Cheers,
- -Jeremy Rand

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJV/m89AAoJEAHN/EbZ1y065CUQANepr7NfxYTGUgf57wxmvRV5
cDz2q2kgTfV/fxRRRKWMlDnfngRPUBQEph2nMSd0vOSTRkTMiz4z77lIrgubfku7
GTXNc05yuml75aWEuqH9nXmesVs9QKYZYIBOgJ673ri/HrMzT9x4ujELl/Z394T9
VGnOP8hGAhc/Os0dACyzbAYmRPAh9IfHYtgv/++kVkR1yfoER5YrrGrBJiwFc1A7
jBcht/YYbKGy1T4XedNU82TyHQgLXCSaE0xrxs4BcKPxwtmQVmb3baqrmj6tDdA4
fntGxGGgA9+DwCIfWMRyNV8xmOr+lErqtv5UezySOf3jt9znPPtUODqm91KVjQkr
vp6Yp+24meEHBEvL542qZJ8E2KU9jE5RzhqqzEsdKduqi22hrS7Uw3+W8YoXUhFc
2WfGKpUlnmjdH3uplIEzpezfWHhSXhZTa1++FuV/RpiYqzmVaTSNPcfPf3nF+wKd
XoDF5b/14+znwgHlBUIW1ZmjkzTbnoiEuuu0NrB48Hs/VIyKg+Mp63fXfgVmzvI/
O4iWYUZmhcd2HEHk9tGcjOziRR0lguDZFBhDxLKHyuMiBgox1c9q7U9uewKDmFVb
N4nn7JRNS4fapZmgOmbuQllyyqtPRPmf7DOuz5c87MQspR9aV1p3QpqkEVhlOt/L
oshlOYWAj7ody8v+yAlQ
=Af1a
-----END PGP SIGNATURE-----

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 09/20/2015 08:33 AM, Jeremy Rand wrote:
> Yep, I was using the LXC mode for exactly that reason. I will
> write up the exact process I used sometime in the next couple days
> and post it here, assuming no one else posts successful
> instructions before then.
>

> Cheers, -Jeremy Rand

Okay, here's what I did. Any differences from the official Bitcoin
instructions for non-Qubes Debian are labeled. I actually attempted
it with a Whonix AppVM this time rather than Debian Jessie.

https://gist.github.com/JeremyRand/1b7536675b9783705001

Anyone have ideas on what might be causing the issue?

Thanks,

- -Jeremy Rand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJV/05SAAoJEAHN/EbZ1y06K5wP/AyBtVP6Wxh7anqfpO1pY0D3
si1DhnAoevnOgCD1NPUSnCuqP8jW80iu8+9tSo4FmT0zJ0FlLrcDkqt9qk8jO/bN
EmlUTy8ZiKy9b4qRgljCxS0tYeRYOyrAvIQ6M8FiBUywlzwe/BYaEv2th3prwK3A
WYrdyzjczV2ADC6z7EX3pLOImlfw6Hrq1noeSVJnujaFVsKDbFTxooEZ6I9RiwJr
/hmLFuq8JYwpA6FMsG+h0AKEYif7BAZFmAEnVCQN1fJ33dg5QjHZgwRgtRx3aqcY
zGDzw0/hpfxY/N9oIAJQzOePsrsS/tdvwDcII/JjVEgn8PNcGlcMn2xptH1v2WbZ
Os0ZGnI29p9Sr6dUSz7vktQDCPdDUh3BN07pUXkDmGhzAE5fLWchGMEYC5ihuqLo
lC/liZNYfTFG+AwMZP8MBVIAiMJVskSjGUzEGrZfdMKxv4Tt8SGnC+x9DMV9Qv2f
+8m88hnJeHNYzVyS1bEmNmINASvPhvGt6/Oik9JDboUHiDYpX+kYdrMXHz0yemgg
+ttrd7eMYl4pGbkwzeWlvS3bzde+rqlo0KBNNIVhu67I+DkekRHOu7tIwbseQ+D1
FimXM0KyQM51qs1Oq/rnwMtBtab3EgmK4e0rpKWA45gSy2dZ0hcPOUUQgLm81rde
JH5Admh5BrUit29Nsl1l
=5pIJ
-----END PGP SIGNATURE-----

[HW42]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Jeremy Rand:

> On 09/20/2015 08:33 AM, Jeremy Rand wrote:
>> Yep, I was using the LXC mode for exactly that reason. I will
>> write up the exact process I used sometime in the next couple
>> days and post it here, assuming no one else posts successful
>> instructions before then.
>
>> Cheers, -Jeremy Rand
>
> Okay, here's what I did. Any differences from the official Bitcoin
> instructions for non-Qubes Debian are labeled. I actually
> attempted it with a Whonix AppVM this time rather than Debian
> Jessie.
>
> https://gist.github.com/JeremyRand/1b7536675b9783705001
>
> Anyone have ideas on what might be causing the issue?

It fails fetching the packages list from the apt-cacher-ng which should
run on the host (the LXC host i.e. the VM you are working with). Is
apt-cacher-ng running on the host? Does it accept connections from the
LXC guest (check config and firewall).

If the connection to apt-cacher-ng works check the logs of apt-cacher-ng.

Are you building offline?
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJWACgSAAoJEOSsySeKZGgWIg0QAIk60f+F+fWVtzJtYUpXivTd
L5FcHIJtnoJ11/6Rk3f288QX4htehpQC0l7Zhi+85XGtZNVG/AP4ZQ8HovBV+nRz
7SdY66ezYYg++cKB+xzYQVtObWC7cw/PfeYIzB0B/KO8hxSilZvNqEFz8OdlCTBQ
u13WSgcR4UM9xiYOf8BQAswm0PjpyuUutYZuIDPWkAVN652rBM2bNBP1fmsPXqFw
+oUi0IK4Zwh5BKnSRi04/XCrD0gl9IKyLBSLSfNbaKv+A5a3zcFKemG+1hZbouBY
QLyZHIPiFyygFU9rrqrV4x+BpFNywk/PYobNsuwiS8HRw3txfycupwDph5TRRpKv
IksruiEL+J4EgQU+fHSEqQL5Q/2IM/s1IIIJLAp9pH/zbiZQzDKWQhZsS+D0KLUx
4m6HGzEgRqBx86CtHUsfAEvLoVr3omstxa2GtGKNvTG8PN0w1nsbaOXlOqNantCU
L2tMG48FEfzjGDf92fxO6JHhkUXp99Lhf/RXMFC25QwUu/d9S/E8JHiBB89g6vhl
ABVmkCUcunP2gAYsJDmiASWOvYUKwNlvuEFhJdFslfr3VVddrl9K9ABBMRk2ezcE
5rxFMoXr/0CAYSC6ZgvF9IHr1Ek8sCWn5iyXtGqGYOiLC7tAPE9hzRca9lq16bZX
LEhQuA+9RRtRQyQ6Z8vZ
=xD5w
-----END PGP SIGNATURE-----

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 09/21/2015 03:53 PM, HW42 wrote:
> It fails fetching the packages list from the apt-cacher-ng which
> should run on the host (the LXC host i.e. the VM you are working
> with). Is apt-cacher-ng running on the host? Does it accept
> connections from the LXC guest (check config and firewall).
>
> If the connection to apt-cacher-ng works check the logs of
> apt-cacher-ng.
>
> Are you building offline?
>

I just updated the Gist with a bit more info:
https://gist.github.com/JeremyRand/1b7536675b9783705001#file-3-apt-cache
r-ng-curl

Basically, I can access http://127.0.0.1:3142/ with curl from the LXC
host, but not http://10.0.3.2:3142/ from the host. netstat suggests
that apt-cacher-ng is running on 0.0.0.0. So I guess this might be a
firewall issue? Any suggestions on how I would check on this?

Thanks,
- -Jeremy Rand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWAkNCAAoJEAHN/EbZ1y06kJAP/RdqPLumgAtfoKyXzIy1Bvyc
F6y+MER16uz/XBr/UKJq8mXoR31BekRZN7mTUHNEgVBcbjLH+PfkF9S8SwhvrvLD
/lzsxoHC4fFKoLh76zKV6fjAYvRAeZOowI+Zb4OuRevcOprSvtcvj7u1C+XFfQ76
n9xRjzhTawj5OoWVbTtTNRDW8d9j6G4uqL8cJ49KPPYBeea77s+eFIq/ndUvXHEp
qJdH4RmU0fwjFcBmVpYjbvImL4X+bpDIZ696SJmMwSJvlBUYfzdGR0ss3DJ7RX61
SSMTFREfUiGVyEQTiP45OBWPcNjaNm9wP66dHegyUcnV7Lx5QNCMcrom5o86agH+
TvI2GuxmeVEGjRzOrRojdYvs1midM7WTWez4N0ZF9FPEzz/NXAwq/fOgyCkmjtb9
UN7C/yTfEL6mPwEGM59sGB04gAqcQt9DYFPEEwMe77STTA3c5OAvr1YfcQXd24jR
BpeZJsP31sbfeZP8Jojr1QLg5Csy61YLcGVxtbib/CzhHfx7pZSkwpFSfAE9/qQw
3WI3rttIml2cwEBu3W7jGbTwwudrwk+LgG3cdQMaOsnnroyFXjp22HJ9t4b37WJt
RoIcBKEgWpZDdCD9xeWT6McghxNQse7lhmVycc9HKDSF1L92snEjtAyKZsZQHwAM
VPnjNB3FK119Ck4NVsFF
=Vyqk
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, Sep 23, 2015 at 06:14:45AM +0000, Jeremy Rand wrote:
> On 09/21/2015 03:53 PM, HW42 wrote:
> > It fails fetching the packages list from the apt-cacher-ng which
> > should run on the host (the LXC host i.e. the VM you are working
> > with). Is apt-cacher-ng running on the host? Does it accept
> > connections from the LXC guest (check config and firewall).
> >
> > If the connection to apt-cacher-ng works check the logs of
> > apt-cacher-ng.
> >
> > Are you building offline?
> >
>
> I just updated the Gist with a bit more info:
> https://gist.github.com/JeremyRand/1b7536675b9783705001#file-3-apt-cache
> r-ng-curl
>
> Basically, I can access http://127.0.0.1:3142/ with curl from the LXC
> host, but not http://10.0.3.2:3142/ from the host. netstat suggests
> that apt-cacher-ng is running on 0.0.0.0. So I guess this might be a
> firewall issue? Any suggestions on how I would check on this?

Most likely - by default firewall in Qubes blocks all incoming traffic.
You can add an exception with (all the traffic on the br0 interface):
sudo iptables -I INPUT -i br0 -j ACCEPT

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBCAAGBQJWAlvIAAoJENuP0xzK19csNRkH/iWRBwFS5Cdkv8wwb6mP9LqI
Blk21tLtiz/pnTG/AIS7nwuq2LYRuHAxH5Whfi11X9+zYOx6desDAq7b6jiL8gJS
AnRVRJTlfgKJQVCWlwJVWr9wzlNsTIJWqpWvtUSCAYva6vv6FKUthzgU6hnZWp4B
yDb/Y4eeSd2bNsu2hEgIZCmhSMcusuoF5jJUcGkQsrd1Ra3mwPDJKueTfcRAnAWG
rnmaTRyoo+tUJ89++qCze2OC7l0fk52MoCDYfzXEJEmxtSI0QAoIgKvXA70H9x0p
vGZLFtvYJxkruy9k2K2BocBRO9xG6Zo69qKCv1jwlXNttgjE+B+5NzHAsse2K18=
=U0z9
-----END PGP SIGNATURE-----

[Jeremy Rand]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 09/23/2015 07:59 AM, Marek Marczykowski-Górecki wrote:
> On Wed, Sep 23, 2015 at 06:14:45AM +0000, Jeremy Rand wrote:
>> I just updated the Gist with a bit more info:
>> https://gist.github.com/JeremyRand/1b7536675b9783705001#file-3-apt-ca
che
>>
>>
r-ng-curl
>
>> Basically, I can access http://127.0.0.1:3142/ with curl from the
>> LXC host, but not http://10.0.3.2:3142/ from the host. netstat
>> suggests that apt-cacher-ng is running on 0.0.0.0. So I guess
>> this might be a firewall issue? Any suggestions on how I would
>> check on this?
>
> Most likely - by default firewall in Qubes blocks all incoming
> traffic. You can add an exception with (all the traffic on the br0
> interface): sudo iptables -I INPUT -i br0 -j ACCEPT
>
>

Hi Marek,

Thank you! Initially I didn't think that your instructions worked,
because I got this:

user@host:~$ curl http://10.0.3.2:3142/
curl: (7) Couldn't connect to server
user@host:~$ sudo iptables -I INPUT -i br0 -j ACCEPT
user@host:~$ curl http://10.0.3.2:3142/
curl: (7) Couldn't connect to server

However, after doing that, the actual Gitian build fully worked (and
produced the correct hashes), even though running curl from the LXC
host didn't work.

Much appreciated.

- -Jeremy Rand
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWB0jKAAoJEAHN/EbZ1y06k5YP/0VxYfeZ1TnoukUnEpGD3kWV
1gnL0G3QtFaP8rw6P0MlxKIdTYLKV0qoen0qFRfYTEC4LThdQ8u3lsRZDbCv1NKu
tThdMO/TnRcFlgmBmzSIJGgei3TZvKRErdnf+hkI2qzILeIooEw0S/v0jnrPFeoL
jnxkPzOgEWX45SB6pcixR0DLOggR7VOE8iRqdMFwooS6BKM+EIbdK2Bkgik6xVa1
H1OBhKa2rhSh5ciA9PNhxv8ZyEtRUTjfhOMN8aDk6/aPhMnGNgea7d3mjRfZkOfR
ah3o+MUlZdKS3MjVheoHmULYMuAh4kX+3LCxxSFfvmXvYeSdfrkQt7Stw6hMVUhB
JihwjKiHfR8ftP0fR42QO7SXtL5N8SHcMQV9TIMQtgk3yNd2bqnIq872wTvOBOdD
eWkftD53VF/lywz0UCxS5jPujxB0OHHpYr/6gfhlYBMvmnrQy51RQYSTVkycZMNo
enIfaiLDQfZNyeaDKwjTFXi1RqncnJV1Fj9NkdYQMnXsqq78chav69yH9kJGTar5
JQxbPJflsFr/XsMe83ift74OSVWjqqInyIMxkMpZGcKSY5XLBDUh0exA6VqbmA8u
/YONco8N9wFR0gMhnmMTbzRdO/5yRLC0HZONcObUvu9xc+vIAYKyX0TvNlGRwrqa
j7TohebzOKx2r3nu7A0c
=fsz+
-----END PGP SIGNATURE-----