Re: Qubes 4.0 FC26
536 views
Skip to first unread message
Marek Marczykowski-Górecki
Sep 9, 2017, 7:43:33 PM9/9/17
to Frédéric Epitre, qubes-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On Sun, Sep 10, 2017 at 12:31:03AM +0200, Frédéric Epitre wrote:
> Hi,
>
> Just to give you some news about what I have done these days on Qubes:
Thanks! Please cc qubes-devel ML for such things. I've added it here
now.
> - I succeed to build all the components (dom0 and vm) for the release
> 4.0 with Fedora 26. For xen-4.8 I had to fix few lines in the patches of
> xen-4.6 for GCC7 and also to add two patches of the version 4.8.2 for
> the current 4.8.1.
Oh, there is already 4.8.2. I'll check how feasible is updating to
4.8.2.
> - I solved one more fallthrough in the core-admin and everything is good.
>
> - The last thing I'm currently stuck is the installer. I have to create
> a fedora 26 branch and import the new version of anaconda and also new
> version of pykickstart(>=1.33 for fedora 26 support) :
This isn't needed for fc26 template, and it's too late to change dom0
distribution version for 4.0. So, don't bother. For Qubes 4.1, there
will be probably newer version at that time.
> * I tried to build pykickstart but tests create errors (need to add more
> than you first patch to skip all the errors). So skipping errors, it
> allows to obtain the packages.
>
> * Then, PUNGI...........During the nosetests, it bothers me with
> something like Pungi:INFO: <type 'exceptions.AttributeError'>
> AttributeError("'F27_RepoData' object has no attribute 'gpgkey'",), or
> other erros always related to "'F27_RepoData' object has no attribute
> 'gpgkey'". I don't understand neither why it is talking about FC27 nor
> gpgkey because this supplementary parameter added should not be
> considered in the test right?
Looks related to a patch adding package signature verification. Probably
needs to be updated. But as said above - don't worry about it now.
> I will debug in the next days with a new fresh branch for fedora 26 in
> the installer.
>
> Best,
>
> Frédéric
>
>
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJZtHyeAAoJENuP0xzK19csP64H/RHDPGvk2toK18ibmP9MZV5c
vLYp6NtHSU6zZrIxqcc+6/yXWEykhYT2FbNx+elJeB+k5dRpJoeZ0e9CnZtFSV9X
CsfN9j5IH59xm2mMUeXxWC4ObgHYVQDcxEtPWeOU4IYq02G45XBdYd4TVk/COtmR
NFJV5lHpzGqeCiDY3O1Sq2VKHbU4DMbXOlz00x6PADAT3GTErC/JcCiEVyoXzN6b
ItiG/qgP8QsqA6VkVDazjvyopRyuNsn3HkLMl/NK8iUlOwIdoDyGgaYT2zgeah38
3pnWcUAJ8LmV79JGH9i39NHI2Ww+px7hVAVTbT7eXQgdn3sta9CZ3ZKAae2iHbQ=
=3RbI
-----END PGP SIGNATURE-----
Hash: SHA256
On Sun, Sep 10, 2017 at 12:31:03AM +0200, Frédéric Epitre wrote:
> Hi,
>
> Just to give you some news about what I have done these days on Qubes:
Thanks! Please cc qubes-devel ML for such things. I've added it here
now.
> - I succeed to build all the components (dom0 and vm) for the release
> 4.0 with Fedora 26. For xen-4.8 I had to fix few lines in the patches of
> xen-4.6 for GCC7 and also to add two patches of the version 4.8.2 for
> the current 4.8.1.
Oh, there is already 4.8.2. I'll check how feasible is updating to
4.8.2.
> - I solved one more fallthrough in the core-admin and everything is good.
>
> - The last thing I'm currently stuck is the installer. I have to create
> a fedora 26 branch and import the new version of anaconda and also new
> version of pykickstart(>=1.33 for fedora 26 support) :
This isn't needed for fc26 template, and it's too late to change dom0
distribution version for 4.0. So, don't bother. For Qubes 4.1, there
will be probably newer version at that time.
> * I tried to build pykickstart but tests create errors (need to add more
> than you first patch to skip all the errors). So skipping errors, it
> allows to obtain the packages.
>
> * Then, PUNGI...........During the nosetests, it bothers me with
> something like Pungi:INFO: <type 'exceptions.AttributeError'>
> AttributeError("'F27_RepoData' object has no attribute 'gpgkey'",), or
> other erros always related to "'F27_RepoData' object has no attribute
> 'gpgkey'". I don't understand neither why it is talking about FC27 nor
> gpgkey because this supplementary parameter added should not be
> considered in the test right?
Looks related to a patch adding package signature verification. Probably
needs to be updated. But as said above - don't worry about it now.
> I will debug in the next days with a new fresh branch for fedora 26 in
> the installer.
>
> Best,
>
> Frédéric
>
>
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJZtHyeAAoJENuP0xzK19csP64H/RHDPGvk2toK18ibmP9MZV5c
vLYp6NtHSU6zZrIxqcc+6/yXWEykhYT2FbNx+elJeB+k5dRpJoeZ0e9CnZtFSV9X
CsfN9j5IH59xm2mMUeXxWC4ObgHYVQDcxEtPWeOU4IYq02G45XBdYd4TVk/COtmR
NFJV5lHpzGqeCiDY3O1Sq2VKHbU4DMbXOlz00x6PADAT3GTErC/JcCiEVyoXzN6b
ItiG/qgP8QsqA6VkVDazjvyopRyuNsn3HkLMl/NK8iUlOwIdoDyGgaYT2zgeah38
3pnWcUAJ8LmV79JGH9i39NHI2Ww+px7hVAVTbT7eXQgdn3sta9CZ3ZKAae2iHbQ=
=3RbI
-----END PGP SIGNATURE-----
Frédéric Pierret (Epitre)
Sep 10, 2017, 5:35:22 AM9/10/17
to qubes-devel
Fedora 26 template works well in R3.2 and R4.0RC. I just want to build the whole to prepare the work for the next release 4.1 as you said.
I probably found where the problem was for the F27_RepoData. It is in pykickstart command for repo.py. I continue to dig in and when I finish I will push all the things on my repos.
Best,
Frédéric
I probably found where the problem was for the F27_RepoData. It is in pykickstart command for repo.py. I continue to dig in and when I finish I will push all the things on my repos.
Best,
Frédéric
Marek Marczykowski-Górecki
Sep 10, 2017, 8:09:01 PM9/10/17
to Frédéric Epitre, qubes-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
On Sun, Sep 10, 2017 at 01:43:24AM +0200, Marek Marczykowski-Górecki wrote:
> On Sun, Sep 10, 2017 at 12:31:03AM +0200, Frédéric Epitre wrote:
> > - I succeed to build all the components (dom0 and vm) for the release
> > 4.0 with Fedora 26. For xen-4.8 I had to fix few lines in the patches of
> > xen-4.6 for GCC7 and also to add two patches of the version 4.8.2 for
> > the current 4.8.1.
>
> Oh, there is already 4.8.2. I'll check how feasible is updating to
> 4.8.2.
Ok, xen 4.8.2 seems to be fine. Do you have those updated GCC7 patches
> On Sun, Sep 10, 2017 at 12:31:03AM +0200, Frédéric Epitre wrote:
> > - I succeed to build all the components (dom0 and vm) for the release
> > 4.0 with Fedora 26. For xen-4.8 I had to fix few lines in the patches of
> > xen-4.6 for GCC7 and also to add two patches of the version 4.8.2 for
> > the current 4.8.1.
>
> Oh, there is already 4.8.2. I'll check how feasible is updating to
> 4.8.2.
somewhere pushed? Would you care to open a pull request?
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
6xWg/qJA4UkdaEK9ytNB2tWF2Gv434YKjbpPzmolrKgHCAeXGE14BCwk1V+Dnnh3
VEbuRGmaPbUPTifkgyzZ8H9D1CjnialtMRCbXj2alLM55Qt+y2g9lexXp0fKXere
QVS0LypC34/VOFOTGu7RkXj0FUY68qNZFtbg7WnoAOVvmojvsaSP/tZBeTwwORvO
wduRDBvgKy8R5/Uf938rgIshouHSAXcZ+YjBcWR82K9RdDLlhpYAVzh3sCnm8Pun
iO7NoujiSSVzQuNMiRTv2nhRsPkQw90E8l3JhmGyoGzY6M1OBiDp6yJdaZqQ55s=
=ODzs
-----END PGP SIGNATURE-----
Frédéric Pierret (Epitre)
Sep 11, 2017, 12:31:05 PM9/11/17
to qubes-devel
I have created pull requests for xen-4.6 and xen-4.8. I will test in the next days xen-4.9 because theses fix are normally included except mini-os linking (need to check) but I'm finishing to debug the build for the "4.1". I updated Anaconda 26.20.11, Pungi 4.1.18, Lorax 26.9 and Pykickstart 2.38 and I'm having an error during iso creation in post-install configuration for the rebuilding of the initramfs images. Just for your information:
2017-09-11 14:20:13,997: rebuilding boot/initramfs-4.12.10-20.pvops.qubes.x86_64.img
rebuilding boot/initramfs-4.12.10-20.pvops.qubes.x86_64.img
Traceback (most recent call last):
File "/sbin/lorax", line 289, in <module>
main()
File "/sbin/lorax", line 135, in main
remove_temp=True, verify=opts.verify)
File "/usr/lib/python3.6/site-packages/pylorax/__init__.py", line 355, in run
treebuilder.rebuild_initrds(add_args=anaconda_args)
File "/usr/lib/python3.6/site-packages/pylorax/treebuilder.py", line 273, in rebuild_initrds
runcmd(cmd, root=self.vars.inroot)
File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 341, in runcmd
return execWithRedirect(cmd[0], cmd[1:], **kwargs)
File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 228, in execWithRedirect
env_add=env_add, reset_handlers=reset_handlers, reset_lang=reset_lang)[0]
File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 201, in _run_program
raise subprocess.CalledProcessError(proc.returncode, argv, output)
subprocess.CalledProcessError: Command '['dracut', '--nomdadmconf', '--nolvmconf', '--xz', '--install', '/.buildstamp', '--no-early-microcode', '--add', 'fips', '--add', 'anaconda pollcdrom
qemu qemu-net', '--force', 'boot/initramfs-4.12.10-20.pvops.qubes.x86_64.img', '4.12.10-20.pvops.qubes.x86_64']' returned non-zero exit status 1.
I'm using my latest kernels builds and it should be not related to that because it works for Qubes 4 with dom0=fc25 but I will try with the 4.9 version just to be sure. I also tried a more recent version of Lorax but I obtain the same error. I will debug tomorrow.
2017-09-11 14:20:13,997: rebuilding boot/initramfs-4.12.10-20.pvops.qubes.x86_64.img
rebuilding boot/initramfs-4.12.10-20.pvops.qubes.x86_64.img
Traceback (most recent call last):
File "/sbin/lorax", line 289, in <module>
main()
File "/sbin/lorax", line 135, in main
remove_temp=True, verify=opts.verify)
File "/usr/lib/python3.6/site-packages/pylorax/__init__.py", line 355, in run
treebuilder.rebuild_initrds(add_args=anaconda_args)
File "/usr/lib/python3.6/site-packages/pylorax/treebuilder.py", line 273, in rebuild_initrds
runcmd(cmd, root=self.vars.inroot)
File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 341, in runcmd
return execWithRedirect(cmd[0], cmd[1:], **kwargs)
File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 228, in execWithRedirect
env_add=env_add, reset_handlers=reset_handlers, reset_lang=reset_lang)[0]
File "/usr/lib/python3.6/site-packages/pylorax/executils.py", line 201, in _run_program
raise subprocess.CalledProcessError(proc.returncode, argv, output)
subprocess.CalledProcessError: Command '['dracut', '--nomdadmconf', '--nolvmconf', '--xz', '--install', '/.buildstamp', '--no-early-microcode', '--add', 'fips', '--add', 'anaconda pollcdrom
qemu qemu-net', '--force', 'boot/initramfs-4.12.10-20.pvops.qubes.x86_64.img', '4.12.10-20.pvops.qubes.x86_64']' returned non-zero exit status 1.
I'm using my latest kernels builds and it should be not related to that because it works for Qubes 4 with dom0=fc25 but I will try with the 4.9 version just to be sure. I also tried a more recent version of Lorax but I obtain the same error. I will debug tomorrow.
Le dimanche 10 septembre 2017 01:43:33 UTC+2, Marek Marczykowski-Górecki a écrit :
Frédéric Pierret (Epitre)
Sep 22, 2017, 5:23:14 PM9/22/17
to qubes-devel
I finally succeeded to build Qubes with Fedora 26 as dom0 and template. Globally, it was a matter of some fixes with respect to GCC7 for xen and other small minor corrections in qubes packages.
There is quite a lot small changes in codes between Fedora 25 and 26+ with respect the installer packages (Pungi, Lorax, Pykickstart and Anaconda).
So to debug it was (still) long because I have not found a shortcut to bypass the whole build of the iso (which is about ~20min on my servers each time), neither a way to debug easily Anaconda.
In other words, in order to not do all the stages of Pungi, it seems that we have to comment several things it the Makefile. Notably, I rewrite all the necessary code (patches) for Anaconda with respect to the previous commits to have, like the others, a source package, a spec file and patches (In fact, I have done it for Fedora 26 and 27). In my opinion, it is more clear and consistent.
For the Fedora 26 template (Qubes 3.2 and 4.x), one has to take into account a very slight modification in the qubes dnf plugin due to newer version (a temporarly fix https://github.com/fepitre/qubes-core-agent-linux/commit/8fffaadb41b9fb5ef5231287483040b31e02d57b).
I should push everything on github on devel branches in the next days, a time to polish and debug a bit more. Stay tuned !
There is quite a lot small changes in codes between Fedora 25 and 26+ with respect the installer packages (Pungi, Lorax, Pykickstart and Anaconda).
So to debug it was (still) long because I have not found a shortcut to bypass the whole build of the iso (which is about ~20min on my servers each time), neither a way to debug easily Anaconda.
In other words, in order to not do all the stages of Pungi, it seems that we have to comment several things it the Makefile. Notably, I rewrite all the necessary code (patches) for Anaconda with respect to the previous commits to have, like the others, a source package, a spec file and patches (In fact, I have done it for Fedora 26 and 27). In my opinion, it is more clear and consistent.
For the Fedora 26 template (Qubes 3.2 and 4.x), one has to take into account a very slight modification in the qubes dnf plugin due to newer version (a temporarly fix https://github.com/fepitre/qubes-core-agent-linux/commit/8fffaadb41b9fb5ef5231287483040b31e02d57b).
I should push everything on github on devel branches in the next days, a time to polish and debug a bit more. Stay tuned !
Le dimanche 10 septembre 2017 01:43:33 UTC+2, Marek Marczykowski-Górecki a écrit :
Marek Marczykowski-Górecki
Sep 22, 2017, 5:41:52 PM9/22/17
to Frédéric Pierret (Epitre), qubes-devel
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hash: SHA256
On Fri, Sep 22, 2017 at 02:23:14PM -0700, Frédéric Pierret (Epitre) wrote:
> I finally succeeded to build Qubes with Fedora 26 as dom0 and template.
Yay!
> I finally succeeded to build Qubes with Fedora 26 as dom0 and template.
> Globally, it was a matter of some fixes with respect to GCC7 for xen and
> other small minor corrections in qubes packages.
> There is quite a lot small changes in codes between Fedora 25 and 26+ with
> respect the installer packages (Pungi, Lorax, Pykickstart and Anaconda).
downloaded packages. Unfortunately, upstream maintainers don't care about
it. Besides porting patches, it's a good idea to test it. See
.travis.yml and conf/travis*.ks - there are configs intentionally
including repositories with unsigned packages.
> So to debug it was (still) long because I have not found a shortcut to
> bypass the whole build of the iso (which is about ~20min on my servers each
> time), neither a way to debug easily Anaconda.
have:
1. PXE boot, like here: https://unix.stackexchange.com/questions/186302/fedora-network-install-via-pxe-boot
2. kickstart file - look at /root/anaconda-ks.cfg after installation
3. wake on lan.
This way, it's possible to automate the whole workflow, so you hit
"build" and come back half an hour later to see result of the test.
For trivial fixes, you can do that from tty2 (edit files in
/usr/...). Make sure to kill crashed
anaconda with SIGKILL, otherwise it will shutdown the system. And AFAIR
you also need to kill X server before starting new one.
bVt0SOIFpG6QLMIzADHq5m7vcEzvBUdfn/ULEdyrulKgPZkj+Tx+ZbHmMP4+SLqi
DypdXyvjBOJLAjMHfhTgVQ4D36ywN+ZYjBtwDaJAb2oYHMBmO1+fYPPM3ZJpSI23
Q3xYDFv0DUqQBuNL39uHN+VJdnWZxbtay+F4DfgptlajiLmsax0tbwcqrx6qGtfO
1s3uxTvASv8MojE5Lpw3ZpFFV3kuLzKx6C8862KK1/vsXqsfCtLrKhuYp22X+d/8
+gGixjaIRUhCL/EZiDE+Tot7A+Eq5Rh2meryrg01y4wRqO3BtI00vZE6B/imLyw=
=/9AM
-----END PGP SIGNATURE-----
Outback Dingo
Sep 27, 2017, 4:57:59 AM9/27/17
to Marek Marczykowski-Górecki, Frédéric Pierret (Epitre), qubes-devel
So curious now, is this duplicable? do you havea working iso image?
did it install properly? it is functional ?
> --
> You received this message because you are subscribed to the Google Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
> To post to this group, send email to qubes...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20170922214145.GV1116%40mail-itl.
> For more options, visit https://groups.google.com/d/optout.
did it install properly? it is functional ?
> You received this message because you are subscribed to the Google Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel...@googlegroups.com.
> To post to this group, send email to qubes...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/20170922214145.GV1116%40mail-itl.
> For more options, visit https://groups.google.com/d/optout.
Frédéric Pierret (Epitre)
Sep 27, 2017, 5:14:02 AM9/27/17
to qubes-devel
Almost but be patient. Note that Qubes with FC26 (or maybe FC27) as dom0 is planed to might be 4.1. This is still in current development and debug.
At the release point of view, there is still some adjustments to do for Anaconda because there is some of changes in the code with FC>=26:
- the check of lock root account
- the check of non empty user password
was done in each spokes respectively.
Now they merged the check classes in helper.py. So we need to adapt it. This is only what is is missing the series of patches: https://github.com/fepitre/qubes-installer-qubes-os/tree/devel-f26/anaconda
But as I said, there is still things to debug for dom0. So if you want to try and help:
- pick a qubes-os-master.conf
- put dom0 to fc26
- override branch for the installer with my repository and devel-f26 branch
- all others modifications should now be in the Qubes master repository
Another point: stable Qubes 4.0 is almost released and you can now build Fedora 26 template for 3.2 and 4.0 (check last commits on Qubes master branches).
Best,
At the release point of view, there is still some adjustments to do for Anaconda because there is some of changes in the code with FC>=26:
- the check of lock root account
- the check of non empty user password
was done in each spokes respectively.
Now they merged the check classes in helper.py. So we need to adapt it. This is only what is is missing the series of patches: https://github.com/fepitre/qubes-installer-qubes-os/tree/devel-f26/anaconda
But as I said, there is still things to debug for dom0. So if you want to try and help:
- pick a qubes-os-master.conf
- put dom0 to fc26
- override branch for the installer with my repository and devel-f26 branch
- all others modifications should now be in the Qubes master repository
Another point: stable Qubes 4.0 is almost released and you can now build Fedora 26 template for 3.2 and 4.0 (check last commits on Qubes master branches).
Best,
Outback Dingo
Oct 7, 2017, 6:49:42 AM10/7/17
to Frédéric Pierret (Epitre), qubes-devel
does anyone have a useable FC26 core domU built ...ive been waiting
1.5 years just to get qubes reloaded... and so far nothing is
working
> https://groups.google.com/d/msgid/qubes-devel/ccc9617f-5dd2-4e21-b3e4-54e317a8c3b4%40googlegroups.com.
1.5 years just to get qubes reloaded... and so far nothing is
working
Frédéric Pierret (fepitre)
Oct 7, 2017, 6:56:06 AM10/7/17
to qubes-devel
Why don't you try Qubes 4.0 rc1 and soon rc2 (dom0 fc25) ? The necessity of having FC26 dom0 currently (in terms of days/few weeks) is, IMHO, meaningless. Currently the core team is focused on releasing Qubes 4.0 and I'm focused on trying to debug things happening for FC26, FC27, CentOS 7 for the dom0 and also for kernels.
We are all doing our best!
We are all doing our best!
Outback Dingo
Oct 7, 2017, 2:36:25 PM10/7/17
to Frédéric Pierret (fepitre), qubes-devel
because i tried qubes 4.0 RC and its also a fail..... So I was
hoping someone had something more up to date
domU and XEN wise....
> https://groups.google.com/d/msgid/qubes-devel/abeed2ef-94d2-41f2-ba0c-8b2b39e5aa6d%40googlegroups.com.
hoping someone had something more up to date
domU and XEN wise....
Reply all
Reply to author
Forward
0 new messages