qvm-create-windows-qube Automatically creates

[crazyqube]

I just made my solution for fully automatically creating and installing new Windows qubes from scratch public! It pre-installs Qubes Windows Tools and Firefox so now you don't even have to open Internet Explorer to download a good browser! (lol)

It's currently ready for use at:

https://github.com/crazyqube/qvm-create-windows-qube

If you have any issues or suggestions then by all means create an issue and I'll look into it.

-crazyqube

P.S. If you use it and find it good then please give it a well-deserved star!

[awokd]

'crazyqube' via qubes-users:

Nice script. What is auto-tools or where does it come from? Also, would
it be possible to make available a deterministic/reproducible
slipstreamed ISO with the Windows updates and QWT drivers integrated?
With a SHA256 sum, it could save some steps.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Mon, Aug 19, 2019 at 11:22:21AM +0000, 'crazyqube' via qubes-devel wrote:
> I just made my solution for fully automatically creating and installing new Windows qubes from scratch public! It pre-installs Qubes Windows Tools and Firefox so now you don't even have to open Internet Explorer to download a good browser! (lol)
>
> It's currently ready for use at:
> https://github.com/crazyqube/qvm-create-windows-qube
>
> If you have any issues or suggestions then by all means create an issue and I'll look into it.

I haven't looked into details nor tried it yet, but on the first sight
looks really cool!

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl1cX68ACgkQ24/THMrX
1ywF4gf+I6MNGnhkiNlujuCpwVOojWyltxU7zpagpHJVr6dax/L+N95ySQlFhynI
cIPN50yCwPT3ZBplTneQstYEZnYxd8QMqz3+0A7eaOr3U+ivZZXy/zSJvhVxEwMf
0/BiIoZMNjskprMzO7lx9FExpx3ginyNTvZt9zfo/J//rOTBrwJF7A8TI+yTFe9T
wfypj/Mtys7KnAlLuCFtnyKlgiZxhtDhjF1IxTrLuPAK+Jy6mSOlGTDCamZrjn+L
ZoHfeX/eEc2hrM1M+0zPJvysdCU8opwX3sdS13m2uq9Kp7byoNeCC2bI9rlX1KSC
84tH9paKxqGK8oP9d2f93eF4H3Pefw==
=YRT5
-----END PGP SIGNATURE-----

[799]

Hello,

On Tue, 20 Aug 2019 at 21:34, 'awokd' via qubes-users <qubes...@googlegroups.com> wrote:

'crazyqube' via qubes-users:
> I just made my solution for fully automatically creating and installing new Windows qubes from scratch public! It pre-installs Qubes Windows Tools and Firefox so now you don't even have to open Internet Explorer to download a good browser! (lol)
>
> It's currently ready for use at:
> https://github.com/crazyqube/qvm-create-windows-qube
>
> If you have any issues or suggestions then by all means create an issue and I'll look into it.
>
> -crazyqube
>
> P.S. If you use it and find it good then please give it a well-deserved star!

if this works,it would be great.

I am trying to run through the process but want to do it by CLI from dom0 only.

This would even allow more automation as we can write a script which will do the last manuell steps like creating the windows-mgmt qube etc.

You should be able to run all steps to setup, via dom0:

# create a new AppVM

qvm-create --class AppVM --template fedora-30 --label black windows-mgmt

# Increase storage capacity

qvm-volume extend windows-mgmt:private 20480M

# Install Git in the AppVM (will be gone on next reboot)

qvm-run --auto --pass-io --no-gui --user root windows-mgmt 'dnf install -y git'

# Clone repository of qvm-create-windows-qube

qvm-run --auto --pass-io --no-gui windows-mgmt 'cd Documents && git clone https://github.com/crazyqube/qvm-create-windows-qube'

# Run the script to download all files

qvm-run --auto --pass-io --no-gui windows-mgmt 'cd Documents/qvm-create-windows-qube && ./download-windows.sh'

#  install windows tools

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing qubes-windows-tools

# copy script to dom0

qvm-run --pass-io windows-mgmt 'cat $HOME/Documents/qvm-create-windows-qube/qvm-create-windows-qube.sh' > qvm-create-windows-qube.sh

Feel free to add this to your script/repo.

[799]

[brenda...@gmail.com]

On Tuesday, August 20, 2019 at 6:54:02 PM UTC-4, 799 wrote:

Hello,

On Tue, 20 Aug 2019 at 21:34, 'awokd' via qubes-users <qubes...@googlegroups.com> wrote:

'crazyqube' via qubes-users:
> I just made my solution for fully automatically creating and installing new Windows qubes from scratch public! It pre-installs Qubes Windows Tools and Firefox so now you don't even have to open Internet Explorer to download a good browser! (lol)
>

> If you have any issues or suggestions then by all means create an issue and I'll look into it.
>

I am trying to run through the process but want to do it by CLI from dom0 only.

This would even allow more automation as we can write a script which will do the last manuell steps like creating the windows-mgmt qube etc.

cq appears to have added your dom0 initiation steps, so kudos to both of you.

I opened an issue with dom0's $HOME value being passed to windows-mgmt, which fails to find the iso (admin vs user account name), but with a quick edit it's running now. Will report back.

Brendan

[brenda...@gmail.com]

Hi crazyqube,

I've used this to generate 20-30 VMs.

I've noticed some incomplete installs (50/50). There do seem to be come timing dependencies that sometimes cause failures. I'll be investigating these further next week.

I have some thoughts on changes I'll work on, if you're not planning to work on them, that might address some of these:

- Defaulting to debug=true so that boot problems can be easily diagnosed, with instructions on how the user should manually disable it when finished.

- Increasing the device-stub VM priority from 256 to 1000 during install utilizing xl sched-credit. This dramatically increases the IO throughput for the installation.

- Defaulting to no-network. For the most qubes usage, I think many of us won't plan to connect Windows to the internet.

- If network is explicitly set, only set it to the given option before/after the final boot cycle, to minimize interference.

- Increasing the run-time of the final boot cycle, and possibly overlapping that shutdown with the next creation. Utilize qvm-run shutdown.exe or qvm-run a script instead of qvm-shutdown.

- Refactor repeated code into bash functions.

-  Ensure loop devices in windows-mgmt are removed when finished (keep the qui-devices menu uncluttered)
- Perhaps restart windows-mgmt between VM creations.
- Automate installation of xenvbd 8.2.2 or 8.2.1 after appropriate Windows 7 updates are installed.
- Document that xenvbd is needed for attaching block devices from qui-devices.

- Utilize double digit counter instead of single digit.

- Option to disable windows update permanantly.
- Option to initiate windows update on last reboot (after QWT is installed).
- Increase qrexec_timeout to 600 by default.

Brendan

[Brendan Hoar]

Couple more:

- As windows 7 does not support SCSI unmap, and C and E are on virtual SCSI devices: install sdelete by default and schedule sdelete.exe -z C:\ and sdelete -z E:\ ... largish zero writes are caught at the lvm later and unallocated from storage - plus passed on as discards to physical storage if you’ve enabled this in Qubes (as per testing).

- Possibly work an initial defrag run into the deployment but before sdelete as it saved about 1GB of LVM storage per VM (prob related to lvm chunk size).

B

[799]

Hello Brendan,

Thanks for the improvement list. Some questions:

<brenda...@gmail.com> schrieb am Do., 29. Aug. 2019, 15:27:

- Increasing the device-stub VM priority from 256 to 1000 during install utilizing xl sched-credit. This dramatically increases the IO throughput for the installation.

How can this be done? what is the device-stub VM priority? Can this be set via qvm-prefs?

- Increasing the run-time of the final boot cycle, and possibly overlapping that shutdown with the next creation. Utilize qvm-run shutdown.exe or qvm-run a script instead of qvm-shutdown.

How can this be done?

- Automate installation of xenvbd 8.2.2 or 8.2.1 after appropriate Windows 7 updates are installed.

xenvbd = Qubes Tools ?

[799]

[Brendan Hoar]

On Fri, Aug 30, 2019 at 2:14 AM 799 <one7...@gmail.com> wrote:

Hello Brendan,

Thanks for the improvement list. Some questions:

<brenda...@gmail.com> schrieb am Do., 29. Aug. 2019, 15:27:

- Increasing the device-stub VM priority from 256 to 1000 during install utilizing xl sched-credit. This dramatically increases the IO throughput for the installation.

How can this be done? what is the device-stub VM priority? Can this be set via qvm-prefs?

xl sched-credit -d ${current_name}-dm -w 1000 # execute after sleep nn seconds after each VM startup. -dm is the stub device VM for HVMs. It is temporary until next restart.

- Increasing the run-time of the final boot cycle, and possibly overlapping that shutdown with the next creation. Utilize qvm-run shutdown.exe or qvm-run a script instead of qvm-shutdown.

How can this be done?

$( sleep 360; qvm-run “${current_name}” “shutdown.exe /s /t 0” )& # I think

- Automate installation of xenvbd 8.2.2 or 8.2.1 after appropriate Windows 7 updates are installed.

xenvbd = Qubes Tools ?

It’s in Xen tools, installed by Qubes tools but that module is not installed by default by Qubes tools as it is buggy with unpatched win 7. Since the script patches Win 7 it should be ok. I downloaded the 8.2.2 version of the xenvbd driver (don’t use unsigned daily build) from the xen site and installed that manually. Then you can use qui-devices widget to attach devices.

It’d be nice to add automating that to the winmgmt VM downloads, iso mounting and installing steps.

B

[crazyqube]

Hi Brendan,

I'm not sure why you're getting only 50/50 success rate on the installations. For me it's been perfect every time. This will need to be investigated.

Some of that stuff about increasing I/O throughput and stub priority stuff sounds great as I was unaware of it. Right now when QWT is installed the automatic installation leaves a checkbox related to increasing I/O performance with an extra Xen driver unchecked. I believe I tested it before and as long as you have decent amount of updates installed it appears to work fine. Maybe we can fine a command-line switch to install that extra driver too?

As for the Windows updates do be informed that we must install a minimum of them or QWT will fail to install causing the system to go into recovery mode on next boot. Just having Service Pack 1 (SP1) isn't enough. Hence why I had to at least use wusa.exe to install those to WSU update packages out of the box. (The Servicing Stack and Convenience Rollup which is a bunch of updates in two update packages)

I don't see why restarting windows-mgmt would be necessary. If you look at the create-media.sh script I've tried to make it as safe as possible by setting a TRAP on exit, ^C, etc so if the process is interrupted in anyway it will do it's best to clean up. However, all this may be fixed by packer (package on Debian) which I'm looking into and could completely streamline this process.

Right now I have updates set to download and install automatically but turned off automatic reboots. I didn't want to turn off updates out of the box because as provided the machine is missing many important security updates. For example, it's vulnerable to MS17-010. However, this technically shouldn't matter as long as port 445 it's port forwarded to the LAN or another qube.

I also never had an issue with the qrexec_timeout but perhaps that's because I have a fast SSD.

I've been working on this lately as it would be able to easily specify programs to pre-install:

https://github.com/crazyqube/qvm-create-windows-qube/tree/chocolatey

(Read the todo in the README for more info about research and future changes)

It's mostly done although it requires testing. Also, this:

https://github.com/chocolatey/chocolatey.org/issues/687

is currently a big issue as I don't want people who want their Windows VM behind Tor to be treated like second-class citizens.

Lastly, this project is in the process of being put into official documentation!

https://github.com/QubesOS/qubes-doc/pull/854

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

--

You received this message because you are subscribed to the Google Groups "qubes-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/aa0b38ae-ec25-40cb-a0c4-0c92b3cd2be7%40googlegroups.com.

[crazyqube]

Well, once we have Chocolatey provisioned we can easily specify for Sysinternals to be pre-insatlled.

As for the zeroing, there is an option in the windows-7.xml answer file that provides an option to zero the disk before installation but I disabled it because I though it would slow down the installation.

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

--

You received this message because you are subscribed to the Google Groups "qubes-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/CAOajFeeBikBT%2B5HJfts5wGrNvYtpZqdy2beDSBCV6s3K%3Dqq%3DqA%40mail.gmail.com.