Fwd: Re: openQA, Automated Testing and 'vbox-unattended'
Bernhard M. Wiedemann
Hash: SHA1
- -------- Original-Nachricht --------
Betreff: Re: openQA, Automated Testing and 'vbox-unattended'
Datum: Tue, 24 Jan 2012 08:22:57 +0100
Von: Bernhard M. Wiedemann <bernh...@lsmod.de>
An: Alexey Eromenko <al4...@gmail.com>
Kopie (CC): os-au...@googlegroups.com
- -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Am 22.01.2012 14:50, schrieb Alexey Eromenko:
> Hello dear SUSERs,
>
> (SUSE users, that is)
Hello Alexey,
> I have, over the years, have tried to do what you did in openQA
> using various approaches:
>
> -keyboard input + screenshots + timer -tried doing checked parts
> of screenshots (rectangles of the original) -tried to do relative
> checking of screenshots (to verify 99% compatibility with
> original, not 100%) -screenshot checksum comparison
>
> I have come to conclusion, that the above methods are highly
> unreliable. Tests gave different results from one run to the next,
> esp. on highly loaded hosts, where operations timed out...
We found that there is mostly one thing that can prevent running 4
tests in parallel on a dedicated quadcore machine with enough RAM. It
is concurrent disk-IO which competes for the read-write-head seeks.
This is why we intially only ran one test at a time and then switched
to using a SLC-SSD with 250MB/s random writes (as opposed to the
1-10MB/s random writes you get on a HDD with multiple accesses).
Also part of the reliability is achieved by reacting on graphics
output (e.g. through the waitstandstill function), allowing for
different timings.
> The only method that is reliable is fully unattended install, by
> using Guest-OS-specific installer, such as SUSE's AutoYast and
> Debian Preseed.
part of the reason to have openQA is to ensure that (non-automatic)
install is working, so we would like to keep this part running for
openSUSE-Factory, Debian-testing/sid and Fedora-rawhide.
> My suggestion: Mix guest-specific unattended install, and then use
> protocol to connect between the installed VM and the host OS
> (either IP packet or VirtualBox Guest Exec).
>
> I have written a program to do unattended installs for VirtualBox,
> here: https://forums.virtualbox.org/viewtopic.php?f=10&t=46798
It is a cool project.
wondering: Does MSWindows also have some autoyast-equivalent?
>
> (it is fully SUSE compatible) After install finishes, just use
> "VBox GuestControl Execute", and read data from guest OS. This
> will guarantee you that it is alive.
>
> Like this: VBoxManage guestcontrol "my Linux VM" execute --image
> "/bin/uname" --username root --password 123456 --wait-stdout -- -a
> (it should return guest OS kernel version)
I can see that this method has some advantages, but it also heavily
relies on VBox as hypervisor and on guest additions (which might not
be available for some OSes supported by our OS-autoinst test-engine).
And it is also impossible to run kvm and VirtualBox on the same
machine at the same time (with intel
https://www.virtualbox.org/ticket/3830).
> Please forward this email to other openQA members, and share
> discussion with me. I have vast amount of VirtualBox experience,
> and can help you with the topic.
you can join the http://groups.google.com/group/os-autoinst?pli=1 ML
to discuss with all two openQA developers :-)
Ciao
Bernhard M.
- -----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8eXFEACgkQSTYLOx37oWTdgwCg0hl6sU1ABbx7txhyaiFwmG4K
p7oAoOn6KnBM21W9vZ+fJgLWgc/ZDoTN
=ac5d
- -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8eXkoACgkQSTYLOx37oWTIKwCg7xrjslAD86yJjEG+iqJnfajo
vOgAn1yFfCuwQEhhmBLFqUXGmZ9UtFxG
=o2pP
-----END PGP SIGNATURE-----