Getting OS-Autoinst working for Mageia 3 testing

[Glen Ogilvie]

Hi,

I've been working recently on doing some QA for Mageia 3.  It seems like using OS-Autoinst would be really helpful for us.

I've been trying to get it working, but so far, not quite as much success as I would like.  So, if this list is active, then, here
are my questions, and what I've done so far:

git clone git://github.com/bmwiedemann/os-autoinst.git
I've installed the following packages:

opencv-devel  ( I only have 2.3, not 2.4)
swig
perl-Data-Dump
perl-devel
ffmpeg2theora

I've tried the: autogen.sh, configure, make, make install.
This seems to end up missing a whole bunch of stuff, when I then try isotovideo (my mageia iso). 

So, then I've tried locally, with env.sh copied from env.sh.sample,
and have modified bmwqemu.pm to support Mageia, and created by own distri/mageia/ directory


Problems I have so far:

When it gets to the end of the distr inst.d scripts (I only have 3 so far), the VM shutdowns, with this output.
I think that tinycv.cc is not working, as features2d.hpp looks like maybe a 2.4 feature of opencv?


FIXME: perl fallback at ../os-autoinst/ppm.pm line 217.
Perl exited with active threads:
        1 running and unjoined
        0 finished and unjoined
        2 running and detached
../os-autoinst/tools/ppmtompg qemuscreenshot video/Mageia-3-beta4-x86_64-DVD
Input #0, image2, from 'ppmtompg.tmp/%08d.ppm':
  Duration: 00:00:01.40, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: ppm, rgb24, 800x600, 25 fps, 25 tbr, 25 tbn, 25 tbc

  0:00:01.50 audio: 0kbps video: 573kbps, time elapsed: 00:00:01
  0:00:01.40 audio: 0kbps video: 573kbps, time elapsed: 00:00:01
qemu: terminating on signal 15 from pid 32234
Building PPM C-libraries...
swig -perl5 -c++ tinycv.i
c++ -O3 -c -fPIC tinycv_wrap.cxx `perl -MExtUtils::Embed -e ccopts`
c++ -O3 -fPIC `pkg-config --cflags opencv` -o tinycv.o -c tinycv.cc
tinycv.cc:10:42: fatal error: opencv2/nonfree/features2d.hpp: No such file or directory
compilation terminated.
make: *** [tinycv.o] Error 1
rm tinycv_wrap.cxx tinycv_wrap.o

====
||| checking check /home/test/tmp/../os-autoinst/distri/mageia/check.pm
||| checking consoletest_setup /home/test/tmp/../os-autoinst/distri/mageia/consoletest.d/010_consoletest_setup.pm
--- {"audiodumps":[],"md5_result":"na","name":"consoletest_setup","screenshots":[],"result":"na"}


consoletest_setup: na
isosize: ok
overall: fail



Also:
On startup, I get an error about audio.  See below:

test@zed tmp]$ ../os-autoinst/tools/isotovideo Mageia-3-beta4-x86_64-DVD.iso
testing /home/test/tmp/Mageia-3-beta4-x86_64-DVD.iso
Building PPM C-libraries...
swig -perl5 -c++ tinycv.i
c++ -O3 -c -fPIC tinycv_wrap.cxx `perl -MExtUtils::Embed -e ccopts`
c++ -O3 -fPIC `pkg-config --cflags opencv` -o tinycv.o -c tinycv.cc
tinycv.cc:10:42: fatal error: opencv2/nonfree/features2d.hpp: No such file or directory
compilation terminated.
make: *** [tinycv.o] Error 1
rm tinycv_wrap.cxx tinycv_wrap.o
+++BACKEND_JSON: {"backend_info":{},"management_port":15223,"backend":"backend::qemu"}
Formatting 'raid/1', fmt=raw size=8589934592
Formatting 'raid/2', fmt=raw size=8589934592
Formatting 'raid/3', fmt=raw size=8589934592
Formatting 'raid/4', fmt=raw size=8589934592
starting: /usr/bin/qemu-kvm -m 1024 -net user -monitor tcp:127.0.0.1:15222,server,nowait -net nic,model=virtio,macaddr=52:54:00:12:34:56 -serial file:serial0 -soundhw ac97 -vga cirrus -S -drive file=raid/l1,if=virtio -drive file=raid/l2,if=virtio -boot dc -cdrom /home/test/tmp/Mageia-3-beta4-x86_64-DVD.iso -vnc :99 -usb -usbdevice tablet -smp 1
audio: Failed to create voice `ac97.pi'
audio: Failed to create voice `ac97.mc'
audio: Failed to create voice `ac97.pi'
audio: Failed to create voice `ac97.mc'
QEMU 1.0 monitor - type 'help' for more information
iso_size=4044357632
(qemu) cont
(qemu)
||| starting bootloader ../os-autoinst/distri/mageia/inst.d/020_bootloader.pm


I am also a little puzzled about the waitidle, as the VM seems to use quite a lot of CPU, even when it should be doing nothing except waiting for user input.

I seem to get messages like:
>>> waitidle: timed out after 19

Not sure what that means?

and when I try using: waitstillimage, I think it crashes, maybe due to the opencv stuff?

So, suggestions, ideas, etc, would be helpful.

I've tried going down the track of updating the opencv package, but a number of packages I already have installed depend on the existing version of opencv, so it's not too easy.

Regards
Glen Ogilvie

[Dominik Heidler]

Hi,

first yust ignore that automake stuff - its not needed and just messing
around. The ppmclibs lib has is's own plain makefile (which is called by
autoinst at runtime, if the library is missing) - which usually works
better.

You may try to install opencv 2.4.
I don't know, if it is because you have 2.3 but you are missing
something like /usr/include/opencv2/nonfree/features2d.hpp

Dominik

Am 03.04.2013 10:37, schrieb Glen Ogilvie:
> Hi,
>
> I've been working recently on doing some QA for Mageia 3. It seems like
> using OS-Autoinst would be really helpful for us.
>
> I've been trying to get it working, but so far, not quite as much
> success as I would like. So, if this list is active, then, here
> are my questions, and what I've done so far:
>
> git clone git://github.com/bmwiedemann/os-autoinst.git
> I've installed the following packages:
>

> opencv-devel (/I only have 2.3, not 2.4)/

> swig
> perl-Data-Dump
> perl-devel
> ffmpeg2theora
>
> I've tried the: autogen.sh, configure, make, make install.
> This seems to end up missing a whole bunch of stuff, when I then try
> isotovideo (my mageia iso).
>
> So, then I've tried locally, with env.sh copied from env.sh.sample,
> and have modified bmwqemu.pm to support Mageia, and created by own
> distri/mageia/ directory
>
>

> *Problems I have so far:*

> *Also:
> On startup, I get an error about audio. See below:*

> --
> Sie haben diese Nachricht erhalten, weil Sie der Google Groups-Gruppe
> OS-Autoinst beigetreten sind.
> Um Ihr Abonnement für diese Gruppe zu beenden und keine E-Mails mehr von
> dieser Gruppe zu erhalten, senden Sie eine Email an
> os-autoinst...@googlegroups.com.
> Weitere Optionen: https://groups.google.com/groups/opt_out
>
>


--
Dominik Heidler
Maxfeldstr. 5, D-90409 Nuernberg, Germany
Phone +49-911-74053-141
SUSE LINUX Products GmbH, Nuernberg
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer
HRB 21284 (AG Nürnberg)
_________________________________________

ドミニク

[Bernhard M. Wiedemann]

Am 03.04.2013 10:37, schrieb Glen Ogilvie:

> Hi,
>
> I've been working recently on doing some QA for Mageia 3. It seems like
> using OS-Autoinst would be really helpful for us.
>
> I've been trying to get it working, but so far, not quite as much
> success as I would like. So, if this list is active, then, here
> are my questions, and what I've done so far:
>
> git clone git://github.com/bmwiedemann/os-autoinst.git
> I've installed the following packages:
>

> opencv-devel (/I only have 2.3, not 2.4)/

> swig
> perl-Data-Dump
> perl-devel
> ffmpeg2theora
>
> I've tried the: autogen.sh, configure, make, make install.
> This seems to end up missing a whole bunch of stuff, when I then try
> isotovideo (my mageia iso).

We usually just run the code from the git checkout (much easier for
development), so the install part may indeed be broken.

> So, then I've tried locally, with env.sh copied from env.sh.sample,
> and have modified bmwqemu.pm to support Mageia, and created by own
> distri/mageia/ directory

That is the right thing to do.

> *Problems I have so far:*
>

> When it gets to the end of the distr inst.d scripts (I only have 3 so
> far), the VM shutdowns, with this output.
> I think that tinycv.cc is not working, as features2d.hpp looks like
> maybe a 2.4 feature of opencv?
>
> FIXME: perl fallback at ../os-autoinst/ppm.pm line 217.
> Perl exited with active threads:
> 1 running and unjoined
> 0 finished and unjoined
> 2 running and detached

It errors out because it wants to use C code via the Inline::C perl
module (perl-Inline rpm in openSUSE, but you can install it via cpan, as
fallback), which needs gcc, make and maybe perl-devel.

> ../os-autoinst/tools/ppmtompg qemuscreenshot video/Mageia-3-beta4-x86_64-DVD
> Input #0, image2, from 'ppmtompg.tmp/%08d.ppm':
> Duration: 00:00:01.40, start: 0.000000, bitrate: N/A
> Stream #0:0: Video: ppm, rgb24, 800x600, 25 fps, 25 tbr, 25 tbn, 25 tbc
>
> 0:00:01.50 audio: 0kbps video: 573kbps, time elapsed: 00:00:01
> 0:00:01.40 audio: 0kbps video: 573kbps, time elapsed: 00:00:01
> qemu: terminating on signal 15 from pid 32234
> Building PPM C-libraries...
> swig -perl5 -c++ tinycv.i
> c++ -O3 -c -fPIC tinycv_wrap.cxx `perl -MExtUtils::Embed -e ccopts`
> c++ -O3 -fPIC `pkg-config --cflags opencv` -o tinycv.o -c tinycv.cc
> tinycv.cc:10:42: fatal error: opencv2/nonfree/features2d.hpp: No such
> file or directory
> compilation terminated.
> make: *** [tinycv.o] Error 1

I checked and we use opencv-2.4.1 in one setup, but the other one runs
completely without opencv atm (which might change soon with updates from
the openSUSE-Team).


>
> *Also:
> On startup, I get an error about audio. See below:*
...

> starting: /usr/bin/qemu-kvm -m 1024 -net user -monitor
> tcp:127.0.0.1:15222,server,nowait -net
> nic,model=virtio,macaddr=52:54:00:12:34:56 -serial file:serial0 -soundhw
> ac97 -vga cirrus -S -drive file=raid/l1,if=virtio -drive
> file=raid/l2,if=virtio -boot dc -cdrom
> /home/test/tmp/Mageia-3-beta4-x86_64-DVD.iso -vnc :99 -usb -usbdevice
> tablet -smp 1
> audio: Failed to create voice `ac97.pi'
> audio: Failed to create voice `ac97.mc'
> audio: Failed to create voice `ac97.pi'
> audio: Failed to create voice `ac97.mc'

I also get these regularly and they dont seem to matter. Audio tests work.

> I am also a little puzzled about the waitidle, as the VM seems to use
> quite a lot of CPU, even when it should be doing nothing except waiting
> for user input.
>
> I seem to get messages like:
> >>> waitidle: timed out after 19
>
> Not sure what that means?

The VM can be busy-waiting in the bootloader (because there is no OS to
schedule time-slices or such), but later when it booted Linux, and you
watch "top", the CPU load of your kvm process should be well below 50%
or are you running it with CPU-emulation?

> and when I try using: waitstillimage, I think it crashes, maybe due to
> the opencv stuff?

rather because of the Inline::C stuff, which should be easy to fix.

> So, suggestions, ideas, etc, would be helpful.
>
> I've tried going down the track of updating the opencv package, but a
> number of packages I already have installed depend on the existing
> version of opencv, so it's not too easy.

At the moment, you should be able to get it working without opencv.

Ciao
Bernhard M.

[Dominik Heidler]

This /Failed to create voice/ messages can be ignored - it works anyway.

Am 03.04.2013 10:37, schrieb Glen Ogilvie:

> udio: Failed to create voice `ac97.pi'
> audio: Failed to create voice `ac97.mc'
> audio: Failed to create voice `ac97.pi'
> audio: Failed to create voice `ac97.mc'

Waitidle is a function which waits until it detects qemu consuming less
cpu-time than a certain threshold, but as all whe wait* functions it has
a timeout. Exceeding that timeout is not fatal - especially with
waitidle, which is not 100% reliable.


waitstillimage is implemented without opencv using inline C.
( see ppm.pl -> maxbytediffC() )
The waitstillimage function has an argument called maxdiff, which
defaults to 20.
If you call waitstillimage with a stilltime of 7 seconds, it will
compare the current image with the image from 7 seconds ago:
Each pixel has three bytes and the difference between a byte in the
current image and a byte in the image from 7 seconds ago may not exceed 20.

When using qemu-kvm or vbox, a maxdiff of 0 would work as well, but we
are also testing against real hardware, and VGA is analog so some values
will change slightly.

[Glen Ogilvie]

On Wednesday, April 3, 2013 10:36:10 PM UTC+13, Bernhard M. Wiedemann wrote:

Am 03.04.2013 10:37, schrieb Glen Ogilvie:
> Hi,
>
> I've been working recently on doing some QA for Mageia 3.  It seems like
> using OS-Autoinst would be really helpful for us.
>
> I've been trying to get it working, but so far, not quite as much
> success as I would like.  So, if this list is active, then, here
> are my questions, and what I've done so far:
>
> git clone git://github.com/bmwiedemann/os-autoinst.git
> I've installed the following packages:
>
> opencv-devel  (/I only have 2.3, not 2.4)/
> swig
> perl-Data-Dump
> perl-devel
> ffmpeg2theora
>
> I've tried the: autogen.sh, configure, make, make install.
> This seems to end up missing a whole bunch of stuff, when I then try
> isotovideo (my mageia iso).

We usually just run the code from the git checkout (much easier for
development), so the install part may indeed be broken.

OK, will do.

> So, then I've tried locally, with env.sh copied from env.sh.sample,
> and have modified bmwqemu.pm to support Mageia, and created by own
> distri/mageia/ directory

That is the right thing to do.


> *Problems I have so far:*
>
> When it gets to the end of the distr inst.d scripts (I only have 3 so
> far), the VM shutdowns, with this output.
> I think that tinycv.cc is not working, as features2d.hpp looks like
> maybe a 2.4 feature of opencv?
 >
> FIXME: perl fallback at ../os-autoinst/ppm.pm line 217.
> Perl exited with active threads:
>          1 running and unjoined
>          0 finished and unjoined
>          2 running and detached

It errors out because it wants to use C code via the Inline::C perl
module (perl-Inline rpm in openSUSE, but you can install it via cpan, as
fallback), which needs gcc, make and maybe perl-devel.

Great, thank you:  In my case, urpmi perl-Inline fixed that problem.  waitstillimage now works. :)
 

> ../os-autoinst/tools/ppmtompg qemuscreenshot video/Mageia-3-beta4-x86_64-DVD
> Input #0, image2, from 'ppmtompg.tmp/%08d.ppm':
>    Duration: 00:00:01.40, start: 0.000000, bitrate: N/A
>      Stream #0:0: Video: ppm, rgb24, 800x600, 25 fps, 25 tbr, 25 tbn, 25 tbc
>
>    0:00:01.50 audio: 0kbps video: 573kbps, time elapsed: 00:00:01
>    0:00:01.40 audio: 0kbps video: 573kbps, time elapsed: 00:00:01
> qemu: terminating on signal 15 from pid 32234
> Building PPM C-libraries...
> swig -perl5 -c++ tinycv.i
> c++ -O3 -c -fPIC tinycv_wrap.cxx `perl -MExtUtils::Embed -e ccopts`
> c++ -O3 -fPIC `pkg-config --cflags opencv` -o tinycv.o -c tinycv.cc
> tinycv.cc:10:42: fatal error: opencv2/nonfree/features2d.hpp: No such
> file or directory
> compilation terminated.
> make: *** [tinycv.o] Error 1

I checked and we use opencv-2.4.1 in one setup, but the other one runs
completely without opencv atm (which might change soon with updates from
the openSUSE-Team).

Is the any flags or variables I can set to tell tinycv.cc to not try to use opencv?
 

>
> *Also:
> On startup, I get an error about audio.  See below:*
...
> starting: /usr/bin/qemu-kvm -m 1024 -net user -monitor
> tcp:127.0.0.1:15222,server,nowait -net
> nic,model=virtio,macaddr=52:54:00:12:34:56 -serial file:serial0 -soundhw
> ac97 -vga cirrus -S -drive file=raid/l1,if=virtio -drive
> file=raid/l2,if=virtio -boot dc -cdrom
> /home/test/tmp/Mageia-3-beta4-x86_64-DVD.iso -vnc :99 -usb -usbdevice
> tablet -smp 1
> audio: Failed to create voice `ac97.pi'
> audio: Failed to create voice `ac97.mc'
> audio: Failed to create voice `ac97.pi'
> audio: Failed to create voice `ac97.mc'

I also get these regularly and they dont seem to matter. Audio tests work.

I was wondering if they might be contributing to high CPU load or something.  VM's I boot using virt-manager, use
 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0

Changing this in: inst/startqemu.pm, changed the error about audio, but did not get rid of it.

> I am also a little puzzled about the waitidle, as the VM seems to use
> quite a lot of CPU, even when it should be doing nothing except waiting
> for user input.
>
> I seem to get messages like:
>  >>> waitidle: timed out after 19
>
> Not sure what that means?

The VM can be busy-waiting in the bootloader (because there is no OS to
schedule time-slices or such), but later when it booted Linux, and you
watch "top", the CPU load of your kvm process should be well below 50%

or are you running it with CPU-emulation??

I think I am using full hardware emulation. My hardware supports it, and the process that is started is qemu-kvm.
It is not started however, with anything like the same options as qemu-kvm is started by virt-manager.

I think something is not right, with the way it's running KVM.  Here is top. 

Below, is the output from top. Both guests are booted to the same place in the installer.  The one running as root, is started uisng
virt-manager, and is sitting at 4% cpu.  The one started by os-autoinst is using 56%.  I have a core i7, with 32GB of RAM and SSD.

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 13513 test    20   0 1344m 356m 6344 S   56  1.1   0:26.02 qemu-kvm
 9850 root     20   0 4441m 504m 6820 S    4  1.6   0:45.76 qemu-kvm

here are the processes:

root      9850     1  7 22:57 ?        00:00:46 /usr/bin/qemu-kvm -S -M pc-1.0 -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -name mageia3beta4t2 -uuid a4eafb7b-cfcd-3841-e31d-118bc66ff79b -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/mageia3beta4t2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/dev/ssd/mageia3beta4t2,if=none,id=drive-virtio-disk0,format=raw -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -drive file=/home/nelg/mageia3/Mageia-3-beta4-x86_64-DVD/Mageia-3-beta4-x86_64-DVD.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:de:e6:86,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga std -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

root      9851     2  0 22:57 ?        00:00:00 [kvm-pit-wq]

test     13513 13455 47 23:06 pts/2    00:00:33 /usr/bin/qemu-kvm -m 1024 -net user -monitor tcp:127.0.0.1:15222,server,nowait -net nic,model=virtio,macaddr=52:54:00:12:34:56 -serial file:serial0 -soundhw ac97 -vga cirrus -S -drive file=raid/l1,if=virtio -drive file=raid/l2,if=virtio -boot dc -cdrom /home/test/tmp/Mageia-3-beta4-x86_64-DVD.iso -vnc :99 -usb -usbdevice tablet -smp 1


I've tried add -enable-kvm to the os-autoinst version, but that does not seem to help with CPU usage.

Thank you all for the quick replies so far.

Regards
Glen Ogilvie

[Bernhard M. Wiedemann]

Did you notice that the fast one is running as root?
Are you sure, that your "test" user has access to /dev/kvm ?
I guess, qemu-kvm could fallback to full CPU emulation otherwise.
# chown test /dev/kvm

> I've tried add -enable-kvmto the os-autoinst version, but that does not

> seem to help with CPU usage.

Adding -enable-kvm in inst/startqemu.pm:84 would have been my first
guess, too.
Did you try running "top" within the VM, to see if it runs some
processing in it?
Otherwise, you could try bisecting - adding kvm options on one side or
dropping options on the other side until the behavior switches.

Ciao
Bernhard M.

[Glen Ogilvie]

On Wednesday, April 3, 2013 11:16:56 PM UTC+13, Glen Ogilvie wrote:

The VM can be busy-waiting in the bootloader (because there is no OS to
schedule time-slices or such), but later when it booted Linux, and you
watch "top", the CPU load of your kvm process should be well below 50%
or are you running it with CPU-emulation??

I think I am using full hardware emulation. My hardware supports it, and the process that is started is qemu-kvm.
It is not started however, with anything like the same options as qemu-kvm is started by virt-manager.

I think something is not right, with the way it's running KVM.  Here is top. 

Below, is the output from top. Both guests are booted to the same place in the installer.  The one running as root, is started uisng
virt-manager, and is sitting at 4% cpu.  The one started by os-autoinst is using 56%.  I have a core i7, with 32GB of RAM and SSD

I've tried add -enable-kvm to the os-autoinst version, but that does not seem to help with CPU usage.

Answering my own post, relating to CPU usage.  I've discovered that if I don't have vncviewer connected, CPU load drops to a reasonable level (about 5%).

So, using (tigervnc)
 vncviewer -PreferredEncoding=raw localhost:99

dropped my CPU load to about 11-12% when vnc is connected, and 4-5% when disconnected