Debian and QEMU virtio-net-device

[Jerry Stuckle]

Hi, all,

I'm trying to get Debian armhf (jessie) running under qemu-system-arm.
It's working OK except for one point.

If I don't specify any nic, QEMU supplies a default which is accepted by
Debian. However, when I specify a virtio-net-device (so I can bridge to
the host nic), I get no nic.

The nic is defined in the QEMU startup with:

-netdev bridge,br=bridge0,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper \
-device virtio-net-device,netdev=net0,mac=DE:AD:BE:EF:37:30 \

and the QEMU monitor shows it as:

virtio-net-device.0:
index=0,type=nic,model=virtio-net-device,macaddr=de:ad:be:ef:37:30
\ net0: index=0,type=tap,helper=/usr/lib/qemu/qemu-bridge-helper,br=bridge0

(bridge0 is the name of the bridge on the host system, and is working
correctly).

I've added the following to /etc/modules:

virtio
virtio_ring
virtio_blk
virtio_net

All modules are loaded and show up in lsmod.

The only odd thing I see in the syslog at startup are lines indicating
eth0 is not found.

Am I missing something to get this to work in Debian? Like another
driver maybe?

I appreciate any help.

Jerry

[Ian Campbell]

On Mon, 2016-11-07 at 22:57 -0500, Jerry Stuckle wrote:
> The only odd thing I see in the syslog at startup are lines indicating
> eth0 is not found.

Wild stab in the dark: Perhaps things have remembered the mac address
of the original (automatically added) device as eth0 and so the virtio
device has been renamed out of the way, meaning that
/etc/network/interfaces's references to eth0 don't work?

Does "ifconfig -a" (as root) show the virtio device with some name
other than eth0? If so then you might need to edit
/etc/udev/rules.d/70-persistent-net.rules to cause it to forget the old
device.

Ian.

[Mark Morgan Lloyd]

Is that file still valid on Jessie?

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]

[Ian Campbell]

On Tue, 2016-11-08 at 08:42 +0000, Mark Morgan Lloyd wrote:
> > Does "ifconfig -a" (as root) show the virtio device with some name
> > other than eth0? If so then you might need to edit
> > /etc/udev/rules.d/70-persistent-net.rules to cause it to forget the
> > old
> > device.
>
> Is that file still valid on Jessie?

I'm not entirely sure to be honest, but I have it on a system installed
from Stretch around Easter this year, so maybe?

https://lists.debian.org/debian-user/2015/08/msg01083.html seems to
suggest that other factors may be at play these days though.

Ian.

[Mark Morgan Lloyd]

Yes, the device naming scheme changes on Stretch, AIUI following other
distreaux. Suffixes indicating aliases and VLANs still appear to work as
expected.

By and large it's an improvement, with interfaces being fixed according
to the external connector on the computer (and with this applying to
both Ethernet and USB connectors). Note there that I've not explored the
situation when an external USB hub is interposed, this could get messy
particularly if a hub is replaced with another which has a different
number of ports.

I suspect that Jessie is somewhere in between, and I don't believe I've
seen a 70-persistent-net.rules on any system here irrespective of
whether it's got a static population of interfaces or is being used
heavily testing hotplugged Ethernet and tethered 'phones.

[Jerry Stuckle]

Hi, Ian, and thanks for the response.

No, the mac address of the original eth0 device was different (and
generated by QEMU). There was also no difference if I didn't specify a
MAC address for the virtio-net-device.

ifconfig -a shows only the lo device.

Any other ideas?

Thanks again,
Jerry

[Ian Campbell]

On Tue, 2016-11-08 at 08:51 -0500, Jerry Stuckle wrote:
> Any other ideas?

I'm at a bit of a loss, maybe someone else has some bright ideas.

A few bits of info whic might jolt someones memory:

What is your full qemu command line for the working case (with the
default nic) and non-working cases?

Might be worth also posting the full dmesg output for both cases.

Do you have any other virtio devices (e.g. block) working ok?

Ian.

[Mark Morgan Lloyd]

On 08/11/16 14:30, Ian Campbell wrote:
> On Tue, 2016-11-08 at 08:51 -0500, Jerry Stuckle wrote:
>> Any other ideas?
>
> I'm at a bit of a loss, maybe someone else has some bright ideas.
>
> A few bits of info whic might jolt someones memory:

I can't usefully help, since while I use Qemu for various targets I tend
to use tun etc. networking for historical reasons. I'm sure what I'm
doing is thoroughly obsolete, not least because it needs root privilege,
but there might possibly be something useful at
http://wiki.freepascal.org/Qemu_and_other_emulators

Of course the one thing that really does make a big difference is making
absolutely sure that Qemu error messages are visible, i.e. at least
initially run it from a standard shell. Some of the Qemu console status
commands might also turn out to be useful.

I suppose this leads on to a question of my own...

[Lennart Sorensen]

On Mon, Nov 07, 2016 at 10:57:29PM -0500, Jerry Stuckle wrote:
> Hi, all,
>
> I'm trying to get Debian armhf (jessie) running under qemu-system-arm.
> It's working OK except for one point.
>
> If I don't specify any nic, QEMU supplies a default which is accepted by
> Debian. However, when I specify a virtio-net-device (so I can bridge to
> the host nic), I get no nic.
>
> The nic is defined in the QEMU startup with:
>
> -netdev bridge,br=bridge0,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper \
> -device virtio-net-device,netdev=net0,mac=DE:AD:BE:EF:37:30 \

Without the rest of the command line it is hard to know.

This works for me:

qemu-system-arm -machine vexpress-a15 \
-device virtio-net-device,netdev=net0 -netdev type=user,id=net0 \
-dtb vexpress-v2p-ca15-tc1.dtb -kernel vmlinuz -initrd initrd.gz \
-drive file=debian-8.6.0-armhf-netinst.iso,if=none,id=cd \
-device virtio-scsi-device -device scsi-cd,drive=cd \
-append "console=ttyAMA0" -nographic

Of course adding a hard disk would be useful.

So what system were you emulating?

--
Len Sorensen

[JF Straeten]

Hi,

On Tue, Nov 08, 2016 at 09:51:37AM -0500, Lennart Sorensen wrote:

[...]

> > If I don't specify any nic, QEMU supplies a default which is
> > accepted by Debian. However, when I specify a virtio-net-device
> > (so I can bridge to the host nic), I get no nic.

[...]
> > -netdev
> > -bridge,br=bridge0,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper
> > -\ device virtio-net-device,netdev=net0,mac=DE:AD:BE:EF:37:30 \

Could you try with another macadress ?

Here (on amd64, though), sometimes qemu doesn't like some macadress,
dunno why...

The tool 'macchanger' can create a valid one.

Hih,


--

JFS.

[Lennart Sorensen]

I tried with that mac address and I did not see a problem with that.

--
Len Sorensen

[Jeffrey Walton]

Another possibility for troubleshooting is to boot with
'biosdevname=0' to avoid the stable name that munges things up on
occasion. Also see "Consistent Network Device Naming in Linux"
(http://linux.dell.com/files/whitepapers/consistent_network_device_naming_in_linux.pdf).

I've found biosdevname=0 is hit or miss. It worked for a while, but
then later 4.x kernels stopped honoring it.

Jeff

[JF Straeten]

Re,

On Tue, Nov 08, 2016 at 02:27:30PM -0500, Lennart Sorensen wrote:

[...]

> > Could you try with another macadress ?
> >
> > Here (on amd64, though), sometimes qemu doesn't like some macadress,
> > dunno why...
> >
> > The tool 'macchanger' can create a valid one.
>
> I tried with that mac address and I did not see a problem with that.

Sorry, it was an advice to the OP ;)

With some macaddress, here, the nic doesn't come up, but no problème
with another macadress...

A+

--

JFS.

[Jerry Stuckle]

On 11/8/2016 9:51 AM, Mark Morgan Lloyd wrote:
> On 08/11/16 14:30, Ian Campbell wrote:
>> On Tue, 2016-11-08 at 08:51 -0500, Jerry Stuckle wrote:
>>> Any other ideas?
>>
>> I'm at a bit of a loss, maybe someone else has some bright ideas.
>>
>> A few bits of info whic might jolt someones memory:
>
> I can't usefully help, since while I use Qemu for various targets I tend
> to use tun etc. networking for historical reasons. I'm sure what I'm
> doing is thoroughly obsolete, not least because it needs root privilege,
> but there might possibly be something useful at
> http://wiki.freepascal.org/Qemu_and_other_emulators
>
> Of course the one thing that really does make a big difference is making
> absolutely sure that Qemu error messages are visible, i.e. at least
> initially run it from a standard shell. Some of the Qemu console status
> commands might also turn out to be useful.
>
> I suppose this leads on to a question of my own...
>

Mark,

Thanks for the input. There are no error messages from QEMU, and the
console status looks good. I started out with this on the QEMU mailing
list, but after lots of looking, people basically threw up their hands.
At this point it doesn't *look* like a QEMU problem - but what do I
know? I'm just a lowly programmer. If I knew the problem, I'd fix it :)

Jerry

[Mark Morgan Lloyd]

On 08/11/16 21:00, Jerry Stuckle wrote:

> Thanks for the input. There are no error messages from QEMU, and the
> console status looks good. I started out with this on the QEMU mailing
> list, but after lots of looking, people basically threw up their hands.
> At this point it doesn't *look* like a QEMU problem - but what do I
> know? I'm just a lowly programmer. If I knew the problem, I'd fix it :)

I'm about to hit the sack or something, but just remember that when a
programmer has a hard time finding a bug it's almost always because he's
looking in the wrong place.

[Jerry Stuckle]

On 11/8/2016 4:25 PM, Mark Morgan Lloyd wrote:
> On 08/11/16 21:00, Jerry Stuckle wrote:
>
>> Thanks for the input. There are no error messages from QEMU, and the
>> console status looks good. I started out with this on the QEMU mailing
>> list, but after lots of looking, people basically threw up their hands.
>> At this point it doesn't *look* like a QEMU problem - but what do I
>> know? I'm just a lowly programmer. If I knew the problem, I'd fix it :)
>
> I'm about to hit the sack or something, but just remember that when a
> programmer has a hard time finding a bug it's almost always because he's
> looking in the wrong place.
>

Hi, Mark,

No argument there. I wish I knew where to look on this one!

Jerry

[Jerry Stuckle]

Hi, Ian,

Sorry for the delay and not posting to the listserv - my mistake.

The entire qemu command line is:

qemu-system-arm -m 1024M \
-sd /export/armhf.qcow2 \
-M vexpress-a9 \
-cpu cortex-a9 \
-kernel /export/boot/vmlinuz \
-initrd /export/boot/initrd.img \
-append "root=/dev/mmcblk0p2" \
-monitor stdio \

-netdev bridge,br=bridge0,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper \
-device virtio-net-device,netdev=net0,mac=DE:AD:BE:EF:37:30

The only difference between this and the working one is the removal of
the last two lines. This creates a default SLIRP device which works
(but has limitations).

No other virtual devices, as you can see, so no way to check if other
virtual devices are working.

I've attached dmesg.good and dmesg.bad, which are the outputs for both
working (non-virtio) and failing (virtio) network devices.

Again, I appreciate any insight you can provide.

Jerry

[Lennart Sorensen]

On Wed, Nov 09, 2016 at 08:18:06AM -0500, Jerry Stuckle wrote:
> Hi, Ian,
>
> Sorry for the delay and not posting to the listserv - my mistake.
>
> The entire qemu command line is:
>
> qemu-system-arm -m 1024M \
> -sd /export/armhf.qcow2 \
> -M vexpress-a9 \
> -cpu cortex-a9 \
> -kernel /export/boot/vmlinuz \
> -initrd /export/boot/initrd.img \
> -append "root=/dev/mmcblk0p2" \
> -monitor stdio \
> -netdev bridge,br=bridge0,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper \
> -device virtio-net-device,netdev=net0,mac=DE:AD:BE:EF:37:30

Could you try adding -dtb with the right dtb for that machine?
ARM kernels really want a DTB to control all the busses and devices
in general.

Did you append the dtb to the kernel instead or something? I can't even
get qemu to boot without the -dtb. Of course I also get no output unless
I pass a console argument to the kernel.

I don't have a bridge setup, but using -netdev user instead of -netdev
bridge, I get working networking.

With working virtio net you will see NOTHING in dmesg. It doesn't
print anything. 'ip link' will show eth0 however, and if your bridge
setup is done right, it will work. Trying user mode first is a lot
simpler and an easier way to check if the virtio is working first before
worrying about the bridge setup.

The smx device is a different one, and not the one used by virtio.

--
Len Sorensen

[Ian Campbell]

On Wed, 2016-11-09 at 08:18 -0500, Jerry Stuckle wrote:
> Again, I appreciate any insight you can provide.

Looks like you are using the armel vexpress kernel, which AFAICT
includes PCI based virtio support, unlike most other ARM configurations
which include the MMIO (discovered via DTB or ACPI) variant. (Other
platforms like x86 use the PCI based stuff though, so I'd expect it to
work)

I think that might mean you need virtio-net-pci instead of virtio-net-
device in your command line, but I'm not 100% sure. (The x86 example on
http://www.linux-kvm.org/page/Virtio seems to suggest this too).

One other approach would be to ditch the vexpress kernel and initrd and
instead use the armhf/armmp kernel and "-M virt" instead of "-M
vexpress" and then virtio-net-device would be the correct thing to use.

In fact unless you have a good concrete reason to use armel and
vexpress then that (armhf/armp) would be what I would most recommend.
You could still run an armel userspace if you have that requirement,
but if not then I'd recommend using armhf userspace too.

BTW your use of vexpress also answers Lennart's question wrt device
tree, since that platform is not DTB based.

Lastly please note https://lists.debian.org/<1478646546.1727.22.camel@d
ecadent.org.uk> i.e. there will likely be no armel/vexpress kernel in
Stretch (so if you have a hard requirement for it then now would be a
good time to speak, ideally on that other thread).

Ian.

[Mark Morgan Lloyd]

On 11/11/16 08:00, Ian Campbell wrote:
> On Wed, 2016-11-09 at 08:18 -0500, Jerry Stuckle wrote:
>> Again, I appreciate any insight you can provide.
>
> Looks like you are using the armel vexpress kernel, which AFAICT
> includes PCI based virtio support, unlike most other ARM configurations
> which include the MMIO (discovered via DTB or ACPI) variant. (Other
> platforms like x86 use the PCI based stuff though, so I'd expect it to
> work)

Is there an equivalent to lspci for this type of device?

[Ian Campbell]

On Fri, 2016-11-11 at 14:38 +0000, Mark Morgan Lloyd wrote:
> On 11/11/16 08:00, Ian Campbell wrote:
> >
> > On Wed, 2016-11-09 at 08:18 -0500, Jerry Stuckle wrote:
> > >
> > > Again, I appreciate any insight you can provide.
> >
> > Looks like you are using the armel vexpress kernel, which AFAICT
> > includes PCI based virtio support, unlike most other ARM
> > configurations
> > which include the MMIO (discovered via DTB or ACPI) variant. (Other
> > platforms like x86 use the PCI based stuff though, so I'd expect it
> > to
> > work)
>
> Is there an equivalent to lspci for this type of device?

Not really AFAIK.

IIRC there is a DTB node for each device, so you ought to be able to
dig around in /sys/firmware/devicetree or the legacy /proc equivalent
(which I've forgotten the name of) and find the nodes. The dtc compiler
can also reconstruct a dts source file from that sysfs tree or from a
dtb which can be grepped.

But even then I don't beleive the device tree identifies the actual
device type, just its existence and location in MMIO, since the device
type is then queried from the device itself and therefore isn't in the
firmware description.

I guess there is also a /sys/bus/virtio full of stuff.

Ian.

[Jerry Stuckle]

Hi, Ian, and thanks again for your time.

No, this is an armhf kernel, installed from
debian-7.1.0-armhf-netinst.iso (and updated). It does not include PCI
support (neither does hardware) and trying to use virtio-net-pci returns
a message about PCI not found (which it isn't).

I could try something else, but this is the kernel being used for the
development hardware and I'd like to stick with the same kernel. It's
been good because I've been able to compile and test a lot of code on my
laptop, then port the binaries to the device. But now I need to do some
testing that requires more network support than is available in the
limited QEMU default emulator.

I didn't realize the vexpress kernel wasn't dtb based - but then I'm not
great on Linux administration, either. I thought I had to have it when
I installed they kernel, but that was over three years ago on another
project, and I don't recall everything I did from that time.

Jerry

[Ian Campbell]

On Fri, 2016-11-11 at 11:55 -0500, Jerry Stuckle wrote:
> Hi, Ian, and thanks again for your time.
>
> No, this is an armhf kernel, installed from
> debian-7.1.0-armhf-netinst.iso (and updated).  It does not include PCI
> support (neither does hardware) and trying to use virtio-net-pci returns
> a message about PCI not found (which it isn't).

I was confused because in Jessie vexpress was merged into the armmp
flavour on armhf and I mistakenly remembered thattra 3.2 was the Jessie
kernel rather than the Wheezy one (which it is).

The virtio options in Wheezy's armhf/vexpress kernel flavour all
involve PCI, not the DTB/mmio based stuff[0], and as you say there is
no PCI on that platform so I guess they are probably useless, sorry.

You could try the Jessie kernel with Wheezy userspace I guess.

Ian.

[0]
wheezy.git$ grep VIRTIO debian//build/config.armhf_none_vexpress 
CONFIG_VIRTIO_BLK=m
CONFIG_VIRTIO_CONSOLE=m
CONFIG_HW_RANDOM_VIRTIO=m
CONFIG_VIRTIO_NET=m
CONFIG_SCSI_VIRTIO=m
CONFIG_VIRTIO_PCI=m
CONFIG_VIRTIO_BALLOON=m
CONFIG_NET_9P_VIRTIO=m

[Jerry Stuckle]

Ian,

Thanks much for the info.

I find it interesting that the virtio options for vexpress would require
PCI, since the vexpress architecture doesn't support PCI.
Unfortunately, I guess that means we can't use Debian, as I really need
to get the virtualization working. I'll be traveling over the holidays,
but can't just let the project drop, and can't easily take the prototype
system with me, and trying to mix Wheezy and Jessie just isn't practical.

So I'll have to look at other distros to see what does support it. It
will be easier to do that now than at some later time.

Thanks again for your help, Ian - I really do appreciate all of your
time. At least I now know it isn't something I did.

Jerry

[Ian Campbell]

On Fri, 2016-11-11 at 20:05 -0500, Jerry Stuckle wrote:
> Ian,
>
> Thanks much for the info.
>
> I find it interesting that the virtio options for vexpress would require
> PCI, since the vexpress architecture doesn't support PCI.

IIRC there was _no_ option to support virtio on any platform without
PCI in the Wheezy era kernels, the option likely ended up set on
vexpress by mistake (i.e. as a consequence of enabling it on other
flavours).

> Unfortunately, I guess that means we can't use Debian, as I really need
> to get the virtualization working.

You should be able to use the Jessie armhf/armmp kernel. It's just
Wheezy era where it isn't possible.

You could also use a non-virtio nic and configure it to meet you needs,
take a looked at the output of "-device ?", it probably contains the
very same nic which is enabled by default and there should be no reason
you can't manually configure that to use the different network config
you want.

Ian.

[Jerry Stuckle]

Ian,

Thanks for clarifying this - I misunderstood your previous statement and
thought it was the Wheezy kernel which supported virtio and not Jessie.

However, I am already running Jessie. The only non-PCI nic QEMU
supports for a bridge is the virtio_device. The one used for SLIRP is
not an option. Not that it would help anyway - it's a very limited
emulation, and the reason I need to get a bridge running.

Jerry

[Ian Campbell]

On Sat, 2016-11-12 at 10:22 -0500, Jerry Stuckle wrote:
> Ian,
>

> Thanks for clarifying this - I misunderstood your previous statement
> and
> thought it was the Wheezy kernel which supported virtio and not
> Jessie.
>
> However, I am already running Jessie.

The logs you posted earlier contain:

[    0.000000] Linux version 3.2.0-4-vexpress (debian...@lists.debian.org) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 SMP Debian 3.2.73-2+deb7u3

which is from the Wheezy kernel (vexpress flavour). You can see in http
s://tracker.debian.org/pkg/linux that 3.2.x is shown as oldstable
(Wheezy) while the Jessie (stable) kernel is 3.16 based.

You can also see in
https://packages.debian.org/search?keywords=linux-image-vexpress
that the vexpress flavour went away after Wheezy, it was subsumed into
https://packages.debian.org/search?keywords=linux-image-armmp in

Jessie.

>   The only non-PCI nic QEMU supports for a bridge is the virtio_device.

You seem to have inferred some linkage between the device model (the
virtual/emulated device, configured with -device) and the networking
backend (the virtual network infrastructure, e.g. bridge etc,
configured with -netdev) where I do not believe such a linkage exists.

Grabbing netboot/vmlinuz, netboot/initrd.gz and device-tree/vexpress-
v2p-ca9.dtb from http://ftp.uk.debian.org/debian/dists/jessie/main/inst
aller-armhf/current/images/ lets me boot and detect networking with:

qemu-system-arm -M vexpress-a9 -cpu cortex-a9 -kernel vmlinuz -initrd initrd.gz -append console=ttyAMA0,115200 -dtb vexpress-v2p-ca9.dtb -net nic,model=lan9118,netdev=net0 -netdev user,id=net0

or I can switch to a tap device with (need nographic due to sudo):

sudo qemu-system-arm -M vexpress-a9 -cpu cortex-a9 -kernel vmlinuz -initrd initrd.gz -append console=ttyAMA0,115200 -dtb vexpress-v2p-ca9.dtb -net nic,model=lan9118,netdev=net0 -netdev tap,id=net0 -nographic

I don't have a bridge setup on this machine so I can't try that
precisely but I hope that has demonstrated that the emulation and the
netdev are independent.

Note that I'm passing -dtb here because it is a Jessie kernel, with the
Wheezy kernel this would not be necessary.

>   The one used for SLIRP is
> not an option.  Not that it would help anyway - it's a very limited
> emulation, and the reason I need to get a bridge running.

As I say, the emulation and the user of bridge vs slirp are
independent. Although it is a bit moot since if you switch the Jessie
kernel virtio net should work fine anyway and will most likely be
faster.

Ian.

[Jerry Stuckle]

Ian,

That's interesting, because when I do lsb_release-a returns

No LSB modules are available.
Distribution ID: Debian
Description: Debian GNU/Linux 8.6 (jessie)
Release: 8.6
Codename: Jessie

/etc/apt/sources.list entries all point at stable, which should be
jessie, I think. And the system is up to date.

Right now I have no idea what's causing the difference in versions. Any
ideas? This could be part of the cause of my problems.

Jerry

[Ian Campbell]

On Sat, 2016-11-12 at 16:16 -0500, Jerry Stuckle wrote:
> Ian,
>

> That's interesting, because when I do lsb_release-a returns
>
> No LSB modules are available.
> Distribution ID: Debian
> Description:     Debian GNU/Linux 8.6 (jessie)
> Release:         8.6
> Codename:        Jessie
>
> /etc/apt/sources.list entries all point at stable, which should be
> jessie, I think.  And the system is up to date.

This is all userspace stuff, which comes from within the VM file
system, but may not correspond to the kernel you are booting.

Your earlier qemu command line seems to indicate that you are picking
up a kernel from the _host_ filesystem. It seems very likely that you
have not updated the kernel and initrd on the host or you are somehow
picking up the older kernel.

Maybe you have tripped over the switch from -vexpress to -armmp kernel
flavour naming -- check that you have the linux-image-armmp installed
and not just the (now out of date) linux-image-vexpress one. Those are
both meta packages which depend on the specific versioned packages.
What does "dpkg -l linux-image-\*" say?

How are you getting the VM filesystem's /boot into your host's
/export/boot where you are booting it from? That process might need
tweaking to pickup the new kernel?

Ian.

[Jerry Stuckle]

Hi, Ian,

Sorry for the delay - I'm really confused now.

Yes, the vmlinuz and initrd are loaded off the local machine - but these
are the same copies that are in the /boot directory on the Debian guest.
I would have thought those were updated when I upgraded Debian.

In any case - I looked for an armmp package and found
linux-image-3.16.0-4-armmp. I installed that package, copied the
vmlinuz-3.16.0-4-armmp and initrd.img-3.16.0-4-armmp files to the host
and rest the symlinks.

Boot only gave me an empty window. No messages, nothing. I had to
force close the machine.

According to
https://wiki.debian.org/DebianKernel/ARMMP#fnref-ab595c5be7b24964461cfb9d5a876e5f0de4d301,
it is unknown whether the armmp kernel supports Versatile Express or
not. From my tries, it looks like it does not.

Is this by any chance a 64bit port? If so, that would explain the
problem because I'm using a 32bit chip. Or is there something else I'm
doing wrong here?

And it looks like I have the same problem on the hardware. Not good...

Thanks,
Jerry

[Ian Campbell]

On Sun, 2016-11-13 at 21:39 -0500, Jerry Stuckle wrote:
> Boot only gave me an empty window.  No messages, nothing.  I had to
> force close the machine.

You need the DTB too. /usr/lib/linux-image-3.16.0-4-armmp should
contain vexpress-v2p-ca9.dtb which seems like the one you want (but
there are others, take a look and try to match to your hardware).

Pass the dtb to qemu with -dtb as I showed yesterday.

For h/w firmware which may not support passing a dtb you can also
append it to the vmlinuz.

> According to
> https://wiki.debian.org/DebianKernel/ARMMP#fnref-ab595c5be7b24964461cfb9d5a876e5f0de4d301,
> it is unknown whether the armmp kernel supports Versatile Express or
> not.

Not it isn't, those columns are headed "LPAE" and "Full Debian
Installer Support".

>   From my tries, it looks like it does not.

I tried this yesterday and reported to you 2 qemu command lines which I
had used. It does work.

Ian.

[Jerry Stuckle]

Hi, Ian,

I tried several variations with different DTBs, and nothing worked. I
also tried your example, with no difference. All I get is an empty console.

My initrd.gz is a symlink to initrd.img-3.16.0-4-armmp and vmlinuz is a
symlink to vmlinuz-3.16.0-4-armmp, which I think are the correct files.

Is my problem possibly that I'm trying to upgrade an existing ARM
installation instead of installing new? But even if it were, I would
expect to see *something* to indicate a boot failure.

Totally perplexing!

Jerry

[Ian Campbell]

On Mon, 2016-11-14 at 12:46 -0500, Jerry Stuckle wrote:
> Hi, Ian,
>
> I tried several variations with different DTBs, and nothing worked.  I
> also tried your example, with no difference.  All I get is an empty console.

Please tell us the actual commands which you have tried to run.

Ian.

[Jerry Stuckle]

Ian,

I have tried with your command:

qemu-system-arm -M vexpress-a9 \
-cpu cortex-a9 \

-kernel /export/armmp/vmlinuz \
-initrd /export/armmp/initrd.gz \
-append console=ttyAMA0,115200 \
-dtb /export/armmp/vexpress-v2p-ca9.dtb \
-net nic,model=lan9118,netdev=net0 \
-netdev user,id=net0 \
-monitor stdio

I have also tried without the last three lines, with

-append root=/dev/mmcblk0p2

both with and without the console parameter,

and

-append root=/dev/mmcblk0

(didn't expect this to work - but worth a try).

None of them give me anything but a blank qemu window. The qemu monitor
says the process is running, but no indication that it's doing anything.

I also remounted the qemu disk and verified the initrd, vmlinuz and .dtb
files matched those downloaded, just in case I corrupted something along
the way.

There's got to be something very basic I'm missing here, but I have no
idea what it is.

Jerry

[Lennart Sorensen]

On Mon, Nov 14, 2016 at 02:48:24PM -0500, Jerry Stuckle wrote:
> I have tried with your command:
>
> qemu-system-arm -M vexpress-a9 \
> -cpu cortex-a9 \
> -kernel /export/armmp/vmlinuz \
> -initrd /export/armmp/initrd.gz \
> -append console=ttyAMA0,115200 \
> -dtb /export/armmp/vexpress-v2p-ca9.dtb \
> -net nic,model=lan9118,netdev=net0 \
> -netdev user,id=net0 \
> -monitor stdio
>
> I have also tried without the last three lines, with
>
> -append root=/dev/mmcblk0p2
>
> both with and without the console parameter,
>
> and
>
> -append root=/dev/mmcblk0
>
> (didn't expect this to work - but worth a try).
>
> None of them give me anything but a blank qemu window. The qemu monitor
> says the process is running, but no indication that it's doing anything.
>
> I also remounted the qemu disk and verified the initrd, vmlinuz and .dtb
> files matched those downloaded, just in case I corrupted something along
> the way.
>
> There's got to be something very basic I'm missing here, but I have no
> idea what it is.

Try hitting control+alt+3 to get to the virtual serial port in qemu.
Normally graphics is 1, monitor is 2, and serial port is 3. Works for me.

Could of course be on 2 given the -monitor stdio passed.

--
Len Sorensen

[Jerry Stuckle]

Lennart,

Bingo! The virtual serial port (on 3, btw) showed the system trying to
boot. An error pointed out that in this iteration I had forgotten the
-sd parameter, so no (virtual) hard disk.

I added -sd /export/armhf.qcow2 to the command, but still no hard disk
in the /dev listing. I'm wondering if I have a problem with the dtb;
I'll have to look into that.

But at least I have something to work with now. Hopefully I can find
why the disk isn't being recognized now.

Thanks much!

Jerry

[Jerry Stuckle]

OK, the problem here is I'm not able to mount the disk. Nothing I do
causes it to show up in /dev. I'm using

qemu-system-arm -M vexpress-a9 \
-cpu cortex-a9 \

-m 1024M \
-kernel /export/armmp/vmlinuz \
-initrd /export/armmp/initrd.gz \
-append "root=mmcblk0p1 console=ttyAMA0,115200" \
-dtb /export/armmp/vexpress-v2p-ca9.dtb \
-sd /export/armhf.qcow2 \
-monitor stdio


I've also tried

-drive if=none,file=armhf.qcow2,format=qcow2,id=hd \
-device virtio-blk-device,drive=hd \

As well as

-drive if=none,file=armhf.qcow2,format=raw,id=hd \
-device virtio-blk-device,drive=hd \

-drive if=none,file=armhf.qcow2,driver=qcow2,id=hd \
-device virtio-blk-device,drive=hd \

and

-drive if=sd,format=qcow2,file=/export/armhf.qcow2,id=hd \
-drive if=sd,format=raw,file=/export/armhf.qcow2,id=hd \
-drive if=sd,driver=qcow2,file=/export/armhf.qcow2,id=hd \
-drive if=sd,file=/export/armhf.qcow2,id=hd \

The virtual disk is formatted with the Debian default (p1 /boot, p2
root, p5 /swap). The mount works in wheezy using the -sd parameter above
and shows up as /dev/mmcblk0, but no luck in jessie.

Everything I've found on the internet indicates the -sd parameter should
be all I need, but it doesn't work. Any ideas here?

I really wish I understood more about how Debian and qemu work together!

Thanks,
Jerry

[Ian Campbell]

On Mon, 2016-11-14 at 23:13 -0500, Jerry Stuckle wrote:
>
> Everything I've found on the internet indicates the -sd parameter should
> be all I need, but it doesn't work.  Any ideas here?

The armmp kernel flavour works on many different platforms and is
therefore more modular than the old vexpress one. I expect you just
need to load a driver module.

Looks like the pl180 on vexpress is supported by "mmci.ko". You will
need to make sure that module is available in your initrd then you
should be able to modprobe it (once you have the system booting then
you should be able to automatically regenerate a correct inird with no
manual intervention required).

Ian.

[Jerry Stuckle]

Ian/Lennart,

Just a quick update - been busy and haven't had much of a chance to work
on it. But I wanted to update you before I hit the hay - you've been so
much help.

I tried adding mmci to the modules list and rebuilding initrd. The
result was a kernel panic. No command line to check status. After a
lot of fooling around, I finally decided to start from scratch.

I did an apt-get purge linux-image-3.16.0-4-armmp and deleted the
related files from /boot. I removed mmci from the
/etc/initramfs-tools/modules file and reinstalled the above package.

The result was the same as before - no disk. So, to set a checkpoint, I
saved the files in /boot and did update-initramfs -u -k 3.16.0-4-armmp.
Copied the files to the host - and voila - when I booted, I had a disk.
I had to add "root=mmcblk0p2" to the -append line, and the system booted
with the correct OS.

After adding the virtio network lines to the startup script, I still
don't have virtual networking working (looks like it's still a SLIRP
connection). But at least I'm making progress. And it's past my
bedtime. Tomorrow comes early. Hopefully I can work on the network
tomorrow, but so far it looks like a pretty busy day.

Thanks again. Not much of an update, but I didn't want to keep you
hanging, either.

Jerry