[9fans] lguest on 2.6.25
Alex Lee
I'd like to set up a cpu/auth/file server for my home network. I
already have a laptop server running Ubuntu 8.04, so the idea is to
also get Plan 9 running on it, via lguest. It's mostly working, but
I'm encountering a few problems.
Here's what I've done so far (maybe it's excessive list every single
step, but just in case...):
1. Grab relevant files.
- Download 9lguestcpu.2.6.25.elf and RUNLGUEST from Ron's contrib directory
- Get thx9.img out http://9grid.net/rminnich/src/lguest/thnx9.bz2
- Download the 2.6.25 kernel: linux-2.6.25.tar.bz2 (dated 2008-04-17;
I tried the latest version, 2.6.25.16, but encountered the same
problems that were reported in http://9fans.net/archive/2008/07/690)
2. Compile and install kernel
- I start with the stock 2.6.25 kernel. I then copy in my current
config (Ubuntu 2.6.24-19-generic), run 'make oldconfig', then answer
the config prompts with mostly the defaults.
- Then I configure the LGUEST, LGUEST_LGUEST, VIRTIO, etc. config
entries as per the lguest.txt documentation.
- Then I compile (here the debian way, with 'make-kpkg kernel_image
kernel_headers').
- I also run 'make' in Documentation/lguest to get my 'lguest' binary.
3. Prepare host machine
- Install the new kernel on the host machine and reboot.
- modprobe lg syscall_vector=64
- iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
- echo 1 > /proc/sys/net/ipv4/ip_forward
4. Run Plan 9 in lguest
- I modified RUNLGUEST so that the command is:
./lguest 256 9lguestcpu.2.6.25.elf --block=thx9.img
--tunnet=192.168.19.1
'venti=#S/sd00/arenas;bootdisk=local!#S/sd00/fossil;bootargs=local!#S/sd00/fossil'
- sudo ./RUNLGUEST
Eventually it prompts for the filesystem, and I pick the default. Here
is the output from that point onwards:
root is from (tcp, il, local)[local!#S/sd00/fossil]:
disk is #S/sd00/fossil
AUTHENTICATE ...
time...
GET VENTI ENV! #S/sd00/arenas
IN THERE AND nf is 1
F 0 is #S/sd00/arenas
venti...configloopbackk!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
DID CONFIG LOOPBACK
setenv venti to tcp!127.1!17034
fossil(#S/sd00/fossil)...version...time...
init[0] = init
init[1] = -c
LGATTACH edev c0440b90 ctlr c0440dd0
LGINIT
SETMAC: NO
ETHERIN: devno 3 ring 0 irq 34
ETHEROUT: devno 3 ring 0 irq 35
lguestnetwork is
ndb/dns: can't read my ip address
rc (cpurc): null list in concatenation
init: starting /bin/rc
%
Networking wasn't working at first. So I modified /lib/ndb/local so
that the dns entry pointed to my router (192.168.1.1), and then I ran
the following:
% ip/ipconfig -g 192.168.19.1 ether /net/ether0 192.168.19.2 255.255.255.0
% ndb/dns -r
And then I can resolve names and ping out. So far, so good, but there
are three problems that I'm still trying to figure out:
1. How can I avoid having to manually configure networking each time I boot?
2. I can ping from the host machine to the lguest interface
(192.168.19.2), but when I try to connect with drawterm the connection
is refused.
3. lguest always runs at full CPU. How can I figure out why this is
happening? (I don't feel comfortable running lguest for more than a
few minutes at a time -- the laptop gets pretty hot.)
Any advice or pointers on these issues would be greatly appreciated. Thanks!
Alex
--
Alex Lee <ale...@uchicago.edu>
ron minnich
will try again this week to get this going.
It's harder now as I'm now on a 64-bit machine and 64-bit lguest is not there.
But I'll try to get you an answer.
Sorry
ron
ron minnich
> Here's what I've done so far (maybe it's excessive list every single
> step, but just in case...):
not at all!
>
> 1. Grab relevant files.
>
> - Download 9lguestcpu.2.6.25.elf and RUNLGUEST from Ron's contrib directory
> - Get thx9.img out http://9grid.net/rminnich/src/lguest/thnx9.bz2
> - Download the 2.6.25 kernel: linux-2.6.25.tar.bz2 (dated 2008-04-17;
> I tried the latest version, 2.6.25.16, but encountered the same
> problems that were reported in http://9fans.net/archive/2008/07/690)
The linux kernel is very fluid nowadays, and 2.6.xx.yy is not always
working as 2.6.22.yy++. It's frustrating. So:
>
> 2. Compile and install kernel
>
> - I start with the stock 2.6.25 kernel. I then copy in my current
> config (Ubuntu 2.6.24-19-generic), run 'make oldconfig', then answer
> the config prompts with mostly the defaults.
> - Then I configure the LGUEST, LGUEST_LGUEST, VIRTIO, etc. config
> entries as per the lguest.txt documentation.
> - Then I compile (here the debian way, with 'make-kpkg kernel_image
> kernel_headers').
> - I also run 'make' in Documentation/lguest to get my 'lguest' binary.
>
good.
> 3. Prepare host machine
>
> - Install the new kernel on the host machine and reboot.
> - modprobe lg syscall_vector=64
> - iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
> - echo 1 > /proc/sys/net/ipv4/ip_forward
good.
>
> 4. Run Plan 9 in lguest
>
> - I modified RUNLGUEST so that the command is:
> ./lguest 256 9lguestcpu.2.6.25.elf --block=thx9.img
> --tunnet=192.168.19.1
> 'venti=#S/sd00/arenas;bootdisk=local!#S/sd00/fossil;bootargs=local!#S/sd00/fossil'
> - sudo ./RUNLGUEST
>
> Eventually it prompts for the filesystem,
that's encouraging.
>and I pick the default. Here
> is the output from that point onwards:
>
> root is from (tcp, il, local)[local!#S/sd00/fossil]:
> disk is #S/sd00/fossil
> AUTHENTICATE ...
> time...
> GET VENTI ENV! #S/sd00/arenas
> IN THERE AND nf is 1
> F 0 is #S/sd00/arenas
> venti...configloopbackk!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> DID CONFIG LOOPBACK
> setenv venti to tcp!127.1!17034
> fossil(#S/sd00/fossil)...version...time...
> init[0] = init
> init[1] = -c
> LGATTACH edev c0440b90 ctlr c0440dd0
> LGINIT
> SETMAC: NO
> ETHERIN: devno 3 ring 0 irq 34
> ETHEROUT: devno 3 ring 0 irq 35
> lguestnetwork is
see this line? It's a variable read in /rc/bin/cpurc. You can check
that script and see how it can be set so that networking is a little
better. More I do not recall and the machine is not nearby ...
> ndb/dns: can't read my ip address
> rc (cpurc): null list in concatenation
>
> init: starting /bin/rc
> %
>
> Networking wasn't working at first. So I modified /lib/ndb/local so
> that the dns entry pointed to my router (192.168.1.1), and then I ran
> the following:
>
> % ip/ipconfig -g 192.168.19.1 ether /net/ether0 192.168.19.2 255.255.255.0
> % ndb/dns -r
>
> And then I can resolve names and ping out. So far, so good, but there
> are three problems that I'm still trying to figure out:
ok
>
> 1. How can I avoid having to manually configure networking each time I boot?
Something has to answer a DHCP request from your guest. Can you run
dhcpd on the host?
> 2. I can ping from the host machine to the lguest interface
> (192.168.19.2), but when I try to connect with drawterm the connection
> is refused.
Try restarting the listen? that's an odd one. If you telnet to the
port from the host to the guest what do you see?
> 3. lguest always runs at full CPU. How can I figure out why this is
> happening? (I don't feel comfortable running lguest for more than a
> few minutes at a time -- the laptop gets pretty hot.)
hmm. I have not seen this one. Again, not sure why this is happening.
But the halt
may not be working. Possibly the API has changed -- not sure.
ron
Alex Lee
[snip]
>> lguestnetwork is
>
> see this line? It's a variable read in /rc/bin/cpurc. You can check
> that script and see how it can be set so that networking is a little
> better. More I do not recall and the machine is not nearby ...
>
>> 2. I can ping from the host machine to the lguest interface
>> (192.168.19.2), but when I try to connect with drawterm the connection
>> is refused.
>
> Try restarting the listen? that's an odd one. If you telnet to the
> port from the host to the guest what do you see?
Ah, I see now. I added "lguestnetwork=NAT" to parameters in RUNLGUEST,
and now both networking and drawterm are working great!
>> 3. lguest always runs at full CPU. How can I figure out why this is
>> happening? (I don't feel comfortable running lguest for more than a
>> few minutes at a time -- the laptop gets pretty hot.)
>
> hmm. I have not seen this one. Again, not sure why this is happening.
> But the halt
> may not be working. Possibly the API has changed -- not sure.
>
> ron
Let me know if there's any information that I can provide to help with
this. Many thanks for all the hard work that you've put into doing
this port.
Best,
John Soros
On Sun, 24 Aug 2008 23:58:21 -0500
"Alex Lee" <ale...@uchicago.edu> wrote:
> On Sun, Aug 24, 2008 at 11:32 PM, ron minnich <rmin...@gmail.com> wrote:
> [snip]
> >> lguestnetwork is
> >
> > see this line? It's a variable read in /rc/bin/cpurc. You can check
> > that script and see how it can be set so that networking is a little
Hello,
I have gotten the lguest port working on the 2.6.25.0 kernel, it works mighty fine here.
The load issue for me is only on plan9, on the host I can see with 'top' that the lguest guest isn't consuming all the cpu, so this might ust be a plan9 problem. I have also seen quite a few messages in /sys/log/cron i get a lot of lines saying:
$sysname Aug 25 08:31:14 time went backward
I found routed networking to be the simplest, nat gives me the creeps.
So here is my way to start the lguest guest, I hope it will be helpful:
====snip
#!/bin/bash
set -x
/root/bin/lguest --tunnet=$1 --block=/dev/mirror/plan9cpu \
512 /home/johnny/9lguestcpu.2.6.25.elf 'bootargs=local!#S/sd00/fossil;sysname=9soul;bootdisk=local!#S/sd00/fossil'
sleep 10
ip route add $3 via $1 dev $2
ip route del 192.168.0.0/24 dev $2
echo 1 > /proc/sys/net/ipv4/conf/$2/proxy_arp
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp
====snip
$1 is the host ip, $2 is the tap interface the guest will use, and $3 is the ip the guest will have. This method only works for static networking, also you will need to install the iproute2 utilities. These command also assume the your net is 192.168.0.0 netmask 255.255.255.0, change that to whatever suits you.
Cheers,
John
erik quanstrom
> I have gotten the lguest port working on the 2.6.25.0 kernel, it works mighty fine here.
> The load issue for me is only on plan9, on the host I can see with 'top' that the lguest guest isn't consuming all the cpu, so this might ust be a plan9 problem. I have also seen quite a few messages in /sys/log/cron i get a lot of lines saying:
> $sysname Aug 25 08:31:14 time went backward
both the excessive load and these messages sound like a timesync problem.
- erik
ron minnich
by 'kernel' here I mean linux kernel, sorry.
ron
ron minnich
yeah, that's really weird. One option is to set up the kernels without
any fancy timers -- just use the old PC timers.
ron
ron minnich
> it any event, fixing timesync properly is likely to fix the excessive
> load problem.
good to know. I probably jumped to the wrong conclusion.
ron
erik quanstrom
>>
>
> yeah, that's really weird. One option is to set up the kernels without
> any fancy timers -- just use the old PC timers.
do you think that's the problem?
time jumping backwards doesn't seem like something that could
be traced to some oddity of the lguest environment. (at least i
lack the imagination to know how this could happen.) it sounds
more like the traditional problem of having started two timsyncs.
it any event, fixing timesync properly is likely to fix the excessive
load problem.
- erik
John Soros
Well, i don't think I am starting two timesync, ps a shows only one:
cpu% ps a |grep timesync
bootes 77 0:00 0:00 92K Pread timesync
I also can not write to /dev/rtc, so the timesync part of cpurc fails, this part in particular
=========snip
if(! ps|grep -s timesync) {
aux/timesync -n pool.ntp.org
if (test -e '#r/rtc') @ {
sleep 10 # let timesync correct the time
awk '{print $1}' /dev/time >'#r/rtc' # fix hw clock
} &
}
=============
awk: i/o error occurred while closing /dev/stdout
input record /dev/time:1
source line 1
I have also tried to compile the kernel by doing:
cpu% fcp /n/sources/contrib/rminnich/lguest/lg25.tgz $home/src; cd $home/src; tar xzf lg25.tgz;
cpu% mkdir src/boot
cpu% mkdir src/ip
cpu% mkdir src/pc
cpu% mkdir src/port
cpu% rc bin/rc/lguest
cpu% mk 'conf=lguestcpu'
.....
8c -FTVw lgkbd.c
../port/portdat.h:536 dat.h:130 lgkbd.c:4 syntax error, last name: Tval
mk: 8c -FTVw lgkbd.c : exit status=rc 887: 8c 889: error
Any help would be appreciated, this timesync problem is causing me quite some headaches :-)
Thanks
John
lu...@proxima.alt.za
If Xen is anything to go by, time keeping is a problem in
virtualisation. And Erik is misguiding you :-)
Thing is, if timesync is wresting with the clock as it seems to do if
two instances are running, it does consume a lot of cpu time,
according to experiences reported on this list. But it's possible
that timesync is wrestling with the virtual executive rather than with
another instance of itself.
Would it hurt you to run without timesync?
++L
John Soros
will try, thanks :-)
>
> ++L
>
>
John
John Soros
For one timesync couldn't write to the rtc, so I commented out the lines in cpurc.
Well, here timesync is not the problem, time is reported incorrectly, and I do not really know how to set it. I tried echoing unix time into '#r' and /dev/rtc, with no luck:
$sysname# ls -l /dev/rtc
--rw-rw-r-- r 0 bootes bootes 0 Apr 23 20:43 /dev/rtc
$sysname# echo -n '1219699386' > /dev/rtc
echo: write error: permission denied
(on the hostowners console)
Also cron still spews messages flooding it's log with lines like this:
Aug 26 02:29:41 time went backward
Which might be related to time issues..
rgds
On Mon, 25 Aug 2008 20:54:53 +0200
lu...@proxima.alt.za wrote:
ron minnich
run any at all. The hardware clock is set from Linux and I don't even
allow it to be set. It makes no sense to do that.
So don't let timesync run.
thanks
ron
John Soros
Best++
John
On Mon, 25 Aug 2008 15:44:39 -0700
"ron minnich" <rmin...@gmail.com> wrote:http://www.unixtimestamp.com/
Alex Lee
> Ok, still, it would be great to know how to set the time, as my time is way off (by more than 4 hours).
> Best++
> John
Hi again,
A couple of updates: I'm getting the same cron messages and time
problems that John is seeing. I've made sure that timesync isn't
running. My lguest time is about 3.5 hours ahead of host time.
As for the load issue: The full cpu usage that I was seeing earlier
actually stops after a several minutes.
After booting, ^T^Tp shows genrandom as Running. Several minutes later
the cpu usage drops to near idle, and genrandom goes to Wakeme. Then
every now and then (I'm not sure if anything in particular triggers
it, or if just randomly happens) I see the cpu usage jump again, and I
can see that genrandom is once again Running. Then after several
minutes it stops again.
When genrandom is running, stats shows a load of around 3000.
Otherwise, load is always around 2000.
The problem looks similar to this:
http://9fans.net/archive/2006/03/588 -- except that the lguest
instance works fine while genrandom is running. Is this genrandom
behavior anything out of the ordinary?
ron minnich
> The problem looks similar to this:
> http://9fans.net/archive/2006/03/588 -- except that the lguest
> instance works fine while genrandom is running. Is this genrandom
> behavior anything out of the ordinary?
>
I don't think so. Genrandom make me want to look at using the hardware
RNGs a few years ago, but then Intel killed firmware hub and the whole
idea kind of went away.
ron
ron minnich
> Ok, still, it would be great to know how to set the time, as my time is way off (by more than 4 hours).
What's the time on your host (Linux) look like? Are you sure it's not
a time zone setup issue?
Is it always four hours? It should not be drifting.
ron
John Soros
On Wed, 27 Aug 2008 21:28:42 -0500
"Alex Lee" <ale...@uchicago.edu> wrote:
> On Tue, Aug 26, 2008 at 1:44 AM, John Soros <sor...@gmail.com> wrote:
> > Ok, still, it would be great to know how to set the time, as my time is way off (by more than 4
> > hours). Best++
> > John
>
> Hi again,
>
> A couple of updates: I'm getting the same cron messages and time
> problems that John is seeing. I've made sure that timesync isn't
> running. My lguest time is about 3.5 hours ahead of host time.
>
> As for the load issue: The full cpu usage that I was seeing earlier
> actually stops after a several minutes.
This is what i've been experiencing. This is fine, from the linux host, the guest takes a lot of load in the begginning, but then it calms down, and the load only goes up once in a while. From inside the guest, though, the load looks like it's maxed out all the tim. This might be because of time issues, though.
As for the timezone issues, well, i have my timezon set up properly on the host, and in plan9 I did cp /adm/timezon/CET /adm/timezone/local.
>
> After booting, ^T^Tp shows genrandom as Running. Several minutes later
> the cpu usage drops to near idle, and genrandom goes to Wakeme. Then
> every now and then (I'm not sure if anything in particular triggers
> it, or if just randomly happens) I see the cpu usage jump again, and I
> can see that genrandom is once again Running. Then after several
> minutes it stops again.
>
> When genrandom is running, stats shows a load of around 3000.
> Otherwise, load is always around 2000.
>
> The problem looks similar to this:
> http://9fans.net/archive/2006/03/588 -- except that the lguest
> instance works fine while genrandom is running. Is this genrandom
> behavior anything out of the ordinary?
>
> Alex
>
>
John
Richard Miller
> actually stops after a several minutes.
> Is this genrandom
> behavior anything out of the ordinary?
/dev/random normally runs at 100% cpu after booting, and whenever
its queue of random bytes is getting empty. Since it uses the
clock to produce entropy, this might take longer than usual if
your clock is behaving strangely.
erik quanstrom
> When genrandom is running, stats shows a load of around 3000.
> Otherwise, load is always around 2000.
>
what are the other two processes that are constantly
running?
- erik
erik quanstrom
> I don't think so. Genrandom make me want to look at using the hardware
> RNGs a few years ago, but then Intel killed firmware hub and the whole
> idea kind of went away.
>
via has release the padlock documentation.
unfortunately one needs to enable floating point
and sse to use the padlock rng.
on reading that, i decided that software rng
wasn't as slow or as important as i'd remembered.
there's also the problem of ensuring that the
hardware rng is at least as good as plan 9's.
- erik
ron minnich
you are four hours off.
A useful thing to do is cat /dev/time and see how it changes.
The time from lguest is simple: you read a 64-bit # which is time.
It's just like Xen that way.
Also, try this to test another issue:
date && sleep 60 && date
Two things: should take 60 seconds by the watch and the two dates
should report 60 seconds apart.
Do they?
ron
John Soros
as far as i could mesure it, it took 60 seconds, here it is:
cpu% date && sleep 60 && date
Thu Aug 28 22:19:21 CET 2008
Thu Aug 28 22:20:22 CET 2008
cpu%
My timezones are all set up to be CET (host and plan9 guest), the hardware clock is set to UTC. I am running vanilla 2.6.25 kernel, on a gentoo system.
Thank you for helping debug this problem, is there any more info I can give to help this process?
John
erik quanstrom
time and not utc?
- erik
Alex Lee
I've doublechecked the timezone setting, and it appears to be correct.
The 'date' test seems to work fine:
# date && sleep 60 && date
Thu Aug 28 15:52:19 CDT 2008
Thu Aug 28 15:53:19 CDT 2008
Within a second of one another, I ran:
lguest: cat /dev/time
1219956674 1219956674748050432 1219956674748050432 4294967296
host: date +%s
1219943774
So lguest is 12900 seconds ahead (= 215 minutes, = 3 hrs 35 minutes).
John, are you getting the same?
Best,
John Soros
plan9:
% date
Thu Aug 28 23:40:17 CET 2008
Linux host:
$ date
Thu Aug 28 20:06:02 CEST 2008
even the time difference seems to be the same. strange!
rgds
John