emerging klibc gets wrong kernel sources?
Nikos Chantziaras
downloaded kernel 2.6.23 and the 2.6.24-rc7 patch. However, emerging
gentoo-sources previously got 2.6.24-r4. So now there are two kernel
trees. Shouldn't klibc compile against the gentoo-sources kernel?
And space is showing signs of getting tight already. I only have a
base-system, and yet it eats 3GB (from 10GB total). It looks like
Gentoo will need outrageous amounts of space for a full KDE desktop.
J.O. Aho
> Following the guide to get uvesafb working, I emerged klib. This
> downloaded kernel 2.6.23 and the 2.6.24-rc7 patch. However, emerging
> gentoo-sources previously got 2.6.24-r4. So now there are two kernel
> trees. Shouldn't klibc compile against the gentoo-sources kernel?
No, there are some trouble with klibc that makes it can't be compiled against
a normal kernel source or against the Linux headers, you need a specially
patched kernel source to make this work, thats why you get the 2.6.24-rc7
kernel source (don't use this one to compile a normal kernel).
> And space is showing signs of getting tight already. I only have a
> base-system, and yet it eats 3GB (from 10GB total). It looks like
> Gentoo will need outrageous amounts of space for a full KDE desktop.
Each time you build something, you download the source and gentoo patches, you
should clean out your /usr/portage/distfiles of all unneeded files. A full
install of KDE will take something like 511M when compiled.
--
//Aho
Nikos Chantziaras
>> And space is showing signs of getting tight already. I only have a
>> base-system, and yet it eats 3GB (from 10GB total). It looks like
>> Gentoo will need outrageous amounts of space for a full KDE desktop.
>
> Each time you build something, you download the source and gentoo
> patches, you should clean out your /usr/portage/distfiles of all
> unneeded files. A full install of KDE will take something like 511M when
> compiled.
I used `eclean --destructive distfiles` to clean, but it's not really
deleting anything. Is it safe to simply `rm -rf
/usr/portage/distfiles/*` as well as `rm -rf /var/tmp/portage/*`? Those
two directory trees eat about 1GB right now.
J.O. Aho
It's safe to do so.
--
//Aho
Arthur Hagen
Yes, it's safe. /var/tmp/portage/* will only contain *failed* compiles.
/usr/portage/distfiles is nice to have for when rebuilding something, but if
you don't mind having to redownload packages you recompile, it's safe to zap
that too.
Regards,
--
*Art
Darin McBride
While it's safe, I wouldn't clear out distfiles. /var/tmp/portage, however,
is safe as long as you don't have a current emerge running ;-)
The reason I keep distfiles (and mine is currently sitting at 3.3GB) is
because I find I'm rebuilding packages intermittently. Keeping the source
around speeds things up, even though I have a 10Mb internet connection.
It's also much kinder to the mirrors in that I'm not re-downloading (much)
each time I do a revdep-rebuild.
That and I have all my gentoo boxes mounting from a single location, so it
actually has the source to all builds on all machines. (Too bad eclean
couldn't check ALL machines for needed sources... luckily, the other
machines are almost solely subsets of my primary machine, so it's pretty
close.) This means that I download each source tarball once, and compile
it up to three times. (Yes, I could just reuse the binaries across the two
P4's, though not the P3, and if I had more P4s, I'd probably do so.) More
if I have to revdep-rebuild.
Aragorn
> Nikos Chantziaras wrote:
>
>> J.O. Aho wrote:
>>>> And space is showing signs of getting tight already. I only have a
>>>> base-system, and yet it eats 3GB (from 10GB total). It looks like
>>>> Gentoo will need outrageous amounts of space for a full KDE desktop.
>>>
>>> Each time you build something, you download the source and gentoo
>>> patches, you should clean out your /usr/portage/distfiles of all
>>> unneeded files. A full install of KDE will take something like 511M when
>>> compiled.
>>
>> I used `eclean --destructive distfiles` to clean, but it's not really
>> deleting anything. Is it safe to simply `rm -rf
>> /usr/portage/distfiles/*` as well as `rm -rf /var/tmp/portage/*`? Those
>> two directory trees eat about 1GB right now.
>
> While it's safe, I wouldn't clear out distfiles. /var/tmp/portage,
> however, is safe as long as you don't have a current emerge running ;-)
If you (and/or the OP) have a sufficient amount of RAM in your system, you
could then make */var/tmp/portage* a /tmpfs./ It's what I do, but then
again, that system has 32 GB of RAM in it, so your mileage may vary. ;-)
--
Aragorn
(registered GNU/Linux user #223157)
J.O. Aho
There are some packages that requires a lot of free RAM memory to compile,
like OpenOffice (think that one may be one of the worst).
--
//Aho
pk
> again, that system has 32 GB of RAM in it, so your mileage may vary. ;-)
And I thought my 8GB were a lot! :-)
Aragorn
> Aragorn wrote:
>
>> If you (and/or the OP) have a sufficient amount of RAM in your system,
>> you could then make */var/tmp/portage* a /tmpfs./ It's what I do, but
>> then again, that system has 32 GB of RAM in it, so your mileage may
>> vary. ;-)
>
> There are some packages that requires a lot of free RAM memory to compile,
> like OpenOffice (think that one may be one of the worst).
According to the documentation, compiling OpenOffice or KDE uses up about 5
to 6 GB in */var/tmp/portage,* but as stated by others already, if the
compile is successful, then */var/tmp/portage* should be empty again after
that, and if it resides on a /tmpfs,/ then this memory is returned to
userspace as normal process memory.
The prerequisite of course is that one has plenty of RAM in the system. If
for instance you have 8 GB of RAM, then - considering that one usually
compiles OpenOffice or KDE only once over a long period of time - one would
still have 2 GB of regular virtual memory available - /tmpfs/ is pageable,
by the way - for running the base install system, which you would
preferably do without having X running anyway and which would thus only
require some 128 to 256 MB max for the operating system itself and the
build process. Using pipes during the build would also avoid a lot of
temporary files being created in */var/tmp/portage.*
Either way, having */var/tmp/portage* on a /tmpfs/ should work quite well
even on systems with a bit less of RAM outside of the building of KDE or
OpenOffice, and I've seen many recommendations for using the binary
OpenOffice installer rather than compiling it from sources.
But of course, these arguments are all moot if you have only 512 MB or so in
your system. I would recommend using a /tmpfs/ for */var/tmp/portage* only
if your system has at least 4 GB of RAM or if you're running a server that
doesn't require X11, and that thus needs neither KDE nor OpenOffice to be
installed.
Like I said, mileage will and does vary... :-)
Aragorn
That machine is destined to be used with multiple Gentoo instances running
atop of the Xen hypervisor. My goal is to set up four Gentoo instances -
the privileged virtual machine included - of which one will be a GUI
workstation installation with KDE, OpenOffice and various other graphical
applications, and the others being basically headless virtual servers. ;-)
The hardware for that machine - Tyan twin-socket ccNUMA board, twin dualcore
HE Opterons (2.6 GHz, 68 Watt), 32 GB ECC registered RAM (ATP pc5300), four
Hitachi 15k SAS 147 GB disks in RAID 5 on an Adaptec 32105 PCIe adapter,
Zippy 800 Watt EPS12V power supply and a CoolerMaster CM-832 chassis with
four 12cm fans - has taken a serious bite out of my piggybank as well...
Let's just say that one could buy a fairly small yet brandnew car for that
money - think "Opel(/Vauxhall) Astra" in basic trim, with the 1.4-liter 16v
engine... :p
J.O. Aho
>>> again, that system has 32 GB of RAM in it, so your mileage may vary. ;-)
>> And I thought my 8GB were a lot! :-)
> That machine is destined to be used with multiple Gentoo instances running
> atop of the Xen hypervisor. My goal is to set up four Gentoo instances -
> the privileged virtual machine included - of which one will be a GUI
> workstation installation with KDE, OpenOffice and various other graphical
> applications, and the others being basically headless virtual servers. ;-)
I have been thinking of my own setup, not as expensive as yours and I was
starting to think about OpenVZ, as I will anyhow just be using Linux so not
needing everything that Xen has to offer. It seems that OpenVZ is kinder to
the CPU ( http://www.hpl.hp.com/techreports/2007/HPL-2007-59.pdf ).
But there is still one trouble I have, dedicating graphics cards to different
virtual environments and USB-ports. At the moment what I have thought about is
"Firewall/Gateway" which has more or less just a read only environment,
File-server to share files, Media-box (dedicated graphics, remote control),
Desktop (dedicated graphics, keyboard and mouse), Server running mail, web...
This would be more or less, everything I have today in one box, still quite
normal off the shelf hardware, but still a bit over 1000 Euro, but still
cheaper than those Indian 5000 Euro cars.
--
//Aho
J.O. Aho
Reading the man pages for the openvz control tool, it seems not be that easy
to dedicate anything else than network cards for a VE, so it seems that Xen
may be the way to go anyway...
--
//Aho
Aragorn
> Aragorn wrote:
>>>> again, that system has 32 GB of RAM in it, so your mileage may vary.
>>>> ;-)
>>> And I thought my 8GB were a lot! :-)
>> That machine is destined to be used with multiple Gentoo instances
>> running
>> atop of the Xen hypervisor. My goal is to set up four Gentoo instances -
>> the privileged virtual machine included - of which one will be a GUI
>> workstation installation with KDE, OpenOffice and various other graphical
>> applications, and the others being basically headless virtual servers.
>> ;-)
>
> I have been thinking of my own setup, not as expensive as yours and I was
> starting to think about OpenVZ, as I will anyhow just be using Linux so
> not needing everything that Xen has to offer. It seems that OpenVZ is
> kinder to the CPU ( http://www.hpl.hp.com/techreports/2007/HPL-2007-59.pdf
> ).
I know OpenVZ, yes, and for a while I have also contemplated using it, along
with checking out the KVM approach. :-) Gentoo has packages for OpenVZ in
Portage, by the way, although they /may/ still be masked - this I don't
know.
> But there is still one trouble I have, dedicating graphics cards to
> different virtual environments and USB-ports.
Yes, and I can help you out there - I _ think_- when it comes to *Xen,* as I
have been playing with an idea, but I have not been able to test its
validity yet.
When building the XenLinux-patched /dom0/ kernel, you have the option to use
either a virtual framebuffer in /domU/ or not, and only upon selecting that
option do you get a second option, namely to select a virtual keyboard
for /domU./
In other words, the kernel configurator is set up to have the virtual
keyboard depend on the virtual framebuffer. The idea here is to be able to
switch the console from /dom0/ to /domU/ via the /xm/ commandline utility;
you can switch back to the privileged virtual machine via the key
combination /Ctrl+]/ - you may need to press /AltGr/ for the "]" character,
depending on your locale.
Now, if you want to use graphics inside /domU/ and you want to be actually
only run X11 there - and not in X11, with a remote connection to an
application running inside /domU,/ which is what most people do but which
is not the right way as it introduces a stability hazard to your management
virtual machine by running X there - then you need to have a dedicated
video adapter for the /domU/ machine.
So far so good, but the problem there is that by default, the Linux kernel
will still be using whatever it considers to be the primary video adapter -
the one found at /dom0/ boot time - for non-graphical output. So if you
want to use the dedicated /domU/ video adapter, you are stuck to only being
able to use it in X11, which you _can_ easily set up by editing
*/etc/xorg.conf* and manually specifying the PCI address for that adapter.
This approach would then of course also mean that you'd have to have that
operating system session configured to boot to runlevel 5, with a display
manager, and that you wouldn't get any output during the character mode
phase of the /domU/ boot and shutdown, unless you're willing to switch
between monitors or monitor channels every time you switch back and forth
between character mode and X11.
Now, my as yet untested solution is the following. First make sure you have
the necessary peripherals installed. Keyboard, mouse, two videocards, each
of which is connected to a channel on your monitor(s). Next, configure
your */dom0/* XenLinux kernel with PCI passthrough, and with support for
both the virtual framebuffer and the virtual keyboard. Finish off your
configuration and save the /.config/ file.
Now, open the /.config/ file again with an editor, and comment out the
virtual framebuffer part, but leave the virtual keyboard enabled. Then
compile your /dom0/ kernel - this is the untested part, so if you break it,
you get to keep both pieces :p - and configure and compile your /domU/
kernel. Also, find out the PCI address of the videocard you intend to
dedicate to /domU/ and hide that device from Xen via its boot parameters,
using for instance...
pciback.hide=(1:00.06)
... or whatever the PCI address for that card is.
If the above works as I think it would, then you should be able to switch
the console from /dom0/ to /domU/ in the usual manner, upon which you would
flick the switch - or push the button, depending on what monitor(s) you are
using - on your monitor(s) to switch channels. You should then have a
fully working console in /domU./
When compiling the evil proprietary nVidia driver in /domU/ however, you
must first export /IGNORE_XEN_PRESENCE/ as a variable inside the shell in
which you are building the driver. It will then be exported to all
subsequent shells created by the build process. Without that, the nVidia
driver will not build.
Eventhough I haven't gotten around to it yet, I believe it would also be
wisest to hide your soundcard - if any - from Xen and /dom0/ in the same
manner, via a Xen boot parameter. You would then of course not be able to
use any sound inside /dom0,/ but that is not what /dom0/ is for anyway. ;-)
Now, the above is of course for a Xen set-up. OpenVZ is very different
there in that you're not running multiple operating system kernels on top
of a hypervisor, but running multiple userspace instances on top of a
common kernel. OpenVZ does (to my knowledge) also not support X11, and
even if it did, it wouldn't be safe to do so, as you would be risking
kernel and hypervisor instability if you were to use the evil proprietary
driver.
With the Xen approach as I have laid out above, if the evil driver hangs the
kernel or even messes up the hardware framebuffer in such a way that you
need a reboot to recover from it, then the damage would still be limited to
that particular /domU/ virtual machine anyway, and your /dom0/ and
hypervisor would be safe, and as such, so would your other /domU/ virtual
machines be. So all of your individual server set-ups would continue to be
available while you are rebooting your GUI workstation virtual machine.
This is why it is wrong to run X11 in /dom0,/ unless /dom0/ is your
workstation installation and is not to be considered mission-critical.
Such a set-up would be good for using /domU/ as a testbed or sandbox only.
My intent however is to have all /domU/ virtual machines be
mission-critical, with the graphical virtual machine preferably being as
stable as possible but acknowledging that it holds a risk due to having to
use the evil proprietary nVidia driver for 3D graphics. If you don't need
3D acceleration, then you can use the more reliable (and less cumbersome to
set up) FOSS "nv" driver.
> At the moment what I have thought about is "Firewall/Gateway" which has
> more or less just a read only environment, File-server to share files,
> Media-box (dedicated graphics, remote control), Desktop (dedicated
> graphics, keyboard and mouse), Server running mail, web...
If you need dedicated graphics in more than one unprivileged virtual
machine, then you'll need to add an additional video adapter card for each
of them. You can't share one videocard among multiple virtual machines,
except via a virtual framebuffer. Likewise, your remote control apparatus
will have to be hidden from /dom0/ and must only be accessed by one
unprivileged virtual machine.
But then again still, this would not be feasible with OpenVZ. You'd need to
use Xen for such a set-up. I would also set up the firewalling, NAT and
gateway thing in /dom0,/ since that is the virtual machine that has access
to the physical NICs in your machine. You should then also set up the Xen
networking configuration to make use of routing instead of the default
choice of bridging.
If you want to use ethernet bridging set-up that Xen defaults to, then each
of your virtual machines must have a dedicated IP address of its own, and
this might be tricky with most ISPs. Mine for instance - Telenet here in
Belgium - gives me only two public IP addresses (via semi-static DHCP) for
my current contract, and when my machine will be fully set up, I intend to
switch to a contract that gives me a guaranteed static IP address, but then
I'll only have one. If I want anything more, then the next step is 8
static IP addresses over ADSL, but that would also require me to rent a
landline or a raw copper from Belgacom, as I'm on cable internet right now
and this 8 IP addresses deal is not possible over cable.
So if you do want bridging and your ISP only gives you one public IP
address, then you should get a router and set that one up as the default
gateway and firewall, and then you can have the router assign as many IP
addresses to your Xen box as you like.
> This would be more or less, everything I have today in one box, still
> quite normal off the shelf hardware, but still a bit over 1000 Euro, but
> still cheaper than those Indian 5000 Euro cars.
You mean that backpack on wheels from Tata Motors? :p Wasn't that 2500
Euro, by the way? ;-)
Aragorn
> [...]
>
> Now, if you want to use graphics inside /domU/ and you want to be actually
> only run X11 there - and not in X11, with a remote connection to an
^^^
That should read "...and not in /dom0..."/.
> application running inside /domU,/ which is what most people do but which
> is not the right way as it introduces a stability hazard to your
> management virtual machine by running X there - then you need to have a
> dedicated video adapter for the /domU/ machine.
>
> [...]
>
> When compiling the evil proprietary nVidia driver in /domU/ however, you
> must first export /IGNORE_XEN_PRESENCE/ as a variable inside the shell in
> which you are building the driver. It will then be exported to all
> subsequent shells created by the build process. Without that, the nVidia
> driver will not build.
The /IGNORE_XEN_PRESENCE/ variable should have a value other than "0".
So, ...
export IGNORE_XEN_PRESENCE=1
... should work. If it's set to "0", then the driver won't compile. I got
that off the nVidia forums. ;-)
Nikos Chantziaras
> [...]
> The hardware for that machine - Tyan twin-socket ccNUMA board, twin dualcore
> HE Opterons (2.6 GHz, 68 Watt), 32 GB ECC registered RAM (ATP pc5300), four
> Hitachi 15k SAS 147 GB disks in RAID 5 on an Adaptec 32105 PCIe adapter,
> Zippy 800 Watt EPS12V power supply and a CoolerMaster CM-832 chassis with
> four 12cm fans - has taken a serious bite out of my piggybank as well...
If it doesn't run Quake 4 with 200FPS it's not worth the money :P
Aragorn
That would largely depend on the video adapter and on the proprietary, evil
nVidia driver. ;-)
On the other hand, I do think that something that compiles a complete Linux
kernel in only just over one minute is worth its money, though... :p
Arthur Hagen
> Darin McBride wrote:
>>
>> While it's safe, I wouldn't clear out distfiles. /var/tmp/portage,
>> however, is safe as long as you don't have a current emerge running
>> ;-)
>
> If you (and/or the OP) have a sufficient amount of RAM in your
> system, you could then make */var/tmp/portage* a /tmpfs./ It's what
> I do, but then again, that system has 32 GB of RAM in it, so your
> mileage may vary. ;-)
A minor problem with that is that /var/tmp/portage is also the home
directory for the portage user account, and some other persistent stuff
might be saved in it (like a .forward for mail or .distcc for distributed
compiling). Having user accounts pointing to temporary directories is
usually a bad idea, and I wish gentoo had used a different directory for
this.
Regards,
--
*Art
J.O. Aho
> I know OpenVZ, yes, and for a while I have also contemplated using it, along
> with checking out the KVM approach. :-) Gentoo has packages for OpenVZ in
> Portage, by the way, although they /may/ still be masked - this I don't
> know.
I haven't looked that far, but it seems to be unmasked according the Gentoo
Howto at the wiki, but this far dedicating hardware don't work more than for
NICs as far as I managed to figure things out, which means that OpenVZ won't
be an option for me.
> Now, if you want to use graphics inside /domU/ and you want to be actually
> only run X11 there - and not in X11, with a remote connection to an
> application running inside /domU,/ which is what most people do but which
> is not the right way as it introduces a stability hazard to your management
> virtual machine by running X there - then you need to have a dedicated
> video adapter for the /domU/ machine.
> So far so good, but the problem there is that by default, the Linux kernel
> will still be using whatever it considers to be the primary video adapter -
> the one found at /dom0/ boot time - for non-graphical output. So if you
> want to use the dedicated /domU/ video adapter, you are stuck to only being
> able to use it in X11, which you _can_ easily set up by editing
> */etc/xorg.conf* and manually specifying the PCI address for that adapter.
I have been thinking of using 3 graphics cards, two PCIe and one PCI, where I
was hoping to be able to use the PCI card as the main graphics adapter, as
console output don't require any monster graphics cards.
While the desktop and the media-box instances I was hoping to give them a PCIe
graphics card each, both should be able to run at the same time.
> This approach would then of course also mean that you'd have to have that
> operating system session configured to boot to runlevel 5, with a display
> manager, and that you wouldn't get any output during the character mode
> phase of the /domU/ boot and shutdown, unless you're willing to switch
> between monitors or monitor channels every time you switch back and forth
> between character mode and X11.
Yes, but in gentoo you default have initlevel 3 for graphical environment, as
people seems to only use "default", I do want to have things setup like in
RedHat where level 3 is multiuser mode with VT and level 5 is Xorg. Myself I
added graphical, no network as level 4 (reserved for future use in RedHat), so
I have an equivalent to level 2.
> With the Xen approach as I have laid out above, if the evil driver hangs the
> kernel or even messes up the hardware framebuffer in such a way that you
> need a reboot to recover from it, then the damage would still be limited to
> that particular /domU/ virtual machine anyway, and your /dom0/ and
> hypervisor would be safe, and as such, so would your other /domU/ virtual
> machines be. So all of your individual server set-ups would continue to be
> available while you are rebooting your GUI workstation virtual machine.
*nods*
That is one of the strong benefits with Xen over the others, but OpenVZ has
easier resource management system, where you can make changes on the fly. As I
have understood Xen, you have to restart the instance before it gets the new
resources.
> This is why it is wrong to run X11 in /dom0,/ unless /dom0/ is your
> workstation installation and is not to be considered mission-critical.
> Such a set-up would be good for using /domU/ as a testbed or sandbox only.
> My intent however is to have all /domU/ virtual machines be
> mission-critical, with the graphical virtual machine preferably being as
> stable as possible but acknowledging that it holds a risk due to having to
> use the evil proprietary nVidia driver for 3D graphics. If you don't need
> 3D acceleration, then you can use the more reliable (and less cumbersome to
> set up) FOSS "nv" driver.
No, I don't want a graphical environment for dom0, I want it to be as clean as
possible, the only thing I will run on it would be a NFS share, as I'm not
that happy about the file I/O in a domU and thinking about some kind of tftp
boots for the domUs.
>> At the moment what I have thought about is "Firewall/Gateway" which has
>> more or less just a read only environment, File-server to share files,
>> Media-box (dedicated graphics, remote control), Desktop (dedicated
>> graphics, keyboard and mouse), Server running mail, web...
>
> If you need dedicated graphics in more than one unprivileged virtual
> machine, then you'll need to add an additional video adapter card for each
> of them. You can't share one videocard among multiple virtual machines,
> except via a virtual framebuffer. Likewise, your remote control apparatus
> will have to be hidden from /dom0/ and must only be accessed by one
> unprivileged virtual machine.
It had been nice if Xen had the same way to deal with hardware as OpenVZ have
for their network devices and you can do the change in realtime with no need
to reboot.
> But then again still, this would not be feasible with OpenVZ. You'd need to
> use Xen for such a set-up. I would also set up the firewalling, NAT and
> gateway thing in /dom0,/ since that is the virtual machine that has access
> to the physical NICs in your machine. You should then also set up the Xen
> networking configuration to make use of routing instead of the default
> choice of bridging.
Was thinking about if it would be possible to dedicate a nic in a similar way
to a domU as a graphics card. IMHO it feels better if someone "hack" a domU
with only read only access to files, than getting access to dom0, and if
something would get messy then just restart the domU instead of a reboot of
the whole system.
> If you want to use ethernet bridging set-up that Xen defaults to, then each
> of your virtual machines must have a dedicated IP address of its own, and
> this might be tricky with most ISPs. Mine for instance - Telenet here in
> Belgium - gives me only two public IP addresses (via semi-static DHCP) for
> my current contract, and when my machine will be fully set up, I intend to
> switch to a contract that gives me a guaranteed static IP address, but then
> I'll only have one.
My ISP allows just one public IP (thanks to someone smart person at that
company, it's a static one), but the majority os ISPs over here offers only
DHCP and with one dynamic ip-address.
I don't want any direct access to the other domUs from the outside, so
bridging ain't the option for me.
> So if you do want bridging and your ISP only gives you one public IP
> address, then you should get a router and set that one up as the default
> gateway and firewall, and then you can have the router assign as many IP
> addresses to your Xen box as you like.
I have a route from my ISP, but I have disabled the routing and firewall, as I
feel I have more control over thing when I run it on my computer, which won't
loose logfiles just doe a blackout or restart.
>> This would be more or less, everything I have today in one box, still
>> quite normal off the shelf hardware, but still a bit over 1000 Euro, but
>> still cheaper than those Indian 5000 Euro cars.
> You mean that backpack on wheels from Tata Motors? :p Wasn't that 2500
> Euro, by the way? ;-)
Yes, that one. It may have dropped in price, but still the hardware solution I
have looked at is cheaper than the car. :)
--
//Aho
J.O. Aho
I would say it's cool to have, but not worth all the money, I can wait 5
minutes and afford to go on vacation. :)
--
//Aho
Aragorn
What's a vacation? :p
Nikos Chantziaras
> J.O. Aho wrote:
>> I would say it's cool to have, but not worth all the money, I can wait 5
>> minutes and afford to go on vacation. :)
>
> What's a vacation? :p
Vacation == Not looking at the output of `emerge -e world` :P
Aragorn
> Aragorn wrote:
>
>> Now, if you want to use graphics inside /domU/ and you want to be
>> actually only run X11 there - and not in X11, with a remote connection to
>> an application running inside /domU,/ which is what most people do but
>> which is not the right way as it introduces a stability hazard to your
>> management virtual machine by running X there - then you need to have a
>> dedicated video adapter for the /domU/ machine.
>>
>> So far so good, but the problem there is that by default, the Linux
>> kernel will still be using whatever it considers to be the primary video
>> adapter - the one found at /dom0/ boot time - for non-graphical output.
>> So if you want to use the dedicated /domU/ video adapter, you are stuck
>> to only being able to use it in X11, which you _can_ easily set up by
>> editing */etc/xorg.conf* and manually specifying the PCI address for that
>> adapter.
>
> I have been thinking of using 3 graphics cards, two PCIe and one PCI,
> where I was hoping to be able to use the PCI card as the main graphics
> adapter, as console output don't require any monster graphics cards.
I agree that you wouldn't need a powerful graphics adapter for character
mode consoles, but unless you follow the - I admit - experimental approach
I have mentioned - i.e. disabling virtual framebuffer support for /domU/ in
your /dom0/ kernel while still allowing virtual keyboard support - you are
going to have to switch your monitor back and forth between two channels
each time you switch between X11 and a character mode console, whereas if
you were to do it the way I suggested, you would switch monitor channels
when switching between /dom0/ and /domU./
Either way, with one graphics adapter being a PCI card, that will be the one
seen as the primary video adapter by the BIOS and by the /dom0/ kernel.
> While the desktop and the media-box instances I was hoping to give them a
> PCIe graphics card each, both should be able to run at the same time.
True, but you only have one console, so you would need to use the /xm/
command in /dom0/ in order to switch to one of the other virtual machines.
A workaround for this could possibly exist - this is pure speculation on my
part, so again if you break anything, the pieces are all yours to keep :p -
if you were to use a plug-in PCI adapter with PS/2 ports for an additional
mouse and keyboard, and hide this adapter from Xen and /dom0./
In theory, it would also be possible to use a dedicated USB keyboard and
mouse for a /domU/ virtual machine, but from what I've read so far on the
web, this theory does not stroke with reality, and all kinds of badness
occurs if you try it that way.
The same could be true for the plug-in adapter card with PS/2 ports for all
I know; nobody seems to have tried it that way yet, or if they did, then
they didn't report on their success or failure.
>> This approach would then of course also mean that you'd have to have that
>> operating system session configured to boot to runlevel 5, with a display
>> manager, and that you wouldn't get any output during the character mode
>> phase of the /domU/ boot and shutdown, unless you're willing to switch
>> between monitors or monitor channels every time you switch back and forth
>> between character mode and X11.
>
> Yes, but in gentoo you default have initlevel 3 for graphical environment,
> as people seems to only use "default", I do want to have things setup like
> in RedHat where level 3 is multiuser mode with VT and level 5 is Xorg.
> Myself I added graphical, no network as level 4 (reserved for future use
> in RedHat), so I have an equivalent to level 2.
I have only used display managers when I was still new to GNU/Linux. My
laptop is still set up that way, but I barely ever use it anymore - it
still has Mandrake 8.2 on it, and it only has 128 MB (minus 16 MB for the
graphics), which was already quite meager (for use with KDE) in the days of
Mandrake 8.2 itself. Just goes to show how often I still boot it up. ;-)
My main reason for wanting to start X11 from the commandline rather than
using a display manager is that without a display manager, you have one
less process running as root, and if something goes wrong in X11 - e.g.
you've b0rk3d your xorg.conf - then you can kill it without that it's
respawning all the time.
>> With the Xen approach as I have laid out above, if the evil driver hangs
>> the kernel or even messes up the hardware framebuffer in such a way that
>> you need a reboot to recover from it, then the damage would still be
>> limited to that particular /domU/ virtual machine anyway, and your /dom0/
>> and hypervisor would be safe, and as such, so would your other /domU/
>> virtual machines be. So all of your individual server set-ups would
>> continue to be available while you are rebooting your GUI workstation
>> virtual machine.
>
> *nods*
> That is one of the strong benefits with Xen over the others, but OpenVZ
> has easier resource management system, where you can make changes on the
> fly. As I have understood Xen, you have to restart the instance before it
> gets the new resources.
From what I know of Xen so far, that is correct. However, the balloon
driver does allow you to allocate more or less memory to a virtual machine
in realtime, albeit that utilities like /top/ don't notice this, because
the memory information in */proc* is what the kernel exported there at boot
time.
>> This is why it is wrong to run X11 in /dom0,/ unless /dom0/ is your
>> workstation installation and is not to be considered mission-critical.
>> Such a set-up would be good for using /domU/ as a testbed or sandbox
>> only. My intent however is to have all /domU/ virtual machines be
>> mission-critical, with the graphical virtual machine preferably being as
>> stable as possible but acknowledging that it holds a risk due to having
>> to use the evil proprietary nVidia driver for 3D graphics. If you don't
>> need 3D acceleration, then you can use the more reliable (and less
>> cumbersome to set up) FOSS "nv" driver.
>
> No, I don't want a graphical environment for dom0, I want it to be as
> clean as possible, the only thing I will run on it would be a NFS share,
> as I'm not that happy about the file I/O in a domU and thinking about some
> kind of tftp boots for the domUs.
My current set-up - which still needs completion - has */usr/src* and
*/usr/portage* on logical volumes in /dom0,/ which I intend to export via
NFS to the /domU/ virtual machines.
Unprivileged virtual machines can be bootstrapped from NFS in Xen. Their
kernels must be available in a /dom0/ filesystem anyway.
>> But then again still, this would not be feasible with OpenVZ. You'd need
>> to use Xen for such a set-up. I would also set up the firewalling, NAT
>> and gateway thing in /dom0,/ since that is the virtual machine that has
>> access to the physical NICs in your machine. You should then also set up
>> the Xen networking configuration to make use of routing instead of the
>> default choice of bridging.
>
> Was thinking about if it would be possible to dedicate a nic in a similar
> way to a domU as a graphics card.
In theory, that should be possible.
> IMHO it feels better if someone "hack" a domU with only read only access
> to files, than getting access to dom0, and if something would get messy
> then just restart the domU instead of a reboot of the whole system.
My intended set-up is to have /dom0/ do the routing, network address
translation and firewalling, and to direct port 22 to an unprivileged
virtual machine. Remote access to /dom0/ would then still be possible, but
via a detour through the unprivileged machine.
In addition, I intend to set up all virtual machines - including /dom0/ - to
disallow direct root logins, so that one first has to log in via an
unprivileged user account and then use /su/ or possibly /sudo/ to perform
root tasks. I've also briefly looked into role-based access control, but
that seems a bit cumbersome as well as being overkill for a machine with
only one or two logins.
And of course, as I have mentioned in earlier posts a while ago, I prefer
having just about all my filesystems mounted read-only, except for...:
- /home
- /tmp
- /var (and everything under it)
- /dev (handled by udev and thus off the root filesystem)
- /proc (virtual anyway)
- /sys (virtual as well)
- /root (kept off the root filesystem)[1]
- /srv
[1] */root* will hold the necessary environment files as copies on the
read-only root filesystem for when this separate filesystem is not mounted.
>> If you want to use ethernet bridging set-up that Xen defaults to, then
>> each of your virtual machines must have a dedicated IP address of its
>> own, and this might be tricky with most ISPs. Mine for instance -
>> Telenet here in Belgium - gives me only two public IP addresses (via
>> semi-static DHCP) for my current contract, and when my machine will be
>> fully set up, I intend to switch to a contract that gives me a guaranteed
>> static IP address, but then I'll only have one.
>
> My ISP allows just one public IP (thanks to someone smart person at that
> company, it's a static one), but the majority os ISPs over here offers
> only DHCP and with one dynamic ip-address.
With Telenet, if you have a standard consumergrade internet connection (as I
currently have) then your IP address is assigned to you via DHCP but is
semi-static, in that your IP address is linked to the MAC address of the
NIC connected to it and stored as such in a memory buffer in your cable
modem.
If you want to connect another computer to your cable modem, then you must
reset the modem first but leaving it unplugged from the wall power socket
for about 15 minutes. As long as you don't do that and as long as there is
no interruption of the cable network between Telenet hubs and your end-user
connection - which could occur due to roadworks, for instance - that
particular IP address stays assigned to that particular MAC.
However, the above is not workable if you want to really rely on your IP
address being static, e.g. if you want to attach a domain name to it. As I
intend to use one of the virtual machines - the one that gets the /ssh/
access to the internet - as a server in our IRC network, I need to be able
to rely on my IP address remaining static no matter what.
I will thus need to upgrade to an "Office Line" connection, which will still
work with all my existing hardware - i.e. it can use the cable internet
connection and the cable modem - and then my theoretical maximum download
speed will be doubled - my upload speed will be throttled to 384 Kbit/sec
max, whereas it is 256 Kbit/sec now and would for end-users in my region
normally be doubled to 512 Kbit/sec this month.
My traffic quota and my number of POP3 mailboxes will also increase, from 12
GB/month now to 52 GB/month and from 4 POP3 mail accounts to 10 accounts.
On the downside, I now have two public IP addresses, and when I switch to
an Office Line, I will only have one - plus the IP address needed for my
digital television contract.
But like I said, if I want anything with more connections, then I'll have to
opt for ADSL or SDSL, and that's very expensive, plus that it requires
bringing in a third party - namely Belgacom, who own all the phone and
copper lines in this country - and renting a line from them. I will then
also have to buy a router - they seem to insist that you buy Cisco - and
either figure out how to set it up myself or have them do it for me, which
costs even more.
I know all about that last kind of deal as well as that is the kind of deal
we have now for our IRC network and our hosting "business" - basically a
beta project intended to help us fund the exploitation of our IRC network,
although we also host our own domain, of course. So we have an ADSL
connection - the most affordable among their professional offering - with 8
public IP addresses, of which the Cisco router takes up one.
> I don't want any direct access to the other domUs from the outside, so
> bridging ain't the option for me.
Well, if you were to use a router between your Xen box and your internet
connection, then you could use bridging and then you can still shield off
several virtual machines from the internet, as the router would then be
your gateway and firewall.
>>> This would be more or less, everything I have today in one box, still
>>> quite normal off the shelf hardware, but still a bit over 1000 Euro, but
>>> still cheaper than those Indian 5000 Euro cars.
>>
>> You mean that backpack on wheels from Tata Motors? :p Wasn't that 2500
>> Euro, by the way? ;-)
>
> Yes, that one. It may have dropped in price, but still the hardware
> solution I have looked at is cheaper than the car. :)
Well, I haven't exactly looked at the specs for that car yet, but I presume
that it won't be winning any "Grand Prix of the Red Lights" anyway, and
unless you're a smurf, that thing doesn't look very spacious either. ;-)
Hmm... Now why am I thinking back of those old three-wheeler bubblecars
from when I was a little boy? :p
J.O. Aho
> J.O. Aho wrote:
>> IMHO it feels better if someone "hack" a domU with only read only access
>> to files, than getting access to dom0, and if something would get messy
>> then just restart the domU instead of a reboot of the whole system.
> My intended set-up is to have /dom0/ do the routing, network address
> translation and firewalling, and to direct port 22 to an unprivileged
> virtual machine. Remote access to /dom0/ would then still be possible, but
> via a detour through the unprivileged machine.
> In addition, I intend to set up all virtual machines - including /dom0/ - to
> disallow direct root logins, so that one first has to log in via an
> unprivileged user account and then use /su/ or possibly /sudo/ to perform
> root tasks.
Yes, I have always run ssh with root login disabled, I have even set users who
don't need to be able to use ssh to be disabled too. Another thing I made when
I got all to many script kiddies trying to force in, was to move the ssh away
fro port 22, it don't make the system more secure, but you get rid those
script kiddies who tries to force login in as john and a million other common
US-English names. Also at the third attempt from the same ip, the ip is
blocked for a hour for additional attempts.
Sure sudo is "great", but I don't feel it as secure as su, as you continue to
use the user password, while using su you need to know the root password
before you can do anything.
>>>> This would be more or less, everything I have today in one box, still
>>>> quite normal off the shelf hardware, but still a bit over 1000 Euro, but
>>>> still cheaper than those Indian 5000 Euro cars.
>>> You mean that backpack on wheels from Tata Motors? :p Wasn't that 2500
>>> Euro, by the way? ;-)
>> Yes, that one. It may have dropped in price, but still the hardware
>> solution I have looked at is cheaper than the car. :)
> Well, I haven't exactly looked at the specs for that car yet, but I presume
> that it won't be winning any "Grand Prix of the Red Lights" anyway, and
> unless you're a smurf, that thing doesn't look very spacious either. ;-)
No, it won't win any prices, specially not traffic safe prices, I think it's
more in the class with original coopers, if you crashed with it you could be
95% sure you would die.
> Hmm... Now why am I thinking back of those old three-wheeler bubblecars
> from when I was a little boy? :p
:) just remember those from movies and telly programs shot "some" years
earlier than my first year, but it was still without safety belts in the back
and only 2 point safety belts in the front on most cars.
--
//Aho
pk
> Sure sudo is "great", but I don't feel it as secure as su, as you continue
> to use the user password, while using su you need to know the root
> password before you can do anything.
Actually, sudo can be configured to request the /root/ password, instead of
that of the user. See the "rootpw" flag in man sudoers.
Graham Murray
> Yes, but in gentoo you default have initlevel 3 for graphical
> environment, as people seems to only use "default",
That is true only if the user/administrator adds X to the default
runlevel.
J.O. Aho
If I recall it right, "rc-update add xdm default" was in the early Gentoo
documentation about setting up an auto starting X11 environment, the rc-update
is from the gentoo-wiki.
--
//Aho
Nikos Chantziaras
That's what I followed to set up X11. I wasn't even aware that Gentoo
actually has runlevels because there are no /etc/init.d/rc.*
directories. I'd prefer runlevels like in other distros.
J.O. Aho
It shouldn't be too difficult to do that
ln -sfn /etc/runlevels /etc/rc.d
mkdir -p /etc/rc.d
ln -sfn /etc/runlevels/1 /etc/rc.d/rc1.d
ln -sfn /etc/runlevels/1 /etc/rc.d/rc2.d
ln -sfn /etc/runlevels/1 /etc/rc.d/rc3.d
ln -sfn /etc/runlevels/1 /etc/rc.d/rc4.d
ln -sfn /etc/runlevels/1 /etc/rc.d/rc5.d
mkdir -p /etc/runlevels/1
mkdir -p /etc/runlevels/2
mkdir -p /etc/runlevels/3
mkdir -p /etc/runlevels/4
mkdir -p /etc/runlevels/5
replace in /etc/inittab:
l1:S1:wait:/sbin/rc single
l2:2:wait:/sbin/rc default
l3:3:wait:/sbin/rc default
l4:4:wait:/sbin/rc default
l5:5:wait:/sbin/rc default
to:
l1:S1:wait:/sbin/rc 1
l2:2:wait:/sbin/rc 2
l3:3:wait:/sbin/rc 3
l4:4:wait:/sbin/rc 4
l5:5:wait:/sbin/rc 5
Just don't forget to update the runlevels to contain the right service symlinks.
--
//Aho
pk
Wouldn't all of this be blown away at the first baselayout update? You'll
have to be careful to not let portage overwrite the files you modified.
J.O. Aho
No, inittab is protected by default, you will get opted if you want to update
it or not, and you can always protect it more permanently in make.conf.
Upgrading baselayout won't affect anything else.
--
//Aho
pk
> No, inittab is protected by default, you will get opted if you want to
> update it or not, and you can always protect it more permanently in
> make.conf.
Protecting a file only means it's not automatically overwritten, but as long
as portage sees the ._cfg* files it will keep asking (through etc-update or
similar tools) whether you want to overwrite it. Not saying that you can't
do that, of course, but you still have to be careful, especially if
etc-update tells you that 144 files need to be updated, /etc/inittab is
among those, and you don't notice that (I'm speaking out of direct
experience, btw; I once unthinkingly replaced my customized inittab with
the default one. Luckily I had a backup :-)).