Virtualization Progress

[Alan Smith]

Hello everyone,

Welcome to the ZestyPanel Google Group! You should join the Google Code project: http://code.google.com/p/zestypanel. Once you've joined I'll elevate your privileges so you can all contribute to the wiki and code base, as well as issue tracker, etc. Please post control panel mockups to the wiki and post an email here saying you've done that.

Now that you're all feeling warm and welcome, let me tell you what I've discovered on the server side of this project. As you know, we want to isolate each server/framework pair to prevent conflicts in configuration, mainly. However, the servers should share users and their files. The heavy solution would be to use something like VirtualBox and run a full-blown OS for each server. That would be resource intensive and difficult to maintain that many operating systems.

There is a much simpler solution: use "containers" within a host OS to isolate the servers. Containers allow isolation of users and processes as well as filesystem access. There are two implementations of containers that I considered: LXC (Linux Containers) and FreeBSD Jails. Both are very powerful but I'm leaning towards jails as FreeBSD is favored as a server for its stability. The jails are also easier to setup/tear-down and maintain in general. Unless anyone has any objections I will proceed with FreeBSD and jails.

Sharing of users' files will be as simple as adding another mount point to each jail, mounting /home as a device. Easy enough?

What do you all think?

Alan

[Alan Smith]

For those of you who are more familiar with Linux than FreeBSD, like myself, I strongly suggest you read this:

http://www.over-yonder.net/~fullermd/rants/bsd4linux/bsd4linux1.php

It was very insightful and made me itch to get my hands on BSD.

Enjoy, Alan

[Alan Smith]

Hello again,

Upon request here are more details about how jails work:

What is a container?

A container virtualizes the operating system creating a "container" where processes and users are isolated from other processes and user on the system. It is essentially a glorified "chroot" (http://en.wikipedia.org/wiki/Chroot). The only component all containers/jails share is the kernel.

How do OS updates work?

Updating the host OS is conducted as it would be normally. Jails have no affect on the update process for the host OS.

When you update the host OS the changes do not apply to the jails. The OS the jails use are a "skeleton" of the host. They have a bare bones copy of the host which has to be recreated when the system is updated. (See "15.6.1.4 Upgrading" at the bottom of http://www.freebsd.org/doc/handbook/jails-application.html.)

How do package updates work?

The packages are isolated in jails and the host OS. If you want to update a package you have to do it on the host OS and then in each jail, individually. You execute the 'jexec' command and you're in the jail; it's very similar to ssh'ing into a server. This might seem like a lot of extra work but it would be no easier if you were running multiple computers/virtual computers. Scripting of the jails should not be difficult if you want to automate the management of packages across the jails. However, the packages being exclusive to each jail has the advantage of making it possible to withhold an update to a package if it breaks something for that server/framework.

If any of this doesn't make sense or you have a question, read the handbook first (http://www.freebsd.org/doc/handbook/jails.html). If you're still perplexed feel free to contact me; I'd be happy to help.

Alan

[Alexis Métaireau]

Le 10/14/2010 05:43 PM, Alan Smith a �crit :

> Both are very powerful but I'm leaning towards jails as FreeBSD is favored
> as a server for its stability. The jails are also easier to setup/tear-down
> and maintain in general. Unless anyone has any objections I will proceed
> with FreeBSD and jails.
> Sharing of users' files will be as simple as adding another mount point to
> each jail, mounting /home as a device. Easy enough?
>
> What do you all think?

Hi Allan,

What I'm doing on my personal server (fbsd) is simply do share the users
by ldap, on the different jails, that way, we can have a central acces
point to all their privileges.

I'm +1 on using fbsd jails.

Why do you want to share the user files on all the jails ? Is there an
use case to do so ?

Cheers,
Alex

[Alan Smith]

Hi Alex,

That's fantastic, thanks for exemplifying your usage of ldap.

The idea behind sharing the users' files is that all files belonging to a user are in one place and still accessible to the servers. That's probably obvious. What I don't want are users creating, for example, a Django project and it going in the Django directory which is outside of their home directory. That would confuse people and complicate the managing of directory permissions. If the users' files are in their home directories then giving the server access to them is as easy as mounting /home.

What I would prefer is to only "connect" the directories of the projects/web apps to the servers that will serve them. For example, if I create a Django project called "pandoraripper" in /home/alan (i.e. /home/alan/pandoraripper) I don't really want the server gaining access to /home/alan; I would instead prefer that /home/alan/pandoraripper be all that is revealed. What would you propose is the best solution? My thought was to simply mount /home/alan/pandoraripper as /projects/pandoraripper (or similar) in the Django server. Does that make sense?

Alan

P.S. I won't be able to setup a FreeBSD server until Monday as the weekend is packed. Once it's up I'll give an account to those who want one.

[Alexis Métaireau]

Le 10/15/2010 12:13 AM, Alan Smith a �crit :

> The idea behind sharing the users' files is that all files belonging to a
> user are in one place and still accessible to the servers. That's probably
> obvious. What I don't want are users creating, for example, a Django project
> and it going in the Django directory which is outside of their home
> directory. That would confuse people and complicate the managing of
> directory permissions. If the users' files are in their home directories
> then giving the server access to them is as easy as mounting /home.

What I do on my server, is to create users on the host, with the same
uid than the users on the jails.

That way, it's easy to do a symbolic link from a specific directory on
the server to a directory on the jail. This, also, needs to have only
one server, but is really quick and usable in term of performances. I'm
not sure about the security issues, and it seems to be a bit weird, but
it can be useful, maybe in another way.

> What I would prefer is to only "connect" the directories of the projects/web
> apps to the servers that will serve them.

Do you mean the "jails" ? If not, how many servers have "we", and what
is the averall organisation of them ?

> For example, if I create a Django
> project called "pandoraripper" in /home/alan (i.e. /home/alan/pandoraripper)
> I don't really want the server gaining access to /home/alan; I would instead
> prefer that /home/alan/pandoraripper be all that is revealed. What would you
> propose is the best solution? My thought was to simply mount
> /home/alan/pandoraripper as /projects/pandoraripper (or similar) in the
> Django server. Does that make sense?

BTW, I think something like /home/alan/django/pandoraripper could be
better, but I agree that the only thing that have to be revealed is the
project files.

Hmm, this make me think about something else: what are you planning to
use to serve WSGI for python/django applications ? There gunicorn, a
great tool for such purposes, which can be installed in front of nginx.

And what infrastructure are you planning to install (which
servers/(reverse)proxy/etc) ?

> P.S. I won't be able to setup a FreeBSD server until Monday as the weekend
> is packed. Once it's up I'll give an account to those who want one.

Okay fine, have a funny weekend !

Alex

[Gregory Mendez]

This is only tangentially related to the current discussion thread but I feel it needs to be brought up sooner rather than later.

When we last did an OS reinstall (~May) we considered switching (we were first Debian and have been Gentoo for many years now). We rather seriously considered FreeBSD but didn't like its package system. While Portage was driving us nuts with how slow it had gotten to list new software as stable, its commands are simple and errors are much nicer than fBSD. It also defaulted to installing in /usr/local/bin rather than the structure Gentoo uses (that may not be so much of an issue since this change is big enough where things will break). 

Some other things we considered were hardened Gentoo, Debian (again), http://www.gentoo.org/proj/en/gentoo-alt/ , http://paludis.pioto.org/, http://emerde.freaknet.org/, http://www.pardus.org.tr/eng/projects/pisi/PiSi.html. In the end we just went with what we were familiar with thinking we would revisit the issue when we started this project. Most of the alternative packaging systems I linked just didn't seem production ready. 

But my main reason for bringing this up is to make sure we discuss and consider packaging systems and not just the virtualization options while considering OSes.

[Alan Smith]

Alex,

 

What I do on my server, is to create users on the host, with the same

uid than the users on the jails.

I was wondering if that would work. Good to know it's possible.

 

That way, it's easy to do a symbolic link from a specific directory on
the server to a directory on the jail. This, also, needs to have only
one server, but is really quick and usable in term of performances. I'm
not sure about the security issues, and it seems to be a bit weird, but
it can be useful, maybe in another way.

 

I don't know why that technique didn't occur to me. Nice thought. We will definitely have to try that and see what the security implications are.

Do you mean the "jails" ? If not, how many servers have "we", and what

is the averall organisation of them ?

The servers are going to be paired with a framework. One server for each framework and vice-a-versa. It'll be a one-to-one relationship. The server will be selected to compliment the framework it is running. Note, server/framework pairs should be easy to add/remove.

Hmm, this make me think about something else: what are you planning to

use to serve WSGI for python/django applications ? There gunicorn, a
great tool for such purposes, which can be installed in front of nginx.

As I said above, each server will be selected as the best match for its framework. That is why we asked for volunteers with experience using some of the popular framework. The hope was that we would find people who knew what servers worked best with which frameworks. Your input will obviously be helpful.

Cheers, Alan

[Duncan]

Alan,

I've read the entire thread; there is already a lot of good information here. Once we get further into development I can set up an AL-hosted virtual machine to do work on.

I agree with keeping stuff simple from a user perspective. My ideal is that if a user needs to know why something needs to be done in X way we should look for a better implementation. As a very long-term Mac user I value transparency for users, and if we can mange to make operations for the users obvious then I would consider that to be a success. Keeping stuff simple for our users makes our support easier which makes my (and Gregory's) lives easier.

Alan, I see you're saying we'd share directories with the jails to serve them. What limits do we have here? Are DBs considered to be outside of the jail, or mounted inside? Could I create a site that uses both django and rails for different parts of the site, while still sharing a back-end DB? I hope I'm not going too deep, just trying to understand what we're trying to do.

Alan, have you looked at KaOS at all? I have no attachment to it beyond the appearance that it could be run on Gentoo.

I hope I haven't gone too deep at this early stage, but those are some of my thoughts at this point.

-

Duncan

Alan

--
This message was scanned by ESVA and is believed to be clean.
Click here to report this message as spam.

[Alan Smith]

Duncan,

The way things were going to be set up, you would not be able to share DBs. Each framework would have a DB running alongside it, which, would not be accessible outside of the jail. Is DB sharing something we need to consider? If not, it would make jail/server management/security much easier as there would be no outward facing DB connections (potential security risk) and no chance of incompatibilities between the frameworks.

I have looked at KaOS but the documentation is very sparse, and the design seems a little backwards (you first create an "application" configuration, and then create VMs to run it, as opposed to creating a VM and then adding apps to it).

You did not go too deep. All concerns should be raised now while our chosen implementation is not yet set in stone.

Duncan, how soon could you have a FreeBSD VM set up? Should I create a dedicated server and give you all access, or is it easy enough for you to simply create a VM?

Alex, do you have any resources to offer?

Cheers, Alan

[Alexis Métaireau]

Le 10/18/2010 07:22 PM, Alan Smith a �crit :

> Alex, do you have any resources to offer?

About FreeBSD ? Well, the handbook, I guess:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/

[Alan Smith]

Alex,

I'm sorry, I wasn't very clear. I was referring to servers, etc.

[Alexis Métaireau]

Le 10/18/2010 07:27 PM, Alan Smith a �crit :

> Alex,
>
> I'm sorry, I wasn't very clear. I was referring to servers, etc.
>

Oh, okay, Unfortunately not. I can set up a jail on a dedicated server
for testing purposes, but I'm not sure this will help a lot.

Cheers

[Alan Smith]

Hi all,

I will be creating a FreeBSD server this afternoon unless I hear back from Duncan before then.

Cheers, Alan

[Duncan]

Alan,

Sorry to have been a bit quiet this week…

I have talked to Gregg about DBs and we'd prefer to have the DBs stored outside the jail. Should we need to migrate to a second DB server somewhere down the line it would make the migration a lot simpler, since we could just change the DB address. I don't believe this can be done easily with the DB in the jail. Aside from that I can't think of any major issues with the jail config; what I've seen discussed so far looks good aside from the DB thing.

Other than that, my understanding of jails and the discussion so far looks good.

We do have some users that use multiple frameworks on their sites, and I would like to be able to support them if possible. I realize this is rather a lofty goal and may not be something we can accomplish.

Alan, go ahead and create your virtual machine. My free time tends to fluctuate a lot and I haven't been able to get one up. My apologies for not getting it created.

Cheers,
Duncan

[Alexis Métaireau]

Hi Duncan, Alan,

Le 10/21/2010 08:44 PM, Duncan a écrit :
> Alan,
>
> Sorry to have been a bit quiet this week…
>
> I have talked to Gregg about DBs and we'd prefer to have the DBs stored
> outside the jail. Should we need to migrate to a second DB server somewhere
> down the line it would make the migration a lot simpler, since we could just
> change the DB address. I don't believe this can be done easily with the DB
> in the jail.

Why not ? I mean, it depends mainly on what the network configuration
is, no ? The jails can access the other jails over the network easily,
and I guess we can use NAT for more complex cases.

When you say "outside the jail", do you mean in another jail, or
directly on the host ? I'm strongly -1 on the second idea, cause it will
expose the whole system to the security fails DB could have.

Maybe I'm not understanding completely what you're saying, and you
simply want to put the DB aside from the jail, and in that case I'm in
favor of that (that's better to do things that way rather than having a
DB and the servers in the same jail).

BTW, maybe to increase our comprehension, we cane name the jails (eg.
the sites-jail, the db-jail, etc).

> Aside from that I can't think of any major issues with the jail
> config; what I've seen discussed so far looks good aside from the DB thing.
>
> Other than that, my understanding of jails and the discussion so far looks
> good.
>
> We do have some users that use multiple frameworks on their sites, and I
> would like to be able to support them if possible. I realize this is rather
> a lofty goal and may not be something we can accomplish.

Maybe can we do something like that.

I'm not sure to understand the whole system architecture (a scheme would
definitely help), but it's possible to do things using nullfs.

For instance, let's say we have one db-jail, and two frameworks jails (a
django-jail and a symfony-jail). We could create a file-jail, to store
the files that will be processed by the different frameworks jails.

To do so, we can mount, from the host, parts of the file-jail
filesystem, with read-only permission, on the framewok-jails. (it's way
better than the symbolic link I was talking last time).

I havnt do that yet, but it's used by the ez-jail system for instance,
and seems to work pretty well.

Cheers !
Alex

>> Click here to report this message as spam.<http://scanner.ambitiouslemon.com/cgi-bin/learn-msg.cgi?id=BB5CE27FFA.1B208>
>>
>