Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Server Building Help

0 views
Skip to first unread message

Suntide

unread,
Jan 10, 2012, 8:33:33 AM1/10/12
to
Hi,

I intend to build a Fedora or CentOS server.

Purpose: This will be essentially a file server for 350 odd users.

Authentication: Users will be accessing the data via Samba and external
users will log in via FTP.

Data Size: Approximate 800GB and increasing

Can someone please let me know the best configuration for SME level
type. I do not want to go for the Xeon/Opteron high end processors
although it can be considered, cost permitting. Also, I would like to
know the recommended size of the hard disks that would obliterate the
need to regularly delete or move the data elsewhere to create/maintain
free space.
As storing the backups on the same machine is not advisable, also please
let me know what other good option (free software) do I have to store
the backups elsewhere.

Thank you.


Dan C

unread,
Jan 10, 2012, 8:53:48 AM1/10/12
to
Does your teacher allow you to get homework answers from Usenet?



--
"Ubuntu" -- an African word, meaning "Slackware is too hard for me".
"Bother!" said Pooh, as he rolled off the prostitute.
Usenet Improvement Project: http://twovoyagers.com/improve-usenet.org/
Thanks, Obama: http://brandybuck.site40.net/pics/politica/thanks.jpg

Aragorn

unread,
Jan 10, 2012, 9:06:16 AM1/10/12
to
On Tuesday 10 January 2012 14:33, Suntide conveyed the following to
alt.os.linux...

> Hi,
>
> I intend to build a Fedora or CentOS server.

Forget Fedora. The word "server" infers stability, and stability and
Fedora are an oxymoron. Go with CentOS. ;-)

> Purpose: This will be essentially a file server for 350 odd users.
>
> Authentication: Users will be accessing the data via Samba and
> external users will log in via FTP.

Use vsftpd for FTP. ;-)

> Data Size: Approximate 800GB and increasing
>
> Can someone please let me know the best configuration for SME level
> type.

That depends on your budget and how important this server is for you.
We cannot make that decision for you.

> I do not want to go for the Xeon/Opteron high end processors
> although it can be considered, cost permitting.

Xeons and Opterons typically use ECC memory, which is more expensive,
but also more reliable. You also want to have a very decent UPS, _and_
a decent PSU. Capacitors wear over time, so the power rating of a PSU
drops over time. You want to have a PSU that delivers at least twice
the actual power requirements when the machine is new. That way, when
the PSU loses its power over time due to capacitor wear - which itself
is the result of a whole batch of faulty capacitors that have been sold
to many different large manufacturers, who are all still using them -
you will still have enough power left to drive your machine.

RAID is a must, preferably in hotswap bays. I would recommend RAID 10.
It's rather expensive in that you lose 50% of your storage capacity to
the redunancy factor, but it offers excellent redundancy - double
failure in some scenarios - and excellent performance.

Use a separate RAID array for the actual system, i.e. "/", "/boot",
"/usr", "/opt", "/var" and "/home". Use a tmpfs for "/tmp", mounted
with noexec. I recommend about 8 to 16 GB of RAM, and "/tmp" should be
no bigger than 30 MB for each user account on the system. Use a
separate RAID array for "/srv/ftp" or - in the event of CentOS and other
RedHat derivatives - "/var/ftp".

> Also, I would like to know the recommended size of the hard disks that
> would obliterate the need to regularly delete or move the data
> elsewhere to create/maintain free space.

Well, the size of the disks is less important here than the number of
disks you'd be putting in the RAID array for data. About six disks of 1
TB in size each should get you a long way. You'd have 3 TB of storage
with a RAID 10 configuration.

> As storing the backups on the same machine is not advisable, also
> please let me know what other good option (free software) do I have to
> store the backups elsewhere.

rsnapshot is one of the best free software backup solutions, which works
over the network. It's essentially a series of Bash scripts that use
rsync. You create an initial full backup, and the subsequent
incremental backups use hardlinks against the initial full backup for
files that haven't changed, while the files that have changed are simply
copied over. As such, the incremental backups themselves consume only
very little diskspace and bandwidth, and you always have access to an
easy full restore capacity.

--
= Aragorn =
(registered GNU/Linux user #223157)

Eef Hartman

unread,
Jan 10, 2012, 9:29:10 AM1/10/12
to
Aragorn <str...@telenet.be.invalid> wrote:
>> Authentication: Users will be accessing the data via Samba and
>> external users will log in via FTP.
>
> Use vsftpd for FTP. ;-)

Unless the external access is NOT over the Internet, I would go for
openSSH & sftp instead, ftp is just not secure enough - especially
for a file server.

>> Data Size: Approximate 800GB and increasing

Use two 2 TB disks in Raid-1 configuration, enough expansion space
and the extra security of a mirrored disk.
--
******************************************************************
** Eef Hartman, Delft University of Technology, dept. SSC/ICT **
** e-mail: E.J.M....@tudelft.nl - phone: +31-15-27 82525 **
******************************************************************

Aragorn

unread,
Jan 10, 2012, 9:49:43 AM1/10/12
to
On Tuesday 10 January 2012 15:29, Eef Hartman conveyed the following to
alt.os.linux...

> Aragorn <str...@telenet.be.invalid> wrote:
>
>> [Suntide wrote:]
>>
>>> Authentication: Users will be accessing the data via Samba and
>>> external users will log in via FTP.
>>
>> Use vsftpd for FTP. ;-)
>
> Unless the external access is NOT over the Internet, I would go for
> openSSH & sftp instead, ftp is just not secure enough - especially
> for a file server.

That's why I'm recommending vsftpd. The OP specified that it has to be
FTP, and of all FTP server daemons, vsftpd is probably the most secure
one.

sftp is better, but considering that the OP mentioned Samba, I gather
that his clients will be using Microsoft Windows, and according to my
Windows-using friends, Windows doesn't support sftp out-of-the-box. So
the clients would have to connect to the server via Putty or similar
software.

>>> Data Size: Approximate 800GB and increasing
>
> Use two 2 TB disks in Raid-1 configuration, enough expansion space
> and the extra security of a mirrored disk.

Yes, but RAID 1 only offers single failure redundancy and does not
increase writing speed, while carrying the same overhead in storage
space, i.e. only 50% total storage capacity. That's why RAID 10 would
be better in my humble opinion.

Java Jive

unread,
Jan 10, 2012, 9:57:26 AM1/10/12
to
Filezilla does SFTP on Windows, and does it pretty well ...

On Tue, 10 Jan 2012 15:49:43 +0100, Aragorn
<str...@telenet.be.invalid> wrote:
>
> sftp is better, but considering that the OP mentioned Samba, I gather
> that his clients will be using Microsoft Windows, and according to my
> Windows-using friends, Windows doesn't support sftp out-of-the-box. So
> the clients would have to connect to the server via Putty or similar
> software.
--
=========================================================
Please always reply to ng as the email in this post's
header does not exist. Or use a contact address at:
http://www.macfh.co.uk/JavaJive/JavaJive.html
http://www.macfh.co.uk/Macfarlane/Macfarlane.html

Eef Hartman

unread,
Jan 10, 2012, 10:16:28 AM1/10/12
to
Aragorn <str...@telenet.be.invalid> wrote:
> Windows doesn't support sftp out-of-the-box.

A product like WinSCP, which is freely available, supports both
the scp and sftp protocols. See winscp.net
OK, you _do_ have to install it separately, but it is the product
"our" Windows users use to access our linux servers.

> Yes, but RAID 1 only offers single failure redundancy and does not
> increase writing speed, while carrying the same overhead in storage
> space, i.e. only 50% total storage capacity. That's why RAID 10
> would be better in my humble opinion.

True, but RAID-10 needs a minimum of 4 disks and even when you use
1 TB ones that's still quite a bit more expensive then just two
2 TB ones. And you also need a more advanced RAID controller.
With 4 disks I would go for RAID-5 instead, still only single
disk failure redundancy, but more flexible (and more usable disk
space).

Aragorn

unread,
Jan 10, 2012, 10:39:20 AM1/10/12
to
[Top-post corrected]

On Tuesday 10 January 2012 15:57, Java Jive conveyed the following to
alt.os.linux...

> On Tue, 10 Jan 2012 15:49:43 +0100, Aragorn
> <str...@telenet.be.invalid> wrote:
>>
>> sftp is better, but considering that the OP mentioned Samba, I gather
>> that his clients will be using Microsoft Windows, and according to my
>> Windows-using friends, Windows doesn't support sftp out-of-the-box.
>> So the clients would have to connect to the server via Putty or
>> similar software.
>
> Filezilla does SFTP on Windows, and does it pretty well ...

But that's not "out-of-the-box", is it? It's a third-party application.
Even Firefox may support sftp - I don't know; I've never tried it.

The point is that one cannot demand from a Windows user that they would
install a third-party application for supporting a service offered by a
server somewhere. Because if you could pose such a demand, then you
could by the same token also demand that they'd use GNU/Linux to access
a network share on a GNU/Linux machine via NFS, rather than that they'd
use Samba.

Clients running Windows are picky, even if only because they generally
know no better than what they have in front of them, "because it came
with the computer" when they bought it.

Aragorn

unread,
Jan 10, 2012, 11:30:51 AM1/10/12
to
On Tuesday 10 January 2012 16:16, Eef Hartman conveyed the following to
alt.os.linux...

> Aragorn <str...@telenet.be.invalid> wrote:
>
>> Windows doesn't support sftp out-of-the-box.
>
> A product like WinSCP, which is freely available, supports both
> the scp and sftp protocols. See winscp.net
> OK, you _do_ have to install it separately, but it is the product
> "our" Windows users use to access our linux servers.

Well, sure, but I am going by the worst case scenario here. The OP
didn't specify what was possible, or who the clients are, or even
whether this is an internet server or an intranet server.
>
>> Yes, but RAID 1 only offers single failure redundancy and does not
>> increase writing speed, while carrying the same overhead in storage
>> space, i.e. only 50% total storage capacity. That's why RAID 10
>> would be better in my humble opinion.
>
> True, but RAID-10 needs a minimum of 4 disks and even when you use
> 1 TB ones that's still quite a bit more expensive then just two
> 2 TB ones. And you also need a more advanced RAID controller.
> With 4 disks I would go for RAID-5 instead, still only single
> disk failure redundancy, but more flexible (and more usable disk
> space).

I once was a fan of RAID 5 as well, until I found out about the infamous
RAID 5 write hole. RAID 10 is more costly in that it wastes more
diskspace, but it is safer and faster.

As for needing a more advanced RAID controller, that's not really true.
You don't even need a hardware RAID controller if you opt to make use of
Linux software RAID.

Personally I also prefer hardware RAID controllers - and those that
support RAID 5 generally also support RAID 10 - but if cost is an issue,
then software RAID is the better solution. Linux software RAID is
definitely highly reliable, and even supports hot or standby spares.

Your mileage may vary, and the OP must decide what they want. I'm just
throwing in my 5 Eurocents. ;-)

Java Jive

unread,
Jan 10, 2012, 12:27:36 PM1/10/12
to
Speaking as someone who used to write for a living scripts for
installing Windows software, Filezilla is comparatively easy to set
up. In fact, as all the site data is held in files, rather than the
registry, installation could be entirely automated very easily. It
could, for example, be done by a batch file run from a logon script.

On Tue, 10 Jan 2012 16:39:20 +0100, Aragorn
<str...@telenet.be.invalid> wrote:
>
> The point is that one cannot demand from a Windows user that they would
> install a third-party application for supporting a service offered by a
> server somewhere.

J.O. Aho

unread,
Jan 10, 2012, 3:04:13 PM1/10/12
to
Java Jive wrote:
On Tue, 10 Jan 2012 16:39:20 +0100, Aragorn
> <str...@telenet.be.invalid> wrote:

[Evil Jeopardy top-post corrected]

>>
>> The point is that one cannot demand from a Windows user that they would
>> install a third-party application for supporting a service offered by a
>> server somewhere.
>
> Speaking as someone who used to write for a living scripts for
> installing Windows software, Filezilla is comparatively easy to set
> up. In fact, as all the site data is held in files, rather than the
> registry, installation could be entirely automated very easily. It
> could, for example, be done by a batch file run from a logon script.
>

Still it's not out of the box and FileZilla may not be on the approved
software list for the company/organization and still microsoft users seems to
also have trouble with software they haven't used before.


--

//Aho

Java Jive

unread,
Jan 10, 2012, 3:32:33 PM1/10/12
to
On Tue, 10 Jan 2012 21:04:13 +0100, "J.O. Aho" <us...@example.net>
wrote:

> Java Jive wrote:
> On Tue, 10 Jan 2012 16:39:20 +0100, Aragorn
> > <str...@telenet.be.invalid> wrote:
>
> [Evil Jeopardy top-post corrected]

Oh! Grow up!

> Still it's not out of the box and FileZilla may not be on the approved
> software list for the company/organization

So? They'll test it and approve it if they need it.

> and still microsoft users seems to
> also have trouble with software they haven't used before.

No more or less than other users, such as Linux users.

It sounds to me as though some of the posters here have no idea of how
these things are done in a business environment. It's not the only
possible way of doing it, but I'll explain how it all happened at my
last place of work.

Either a 'line of business', that is one particular department
employing thousands of 'users', or the IT department itself, would
perceive a need for some additional software on the firm's PCs.

The Desktop Group in IT would get hold of the software, install it,
test it for compatibility with the firm's then current standard build,
and all being well, would write an installation script for it, and
instructions for installing it.

These would then be passed to an independent testing group who would
follow the instructions, install the software, and trial use, ensuring
that generally everything functions as the IT department has
described.

Then groups of specific end-users who were more IT savvy would test
the software for a time.

Then it would be released to anyone who had a need for it to download
it from a server, or else it would be added to the standard build.

The firms' 'standard' build actually came in several flavours to suit
different departments. IT would get just a standard build, Taxation
would have additional financial software related to UK taxation,
Insolvency would get a build geared towards insolvency practice, etc.
All departments had to use a wide range of software, many of which
were bespoke and could not have been encountered befoe joining the
firm. Employees were expected to be of graduate class and thoroughly
used to computers. If they hadn't been, they simply could not have
done the job.

So to say that ...

> microsoft users seems to
> also have trouble with software they haven't used before.

... is simply untrue, and sounds like flaming by an ignorant
pointy-headed Linux geek who needs to get out into the real world
more.

Jasen Betts

unread,
Jan 11, 2012, 3:05:35 AM1/11/12
to
On 2012-01-10, Suntide <sun...@example.com> wrote:
> Hi,
>
> I intend to build a Fedora or CentOS server.
>
> Purpose: This will be essentially a file server for 350 odd users.
>
> Authentication: Users will be accessing the data via Samba and external
> users will log in via FTP.

install a good FTP server, enforce ancryption.

> Data Size: Approximate 800GB and increasing

about 2G each user, same as gmail :)

350 users sharing a flat file database is completly different to 250
users doing spreadsheets and word processing or doing graphic design

> Can someone please let me know the best configuration for SME level
> type. I do not want to go for the Xeon/Opteron high end processors
> although it can be considered, cost permitting.

you probably don't need high end processors, for a fileserver you
mainly need a fast bus

you're probably going to want 10G ethernet too.

invest in a good server mainboard attach lots of cheap disk.

> Also, I would like to
> know the recommended size of the hard disks that would obliterate the
> need to regularly delete or move the data elsewhere to create/maintain
> free space.

it's called "big enough" (see also Moore's law)

> As storing the backups on the same machine is not advisable, also please
> let me know what other good option (free software) do I have to store
> the backups elsewhere.

rsync over the lan to a linux PC with external disk (esata/usb3 etc)

rotate the external disk to an off-site location. have atleast three sets
so that atleast one set is always off-site. consider using filesystem
encryption on these disks incase they go missing.

Remeber to keep a backup copy of the encryption key at a secure offsite
location

--
āš‚āšƒ 100% natural

--- Posted via news://freenews.netfront.net/ - Complaints to ne...@netfront.net ---

Jasen Betts

unread,
Jan 11, 2012, 3:16:32 AM1/11/12
to
On 2012-01-10, Eef Hartman <E.J.M....@tudelft.nl> wrote:
> Aragorn <str...@telenet.be.invalid> wrote:
>> Windows doesn't support sftp out-of-the-box.

>> Yes, but RAID 1 only offers single failure redundancy and does not
>> increase writing speed, while carrying the same overhead in storage
>> space, i.e. only 50% total storage capacity. That's why RAID 10
>> would be better in my humble opinion.
>
> True, but RAID-10 needs a minimum of 4 disks and even when you use
> 1 TB ones that's still quite a bit more expensive then just two
> 2 TB ones. And you also need a more advanced RAID controller.
> With 4 disks I would go for RAID-5 instead, still only single
> disk failure redundancy, but more flexible (and more usable disk
> space).

more load on the CPU,

raid-10 gives the speed boost of raid-0 and the redundancy of raid-1

raid-5 chews processors cycles or requires a more-expensive raid controller
and only give a 50% increase in storage space over raid-10

disk is cheap (the whole point ot raid is to use cheap disks - if
you're using $600 branded disks you're doing it wrong)

Vahis

unread,
Jan 11, 2012, 10:47:45 AM1/11/12
to
On 2012-01-11, Jasen Betts <ja...@xnet.co.nz> wrote:

> and only give a 50% increase in storage space over raid-10
>
> the whole point ot raid is to use cheap disks

I came to know about raid in the eighties, even sold a few
Sun Microsystem ones.

Those were the days, you still made money out of HW

R.A.I.D those days = Redundant Array of Inexepensive Disks

In Sun's case not so _very_ inexpensive though :)

Vahis
--
http://waxborg.servepics.com
openSUSE 11.4 (x86_64) 2.6.37.6-0.9-default main host
openSUSE 12.1 (x86_64) 3.2.0-2-desktop Tumbleweed in VirtualBox
openSUSE 12.1 (i586) 3.1.0-1.2-desktop in EeePC 900

Vahis

unread,
Jan 11, 2012, 10:49:12 AM1/11/12
to
On 2012-01-11, Jasen Betts <ja...@xnet.co.nz> wrote:

> the whole point ot raid is to use cheap disks

Eef Hartman

unread,
Jan 11, 2012, 10:52:51 AM1/11/12
to
Vahis <wax...@gmail.com.invalid> wrote:
> In Sun's case not so _very_ inexpensive though :)

Compared to some brands, still reasonably cheap:
when I first started with HP in the late 80's a disk for their
"multi-user mini's" was something like 33 000 guilders (about
15 000 Euro's), and that for "only" a 404 MB one.
That one was about the size of a washing machine, by the way, and
could take up to 9 Amps of current (14" disk, 11 platters, removable).

Aragorn

unread,
Jan 11, 2012, 10:57:38 AM1/11/12
to
On Wednesday 11 January 2012 16:47, Vahis conveyed the following to
alt.os.linux...

> On 2012-01-11, Jasen Betts <ja...@xnet.co.nz> wrote:
>
>> and only give a 50% increase in storage space over raid-10
>>
>> the whole point ot raid is to use cheap disks
>
> I came to know about raid in the eighties, even sold a few
> Sun Microsystem ones.
>
> Those were the days, you still made money out of HW
>
> R.A.I.D those days = Redundant Array of Inexepensive Disks
>
> In Sun's case not so _very_ inexpensive though :)

Considering that most of the old RAID adapters were specifically for
SCSI disks, the "inexpensive" was soon replaced in the mnemonic by the
word "independent". SCSI disks were (and still are) not exactly
inexpensive.

Fortunately, with the move from PATA to SATA for commodity hard disks
and even for a bunch of enterprise-grade hard disks today, it becomes
possible to use them in conjunction with SAS RAID controllers. I have
such a controller (from Adaptec) in one of my machines, and it hosts
four SAS disks and four (hotswappable) SATA disks.

The SAS disks are smaller in capacity and a lot more expensive, but
they're also a lot faster - they spin at 15'000 rpm. The SATAs are
slower (at 7200 rpm), but their capacity is almost an order of magnitude
of that of the SAS disks, and they're a lot more affordable. The
difference in speed between the two isn't exactly a 2:1 ratio, given
that the SATA disks have much higher data density.

Aragorn

unread,
Jan 11, 2012, 11:00:49 AM1/11/12
to
On Wednesday 11 January 2012 16:52, Eef Hartman conveyed the following
to alt.os.linux...

> Vahis <wax...@gmail.com.invalid> wrote:
>> In Sun's case not so _very_ inexpensive though :)
>
> Compared to some brands, still reasonably cheap:
> when I first started with HP in the late 80's a disk for their
> "multi-user mini's" was something like 33 000 guilders (about
> 15 000 Euro's), and that for "only" a 404 MB one.
> That one was about the size of a washing machine, by the way, and
> could take up to 9 Amps of current (14" disk, 11 platters, removable).

The two Unisys minis we had in college also had such hard disks. They
were in translucent "cartridges", with a carrying handle on top. ;-)

Vahis

unread,
Jan 11, 2012, 12:29:26 PM1/11/12
to
On 2012-01-11, Eef Hartman <E.J.M....@tudelft.nl> wrote:
> Vahis <wax...@gmail.com.invalid> wrote:
>> In Sun's case not so _very_ inexpensive though :)
>
> Compared to some brands, still reasonably cheap:
> when I first started with HP in the late 80's a disk for their
> "multi-user mini's" was something like 33 000 guilders (about
> 15 000 Euro's), and that for "only" a 404 MB one.
> That one was about the size of a washing machine, by the way, and
> could take up to 9 Amps of current (14" disk, 11 platters, removable).

Here's a disk drive, like I saw them first:
http://waxborg.servepics.com/wpg2?g2_itemId=4596

Eef Hartman

unread,
Jan 12, 2012, 9:26:59 AM1/12/12
to
Aragorn <str...@telenet.be.invalid> wrote:
> I once was a fan of RAID 5 as well, until I found out about the infamous
> RAID 5 write hole. RAID 10 is more costly in that it wastes more
> diskspace, but it is safer and faster.

At the cost of still more disks (6 minimum) you can go to RAID-50 (5 + 0,
so redundancy PLUS mirroring).

> Personally I also prefer hardware RAID controllers - and those that
> support RAID 5 generally also support RAID 10 - but if cost is an issue,
> then software RAID is the better solution. Linux software RAID is
> definitely highly reliable, and even supports hot or standby spares.
But - of course - cannot support more disks then the motherboard has
connectors for. If you have to buy an additional adaptor, you might
as well go for a hardware RAID one for i.e. 8 disks.

I originally came up with RAID-1 as compared to "a single disk", which
seemed to me what the OP was considering. At the cost of only one
extra disk RAID-1 _does_ contribute more reliability.

But - of course - you can go as high as you want in that

> Your mileage may vary, and the OP must decide what they want. I'm just
> throwing in my 5 Eurocents. ;-)

Which is what I did too ....

dx...@albury.nospam.net.au

unread,
Jun 20, 2012, 9:03:18 AM6/20/12
to
Vahis wrote:

<Snip>

> Here's a disk drive, like I saw them first:
> http://waxborg.servepics.com/wpg2?g2_itemId=4596
>
> Vahis
>

(I know this is an old thread, but just catching up!!)

The photo reminds me of a display at a Computer show I attended back in
approx 1993.

Had on display a 1GB Hard Drive (yes, 1GB). Had two armed guards with-in
arms distance, as well!!

Daniel
0 new messages