OpenBSD Benchmarked... results: poor!

97 views
Skip to first unread message

James Herbert

unread,
Oct 19, 2003, 7:19:31 AM10/19/03
to
Found this today and was intrigued.

http://bulk.fefe.de/scalability/

Wonder what you guys think about this?

J

James Herbert

unread,
Oct 19, 2003, 7:33:17 AM10/19/03
to

sorry to follow up my own post... but thought I should point out this
quote:

" OpenBSD 3.4 was a real stinker in these tests. The installation
routine sucks, the disk performance sucks, the kernel was unstable, and
in the network scalability department it was even outperformed by it's
father, NetBSD. OpenBSD also gets points deducted for the sabotage they
did to their IPv6 stack. If you are using OpenBSD, you should move away
now. "

J

James Herbert

unread,
Oct 19, 2003, 7:36:01 AM10/19/03
to
> Found this today and was intrigued.
>
> http://bulk.fefe.de/scalability/
>
> Wonder what you guys think about this?
>
> J

blah. already posted. my bad.

must remember to drink coffee prior to posting.

J

Claudio Jeker

unread,
Oct 19, 2003, 9:41:25 AM10/19/03
to
On Sun, Oct 19, 2003 at 12:29:15PM +0100, James Herbert wrote:
> > Found this today and was intrigued.
> >
> > http://bulk.fefe.de/scalability/
> >
> > Wonder what you guys think about this?
> >
> > J
>
> sorry to follow up my own post... but thought I should point out this
> quote:
>
> " OpenBSD 3.4 was a real stinker in these tests. The installation
> routine sucks, the disk performance sucks, the kernel was unstable, and
> in the network scalability department it was even outperformed by it's
> father, NetBSD. OpenBSD also gets points deducted for the sabotage they
> did to their IPv6 stack. If you are using OpenBSD, you should move away
> now. "
>

At least he shows his bias. So you know that you have to read it with this
in mind.

Anyway I had a closer look at one of his test -- the bind to port 0 one.

So why is OpenBSD O(N) while FreeBSD is O(1)?
The answer is simple. FreeBSD uses a hash-list to store in pcb strucures
while OpenBSD uses a cirqular queue (like NetBSD).
The function in_pcblookup has to circle through the complete list of pcbs
to find out if a port is in use or not. So for each bind with port 0 this
has to be done at least one time.

But why is OpenBSD even worst than O(N) (e.g as NetBSD)?
This answer is even simpler. OpenBSD uses random outgoing port. At the
begining using a random port should not slow down the bind but as we add
more and more binded sockets the chance to have a conflict (the new
random port is already used) raises and so suddenly we have to make more
then one in_pcblookup.

How to fix it:
Speed up in_pcblookup to a faster hash or tree based search function.
Probably some code of FreeBSD or from the pf state table could be used.

PS: I doubt that linux and FreeBSD are real O(1) implementations as
mentioned in the paper. They are just near O(1) for small numbers.

--
:wq Claudio

Adam

unread,
Oct 19, 2003, 10:20:49 AM10/19/03
to
On Sun, Oct 19, 2003 at 12:29:15PM +0100, James Herbert wrote:
> > Found this today and was intrigued.
> >
> > http://bulk.fefe.de/scalability/
> >
> > Wonder what you guys think about this?
> >
> > J
>
> sorry to follow up my own post... but thought I should point out this
> quote:
>
> " OpenBSD 3.4 was a real stinker in these tests. The installation
> routine sucks, the disk performance sucks, the kernel was unstable, and
> in the network scalability department it was even outperformed by it's
> father, NetBSD. OpenBSD also gets points deducted for the sabotage they
> did to their IPv6 stack. If you are using OpenBSD, you should move away
> now. "
>
> J

Is that bias though? Which part of that isn't true? OpenBSD doesn't
perform well, and does crash when you try to use it for anything big.

Adam

Dom De Vitto

unread,
Oct 19, 2003, 12:00:14 PM10/19/03
to
Just for the archives:

"When my implementation worked on Linux, I decided to port the API
to BSD as well"

So, he writes the benchmark on Linux, then PORTS it to BSD ?

I'm sure if he wrote it on VAX/VMS and ported it to Sun E15k, the
VAX would seem to fly too.....

It's clear from his language that he's proud of "his" Linux (Leanux),
and wasn't looking for objective tests, or data that went against
his starting premise, "Linux is best".

Lack of any details of hardware used makes his results pointless,
and his inexperience and inability to INSTALL OpenBSD
shows that he clearly isn't qualified to set up any kind of
scalable server using OpenBSD, let alone test one.

All of his problems are either:
a) In the install notes of FAQ.
b) Fly in the face of other users experiences.
c) Coloured by his Linux-only viewpoint.

If he looked past his nose, he's see that the *real* answers would be
things like "OpenBSD doesn't support SMP", which are valid and, for
his "scalable network programming research" goal quite obvious.

If I was his tutor I'm strongly suggest he be sent back to do more
reading and less coding, and certainly he should document, as the
FIRST RULE OF SCIENCE, the test conditions. Quite obviously
nobody can replicate his tests because he doesn't state when he
took -current, and obviously this could have been when Theo put
in some experimental stuff - who knows?

Dom
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dom De Vitto Tel. 07855 805 271
http://www.devitto.com mailto:d...@devitto.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Adam

http://members.shaw.ca/infonick/slashdot/scalability.htm
http://members.shaw.ca/infonick/slashdot/scalability.htm
http://members.shaw.ca/infonick/slashdot/scalability.htm
http://members.shaw.ca/infonick/slashdot/scalability.htm
http://members.shaw.ca/infonick/slashdot/scalability.htm
http://members.shaw.ca/infonick/slashdot/scalability.htm
http://members.shaw.ca/infonick/slashdot/scalability.htm
http://members.shaw.ca/infonick/slashdot/scalability.htm

Benchmarking BSD and Linux
Benchmarking BSD and Linux
Benchmarking BSD and Linux
Benchmarking BSD and Linux
Benchmarking BSD and Linux
Benchmarking BSD and Linux
Benchmarking BSD and Linux
Benchmarking BSD and Linux

Claus Assmann

unread,
Oct 19, 2003, 12:14:37 PM10/19/03
to
Instead of whining about "the guy is biased", "his benchmarks suck",
please run his (or others) benchmarks and publish the numbers.

At least this guy did some actual work and backed it up with numbers.

Henning Brauer

unread,
Oct 19, 2003, 12:25:58 PM10/19/03
to
On Sun, Oct 19, 2003 at 10:20:11AM -0400, Adam wrote:
> Is that bias though? Which part of that isn't true? OpenBSD doesn't
> perform well, and does crash when you try to use it for anything big.

oh yeah, right, that is what I see here daily on my ~70 OpenBSD
machines of which quite some heave some quite heavy load.

wait, something's wrong.

--
http://2suck.net/hhwl.html - http://www.bsws.de/
Unix is very simple, but it takes a genius to understand the simplicity.
(Dennis Ritchie)

Marc Matteo

unread,
Oct 19, 2003, 12:44:25 PM10/19/03
to

Much like when that MS sponsored company ran "tests" between a tweaked
Windows box and an out-of-the-box Linux box running Samba with Windows
crushing the Linux box, most OpenBSD developers realize while the fefe
site is bogus... but worth examining to some extent anyway.

But who can't figure out IPv6 on an OpenBSD box?!?

Marc

David Benfell

unread,
Oct 19, 2003, 12:46:29 PM10/19/03
to
On Sun, 19 Oct 2003 09:14:22 -0700, Claus Assmann wrote:
> Instead of whining about "the guy is biased", "his benchmarks suck",
> please run his (or others) benchmarks and publish the numbers.
>
> At least this guy did some actual work and backed it up with numbers.
>
Both sides are a little unfair.

For the results to be accepted on a scientific basis, the exact test
methodology should be documented. However, this could result in
"vendors" tuning their operating systems to perform well on the
benchmarks. Therefore, those who publish benchmark results are
reluctant to describe exact methodology.

Nonetheless, some of the criticisms of this particular study seem
reasonable.

Writing a benchmark for one operating system, then porting it to
another certainly catches one's eye. Ideally, the benchmark should be
written such that the only "porting" required would be a re-compile --
which introduces issues of compiler versions.

Benchmarks generally have fully earned discredit for these and other
flaws. The biggest problem seems to be that to do it properly
requires a bunch of money. And the only people who are sufficiently
interested in the results to actually pay for them are the "vendors",
which, once again, introduces, at minimum, a suspicion of bias.

--
David Benfell, LCP
ben...@parts-unknown.org
---
Resume available at http://www.parts-unknown.org/resume.html

[demime 0.98d removed an attachment of type application/pgp-signature]

Nick Holland

unread,
Oct 19, 2003, 12:50:08 PM10/19/03
to
James Herbert wrote:
>
> > Found this today and was intrigued.
> >
> > http://bulk.fefe.de/scalability/
> >
> > Wonder what you guys think about this?
> >
> > J
>
> sorry to follow up my own post... but thought I should point out this
> quote:
>
> " OpenBSD 3.4 was a real stinker in these tests.

Gee, that's objective.

> The installation routine sucks,

Well, having gone the "other direction" recently, attempting a Linux
install after many years of OpenBSD work, I gotta say this statement
alone discredits his report.

> the disk performance sucks,

He indicated he ran the OpenBSD tests on a different part of his
laptop's drive than the other systems. Draw your own conclusions
about its direct validity.

> the kernel was unstable,

oh, yeah, my OpenBSD boxes fall over all the time out in the field
</sarcasm>.

> and in the network scalability department it was even outperformed by it's
> father, NetBSD.

oh, you mean, OpenBSD isn't just an ego trip of Theo de Raadt? NetBSD
plus a few patches and changes of the copyright notices? You mean,
there are real differences between the projects? Wow. Who wouldda
thunk?

> OpenBSD also gets points deducted for the sabotage they
> did to their IPv6 stack.

Wow. We do things differently from Linux! How dare we! We are
clearly incompetent! We've gone from being a bad implementation of
NetBSD to being a bad Linux implementation! Will his praise of
OpenBSD never end? 8)

All things considered... when it comes to IPv6, if Itojun@ says "do
it this way", I'm inclined to believe him. His credentials are pretty
darned good here.

> If you are using OpenBSD, you should move away
> now. "

It does appear this was the starting premise, not the conclusion of
the "research".

Simply by pointing at bugtraq or similar, or even using personal
experience, I could come up with the exact opposite conclusion: "If
you are using Linux, you should have left years ago"


It all boils down to what you consider important. Personally, I find
the Micr..er..Linux design philosophy of "Add features then make it
'secure' later [i.e., when someone punches a hole in it]" a
non-starter for me. The Linux "Thrash from one design idea to
another" concept is also a non-starter for me. But the free world is
about choice. Felix von Leitner is free to chose his priorites, I am
free to chose mine, you chose yours. If absolute raw performance is
your goal, OpenBSD is probably not your ideal platform. However, the
number of people for whom OpenBSD can't perform sufficiently for is
pretty small, and they probably have enough money to hire people to
keep their systems patched...

I know many people look at a task and say, "What is the best tool to
accomplish this task". I add an extra question: "Should the task even
be considered reasonable with existing tools?" I.e., just because
there are five bad tools to choose from, doesn't mean I'm going to
pick the least bad -- I am willing to say "No" to the entire project.
Many people come to OpenBSD after having fought with Linux. I gave up
on Linux being a tool of value to my clients quite some time BEFORE I
found OpenBSD.

Gee, I gave this more time than it was worth...

Nick.
--
http://www.holland-consulting.net

Marco Feenstra

unread,
Oct 19, 2003, 1:12:27 PM10/19/03
to
Nick Holland wrote:

>
>>the disk performance sucks,
>>
>>
>
>He indicated he ran the OpenBSD tests on a different part of his
>laptop's drive than the other systems. Draw your own conclusions
>about its direct validity.
>
>

I would go one step further since he mentioned he had to copy the log
files over from the filesystem after each benchmark.
So either he is running on an extremely low diskspace filesystem, which
normally means an performance wise severly degraded filesystem.
And / or he is creating lots of logging data, coupled with the low disk
performance statement could explain lower test scores.

Eitherway this benchmark of small selective subsystems of an Operating
System does not really reflect the performance characteristics of said
Operating System for any actual purpose.

Marco Feenstra.

Rico -mc- Gloeckner

unread,
Oct 19, 2003, 1:57:35 PM10/19/03
to
On Sun, Oct 19, 2003 at 05:02:28PM +0100, Dom De Vitto wrote:
> Lack of any details of hardware used makes his results pointless,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I suggest you go back reading. atleast the Speed of the CPU is
documented. You just have to open your eyes.


> and his inexperience and inability to INSTALL OpenBSD
> shows that he clearly isn't qualified to set up any kind of
> scalable server using OpenBSD, let alone test one.
>
> All of his problems are either:
> a) In the install notes of FAQ.
> b) Fly in the face of other users experiences.
> c) Coloured by his Linux-only viewpoint.
>
> If he looked past his nose, he's see that the *real* answers would be
> things like "OpenBSD doesn't support SMP", which are valid and, for
> his "scalable network programming research" goal quite obvious.


If any of the whiners on this list would look past his nose, he would
find that fefe actually open-sourced all his benchmark stuff. If any of
the whiners on this list would look further, they would use the source
(Luke!) to actually use the benchmarks on his/her own box, sending the
results or -rather as fefe wants- a link of his own results to fefe.

(Mind you, fefe explicitely asked for benchmarks done by others.)

Now, if any of the whiners would go even one step further, they would
optimize the code and document the Optimizations, thus helping any
application-coder out there to bring an optimum of perfomance on their
openbsd ports of their application.


Oh, wait. The whiners on this list are not here to make the
world better, instead they are here to whine about being treated
unfairly.

Doh, i see, the situation is helpless. Excuse my lame suggestions and
go back whining.


have a nice day,
(an ex-openbsd user, who got tired by exactly these lame
attitudes showed by several people in this thread)
--
| Rico -mc- Gloeckner | mv ~/.signature `finger m...@ukeer.de` |
| gpg/mail: 3D67 D42F 2D50 4B68 1D62 E999 EFCB CDFF 61F0 5B8C |
| gpg/jabber: 7A1C D378 CAB7 C545 5E5D CC44 C44F 9657 3D9C 8389 |

Raymond Morsman

unread,
Oct 19, 2003, 2:03:08 PM10/19/03
to
Op zo 19-10-2003, om 19:56 schreef Rico -mc- Gloeckner:

> On Sun, Oct 19, 2003 at 05:02:28PM +0100, Dom De Vitto wrote:
> > Lack of any details of hardware used makes his results pointless,
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> I suggest you go back reading. atleast the Speed of the CPU is
> documented. You just have to open your eyes.

Still doesn't say very much. It was a notebook.

Raymond.

Adam

unread,
Oct 19, 2003, 2:26:34 PM10/19/03
to
On Sun, Oct 19, 2003 at 06:25:48PM +0200, Henning Brauer wrote:
> On Sun, Oct 19, 2003 at 10:20:11AM -0400, Adam wrote:
> > Is that bias though? Which part of that isn't true? OpenBSD doesn't
> > perform well, and does crash when you try to use it for anything big.
>
> oh yeah, right, that is what I see here daily on my ~70 OpenBSD
> machines of which quite some heave some quite heavy load.
>
> wait, something's wrong.

That's fantastic for you. It doesn't help all the people who's OpenBSD
servers crash under load. If I don't recompile a custom kernel, I get
panics from lack of resources, if I do compile a custom kernel, the new
panics I get don't count because I am running a custom kernel.

panic: uvm_mapent_alloc: out of static map entries, check MAX_KMAPENT
panic: pipespace: out of kvm
panic: malloc: out of space in kmem_map
panic: smashed stack in cpu_coredump

That's just what I pulled from core dumps sitting around at this moment.
I don't get those with freebsd or linux, doing more work.

So, while I am sure an openbsd developer can manage to get a machine to
stay up under load, normal users get kernel panics trying to use openbsd
for any real work.

Adam

Alexander Yurchenko

unread,
Oct 19, 2003, 2:36:45 PM10/19/03
to
On Sun, Oct 19, 2003 at 02:25:43PM -0400, Adam wrote:
> panic: smashed stack in cpu_coredump

JFYI, it was fixed...

>
> Adam

--
Alexander Yurchenko (aka grange)

Henning Brauer

unread,
Oct 19, 2003, 2:42:49 PM10/19/03
to
On Sun, Oct 19, 2003 at 02:25:43PM -0400, Adam wrote:
> On Sun, Oct 19, 2003 at 06:25:48PM +0200, Henning Brauer wrote:
> > On Sun, Oct 19, 2003 at 10:20:11AM -0400, Adam wrote:
> > > Is that bias though? Which part of that isn't true? OpenBSD doesn't
> > > perform well, and does crash when you try to use it for anything big.
> >
> > oh yeah, right, that is what I see here daily on my ~70 OpenBSD
> > machines of which quite some heave some quite heavy load.
> >
> > wait, something's wrong.
>
> That's fantastic for you. It doesn't help all the people who's OpenBSD
> servers crash under load. If I don't recompile a custom kernel, I get
> panics from lack of resources, if I do compile a custom kernel, the new
> panics I get don't count because I am running a custom kernel.
>
> panic: uvm_mapent_alloc: out of static map entries, check MAX_KMAPENT
> panic: pipespace: out of kvm
> panic: malloc: out of space in kmem_map
> panic: smashed stack in cpu_coredump
>
> That's just what I pulled from core dumps sitting around at this moment.
> I don't get those with freebsd or linux, doing more work.
> So, while I am sure an openbsd developer can manage to get a machine to
> stay up under load, normal users get kernel panics trying to use openbsd
> for any real work.

the right way to deal with that is analyze why this happens and see
what is at fault - either the application, or unsuitable defaults in
OpenBSD, or some real bug.
That's what I did years back when I had some trouble with my stuff on
OpenBSD. guess what? nowadays most of my boxes don't have any special
tuning - most could run GENERIC, and many do.
the exceptions are my bgp boxes. they need increased NMBCLUSTERS.
quite acceptable, a kernel routing table with over 100000 entries is
not exactly a normal case.

Mike Shaw

unread,
Oct 19, 2003, 2:47:15 PM10/19/03
to
> Oh, wait. The whiners on this list are not here to make the
>world better, instead they are here to whine about being treated
>unfairly.

Good grief...Not content with just dominating market share and converting
Linux users, now OpenBSD is out to make the world better.

I think most people who know enough to work on these benchmarks are too
busy CODING the OS to spend time on stupid "Mye OS R Teh RawK!" grandstanding
tricks like this. Frankly, I stopped reading when the dude said they
used "-current". He doesn't have a clue, much less the skill to create
benchmarks that mean anything to anyone.

I didn't choose OpenBSD because of stupid exothermic 'net debates. I
chose it because it suits my needs far better than other OS'. Everyone
on this list should understand that.

So move along...nothing to see here folks...just another trolling penguin...please
disburse.

-Mike

Dom De Vitto

unread,
Oct 19, 2003, 2:55:02 PM10/19/03
to
Exactly.

So fefe's results say that if our using a notebook to run your
high-intensity distributed network system OpenBSD isn't so good.

Surprising.

The lack of hardware variation alone should prevent any firm
deductions.

Maybe there is something about this notebook which OpenBSD
3.4-current-of-somewhen-mid-release or maybe it's easier to use words
like "stinks" than actually do proper tests, or even INVITE tests,
publish results and then make deductions.

The bottom line is that the whole testing methodology is bent,
and consequently his published "findings" are too.

Theo could easily rewrite OpenBSD to thrash these other OSes,
real things like multiprocessor support are a real drag for them,
so OpenBSD could be heaps faster. But who cares how many binds/second
can be done, this isn't real "work", so what does it prove?

I'm done the archives & google should hold my comments for as long
as is needed.

Dom
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Dom De Vitto Tel. 07855 805 271
http://www.devitto.com mailto:d...@devitto.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-----Original Message-----
From: owner...@openbsd.org [mailto:owner...@openbsd.org] On Behalf Of
Rico -mc- Gloeckner
Sent: Sunday, October 19, 2003 6:57 PM
To: Dom De Vitto
Cc: mi...@openbsd.org
Subject: Re: OpenBSD Benchmarked... results: poor!

On Sun, Oct 19, 2003 at 05:02:28PM +0100, Dom De Vitto wrote:
> Lack of any details of hardware used makes his results pointless,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I suggest you go back reading. atleast the Speed of the CPU is documented.
You just have to open your eyes.

> and his inexperience and inability to INSTALL OpenBSD shows that he
> clearly isn't qualified to set up any kind of scalable server using
> OpenBSD, let alone test one.
>
> All of his problems are either:
> a) In the install notes of FAQ.
> b) Fly in the face of other users experiences.
> c) Coloured by his Linux-only viewpoint.
>
> If he looked past his nose, he's see that the *real* answers would be
> things like "OpenBSD doesn't support SMP", which are valid and, for
> his "scalable network programming research" goal quite obvious.


If any of the whiners on this list would look past his nose, he would find
that fefe actually open-sourced all his benchmark stuff. If any of the
whiners on this list would look further, they would use the source
(Luke!) to actually use the benchmarks on his/her own box, sending the
results or -rather as fefe wants- a link of his own results to fefe.

(Mind you, fefe explicitely asked for benchmarks done by others.)

Now, if any of the whiners would go even one step further, they would
optimize the code and document the Optimizations, thus helping any
application-coder out there to bring an optimum of perfomance on their
openbsd ports of their application.

Oh, wait. The whiners on this list are not here to make the world better,
instead they are here to whine about being treated unfairly.

Doh, i see, the situation is helpless. Excuse my lame suggestions and go
back whining.


have a nice day,
(an ex-openbsd user, who got tired by exactly these lame
attitudes showed by several people in this thread)
--
| Rico -mc- Gloeckner | mv ~/.signature `finger m...@ukeer.de` |
| gpg/mail: 3D67 D42F 2D50 4B68 1D62 E999 EFCB CDFF 61F0 5B8C |
| gpg/jabber: 7A1C D378 CAB7 C545 5E5D CC44 C44F 9657 3D9C 8389 |

Rico -mc- Gloeckner

unread,
Oct 19, 2003, 3:02:11 PM10/19/03
to
On Sun, Oct 19, 2003 at 11:43:07AM -0700, Mike Shaw wrote:
> So move along...nothing to see here folks...just another trolling
> penguin...please disburse.

Quite frankly, i do not care what your goals to use obsd are.

Quite frankly, the lame attitude showed by some people here do not only
suck, they suck complete universes through nanotubes.


Yes, i understand that bashing people who are contributing benchmarks
with an open invitation to contribute to the benchmarks with optimized
machines is easier than to follow that invitation and showing the world
that the benchmark done originally didnt show the full perfomance.

Quite frankly, you are the troll.

Adam

unread,
Oct 19, 2003, 3:18:51 PM10/19/03
to

happens consistantly with a busy squid server. What am I doing wrong
there? Does openbsd just not support squid?

Its just like I said, maybe you can analyze why it happens, but most
people can't. How am I supposed to figure out why malloc ran out of
space in kmem_map? That you figure kernel debugging should be a normal
part of running a busy server is the problem.

Adam

julien mabillard

unread,
Oct 19, 2003, 3:53:41 PM10/19/03
to
well,
i don't like these kind of benchmarks because they just
show you what the author wanted to.

i can agree about some peformance results for some
specific tests.

but, this is absolutely not enough as a scientific approach
for an operating system choice.

who would choose hardware only on its cpu frequency?

OpenBSD is really more than processes performance
against Linux. and if people don't see why, they haven't looked
at what OpenBSD is.

cheers.

--
Key fingerprint = C549 46E1 1B75 116E 3321 BC0A E502 9457 319E B340

Marc Balmer

unread,
Oct 19, 2003, 4:01:50 PM10/19/03
to
> Its just like I said, maybe you can analyze why it happens, but most
> people can't. How am I supposed to figure out why malloc ran out of
> space in kmem_map? That you figure kernel debugging should be a normal
> part of running a busy server is the problem.

Windows was created to keep the idiots from Unix away. So stick with it.

Elliott Liggett

unread,
Oct 19, 2003, 4:09:18 PM10/19/03
to
Well, I have a spare desktop, and a 30 gig drive. I'm willing to do
whatever benchmarks you guys would like, and I'll even wipe and install
linux and run them as well (and freebsd too, of course). I've never
installed NetBSD before, but I'll try it.

A few notes:

1) The OpenBSD installer is awesome. But, coming from linux a few years
ago, I hate it. Its just a 180 of what you expect. Recently I installed
debian for a friend, and while debian is by far the nicest linux you'll
find anywhere, the installer left me in the dark a few times. So, I
think its just whatever you're used to.

2) On kernel panics, I've never seen my OpenBSD panic. Ever. But, when
I tried a desktop OpenBSD install, I started seeing panics right and
left. The server is a 486/100, and the desktop was a celeron 400. Go
figure. I guess the 486 is bound to have better support simply due to
how long it has been on the market. Plus, celeron's suck. But thats
another story :)

3) The best thing we can do with these benchmarks, is to a) run them
ourselves on our own hardware, with various BSDs and Linux. (just to
see how accurate it is on other hardware), and b) we should try and
more forward. The OpenBSD benchmarks didn't display ANY scalability.
Granted, nobody needs scalability on a laptop. Duh. But it should show
more than it did. It would be silly to sit here and honestly think that
OpenBSD is a clear winner, until you perform the same tests.

The code is available, and I'm going to see if I can simply compile it
on a few boxes of mine. Note I did not say 'port'.

One thing which was not benchmarked was security. Or ease of use. Or
documentation. Or install footprint. So, don't think too much of it.
Benchmarks are always purpose-specific.

Anyway, like I said, I'll run them on my boxes, and we'll see. Unless
the code confuses the crap out of me :D

--Elliott

> Its just like I said, maybe you can analyze why it happens, but most
> people can't. How am I supposed to figure out why malloc ran out of
> space in kmem_map? That you figure kernel debugging should be a normal
> part of running a busy server is the problem.
>

> Adam

Adam

unread,
Oct 19, 2003, 4:12:59 PM10/19/03
to
On Sun, Oct 19, 2003 at 09:59:14PM +0200, Marc Balmer wrote:
> > Its just like I said, maybe you can analyze why it happens, but most
> > people can't. How am I supposed to figure out why malloc ran out of
> > space in kmem_map? That you figure kernel debugging should be a normal
> > part of running a busy server is the problem.
>
> Windows was created to keep the idiots from Unix away. So stick with it.

That's a good attitude. Clearly not being an openbsd kernel developer
makes one an idiot. Its not a unix problem, like I said, freebsd and
linux both have no problems with these exact same tasks. Its just
openbsd that poor idiots like me can't figure out.

Adam

Ted Unangst

unread,
Oct 19, 2003, 4:18:23 PM10/19/03
to
On Sun, 19 Oct 2003, Rico -mc- Gloeckner wrote:

> Now, if any of the whiners would go even one step further, they would
> optimize the code and document the Optimizations, thus helping any
> application-coder out there to bring an optimum of perfomance on their
> openbsd ports of their application.

if you are writing a web server to run on openbsd, i suggest that instead
of mapping every other page of each file, unmapping it, and mapping it
again, you map it one time in a contiguous region and leave it there. i
think that is a not too difficult optimization that will lead to better
performance.


--
we don't run washington and no one really does

Elliott Liggett

unread,
Oct 19, 2003, 4:25:44 PM10/19/03
to
Heh Mark Balmer, thats downright mean! Are you related to Steve at all?

Thats a great marketing strategy for windows though - present unix as
something impossible for all but the elite kernel hackers.

Anyway, thats the sort of flame that will just keep people away from
unix and unix-like systems.

And, keep that web site of yours happily running on Linux.

Thanks, and keep up the good work.

Andreas Krennmair

unread,
Oct 19, 2003, 4:27:22 PM10/19/03
to
* Raymond Morsman <ray...@dyn.org> [gmane.os.openbsd.misc]:

> Op zo 19-10-2003, om 19:56 schreef Rico -mc- Gloeckner:
> > On Sun, Oct 19, 2003 at 05:02:28PM +0100, Dom De Vitto wrote:
> > > Lack of any details of hardware used makes his results pointless,
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > I suggest you go back reading. atleast the Speed of the CPU is
> > documented. You just have to open your eyes.
>
> Still doesn't say very much. It was a notebook.

So what? Do you really think that it was just the notebook that made
OpenBSD perform so much worse than the other operating systems? Do you
really think that in a Dual Xeon machine (oops!) OpenBSD would perform
the bind part of Fefe's benchmark with a better complexity than the
observed O(n)?

What really makes me laugh are all these whining fanboys here on the
list. Hey, some obscure and mostly uninteresting Java benchmark done by
IBM showed problems with the Linux scheduler. And what did the Linux
kernel hackers do? They didn't whine, but instead they simply improved
the scheduler.

ak
--
> Write Secure Code: a summary: Don't use system(), and be careful with
> buffers.
Brain Surgery: a summary: open skull, and poke around with a stick.
-- seen on slashdot.org

Alejandro G. Belluscio

unread,
Oct 19, 2003, 4:30:23 PM10/19/03
to
Sunday, October 19, 2003, 8:17:23 AM, you wrote:
James> Found this today and was intrigued.
James> http://bulk.fefe.de/scalability/
James> Wonder what you guys think about this?
I've emailed Felix and he told me he was sorry that he had used
CURRENT. He had thought it was STABLE. That he'll test it with RELEASE
as soon as he gets his CD. So I think that if you write to him
politely, he's a reasonable man. At least as reasonable as someone who
actually enjoys dealing with Dan Berenstein's licenses.

--
Best regards,
Alejandro Belluscio

Tobias Weingartner

unread,
Oct 19, 2003, 4:50:47 PM10/19/03
to
On Sunday, October 19, "James Herbert" wrote:
>
> sorry to follow up my own post... but thought I should point out this
> quote:
>
> " OpenBSD 3.4 was a real stinker in these tests. The installation
> routine sucks, the disk performance sucks, the kernel was unstable, and

> in the network scalability department it was even outperformed by it's
> father, NetBSD. OpenBSD also gets points deducted for the sabotage they
> did to their IPv6 stack. If you are using OpenBSD, you should move away
> now. "

Quite the quote. Now, where is your patch to make this better? :-)

--Toby.

Raymond Morsman

unread,
Oct 19, 2003, 6:00:46 PM10/19/03
to
Op zo 19-10-2003, om 20:25 schreef Adam:


> That's fantastic for you. It doesn't help all the people who's OpenBSD
> servers crash under load. If I don't recompile a custom kernel, I get
> panics from lack of resources, if I do compile a custom kernel, the new
> panics I get don't count because I am running a custom kernel.

Oh yeah...

It's troll season, I forgot.

A friend of me told me that irritation and frustration are energyforms.

There's a lot of energy for the losers...

Raymond.

Raymond Morsman

unread,
Oct 19, 2003, 6:13:45 PM10/19/03