[gentoo-user] Gentoo speed comparison to other distros

[Dale]

Howdy,

I was wondering. Has anyone ever seen where a test as been done to
compare the speed of Gentoo with other distros? Maybe Gentoo compared
to Redhat, Mandrake, Ubuntu and such?

Also, I read that Nasdaq runs a modified version of Gentoo. Do any
other large corps run it that we know of?

I googled a bit but couldn't find anything. Maybe my search terms
wasn't good enough.

Links would be nice.

Dale

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

[Rafa Griman]

Hi !!

On Thu, Mar 14, 2013 at 9:15 AM, Dale <rdale...@gmail.com> wrote:
> Howdy,
>
> I was wondering. Has anyone ever seen where a test as been done to
> compare the speed of Gentoo with other distros? Maybe Gentoo compared
> to Redhat, Mandrake, Ubuntu and such?

First things first ... What do you mean by "speed". Benchmarking is a
very complicated job ;) Do you mean boot time, network bandwidth, HDD
bandwidth, number crunching, graphics, ...? What application(s)? What
data volume? ...

MHO: never trust benchmarks unless you do them and you know what you
are doing ;) Even then ... be careful ;)

If you decide to run some benchmarks, take into account that Gentoo
has so many USE flags ... youo might not use one of those flags ...
but the other distros do use them. Same applies to compiler flags so
... would it be a fair comparison ? ;)

Last, but not least ... Imagine Gentoo is "faster" ... would compile
time be worth it? IOW: installing a precompiled distro (like RHEL,
SLES, ...) can about 30 - 60 minutes. Gentoo can take 24 hours (or
more ... or less, depending on what you install, your experience,
...). Now imagine speed up is 0.1% ... is it worth it?

> Also, I read that Nasdaq runs a modified version of Gentoo. Do any
> other large corps run it that we know of?

Sorry, can't be of any help here :(

> I googled a bit but couldn't find anything. Maybe my search terms
> wasn't good enough.
>
> Links would be nice.

MHO

Rafa

[Neil Bothwick]

On Thu, 14 Mar 2013 03:15:36 -0500, Dale wrote:

> I was wondering. Has anyone ever seen where a test as been done to
> compare the speed of Gentoo with other distros? Maybe Gentoo compared
> to Redhat, Mandrake, Ubuntu and such?

Mandrake? Where have you been for the last ten years, Dale? ;)


--
Neil Bothwick

... "I just forgot to increment the counter," Tom said, nonplussed.

[Dale]

Neil Bothwick wrote:
> On Thu, 14 Mar 2013 03:15:36 -0500, Dale wrote:
>
>> I was wondering.  Has anyone ever seen where a test as been done to
>> compare the speed of Gentoo with other distros?  Maybe Gentoo compared
>> to Redhat, Mandrake, Ubuntu and such?
>
> Mandrake? Where have you been for the last ten years, Dale? ;)
>
>

Sorry, it was called Mandrake when I used it last.  It's Mandriva now.  Odd, it was about 10 years ago that I switched to Gentoo from Mandrake.  That 9.1 to 9.2 upgrade was awful.

Dale

:-)  :-)

[William Kenworthy]

Did this few years back for an online magazine sponsored by a local
linux sysadmin company who wanted to see the difference between generic
debian and optimised (not necessarily gentoo, but thats what I used.)

Difference in times was ~10% across the board for graphics manipulations
(gimp scripts), spreadsheet tasks (gnumeric) and the like.

The "kicker" - simple optimisations gained far, far more than generic
compiler settings. e.g., initially, the gnumeric versions were slightly
different, with some wild times across the tasks. Make em the same
version (and cuedos to the gnumeric maintainer for jumping in and
helping diagnose/fix the problem - newer version on gentoo was heaps
slower :) and there was little difference.

Shared libs like glibc didnt make a huge difference, but being smart
about how/what a "particular" task was handled gained more. If a debian
app was compiled with similar options as to gentoo, little difference
between them in performance which considering shared libs etc wasn't
what I expected.

The intel compilers are/were said to be a lot better than gcc, not sure
if the gap is still there (supposedly 20% better again)

Its how long is a piece of string kind of question if considered OS
wide, but pick a narrow task and optimise away with smart programmers
and you will do well on almost anything.

Big advantage of gentoo - configurability, version control (what version
is installed and changing it at short notice) and general flexibility.

10 % on software is a lot better than forking out $$$ on faster hardware
to do that (as gamers do!), but at the end of the day, I can also make
my car go faster by painting the diff red (urban myth/joke from my
hotrodding days:) and see roughly the same performance boost - i.e.,
probably wont notice it in real life)

BillK

[Michael Hampicke]

2013/3/14 Dale <rdale...@gmail.com>

Howdy,

Also, I read that Nasdaq runs a modified version of Gentoo.  Do any
other large corps run it that we know of?

I googled a bit but couldn't find anything.  Maybe my search terms
wasn't good enough.

Yeehaw,

domainfactory (http://df.eu) uses a modified version of gentoo on their servers. df is one of the largest domain/hosting/mail providers in german-speaking countries.

[Francisco Ares]

Hi

Just my  $0.02:

Of course there are distro-related issues on performance, but once the system is up and running, wouldn't it be a matter of compiler/linker optimization differences?

Francisco

2013/3/14 Michael Hampicke <mgeha...@gmail.com>

--
"If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have one idea and we exchange these ideas, then each of us will have two ideas." - George Bernard Shaw

[Alan McKinnon]

On 14/03/2013 13:29, Mark David Dumlao wrote:

> On 03/14/2013 04:15 PM, Dale wrote:
>> Also, I read that Nasdaq runs a modified version of Gentoo. Do any
>> other large corps run it that we know of?
>>

> What exactly does it mean to run a "modified version of Gentoo"? Don't
> we all? ;)


I've always claimed to colleagues that there is no such thing as
"a running Gentoo".

There's an AlanOS, and a DaleOS and a MarkOS and they are all forks of
Gentoo, but nobody actually ever runs "Gentoo"

:-)


--
Alan McKinnon
alan.m...@gmail.com

[Pandu Poluan]

LOL... that's why I got into the habit of saying "Gentoo-based system" :-)

Rgds,
--

[Pandu Poluan]

On Mar 14, 2013 4:14 PM, "William Kenworthy" <bi...@iinet.net.au> wrote:
>
> Did this few years back for an online magazine sponsored by a local
> linux sysadmin company who wanted to see the difference between generic
> debian and optimised (not necessarily gentoo, but thats what I used.)
>
> Difference in times was ~10% across the board for graphics manipulations
> (gimp scripts), spreadsheet tasks (gnumeric) and the like.
>
> The "kicker" - simple optimisations gained far, far more than generic
> compiler settings.  e.g., initially, the gnumeric versions were slightly
> different, with some wild times across the tasks.  Make em the same
> version (and cuedos to the gnumeric maintainer for jumping in and
> helping diagnose/fix the problem - newer version on gentoo was heaps
> slower :) and there was little difference.
>
> Shared libs like glibc didnt make a huge difference, but being smart
> about how/what a "particular" task was handled gained more.  If a debian
> app was compiled with similar options as to gentoo, little difference
> between them in performance which considering shared libs etc wasn't
> what I expected.
>
> The intel compilers are/were said to be a lot better than gcc, not sure
> if the gap is still there (supposedly 20% better again)
>
> Its how long is a piece of string kind of question if considered OS
> wide, but pick a narrow task and optimise away with smart programmers
> and you will do well on almost anything.
>
> Big advantage of gentoo - configurability, version control (what version
> is installed and changing it at short notice) and general flexibility.
>

This.

Why I prefer Gentoo over other distros: Full control.

I mean, I can (and do) leverage "-march=native". And I certainly have an overly long USE flags... but it's the sheet satisfaction of knowing that my system is MY system that made me stick with Gentoo...

It's eminently satisfying -- a geekgasm, if you will -- to know that one's kernel is lean and customized, all the toolchains have been tuned, and there are no useless things being installed...

In regards to performance, the benefits might not be groundbreaking, but it's there, and when your server is being relentlessly hammered by requests, Gentoo seems to have additional breathing space where other distros choke...

Rgds,
--

[Alan McKinnon]

On 14/03/2013 14:31, Mark David Dumlao wrote:

> Smart call that you called it a "running" Gentoo rather than an
> "installed" one, because my followup question would have been, "Well
> what exactly does it mean to have installed Gentoo? I've had this laptop
> two years now and I'm still not done tinkering with it!" ;)


This is my fifth Dell laptop in a row with Gentoo installed. Tinkering?
yeah I do that too :-)

Some days this system looks like one of those crazy Wily E. Coyote
machines with all the bits I bolt on the back :-)



--
Alan McKinnon
alan.m...@gmail.com

[Alan McKinnon]

On 14/03/2013 14:12, Pandu Poluan wrote:
>
> On Mar 14, 2013 4:14 PM, "William Kenworthy" <bi...@iinet.net.au

Gentoo excels as a -dev system where your devs need to test things in
different environments.

A classic case is different pythons. We have many Centos 4 machines in
production that run python-2.4, the developers naturally run something
bleeding edge like 2.7 or 3.3 on their laptops.

Many many times they need to know if their bespoke code runs properly on
Centos, or PyPy or whatever other valid environment difference could
happen in the real world.

Tweak USE, tweak the masking and let emerge world do it's thing. Now the
dev can do valid tests. If the dev machines are VMs, snapshot them just
before starting this and you have the best possible solution for my money.

Or, try remove LDAP, NIS and PAM support for auth from a RHEL machine to
test if it works without those things in place.
RHEL? Impossible.
Gentoo? Trivially easy.



--
Alan McKinnon
alan.m...@gmail.com

[Mark David Dumlao]

"Trivially easy", of course, means an emerge -euDNtv world && emerge
-ctv && revdep-rebuild -i && revdep-rebuild ... ehehehe

I dunno, it might actually be easier to setup the said distros in a VM.
And if those configurations don't work, you shouldn't have to support
them, eh? ;)

[Alan McKinnon]

Well, devs tend to ask questions like "would this thing X work in
practice? or do I have to munge my code?"

They want to know if shipped code supports something. And, I don't get
to say "I'm sorry, I cannot support Centos 4 on this"

Business has a stock answer "Well, find a way to make it work."

Flexibility is the key. At least with

"emerge -euDNtv world && emerge -ctv && revdep-rebuild -i && revdep-rebuild"

I can walk away and come back in three hours, look at logs and tell them
to test. Plus I don't have to re-install their customer code everyt time
from scratch (said code *never*, of course, coming with anything
resembling a MakeFile)



--
Alan McKinnon
alan.m...@gmail.com

[Grant Edwards]

On 2013-03-14, Dale <rdale...@gmail.com> wrote:

> I was wondering. Has anyone ever seen where a test as been done to
> compare the speed of Gentoo with other distros? Maybe Gentoo compared
> to Redhat, Mandrake, Ubuntu and such?

I just did a test, and they're all the same.

CDs/DVDS of various distros dropped from a height of 1m all hit the
floor simultaneously [there are random variations due to aerodynamic
instability of the disk shape, but it's the same for all distros]. If
launched horizontally with spin to provide attitude stability (thrown
like a frisbee), they all fly the same.

The point being, you're going to have to define "speed".

Does speed refer to

Installation time?

Boot time?

Linpack?

Dhrystone?

Whetstone?

Time for me to figure out how to fix a configuration problem?

Time to do to an update on a machine that's been unplugged for a year?

Time to to produce a packaged version of some random C program that
comes with a Makefile that uses autotools?

Time for a reported bug to get fixed?

--
Grant Edwards grant.b.edwards Yow! Is it 1974? What's
at for SUPPER? Can I spend
gmail.com my COLLEGE FUND in one
wild afternoon??

[Alan McKinnon]

On 14/03/2013 16:07, Grant Edwards wrote:
> On 2013-03-14, Dale <rdale...@gmail.com> wrote:
>
>> I was wondering. Has anyone ever seen where a test as been done to
>> compare the speed of Gentoo with other distros? Maybe Gentoo compared
>> to Redhat, Mandrake, Ubuntu and such?
>
> I just did a test, and they're all the same.
>
> CDs/DVDS of various distros dropped from a height of 1m all hit the
> floor simultaneously [there are random variations due to aerodynamic
> instability of the disk shape, but it's the same for all distros]. If
> launched horizontally with spin to provide attitude stability (thrown
> like a frisbee), they all fly the same.

nonononononono, gentoo is much faster.

I did the same test, but comparing Centos on a DVD with Gentoo on a USB
stick. The stick tends to fall about 8% faster, mostly due to removing
those aerodynamic instabilities causing lift effects from the wing-like
shape of the DVD.

I consider this a perfectly valid test as Gentoo is designed to let me
remove unwanted side-effects from the environment. The shape of a DVD
was unwanted, so I made a tweak to take it out.



p.s. good joke on your part :-)
Dale is never going to live this one down. But he's a big boy, he can
take it.

>
> The point being, you're going to have to define "speed".
>
> Does speed refer to
>
> Installation time?
>
> Boot time?
>
> Linpack?
>
> Dhrystone?
>
> Whetstone?
>
> Time for me to figure out how to fix a configuration problem?
>
> Time to do to an update on a machine that's been unplugged for a year?
>
> Time to to produce a packaged version of some random C program that
> comes with a Makefile that uses autotools?
>
> Time for a reported bug to get fixed?
>
>


--

Alan McKinnon
alan.m...@gmail.com

[Paul Hartman]

On Thu, Mar 14, 2013 at 7:12 AM, Pandu Poluan <pa...@poluan.info> wrote:
> Why I prefer Gentoo over other distros: Full control.

That's it, in a nutshell.

> I mean, I can (and do) leverage "-march=native".

I've been scared away from -march and instead of -mtune in case i need
to drop my hard drive into another system for recovery which might
have an incompatible CPU.

[Neil Bothwick]

On Thu, 14 Mar 2013 21:40:49 +0800, Mark David Dumlao wrote:

> > Or, try remove LDAP, NIS and PAM support for auth from a RHEL machine
> > to test if it works without those things in place.
> > RHEL? Impossible.
> > Gentoo? Trivially easy.

> "Trivially easy", of course, means an emerge -euDNtv world && emerge
> -ctv && revdep-rebuild -i && revdep-rebuild ... ehehehe

There's no need to rebuild everything, and those other flags make no
sense when using -e. Generally you only need

emerge -uaD --changed-use @world


--
Neil Bothwick

Set phasers to extreme itching!

[William Kenworthy]

Ok, thats another "valid comparison" to go with dropping Alans gentoo on
a USB stick, centos on a DVD ... so what OS goes with a hard drive when
its "dropped"?

Is anyone near Piza ... Ive been told they have a tower that's been used
for these types of test in the past.

BillK

[Bruce Hill]

On Thu, Mar 14, 2013 at 07:29:54PM +0800, Mark David Dumlao wrote:
> <html>
> <head>
> <meta content="text/html; charset=ISO-8859-1"
> http-equiv="Content-Type">
> </head>
> <body bgcolor="#FFFFFF" text="#000000">
> <div class="moz-cite-prefix">On 03/14/2013 04:15 PM, Dale wrote:<br>
> </div>
> <blockquote cite="mid:51418728...@gmail.com" type="cite">
> <pre wrap="">Also, I read that Nasdaq runs a modified version of Gentoo. Do any

> other large corps run it that we know of?
>

> </pre>
> </blockquote>

> What exactly does it mean to run a "modified version of Gentoo"?

> Don't we all? ;)<br>
> </body>
> </html>

What kind of crap email do you call that ^^^ ?
--
Happy Penguin Computers >')
126 Fenco Drive ( \
Tupelo, MS 38801 ^^
sup...@happypenguincomputers.com
662-269-2706 662-205-6424
http://happypenguincomputers.com/

A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Don't top-post: http://en.wikipedia.org/wiki/Top_post#Top-posting

[Bruce Hill]

On Thu, Mar 14, 2013 at 08:31:54PM +0800, Mark David Dumlao wrote:
> <html>
> <head>
> <meta content="text/html; charset=ISO-8859-1"
> http-equiv="Content-Type">
> </head>
> <body bgcolor="#FFFFFF" text="#000000">

> <div class="moz-cite-prefix">On 03/14/2013 07:36 PM, Alan McKinnon
> wrote:<br>
> </div>
> <blockquote cite="mid:5141B649...@gmail.com" type="cite">
> <pre wrap="">On 14/03/2013 13:29, Mark David Dumlao wrote:
> </pre>
> <blockquote type="cite">
> <pre wrap="">On 03/14/2013 04:15 PM, Dale wrote:
> </pre>
> <blockquote type="cite">
> <pre wrap="">Also, I read that Nasdaq runs a modified version of Gentoo. Do any

> other large corps run it that we know of?
>

> </pre>
> </blockquote>
> <pre wrap="">What exactly does it mean to run a "modified version of Gentoo"? Don't
> we all? ;)
> </pre>
> </blockquote>
> <pre wrap="">

>
> I've always claimed to colleagues that there is no such thing as
> "a running Gentoo".
>
> There's an AlanOS, and a DaleOS and a MarkOS and they are all forks of
> Gentoo, but nobody actually ever runs "Gentoo"
>
> :-)

> </pre>
> </blockquote>

> Smart call that you called it a "running" Gentoo rather than an
> "installed" one, because my followup question would have been, "Well
> what exactly does it mean to have installed Gentoo? I've had this

> laptop two years now and I'm still not done tinkering with it!" ;)<br>
> </body>
> </html>

Kindly turn off your HTML ... this is email, not your personal web page. ;)

[Michael Mol]

On 03/14/2013 11:17 AM, Bruce Hill wrote:
> On Thu, Mar 14, 2013 at 07:29:54PM +0800, Mark David Dumlao wrote:
>> <html>
>> <head>
>> <meta content="text/html; charset=ISO-8859-1"
>> http-equiv="Content-Type">
>> </head>
>> <body bgcolor="#FFFFFF" text="#000000">
>> <div class="moz-cite-prefix">On 03/14/2013 04:15 PM, Dale wrote:<br>
>> </div>
>> <blockquote cite="mid:51418728...@gmail.com" type="cite">
>> <pre wrap="">Also, I read that Nasdaq runs a modified version of Gentoo. Do any
>> other large corps run it that we know of?
>>
>> </pre>
>> </blockquote>
>> What exactly does it mean to run a "modified version of Gentoo"?
>> Don't we all? ;)<br>
>> </body>
>> </html>
>
> What kind of crap email do you call that ^^^ ?
>

From the headers of his email:

Subject: Re: [gentoo-user] Gentoo speed comparison to other distros
References: <51418728...@gmail.com>
In-Reply-To: <51418728...@gmail.com>
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

It's perfectly compliant. You may want to correct your mail client to
understand HTML.

(Admittedly, it's unusual to see email clients send *only* text/html,
rather than a multipart message with two different encodings.)

[João Matos]

2013/3/14 Michael Mol <mik...@gmail.com>

At least one link: http://www.linux-mag.com/id/7574/ . It is kinda old, but I liked the reading.

--
João de Matos
Linux User #461527

[Dale]

Grant Edwards wrote:
> On 2013-03-14, Dale <rdale...@gmail.com> wrote:
>
>> I was wondering. Has anyone ever seen where a test as been done to
>> compare the speed of Gentoo with other distros? Maybe Gentoo compared
>> to Redhat, Mandrake, Ubuntu and such?
> I just did a test, and they're all the same.
>
> CDs/DVDS of various distros dropped from a height of 1m all hit the
> floor simultaneously [there are random variations due to aerodynamic
> instability of the disk shape, but it's the same for all distros]. If
> launched horizontally with spin to provide attitude stability (thrown
> like a frisbee), they all fly the same.
>
> The point being, you're going to have to define "speed".
>
> Does speed refer to
>
> Installation time?
>
> Boot time?
>
> Linpack?
>
> Dhrystone?
>
> Whetstone?
>
> Time for me to figure out how to fix a configuration problem?
>
> Time to do to an update on a machine that's been unplugged for a year?
>
> Time to to produce a packaged version of some random C program that
> comes with a Makefile that uses autotools?
>
> Time for a reported bug to get fixed?
>

OK. It appears not very many can figure out what I asked for. So, let
me spell it out for those who are challenged. LOL ;-) Read some
humor into that OK.

Install a OS. Run tests on a set of programs and record the time it
takes to complete a certain task. More tasks the better.

Then install another OS on the same hardware. Run tests on a set of
programs and record the time it takes to complete a certain task. More
tasks the better.

The object of this is, does Gentoo with the customization it allows run
faster than some binary install that does NOT allow those controls? In
other words, can a Gentoo based install perform more efficiently than a
binary based install like Redhat, Ubuntu or some other distro?

I am NOT concerned about compile times or the install itself.

Does that put the dots closer together for the challenged ones? ROFL

Dale

:-) :-)

[Mateusz Kowalczyk]

The point of the challenged ones was that while we can take measurements
like these, it's rather meaningless to do so. The result will be
different for every single person out there depending on their
configuration, USE, CFLAGS and who knows what else.

I can compile a package with support for 3 different DEs, few WMs, oss
and alsa and about a billion things I will never use. Does this make for
a more or less of a meaningful test than doing the same test with no
flags what so ever? There is no correct answer as it varies per user
basis. The most meaningful measurements that we can probably take would
be between different USE flags configurations. Maybe we can say that
package ‘foo’ with certain USE and CFLAGS runs in less average time than
the same package on a distro Bar.

In my opinion, it would be far more meaningful to measure the effect of
different USE flags on the same package, *in relative time* on the same
system. This would give us more idea about the impact of each flag as
opposed to a very limited view of ‘package foo with certain specific USE
flags runs 10ms faster than the same package on the same hardware on a
binary distribution’. If you still want such measurements and you want
them to be somewhat meaningful to you, it is you who will have to take
them. Unless there are some gross inconsistencies in run times on
different distributions, we have no use for such measurement.

Everyone understood what you asked for. It's _you_ that misunderstood
their explanation for why it's meaningless to ask such a question in the
first place.

--
Mateusz K.

[Dale]

I didn't miss anything. I get what some are saying. The reason for my
question is this. Gentoo allows a person to customize the OS to the
specific hardware it is being run on. Redhat and other binary distros
don't allow this, unless you compile your own packages which is no
longer really a binary install.

So, if I install Redhat on my machine, would it be less efficient than
my Gentoo install which is customized for my hardware? Has someone else
tested this and made it public?

If people can't get this, never mind.

[Neil Bothwick]

On Thu, 14 Mar 2013 18:52:41 -0500, Dale wrote:

> So, if I install Redhat on my machine, would it be less efficient than
> my Gentoo install which is customized for my hardware? Has someone else
> tested this and made it public?

No. They may have tested it on their machine, but not on yours, so their
results aren't applicable to running the same tests on DaleOS[tm].

The times this really matters is when running resource-intensive or
time-critical applications (like the NASDAQ example you gave) and on
those situations it is possible to define a set of test that give meaning
results, but only n those situations.

Your question makes "how long is a piece of string" seem a model of
precision.


--
Neil Bothwick

A Smith & Weason beats Four Aces everytime.

[Mateusz Kowalczyk]

I don't think that it's plausible to take such measurement. We could set
every USE flag possible for the package we are benchmarking to try and
replicate the support for everything that the binary package is likely
to have. We also have to do this for all its dependencies (and their
dependencies and so on) to have nothing that could potentially influence
the measurement. Assuming that portage complies with this (it won't), we
compile the package with optimizations for our hardware. The result? We
probably have the same result on Gentoo and the other distro.

Why? The reason is simple: binary distributions provide packages
compiled with optimizations turned on for specific architectures. Unless
you are doing some unheard of optimizations for your obscure model of
the CPU, I don't imagine you'd get much advantage at all. If the
maintainer of the binary package compiles it with optimizations for an
i7 and you do the same, why should the performance be different?

If you install RedHat on your machine, it probably will be less
efficient than Gentoo but for a different reason. It won't be
(noticeably) more efficient because you've got your CFLAGS set in a
particular way but it because you only install the packages you actually
want and remove support for things you don't need. Why bloat a package
and waste cycles having it try to poll a service you might never
actually use? Gentoo lets you get rid (or never install in the first
place) of this kind of… bloat while you don't have much choice on a
binary package short of compiling everything by hand at which point you
should be using Gentoo to do it for you.

RedHat maintainers aren't stupid (you can probably tell I've never used
RH) – they will release packages optimized for architectures they will
run on. Overall you might get very slight performance boost because of
some CFLAG you enable but you might as well have worse performance
because you don't know as much about optimizations as the RH maintainers
and developers. Bah, you can even find examples on Gentoo wiki where
compiling certain packages with certain flags actually makes them slower
and not faster where usually the opposite is the case.

--
Mateusz K.

[William Kenworthy]

On 15/03/13 08:31, Mateusz Kowalczyk wrote:
> On 14/03/13 23:52, Dale wrote:
>> Mateusz Kowalczyk wrote:
>>> On 14/03/13 22:41, Dale wrote:
>>>> Grant Edwards wrote:
>>>>> On 2013-03-14, Dale <rdale...@gmail.com> wrote:
>>>>>

>>>>> ...

>>>>> RedHat maintainers aren't stupid (you can probably tell I've never
>>>>> used RH) – they will release packages optimized for architectures
>>>>> they will run on. Overall you might get very slight performance
>>>>> boost because of some CFLAG you enable but you might as well have
>>>>> worse performance because you don't know as much about
>>>>> optimizations as the RH maintainers and developers. Bah, you can
>>>>> even find examples on Gentoo wiki where compiling certain packages
>>>>> with certain flags actually makes them slower and not faster where
>>>>> usually the opposite is the case.

Further, when we did the tests I mentioned before (exactly what Dale was
asking about in fact) - we had 3 identical machines for testing in
parallel ... Celerons at the time. While setting up, it became clear
that while gentoo was working well on my P4 laptop, cloning it onto the
celeron gave performance worse than a default i386 debian. So after a
bit of swatting on compiler flags I tuned it closer to the architecture,
did an overnight rebuild and we went from there ... and it could only
"shade" i386 default debian about 10% ... mostly.

1. The upshot is that I consider its actually easier to shoot yourself
in the foot performance wise if you get it wrong than it is to get it right.

2. Tuning for a particular load/job *WILL* make the machine more
unsuitable to other load/job profiles.

3. On the same hardware, any distro can/should be made to perform
identically if tuned by someone in the know (or made worse)

4. Gentoo is easier to tune (make better ... or worse :)

BillK

[Pandu Poluan]

AFAIK, binary distros have only two kinds of targets for Intel/AMD processors: x86 or amd64.

I think most binary distros set their -march or -mtune to nothing newer than a Pentium Pro (x86 case) or Pentium 4 (amd64 case). This means, newer generation instruction sets are not used.

Of course, the amount (%) of code that can be optimized is very small compared to the rest of the code. On most programs, there will be no benefit of mtune/march to a specific processor.

However, if you happen to frequently use a program whose major processing time happens to be optimizable with the newer instructions, you hit a jackpot.

TL;DR : Just like everybody say, Gentoo might be faster, but depends totally on the jobs one do on it. On majority of cases, the faster speed is totally inconsequential.

Rgds,
--

[Mark David Dumlao]

On Thu, Mar 14, 2013 at 10:12 PM, Alan McKinnon <alan.m...@gmail.com> wrote:
> On 14/03/2013 16:07, Grant Edwards wrote:
>> On 2013-03-14, Dale <rdale...@gmail.com> wrote:
>>
>>> I was wondering. Has anyone ever seen where a test as been done to
>>> compare the speed of Gentoo with other distros? Maybe Gentoo compared
>>> to Redhat, Mandrake, Ubuntu and such?
>>
>> I just did a test, and they're all the same.
>>
>> CDs/DVDS of various distros dropped from a height of 1m all hit the
>> floor simultaneously [there are random variations due to aerodynamic
>> instability of the disk shape, but it's the same for all distros]. If
>> launched horizontally with spin to provide attitude stability (thrown
>> like a frisbee), they all fly the same.
>
>
> nonononononono, gentoo is much faster.
>
> I did the same test, but comparing Centos on a DVD with Gentoo on a USB
> stick. The stick tends to fall about 8% faster, mostly due to removing
> those aerodynamic instabilities causing lift effects from the wing-like
> shape of the DVD.
>
> I consider this a perfectly valid test as Gentoo is designed to let me
> remove unwanted side-effects from the environment. The shape of a DVD
> was unwanted, so I made a tweak to take it out.
>
>

Nice try, but CentOS has a network install option. So before you guys
get up the elevator to the tenth floor, the sysad on the ground has
already smashed the machine in frustration.

[Mark David Dumlao]

I know that, in general principle. But it's a test environment. I'd
assume stricter standards of "purity" there than elsewhere. simply
going by changed-use can break some library dependencies. We need to
use depclean to remove build deps junk after the emptytree, and we're
revdep-rebuilding twice in case the depclean borked something. (To be
really strict, revdep-rebuild should be repeated until it stops
building things...)

Heck in some setups empty-tree will simply fail thanks to circular
deps of the global use flags and you'll need manual intervention to
bootstrap a package with less USE...

[Mark David Dumlao]

On Thu, Mar 14, 2013 at 10:02 PM, Alan McKinnon <alan.m...@gmail.com> wrote:
> On 14/03/2013 15:40, Mark David Dumlao wrote:
>> On 03/14/2013 09:28 PM, Alan McKinnon wrote:
>>> Or, try remove LDAP, NIS and PAM support for auth from a RHEL machine to
>>> test if it works without those things in place.
>>> RHEL? Impossible.
>>> Gentoo? Trivially easy.
>> "Trivially easy", of course, means an emerge -euDNtv world && emerge
>> -ctv && revdep-rebuild -i && revdep-rebuild ... ehehehe
>>
>> I dunno, it might actually be easier to setup the said distros in a VM.
>> And if those configurations don't work, you shouldn't have to support
>> them, eh? ;)
>>
>
>
> Well, devs tend to ask questions like "would this thing X work in
> practice? or do I have to munge my code?"
>

No, that doesn't make sense. The situation you presented above was
removing "impossible to remove" components on an OS and asking if the
software still works. You don't get to call that a vaild test
environment if the test environment itself doesn't work in the first
place.

> They want to know if shipped code supports something. And, I don't get
> to say "I'm sorry, I cannot support Centos 4 on this"
>
> Business has a stock answer "Well, find a way to make it work."

Actually, business has a stock answer of "Supported on Windows XP or
later, Mac OS X some cat, Red Hat version foo, SuSE...."

In general they target actual known platforms, and YES they get to say
"I cannot support Centos 4 on this" all the time.

> Flexibility is the key. At least with
>
> "emerge -euDNtv world && emerge -ctv && revdep-rebuild -i && revdep-rebuild"
>
> I can walk away and come back in three hours, look at logs and tell them
> to test. Plus I don't have to re-install their customer code everyt time
> from scratch (said code *never*, of course, coming with anything
> resembling a MakeFile)
>

Hoo boy what I would give for -euDNtv to take less than 3 hours on my setup ;)

[Walter Dnes]

On Thu, Mar 14, 2013 at 06:52:41PM -0500, Dale wrote

> I didn't miss anything. I get what some are saying. The reason for
> my question is this. Gentoo allows a person to customize the OS to
> the specific hardware it is being run on. Redhat and other binary
> distros don't allow this, unless you compile your own packages which
> is no longer really a binary install.
>
> So, if I install Redhat on my machine, would it be less efficient than
> my Gentoo install which is customized for my hardware? Has someone
> else tested this and made it public?

I'm not aware of any. There is my experience with NHL Gamecenter
Live on my 2007 Dell Inspiron D530 desktop with onboard Intel GPU. The
initial Gentoo install could not handle even the slowest feed. That was
the generic i686 code from the initial stage 3. After I ran "emerge
system" and "emerge world" including the "-march=native" flag it handles
the lowest speed stream just fine. CPU is Intel(R) Core(TM)2 Duo CPU
E4600 @ 2.40GHz.

--
Walter Dnes <walt...@waltdnes.org>
I don't run "desktop environments"; I run useful applications

[Neil Bothwick]

On Fri, 15 Mar 2013 13:47:49 +0800, Mark David Dumlao wrote:

> > There's no need to rebuild everything, and those other flags make no
> > sense when using -e. Generally you only need
> >
> > emerge -uaD --changed-use @world
> >
>
> I know that, in general principle. But it's a test environment. I'd
> assume stricter standards of "purity" there than elsewhere. simply
> going by changed-use can break some library dependencies. We need to
> use depclean to remove build deps junk after the emptytree, and we're
> revdep-rebuilding twice in case the depclean borked something. (To be
> really strict, revdep-rebuild should be repeated until it stops
> building things...)

portage should handle that itself nowadays, but it doesn't hurt to run
revdep-rebuild to be sure. You could use -N instead of --changed-use but I
still think -e is unnecessary.

>
> Heck in some setups empty-tree will simply fail thanks to circular
> deps of the global use flags and you'll need manual intervention to
> bootstrap a package with less USE...

And that's a good reason to not use -e. If you do use -e, none of the
other options make any sense, -u -D and -N are meaningless if the system
thinks nothing is installed and there's no point in using -t without -a
or -p, and with -e it would generate so much output I'm not sure many
people would bother reading it all.


--
Neil Bothwick

Cross-country skiing is great in small countries.

[Mark David Dumlao]

I'm pretty sure I just recycled the emptytree + deep/newuse advice
from one of the docs. I see it mentioned in the wiki at least.

http://en.gentoo-wiki.com/wiki/Freeing_Up_Disk_Space

Honestly, though, it's just a case of muscle memory at work. Usually I
just -uDNtv everything and just add options after that like -1, -a...

[Mark David Dumlao]

ROFL. It's called "me wrestling with thunderbird to try to remove html
formatting but failing".

[Nuno Silva]

On 2013-03-15, Dale wrote:

> I didn't miss anything. I get what some are saying. The reason for my
> question is this. Gentoo allows a person to customize the OS to the
> specific hardware it is being run on. Redhat and other binary distros
> don't allow this, unless you compile your own packages which is no
> longer really a binary install.
>
> So, if I install Redhat on my machine, would it be less efficient than
> my Gentoo install which is customized for my hardware? Has someone else
> tested this and made it public?
>
> If people can't get this, never mind.

I have not tested this nor seen data on this, but I'd look for
comparisons on the efficiency and gains from gcc optimizations. These
would be what benefits source-based distros on a specific system
compared to binary distros, and a benchmark made with gcc will be
simpler and easier to deal with than an os-wide benchmark.

--
Nuno Silva (aka njsg)
http://njsg.sdf-eu.org/

[Grant Edwards]

On 2013-03-14, Dale <rdale...@gmail.com> wrote:
> Grant Edwards wrote:
>> On 2013-03-14, Dale <rdale...@gmail.com> wrote:
>>
>>> I was wondering. Has anyone ever seen where a test as been done to
>>> compare the speed of Gentoo with other distros? Maybe Gentoo compared
>>> to Redhat, Mandrake, Ubuntu and such?
>> I just did a test, and they're all the same.
>>
>> CDs/DVDS of various distros dropped from a height of 1m all hit the
>> floor simultaneously

>> [...]

>> The point being, you're going to have to define "speed".

> OK. It appears not very many can figure out what I asked for. So,
> let me spell it out for those who are challenged. LOL ;-) Read some
> humor into that OK.
>
> Install a OS. Run tests on a set of programs and record the time it
> takes to complete a certain task. More tasks the better.

The results are going to vary depending on what task(s) are chosen.

If app/library/compiler versions are the same, all of the results I've
read about show you're not going to see a noticable difference. You
might be able to _measure_ a difference, but it's not something you'll
ever notice.

IOW, if you spend a few days tweaking CFLAGS, you might be able to
increase the number of FFTs per second you can run by a few percent
when compared to an off-the-shelf Ubuntu, RedHat, or Scientific Linux
install. But, if that's what you care about, then using a better
library/algorithm or better hardware is what you do.

The advantage of Gentoo is ease of administration, ease of
customization, ease of getting non default/mainstream things installed
and working right.

--
Grant Edwards grant.b.edwards Yow! The entire CHINESE
at WOMEN'S VOLLEYBALL TEAM all
gmail.com share ONE personality --
and have since BIRTH!!

[Kevin Chadwick]

> >
> > From the headers of his email:
> >
> > Subject: Re: [gentoo-user] Gentoo speed comparison to other distros
> > References: <51418728...@gmail.com>
> > In-Reply-To: <51418728...@gmail.com>
> > Content-Type: text/html; charset=ISO-8859-1
> > Content-Transfer-Encoding: 7bit
> >
> > It's perfectly compliant. You may want to correct your mail client to
> > understand HTML.
> >
> > (Admittedly, it's unusual to see email clients send *only* text/html,
> > rather than a multipart message with two different encodings.)
> >
>
> ROFL. It's called "me wrestling with thunderbird to try to remove html
> formatting but failing".

Compulsory html annoys me on Android (If only you could have proper
programs like Nokias N9 had claws)

Claws would mean you needn't bother and still have html to text by
default and can even enable html plugins if desired (right way around).


--
_______________________________________________________________________

'Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface'

(Doug McIlroy)
_______________________________________________________________________

[Kevin Chadwick]

Or the real difference maker, designing the program itself to be faster
or using a really fast storage device bearing in mind any draw backs
like storage space.

If you use hardened Gentoo or OpenBSD or a PAE gentoo like Sabayon it
may be slightly slower but more secure but you won't notice any
difference when waiting for firefox to open until the second time.

If you use the Gentoo hardened Tinfoil Linux you will need lots of ram
and wait ages to boot but firefox will just pop up.

Compiling speed, well I would just get better hardware or do
distributed compiles as otherwise chances are your taking risks
especially if you don't test and understand exactly what you are
changing very well bearing in mind that with compilers everything may
work fine 97% instead of 99% of the time.

[Mick]

On Friday 15 Mar 2013 17:36:48 Kevin Chadwick wrote:
> > > From the headers of his email:
> > >
> > > Subject: Re: [gentoo-user] Gentoo speed comparison to other distros
> > > References: <51418728...@gmail.com>
> > > In-Reply-To: <51418728...@gmail.com>
> > > Content-Type: text/html; charset=ISO-8859-1
> > > Content-Transfer-Encoding: 7bit
> > >
> > > It's perfectly compliant. You may want to correct your mail client to
> > > understand HTML.
> > >
> > > (Admittedly, it's unusual to see email clients send *only* text/html,
> > > rather than a multipart message with two different encodings.)
> >
> > ROFL. It's called "me wrestling with thunderbird to try to remove html
> > formatting but failing".
>
> Compulsory html annoys me on Android (If only you could have proper
> programs like Nokias N9 had claws)
>
> Claws would mean you needn't bother and still have html to text by
> default and can even enable html plugins if desired (right way around).

I understand that you can specify what sort of mail format you want to send
per email recipient, including of course <gento...@lists.gentoo.org>, but I
don't have T'bird installed to check:

http://kb.mozillazine.org/Plain_text_e-mail_(Thunderbird)

HTH.
--
Regards,
Mick

[Mick]

On Friday 15 Mar 2013 20:34:14 Mark David Dumlao wrote:

> On 03/16/2013 04:06 AM, Mick wrote:

> I understand that you can specify what sort of mail format you want to send
> per email recipient, including of course <gento...@lists.gentoo.org>,
> but I don't have T'bird installed to check:
>
> http://kb.mozillazine.org/Plain_text_e-mail_(Thunderbird)
>
> HTH.
>
>

> I know about that. But it fails to work on compose windows opened by the
> thunderbird conversations plugin. Quotes there seem to be hard-quoted as
> HTML and no amount of fiddling converts those into plaintext quotes.

OK, I am not a T'bird user, let alone plugins for this application - but
Google tells me that the 'Quick Reply' feature creates plain text responses.
Is this the case?

--
Regards,
Mick

[Michael Mol]

On 03/15/2013 04:34 PM, Mark David Dumlao wrote:
> On 03/16/2013 04:06 AM, Mick wrote:

> I know about that. But it fails to work on compose windows opened by the
> thunderbird conversations plugin. Quotes there seem to be hard-quoted as
> HTML and no amount of fiddling converts those into plaintext quotes.

Reply created from conversation view in Thunderbird.

(Though I've got some configuration item set somewhere to only send in
plaintext; Enigmail complains that text/html emails don't always work
right with PGP signing.)

[Joshua Murphy]

On Thu, Mar 14, 2013 at 4:15 AM, Dale <rdale...@gmail.com> wrote:

Howdy,

I was wondering.  Has anyone ever seen where a test as been done to
compare the speed of Gentoo with other distros?  Maybe Gentoo compared
to Redhat, Mandrake, Ubuntu and such?

Also, I read that Nasdaq runs a modified version of Gentoo.  Do any
other large corps run it that we know of?

I googled a bit but couldn't find anything.  Maybe my search terms
wasn't good enough.

Links would be nice.

Dale

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!

While I'll start by backing up everything said by others regarding the differences being nearly negligible in a truly equal test, same feature set from source vs the feature set provided by a binary distro, and even a loss in terms of total time when you include the compile times involved, I do have a bit of anecdotal evidence in Gentoo's favor.

On the majority of x86 or x86_64 hardware there's very little room for across the board gains in performance over otherwise standard cflags. On slightly less 'normal' hardware, like, say, an Atom N270 based netbook with 1GB of ram, however, a few cflags go a *long* way towards having a usable system. My Mini9 shipped with a variant of Ubuntu that's actually built with general optimizations to make it usable on that hardware, and having run the same version of Ubuntu without those optimizations for a day or two on it, the amount of stutter and stalling was almost unbearable. Then, with the help of a desktop (or three) to handle the bulk of the compilation, I moved to Gentoo on it. I hadn't sorted out what cflags would be best, and simply built what I needed to get back to work on it with fairly minimal use flags, and I was rather frustrated to find that it still ran worse than the factory install, once programs had started (though that process was noticeably faster, as it generally is with so much less running in the background). Once I adjusted to the appropriate cflags, the stutter cleared up, things didn't stall frequently, and the system was simply more responsive. I could even watch flash videos full screen without it stuttering, which I'd given up on as a possibility on the system. A vast majority of the gains I saw were simply from clearing away the 80% of Ubuntu's features I have no use for, but when you have a processor that approaches things just a little differently, like an Atom, you really can gain a bit from letting the compiler put things in an order the processor will agree with better.

--
Poison [BLX]
Joshua M. Murphy

[Stroller]

On 15 March 2013, at 17:32, Kevin Chadwick wrote:
>
> If you use the Gentoo hardened Tinfoil Linux you will need lots of ram
> and wait ages to boot but firefox will just pop up.

I'm sorry, I don't understand this statement. Could you possibly explain, please?

Stroller.

[Stroller]

On 15 March 2013, at 17:36, Kevin Chadwick wrote:
>> ...

>> ROFL. It's called "me wrestling with thunderbird to try to remove html
>> formatting but failing".
>
> Compulsory html annoys me on Android (If only you could have proper
> programs like Nokias N9 had claws)

Wait, K9 Mail doesn't have a plain text option?

Perhaps I shouldn't be surprised, as I am also unable to comprehend why K9 might enforce top-posting on replies.

Stroller.

[Chris Walters]

On Thu, 14 Mar 2013 18:52:41 -0500
Dale <rdale...@gmail.com> wrote:

> I didn't miss anything. I get what some are saying. The reason for
> my question is this. Gentoo allows a person to customize the OS to
> the specific hardware it is being run on. Redhat and other binary
> distros don't allow this, unless you compile your own packages which
> is no longer really a binary install.
>
> So, if I install Redhat on my machine, would it be less efficient than
> my Gentoo install which is customized for my hardware? Has someone
> else tested this and made it public?
>
> If people can't get this, never mind.
>
> Dale

I have not really searched for such benchmarks for a while. I would
take any such things with a grain of salt, if I did. Unless I want to
devote the time to set up a series of application specific tests, and
test them on multiple platforms on MY hardware, such results would be
almost useless to me. I am sure that there are such tests out there -
Computer Science and Engineering departments at Universities have
certainly run such tests - whether they are public or not, is another
story.

Also, if you could specifically define "efficient", it would be
helpful, at least to narrow down what you're looking for. If you mean,
accomplishing the same task with the fewest instructions, that is
difficult to test, and very dependent on Kernel, library and compiler
versions. If you mean faster, that is more hardware dependent, unless
you're dealing with some very poorly written code.

Efficient does not always mean faster - as others have pointed out,
binary distos are faster to install than source based distros. FreeBSD
is faster than GNU/Linux on some tasks, and overall is more UNIX-like,
since it was a Berkeley developed version of UNIX. This has been
tested on other people's hardware. Would that necessarily compare to
mine or yours? We don't know that, since we have NOT tested them on
each of our sets of hardware.

E.g. FreeBSD uses an older version of just about everything, so I
cannot optimize the compiler for my Core-i7-avx processor. I can do
this with Gentoo (or any other GNU/Linux distro that I choose to
'optimize' by compiling from source).

Just my $0.02,
Chris

[Mysterious Mose]

Stroller wrote:
> Wait, K9 Mail doesn't have a plain text option?
>
> Perhaps I shouldn't be surprised, as I am also unable to comprehend why K9 might enforce top-posting on replies.

K9 Mail can do both plain text and bottom posting.
Both set in Account settings/Sending mail.

[Marc Stürmer]

Am 14.03.2013 09:15, schrieb Dale:

> I was wondering. Has anyone ever seen where a test as been done to
> compare the speed of Gentoo with other distros? Maybe Gentoo compared
> to Redhat, Mandrake, Ubuntu and such?

Running Gentoo is not a choice about raw _speed_. I mean, even if you
claim that your binaries are running 2-3% faster than e.g. on Debian,
this is something really negligable on a production system.

I mean, you may gain a small percentage of running speed, but on the
other hand you get the need to have a compiler installed on your system,
which could be quite a security hole, and having binaries produced by
yourself.

If you are not that lucky to have your own binary package building host
for Gentoo that's something, that you don't want to have on heavy duty
production systems, like e.g. database hosts. Compiler runs on such
systems are a big nono to me.

So running Gentoo is about another thing - _choice_ and _flexibility_.
It fits that hole quite nicely if you need package switches enabled most
binary based distributions don't have enabled. Otherwise running those
distributions is the way to go.

Of course, if you like to tinker with your system to shape it the way
you like it, Gentoo is a good choice.

[Kevin Chadwick]

It can write but forces html onto users, which potentially includes jpg
exploits, png exploits, html exploits, script exploits, font exploits...

And before you say anything. For what benefit, annoying ads from
paypal. I am quite capable of opening a browser and deciding which
domains *I* trust??

Google's network fell into this trap and banned Windows, but did they
fix the real problem or just raise the bar a little (though I expect
they took other unreleased measures that would be more interesting)?

Would be even worse on Iphones where webkit is forced and so as old as
the rom image. Rom cycle time is a major reason why even on cyanogenmod
I use firefox over the chrome package which is ancient.

Of course on Apple laptops even, Safari's webkit is sometimes months old
anywhow.

Having knocked Android, I haven't found the time to try the latest
native email app. I'm not expecting a no html option but I'm pretty
sure it will have some major pluses over k9mail, which was a trade of
good for bad on Gingerbread.

[Kevin Chadwick]

It's one of Blueness projects based on Hardened Gentoo. It loads into
ram at boot (you need something like 4 gig of ram) which takes ages
from dvd but could be from an ssd/hdd (defeating half the point
without a ro switch though). It can update from the net once booted too.

Once done everythings in ram so firefox can literally pop up like a
web advert upon execution.

[Michael Mol]

On 03/18/2013 04:38 PM, Kevin Chadwick wrote:
>>> Wait, K9 Mail doesn't have a plain text option?
>>>
>>> Perhaps I shouldn't be surprised, as I am also unable to comprehend why K9 might enforce top-posting on replies.
>>
>> K9 Mail can do both plain text and bottom posting.
>> Both set in Account settings/Sending mail.
>
> It can write but forces html onto users, which potentially includes jpg
> exploits, png exploits, html exploits, script exploits, font exploits...
>
> And before you say anything. For what benefit, annoying ads from
> paypal. I am quite capable of opening a browser and deciding which
> domains *I* trust??
>
> Google's network fell into this trap and banned Windows, but did they
> fix the real problem or just raise the bar a little (though I expect
> they took other unreleased measures that would be more interesting)?
>
> Would be even worse on Iphones where webkit is forced and so as old as
> the rom image. Rom cycle time is a major reason why even on cyanogenmod
> I use firefox over the chrome package which is ancient.
>
> Of course on Apple laptops even, Safari's webkit is sometimes months old
> anywhow.
>
> Having knocked Android, I haven't found the time to try the latest
> native email app. I'm not expecting a no html option but I'm pretty
> sure it will have some major pluses over k9mail, which was a trade of
> good for bad on Gingerbread.
>

I don't know what mail client you use (I suppose I could check your
headers), but *every* mail client I've used disables loading remote
content by default.

Further, you're ranting about users being "forced" to send email with
HTML, intimating that this means they'll send exploit-laden messages to
their recipients. That's patently silly; the people "forced" to send
HTML emails aren't going to be sending exploits. That's like suggesting
that people forced to drive to work are forced to commit vehicular
manslaughter...

It's the recipient of the email who has the burden of remaining secure,
and this is possible largely through simply disabling loading rich media
by default. Again, most mail clients disable loading remote media by
default, and most I've used support disabling packaged media as well.

[Michael Mol]

On 03/18/2013 04:21 PM, Kevin Chadwick wrote:
>> On 15 March 2013, at 17:32, Kevin Chadwick wrote:
>>>
>>> If you use the Gentoo hardened Tinfoil Linux you will need lots of ram
>>> and wait ages to boot but firefox will just pop up.
>>
>> I'm sorry, I don't understand this statement. Could you possibly explain, please?
>
> It's one of Blueness projects based on Hardened Gentoo. It loads into
> ram at boot (you need something like 4 gig of ram) which takes ages
> from dvd but could be from an ssd/hdd (defeating half the point
> without a ro switch though). It can update from the net once booted too.
>
> Once done everythings in ram so firefox can literally pop up like a
> web advert upon execution.
>

In other words, it's a distribution designed to not allow persistent
storage that might possibly be poisoned, and instead get much of its
security-conscious code updated over the network.

The "just pops up" being referred to simply comes from everything being
loaded into the kernel file cache before you can do anything with the
system.

(Frankly, this sounds quite nice for kiosk environments.)

[Kevin Chadwick]

> I don't know what mail client you use (I suppose I could check your
> headers), but *every* mail client I've used disables loading remote
> content by default.
>

Except the content within the message. Why do you assume I am talking
about remote content.

> Further, you're ranting about users being "forced" to send email with
> HTML, intimating that this means they'll send exploit-laden messages to
> their recipients.

I am not.

On 03/18/2013 04:38 PM, Kevin Chadwick wrote:
>> It can write but forces html onto users,

You seem to miss some of the details. I'll find time to respond on ipv6
too at some point ;-)

[Kevin Chadwick]

> >
> > It's one of Blueness projects based on Hardened Gentoo. It loads into
> > ram at boot (you need something like 4 gig of ram) which takes ages
> > from dvd but could be from an ssd/hdd (defeating half the point
> > without a ro switch though). It can update from the net once booted too.
> >
> > Once done everythings in ram so firefox can literally pop up like a
> > web advert upon execution.
> >
>
> In other words, it's a distribution designed to not allow persistent
> storage that might possibly be poisoned,

Not really, that is one benefit, but don't forget that BIOS, HDD
or Video card firmware could have been altered.

The main goals are reliability and leave no trace elements but it does
have some added tamper ensurance yes.

I didn't spell it out because you should check the site to see all the
details and would be bound to get it a little wrong without checking
myself.

> and instead get much of its
> security-conscious code updated over the network.
>

Security conscious code??? What do you mean? That says to me things
like PAX brute force protection??

Even though it is from a DVD it can be updated just like standard linux.
The problem is, if you run out of ram then things get killed.

> (Frankly, this sounds quite nice for kiosk environments.)

Could be if you have a good enough network connection for Linux kernel
updates or cut it right down ;-)

[Michael Mol]

On 03/18/2013 05:38 PM, Kevin Chadwick wrote:
>>>
>>> It's one of Blueness projects based on Hardened Gentoo. It loads
>>> into ram at boot (you need something like 4 gig of ram) which
>>> takes ages from dvd but could be from an ssd/hdd (defeating half
>>> the point without a ro switch though). It can update from the net
>>> once booted too.
>>>
>>> Once done everythings in ram so firefox can literally pop up like
>>> a web advert upon execution.
>>>
>>
>> In other words, it's a distribution designed to not allow
>> persistent storage that might possibly be poisoned,
>
> Not really, that is one benefit, but don't forget that BIOS, HDD or
> Video card firmware could have been altered.

Sure.

>
> The main goals are reliability and leave no trace elements but it
> does have some added tamper ensurance yes.
>
> I didn't spell it out because you should check the site to see all
> the details and would be bound to get it a little wrong without
> checking myself.
>
>> and instead get much of its security-conscious code updated over
>> the network.
>>
>
> Security conscious code??? What do you mean? That says to me things
> like PAX brute force protection??

I mean everything that gets updated more frequently owing to its being a
high-profile target in security contexts. Web browsers. Mail clients.
Listening daemons.

Having a static image that you need to update every time you boot is a
bit like plugging in an unpatched Windows machine that you need to run
updates on...every time you boot. It's a tad silly in that respect.

>
> Even though it is from a DVD it can be updated just like standard
> linux. The problem is, if you run out of ram then things get killed.
>
>
>> (Frankly, this sounds quite nice for kiosk environments.)
>
> Could be if you have a good enough network connection for Linux
> kernel updates or cut it right down ;-)

Local gigabit is cheap, and a gigabit connection would transfer the
image in under a minute. A bit more, of course, if you've got an
overloaded server being slammed by ten or twenty machines.

(I wonder if one can anycast TFTP on a local segment. Hm. I think you
could just barely pull it off, since you'd have resolved the layer 2
address for your syn packet, and that should stick with the connection.)

[Neil Bothwick]

On Mon, 18 Mar 2013 20:38:11 +0000, Kevin Chadwick wrote:

> > K9 Mail can do both plain text and bottom posting.
> > Both set in Account settings/Sending mail.
>
> It can write but forces html onto users, which potentially includes jpg
> exploits, png exploits, html exploits, script exploits, font exploits...

What are you talking about? K9 forces HTML on no one, it sends plain
text if you set it to do so.

> Having knocked Android, I haven't found the time to try the latest
> native email app. I'm not expecting a no html option but I'm pretty
> sure it will have some major pluses over k9mail, which was a trade of
> good for bad on Gingerbread.

K9 is not Android, any more than yourfavouriteemailer is Linux. It is a
program that runs on Android. As for being less capable than the native
app, the opposite is the case as it is based on the code from the native
app, but actively developed.


--
Neil Bothwick

Pedestrians come in two types: Quick or Dead.

[Kevin Chadwick]

On Mon, 18 Mar 2013 23:38:11 +0000
Neil Bothwick <ne...@digimed.co.uk> wrote:

> > > K9 Mail can do both plain text and bottom posting.
> > > Both set in Account settings/Sending mail.
> >
> > It can write but forces html onto users, which potentially includes
> > jpg exploits, png exploits, html exploits, script exploits, font
> > exploits...
>
> What are you talking about? K9 forces HTML on no one, it sends plain
> text if you set it to do so.
>

If you receive a html email you have no choice but to execute code to
handle as per my above examples.

> > Having knocked Android, I haven't found the time to try the latest
> > native email app. I'm not expecting a no html option but I'm pretty
> > sure it will have some major pluses over k9mail, which was a trade
> > of good for bad on Gingerbread.
>
> K9 is not Android, any more than yourfavouriteemailer is Linux. It is
> a program that runs on Android. As for being less capable than the
> native app, the opposite is the case as it is based on the code from
> the native app, but actively developed.

Googles mail is part of android and they do maintain it. I maintain
that while k9 has some improvements it also breaks things and I guess
would have not seen light without Googles initial efforts.

[Kevin Chadwick]

On Mon, 18 Mar 2013 19:16:52 -0400
Michael Mol <mik...@gmail.com> wrote:

> >
> > On 03/18/2013 04:38 PM, Kevin Chadwick wrote:
> >>> It can write but forces html onto users,
> >
> > You seem to miss some of the details.
>

> About that. See the attachment. It's a screenshot of the setting in
> K-9 where you can select composition methods. I took the screenshot
> on my own phone. (And then ran it through pngcrush -brute in
> deference to ML bandwidth...)

I knew that perfectly well??

You even missed the quote? I only wrote two lines and you still
missed it never mind the examples I had given in my original mail that
do not only apply to remote content and that you wrongly interpreted.

There is a security saying.

Assumption is the mother of all f....

[Kevin Chadwick]

On Mon, 18 Mar 2013 19:28:04 -0400
Michael Mol <mik...@gmail.com> wrote:

> >
> > Even though it is from a DVD it can be updated just like standard
> > linux. The problem is, if you run out of ram then things get killed.
> >
> >
> >> (Frankly, this sounds quite nice for kiosk environments.)
> >
> > Could be if you have a good enough network connection for Linux
> > kernel updates or cut it right down ;-)
>
> Local gigabit is cheap, and a gigabit connection would transfer the
> image in under a minute. A bit more, of course, if you've got an
> overloaded server being slammed by ten or twenty machines.
>
> (I wonder if one can anycast TFTP on a local segment. Hm. I think you
> could just barely pull it off, since you'd have resolved the layer 2
> address for your syn packet, and that should stick with the
> connection.)

Kiosks are notorious for having difficulty in getting to connections
as there place is determined by other factors. Still it may make a good
choice of OS except for reboot time.

[Neil Bothwick]

On Tue, 19 Mar 2013 00:15:34 +0000, Kevin Chadwick wrote:

> > What are you talking about? K9 forces HTML on no one, it sends plain
> > text if you set it to do so.
> >
> If you receive a html email you have no choice but to execute code to
> handle as per my above examples.

That applies to mails from any software set to send as email, it is not
specific to K9, Android or the price of fish.

> > K9 is not Android, any more than yourfavouriteemailer is Linux. It is
> > a program that runs on Android. As for being less capable than the
> > native app, the opposite is the case as it is based on the code from
> > the native app, but actively developed.
>
> Googles mail is part of android and they do maintain it. I maintain
> that while k9 has some improvements it also breaks things and I guess
> would have not seen light without Googles initial efforts.

Are you referring to the Googlemail or the Mail program on Android, they
are completely different? But I guess there's no defence against such
specific accusations as "it breaks things".


--
Neil Bothwick

Bang on the LEFT side of your computer to restart Windows

[Michael Mol]

On 03/18/2013 08:15 PM, Kevin Chadwick wrote:
> On Mon, 18 Mar 2013 23:38:11 +0000
> Neil Bothwick <ne...@digimed.co.uk> wrote:
>
>>>> K9 Mail can do both plain text and bottom posting.
>>>> Both set in Account settings/Sending mail.
>>>
>>> It can write but forces html onto users, which potentially includes
>>> jpg exploits, png exploits, html exploits, script exploits, font
>>> exploits...
>>
>> What are you talking about? K9 forces HTML on no one, it sends plain
>> text if you set it to do so.
>>
>
> If you receive a html email you have no choice but to execute code to
> handle as per my above examples.

Either you ignored what I said about being able to disable loading
remote content and being able to disable showing inline rich content, or
you're seriously concerned about HTML parser vulnerabilities.

If that's the case, set up a defanging filter for your email.

>
>>> Having knocked Android, I haven't found the time to try the latest
>>> native email app. I'm not expecting a no html option but I'm pretty
>>> sure it will have some major pluses over k9mail, which was a trade
>>> of good for bad on Gingerbread.
>>
>> K9 is not Android, any more than yourfavouriteemailer is Linux. It is
>> a program that runs on Android. As for being less capable than the
>> native app, the opposite is the case as it is based on the code from
>> the native app, but actively developed.
>
> Googles mail is part of android and they do maintain it. I maintain
> that while k9 has some improvements it also breaks things and I guess
> would have not seen light without Googles initial efforts.

I'm really not sure what Google's native client (or K9) breaks. I use K9
because I require GPG support for communicating with one of my clients.

[Michael Mol]

On 03/18/2013 08:05 PM, Kevin Chadwick wrote:
> On Mon, 18 Mar 2013 19:16:52 -0400 Michael Mol <mik...@gmail.com>
> wrote:
>
>>>
>>> On 03/18/2013 04:38 PM, Kevin Chadwick wrote:
>>>>> It can write but forces html onto users,
>>>
>>> You seem to miss some of the details.
>>
>> About that. See the attachment. It's a screenshot of the setting in
>> K-9 where you can select composition methods. I took the screenshot
>> on my own phone. (And then ran it through pngcrush -brute in
>> deference to ML bandwidth...)
>
> I knew that perfectly well??

You say 'It can write but forces html onto users'. So I pointed out
that, no, it doesn't.

So I take it you're complaining that *other peoples'* HTML clients force
HTML on you. That's a complete and total abdication of responsibility on
your part!

You can ignore these people if you wish. You can ignore the HTML parts
of emails if you wish. You can defang incoming emails if you wish. You
have no obligation to do any more than the minimum required for you to
selectively ignore emails with data you don't want.

>
> You even missed the quote?

If you're going to call me out for ignoring things, missing things or
simply not knowing things, please highlight what it is. "the quote"
isn't very enlightening in this context. You have a nasty habit of
referencing things without inlining them or referencing them directly,
and this has gotten in the way of clear communication *multiple* times
over the last week.

> I only wrote two lines and you still missed it

I respond to what's written in the email I'm replying to, because that's
what I've just read, and that's the context of the email.

> never mind the examples I had given in my original mail that do not
> only apply to remote content and that you wrongly interpreted.

Honestly, I never expected you to be up in arms over being exposed to
HTML syntax.

I presumed you were concerned about libpng, libjpeg, swf and gif. I
presumed you were concerned about privacy concerns. Those are what most
people who gripe about HTML email security are concerned with.

Being concerned with HTML syntax is a new one.

Being angry with mail clients for allowing people to send emails you
don't want to read? That'd ridiculous.

>
> There is a security saying.
>
> Assumption is the mother of all f....
>

Try including more context, and I won't have to assume as much or as often.

[Michael Mol]

I was thinking POS-style setups in a makerspace I help with.


If I had to cope with wireless or cellular, and I was seriously
concerned about security on a budget, I'd use an internal USB stick with
a fuse diode to prevent further writing, or an SD card with a similar
fuse tripped. Expire on a schedule. Send updates as replacement data
devices.

[Kevin Chadwick]

> Either you ignored what I said about being able to disable loading
> remote content and being able to disable showing inline rich content, or
> you're seriously concerned about HTML parser vulnerabilities.

You can't disable incoming rich content (which is the important one)
like jpg logos on Android and which was the whole point. Considering
most phones run Gingerbread it should be noted that this practice is
actually rather dangerous.

[Kevin Chadwick]

> If you're going to call me out for ignoring things, missing things or
> simply not knowing things, please highlight what it is. "the quote"
> isn't very enlightening in this context. You have a nasty habit of
> referencing things without inlining them or referencing them directly,
> and this has gotten in the way of clear communication *multiple* times
> over the last week.
>
> > I only wrote two lines and you still missed it
>
> I respond to what's written in the email I'm replying to, because that's
> what I've just read, and that's the context of the email.
>
> > never mind the examples I had given in my original mail that do not
> > only apply to remote content and that you wrongly interpreted.
>
> Honestly, I never expected you to be up in arms over being exposed to
> HTML syntax.
>
> I presumed you were concerned about libpng, libjpeg, swf and gif.

As I clearly said both, but actually less so html. You seem to be under
the impression Androids mail clients let you avoid all that but they do
not. Talk about hitting your head against a brick wall.

> I
> presumed you were concerned about privacy concerns. Those are what most
> people who gripe about HTML email security are concerned with.

That would be to do with scripts and remote content.

Remote content Is as you have said almost always switchable and so was
not a concern/thought of mine but yes, what people shout about. Scripts,
well with Googles love of javascript (for obvious tracking reasons) I
wouldn't be too surprised if that is enabled without recourse on
android email.

[Michael Mol]

On 03/19/2013 05:09 PM, Kevin Chadwick wrote:
>> If you're going to call me out for ignoring things, missing things or
>> simply not knowing things, please highlight what it is. "the quote"
>> isn't very enlightening in this context. You have a nasty habit of
>> referencing things without inlining them or referencing them directly,
>> and this has gotten in the way of clear communication *multiple* times
>> over the last week.
>>
>>> I only wrote two lines and you still missed it
>>
>> I respond to what's written in the email I'm replying to, because that's
>> what I've just read, and that's the context of the email.
>>
>>> never mind the examples I had given in my original mail that do not
>>> only apply to remote content and that you wrongly interpreted.
>>
>> Honestly, I never expected you to be up in arms over being exposed to
>> HTML syntax.
>>
>> I presumed you were concerned about libpng, libjpeg, swf and gif.
>
> As I clearly said both, but actually less so html. You seem to be under
> the impression Androids mail clients let you avoid all that but they do
> not. Talk about hitting your head against a brick wall.

I can't tell any more whether you're complaining about people sending
HTML, whether you're complaining about receiving HTML emails without
being able to avoid parsing them, or whether you're complaining about
other people receiving HTML emails and their being placed at risk of
parsing bugs as a result.

If you're complaining about other people sending HTML emails: OK, fine.
Politely point out to them that it's common courtesy not to send HTML
emails. PLONK them if you need to. But make it clear this is what you're
complaining about. I don't see the relevance of most of your arguments
if your complaint is with other people sending HTML messages.

If you're complaining about receiving HTML emails without being able to
avoid parsing them: You're clearly technical enough to implement some
solution to avoid it. One solution would be to grab the source of an
existing mail client and patch it to not handle the HTML parts. Another
solution would be to have your mail pass through a server which strips
messages of those parts, or modifies them in some way to make them safe.
Yet another solution would be to find a mail client which does this for
you. I see no reason to continue raging about the state of the mail
clients you use, if this is your argument.

If you're complaining about other people receiving HTML emails and their
being placed at risk of parsing bugs, then provide a solution (I
detailed a few in the above paragraph) and allow them to adopt it if
they wish.

If what you're complaining about isn't enumerated above, please try to
state it simply and clearly.

>
>> I
>> presumed you were concerned about privacy concerns. Those are what most
>> people who gripe about HTML email security are concerned with.
>
> That would be to do with scripts and remote content.
>
> Remote content Is as you have said almost always switchable and so was
> not a concern/thought of mine but yes, what people shout about. Scripts,
> well with Googles love of javascript (for obvious tracking reasons) I
> wouldn't be too surprised if that is enabled without recourse on
> android email.

I'm pretty sure I've never seen JS in email. Traditionally, tracking is
done with image bugs. There's little to no point in using scripting in
emails. And given Google is pushing as fast as they can away from RSS
and toward Google+, I'm rather expecting them to look for ways to get
away from email and XMPP, too.

Further, most GMail users use the web interface; there's No Way In Hell
Google would allow mail-delivered code to be executed from within that
security context. That would be the fastlane to account hijacking.

This argument boils down to: "I don't trust Google, so I'd like to
suggest they would use JS in emails, because that's scary, too."