freebsd-chat-digest V5 #702
owner-freebs...@freebsd.org
freebsd-chat-digest Wednesday, February 12 2003 Volume 05 : Number 702
In this issue:
Re: Email push and pull (was Re: matthew dillon)
Re: Bugzilla? (was Re: Okay, I think I need some serious introduction ;-)
Re: Modelling complexity (was: Re: matthew dillon)
Re: Email push and pull (was Re: matthew dillon)
Re: Email push and pull (was Re: matthew dillon)
Fwd: Flash on FreeBSD
[private and confindential]
Re: Email push and pull (was Re: matthew dillon)
----------------------------------------------------------------------
Date: Wed, 12 Feb 2003 12:58:40 +0100
From: Brad Knowles <brad.k...@skynet.be>
Subject: Re: Email push and pull (was Re: matthew dillon)
At 7:49 PM -0800 2003/02/11, Terry Lambert wrote:
> Why did you cut out my arguments against actually doing it, in your
> reply?
I wasn't disagreeing with them, it's just that this is an issue
that I think is also important, and you hadn't addressed it.
>> Why not just give everyone in the world an IMAP account on your
>> mail server and make everyone use shared folders?
>
> Security. Privacy. Concurrent access. Storage. Backup.
>
> You can actually get around all but "storage" and "backup", and
> those can be brute-forced.
There's no way to get around the storage issue. This stuff has
to be stored somewhere, and if we're not allowed to store it on the
machine owned by the recipient, then we have to store it on the
machine owned by the sender. Thus, we create huge SPOFs throughout
the system.
> But you have to be willing to change
> IMAP4 semantics, slightly.
In what way does IMAP4 need to be changed?
>> No, not really. Besides, we all know how poorly LAN e-mail
>> packages scale. Been there, done that.
>
> Just because the code you are familiar with was written by idiots,
> doesn't tar all code with the same brush. 8-).
I've seen quite a few LAN e-mail systems. Not a one of them
scaled. They were all written under many false assumptions.
I'm not saying all code is bad, but I have seen enough LAN e-mail
systems to say that, in general, they don't scale.
>> Again, not really. Nice idea, but the OP said that the messages
>> themselves were never sent, only notices -- the message bodies would
>> then be retrieved via a pull mechanism.
>
> From a flood-filled, replicated store. I didn't say that the nodes
> containing the message replicas had to be local to the recipient. The
> usenet model is a good flood-fill replicated transport model.
No, a flood-fill replicated store can't be used for the actual
messages. The OP stipulated that they were stored only on the system
belonging to the sender. You could flood-fill the notices all you
want, but if the original system on which this stuff is stored goes
tango-uniform, then you are well and truly screwed.
Of course, if you flood-filled the notices, you'd have to find
some way to force everyone to actually subscribe to the appropriate
newsgroups or something, so that they'd actually see the notices.
- --
Brad Knowles, <brad.k...@skynet.be>
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
-Benjamin Franklin, Historical Review of Pennsylvania.
GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)
------------------------------
Date: Wed, 12 Feb 2003 09:21:59 -0500
From: Rahul Siddharthan <rs...@online.fr>
Subject: Re: Bugzilla? (was Re: Okay, I think I need some serious introduction ;-)
Mark Murray said on Feb 12, 2003 at 07:59:04:
> PLEASE REMOVE ME FROM THE CC!!
How about the following: maybe some procmail guru can improve it but
it should do what you seem to want, ie, keep the list copy but reject
any freebsd-chat-addressed mail that did not arrive via the list.
Trimming obviously irrelevant cc's is good, but you can't expect
people to always do it when the normal practice is to retain
cc's (necessary since the list policy allows posting from
non-subscribers).
- - R
CHATFOLDER=$MAILDIR/freebsdchat # define this as you like
:0:
* ^(Delivered-To:).*(freebs...@freebsd.org)
| $FORMAIL -A"X-Sorted: Bulk" >>$CHATFOLDER
:0:
* ^(To:|Cc:).*(ch...@freebsd.org)
| $FORMAIL "X-Sorted: Bulk" >>/dev/null
:0:
* ^(To:|Cc:).*(freebs...@freebsd.org)
| $FORMAIL "X-Sorted: Bulk" >>/dev/null
------------------------------
Date: Wed, 12 Feb 2003 08:51:02 -0800
From: Terry Lambert <tlam...@mindspring.com>
Subject: Re: Modelling complexity (was: Re: matthew dillon)
Dave Hayes wrote:
> Terry Lambert <tlam...@mindspring.com> writes:
> > Dave Hayes wrote:
> >> > I understood you perfectly.
> >>
> >> Then why did you paraphrase what I said incorrectly?
> >
> > I paraphrased the *consequences* of what you said, were less than
> > 100% of us to adopt your world view.
>
> Technically, you paraphrased your assumptions about the consequences
> of what you -think- I meant.
Technically, I paraphrased the results of gross calculations
about the consequences of several of the highest probability
interpretations of what you meant, and then stated the common
properties of all interpretations with "reasonably high"
probabilities. 8^p.
> Yes, this is your theory. You still haven't demonstrated how the
> system is currently in a steady state,
I have. You've just refused to accept/consult my references, and
I don't have the time or inclination to teach you mathematics in
this forum.
> why any steady state is desirable,
So that state transitions can be planned, of course. Otherwise,
how do you know in advance whether a particular state transition
will result in greater or lesser entropy?
> nor have you seemed to grasp what I am advocating. This
> could be because I am not explaining it in terms of chaos theory or
> mathematical rigor.
Or even in terms of the desired consequences of behaviour according
to your model, or how such behaviours would be self maintaining,
once established. Feel free to engage in a generalization, and
permit me to decide on my own whether the steady-state you propose
is actually desirable.
> > You hypothesize that all behaviour is tolerable, and should be
> > tolerated (if only by ignoring it).
>
> Not exactly. I maintain that the most efficient change to any
> system you are participating in is the change in your behavior.
> If you try to change other's behavior, it will not work for
> some small select group of people.
I maintain that the most efficient change to any system you are
participating in is the change in the ground rules for the system,
such that, given a system S, the resulting system S' has the
emergent properties that you desire, which the system S did not.
"Give me a lever, and a place to stand, and I shall move
the world!" -- Archimedes
> > Why don't you take your simple system, and posit a strange attractor
> > of a willful disrupter, and see if your system maintains itself in
> > some acceptable range +/- some small value, independent of the
> > amount of force driving the occilations?
>
> Frankly, because people don't all work that way.
They all *do* work that way, as a group; it's called a "mutual
security game".
And since what we are interested in here is the predictability of
group dynamics, given a set of circumstances, such as an external
threat, we can make useful adjustments to the system in question
so that the group utilizing the system as a tool will be able to
deal with the threat.
The most absolutely crude form of this is "list moderation",
but we are engineers, and, as engineers, our tools are better
than a single on/off switch, which we must wield with an iron
fist.
> >> BTW, you cannot prove that you understood what I said and neither
> >> can I. We might agree that you understand, but the possibility
> >> would still exist that you do not.
> >
> > And how is that relevent?
>
> It's always relavent, any time someone is communicating. Failure to
> account for this would be like failure to account for a key piece
> of datum in your experimental verifications. Ignoring it's effect
> is tantamount to religious behavior...for a real scientist.
It's never relevent, in fact. Are you aware of NIM-like games?
Ones such game is called "The Prisoner's Dilemma". Another
version of this game is called "The Iterated Prisoner's Dilemma".
The difference between the two is that in the iterated version,
you cn take playing history into account.
Now consider that the prisoner's in the game have no way of
communicating, apart from knowledge of whether or not their
counterpart has betrayed them, and the final score at the end
of each iteration.
What is the optimal playing stratey for this game?
The answer is called "Modified Tit-For-Tat With Forgiveness"
(you may look this up on Google with search phrase "iterated
prisoner's dilemma").
The reason that this is the optimal playing strategy is that
it uses the outcome of the game over a number of iterations in
order to establish a covert communications channel, and thus
to offer to cooperate to achieve the highest possible score.
This communications channel was not designed into the game; it
is an *emergent property* of the playing strategy, and it's
what makes the playing stragey optimal.
> > If you want to be a Solipsist, then you aren't really going to be
> > useful to anyone else, because your going to act like everyone else
> > is a figment of your imagination anyway. Where is *our* utility in
> > *you* acting like that?
>
> Gee. If you -are- a figment of someone's imagination, would not the
> greatest utility always be in interacting with the person who is
> imagining you? ;)
>
> In fact, if I -was- imagining you, and I gave you such free will,
> belligerence, and intelligence as you display, where would -my-
> utility in that be? By your own standards, wouldn't I rather
> imagine a bunch of people who agreed with everything I said?
>
> Point of fact is, I'm not really a solipsist. Being accused of this is
> the method most use to convienently misunderstand what I am saying.
Your argument was Solipsist in nature. In fact, if I were to
ascribe a philosophy to you from my own opinion, it would be
"contratian constructivist", where you insist everyone argue
with you on the basis of first principles, but then refuse to
give the same courtesy to the person you are arguing with, so
as to try to make them "work for it". Thus the easiest path
becomes agreeing with you, and getting you to accept any points
which are not in correspondance with your worldview is a Sisyphean
task. People must then weigh their attachment to their own
philosophy vs. the amount of effort required to overcome yours;
this approach is aided by the fact that, at any perceived mistake
on their part, you force a return to first principles (disallowing
return to a checkpoint), AND you will not go away until you
brow-beat them into your point of view.
Vis-a-vis, your insistance on picking up an argument that was
ended over six months ago.
That is, of course, just my opinion... ;^).
> >> > By designing the situation, you design the constraints, and therefore
> >> > you limit the possibilities for behaviour.
> >>
> >> Now you know why I don't take most of your analogies seriously. =)
> >
> > I have no idea.
>
> You design analogies to represent a concept you are trying
> to substantiate. Unfortunately, you design the situation and the
> constraints, which limit the possibilities for behavior to that
> which support your concept.
Yes, of course. To do otherwise would be to argue from the
specific to the general, which is a logical fallacy. I argue
only in terms of the limit of the problem space currently
under consideration. It is an engineering approach to arguement:
I do not like to solve a member of a class of problems more than
once, if I can avoid it, so I will determine the boradest possible
class which is capable of containing the problem statement, and I
solve that, instead.
Engineers will generally take this approach, if they are given
free rein to solve a problem: it's precisely why an engineer
will spend days writing and perfecting a shell script to perform
some function which the engineer could have performed manually
in less or equal time. By creating a tool suited to the problem,
instead of merely immediately addressing the problem, when or if
they or someone else are faced with the same problem in the future,
they can treat it as already solved.
If the have generalized sufficiently, then the same solution applies
to a much larger class of problems than the original subset being
solved for by the original engineer.
The primary difference between a good engineer and a bad engineer
is whether they solve the immediate problem, or if they solve the
general problem, when given a problem to solve.
When interviewing someone, I always pose at least one problem with
an obvious general solution that is no less efficient than several
"brute force" solutions. If the candidate picks one of the "brute
force" solutions, then I know that they will be "Mr. Right Now",
rather than "Mr. Right", and any code they write will have to be
revisited later on, as the product, specifications, or user
expectations for the product, evolve. I will hire this person if
I need someone to create prototypes, and there is no better candidate,
but not otherwise.
Likewise, asking a question where the implementation cost is
higher for the more general solution, if someone comes up with
the more general solution, I will ask them for possible solutions
with quicker times to implement. If they cannot give them, then
I will hire this person if the specifications are written in stone,
but not if I need an agile team member capable of rapidly creating
prototypes.
Some people can do both; even people you would pidgeon-hole as one
type or the other, based on past experience, can surprise you, if
you interview them this way. Generally speaking, in that case,
your past experience is based on them being forced into one role
or the other, because your team is otherwise unblanced. If you
find this polarization happening, even with a number of people
who can operate in both modes, then it's probably management that's
at fault (e.g. setting unrealistic deadlines, goals, milestones, or
specifying overy-large feature sets). Pointing the finger at
management is always the last resort, in most cases, and as long as
management feels confortable managing the people friction, they will
often not do anything about the problem. But the organization, as a
whole, will not be as efficient, effective, or have as high an
economic return as it would otherwise.
Time to recommend another book, for people who are recognizing
their own organization in this discussion:
Out of the Crisis
W. Edwards Deming
MIT Press
ISBN: 0262541157
This book deals with organizations which engage in the practice of
"crisis managment", rather than running smoothly and effectively.
> > Is it just that you can't follow the mathematics behind emergent
> > properties of chaotic systems, or is it just that you disagree that
> > the chaotic systems can be constrained in such a way as to have a
> > particular desirable set of emergent properties?
>
> Neither. I disagree that you can usefully model the world of
> people's interactions mathematically. If you can do this,
> why aren't you playing the stock market and getting so rich
> that you can buy my agreement with your principles? ;)
Modelling the stock market is not the same as modelling group
behaviour. It's well known that you can model the stock market
nearly perfectly. The formula is called the Black, Scholes, and
Merton formula. Using this formula to price options, you can
subtract risk out (or determine the level of acceptable risk,
and the risk/reward ratio you are willing to accept t be paid
for taking the risk).
As for bribing you, I'm not so naive to believe I would be buying
agreement, so much as I would be buying lack of public dissent.
8-).
> > No. You are mistaking my use of the word "law" for the common use
> > of the word "rule", as in "some arbitrary value whose compliance is
> > enforced by a larger society". A "law" is "the way things are". For
> > example, it is a "law" on the Internet that you can only send packets
> > with certain protocol number values (IP, IPv6, ICMP, etc.). This is
> > "the law", because no matter how hard you try to send packets with
> > protocol numbers other than that, they will not get from point A to
> > point B over an arbitrary segment of the Internet reliably, because
> > the core routers will simply discard, not forward, them. That's
> > "just the way things work": it is a law. Like gravity.
>
> So I hack into a router, upload a new core image, and now it transmits
> what I want. Yet gravity isn't something I can hack into and change.
> That difference was what I was attempting to describe.
You would have to hack all routers intermediate to point A and B,
and all routers for which there was an equal cost for intermediate
hops, for all intermediate alternatives intermediate to A and B,
and you would have to hack all routers that may be involved in
failover, should you wish to maintain end-to-end communication in
failure situations in which end-to-end communication would normally
be maintained with unhacked routers.
At which point, you might as well buy Cisco and make them do it,
or take over the government, and issue a DOD "request" that Cisco
do it, etc..
And at that point, you are competent enough to compete and win in
the market place yourself, and competent people don't need to hack
systems, because when they want them, they build them (or buy them).
That's why you hardly ever see a competent act of terrorism: if they
are competent enough, society will pay them more for playing the game
than they could ever obtain by willfully refusing to play the game.
> Or I come up with an RFC for DaveP, write a sample implementation,
> demonstrate that it is more efficient than IP, go through the
> standards process, get a few sites to run it, the router mfrs add code
> for it, and now I can send packets without protocol values.
Yes, you could do that. It would take at least as long to get
deployed as IPv6. Which is why I suggested that you fix the
FIN-WAIT-2 bug, while you were at it. 8-).
> The point is, it's less of a law than gravity.
Actually, it's not. Until there is an alternative to falling when
you step off a cliff, falling is your only option: that's the law.
After millions of years of evolution, and 12,000 years of civilization,
and 6,000 years of recorded history, THEN you get the option of having
a hang-glider hold you up.
For DaveP, you will need to gain something that you cannot "brute
force" this way: you will need willing cooperation.
> Popping the stack here, email standards are difficult, but not
> impossible, to change.
Go to http://www.imc.org/ and start today, if you want us to have
different laws. Just be sure those laws have the emergent
properties you want the final system to exhibit, before you
carve them in stone. 8-).
> Gravity is extremely difficult to change, if not impossible. Email
> standards are relatively new, Gravity is fairly everpresent in the
> timeline of human endeavor.
Some people would argue about gravity. It depends on what you
mean when you say you are "defying" it, whther you meant the
underlying fundamental principle, or whether you mean one of
the emergent properties of the underlying fundamental principle.
For example, it's possible to redesign the "ball in the falling
elevator" experiment so that you *can* tell you are freefalling
in a gravitational field, rather than being out in space. 8-).
> Popping back to the original frame, it's in our power to change how
> email works if we don't like how it works now. Complaining about how a
> strange attractor of a willful disruptor (which actually comes into
> being _because_ of how people are) is less useful than actually
> figuring out how to make the email system better.
Well, I was never aruing about that, per se, I was arguing about
how to overlay a system in the context of an existing system, in
order to damp the effects of the strange attractor.
When we talk about SPAM countermeasures, like blacklisting, what
we are talking about is active damping, in place of replacing the
system with a system which has an emergent property of passive
damping.
You argue that active damping is evil; my response is "replace
the system with one that contains passive damping as an emergent
property, and we will stop active damping".
It's incumbent upon you, as the person who wants to prevent the
*natural and predictable* response to the strange attractor of
tighter controls on identity and content, to provide an alternative
in the form of systems engineering on the underlying transport
system. Because as it stands, you aren't convincing anyone.
> Finally, asserting that a particular technology _which we have control
> over_ demands that people behave in a certain manner is naive and
> borders on the dishonorable. We can change the technology to adapt
> to how people behave and limit "damage". For any arbitrary value
> of damage, this has a better chance of working than demanding
> that people change their behavior.
We have such a technology, already widely deployed in the world:
we call it "government".
If you don't like it, engineer a different system, and convince
the world to deploy it, in place of the current system.
But any feedback system, even a passive one "built into the rules"
and operating "because that's the way things are" is going to
"demand" people's behaviour conform to certain normative values
within the tolerances of the system. Or the people exhibiting
that behaviour *will be* damped.
- -- Terry
------------------------------
Date: Wed, 12 Feb 2003 09:17:23 -0800
From: Terry Lambert <tlam...@mindspring.com>
Subject: Re: Email push and pull (was Re: matthew dillon)
Brad Knowles wrote:
> At 7:42 PM -0800 2003/02/11, Terry Lambert wrote:
> > Actually, I disagree with this presentation. There is an assumption
> > of "sufficient storage" there in the first place, when in practice,
> > sufficient storage is commonly a prime economic issue. If people
> > were satisfied with the job "the big providers" were doing, there
> > would be no little providers.
>
> In practice, the sole limiting factor of mail systems is
> synchronous meta-data update latency and related I/O throughput.
> You're far, far better off getting large numbers of smaller disks and
> putting them in a RAID 1+0 environment (or even RAID 1+0+0), with a
> large enough stripe size that almost all transactions can be taken
> care of as one logical operation.
In terms of I/O throughput, you are right.
But we are not interested in I/O throughput, in this case, we
are interested in minimizing dynamic pool size, for a given
pool retention time function, over a given input and output
volume.
It's best if you consider each POP3 maildrop as a queue.
> This is very much like running a large-scale USENET news server.
> The total quantity of disk space is meaningless, if you don't have
> enough heads working for you in parallel.
The Usenet parallel is probably not that apt. Usenet provides
an "Expires:" header, which bounds the pool retention time to a
fixed interval, reagardless of volume.
In the POP3 case, the volume is bounded, reagardless of time. It
is a different problem, entirely, in terms of email servers which
contain maildrops.
I think the problem is that you are trying to apply transit server
mechanics to what is, effectively, a destination queue, as opposed
to a destination maildrop.
Even if it were a maildrop, if you enforce a quota, you can make
the argument for fixed time: once the pool fills (the maildrop
goes over quota), then the messages pile up in the main mail
queue (without an MDA reservation protocol, anyway), and the
time a message may remain in the main queue undelivered is not
relevent to volume, but the volume is bound to the quota. There
is just an added element of hysteresis in the main queue retention
limit (e.g. 4 days). Even so, this is only effective is you can
guarantee delivery will occur through the maildrop being brought
below quota in that interval -- which you can not do, since that
is not under your control [ Actually, I'd argue that your queue
retention time would have to be in excess of twice the maximum
polling interval, plus one, for it to become a factor again ].
> So, to solve the disk space issue, you just buy some slightly larger disks.
>
> Been there, done that. We learned this lesson a long time ago at
> AOL. Doing single-instance-store is a false economy, and indeed is
> one of the key limiting factors for LAN e-mail packages like cc:Mail,
> Lotus Notes, Microsoft Mail, or Microsoft Exchange. This is one of
> the primary reasons why they don't scale.
Again, I disagree. Poor design is why they don't scale.
> > As to the metadata updates: I'm not positive this is the case,
> > though it is certainly the case in "sendmail" and certain other
> > MTAs. It's really implementation dependent, I think, and the
> > slide assumes a particular implementation.
>
> These slides have absolutely nothing whatsoever to do with the
> MTA. They have to do with the mailbox, mailbox delivery, mailbox
> access, etc.... You need message locking in some fashion, you may
> need mailbox locking, and most schemes for handling mailboxes involve
> either re-writing the mailbox file, or changing file names of
> individual messages (or changing their location), etc.... These are
> all synchronous meta-data operations.
You do not need all the locking you state you need, at least not
at that low a granularity. If you want to talk AOL, AOL used to
use VMS systems, which used record locking.
> See
> <http://www.shub-internet.org/brad/papers/dihses/lisa2000/sld062.htm>
> through
> <http://www.shub-internet.org/brad/papers/dihses/lisa2000/sld081.htm>.
> Then ask yourself how these issues relate to the operation of
> UW-IMAP, Courier-IMAP, and Cyrus.
I will have to examine these references at a later time.
> If you want a detailed discussion of these points, I'll be more
> than happy to get into this.
...and then decide on this.
> However, keep in mind that I've already had a deep discussion of
> these points with Nick Christenson (author of the book _Sendmail
> Performance Tuning_,
Sendmail performance tuning is not the issue, although if you
are a transit server for virtual domains, you should rewrite the
queueing algorithm. See:
ftp://ftp.whistle.com/pub/misc/sendmail/
> as well as the classic paper "A Highly Scalable
> Electronic Mail Service Using Open Systems" at
> <http://www.jetcafe.org/~npc/doc/mail_arch.html>, among others) and
> other large-scale Internet e-mail experts, and I don't think that
> we're likely to add much to this topic here.
The Open Source book is wrong. You can not build such a system
without significant modification. My source tree, for example,
contains more than 6 million lines of code at this point, and
about 250,000 of those are mine, modifying Cyrus, modifying
OpenLDAP, modifying Sendmail, modifying BIND, etc..
> > The scaling issue at 150 is an implementation detail specific to
> > the implementations you are referencing. My advice is "get some
> > real software". 8-). You can scale something like this from
> > Open Source components and about 3 months of concentrated coding
> > to at least 50,000 per indivisible component cluster, and then
> > throw hardware at it.
>
> Concentrated coding? Open source? Where? What specific pieces
> of software are you envisioning? Who would be doing the coding? If
> it's that simple, then why hasn't this code already been written?
Because Open Source projects are inherently incapable of doing
productization work. It's antithetical to their nature. They
are also generally incapable of doing systems integration. That's
also antithetical to their nature, since systems integration
required metacooperation between projects.
This is why commercial software really has nothing to fear from
pure Open Source.
> I'm trying to build a LAN e-mail system today using open source
> software because certain constraints have been put on this project
> (e.g., they have literally $0 to spend on new hardware, mailboxes
> must be stored on NFS, the old multi-purpose machines must be
> replaced and a new mail system must be able to gradually take over),
> and I was painted into a corner before I ever became a member of this
> project.
$0 is not really true. They are paying for you, in the hopes
that it will end up costing less than a prebuilt system.
> I've done the best I can in the circumstances available, but I am
> still very, very concerned that given the best available solutions I
> can find for each of the components involved, the replacement still
> is not going to perform well enough to be considered anything other
> than "b0rken".
Contact Stanford, MIT, or other large institutions which have
already deployed such a system.
> > The normal implementation to avoid single point of failure is
> > replication of the data. At that point, the replication
> > protocol itself becomes (essentially) a transport. The worst
> > case failure mechanism, in that case, is a previously deleted
> > message reappears.
>
> Correct, in theory. Where's the practice?
Not in Open Source; Open Source does not perform productization or
systems integration.
> > I think they do it on the basis of their name, and on the basis
> > of the idea that "you can put any SQL server behind MS Exchange,
> > so use ours, it's better than Microsoft's".
>
> Uh, no. Take a closer look at the pitch. They are completely
> and totally replacing Exchange. There are no Microsoft components
> left. The architecture is totally unlike Exchange. The only thing
> they're doing is pitching a solution that is compatible with Exchange
> at the highest protocol levels, enough to fool Outlook.
I was unaware of this from the pitch on the billboards and WSJ
advertisements. I rather expect anyone buying it will make the
same assumptions I did, that when they were talking about making
"MS Exchange Rock Solid", they were talking about leaving it there.
8-).
- -- Terry
------------------------------
Date: Wed, 12 Feb 2003 09:26:20 -0800
From: Terry Lambert <tlam...@mindspring.com>
Subject: Re: Email push and pull (was Re: matthew dillon)
Brad Knowles wrote:
> > You can actually get around all but "storage" and "backup", and
> > those can be brute-forced.
>
> There's no way to get around the storage issue. This stuff has
> to be stored somewhere, and if we're not allowed to store it on the
> machine owned by the recipient, then we have to store it on the
> machine owned by the sender. Thus, we create huge SPOFs throughout
> the system.
"all but storage and backup"...
> > But you have to be willing to change
> > IMAP4 semantics, slightly.
>
> In what way does IMAP4 need to be changed?
Add domain support. There is no support in it in the login
process, which asks only username and password. The trick of
specifying "username@domain" will not work for a number of mail
clients, such as some versions Netscape, strip trailing "@*"
before sending it to the server, on the assumption that they
know better than you do.
> >> No, not really. Besides, we all know how poorly LAN e-mail
> >> packages scale. Been there, done that.
> >
> > Just because the code you are familiar with was written by idiots,
> > doesn't tar all code with the same brush. 8-).
>
> I've seen quite a few LAN e-mail systems. Not a one of them
> scaled. They were all written under many false assumptions.
>
> I'm not saying all code is bad, but I have seen enough LAN e-mail
> systems to say that, in general, they don't scale.
Well, the Open Source answer would be "Send Patches, and if they
don't step on anyone's toes, we'll think about commiting them".
> >> Again, not really. Nice idea, but the OP said that the messages
> >> themselves were never sent, only notices -- the message bodies would
> >> then be retrieved via a pull mechanism.
> >
> > From a flood-filled, replicated store. I didn't say that the nodes
> > containing the message replicas had to be local to the recipient. The
> > usenet model is a good flood-fill replicated transport model.
>
> No, a flood-fill replicated store can't be used for the actual
> messages. The OP stipulated that they were stored only on the system
> belonging to the sender. You could flood-fill the notices all you
> want, but if the original system on which this stuff is stored goes
> tango-uniform, then you are well and truly screwed.
Well, the "OP" in this case is actually a reference to a DJB
document, and what we're interested in here is solving the
problem. We don't have to accept both the data store and the
"don't send messages" parts of the document, all or nothing.
> Of course, if you flood-filled the notices, you'd have to find
> some way to force everyone to actually subscribe to the appropriate
> newsgroups or something, so that they'd actually see the notices.
No, you'd direct-send notices. You'd flood-fill the messages
the notices referred to, to address the scalability and availability
problems, and to partially address the "revisionist history" and
"delete before receipt" problems.
- -- Terry
------------------------------
Date: Wed, 12 Feb 2003 21:16:59 +0200
From: Alexandr Kovalenko <ne...@nevermind.kiev.ua>
Subject: Fwd: Flash on FreeBSD
I'd urge anyone, who is interested in native flash on FreeBSD, to fill
out with request form. It should help, as it was with nVidia drivers.
- ----- Forwarded message from MikkelFJ <mikkelfj-...@bigfoot.com> -----
Date: Thu, 13 Feb 2003 03:44:38 +0900
From: "MikkelFJ" <mikkelfj-...@bigfoot.com>
To: ruby...@ruby-lang.org (ruby-talk ML)
Subject: Flash GUI on Ruby vs. FreeBSD
Recently there were yet another Ruby Flash discussion.
It was mentioned that Flash was not supported on FreeBSD.
So I asked their customer support about the status.
Here is their answer:
<snip>
At the moment there are no plans to compile the Macromedia Flash Player for
FreeBSD. However, we do offer licensing for developers if they are
interested in compiling the source code on their OS of choice. This page has
more information on licensing the source code:
<http://www.macromedia.com/software/flashplayer/licensing/sourcecode/>
We also have an open SWF forum for developers:
<http://webforums.macromedia.com/openswf/categories.cfm?catid=221&zb=4438252
>
The best thing to do would be to fill out the Feature Request form here:
http://www.macromedia.com/support/email/wishform/
and have other FreeBSD users do the same.
The engineers view the wishlist regularly and would consider porting the
player to UNIX if enough users request for it.
</snip>
Mikkel
- ----- End forwarded message -----
- --
NEVE-RIPE, will build world for food
Ukrainian FreeBSD User Group
http://uafug.org.ua/
------------------------------
Date: Thu, 13 Feb 2003 22:53:20 +0000
From: <jame...@netscape.net>
Subject: [private and confindential]
This is a multi-part message in MIME format.
- ------=_NextPart_000_01BC2B74.89D1CCC0
Content-type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: quoted-printable
AMSTERDAM,
THE NETHERLANDS.
CONFIDENTIAL BUSINESS PROPOSAL.
You may be surprised to receive this letter from me
since you do not know me personally. I am Carol
Makali,
the only child of James Makali,the most popular black
farmer in Zimbabwe who was murdered in the land
dispute
in my country. I got your contact
through network online hence decided to write you.
Before the death of my father, he had taken me to
Johannesburg to deposit the sum of US8.6 million
(Eight million, Six Hundred thousand United States
dollars), in one of the private security company,
as he foresaw the looming danger in Zimbabwe this
money was deposited in a box as gem stones to avoid
much demurrage from security company. This amount
was meant for the purchase of new machines and
chemicals for the Farms and establishment of new
farms in Swaziland.
This land problem came when Zimbabwean President Mr.
Robert Mugabe introduced a new Land Act Reform
wholly affecting the rich white farmers and some
few black farmers, and this resulted to the killing
and mob action by Zimbabwean war veterans and some
lunatics in the society. In fact a lot of people
were killed because of this Land reform Act for
which my father was one of the victims.
It is against this background that, I and my mother
fled Zimbabwe for fear of our lives and are
currently staying in the Netherlands where we are
seeking political asylum and moreso have decided to
transfer my father’s money to a more reliable
foreign account. since the law of Netherlands
prohibits a refugee (asylum seeker) to open any
bank account or to be involved in any financial
transaction throughout the territorial zone of
Netherlands, As the only child of my father, I am
saddled with the responsibility of seeking a genuine
foreign account where this money could be transferred
without the knowledge of my government who are bent
on taking everything we have got. The South African
government seems to be playing along with them.
I am faced with the dilemma of moving this amount of
money out of South Africa for fear of going through
the same experience in future, both countries have
similar political history. As a businesslady,I am
seeking for a partner who I have to entrust my
future and that of my family in his hands, I must let
you know that this transaction is risk free. If you
accept to assist me and my mother, all I want you to
do for me, is to make an arrangements with the
security
company to clear the consignment(funds) from their
afiliate
office here in the Netherlands as i have already
given
directives for the consignment to be brought to the
Netherlands from South Africa.But before then all
modalities will have to be put in place like change
of ownership to the consignment
and more importantly this money I intend to use for
investment.
I have two options for you. Firstly you can choose
to have certain percentage of the money for
nominating
your account for this transaction. Or you can go into
partnership with me for the proper profitable
investment
of the money in your country. Whichever the option
you
want, feel free to notify me. I have also mapped out
5%
of this money for all kinds of expenses incurred
in the process of this transaction.
If you do not prefer a partnership I am willing to
give you 20% of the money while the remaining 75%
will be for my investment in your country. Contact me
with the above telephone number or my E-mail address
while I implore you to maintain the absolute
secrecy required in this transaction.
Thanks, GOD BLESS YOU
Yours Faithfully,
Carol Makali.
------------------------------
Date: Thu, 13 Feb 2003 00:07:36 +0100
From: Brad Knowles <brad.k...@skynet.be>
Subject: Re: Email push and pull (was Re: matthew dillon)
At 9:17 AM -0800 2003/02/12, Terry Lambert wrote:
> In terms of I/O throughput, you are right.
>
> But we are not interested in I/O throughput, in this case, we
> are interested in minimizing dynamic pool size, for a given
> pool retention time function, over a given input and output
> volume.
Under what circumstances are you not interested in I/O throughput?!?
I have seen some mail systems that were short of disk space, but
when we looked carefully at the number of messages in the mailboxes
and the number of recipients per message, there just wasn't a whole
lot of disk space that we could potentially have recovered. This was
across 100,000+ POP3/dialup users at an earlier time in the life of
Belgacom Skynet, the largest ISP in Belgium.
Virtually all other mail systems I've ever seen have not had disk
space problems (unless they didn't enforce quotas), but were instead
critically short of I/O capacity in the form of synchronous meta-data
updates. This was true for both the MTA and the mailbox storage
mechanism.
> The Usenet parallel is probably not that apt. Usenet provides
> an "Expires:" header, which bounds the pool retention time to a
> fixed interval, reagardless of volume.
Almost no one ever uses the Expires: header anymore. If they do,
it's in an attempt to ensure that the message stays around much, much
longer than normal as opposed to the reverse.
No, what I was talking about was the fundamental fact that you
cannot possibly handle a full USENET feed of 650GB/day or more, if
you don't have enough spindles going for you. It doesn't matter how
much disk space you have, if you don't have the I/O capacity to
handle the input.
> Again, I disagree. Poor design is why they don't scale.
Right, and another outcome of poor design is their stupid choice
of single-instance store -- a false economy.
>> These slides have absolutely nothing whatsoever to do with the
>> MTA. They have to do with the mailbox, mailbox delivery, mailbox
>> access, etc.... You need message locking in some fashion, you may
>> need mailbox locking, and most schemes for handling mailboxes involve
>> either re-writing the mailbox file, or changing file names of
>> individual messages (or changing their location), etc.... These are
>> all synchronous meta-data operations.
>
> You do not need all the locking you state you need, at least not
> at that low a granularity.
Really? I'd like to hear your explanation for that claim.
> If you want to talk AOL, AOL used to
> use VMS systems, which used record locking.
At what time? I worked there from 1995 through 1997, and I
worked very closely with the people who had designed the original
mail system for Quantum Computer Systems. The founding four came
from a company where they had tried to use Tandem computers, and at
that time Tandem couldn't deliver on their fault-tolerant claims.
Contrariwise, Stratus could, so they build the system on Stratus.
It was in 1996 that they started moving everything off Stratus
and onto Unix systems. I know quite a bit about the details of that
mail system, because I was materially involved in the implementation
from the Operations side.
I would be very interested to know at what time they have ever
used any Vax/VMS systems anywhere in the entire history of the
company. I have plenty of contacts that I can use to verify any
claims.
>> However, keep in mind that I've already had a deep discussion of
>> these points with Nick Christenson (author of the book _Sendmail
>> Performance Tuning_,
>
> Sendmail performance tuning is not the issue, although if you
> are a transit server for virtual domains, you should rewrite the
> queueing algorithm.
My point is not that Sendmail is the issue. My point is that
Nick has designed and built some of the largest open-source based
mail systems in the world, and he and I worked extensively to create
the architecture laid out in my LISA 2002 talk.
If you look at <http://www.networkcomputing.com/1117/1117f1.html>
and <http://www-1.ibm.com/servers/esdd/articles/sendmail/>, you will
find virtually the same architecture being used by the major
commercial vendors for their high-volume/high-SLA clients.
> See:
>
> ftp://ftp.whistle.com/pub/misc/sendmail/
This was written for sendmail 8.9.3, way before the advent of
multiple queues and all other sorts of new features. It is no longer
relevant to modern sendmail.
> The Open Source book is wrong. You can not build such a system
> without significant modification. My source tree, for example,
> contains more than 6 million lines of code at this point, and
> about 250,000 of those are mine, modifying Cyrus, modifying
> OpenLDAP, modifying Sendmail, modifying BIND, etc..
IIRC, Nick talks about the changes that were required -- some,
but not excessive. Read the paper.
> Because Open Source projects are inherently incapable of doing
> productization work.
True enough. However, this would imply that the sort of thing
that Nick has done is not possible. He has demonstrated that this is
not true.
Yes, using open source to do this sort of thing can be difficult
(as I am finding out), but it doesn't have to be impossible.
> $0 is not really true. They are paying for you, in the hopes
> that it will end up costing less than a prebuilt system.
It's a different color of money. They had already signed the
contract stating that I would be working for them through April (at
the earliest), before this project was dumped in my lap. So, that's
not anything extra. Buying new machines, or buying software, now
that's spending extra.
> Contact Stanford, MIT, or other large institutions which have
> already deployed such a system.
I've already read much of Mark Crispin's writings. I know how
they did it at UW, and they didn't use NFS. I've read the Cyrus
documentation, and they didn't use NFS either.
That only leaves Courier-IMAP, and while I've read the
documentation they have available, I am finding it difficult to find
anyone who has actually built a larger-scale system using
Courier-IMAP on NFS. Plenty of people say they've heard of it being
done, or it should be easily do-able, but I'm not finding the people
themselves who've actually done it.
>> Correct, in theory. Where's the practice?
>
> Not in Open Source; Open Source does not perform productization or
> systems integration.
Therein lies the problem. You may be able to write or re-write
all of the open source systems in existence, but that sort of thing
is not within my capabilities, and would not be accepted for this
project. They're looking askance at my modifications to procmail to
get it to use Maildir format and hashed mailboxes -- there's no way
they'd accept actual source code changes.
> I was unaware of this from the pitch on the billboards and WSJ
> advertisements. I rather expect anyone buying it will make the
> same assumptions I did, that when they were talking about making
> "MS Exchange Rock Solid", they were talking about leaving it there.
Which is exactly what they want you to think. Up to the point
where they've gotten you to sign the contract.
> 8-).
As you well know. ;-)
- --
Brad Knowles, <brad.k...@skynet.be>
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
-Benjamin Franklin, Historical Review of Pennsylvania.
GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)
------------------------------
End of freebsd-chat-digest V5 #702
**********************************
To Unsubscribe: send mail to majo...@FreeBSD.org
with unsubscribe freebsd-chat-digest in the body of the message