corba or sockets?

423 views
Skip to first unread message

Fernando Rodríguez

unread,
Oct 30, 2000, 5:24:35 AM10/30/00
to
Hi!

I have to comunicate a lisp process with a c++ app and I was
considering corba or using sockets. Both are rather new to me, so wich one
would you recommend and why? O:-)

TIA


//-----------------------------------------------
// Fernando Rodriguez Romero
//
// frr at mindless dot com
//------------------------------------------------

Marc Battyani

unread,
Oct 30, 2000, 6:46:00 AM10/30/00
to

"Fernando Rodríguez" <spa...@must.die> wrote in message
news:hriqvscgjd0pcql56...@4ax.com...

> Hi!
>
> I have to comunicate a lisp process with a c++ app and I was
> considering corba or using sockets. Both are rather new to me, so wich one
> would you recommend and why? O:-)

I use sockets. It works well and is quite simpler to use than corba and the
performance is very good over a 100 Mbit LAN. You just have to be able to do
both sides of the communication. With Corba you can inter-operate with
already existing Corba enabled software. You also need a lisp with Corba.

Marc Battyani

Philip Lijnzaad

unread,
Oct 30, 2000, 6:55:18 AM10/30/00
to

Fernando> Hi!
Fernando> I have to comunicate a lisp process with a c++ app and I was
Fernando> considering corba or using sockets. Both are rather new to me, so wich one
Fernando> would you recommend and why? O:-)

Unless the communication involves simple but big data (kind of like ftp,
say), use CORBA, by a very long shot.
Philip

--
Time passed, which, basically, is its job. -- Terry Pratchett (in: Equal Rites)
-----------------------------------------------------------------------------
Philip Lijnzaad, lijn...@ebi.ac.uk \ European Bioinformatics Institute,rm A2-24
+44 (0)1223 49 4639 / Wellcome Trust Genome Campus, Hinxton
+44 (0)1223 49 4468 (fax) \ Cambridgeshire CB10 1SD, GREAT BRITAIN
PGP fingerprint: E1 03 BF 80 94 61 B6 FC 50 3D 1F 64 40 75 FB 53

Erik Naggum

unread,
Oct 30, 2000, 6:56:44 AM10/30/00
to
* Fernando Rodríguez <spa...@must.die>

| I have to comunicate a lisp process with a c++ app and I was
| considering corba or using sockets. Both are rather new to me, so
| wich one would you recommend and why? O:-)

CORBA is already badly designed, so if you are new to sockets and
protocol design (which you will get yourself into), the likelihood
that you, too, will design your protocol badly is so high that it is
probably better to go with CORBA.

If you are very dedicated, you will probably spend a few months just
getting up to speed with protocol design before you can do something
that is not a total drag, and upwards of a year to get it fully
right. Such braindamaged disasters as HTTP (any version) would not
have been possible if there were at least some reasonably accessible
material on how to do these things.

Security problems in protocols are so common and so hard to avoid
that you _really_ don't want to expose yourself to the process of
learning by doing.

#:Erik
--
Does anyone remember where I parked Air Force One?
-- George W. Bush

Eric Marsden

unread,
Oct 30, 2000, 7:18:52 AM10/30/00
to
CORBA provides a lot more than inter-application communication. It
provides a design for developing distributed applications and useful
services such as naming, trading, event propagation. CORBA is
therefore large, and will require time to understand.


>>>>> "mb" == Marc Battyani <Marc.B...@fractalconcept.com> writes:

mb> With Corba you can inter-operate with already existing Corba
mb> enabled software. You also need a lisp with Corba.

CLORB (under development, but already useful, particularly for
client-side CORBA) works with CMUCL, CLISP, SBCL and ACL. Available
under GNU LGPL.

<URL:http://clorb.sourceforge.net/>

--
Eric Marsden <URL:http://www.laas.fr/~emarsden/>

Erik Naggum

unread,
Oct 30, 2000, 8:22:04 AM10/30/00
to
* Philip Lijnzaad <lijn...@ebi.ac.uk>

| Unless the communication involves simple but big data (kind of like ftp,
| say), use CORBA, by a very long shot.

Really? I'd use CORBA for fairly simple stuff and roll my own if I
get above a fairly low complexity threshold, but I know I'm about
three orders of magnitude better at protocol design than the CORBA
team could ever dream of becoming. For instance, CORBA is a single-
threaded protocol, with lock-step properties where you wait for the
answer before you continue with the next transaction. This is of
course a consequence of serial computations and CORBA being a very,
very slow way of doing serial computations on disjoint processes and
processors.

With all this hoopla about multi-threading and other uses for real
and pseudo-parallelism, you might expect people to think through the
multi-threading implications of their protocols. The hardware folks
are gung-ho about _real_ parallelism, trying very hard to answer the
demand for faster on-chip communication, but then what do we get in
the software world? People who use all this new hardware power only
to wait for ages for incoming data before they turn around _real_
fast to send something that did not even need to wait for that value
and then proceed to wait for ages, again, only because the protocol
is designed by a committee of morons?

If you cannot deal with multiple objects "on the wire" and are not
using the bandwidth at full throttle even while you _are_ waiting
for something to come in, you are not programming in the dynamic,
networked world of the new millennium. (How's that for buzzwords?)

E.g, how many applications do you know which continue to work just
as well if you move one of the computers 3000 miles away? Do you
think CORBA handles this situation well? From what I have seen, the
application grinds to a virtual stand-still. Latencies higher than
a few millisecond cause all sorts of interesting behavior in modern
software. It's pretty pathetic that such things are touted as
workable solutions.

Fernando Rodríguez

unread,
Oct 30, 2000, 8:33:08 AM10/30/00
to
On 30 Oct 2000 11:56:44 +0000, Erik Naggum <er...@naggum.net> wrote:

>* Fernando Rodríguez <spa...@must.die>
>| I have to comunicate a lisp process with a c++ app and I was
>| considering corba or using sockets. Both are rather new to me, so
>| wich one would you recommend and why? O:-)
>
> CORBA is already badly designed, so if you are new to sockets and
> protocol design (which you will get yourself into), the likelihood
> that you, too, will design your protocol badly is so high that it is
> probably better to go with CORBA.

What's so wrong about corba?

Marc Battyani

unread,
Oct 30, 2000, 8:38:55 AM10/30/00
to
"Erik Naggum" <er...@naggum.net> wrote in message
news:31818958...@naggum.net...

> * Fernando Rodríguez <spa...@must.die>
> | I have to comunicate a lisp process with a c++ app and I was
> | considering corba or using sockets. Both are rather new to me, so
> | wich one would you recommend and why? O:-)
>
> CORBA is already badly designed, so if you are new to sockets and
> protocol design (which you will get yourself into), the likelihood
> that you, too, will design your protocol badly is so high that it is
> probably better to go with CORBA.
>
> If you are very dedicated, you will probably spend a few months just
> getting up to speed with protocol design before you can do something
> that is not a total drag, and upwards of a year to get it fully
> right. Such braindamaged disasters as HTTP (any version) would not
> have been possible if there were at least some reasonably accessible
> material on how to do these things.

There are lots of text based protocols (ok it's a weak argument they could
all be just crap). Why are you against simple lisp reader friendly text
based protocols when 1) the task is really easy and 2) you master both sides
of the communication?

I drive lots of measuring instruments from Lisp and they all have text based
protocols.

> Security problems in protocols are so common and so hard to avoid
> that you _really_ don't want to expose yourself to the process of
> learning by doing.

By security do you mean cryptographic security or protocol security (fail
safe, resync, etc.)?
If it's the former then SSL can be used. Or do you know security holes in
SSL ?

Marc Battyani

Michael Livshin

unread,
Oct 30, 2000, 9:19:42 AM10/30/00
to
Erik Naggum <er...@naggum.net> writes:

> Really? I'd use CORBA for fairly simple stuff and roll my own if I
> get above a fairly low complexity threshold, but I know I'm about
> three orders of magnitude better at protocol design than the CORBA
> team could ever dream of becoming. For instance, CORBA is a single-
> threaded protocol, with lock-step properties where you wait for the
> answer before you continue with the next transaction. This is of
> course a consequence of serial computations and CORBA being a very,
> very slow way of doing serial computations on disjoint processes and
> processors.

do you imply that it *was* possible to come up with a better wire
protocol for CORBA as it is, or that the whole concept of "remote
procedure call" (i.e. the CORBA semantics) is not an adequate
abstraction in situations involving non-trivial network latencies?

I would agree with the latter, I guess...

--
(only legal replies to this address are accepted)
newsgroup volume is a measure of discontent. -- Erik Naggum

Erik Naggum

unread,
Oct 30, 2000, 9:29:20 AM10/30/00
to
* "Marc Battyani" <Marc.B...@fractalconcept.com>

| There are lots of text based protocols (ok it's a weak argument they
| could all be just crap). Why are you against simple lisp reader
| friendly text based protocols when 1) the task is really easy and 2)
| you master both sides of the communication?

Can you back up and at least explain _how_ you arrived at the
assumptions that underlie your questions?

Erik Naggum

unread,
Oct 30, 2000, 10:20:33 AM10/30/00
to
* Michael Livshin <mliv...@yahoo.com>

| do you imply that it *was* possible to come up with a better wire
| protocol for CORBA as it is, or that the whole concept of "remote
| procedure call" (i.e. the CORBA semantics) is not an adequate
| abstraction in situations involving non-trivial network latencies?

Yes, that the remote procedure call model is fundamentally flawed.

It is, however, possible to do remote procedure calls intelligently,
but it requires a programming language that can deal with unfinished
computations and actually calculate with them for a while. This is
not terribly difficult stuff, but not trivial, either.

Erik Naggum

unread,
Oct 30, 2000, 10:18:19 AM10/30/00
to
* Fernando Rodríguez <spa...@must.die>

| What's so wrong about corba?

Almost everything. I have sketched how in a response to a another
article. CORB's only redeeming quality is that it is a standard, of
sorts, and facilitates communication with library-based "tools".

Real programmers want something that actually works efficiently, too.

Fernando Rodríguez

unread,
Oct 30, 2000, 10:20:45 AM10/30/00
to
On Mon, 30 Oct 2000 10:24:35 GMT, Fernando Rodríguez <spa...@must.die> wrote:

BTW, what about ILU? O:-)


>Hi!
>
> I have to comunicate a lisp process with a c++ app and I was
>considering corba or using sockets. Both are rather new to me, so wich one
>would you recommend and why? O:-)

//-----------------------------------------------

Eric Marsden

unread,
Oct 30, 2000, 10:34:26 AM10/30/00
to
>>>>> "en" == Erik Naggum <er...@naggum.net> writes:

en> For instance, CORBA is a single- threaded protocol, with
en> lock-step properties where you wait for the answer before you
en> continue with the next transaction.

this is no longer true. While the standard CORBA communication model
is synchonous twoway (the client thread which sends the request is
blocked while waiting for the server to answer), several other types
of communication are possible:

* oneway invocations, where the client doesn't wait for a response

* deferred synchonous, where the client invokes a request and can
poll the server later to see whether a response is available (this
is available by using the dynamic invocation interface)

* over EventChannels specified by the CORBA Event Service, or the more
recent Notification Service. The Event Channel mediates
communication between consumers and suppliers, and allows push,
pull, push/pull and pull/push interactions.

* asynchronous method invocations provided by CORBA Messaging, either
by polling or with callbacks. [This has been specified fairly
recently, so isn't supported by all implementations.]

See <URL:http://www.iona.com/hyplan/vinoski/> for some excellent
papers on these subjects (from the C/C++ Users Journal :-).

Tim Bradshaw

unread,
Oct 30, 2000, 10:48:35 AM10/30/00
to
Erik Naggum <er...@naggum.net> writes:

>
> Yes, that the remote procedure call model is fundamentally flawed.
>
> It is, however, possible to do remote procedure calls intelligently,
> but it requires a programming language that can deal with unfinished
> computations and actually calculate with them for a while. This is
> not terribly difficult stuff, but not trivial, either.
>

I think this is yet another one of those cases where most software
people (clearly not Erik) are living in the world of the PDP11.
Hardware people spend all their time dealing with these issues -- just
look at the design of a modern processor, which is dealing exactly
this kind of thing. Memory access is essentially just like RPC. The
PDP11 approach is: issue a request, wait for a result, do something,
issue another one and so on. Modern machines have to deal with huge
memory latencies, and so do something much more complex and
interesting involving multiple outstanding requests, caching of
requests, and speculatively executing code for which all the
prerequisites are not yet known. And they can make this stuff work on
machines with multiple processors, all doing their own caching, and in
a context where `fixes' to the algorithms can often not be made
because they're cast in silicon.

And to cap it all, they manage to make the resulting design *look like
a PDP11* because they know that's all SW people can deal with!

So I suppose either SW people will have to work out that you can't
program network protocols as if they were memory access on a PDP11, or
some *really* cool HW people will manage to make RPC look like is *is*
memory access on a PDP11!

--tim

William Deakin

unread,
Oct 30, 2000, 11:25:48 AM10/30/00
to
Tim wrote:
> So I suppose either SW people will have to work out that you can't
> program network protocols as if they were memory access on a PDP11, or
> some *really* cool HW people will manage to make RPC look like is *is*
> memory access on a PDP11!

Is this because most SW people are stupid and most HW people are clever?
Is this why SW people like java, C++ ... and not Common Lisp?

;)w

Fernando Rodríguez

unread,
Oct 30, 2000, 12:02:49 PM10/30/00
to
On 30 Oct 2000 15:20:33 +0000, Erik Naggum <er...@naggum.net> wrote:


> It is, however, possible to do remote procedure calls intelligently,
> but it requires a programming language that can deal with unfinished
> computations and actually calculate with them for a while. This is
> not terribly difficult stuff, but not trivial, either.

Could you elaborate on this, with some example, please?

Jason Trenouth

unread,
Oct 30, 2000, 11:59:07 AM10/30/00
to
On 30 Oct 2000 13:22:04 +0000, Erik Naggum <er...@naggum.net> wrote:

> * Philip Lijnzaad <lijn...@ebi.ac.uk>
> | Unless the communication involves simple but big data (kind of like ftp,
> | say), use CORBA, by a very long shot.
>
> Really? I'd use CORBA for fairly simple stuff and roll my own if I
> get above a fairly low complexity threshold, but I know I'm about
> three orders of magnitude better at protocol design than the CORBA
> team could ever dream of becoming. For instance, CORBA is a single-
> threaded protocol, with lock-step properties where you wait for the
> answer before you continue with the next transaction. This is of
> course a consequence of serial computations and CORBA being a very,
> very slow way of doing serial computations on disjoint processes and
> processors.
>

> ...

CORBA calls are synchronous by default. The call site effectively blocks
waiting for a reply even for operations returning void and containing no 'out'
or 'inout' arguments. Part of the reason for this is probably because system
exceptions may be thrown by ORB, or by the Object Adaptor on the serverside (eg
OBJECT_NOT_EXIST), even when no user exceptions have been declared.

Client code that isn't interested in waiting for potential failures can spawn a
thread in which to make the call or could declare the operation as 'oneway' in
IDL.

The CORBA community have recently been defining new asynchronous calling
mechanisms for themselves:

http://cgi.omg.org/cgi-bin/doc?orbos/98-05-05

__Jason

Marc Battyani

unread,
Oct 30, 2000, 1:23:14 PM10/30/00
to

"Erik Naggum" <er...@naggum.net> wrote in message
news:31819049...@naggum.net...

> * "Marc Battyani" <Marc.B...@fractalconcept.com>
> | There are lots of text based protocols (ok it's a weak argument they
> | could all be just crap). Why are you against simple lisp reader
> | friendly text based protocols when 1) the task is really easy and 2)
> | you master both sides of the communication?
>
> Can you back up and at least explain _how_ you arrived at the
> assumptions that underlie your questions?

May be it's an English issue (from me) but when you write:

> CORBA is already badly designed, so if you are new to sockets and
> protocol design (which you will get yourself into), the likelihood
> that you, too, will design your protocol badly is so high that it is
> probably better to go with CORBA.

My understanding of this is that "CORBA is bad" and "you probably won't do
better so use it anyway"

This is why I ask you if you still think so if the protocol is a simple lisp
reader friendly text based protocol when 1) the task is really easy and 2)


you master both sides of the communication?

Marc Battyani

Paolo Amoroso

unread,
Oct 30, 2000, 2:22:49 PM10/30/00
to
On 30 Oct 2000 11:56:44 +0000, Erik Naggum <er...@naggum.net> wrote:

> right. Such braindamaged disasters as HTTP (any version) would not
> have been possible if there were at least some reasonably accessible
> material on how to do these things.

Could you please mention a few examples of well designed protocols that are
worth studying? Thanks in advance.


Paolo
--
EncyCMUCLopedia * Extensive collection of CMU Common Lisp documentation
http://cvs2.cons.org:8000/cmucl/doc/EncyCMUCLopedia/

Will Hartung

unread,
Oct 30, 2000, 3:04:05 PM10/30/00
to

Marc Battyani wrote in message <8tkf8h$4ka$1...@reader1.fr.uu.net>...

>
>May be it's an English issue (from me) but when you write:
>
>> CORBA is already badly designed, so if you are new to sockets and
>> protocol design (which you will get yourself into), the likelihood
>> that you, too, will design your protocol badly is so high that it is
>> probably better to go with CORBA.
>
>My understanding of this is that "CORBA is bad" and "you probably won't do
>better so use it anyway"
>
>This is why I ask you if you still think so if the protocol is a simple
lisp
>reader friendly text based protocol when 1) the task is really easy and 2)
>you master both sides of the communication?


The issue that you are missing is that there is more to a protocol than
simply the data being transmitted. There's also the timing and sequencing
details. You can certainly have a very sophisticated and complicated
protocol based on either a Lisp text stream or a binary stream.

HTTP is, essentially, a text based protocol, but that fact alone doesn't
make it good or bad, just readable to humans.

Regards,

Will Hartung
(will.h...@havasint.com)

Erik Naggum

unread,
Oct 30, 2000, 3:13:52 PM10/30/00
to
* Fernando Rodríguez <spa...@must.die>

| Could you elaborate on this, with some example, please?

Why?

Consider the "normal" way people implement streams, with a single
buffer that is over-written with new data as the reading crashes
into the right buffer wall and then continues from the left end of
the buffer. This is not smart in a modern world.

Instead of crashing into a buffer wall, have several buffers. Ask
the operating system to fill them for you while you're more or less
busy processing the data already read, as in "asynchronously". The
operating system is doing a decent job of predicting your linear
progression through the file, anyway, so it sort of works to crash
into buffer walls and get first aid from the operating system and
get back on your feet, but it is far from optimal, and no matter
what you do with that system call, it at _least_ has to copy the
buffer contents from system memory to user memory, which is also
probably at a really bad alignnment for hardware support for such
copying. Several page-aligned buffers can cause I/O to occur at the
operating system's discretion, copying is not needed if the
operating system can map the page directly into user space, and you
can scan through a large file in no time compared to these false
stops and starts as you crash into buffer walls.

Consider the sending of mail to someone with SMTP. As long as you
have well-defined state transitions that are (fairly) independent,
send a whole bunch of commands down the wire and match the responses
to the requests sent instead of waiting for each response by itself.
Consequence? _Dramatic_ speedup when sending to many recipients.

In normal human life, you don't get paralyzed and die if you don't
an answer to a question, but (carefully) go on with assumptions
because you probably have a pretty good idea what kind of answer
you're going to get. Just how many code paths could you possibly
walk down depending on the answer? Do them all! You certainly have
the time while the bits trickle across the globe.

Erik Naggum

unread,
Oct 30, 2000, 3:17:42 PM10/30/00
to
* Eric Marsden <emar...@mail.dotcom.fr>

| this is no longer true.

I appreciate the update.

I _still_ think CORBA sucks, though.

Erik Naggum

unread,
Oct 30, 2000, 3:51:10 PM10/30/00
to
* William Deakin <w.de...@pindar.com>

| Is this because most SW people are stupid and most HW people are clever?
| Is this why SW people like java, C++ ... and not Common Lisp?

Is that a SW person or a HW person asking?

Erik Naggum

unread,
Oct 30, 2000, 3:31:36 PM10/30/00
to
* "Marc Battyani" <Marc.B...@fractalconcept.com>

| This is why I ask you if you still think so if the protocol is a
| simple lisp reader friendly text based protocol when 1) the task is
| really easy and 2) you master both sides of the communication?

The likelihood that novices will do worse than CORBA is still very,
very high. Look at how people design their input languages from
simple files that are under total programmer control, and often they
blow it so bad applications crash and burn, a missed version update
causes serious bit rot, and changing your mind about something means
you lose a _lot_ of old information. Using Lisp for this does not
really help.

No networked task is really easy. If it were, you wouldn't be doing
it. If you're still doing it, how could that possibly impact any
decision on how to design protocols for tasks _worth_ doing?

I actually _favor_ Lisp-based protocols, but not because "it's text,
so it's easy" is even close to relevant (it isn't -- syntax is an
important issue to humans -- it is _not_ important to representing
the data), but because it means I have a stable and proven framework
to work from.

It is actually quite important to realize that just because it's
text doesn't make it any less fraught with all the dangers of other
network protocols. For one thing, network communications is subject
to a failure modes that novices never consider. Even with the best
of checksum algorithms, you _may_ get rotten data once in a while.
Deadlocks may occur whether you use a hariy binary protocol or a
Lisp-based protocol. All these things are unlikely to happen if you
run on a local machine, and they don't happen if you work within the
same memory. (Unless you run without error correcting memory, of
course, but then again, if you do that, you have already proclaimed
to the world that you don't care, so nobody else should bother.)

Marc Battyani

unread,
Oct 30, 2000, 4:27:15 PM10/30/00
to

"Will Hartung" <will.h...@havasint.com> wrote in message
news:8tkk7...@news1.newsguy.com...

Sure. I do agree with all this and this is why I think that for easy tasks
(T.B.D.) it's better to use sockets than CORBA (or COM for M$ fans). When I
don't write Lisp, I write VHDL where I implement lots of (fast!) protocols
at the bit/hardware level. Well, it seems that I should communicate in VHDL
or Lisp rather than English today...

Another communication channel I used between applications is mapped memory +
system events. This gives the fastest communication I could achieve between
2 applications.

Marc Battyani

Marc Battyani

unread,
Oct 30, 2000, 4:45:18 PM10/30/00
to

"William Deakin" <w.de...@pindar.com> wrote in message
news:39FDA10C...@pindar.com...

Now that HW is written in VHDL (SW) the frontier is becoming rather fuzzy.
Lisp has been very present in the electronics CAD software industry and the
standard file format (EDIF) should look familiar here. See yourself, here
are the first lines of an EDIF file...

(edif time_sim
(edifVersion 2 0 0)
(edifLevel 0)
(keywordMap (keywordLevel 0))
(status
(written
(timestamp 2000 5 24 12 0 29)
(program "Xilinx NGD2EDIF" (version "C.18"))
(comment "Command line: -w -v fndtn testus.nga time_sim.edn ")))
(external SIMPRIMS
(edifLevel 0)
(technology (numberDefinition
(scale 1 (E 1 -12) (unit TIME))))
(cell x_and3
(cellType GENERIC)
(view view_1
(viewType NETLIST)
(interface
(port IN0
(direction INPUT))
(port IN1
(direction INPUT))
(port IN2
(direction INPUT))
(port OUT
(direction OUTPUT))
....

Looks like HW guys don't need XML though I'm sure some brilliant SW guy will
explain some day that these file should be written in XML to be used by
C#.NET...
When this arrives we could start to have to reboot, several time per day,
our watches, radio receivers, phones, lifts, flash lights,...

Marc Battyani

Marc Battyani

unread,
Oct 30, 2000, 5:15:19 PM10/30/00
to

"Erik Naggum" <er...@naggum.net> wrote in message
news:31819266...@naggum.net...

Ok, I fully agree with this. I missed the emphasis you made on the
difficulty of designing a correct protocol, and I thought you were against
using Lisp based protocols.

Marc Battyani


Erik Naggum

unread,
Oct 30, 2000, 5:41:40 PM10/30/00
to
* Paolo Amoroso <amo...@mclink.it>

| Could you please mention a few examples of well designed protocols
| that are worth studying? Thanks in advance.

TCP and IPv4 are _very_ good protocols. (I don't know enough about
IPv6 to rate it.) The telecommunications protocols used on the
gigabit trunks and on the long-haul high-speed interconnects are
amazing feats of science. Of the really, _really_ large protocol
suites, I find ITU's Signalling System #7, the Digital Subscriber
Signalling System, and the Integrated Digital Services Network
protocols as well-designed as they are complex. The Telecom people
have probably done some of the best work there is in protocol
design. Just take a look at how they started out with very low
speeds, like 300 bps, but over time managed to squeeze 56kbps
through the feeble phone lines that were not upgraded, then got DSL
to work across that same old copper wire. Impresses me, anyway.

FTP and SMTP are well-designed and fairly small protocols, too.
(Small does not mean quick-and-dirty: If you spend less than 3
months implementing a server or client for either of them, however,
you're doing it wrong, as almost every implementation that is not
regarded as a huge monstrosity are.)

As an example of a very good way of separating abstractions and
representation, ASN.1 and its encoding rules are worth a study, such
as what is done with SNMP.

Finally, if you don't know X.25, you don't know packet exchange, but
it's _real_ old, now, and probably doesn't bring anything directly
applicable to people who think in streams implemented on top of
record-based protocols, except what _not_ to do with streams and
when it really does make sense to use records.

_Very_ few of the recent application-level Internet protocols are
worth the paper they aren't published on. The sheer lack of design
behind the protocol that is _actually_ at work in the MIME cruft is
astonishing, but trying to track that braindamaged piece of shit
from its inception to its current specification level through the
numerous RFCs will tell you what a few over-sized egos who had no
clue what they were doing can do to a conceptually _very_ simple
thing: Wrap various objects in a textual representation.

On the other hand, the Network Time Protocol was done by people who
really cared and also who managed to keep the dolts out of the way.

Jon S Anthony

unread,
Oct 30, 2000, 6:10:30 PM10/30/00
to
Erik Naggum wrote:
>
> * Philip Lijnzaad <lijn...@ebi.ac.uk>
> | Unless the communication involves simple but big data (kind of like ftp,
> | say), use CORBA, by a very long shot.
>
> Really? I'd use CORBA for fairly simple stuff and roll my own if I
> get above a fairly low complexity threshold, but I know I'm about
> three orders of magnitude better at protocol design than the CORBA

The real question should be whether you believe your new protocol adds
real value to what it is you are producing. If you are in the
protocol business this is obvious. If you are in something else
entirely and only need to make use of protocols, it is highly unlikely
that the time spent on inventing yet another one will be of any real
value for your project/product.

CORBA is _not_ anything like the best distributed object protocol that
one could define - probably a legacy of the C++ mentality of most of
the original vendors involved. However, it _does_ work rather well
for many scenarios - most any that someone not in the protocol tool
business would care about.


> team could ever dream of becoming. For instance, CORBA is a single-
> threaded protocol, with lock-step properties where you wait for the
> answer before you continue with the next transaction. This is of
> course a consequence of serial computations and CORBA being a very,
> very slow way of doing serial computations on disjoint processes and
> processors.

This has not been true since at least CORBA 2.0. That's not to say
that you have "true" asynchronous communication (request and asynch
notification of completion) out of the box (certainly not in any of
the ORBs we've used). However, you certainly have one way calls
(basically datagrams...) and can also use of these, or more likely
synchronous "posts", to get this. We've done this here and the
results are actually _very_ good.

The nice thing about using CORBA to do this is that it at least
eliminates all the marshalling and unmarshalling of data items
(primitive and structured) in a platform independent way. Unless your
business is really line protocols or something similar, rolling your
own here with sockets is just a distracting waste of resources.


> the software world? People who use all this new hardware power
> only to wait for ages for incoming data before they turn around
> _real_ fast to send something that did not even need to wait for
> that value and then proceed to wait for ages, again,

Only if you don't make use of threads in making the calls in the first
place. If each call is a thread, the "waiting for the synchronous
reply" is irrelevant. Most ORBs are multithreaded and can easily
handle this.


> E.g, how many applications do you know which continue to work just
> as well if you move one of the computers 3000 miles away? Do you
> think CORBA handles this situation well? From what I have seen, the

Well we have done it at 2000 miles and the results were basically
"instantaneous" (well under a second). Using straight X was death.
Using HTTP was (of course) worse than death.

/Jon

--
Jon Anthony
Synquiry Technologies, Ltd. Belmont, MA 02478, 617.484.3383
"Nightmares - Ha! The way my life's been going lately,
Who'd notice?" -- Londo Mollari

Erik Naggum

unread,
Oct 30, 2000, 7:56:07 PM10/30/00
to
* Jon S Anthony <j...@synquiry.com>

| The real question should be whether you believe your new protocol adds
| real value to what it is you are producing. If you are in the
| protocol business this is obvious. If you are in something else
| entirely and only need to make use of protocols, it is highly unlikely
| that the time spent on inventing yet another one will be of any real
| value for your project/product.

I disagree with this assessment. It is the same argument that can
be made for something overly complex and badly implemented, like
SGML, It has also been touted as solving problems that people would
not be able to solve cheaper by themselves. That was just plain
false. Similar problems crop up with other committee products that
have tried to solve overly broad and thus non-existing problems
instead of trying to solve real problems of real complexity.

How much effort does it take a good system designer to come up with
something better than XML for any _particular_ purpose? If he
grasps the good and useful qualities of SGML, a simplified syntax,
slightly general tools, etc, can be written in a fairly short amount
of time, and it costs less to develop, maintain, and use than a
full-fledged SGML system does, because, importantly, SGML's full
generality has surpassed the point where additional features bring
nothing but more costs than benefits to the equation.. A particular
markup language (or something similar) is also easier to learn for
someone who has _not_ spent much too much time studying the original
-- like one one who is only moderately familiar with the original.

| CORBA is _not_ anything like the best distributed object protocol
| that one could define - probably a legacy of the C++ mentality of
| most of the original vendors involved. However, it _does_ work
| rather well for many scenarios - most any that someone not in the
| protocol tool business would care about.

Precisely, but having designed a few protocols that have been in
wide use for over a decade each, just how much effort do you think
it is to design new ones? The reason protocol design is hard is
that people are told not to do it, and thus do not investigate the
issues involved. This also leads to incredibly bad protocols in
use, because the users are told they are too hard to understand.

| This has not been true since at least CORBA 2.0.

I'm "happy" to hear that.

| The nice thing about using CORBA to do this is that it at least
| eliminates all the marshalling and unmarshalling of data items
| (primitive and structured) in a platform independent way.

You know, this is _not_ a hard problem. It is a hard problem in C++
and the like, but that is because those languages have lost every
trace of the information the compiler had when building the code.
That design mistakes has cost the world _billions_ of dollars. If
you don't make that design mistake, you don't have to worry. I
know, this sounds "too good to be true", but the inability of any
small C++ program to read C++ source code and do useful things with
it is the reason for all these weird data-oriented languages.

| Unless your business is really line protocols or something similar,
| rolling your own here with sockets is just a distracting waste of
| resources.

Well, I have tried both approaches. Have you?

| Only if you don't make use of threads in making the calls in the
| first place. If each call is a thread, the "waiting for the
| synchronous reply" is irrelevant. Most ORBs are multithreaded and
| can easily handle this.

Right. Not my experience. Of course, everything gets better in the
mythical "future", so do stay with CORBA if you have other things to
do while waiting for the synchronous request for more features sees
a response from the CORBA people. (Sorry, a tad rambling here.)

| Well we have done it at 2000 miles and the results were basically
| "instantaneous" (well under a second). Using straight X was death.
| Using HTTP was (of course) worse than death.

How long time do you think you would have spent designing a protocol
that would have been just as good?

Christopher Browne

unread,
Oct 30, 2000, 11:12:09 PM10/30/00
to
In our last episode (30 Oct 2000 16:34:26 +0100),

the artist formerly known as Eric Marsden said:
>>>>>> "en" == Erik Naggum <er...@naggum.net> writes:
>
> en> For instance, CORBA is a single- threaded protocol, with
> en> lock-step properties where you wait for the answer before you
> en> continue with the next transaction.
>
>this is no longer true. While the standard CORBA communication model
>is synchonous twoway (the client thread which sends the request is
>blocked while waiting for the server to answer), several other types
>of communication are possible:
>
> * oneway invocations, where the client doesn't wait for a response

Deprecated, due to the availability of AMI...

> * deferred synchonous, where the client invokes a request and can
> poll the server later to see whether a response is available (this
> is available by using the dynamic invocation interface)

> * over EventChannels specified by the CORBA Event Service, or the more
> recent Notification Service. The Event Channel mediates
> communication between consumers and suppliers, and allows push,
> pull, push/pull and pull/push interactions.

Essentially, these two approaches amount to a _simulation_ of
asynchronous communications by using some sort of "proxy" to manage
the connection. They still use synchronous twoway communications; if
whatever server is sitting there ready to accept the requests can
queue them so they _appear_ async, then you get something that _looks_
async.

Reality doesn't change...

> * asynchronous method invocations provided by CORBA Messaging,
> either by polling or with callbacks. [This has been specified
> fairly recently, so isn't supported by all implementations.]

Are there any non-C++ implementations of AMI? All the documentation
I've seen on AMI has been quite tightly tied to C++...

>See <URL:http://www.iona.com/hyplan/vinoski/> for some excellent
>papers on these subjects (from the C/C++ Users Journal :-).

The thing that is irritating about CORBA is that once you start trying
to get into anything that deviates from "just plain vanilla
synchronous connections," it gets _real_ complex and _real_ strongly
tied to platform-dependancies _real_ fast. They've got a standard for
everything, but only a few that are ubiquitously available.

It's a bit like comparing CL to Scheme; both involve standards; the
problem with Scheme, like CORBA, is that getting Full Functionality
means having to have huge quantities of extensions that don't work
with all the implementaitons. (Contrast with the fact that the SERIES
extension for CL can at least be _hoped_ to work with many CL
implementations...)
--
(concatenate 'string "cbbrowne" "@" "acm.org")
<http://www.hex.net/~cbbrowne/corba.html>
"Popularity is the hallmark of mediocrity." --Niles Crane, "Frasier"

William Deakin

unread,
Oct 31, 2000, 4:14:53 AM10/31/00
to
Erik wrote:
> Is that a SW person or a HW person asking?

Clearly a SW person or else I wouldn't be talking about it :)

Peter Vaneynde

unread,
Oct 31, 2000, 5:12:45 AM10/31/00
to
Erik Naggum <er...@naggum.net> writes:

> * Fernando Rodríguez <spa...@must.die>


> | I have to comunicate a lisp process with a c++ app and I was
> | considering corba or using sockets. Both are rather new to me, so
> | wich one would you recommend and why? O:-)
>

> CORBA is already badly designed, so if you are new to sockets and

IMHO CORBA is badly designed, and the problems this produces are only
amplified by the legions of C++ "programmers" that want to design
systems using only the knownledge of "Corba in 21 days".

Network programming isn't easy, and using CORBA in a slightly
different way the designers immagined will get you into a world of
pain. But it all appears so easy in the examples people are lured into
a false sense of security...

I think CORBA should be seen as "just" a RPC call round ASN.1, and
people should use its 'oo'-ness as sparingly as possible. Just have a
Server object replying to simple (non-object) commands with simple
(structure) replies. Stick to this and you'll be safe. Ignore the
C++/java people why cry you don't know OOP and want to make everything
an object. Remember that a RPC is inherently slow, so limit the amount
of calls needed to do a typical operation, also (unless you use
advanced stuff, not implemented in the lisp servers AFAIK) objects in
CORBA as horribly costly: they take about 1K of space on the wire and
most deadly of all *the never, ever die*. An object in CORBA, once
created and given out *has* to remain alive for the duration of the
lisp process.


> If you are very dedicated, you will probably spend a few months just
> getting up to speed with protocol design before you can do something
> that is not a total drag, and upwards of a year to get it fully
> right.

Right. Be prepared to junk the first few versions of your code. (or
just give up and leave that project)

> Security problems in protocols are so common and so hard to avoid
> that you _really_ don't want to expose yourself to the process of
> learning by doing.

In my experience people don't care, and I don't expect this to change
soon :-(.

Groetjes, Peter

--
LANT nv/sa, Research Park Haasrode, Interleuvenlaan 15H, B-3001 Leuven
mailto:Peter.V...@lant.be Phone: ++32 16 405140
http://www.lant.be/ Fax: ++32 16 404961

Eric Marsden

unread,
Oct 31, 2000, 5:44:39 AM10/31/00
to
>>>>> "cb" == Christopher Browne <cbbr...@news.hex.net> writes:

ecm> asynchronous method invocations provided by CORBA Messaging,
ecm> either by polling or with callbacks. [This has been specified
ecm> fairly recently, so isn't supported by all implementations.]

cb> Are there any non-C++ implementations of AMI? All the
cb> documentation I've seen on AMI has been quite tightly tied to
cb> C++...

not that I know of. Probably future releases of Orbix2000 for Java
will include AMI.

Eric Marsden

unread,
Oct 31, 2000, 7:51:51 AM10/31/00
to
>>>>> "pve" == Peter Vaneynde <Peter.V...@lant.be> writes:

pve> Remember that a RPC is inherently slow, so limit the amount of
pve> calls needed to do a typical operation

a CORBA method invocation does not necessarily involve network
communication: if the client and server are collocated (reside in the
same address space), the ORB can use a simple method call.


pve> objects in CORBA as horribly costly: they take about 1K of
pve> space on the wire and most deadly of all *the never, ever die*.
pve> An object in CORBA, once created and given out *has* to remain
pve> alive for the duration of the lisp process.

I haven't used the commercial CL ORB implementations, so I don't know
what they are capable of. However, I don't see anything in the CORBA
specifications which requires an object to stay alive indefinitely.

You publish the availability of a service through an object reference.
On the host+port specified by that object reference, the object
adapter is responsible for dispatching incoming invocations to the
appropriate servant (a programming language entity which implements
the service). In CL, a servant is probably represented by an instance
of a CLOS class.

The object adapter maintains a mapping between object identifiers (the
server-specific component of an object reference) and servants. When
publishing a service, the only thing which has to remain alive
server-side is its object identifier (a key in a hashtable, say). You
aren't obliged immediately to incarnate a servant: the object adapter
can do so on-demand. Once a servant has serviced a request the object
adapter can decide to kill it (called "etherealization"), to be
restarted when a new request comes in. You can also have one servant
which services requests for multiple objects.

Peter Vaneynde

unread,
Oct 31, 2000, 10:27:59 AM10/31/00
to
Eric Marsden <emar...@mail.dotcom.fr> writes:

> a CORBA method invocation does not necessarily involve network
> communication: if the client and server are collocated (reside in the
> same address space), the ORB can use a simple method call.

AFAIK no common lisp ORB can collocate.

> pve> objects in CORBA as horribly costly: they take about 1K of
> pve> space on the wire and most deadly of all *the never, ever die*.
> pve> An object in CORBA, once created and given out *has* to remain
> pve> alive for the duration of the lisp process.
>
> I haven't used the commercial CL ORB implementations, so I don't know
> what they are capable of. However, I don't see anything in the CORBA
> specifications which requires an object to stay alive indefinitely.

In at least one cl-ORB you can force the deletion of the object (this
was added on request) and further operations on the object result in
an exception. This was only added later and I don't know how well it
works, nor if it is in the base product now (doesn't seem to be). The
position of the designers was that after publication something had to
be there to reply to RPC's. Note that the published CORBA mapping
claims that Lisp objects do not need the destroy method (sorry don't
know the exact name) as Lisp doesn't need it...

> The object adapter maintains a mapping between object identifiers (the
> server-specific component of an object reference) and servants. When
> publishing a service, the only thing which has to remain alive
> server-side is its object identifier (a key in a hashtable, say). You
> aren't obliged immediately to incarnate a servant: the object adapter
> can do so on-demand. Once a servant has serviced a request the object

The POA right?

> adapter can decide to kill it (called "etherealization"), to be
> restarted when a new request comes in. You can also have one servant
> which services requests for multiple objects.

Not implemented AFAIK. IMHO if you have a need for a large number of
objects in your design it's likely you're doing something wrong.

See:
http://www.franz.com/support/documentation/6.0/orblink/doc/orblink-idl.htm
http://www.franz.com/support/documentation/6.0/orblink/doc/standards.htm

Wade Humeniuk

unread,
Oct 31, 2000, 11:31:05 AM10/31/00
to

> TCP and IPv4 are _very_ good protocols. (I don't know enough about
> IPv6 to rate it.) The telecommunications protocols used on the
> gigabit trunks and on the long-haul high-speed interconnects are
> amazing feats of science. Of the really, _really_ large protocol

They are fine protocols, but only as transport and network layer
protocols. The whole conceptual framework of protocols I think is best
described by the OSI reference model. TCP and IPv4 only live within the
context of the data link and physical layer protocols that carry them.
All the issues of reliability, delivery and such need to be looked at as
protocol stacks.

For those that want to learn about protocols break open the OSI spec on
the reference model. Anybody remember the ISO spec number? I don't have
it handy anymore.

When I think what are good protocols I think they should have state
models that are totally defined. Modern protocols seem to be lacking
proper definitions of their state machines and are more ambiguous in
their specification. X.25 is very well defined with state tables
unambiguously placed in the specification. Also I have found a good
feature of a protocol is that is does not try to do too much. This
means that its finite state machine does not exceed 15 states with about
15 events. Do not ask me to justify these arbitrary numbers. I have
just found from experience that if the state machine exceeds these
limits there is probably more than one state machine that can be layered
into a protocol stack.

> protocols as well-designed as they are complex. The Telecom people
> have probably done some of the best work there is in protocol

The major problem with the Telecom protocols I have found has been their
defintions of their protocol data units which place a premimum on
reducing the number of bits in network traffic. This makes the PDUs
hard to parse. A great example of this, if you can get the spec is
IS-634, the control protocol between Cellular Base Stations and
Management systems. Whoever dreamed that up has never programmed a
computer before. Also the telecom protocols (at least the cellular
ones) are tending to define the protocol state machines in SDL which
allows protocol designers to leave gaps of what happens when error
conditions occur.

>
> As an example of a very good way of separating abstractions and
> representation, ASN.1 and its encoding rules are worth a study, such
> as what is done with SNMP.

ASN.1, what can I say, another language to learn. Liking Lisp, I think
a Lisp external syntax should be used for a platform independent
encoding of PDUs. It takes more bytes but it transports well (and I can
see it clearly on a data scope).

I am still divided on the issue of creating protocols between
cooperative machines/processes or going for a distributed OS view where
everything is one large memory space. I thought that is what CORBA is
offering, one large memory space. There is room for both of them, a
distributed OS at some lower level must implement protocols. Does a
programmer have to see that?

Wade

Philip Lijnzaad

unread,
Oct 31, 2000, 11:36:54 AM10/31/00
to

pve> objects in CORBA as horribly costly: they take about 1K of
pve> space on the wire

You're talking about object references, which can actually be much smaller
than 1 k (IOR strings don't tell you that much); it depends a bit on the ORB
vendor. The minimum it must contain (for IIOP) is protocol name +version,
hostname, portnumber, adaptor name and object identifier (the last two are
mangled into the object key, in a vendor-specific fashion). These components
would be necessary for any object reference that is supposed to work across a
network. I believe I read that an estimate of the lower bound on the data in
of an object reference is below 100 bytes (sorry, no 'references').

pve> and most deadly of all *the never, ever die*.


pve> An object in CORBA, once created and given out *has* to remain
pve> alive

Same is true for telephone numbers and URLs ... Somehow, the real world isn't
quite aware of this :-)

pve> for the duration of the lisp process.

Peter> In at least one cl-ORB you can force the deletion of the object (this
Peter> was added on request)

This sounds very suspicious to me. If an object should be deletable by a
client, then it's interface should bloody well have a delete() or destroy()
method or some such (and many do).


Peter> and further operations on the object result in
Peter> an exception.

exactly.

Peter> Note that the published CORBA mapping claims that Lisp objects do not
Peter> need the destroy method (sorry don't know the exact name)

'the destroy method' ? There is no such thing. It depends on the IDL, which
has nothing to do with languages.

Peter> as Lisp
Peter> doesn't need it...

??? If an IDL has interface Car { ...; destroy(); }

then this has nothing to do with Lisp garbage collection: it's simply a
method to have clients delete a particular Car object.

>> The object adapter maintains a mapping between object identifiers (the
>> server-specific component of an object reference) and servants. When
>> publishing a service, the only thing which has to remain alive
>> server-side is its object identifier (a key in a hashtable, say). You
>> aren't obliged immediately to incarnate a servant: the object adapter
>> can do so on-demand. Once a servant has serviced a request the object

Peter> The POA right?

(yes)

>> adapter can decide to kill it (called "etherealization"), to be
>> restarted when a new request comes in. You can also have one servant
>> which services requests for multiple objects.

Peter> Not implemented AFAIK.

This so-called DefaultServant approach is part of the POA, and therefore must
be implemented in _all_ ORBs that CORBA 2.1 compliant (were now at version
2.3a, btw). If not, they are not compliant. Incidentally, the DefaultServant
can be simulated by using a ServantLocator approach, where the preinvoke()
method always returns the same servant. It prolly is slightly more costly
though.

Peter> IMHO if you have a need for a large number of objects in your design
Peter> it's likely you're doing something wrong.

Why? It turns out that using objects in our field (bioinformatics) is very
natural and does make things easier. Since we're talking biggish databases
(1.5 million biological sequences, with many more sub-objects) here, we
inevitably have large numbers of objects. This has nothing to do with the
design: using 1.5 million little pieces of text (which is actually were we
come from) is no option.

When you're talking about large number of objects (all of them being equally
important), you _must_ have persistency behind it. Given this, CORBA
makes it easy to -- completely transparently to clients -- instantiate
some 'working set' of all those objects on demand (which, BTW, no other
distributed (object?) technology comes close to provding).

Philip
--
Time passed, which, basically, is its job. -- Terry Pratchett (in: Equal Rites)
-----------------------------------------------------------------------------
Philip Lijnzaad, lijn...@ebi.ac.uk \ European Bioinformatics Institute,rm A2-24
+44 (0)1223 49 4639 / Wellcome Trust Genome Campus, Hinxton
+44 (0)1223 49 4468 (fax) \ Cambridgeshire CB10 1SD, GREAT BRITAIN
PGP fingerprint: E1 03 BF 80 94 61 B6 FC 50 3D 1F 64 40 75 FB 53

Lieven Marchand

unread,
Oct 30, 2000, 5:00:51 PM10/30/00
to
Erik Naggum <er...@naggum.net> writes:

> * Eric Marsden <emar...@mail.dotcom.fr>
> | this is no longer true.
>
> I appreciate the update.
>
> I _still_ think CORBA sucks, though.

If you want to see protocol design gone berserk, try ASN.1. They
haven't solved any of the real problems but they added heaps of
complexity in the process. Unfortunately, it's caught on in some part
of the IETF world, so I'm considering writing some tools in CL to make
it livable.

--
Lieven Marchand <m...@bewoner.dma.be>
Lambda calculus - Call us a mad club

Rainer Joswig

unread,
Oct 31, 2000, 12:17:59 PM10/31/00
to
In article <m3n1fmh...@localhost.localdomain>, Lieven Marchand
<m...@bewoner.dma.be> wrote:

> Erik Naggum <er...@naggum.net> writes:
>
> > * Eric Marsden <emar...@mail.dotcom.fr>
> > | this is no longer true.
> >
> > I appreciate the update.
> >
> > I _still_ think CORBA sucks, though.
>
> If you want to see protocol design gone berserk, try ASN.1. They
> haven't solved any of the real problems but they added heaps of
> complexity in the process. Unfortunately, it's caught on in some part
> of the IETF world, so I'm considering writing some tools in CL to make
> it livable.

Have you looked at http://www.switch.ch/misc/leinen/snmp/sysman.html ?

--
Rainer Joswig, Hamburg, Germany
Email: mailto:jos...@corporate-world.lisp.de
Web: http://corporate-world.lisp.de/

Jason Trenouth

unread,
Oct 31, 2000, 11:53:12 AM10/31/00
to
On Tue, 31 Oct 2000 15:27:59 GMT, Peter Vaneynde <Peter.V...@lant.be>
wrote:

> Eric Marsden <emar...@mail.dotcom.fr> writes:
>
> > a CORBA method invocation does not necessarily involve network
> > communication: if the client and server are collocated (reside in the
> > same address space), the ORB can use a simple method call.
>
> AFAIK no common lisp ORB can collocate.

Xanalys HCL ORB does co-location optimization for Common Lisp clients and
servers within the same LispWorks process.

__Jason

Erik Naggum

unread,
Oct 31, 2000, 1:50:58 PM10/31/00
to
* Lieven Marchand

| If you want to see protocol design gone berserk, try ASN.1.

ASN.1 is not a protocol design. It's a data description language.
The full name is Abstract Syntax Notation #1. It does no more than
define structures that are named and identified by mutual agreement.
To use ASN.1, you have to choose some encoding rules, and there are
unfortunately too many of them, but the Basic and Distinguished
Encoding Rules are actually OK. (C programmers hate them, because
they make life so much harder in an untyped language like theirs,
but if you know what kinds of objects you get, as in dynamic types,
you can easily deal with these things.)

| They haven't solved any of the real problems but they added heaps of
| complexity in the process.

I can't agree with your assessment. Would you like to explain what
you consider the real problems they have not solved and what the
heaps of complexity added have been?

| Unfortunately, it's caught on in some part of the IETF world, so I'm
| considering writing some tools in CL to make it livable.

I'm surprised that you haven't, already. ASN.1 is about typed data.
It does not live well in a "typed variable" approach to programming.

Erik Naggum

unread,
Oct 31, 2000, 1:36:55 PM10/31/00
to
* Wade Humeniuk <hume...@cadvision.com>

| They are fine protocols, but only as transport and network layer
| protocols.

What does this mean? It makes absolutely no sense at it stands.

| The whole conceptual framework of protocols I think is best
| described by the OSI reference model.

Not at all. Only some frameworks are describable in ISORMOSI,
specifically those that actually used it while being described.

Please note that the full name is the _ISO_ Reference Model for Open
Systems Interconnection. It's their model. It is _not_ a model
that explains or can describe all Open Systems Interconnection.

See The Elements of Networking Style: And Other Essays &
Animadversions on the Art of Intercomputer Networking by
M. A. Padlipsky for a full coverage of this. ISBN 0-595-08879-1.



| TCP and IPv4 only live within the context of the data link and
| physical layer protocols that carry them.

This is false.

| All the issues of reliability, delivery and such need to be looked
| at as protocol stacks.

If you use the ISORMOSI on the Internet, you will end up with very,
very weird attitudes towards a very fine technology, sort of like
the people who believe that now that we have 8-bit byte-adressable
machines, the concept "byte" is conflatable with "octet", and then
their heads explode when other concepts of "byte" present themselves
to them.

| For those that want to learn about protocols break open the OSI spec
| on the reference model. Anybody remember the ISO spec number? I
| don't have it handy anymore.

I would recommend against ISORMOSI until you understand whence it
came, that is: the history of protocol design preceding it, both
within the CCITT/ITU and the ISO camps and without either. There
are very different cultures at work, here. That's a contributory
reason that TCP/IP won. "If ISORMOSI really was so great, how come
nobody talks their protocols?" actually applies.

| The major problem with the Telecom protocols I have found has been
| their defintions of their protocol data units which place a premimum
| on reducing the number of bits in network traffic.

You _really_ want this, but perhaps you need to work with these guys
to realize that 1% more protocol overhead means 1% more money to buy
the same bandwidth, or slower deployment due to higher costs, etc.

| This makes the PDUs hard to parse.

That does not follow _at_all_.

| A great example of this, if you can get the spec is IS-634, the
| control protocol between Cellular Base Stations and Management
| systems. Whoever dreamed that up has never programmed a computer
| before.

I'm making an unsubstantiated guess here, much like you do, and
would counter that he probably does protocol implementation in
dedicated chips. So would you if you dealt that these things.

| Also the telecom protocols (at least the cellular ones) are tending
| to define the protocol state machines in SDL which allows protocol
| designers to leave gaps of what happens when error conditions occur.

I have worked a little with SDL, but that is years ago, but most of
what I have worked with in the Telecom world has been described with
SDL (Z.100) or 'variations" like Estelle.

| ASN.1, what can I say, another language to learn. Liking Lisp, I
| think a Lisp external syntax should be used for a platform
| independent encoding of PDUs. It takes more bytes but it transports
| well (and I can see it clearly on a data scope).

Notice that ASN.1's encoding rules are actually parsable with the
same ease as (most) Lisp objects (that is, excluding symbols, which
are a pain insofar as they overlap syntactically with numbers).

I see a very clean mapping between ASN.1 and Lisp, qua languages to
describe and encode data.

| I am still divided on the issue of creating protocols between
| cooperative machines/processes or going for a distributed OS view
| where everything is one large memory space. I thought that is what
| CORBA is offering, one large memory space. There is room for both
| of them, a distributed OS at some lower level must implement
| protocols. Does a programmer have to see that?

If you do the global memory thing, at least figure out the need for
cache coherence and propagation protocols. Nobody I talked to about
CORBA have had the faintest clue what that would entail for CORBA,
so I gave up talking to these guys. (This was not in a Lisp
context, BTW.)

About half of what I have done over the years has had to do with
ensuring that lots of computers very far apart agree to what the
world looks like at exactly the same time. "One large memory space"
sort of covers it, since the idea is that through intelligent
protocols and very careful programming, lots of computers actually
get the same values without having to ask anyone _else_ for them.
There are several interesting products that offer some of this
functionality for databases, messages servers, etc, these days.

I'm not sure a programmer needs to see everything, but being able to
ensure that you have the information you need in time requires work,
and I fail to see how anyone but programmers should do that. Like
TCP and IP do not live quietly down in the basement, but actually
extend their services and _presence_ all the way up to the
"application layer" (and you _really_ want this, by the way), there
is no way you can avoid knowing about "the underworld" relative to
your actual interests. If you think you can avoid it, you are only
saying that you are willing to accept some set of default behaviors,
but if you don't know what they are, you are also likely to be wrong.

The belief in cleanly separated layers is almost a religion. It has
no basis in reality or experience, but there is nothing that can
make someone who believes in it stop believing in it. There's
always an answer for why it's a good idea despite all the hairy
stuff they have to deal with. Just like the whole ISORMOSI thing.
And almost like the belief in structured programming and object
orientation where the only time anyone can be trusted to use them
well is when they know when _not_ to use them.

Make that a more general assertion: If you think you have an answer
for everything, the answer is probably not right for anything. If
you have an answer for some things, the answer is probably right for
those things.

Fernando Rodríguez

unread,
Oct 31, 2000, 3:51:31 PM10/31/00
to
On Tue, 31 Oct 2000 16:53:12 +0000, Jason Trenouth <ja...@harlequin.com>
wrote:


>> > a CORBA method invocation does not necessarily involve network
>> > communication: if the client and server are collocated (reside in the
>> > same address space), the ORB can use a simple method call.
>>
>> AFAIK no common lisp ORB can collocate.
>
>Xanalys HCL ORB does co-location optimization for Common Lisp clients and
>servers within the same LispWorks process.

Sorry, but what's the need to use something like corba within a single
process? =:-O

TIA O:-)

//-----------------------------------------------
// Fernando Rodriguez Romero
//
// frr at mindless dot com
//------------------------------------------------

Erik Naggum

unread,
Oct 31, 2000, 4:43:13 PM10/31/00
to
* Fernando Rodríguez

| Sorry, but what's the need to use something like corba within a single
| process? =:-O

Well, besides the hammer, nail/thumb analogy, you might decide not
to "know" where your objects are. This could lead to all _kinds_ of
interesting performance pessimizations, just one of which is CORBA.

Marius Vollmer

unread,
Oct 31, 2000, 4:22:03 PM10/31/00
to
Erik Naggum <er...@naggum.net> writes:

> The Telecom people have probably done some of the best work there
> is in protocol design. Just take a look at how they started out
> with very low speeds, like 300 bps, but over time managed to
> squeeze 56kbps through the feeble phone lines that were not
> upgraded, then got DSL to work across that same old copper wire.

I don't know how 56kbps or ISDN etc really work, but I guess they are
more about very high modulation schemes, good channel estimations and
equalizations, error correcting codes and data compression than they
are about clever protocols.

And the old copper wires aren't probably the problem, anyway. They
should not be much worse than your ordinary 10Base-T twisted pair
ethernet wire. The problems are probably more with the filters along
the way that have been optimized for base-band speech signals.

I'm very impressed by these techniques as well, but I don't think you
can learn from them how to design a better CORBA. They are just
worlds apart.

Jon S Anthony

unread,
Oct 31, 2000, 7:09:42 PM10/31/00
to
Erik Naggum wrote:
>
> * Jon S Anthony <j...@synquiry.com>
> | The real question should be whether you believe your new protocol adds
> | real value to what it is you are producing. If you are in the
> | protocol business this is obvious. If you are in something else
> | entirely and only need to make use of protocols, it is highly unlikely
> | that the time spent on inventing yet another one will be of any real
> | value for your project/product.
>
> I disagree with this assessment. It is the same argument that can
> be made for something overly complex and badly implemented, like
> SGML, It has also been touted as solving problems that people would
> not be able to solve cheaper by themselves. That was just plain

Yes, this is the obvious counter argument, but it really doesn't
address the issue. Those involved in things like defining and
implementing SGML (or <fill in the blank>) _should_ be concerned about
"doing it right" because the result of that work _is_ their primary
value added (it _is_ their product/project).

Everyone going off and reinventing this stuff on their own, _even if
they know they can do a better job of it_, when it is not the primary
work they are involved in, and when the available offerings will not
negatively impact this primary work is just counter productive.

This in no way suggests that it is not appropriate or sensible or even
"a good thing" for someone to focus on defining and producing better
such offerings and making them generally available.


> How much effort does it take a good system designer to come up with
> something better than XML for any _particular_ purpose? If he
> grasps the good and useful qualities of SGML, a simplified syntax,
> slightly general tools, etc, can be written in a fairly short amount
> of time, and it costs less to develop, maintain, and use than a

> full-fledged SGML system does,...

This assumes that the tradeoff analysis for use is clearly negatively
disposed towards the available offering. For XML and SGML this is
likely true for most of the suggested uses as most of the suggested
uses are simply inappropriate. Perhaps you would include CORBA as
well, but I would not.


> Precisely, but having designed a few protocols that have been in
> wide use for over a decade each, just how much effort do you think
> it is to design new ones? The reason protocol design is hard is

If it doesn't provide you any advantage, the amount of effort it would
take is irrelevant. Again, if you're interested in getting into the
protocol business, that's a different story.


> | The nice thing about using CORBA to do this is that it at least
> | eliminates all the marshalling and unmarshalling of data items
> | (primitive and structured) in a platform independent way.
>
> You know, this is _not_ a hard problem. It is a hard problem in C++
> and the like, but that is because those languages have lost every

Whether it is hard or not is irrelevant. It still needs to be done
and it still needs to be maintained and it still needs all this on
various platforms. And for what? Additionally, you then need to make
it available for use in things like C++, Java, Perl, etc., at least
this is true for the sort of positioning we need. With CORBA all of
this is already done, and it just works. Let the armies of drones
hacking at C++, Java and the rest spend their time doing this.


> | Unless your business is really line protocols or something similar,
> | rolling your own here with sockets is just a distracting waste of
> | resources.
>
> Well, I have tried both approaches. Have you?

Yes.


> | Only if you don't make use of threads in making the calls in the
> | first place. If each call is a thread, the "waiting for the
> | synchronous reply" is irrelevant. Most ORBs are multithreaded and
> | can easily handle this.
>
> Right. Not my experience. Of course, everything gets better in the
> mythical "future", so do stay with CORBA if you have other things to

Just to be clear, the multithreaded ORBs are available now and have
been for years.


> | Well we have done it at 2000 miles and the results were basically
> | "instantaneous" (well under a second). Using straight X was death.
> | Using HTTP was (of course) worse than death.
>
> How long time do you think you would have spent designing a protocol
> that would have been just as good?

Designing is only a fraction of the effort. You then need to
implement it across several platforms and make it available for use in
an effectively open ended set of languages (at the least C++ and
Java). Given what we are doing, I would guess it would have taken
well over a person year or more of resources. And for what? To get
something just as good? Probably not even as good if you factor in
such things as the IDL compilers, exceptions, and the like.

Now, an order of magnitude better (depending on how that gain is
distribted), that would be a different story, but the effort would go
up dramatically and you are then basically in the distributed object
protocol _business_ (or at least you _should_ be).

Bruce Hoult

unread,
Oct 31, 2000, 8:11:45 PM10/31/00
to
In article <87itq8l...@zagadka.ping.de>, Marius Vollmer
<m...@zagadka.ping.de> wrote:

> And the old copper wires aren't probably the problem, anyway. They
> should not be much worse than your ordinary 10Base-T twisted pair
> ethernet wire. The problems are probably more with the filters along
> the way that have been optimized for base-band speech signals.

Well there is the minor detail that 10baseT is only specced to go 100m
while ADSL has to go for several km.

Humble old AppleTalk has been quietly driving up to 2 km of phone wire
(using the Farallon "PhoneNet" adaptors) for nearly 15 years now at
230,400 bps. That's the same speed as a lot of people's budget-price
ADSL runs at, acheived using a handful of simple passive components
(transformer, resistor, capacitor) plugged into a serial port. Now of
course ADSL can do better than that -- more like 2 Mbps - 6 Mbps -- if
the telco wants you to (or you pay them enough), but it *has* had a few
more years of development and those Nokia M10's aren't cheap at all.

-- Bruce

Christopher Browne

unread,
Oct 31, 2000, 8:28:58 PM10/31/00
to
In our last episode (Tue, 31 Oct 2000 20:51:31 GMT),

the artist formerly known as Fernando Rodríguez said:
>On Tue, 31 Oct 2000 16:53:12 +0000, Jason Trenouth <ja...@harlequin.com>
>wrote:
>>> > a CORBA method invocation does not necessarily involve network
>>> > communication: if the client and server are collocated (reside in the
>>> > same address space), the ORB can use a simple method call.
>>>
>>> AFAIK no common lisp ORB can collocate.
>>
>>Xanalys HCL ORB does co-location optimization for Common Lisp clients and
>>servers within the same LispWorks process.
>
>Sorry, but what's the need to use something like corba within a
>single process? =:-O

Um. Because that provides the highest conceivable speed of access
between client and server?

I might have an application that is distributable via CORBA so that
parts can run on separate hosts _if need be_; efficient colocation
means that this results in minimal degradation of performance if the
components run on the same host.
--
(concatenate 'string "cbbrowne" "@" "ntlug.org")
<http://www.ntlug.org/~cbbrowne/corba.html>
C is almost a real language. (see assembler) Even the name sounds like
it's gone through an optimizing compiler. Get rid of all of those
stupid brackets and we'll talk. (see LISP)

Christopher Browne

unread,
Oct 31, 2000, 8:29:10 PM10/31/00
to
In our last episode (31 Oct 2000 18:50:58 +0000),

the artist formerly known as Erik Naggum said:
>* Lieven Marchand
>| If you want to see protocol design gone berserk, try ASN.1.

> ASN.1 is not a protocol design. It's a data description language.
> The full name is Abstract Syntax Notation #1. It does no more than
> define structures that are named and identified by mutual
> agreement. To use ASN.1, you have to choose some encoding rules,
> and there are unfortunately too many of them, but the Basic and
> Distinguished Encoding Rules are actually OK. (C programmers hate
> them, because they make life so much harder in an untyped language
> like theirs, but if you know what kinds of objects you get, as in
> dynamic types, you can easily deal with these things.)

Ah... And therein lies where I've been "a little suspicious" of CORBA
IDL; the thing that it _should_ be closest to is a "protocol
definition language," but the lack of indication of how methods should
relate is "makes my spider-sense tingle."

For instance, the anticipated use of the following should be clear:
interface file_reader {
fileref open (in filename fn);
okcode read (in fileref f, in quantity q, out filebuf fb);
void fileref close (in fileref f);
};

One would do use this like:

(let ((fr (file_reader::open "/home/cbbrowne/whatever"))
(ok_code t)
(if fr
(progn
(while ok_code
(setf ok_code (file_reader::read fr 240 filebuffer)))
(file_reader::close fr)))))

The thing is, it's an _obvious_ thing that there should be some form
of state machine indicating the manner in which the methods should be
called; OBVIOUSLY we start with an "open," follow that with some
"read" calls, and then end off with a "close."

That relationship, which should, for many protocols, be expressed as a
state machine, doesn't enter in at all.

>| They haven't solved any of the real problems but they added heaps of
>| complexity in the process.
>
> I can't agree with your assessment. Would you like to explain what
> you consider the real problems they have not solved and what the
> heaps of complexity added have been?

Here's one commentary that gripes over it...
<http://www.alvestrand.no/x400/debate/asn1.html>

The really flameworthy bit:

Carl M. Ellison <c...@tis.com>:

"ASN.1 might be an interesting exercise for people who believe
LISP is the only real language or who really like to play with
abstract syntax or who like to write specs while ignoring
implementations (ie., write standards). It's *really* easy to
write structure declarations in ASN.1 -- as long as you don't try
to implement from them.

However, as one who wants computer programs to be written
efficiently and legibly (with small, easy to handle names,
allowing complete statements to be expressed in a small space)
and to have these programs communicate between machines with
different byte ordering -- and almost nothing else -- I find
ASN.1 is the *wrong* tool."

So apparently according to Ellison, Lisp must be intended for computer
programs that are to be written inefficiently, illegibly, with long,
ridiculous-to-process names, blah, blah, blah...

>| Unfortunately, it's caught on in some part of the IETF world, so I'm
>| considering writing some tools in CL to make it livable.
>
> I'm surprised that you haven't, already. ASN.1 is about typed data.
> It does not live well in a "typed variable" approach to programming.

Some Lisp-relevant links...
<http://asn1.elibel.tm.fr/fr/outils/emacs/manuel-utilisation.htm>
<http://www.switch.ch/misc/leinen/snmp/lisp/>

The latter bit includes some ASN.1-processing code...


--
(concatenate 'string "cbbrowne" "@" "ntlug.org")

<http://www.ntlug.org/~cbbrowne/lisp.html>

Christopher Browne

unread,
Oct 31, 2000, 8:29:15 PM10/31/00
to
In our last episode (31 Oct 2000 18:36:55 +0000),

the artist formerly known as Erik Naggum said:
>* Wade Humeniuk <hume...@cadvision.com>
>| They are fine protocols, but only as transport and network layer
>| protocols.
>
> What does this mean? It makes absolutely no sense at it stands.

The point is that they're not directly "application" layer protocols.

A good comparison seems to me to be that they generally represent the
"assembly language" of the networking world.

TCP/IP does indicate that it includes "application" layers that
provide _some_ of the higher level stuff; you have made some
desparaging remarks about recent RFCs that would go along with the
notion of separating "application layers" from the remainder of it...

--
(concatenate 'string "cbbrowne" "@" "acm.org")
<http://www.ntlug.org/~cbbrowne/>
Who wants to remember that escape-x-alt-control-left shift-b puts you
into super-edit-debug-compile mode? (Discussion in comp.os.linux.misc
on the intuitiveness of commands, especially Emacs.)

Erik Naggum

unread,
Oct 31, 2000, 8:51:50 PM10/31/00
to
* Marius Vollmer <m...@zagadka.ping.de>

| I don't know how 56kbps or ISDN etc really work, but I guess they are
| more about very high modulation schemes, good channel estimations and
| equalizations, error correcting codes and data compression than they
| are about clever protocols.

I didn't say "clever". I answered a question on which protocols I
thought were good and worth a study. The issues you raise are taken
care of at different layers of the protocol. The interesting aspect
for protocol designers is the _signalling_system_. That's why I
explicitly spelled out the names of SS#7 and DSS#1.

Rather than list the results of your guesswork, how about their
foundation?

| And the old copper wires aren't probably the problem, anyway. They
| should not be much worse than your ordinary 10Base-T twisted pair
| ethernet wire. The problems are probably more with the filters
| along the way that have been optimized for base-band speech signals.

This is completely false and just more random guesswork. Quit it
and talk to people who know this stuff. It's been _years_ since I
worked with this stuff, but any competent telecom engineer will be
able to tell you about the electrical qualities of those copper
wires and why that is _completely_ irrelevant to ISDN signalling.

| I'm very impressed by these techniques as well, but I don't think
| you can learn from them how to design a better CORBA. They are just
| worlds apart.

I'd like to know why you don't think so, apart from being "worlds
apart", which is just too funny when referring to the telecom world.

Incidentally, most of the interesting things appeared to be "worlds
apart" at one time or another. I believe good high-level language
programmers need to understand machine architectures more than good
low-level language programmers do, for the _reason_ that they are
worlds apart. If you don't understand what's going on on the wire,
the likelihood that you will not understand how to make a high-level
protocol work _with_ the intermediate layers is very high.

Erik Naggum

unread,
Oct 31, 2000, 9:32:55 PM10/31/00
to
* Jon S Anthony <j...@synquiry.com>
| Again, if you're interested in getting into the protocol business,
| that's a different story.

This seems to be your key argument, that there is a primary business
and lots of ancillary concerns for which it is better to use the
results of somebody else's primary business than dabble in it. I do
not agree with this for several reasons. First, if you discover
that you need better control over some ancillary concern, you may
have to make it the primary business of some person or group in your
company. Second, if you find that you cannot afford to do it on
your own, but need something different from the available offerings,
you may cause somebody else to spawn a similar new primary business,
such as a consortium. Third, you may discover that as you go abaout
your business, you gravitate towards certain concerns that are very
different from what you set out to do, and your primary business may
change to a previously ancillary concern, not the least because the
only way to improve your previous primary business to do something
else entirely. All of these have happened to me, and I claim that
if you're making any effort to be good at what you do, you will not
be able to tell beforehand what you will do best in.

| Designing is only a fraction of the effort.

I see that it is somehow important to you to exaggerate the costs of
"rolling your own", but I'd like to know why. It may be necessary
to defend the choice of using CORBA, but I have _already_ stated in
plain text and simple terms that if you can't do it better yourself,
by all means, stick with what somebody else did even if that is not
particularly good, so I hoped that we would have that condition
behind us, but you keep carping on this cost of not doing it better.
I fail to see the relevance in context.

I'm effectively arguing that out-doing CORBA is not that hard, but
having said

CORBA is already badly designed, so if you are new to sockets and

protocol design (which you will get yourself into), the likelihood
that you, too, will design your protocol badly is so high that it
is probably better to go with CORBA.

the rather obvious (to me, anyway) ramification is that you should
be good enough at what you do to out-do CORBA.

| You then need to implement it across several platforms and make it
| available for use in an effectively open ended set of languages (at
| the least C++ and Java). Given what we are doing, I would guess it
| would have taken well over a person year or more of resources. And
| for what? To get something just as good? Probably not even as good
| if you factor in such things as the IDL compilers, exceptions, and
| the like.

I started out overhauling a system that spent 6 seconds from end
system to end system at best, with more than 120 seconds worst case.
It was the third generation of a system I built in 1989 that then
guaranteed 2 seconds from end system to end system. It was simply
so incompetently done that it had to be rewritten. I got it down to
the old standards in the summer of 1998. To move beyond that into
the 500 ms guaranteed end system to end system transmission times,
including more and more clients on the randomly performing Internet
instead of dedicated lines with known characteristics, much higher
bandwidth and even higher transmission needs, I had months and
months of hard work cut out for me.

This stuff is not for sale to random clients as a packaged product,
and it won't be, either. It is not in my employer's interest to
sell the server side of my protocol, because that has become one of
the main reasons we're ahead of the pack. The protocol is intended
to be open to the public and a tremendous amount of work has been
put into ensuring that a client can be written in a short time, like
a week for a reasonably competent programmer regardless of language,
while the server has taken almost 18 months.

| Now, an order of magnitude better (depending on how that gain is
| distribted), that would be a different story, but the effort would
| go up dramatically and you are then basically in the distributed
| object protocol _business_ (or at least you _should_ be).

I think you have a fairly naive view of the separation between the
primary business and the ancillary concerns of an endeavor. Our
_primary_ business is delivering financial news to investors and
brokers. The protocol design became _my_ primary business when I
found that we were destined to waste a lot of time if we stuck with
off-the-shelf products, and I'm paid exceedingly well to develop,
maintain, and promulgate this protocol. This came to be because I
have managers who saw the value of my work and listened to my
concerns and honored my request to be free to work on this for as
long as I wanted. Now I can honestly say that whatever I take home
from this project is miniscule compared to what it brings in. This
was not something that could have been realized if anyone had had
the naive "primary business" view of what we intended to be good at.
Nowhere in our business plans would you find mention of what I do
for this company, because it isn't what we tell people about, and we
don't make any money from my work, we make the money _with_ my work.

Rob Warnock

unread,
Oct 31, 2000, 11:03:16 PM10/31/00
to
Erik Naggum <er...@naggum.net> wrote:
+---------------
| * Wade Humeniuk <hume...@cadvision.com>

| | The whole conceptual framework of protocols I think is best
| | described by the OSI reference model.
...

| Please note that the full name is the _ISO_ Reference Model for Open
| Systems Interconnection. It's their model. It is _not_ a model
| that explains or can describe all Open Systems Interconnection.
|
| See The Elements of Networking Style: And Other Essays &
| Animadversions on the Art of Intercomputer Networking by
| M. A. Padlipsky for a full coverage of this. ISBN 0-595-08879-1.
+---------------

Damn! You beat me to it!! ;-} ;-}

Padlipsky's book is classic ammunition for refuting ignorant/naive
ISORMOSI advocates (or as he says, ISORM, pronouonced "eye sore...M").

+---------------


| | TCP and IPv4 only live within the context of the data link and
| | physical layer protocols that carry them.
|
| This is false.

+---------------

Indeed. As Padlipsky says:

If you know what you're doing, three levels is
enough; if you don't, even seventeen won't help.

Granted, the ARPANET (now IETF) Reference Model *did* eventually grow
in practice from three to four levels, since it was recognized that it
was useful to have a common "Protocols/Services" layer separate from
Applications. So now the Internet protocol architecture's four levels
are (per "Figure 1" of RFC 791):

Applications
Protocols/Services
Internetwork Protocol (incl. ICMP)
Local Network

ISO needlessly complicated thing by breaking up Applications into
Applications & Presentation (as if presentation could even be separated
from the application context -- hah!), breaking Protocols into Session
and Transport, and breaking Local Net into Data Link & Physical. But the
802.3 & FDDI Standards Committees didn't stop *there* -- they further
subdivided the Physical into MAC, PHY, and PMD (stealing some of Data
Link for the MAC layer), and now we have even *more* sub-layers, such
as "AUI" and "MII".

+---------------


| | All the issues of reliability, delivery and such need to be looked
| | at as protocol stacks.
|
| If you use the ISORMOSI on the Internet, you will end up with very,

| very weird attitudes towards a very fine technology...
+---------------

As Dave Clark's classic paper "Modularity & Efficiency" addresses.
Or as Padlipsky puts it:

Layering makes a good servant but a bad master.

+---------------


| | For those that want to learn about protocols break open the OSI spec
| | on the reference model.
|

| I would recommend against ISORMOSI until you understand whence it
| came, that is: the history of protocol design preceding it, both
| within the CCITT/ITU and the ISO camps and without either.

+---------------

Well, as I heard the story, there were seven strong personalities who
each wanted their own sub-committee, so -- guess what? -- they ended up
with seven layers! (Surprise, surprise.)

+---------------


| TCP and IP do not live quietly down in the basement, but actually
| extend their services and _presence_ all the way up to the
| "application layer" (and you _really_ want this, by the way), there
| is no way you can avoid knowing about "the underworld" relative to
| your actual interests.

+---------------

In the very early days, some of you may recall, the protocol implementations
*did* live in user mode, but that later became unfashionable for "security"
and (mistaken) "efficiency" reasons. However, this is now getting a revival
with today's higher-speed network links. See, for example:

<URL:http://www.cl.cam.ac.uk/~iap10/gige.ps>
"Arsenic: A User-Accessible Gigabit Ethernet Interface"
Ian Pratt (University of Cambridge Computer Laboratory).

"Arsenic" is a way of moving the IP/UDP/TCP functions back up into
user-mode, *without* sacrificing inter-user security, so that applications
can access the data directly, where the network interface dropped it,
with no copying.

We do something very similar with our MPI-over-"O/S Bypass"-over-STP-
over-GSN. User programs on one system can exchange MPI messages with
user programs on other systems at a very high rate *without* doing
system calls or causing any interrupts!! Yes, applications (safely,
in a protected way) talking *directly* to the hardware!! To paraphrase
Dogbert, "I wave my paw, Bah! You layer demons, be gone!"

MPI-over-Myrinet or -Giganet does something similar as well.

[Yes, some people (such as SGI) implemented (and shipped) "zero-copy DMA"
protocol stacks before, but they usually depended on the network MTU being
a multiple of the host operating system's page size... which is becoming a
problem on systems which have shifted to larger page sizes for efficiency.]

We're going to *have* to embrace this, guys! 10-gigabit/sec Ethernet is right
around the corner (hardware demos are already being shown), and we're going
to *need* some clever "layer breaking" to provide full 10GbE performance.

+---------------


| The belief in cleanly separated layers is almost a religion. It has
| no basis in reality or experience, but there is nothing that can
| make someone who believes in it stop believing in it.

+---------------

Again, people like Dave Clark & Mike Padlipsky & even Van Jacobsen (his
papers on the "Witless" interface, as well as TCP "header prediction")
blew this religion away years ago, but its adherents never got the message.
(They never do seem to. *sigh*)


-Rob

-----
Rob Warnock, 31-2-510 rp...@sgi.com
Network Engineering http://reality.sgi.com/rpw3/
Silicon Graphics, Inc. Phone: 650-933-1673
1600 Amphitheatre Pkwy. PP-ASEL-IA
Mountain View, CA 94043

Wade Humeniuk

unread,
Oct 31, 2000, 11:28:01 PM10/31/00
to
Erik Naggum wrote:

>
> About half of what I have done over the years has had to do with
> ensuring that lots of computers very far apart agree to what the
> world looks like at exactly the same time. "One large memory space"
> sort of covers it, since the idea is that through intelligent
> protocols and very careful programming, lots of computers actually
> get the same values without having to ask anyone _else_ for them.
> There are several interesting products that offer some of this
> functionality for databases, messages servers, etc, these days.

The only one I ever read about is Amoeba. A research distributed OS.
Any others?

> The belief in cleanly separated layers is almost a religion. It has
> no basis in reality or experience, but there is nothing that can
> make someone who believes in it stop believing in it. There's
> always an answer for why it's a good idea despite all the hairy
> stuff they have to deal with. Just like the whole ISORMOSI thing.
> And almost like the belief in structured programming and object
> orientation where the only time anyone can be trusted to use them
> well is when they know when _not_ to use them.
>

ISORMOSI (first time I've seen this acronym!) for me still has a place.
The example I use is that when I design some software the written design
in a way has to lie. I have to omit details to have clarity, so I can
discuss it with other software developers (and have some aesthetic
appeal). I will usually try to design a protocol which follows a
layered model, because I have to apply some technique to get where I am
going. If I find problems with the layering I can try to resolve them
when I get there, but I have to start somewhere (with the _faith_ that I
will get there). This is one of the weird parts I find about life is
that I seem to need flawed (religious? dogmatic?) views of the world to
approach the truth. I guess its called learning. Is there a way out of
that morass?

Just some more words, a description of something can never be the thing.

As for the triumph of TCP/IP over ISORMOSI. I think it was things like:

Unix
FTP being simpler than FTAM
SNMP being simpler than CMIP
HTTP
the Session and Presentation Layers in OSI
TCP/IP was mostly American
random blind chance
and should I say "Simpler is Better"

that killed ISORMOSI (may it rest in layers).

BTW, isn't ISORMOSI still kicking in Europe?

Wade

Espen Vestre

unread,
Nov 1, 2000, 4:01:09 AM11/1/00