For instance, the anticipated use of the following should be clear:
interface file_reader {
  fileref open (in filename fn);
  okcode read (in fileref f, in quantity q, out filebuf fb);
  void fileref close (in fileref f);

(let ((fr (file_reader::open "/home/cbbrowne/whatever"))
      (ok_code t)
  (if fr
      (progn
        (while ok_code
          (setf ok_code (file_reader::read fr 240 filebuffer)))
        (file_reader::close fr)))))

The thing is, it's an _obvious_ thing that there should be some form
of state machine indicating the manner in which the methods should be
called; OBVIOUSLY we start with an "open," follow that with some
"read" calls, and then end off with a "close."

That relationship, which should, for many protocols, be expressed as a
state machine, doesn't enter in at all.

The really flameworthy bit:

Carl M. Ellison <c...@tis.com>:

     "ASN.1 might be an interesting exercise for people who believe
     LISP is the only real language or who really like to play with
     abstract syntax or who like to write specs while ignoring
     implementations (ie., write standards). It's *really* easy to
     write structure declarations in ASN.1 -- as long as you don't try
     to implement from them.

     However, as one who wants computer programs to be written
     efficiently and legibly (with small, easy to handle names,
     allowing complete statements to be expressed in a small space)
     and to have these programs communicate between machines with
     different byte ordering -- and almost nothing else -- I find
     ASN.1 is the *wrong* tool."

So apparently according to Ellison, Lisp must be intended for computer
programs that are to be written inefficiently, illegibly, with long,
ridiculous-to-process names, blah, blah, blah...

A good comparison seems to me to be that they generally represent the
"assembly language" of the networking world.

TCP/IP does indicate that it includes "application" layers that
provide _some_ of the higher level stuff; you have made some
desparaging remarks about recent RFCs that would go along with the
notion of separating "application layers" from the remainder of it...

  I didn't say "clever".  I answered a question on which protocols I
  thought were good and worth a study.  The issues you raise are taken
  care of at different layers of the protocol.  The interesting aspect
  for protocol designers is the _signalling_system_.  That's why I
  explicitly spelled out the names of SS#7 and DSS#1.

  Rather than list the results of your guesswork, how about their
  foundation?

| And the old copper wires aren't probably the problem, anyway.  They
| should not be much worse than your ordinary 10Base-T twisted pair
| ethernet wire.  The problems are probably more with the filters
| along the way that have been optimized for base-band speech signals.

  This is completely false and just more random guesswork.  Quit it
  and talk to people who know this stuff.  It's been _years_ since I
  worked with this stuff, but any competent telecom engineer will be
  able to tell you about the electrical qualities of those copper
  wires and why that is _completely_ irrelevant to ISDN signalling.

| I'm very impressed by these techniques as well, but I don't think
| you can learn from them how to design a better CORBA.  They are just
| worlds apart.

  I'd like to know why you don't think so, apart from being "worlds
  apart", which is just too funny when referring to the telecom world.

  Incidentally, most of the interesting things appeared to be "worlds
  apart" at one time or another.  I believe good high-level language
  programmers need to understand machine architectures more than good
  low-level language programmers do, for the _reason_ that they are
  worlds apart.  If you don't understand what's going on on the wire,
  the likelihood that you will not understand how to make a high-level
  protocol work _with_ the intermediate layers is very high.

  This seems to be your key argument, that there is a primary business
  and lots of ancillary concerns for which it is better to use the
  results of somebody else's primary business than dabble in it.  I do
  not agree with this for several reasons.  First, if you discover
  that you need better control over some ancillary concern, you may
  have to make it the primary business of some person or group in your
  company.  Second, if you find that you cannot afford to do it on
  your own, but need something different from the available offerings,
  you may cause somebody else to spawn a similar new primary business,
  such as a consortium.  Third, you may discover that as you go abaout
  your business, you gravitate towards certain concerns that are very
  different from what you set out to do, and your primary business may
  change to a previously ancillary concern, not the least because the
  only way to improve your previous primary business to do something
  else entirely.  All of these have happened to me, and I claim that
  if you're making any effort to be good at what you do, you will not
  be able to tell beforehand what you will do best in.

| Designing is only a fraction of the effort.

  I see that it is somehow important to you to exaggerate the costs of
  "rolling your own", but I'd like to know why.  It may be necessary
  to defend the choice of using CORBA, but I have _already_ stated in
  plain text and simple terms that if you can't do it better yourself,
  by all means, stick with what somebody else did even if that is not
  particularly good, so I hoped that we would have that condition
  behind us, but you keep carping on this cost of not doing it better.
  I fail to see the relevance in context.

  I'm effectively arguing that out-doing CORBA is not that hard, but
  having said

    CORBA is already badly designed, so if you are new to sockets and
    protocol design (which you will get yourself into), the likelihood
    that you, too, will design your protocol badly is so high that it
    is probably better to go with CORBA.

  the rather obvious (to me, anyway) ramification is that you should
  be good enough at what you do to out-do CORBA.

| You then need to implement it across several platforms and make it
| available for use in an effectively open ended set of languages (at
| the least C++ and Java).  Given what we are doing, I would guess it
| would have taken well over a person year or more of resources.  And
| for what?  To get something just as good?  Probably not even as good
| if you factor in such things as the IDL compilers, exceptions, and
| the like.

  I started out overhauling a system that spent 6 seconds from end
  system to end system at best, with more than 120 seconds worst case.
  It was the third generation of a system I built in 1989 that then
  guaranteed 2 seconds from end system to end system.  It was simply
  so incompetently done that it had to be rewritten.  I got it down to
  the old standards in the summer of 1998.  To move beyond that into
  the 500 ms guaranteed end system to end system transmission times,
  including more and more clients on the randomly performing Internet
  instead of dedicated lines with known characteristics, much higher
  bandwidth and even higher transmission needs, I had months and
  months of hard work cut out for me.

  This stuff is not for sale to random clients as a packaged product,
  and it won't be, either.  It is not in my employer's interest to
  sell the server side of my protocol, because that has become one of
  the main reasons we're ahead of the pack.  The protocol is intended
  to be open to the public and a tremendous amount of work has been
  put into ensuring that a client can be written in a short time, like
  a week for a reasonably competent programmer regardless of language,
  while the server has taken almost 18 months.

| Now, an order of magnitude better (depending on how that gain is
| distribted), that would be a different story, but the effort would
| go up dramatically and you are then basically in the distributed
| object protocol _business_ (or at least you _should_ be).

  I think you have a fairly naive view of the separation between the
  primary business and the ancillary concerns of an endeavor.  Our
  _primary_ business is delivering financial news to investors and
  brokers.  The protocol design became _my_ primary business when I
  found that we were destined to waste a lot of time if we stuck with
  off-the-shelf products, and I'm paid exceedingly well to develop,
  maintain, and promulgate this protocol.  This came to be because I
  have managers who saw the value of my work and listened to my
  concerns and honored my request to be free to work on this for as
  long as I wanted.  Now I can honestly say that whatever I take home
  from this project is miniscule compared to what it brings in.  This
  was not something that could have been realized if anyone had had
  the naive "primary business" view of what we intended to be good at.
  Nowhere in our business plans would you find mention of what I do
  for this company, because it isn't what we tell people about, and we
  don't make any money from my work, we make the money _with_ my work.

Damn! You beat me to it!! ;-}  ;-}

Padlipsky's book is classic ammunition for refuting ignorant/naive
ISORMOSI advocates (or as he says, ISORM, pronouonced "eye sore...M").

+---------------
| | TCP and IPv4 only live within the context of the data link and
| | physical layer protocols that carry them.
|
|   This is false.
+---------------

Indeed. As Padlipsky says:

        If you know what you're doing, three levels is
        enough; if you don't, even seventeen won't help.

Granted, the ARPANET (now IETF) Reference Model *did* eventually grow
in practice from three to four levels, since it was recognized that it
was useful to have a common "Protocols/Services" layer separate from
Applications. So now the Internet protocol architecture's four levels
are (per "Figure 1" of RFC 791):

        Applications
        Protocols/Services
        Internetwork Protocol (incl. ICMP)
        Local Network

ISO needlessly complicated thing by breaking up Applications into
Applications & Presentation (as if presentation could even be separated
from the application context -- hah!), breaking Protocols into Session
and Transport, and breaking Local Net into Data Link & Physical. But the
802.3 & FDDI Standards Committees didn't stop *there* -- they further
subdivided the Physical into MAC, PHY, and PMD (stealing some of Data
Link for the MAC layer), and now we have even *more* sub-layers, such
as "AUI" and "MII".

+---------------
| | All the issues of reliability, delivery and such need to be looked
| | at as protocol stacks.
|
|   If you use the ISORMOSI on the Internet, you will end up with very,
|   very weird attitudes towards a very fine technology...
+---------------

As Dave Clark's classic paper "Modularity & Efficiency" addresses.
Or as Padlipsky puts it:

        Layering makes a good servant but a bad master.

+---------------
| | For those that want to learn about protocols break open the OSI spec
| | on the reference model.
|
|   I would recommend against ISORMOSI until you understand whence it
|   came, that is: the history of protocol design preceding it, both
|   within the CCITT/ITU and the ISO camps and without either.
+---------------

Well, as I heard the story, there were seven strong personalities who
each wanted their own sub-committee, so -- guess what? -- they ended up
with seven layers! (Surprise, surprise.)

+---------------
|   TCP and IP do not live quietly down in the basement, but actually
|   extend their services and _presence_ all the way up to the
|   "application layer" (and you _really_ want this, by the way), there
|   is no way you can avoid knowing about "the underworld" relative to
|   your actual interests.
+---------------

In the very early days, some of you may recall, the protocol implementations
*did* live in user mode, but that later became unfashionable for "security"
and (mistaken) "efficiency" reasons. However, this is now getting a revival
with today's higher-speed network links. See, for example:

        <URL:http://www.cl.cam.ac.uk/~iap10/gige.ps>
        "Arsenic: A User-Accessible Gigabit Ethernet Interface"
        Ian Pratt (University of Cambridge Computer Laboratory).

"Arsenic" is a way of moving the IP/UDP/TCP functions back up into
user-mode, *without* sacrificing inter-user security, so that applications
can access the data directly, where the network interface dropped it,
with no copying.

We do something very similar with our MPI-over-"O/S Bypass"-over-STP-
over-GSN. User programs on one system can exchange MPI messages with
user programs on other systems at a very high rate *without* doing
system calls or causing any interrupts!! Yes, applications (safely,
in a protected way) talking *directly* to the hardware!! To paraphrase
Dogbert, "I wave my paw, Bah! You layer demons, be gone!"

MPI-over-Myrinet or -Giganet does something similar as well.

[Yes, some people (such as SGI) implemented (and shipped) "zero-copy DMA"
protocol stacks before, but they usually depended on the network MTU being
a multiple of the host operating system's page size... which is becoming a
problem on systems which have shifted to larger page sizes for efficiency.]

We're going to *have* to embrace this, guys! 10-gigabit/sec Ethernet is right
around the corner (hardware demos are already being shown), and we're going
to *need* some clever "layer breaking" to provide full 10GbE performance.

+---------------
|   The belief in cleanly separated layers is almost a religion.  It has
|   no basis in reality or experience, but there is nothing that can
|   make someone who believes in it stop believing in it.
+---------------

Again, people like Dave Clark & Mike Padlipsky & even Van Jacobsen (his
papers on the "Witless" interface, as well as TCP "header prediction")
blew this religion away years ago, but its adherents never got the message.
(They never do seem to. *sigh*)

-Rob

Just some more words, a description of something can never be the thing.

As for the triumph of TCP/IP over ISORMOSI.  I think it was things like:

Unix
FTP being simpler than FTAM
SNMP being simpler than CMIP
HTTP
the Session and Presentation Layers in OSI
TCP/IP was mostly American
random blind chance
and should I say "Simpler is Better"

that killed ISORMOSI (may it rest in layers).

BTW, isn't ISORMOSI still kicking in Europe?

Erik>   you might decide not to "know" where your objects are.  This could
Erik>   lead to all _kinds_ of interesting performance pessimizations, just
Erik>   one of which is CORBA.

Are you suggesting that there is a performance bottleneck in that you have to
know where (host,port or so) they are in order to use CORBA objects? This is,
of course, not true. If this thing is a concern, you would typically look up,
by name (or other) the 'where' of the real objects using some service
(Naming/Trading or so). Moreover, the resolving of a CORBA object reference
can result in a LocationForward by its server. This all happens transparently
to clients, since I believe even CORBA 1.0.

Lastly, a CORBA object reference can actually contain multiple locations
(and/or, incidentally, multiple protocols), offering fault tolerance options
(the standard for this is I believe being finalized; I haven't followed it).  

Some of the critcism that I see raised against CORBA derive from the fact
that people don't appreciate the kinds of things that CORBA solves. When you
look closely at it, no other technology (cgi, RMI, SOAP, DCOM) comes close to
offering its functionality. Just to be explicit about this:

 - language independence
 - platform/vendor independence
 - location independence
 - network protocol independence
 - separation of 'distributed objects' from their implementations
 - security model
 - location forwarding
 - on-demand launching of servers
 - fault-tolerance capabilities
 - async messaging
 - footprint
 - performance

and there's prolly a few things I overlooked.

  I thought this was an obvious tongue-in-cheek comment.

* Philip Lijnzaad <lijnz...@ebi.ac.uk>
| Are you suggesting that there is a performance bottleneck in that
| you have to know where (host,port or so) they are in order to use
| CORBA objects?

  No, I am "suggesting" that if you make a decision not to know the
  location of your objects, you open up for a number of ways to waste
  time and resources, relative to knowing where they are, obviously.
  You are most probably not arguing that there is _no_ performance
  cost in using COBRA compared to a direct in-memory object reference,
  so I am a little uncertain what you are actually thinking of and
  responding to.

| Some of the critcism that I see raised against CORBA derive from the
| fact that people don't appreciate the kinds of things that CORBA
| solves.

  That could also be true, but in my case, I do not care for the kinds
  of things COBRA does to accomplish what it does solve.  I don't have
  anything against what CORBA tries to solve, which is all very smart
  and very good and all that.

| When you look closely at it, no other technology (cgi, RMI, SOAP,
| DCOM) comes close to offering its functionality.

  This looks like marketing to me, with fairly automatic responses,
  and I have to reopen my ears to continue to read what you're saying.

| Just to be explicit about this:
|
|  - language independence
|  - platform/vendor independence
|  - location independence
|  - network protocol independence
|  - separation of 'distributed objects' from their implementations
|  - security model
|  - location forwarding
|  - on-demand launching of servers
|  - fault-tolerance capabilities
|  - async messaging
|  - footprint
|  - performance
|
| and there's prolly a few things I overlooked.

  Thanks for the list of features.  Do you see a list of misfeatures
  or at least problems that would impel someone _not_ to use CORBA if
  they wanted some of these features?

  I have used it to describe a protocol tha was very messy and that
  definition was the basis for a clean reimplementation, so I won't
  knock the model too hard, but it doesn't work for the Internet, and
  one must be acutely aware of when to break that model.  Sometimes
  you need a lift in that 7-story high-rise.

| I will usually try to design a protocol which follows a layered
| model, because I have to apply some technique to get where I am
| going.

  Protocol layering is as hard as doing class inheritance right.  The
  interesting thing is that you can start with the lowest layer and
  have a slot called "payload" that a subclass would interpret with
  sub-slots, or you could start with the highest layer and add slots
  in subclasses as you move down the stack.  As long as the needs of a
  class are dictated from the outside, this is pretty easy.  It gets
  real difficult to find the One True Layering when you move your own
  functionality around as you experience changing needs.

| This is one of the weird parts I find about life is that I seem to
| need flawed (religious? dogmatic?) views of the world to approach
| the truth.  I guess its called learning.  Is there a way out of that
| morass?

  Apart from dying?  I don't think so.  :)

| As for the triumph of TCP/IP over ISORMOSI.  I think it was things like:
|
| Unix
| FTP being simpler than FTAM
| SNMP being simpler than CMIP
| HTTP
| the Session and Presentation Layers in OSI
| TCP/IP was mostly American
| random blind chance
| and should I say "Simpler is Better"
|
| that killed ISORMOSI (may it rest in layers).
|
| BTW, isn't ISORMOSI still kicking in Europe?

  Nah, even the OSI Profiles (like GOSIP) have moved to TCP/IP, but
  there are still some large commercial X.400 software vendors and
  service providers.  Some believe that EDI needs X.400 to work.

  I used to study X.400.  I made a guess that I would spend 10 years
  writing a fully compliant mail system based on X.400.  Then I wrote
  a fully compliant SMTP-based mail system in three weeks, added MIME
  stuff experimentally (while I was still contributing to that work)
  and figured that if you had to spend more than a man-year on a mail
  system, you'd need to make it a highly successful mass product, and
  that was very unlikely to happen to anything as long as decent and
  simple mail systems were available essentially for free.  I don't
  think I was too far from how people with real money, resources, and
  vested interests were thinking.  Today, we have the Evil Behemoth
  doing about 10% of what X.400 offered, and they still don't comply
  with the necessary RFCs, so chances are they can't even _read_ the
  X.400 specification.

  The WWW idea hit the world with unprecedented force.  It's a crying
  shame that HTTP and HTML had such staggeringly idiotic designs, and
  still do.  If I were Tim Berners-Lee, I'd blame someone else for it,
  so I guess he couldn't find anyone who would accept that.  But I
  digress.

Erik> * Erik Naggum
Erik> | ... you might decide not to "know" where your objects are.  This
Erik> | could lead to all _kinds_ of interesting performance pessimizations,
Erik> | just one of which is CORBA.

Erik>   I thought this was an obvious tongue-in-cheek comment.

not to me, I'm afraid. It looked like you '"suggested"' that one of the
problems with CORBA is performance if/when you don't exactly know where your
objects are.

Erik> |  - language independence
Erik> |  - platform/vendor independence
Erik> |  - location independence
Erik> |  - network protocol independence
Erik> |  - separation of 'distributed objects' from their implementations
Erik> |  - security model
Erik> |  - location forwarding
Erik> |  - on-demand launching of servers
Erik> |  - fault-tolerance capabilities
Erik> |  - async messaging
Erik> |  - footprint
Erik> |  - performance
Erik> |
Erik> | and there's prolly a few things I overlooked.

Erik>   Thanks for the list of features.  Do you see a list of misfeatures
Erik>   or at least problems that would impel someone _not_ to use CORBA if
Erik>   they wanted some of these features?

Good question. There are two main problems: (perceived) complexity and
learning curve of the thing (relative to cgi and rmi and SOAP), and more
importantly: lack of widely available (different) implementations of all the
useful extra standards such as security, messenging, etc.  Lesser problems
are the overhead (use sockets if you need real high-speed), and I think
CORBA is prolly not yet mature enough in the transaction monitor area.  So,
for high speed transaction processing, CORBA may not be the right
techonology.

Lastly, the standardization process is a bit long-winded, but that
doesn't say much about the technology.

1) Corba has a large learning curve, complicates builds, is harder to
debug in complex scenarios, and makes you alot more vulnerable to
vendor bugs.  It can make sense if you are using it as a framework for
many interoperable applications which need to interact in diverse
ways.

2) If you don't understand sockets, you are going to have a hard time
reasoning about the operational behavior of your system.  Doing at
least a prototype using plain sockets will give you a sound basis for
understanding the salient qualities of a particular corba
implementation.

uh  ... not really. Network protocol independence doesn't seem to be so, but
then again it doesn't seem to matter. The vendor independence is usually but
not always OK.

No, I know, and it's a big pity really.

you're right; on-demand launching should have been listed under the location
forwarding: you can have auto-launching thanks to this feature, but the
solutions are (probably necessarily) implementation-specific.

cgi? SOAP? DCOM ? rmi? They can be made fault-tolerant by hand; the point is
that (OK, OK, once the fault tolerance thing is finilized and implemented ...),
this thing will be available out of the box.

I thought Iona has, or has annnounced it; not sure.

:-) Again, this is meant relative to other technologies. ORBExpress claims an
ORB runtime of 150kByte. I think this compares favourably with the avarage
XML parser package or httpd process. Same thing for speed: XML or text/plain
is just slower to parse than a binary datastream of which you know the layout.

yes, I agree, I'm sorry if I appeared to be claiming that.
                                                                      Philip

  This claim is not at all supported by the evidence.   Parsing a
  binary data stream that claims to be general and not just a dump of
  bytes from memory, is just as expensive to parse as text/plain and
  can quickly become _slower_ if you do it wrong and need a lot of
  overhead to overcome the inherent problems of binary representation.
  The arguments for binary datastreams are space and bandwidth, _not_
  time to process.

  cb> "May contain" and "standard still being finalized" and "not
  cb> nearly ubiquitously-implemented" adds up to vapourware...

the CORBA fault tolerance specification was published in January.

   <URL:http://cgi.omg.org/cgi-bin/doc?ptc/00-04-04>

  cb> ... AMI isn't yet ubiquitous; I'm not sure it is actually
  cb> implemented yet ...

AMI is implemented by TAO and Orbix2000 for C++, at least.

My feeling that the real key to reducing network traffic is to make the
protocols "higher level" and thus say more with less.

I was at a seminar about ten years ago put on by a member of Sun
Microsystems's Corba team.  He discussed bandwidth considerations but
brushed them off as "orthogonal problems" (the same for network
management, et. al)

As to the question of Corba or Sockets, go with what attracts you, once
you have implemented the solution you will actually know how to actually
do it. :) :)

And bandwidth is *expensive* -- the European telcos are in the process
of spending over $100 billion on licenses for more bandwidth alone,
with the total expenditure on licenses and hardware estimated at more
than $300 billion over the next few years.  That's really a lot of
money.

And it's also not really the case that memory is cheap for instance,
although the PDP11 mindset I referred to elsewhere would lead you to
believe it is.  It's cheap *if you don't want to access it*: if you do
want to access it you soon discover that you have a small amount of
memory you can actually get at quickly, and a much larger amount which
is depressingly far away from the CPU.  If you have a decent machine,
it may be able to deliver stuff in the quantities the CPU needs, but
the latency is still severe.  And this situation will not ever get
better.  And lo and behold there are seriously hairy algorithms to
deal with this problem, implemented mostly in hardware.

Similar things go for network bandwidth -- wire-based bandwidth may be
plentiful, but latencies are long and getting longer (light travels
0.3 m in one cycle for a 1GHz chip).

--tim

Well, for some time now, I have put off studying some processor
designs.  Since I did never before study a processor design, I would
be glad, if you could recommend a processor that's modern and that
someone with no (at least not very much) previous experience in this
area can understand.  I'd also be glad for some literature
recommendations in this area.

Thanks in advance,
Boris

--
bo...@uncommon-sense.net - <http://www.uncommon-sense.net/>

  Just about nothing beats David A. Patterson and John L Hennessy's
  seminal works "Computer Architecture (a quantitative approach)" and
  "Computer Organization and Design (the hardware/software interface)".

  Read them before you pick up, say, the processor reference manuals
  for the Intel Pentium III.  (Despite the braindamaged instruction
  set and register model, the internals are amazingly brilliant.)

For books, it's really easy to decide on a recommendation:

  Computer Organization and Design:
  the hardware/software interface

  <http://www.amazon.com/exec/obidos/ASIN/1558604286>

It covers everything you need to know, all fucussed around the MIPS ISA.  
It looks at gates and hardware design, instruction sets,
assembly-language programming.  It presents a series of
increasingly-sophisticated possible implementations of the MIPS
architecture, with the first ones being simple enough that you could
actually go and build them yourself using TTL chips (or just model them).

You can't beat it and these guys *know* what they're talking about.

Hennessy, btw, just got a big promotion at Stanford...