Can you reccommend any books or URLs which could help me to learn
about XML, whith the aim of being able to discuss intelligently the
relative merits of the two.

Cheers

-o- both are regarded by some as the best thing since sliced bread

-o- both go in heavily for balanced delimiters

-o- both are regarded as overly-bracketful by many people

Differences:

-o- One is a text markup language with little or no semantics

        To a Lisp hacker, XML is S-expressions in drag.

Cheers

g

  I have come to _loathe_ the half-assed hybrid that some XML-in-Lisp tools
  use and produce, because it makes XML just as evil in Lisp as it was in
  XML to begin with, and we have gained absolutely nothing in either power
  of processing or in abstraction, which is so very un-Lisp-like.

<foo bar="zot">quux</foo>

  should be read as

(foo (bar "zot") "quux")

  and most definitely _NOT_ as ((:foo :bar "zot") "quux"), which turns this
  fairly reasonable structure into a morass of complexity worse than it was
  to begin with.  And it does _NOT_ help to represent empty elements only
  with a keyword.  Using three different levels of nesting to represent a
  single concept is Just Plain Wrong.  Also, using keywords is not a good
  idea because there needs to be a lot of related information associated
  with elements and attributes, in different contexts, not to mention all
  the things they do with their funny "namespaces" these days.

  Whether something is an attribute or element is _completely_ arbitrary.
  It is based on some arbitrary choices in the design process that reveal
  absolutely no inherent qualities.  For purely pragmatic reasons, SGML
  folks will use attributes for some things and elements for others because
  their tools can deal with some things in attributes and some things in
  elements.  The faulty idea that attributes say something "about" the
  element and sub-elements somehow constitute be their contents is the same
  premature structuring that premature optimization of code suffers from.
  The whole language is incredibly misdesigned in making that distinction.

  As for writing SGML/XML/HTML/whatever, I have a simple way to get rid of
  the annoying verbosity of these stupid languages while _retaining_ that
  mistake between attribute values and elements, because it is quite hard
  to make simple regular expression-based conversions retain enough data
  about an element to decide what should be attribute and element.  An
  element has the form <name [attributes] | [contents]>.  Attribute have
  the form <name | value>.  Internal whitespace is only for readability.

XML                             Enamel (NML)            CL
<foo/>                          <foo>                   (foo)
<foo bar="zot"/>                <foo <bar|zot>>         (foo (bar "zot"))
<foo>zot</foo>                  <foo|zot>               (foo "zot")
<foo bar="zot">quux</foo>       <foo <bar|zot> |quux>   (foo (bar "zot") "quux")
<foo>Hey, &quux;!</foo>         <foo|Hey, [quux]!>      (foo "Hey, " quux "!")
<foo>AT&amp;T you will</foo>    <foo|AT&T you will>     (foo "AT&T you will")
<foo><bar>zot</bar></foo>       <foo|<bar|zot>>         (foo (bar "zot"))

  So I have almost none of the annoying and arbitrary quote/escape mania in
  attribute values or contents alike, either.  Entities I write as [name],
  and they end up in the Lisp version as symbols if not the character they
  represent purely for syntactic reasons.  Writing "code" in this language
  is actually amazingly painless compared to the produced noise.  Besides,
  with a few simple modify-syntax-entry calls in Emacs, I get < and > to
  match and blink and I can move up and down the structure very easily.

  For processing this stuff in Common Lisp, it is _sometimes_ neat to
  convert the single | attribute/content marker into the zero-length
  symbol, ||, so pathological cases like

<foo bar="zot"><bar>"zot"</bar></foo>

  which could have been written like this to show how arbitrary the
  syntactic disctinction in SGML/XML is

<foo <bar|zot>|<bar|zot>>

  come out as

(foo (bar "zot") || (bar "zot"))

  The really interesting thing is that writing in Enamel and producing XML
  is so easy that a simple Perl or Lisp function that takes an Enamel
  string as argument and produces XML is quite simple and straight-
  forward.  This makes for some interesting-looking "scripting" that blows
  the mind of the miserable little wrecks that think they have to type the
  endtag, the quotes and all the other user-inimical features of SGML/XML.

  In my personal view, Lisp "markup" has the disadvantage of needing lots
  of quotes, while Enamel has the strong advantage that in <xxx|yyy>, xxx
  is always symbolic and yyy is always a string of characters subject to
  interpretation by whatever the symbolic part instructs in context.

  Since the key feature of markup languages is the separation of text from
  markup, the simple idea in Enamel should carry enough force to make this
  a fully realizable goal without making an artificial syntactic separation
  between information and meta-information at any level.  If the syntax is
  good enough for the information, it should be good enough for the meta-
  information, and I think Enamel is.  Fortunately, I do not have to create
  a whole new international following and engage in godawful politics to
  use a better syntax for XML and the like, since XML and the like are only
  used as interchange syntaxes these days.  Nobody in their right mind
  actuslly writes anything by hand in such stupid languages that require so
  much attention to incredibly insignificant details and incomprehensibly
  irrelevant redundancy, anyway, do they?  :)

  Finally, note that in Enamel, a complete element is enclosed in <...> and
  that means it can be subject to a nice little Common Lisp reader macro,
  and it can be taught to recognize other stuff, as well, such as the neat
  concept of interpolating expression values where {expression} occurs.

  Still at "internal use" stage, I plan to publish some stuff about Enamel
  not too far into the future.

I'm curious about your syntax though: If I want to go from Lisp to
something (rather than from something to Lisp), it seems that the
syntax you give is amiguous because of this (I cut the lines that
don't seem relevent).

(foo (bar "zot")) and (foo () (bar () "zot")) respectively.  But for
most cases that was more typing than I liked (since I was typing in
Lisp not a better syntax).

I wrote a package in Java at a prior employer which automatically generated
XML representations for classes as elements based on Java metadata, and the
tack I took was not that the XML attributes contain meta-data and the contents
data but rather that the XML attributes contain atomic data and the
contents contain compound data, since this is IN FACT what the real distinction
is. Some type Foo with

 int x;
 Date y;
 ElementList z;

might produce

 <Foo x="3" date="3207653971"><z><Element>...</Element>...</z></Foo>

In effect, what I got out of this was a description that allowed two
syntaxes: an easy syntax for easy things, and a hard syntax for hard
things.  Of course, there are all kinds of problems even then because of
subclass relationships (the analog of the problem of *print-nreadably*
and strings of base-char vs char, or the loss of fill-pointer, etc. in
printing a string.  Fixing these gets very verbose very quickly.
So I'm mnot defending the notation in that regard.)

But what I'm really wondering is whether SGML has some "intended use" spec
that tells you that you have to put meta-info in the "car" of the "form",
and info in the "cdr".  I thought the use of these containers was
semantics-free.

 <foo bar=<zot/>>

or

 <foo bar=<string>zot</string>>quux</foo>

This would keep people from feeling the attribute list was a shorthand
area and would also allow the storing of complex meta-data.  Right now,
the fact that a use of <...> in the attribute thing seems a terrible waste.
The only rationales I can figure for this were either the desire to
periodically beat someone on the back of the hand for syntax errors
by having a regular application of over-applied syntax or else some sort of
efficiency bum to make the acquisition of strings in the attribute list
uselessly faster.  Do you know what the reason was that recursive structures
were not allowed in this position in XML?

Or perhaps it was the fact that the "real world" substitutes for "parsed
structure" things like that weird assembly code like notation which looks
like

 (A
 AHREF=foo.html
 -Text
 )A

Perhaps someone was just being uncreative about how a compound-structure
could be offered as an attribute.

    As for writing SGML/XML/HTML/whatever, I have a simple way to get rid
    of the annoying verbosity of these stupid languages while _retaining_
    that mistake between attribute values and elements, because it is quite
    hard to make simple regular expression-based conversions retain enough
    data about an element to decide what should be attribute and element.

  ... implying that I would normally have such information and use it when
  generating attribute/value or sub-element/contents.

(foo ((bar "zot")) "text")

or for some HTML:

(font ((size 10) (color :yellow)) "text")

Which is just a lisp program, after applying even my minimal skills
with macros.  The hard part is not overwelming the text.

(fragment ((layout :html-like) (feeling :pompous)) "
SGML and TeX, being just markup, did their best to preserve the bulk
of text without any transformation.  Their goal is to take a normal
text document and " (tquote "mark it up") " for computer interpretation.
SGML markers are ugly but they weren't intended to dominate the file.
" (p) "
As people's interest has moved from SGML to XML, they now talk more off
" (italic "structured data") ", and although this is a somewhat subtle
change of mindset, it makes the markup the dominant part of the file.
Unfortunately, once people start down a course of action they rarely
stop to consider if the original design guidelines and intent may have
been lost.
" (p) "
Just by " (emph "standardizing") " a straightforward mapping of XML
into and back from lisp, the uglyness and verbosity of XML would be
less of an issue.  You could use the syntax you liked.  I suspect when
the enthusiasm for XML has died down a bit, the benefits of a
standardized lisp notation could become better recognized.
" (p) "
Without such standards, of course, forget it.
")

You do need to step around the native lisp functions like quote.

  However, given that the mechanism was created, and I will argue that it
  was not so much created as it was never thought possible to be any other
  way, it was used to define several language properties.  "Now that we
  have this, would it not also be nice to have that."  This means that
  several of the attribute types grew very far apart from the contents of
  sub-elements and you sort of "had" to use them as attributes, but only
  sort of, because the application can and does define the semantics of
  everything, and if you want ID and IDREF, you can make the same choice as
  you would in Common Lisp to use symbols or a hash tables of strings.

  Whoever first decided to define HTML in such a way that unknown elements
  should be displayed suffered from exactly the same problem.  As a sorry
  consequence, we have elements that have to contain _comments_ that are
  the real contents because that somebody did not foresee the need to have
  meta-information in contents.  I argue that this is a result of "getting"
  the invalid meta-information/information dichotomy.  If that person had
  not been bitten by the false idea that meta-information is fundamentally
  different from information, he would have realized that there would be a
  need to use element contents for meta-information, as well.

The only useful information was the name of some data, and the assumed or
explicit type of the data value. It made no difference in terms of processing
if the data was logically an "attribute" or an "element" -- the problem of
extraction is exactly the same in both cases.

That there is a difference with XML is only due to its artifical distinction
of attributes vs elements.

The only useful distinction I have found for attributes vs elements was
aesthetical: how did the element look to a human reader (i.e. me) of the XML
file? Whether I would choose a simple vs compound approach depended solely on
my mental picture of the data in question, e.g.

  <foo name="joe" size="big"/>

vs

  <foo>
    <name>joe</name>
    <size>big</size>
  </foo>

In an S-expression format however, it doesn't matter, and the aesthetic
distinction is only:

  (foo (name joe) (size big))

vs

  (foo
    (name joe)
    (size big))

As a separate topic, I just *hate* when people encode complicated data into
attributes, forcing applications solve yet another parsing problem. The whole
point of something like XML is to have a standard encoding structure. The
"parsing" problem is supposed to be solved once, and only the semantic
interpretation should remain.

E.g.

<this_bad choice="a|b|c"/>

<this_better>
  <choice>
    <alternative>a</alternative>
    <alternative>b</alternative>
    <alternative>c</alternative>
  </choice>
</this_better>

</rant>

Of course, S-expressions are much more preferrable. The only "cool" things
about XML that I like is the ability to specify character encodings (ASCII vs
Unicode, etc.) and the schema namespaces business, such that one can mix
"tags" from semantically different spaces.

Mind you, the details of Schemas are overly complicated and gross to use, but
they are better than DTDs which should just die die die. DTDs are a lesson in
why a separate language should *not* be invented.

If I just treat attributes as subordinate elements, I lose the ability
to simply translate from lisp into XML.  In other news articles
you seem to suggest that you use information outside the lisp
representation to make that determination.  This means that my tools
would require priori knowledge, which I feel a simple lisp->XML
(non-interpretating) translator should not need.  I don't think
lisp->XML translators should have constraints that XML parsers don't
have.

In code which interprets the lispified XML, I know what the grammar is,
so can't I (at that time) bury any abstraction issues in the access
methods?  I admit I don't fully understand the abstraction benefits
with which you are concerned.  I've been overwelmed in tracking
all the XML languages which are being defined.  I was hoping that
being able to map them into lisp syntax would help avoid being buried
in XML's confusing syntax.  When looking at them in a lisp syntax,
thing can become clearer (and seem less innovative).

I don't agree that the distinction between attributes and entities is
always arbitrary.  SGML does stands for Simple Graphical *Markup*
Language, and in a markup language, I think it is important to
distinguish the text of a document from it markup.  Multiple
translators may be used, and they should not need to be kept up to
date on what attributes are used in the other translators.  In an
expression like:

<header1><italic>Wow</italic>, this is difficult.</header1>

or as lisp (which I think is more readable):
(header1 (italic "Wow") " this is difficult")

it isn't clear whether "Wow" is text or the value of an attribute
unless you have prior knowledge of whether `italic` is a attribute in
the context of a header1 directive.  So here the distinction is
simple, clear, and useful.  (I am not commenting on the syntax.)

This is still important for things like xhtml -- and probably docbook,
whose standard I have not yet assimilated.

In my previous message I suggested that:
   <header1 italic="Wow"> this is difficult</header1>

become:
   (header1 ((italic "Wow")) " this is difficult")

With mimimal (but I admit real) damage to the syntax.

  Those who grasp the concepts involved, will see that attributes are just
  another form of contents.  Those who do not grasp the concepts involved,
  will think that attributes are different from contents because they have
  been given syntactically different expression.  But it is always the
  syntax that follows the function.  Someone believed that meta-information
  should be fundamentally different from information.  Someone believed
  that the contents of elements should be text that wound up in the final
  document on the printed page and the values of attributes should not, but
  should only influence the processing of the information.  This worked
  only as long as SGML was used as a markup language for documents and had
  no aspirations towards being an abstract structuring syntax.  When it
  came to use it as a more abstract syntax, there _is_ no inherent quality
  that determines whether some value ends up displayed or not.  That has to
  be supplied by the software that processes the information, which is
  precisely prior knowledge of the structure and its meaning.

I wouldn't be taking the time to learn and work in lisp, if I didn't
recognize that it could significantly improve the ways I analyze
and solve problems.  This was made obvious by looking at (what I
presume is) good lisp code.

I will follow your suggestion and remove the entity/attribute
distinctions in my lisp code.  I am then left with a strong desire to
keep the names of XML attribute names in a list, and use that in a
generic XML output translator.

I suspect this is still avoiding the issues you have raised.  Instead
I will start by writing specific code for each case and see if a less
"C" like way of sharing code becomes evident.

I am open to any suggestions, although I can not guarantee I will
immediately grasp their rational.  (I think I am past my cranky
stage.  I am never cranky when I get to write code.)

Well, I agree that in most cases you will know whether something was
an attribute or contents, when you're processing it, but what about:

  <foo bar="1"><bar>2</bar></foo>

If I understand you correctly (and I'm not exactly sure about that),
you would represent this in Lisp as:

  (foo (bar 1) (bar 2))

I don't see how you can distinguish attributes and contents in this
case, and how you can translate this back into the same XML.  Probably
I'm missing something.

Boris

--
bo...@uncommon-sense.net - <http://www.uncommon-sense.net/>

having this knowledge _only_ in the output translator frees you from
distinguishing attributes and contents in your program.

this does mean that the value of foo's `bar' attribute (er, content)
should magically always be atomic, because if it's not you won't be
able to output valid XML.  but note that if your program can deal with
non-atomic values of `bar', then you probably have chosen wrong XML
format in the first place...

  However, I apparently need to insist that you understand that in SGML and
  XML alike, you do in fact know what attributes an element has.  It cannot
  possibly be ambiguous.  If you decide to name a sub-element the same as
  an attribute, however massively stupid that is even with SGML/XML as it
  is, you _still_ know that you have an attribute with that name.  That
  there is a sub-element with that name, as well, is coincidental to the
  representation.  There simply is no way you can _not_ know that, unless
  you go out of your way to destroy the information that SGML provides you
  with.  If you destroy the information that is available to you, you will
  not get me to do stupid human tricks answering your resulting questions.

  I truly wonder what is so hard to understand about this.  We Lisp people
  are quite used to association lists, right?  Keyword-value pairs do not
  need to be in property lists to be understandable by Lisp people, do
  they?  To my mind, whether you store something in a property list or an
  association list is arbitrary.  However, in the reactions that I have
  seen to obliterating the false dichotomy between attributes and contents
  in SGML, there somehow seems to be a _fundamental_ difference between
  property lists and association lists.  I completely fail to understand
  how that can be.

  The whole deal is so simple I do not even know how to explain it so
  people get it if they do not get it immediately.  It is somewhat like
  seeing someone struggle with fractions.  They either get it or they do
  not, and although I have managed to make many a struggling child get the
  idea, I have _no_ idea what precisely caused them to grasp it.  It just
  happened, and they laughed in relief.  This attribute/container thing is
  equally intuitively evident.

  Case in point: An element has a fixed number of attributes.  That is
  reflected in a fixed length of the association list that makes up the
  attributes.  Attributes are not repetable and not omissible, so if there
  are n attributes in the attribute list for an element, there will be n
  conses with attributes in the cdr of the element representation.  There
  are no two ways about this.  It is completely and irrevocably unambiguous.

  By exploiting the rich information we have about the elements and their
  makeup in SGML, we can reason about things with much simpler means than
  by adhering strictly to the particular representational issues in SGML.
  If it matters to you that some values are attributes, you ask for the
  attribute information.  If it does not matter to you, you can be relieved
  of the distinction.  If you want to transform attribute to contents or
  vice versa, modify the information about the element, not the element; if
  and when you print it out, the modifications will manifest themselves in
  new SGML/XML syntax, but nothing happened to your internal representation.

   <foo bar="1"><bar>2</bar></foo>

would be represented as

   (foo (bar 1) (bar 2))

and you've sort of hinted yes.  You've made allusions to alists as a way
of understanding this, but as a sense of intuition, of course, that doesn't
help a Lisp programmer a lot since plainly an alist is about th leftmost
of each named thing, and people are uneasy about accessing the next-leftmost
element behind it--that usually violates some sense of a-list/stack discipline.

You haven't offered an operator whose goal is to be like destructuring-bind
and so to get around this, so the burden seems, to those looking on, to be
on the programmer to pick apart this structure manually and the set of tools
seems light.  That's probably only an artifact of not seeing your tools,
rather than anyone's belief that you have no such tools.

Likewise you haven't shown any syntax which is, by loose analogy, the
equivalent of Lisp's arglist strangeness for keywords where you map a keyword
to a differently-named variable by doing
   (lambda ((:foo fu) 3) fu)
It is by having an abstraction like this that you can assure the person
that the caller's name for things will not confuse the callee.  I toyed with
coming up with an analogously absurd example for Lisp and the following
was my best go of it.  If it seems unhelpful, just ignore it.  But the point
is just to show that you can manage
(let ((weird 'ee) (apartheid 'ii) (pie 'ii) (pier 'ee))  
  (labels ((fn1 (&key ((:ei e)) ((:ie i))) (list 'ee e 'ii i))
           (fn2 (&key ((:ie e)) ((:ei i))) (list 'ee e 'ii i))
           (sort-by-sound (&rest keys &key (first-vowel-wins-p t)
                           &allow-other-keys)
             (apply (if first-vowel-wins-p #'fn1 #'fn2)
                    :allow-other-keys t
                    keys)))
    (list (sort-by-sound :first-vowel-wins-p t   :ei 'weird     :ie 'pie)
          (sort-by-sound :first-vowel-wins-p nil :ei 'apartheid :ie 'pier))))
((EE WEIRD II PIE)
 (EE PIER  II APARTHEID))

That is, somehow you'd expect the external representation (:ie vs :ei) to
have a fixed effect on what two functiosnt that each have the same body
might return, but the arglist mappings (the "magic" in your example, of a
different kind than the "magic" here of &key, but still magic in a way)
manage to sort things out.  It isn't their behavior but the cross-bar you
plug between them that is doing something cool, and people don't see what
that cool thing is, probably only for lack of specificity rather than
disbelief that what you say might be true.  Just as my example above is ho-hum
to a Lisp programmer, not mysterious, once they understand how keyargs work.