Our primary goal in considering a switch to SGML is to position ourselves to respond to the developing
market for electronic delivery of professional journals and books, whether that be in the form of
CDROM, online access, or something other. Of course we would still deliver our print products and would
expect that the cost to do so would be maintained or reduced as would the length of the publishing cycle.

I recently attended the GCA SGML tutorial (all 5 days) and feel that I have a workable understanding of
the basic syntax. I have, however, a couple of questions regarding the selection or creation of a DTD.
My questions are listed below. I welcome all replies.

As I sidenote, this is my first time using the InterNet and I don't know how to get around very well.
Please keep this in mind if you reference other files, systems, or newsgroups.

1) Are there any disadvantages to using a variant syntax?
Because the system we currently use is ``SGML-like'', one of the options I am considering is developing
our own DTD and making as few changes as possible to our existing coding structure. This would have the
advantage of reducing the amount of training required to transition our editors to the new system. Our
current system uses a long sequence of codes as open and close delimiters. For instance a start tag might
look like the following:

     [md1]{tagname}[mdnm]

and an end tag like:

     [md1]{/tagname}[mdnm]

*NOTE* the [ and ] characters in the above syntax are ASCII characters 174 and 175.

As I understand the SGML standard, it would be possible for me to continue using this structure if I define
it in an SGML declaration. This would mean that our system would be based on a variant syntax. Are there any
disadvantages to using a variant syntax? Is this supported by the available SGML software tools? Would I
encounter problems delivering these files to other SGML systems? These files will be used for print composition
on a variety of systems at outside vendors as well as in house. They will also be used for some form(s) of
electronic delivery.

Mark T. Jacobson asked:

"If I create my own DTD will I reduce the reusablilty of my files as
opposed to using an industry standard DTD such as the AAP's.  Are
there other DTDs I should consider?"

Yes!  There will shortly be de jur standard DTDs for articles, books
and serials, based on the AAP DTDs.  I don't know if they have a
catchy name, but the standard is ISO 12083.   Let's hope these don't
remain only de jur, but become widely accepted and used.

For more information, you may want to contact the editor of the standard,
Eric van Herwijnen, at E...@crnvma.cern.ch.

The advantage of using standard DTDs is the advantage of using any
standard: one can hope that a support industry will grow up around the
standard.

Tom Gordon

p.s. I am in the process of porting my "qwertz" SGML to LaTeX system to
use these ISO DTDs.   The converter from qwertz to ISO is working, but
I've yet to write the converter from ISO to LaTeX.

|   If I create my own DTD will I reduce the reusablilty of my files as
|   opposed to using an industry standard DTD such as the AAP's.  Are
|   there other DTDs I should consider?

[Tom Gordon]

|   Yes!  There will shortly be de jure standard DTDs for articles, books
|   and serials, based on the AAP DTDs.  I don't know if they have a catchy
|   name, but the standard is ISO 12083.  Let's hope these don't remain
|   only de jure, but become widely accepted and used.

To say that ISO 12083 is based on the AAP DTDs is akin to consult the Devil
about the Bible.  ISO 12083 was voted on as the AAP DTDs in ISO's "fast-
track" procedure, and thus people have reason to believe it will continue
to be based on them.  However, ISO 12083 has had all its GI's altered, much
of the structure changed, and has numerous extra features compared to the
original DTD's, many of them even useful.

|   The advantage of using standard DTDs is the advantage of using any
|   standard: one can hope that a support industry will grow up around the
|   standard.

One can also regard standard DTDs as a complete waste of time and effort.
DTDs should be defined within specific communities that expect to exchange
information, as the AAP did among American Publishers.  Elevating such
particular communities and their interchange needs to the level and
prestige of International Standard is to tell people who do not have those
needs that they are not important and if an industry grows around ISO 12083
instead of ISO 8879, we will have lost 25 years of progress in information
processing.  Already, some people make editors for particular DTD's, quite
contrary to the intent of the SGML standard and the raison d'etre for the
whole language.  Now, with this souped-up and rehashed AAP DTD set, they
will have even less reason to do things right.

ISO 12083 was accepted by the international standards community as quite a
different standard than what will hit the streets.  Regardless of whether
this is politically correct or not, or whether the people who voted on this
thing was just rubber-stamping it through ISO and didn't even look at the
horribly ugle mess that the AAP DTD's were at that time, and it is a great
service to Mankind to override their decision to accept the AAP DTD's, it
still would have been polite, if not moral, to ask people to accept what
will actually be published.  Fair enough that a lot of the voting that goes
on in ISO is done by people who don't even read the title of the drafts,
but not letting those who might have cared comment on an _entirely_ _new_
_standard_ that differs not only in word, but in spirit, from the subject
of the vote, rings all my alarm bells and spells disaster for the process
of fast-tracking other standards.  There is no telling _what_ ISO will
publish after things are voted on and accepted.

I have no doubt that ISO 12083 will be adopted, in whole or in part, by a
large number of people, and that we will see several companies offering
ISO 12083-compatible, ISO 8879-violating products in the near future.  The
marketdroids will no doubt be unable to distinguish between "SGML" and "ISO
12083" and the used car salesmen in the software market will lead unwary
customers to buy Edsels labeled "SGML".  This will hurt SGML in the short
run, and the investment people make in their information in the long run.

It was a mistake to accept ISO DIS 12083 because of its lack of quality and
because DTDs simply are not standards material, and it will be a mistake to
publish ISO 12083 as a completely different standard because of the
procedural problems involved.  The present ISO 12083 text may be useful and
may even be good as an application, but it still should not be an ISO
standard.  That it became an ANSI standard in the first place is hard to
believe.

There are merits to providing commonly known generic identifiers for a
number of commonly used element types, and the structure of some low-level
element types can be hard to get right.  This is akin to libraries that
come with programming languages, most of whose routines are very hard to
get right.  But to standardize whole applications, whole programs, is just
plain non-sense.  If ISO 12083 is used as a library, as a resource, that is
very good.  If it is used by software developers to hard-wire semantics to
particular generic identifiers, it is a disaster.

This is just like conformance testing standards -- nice to have, but not as
an international standard.  Standards often increase their value beyond
their technical merits because they are standards, but things do not become
good just because they are standards.  Some standards are really bad, such
as ODA (T.410), OSI (X.200), and MHS (X.400), and they only get worse once
the standards people start to "improve" it at the rate of several technical
addenda and corrigenda a year.

I suggest that organizations who consider using ISO 12083 take good, long
look at it and see what they can use.  Do not under any circumstance use it
as a whole, and do not believe that there is anything more in a generic
identifier in SGML than there was before this "standard" came about.  Such
special semantics should be expressed using architectural forms, and
identified as such.  HyTime got it right, and although HyTime is complex
because of it, we are all much better off.  Some of the stuff in ISO 12083,
at least last time I saw it, used attributes where processing links would
have been much, much better, as Steve Pepper demonstrated in his excellent
article.  There are many ways to standardize semantics and to allow people
to share information in useful ways.  Standardizing DTD's is not among them.

Best regards,
</Erik>
Sunnyvale, CA

If they don't already know about it, European followers of this news group
may be interested in a two-day workshop in Luxembourg, 2-3 December 1993.
This is being organized as part of the CEC's Open Information Interchange
(OII) initiative. Day 2 is entitled "Standardization of SGML DTDs". I will
try to have the programme posted here.

On balance I'm in favour of publishers agreeing a common DTD for interchange
but developing their own DTD - closely related to the common DTD, of course -
to satisfy internal idiosyncrasies.

I don't have much to say about whether or not the procedure by which
ISO 12083 is coming about was proper or desirable.  I'm just a user
and have had no involment in the standardization process.

I also do not feel particularly competent to judge its technical
merits as a general purpose DTD for articles, books and serials.  My
only previous experience in this area is as the author of the qwertz
DTD, which was intended to be a reconstruction of LaTeX in SGML.

In this reply I just want to defend the *idea* of a standard, general
purpose DTD for articles and books.  In my community, which is
academic computer science, (in particular artificial intelligence),
LaTeX is the de facto standard document exchange format.  It is far
from uncommon for authors to work together on an article or book,
using LaTeX and electronic mail.  

For *my* purposes, the main problem with LaTeX is that there is no
nice structure editor for it, let alone WYSIWYG editor, along the
lines of Author/Editor for SGML. I created the qwertz system so as
to be able to use Author/Editor. But LaTeX, not SGML, remained the
document exchange format of choice. It may not be a de jur standard,
but is what (most) people in my community want and expect.

The qwertz system makes it possible to use SGML editors to create
LaTeX documents, but so long as LaTeX remains the accepted exchange
format, there is a natural tendency to remain tied to TeX for
formatting and printing.  Whatever TeX's merits, and they are
numerous, this is unnecessarily restrictive. It should be possible,
even easy, to format documents marked up in a descriptive markup
language using just about any formatting system, such as troff, lout,
Framemaker or even Word.

This is where I would hope that a set of standard DTDs, like ISO
12083, would have its merit. I agree completely with Eric that no DTD
is suitable for every task, but presumably a DTD with comparable
expressiveness to LaTeX could be as useful to the same community which
now regularly uses LaTeX.

Despite what I said about not wanting to defend ISO 12083 in
particular, there is at least some anecdotal evidence that it would be
suitable as a de jur functional equivalent to LaTeX: I wrote a
translator from the qwertz DTD -- which again is modelled after LaTeX,
into ISO 12083 and then used this to translate my dissertation.  There
are few (see below) LaTeX constructs in my qwertz reconstruction which
cannot be adequately modelled by ISO 12083.

This said, it remains to see whether being official, ISO standard DTDs
will do much to help SGML compete with LaTeX in my community. The AAP
DTDs, despite their technical limitations, would also have done the
job, but have not caught on.  

The main reasons that neither the AAP DTDs nor SGML in general have
caught on in my community are, I believe:

        1) The lack of a standard DTD comparable to LaTeX, and
        2) The lack of free and widely available and supported systems
for formatting and printing SGML documents.

There are a few systems, like qwertz, which address point 2, but both
points have to be addressed before SGML stands a chance to displace
LaTeX.

Therefore, as an SGML fan, I hope that Erik is wrong about ISO 12083.

One more thing: I implied above that there were indeed a few LaTeX
constructs I needed with no equivalent in ISO 12083.  These are the
various kinds of "theorem environments" for definitions, theorems,
corollaries, etc.  In such cases, my suggestion is to *extend* ISO
12083 with custom elements for your application.  This is much easier
than starting from scratch, and your documents will be fail to conform
to the standard only in minor and well-documented ways.

Dr. Thomas F. Gordon
GMD, FIT-KI; Schloss Birlinghoven
53757 Sankt Augustin / Germany
email: thomas.gor...@gmd.de;  phone: (+49 2241) 14-2665

[Tom Gordon]

|   In this reply I just want to defend the *idea* of a standard, general
|   purpose DTD for articles and books.  In my community, which is academic
|   computer science, (in particular artificial intelligence), LaTeX is the
|   de facto standard document exchange format.  It is far from uncommon
|   for authors to work together on an article or book, using LaTeX and
|   electronic mail.

It seems that you are just stressing my point, namely that communities not
only should, but in fact will, develop common document applications.  LaTeX
came about because one man (Les Lamport) decided to build this system.  It
is great, it is free (and so is TeX, also one man's work), and it is so
widespread as to be ubiquitous, but _only_ in the academic world.  It is
not a commercially supported product, and all the support tools I have seen
(only a few) for LaTeX are also free and are mainly used in Academia.
Publishers who accept LaTeX documents are usually in or around Academia,
too, or they would much rather accept PostScript pages.  I do not use LaTeX
myself, so I do not know too much about it, but what I do know is that
"LaTeX" is not an unambiguous term -- lots of local variations exist, and
not infrequently, the latex.tex file has to be included for things to work
properly.

LaTeX is not a standard defined and agreed upon by an international
community as an independent third party, but a tool offered to this
community by its designer, and that continues to evolve and minor changes
are communicated among those who use it.  This is the "community spirit"
that makes community-wide standards such a brilliant idea.

However, my point is to argue that the whole wide world is not a single
community with any common "community spirit", that when the system takes
years to absorb desirable changes, deviations will supplant the standard,
that when users are faced with hard-wired semantics and little or no
ability to make the small changes that have made LaTeX so able to survive,
we are _worse_ off with an international standard than if the same thing
were just a commonly used tool in a (large) community.

|   But LaTeX, not SGML, remained the document exchange format of choice.
|   It may not be a de jure standard, but is what (most) people in my
|   community want and expect.

Precisely!  LaTeX is a community standard, and this is great (although not
as great as if it were SGML :-), but whether it is de facto or de jure has
no influence on this.  Academicians tend to want more than de jure
standards can give them, anyway.  That is how the Internet and USENET
became so popular.  Now the U.S. has formally proposed that X.400 MHS use
Internet-style addresses, too.

Good ideas are used whether they are international standards or not, and
the prestige and force behind international standards should be used for
things that are enhanced by them, not by things that are hampered by them.
Languages are typically good candidates to become standards, likewise
protocols, connectors and interfaces, physical storage formats, etc.
Things that can be mass-produced and that are not likely to be customizable
beyond what the specification allows.  For inherently customizable things,
such as user applications, standards are just plain bad ideas.  Look at
OSI, and how convoluted all their standards at the application level have
become.  I contend that applications are hampered by being standardized.
The market can handle applications, but standards should handle all the
bits and pieces that go into them, so the prices can fall, repair can be
much cheaper, etc.  That is what happened when the U.S. car industry adopted
standard nuts, bolts and screws instead of having engineers design their
own for each new car model.  A standard _car_, however, is not a good idea.

Also keep in mind that the "International Standard" label tends to focus on
what is useful _internationally_.  Does ISO 12083 qualify?  As far as I
have seen, the AAP DTD's were barely acceptable as _American_ standards.

|   I agree completely with Erik that no DTD is suitable for every task,
|   but presumably a DTD with comparable expressiveness to LaTeX could be
|   as useful to the same community which now regularly uses LaTeX.

Agreed, but you do not need an international standard to accomplish this,
and what about the other communities?  Remember that Academia is used to
research tools, normally available for free within the community, and there
is not going to be any "industry" growing up around free research tools.
Admittedly, graduate students are turning to Word, WordPerfect, Framemaker,
etc, to produce their theses, because LaTeX is so much harder to use.  Of
course, the advisors who allow that must not think very highly of their
students work to let their work go down the drain instead of requiring an
electronic document format searchable by their peers and the next
generation of students.

|   The main reasons that neither the AAP DTDs nor SGML in general have
|   caught on in my community are, I believe:
|      
|       1) The lack of a standard DTD comparable to LaTeX, and
|       2) The lack of free and widely available and supported systems for
|          formatting and printing SGML documents.
|  
|   There are a few systems, like qwertz, which address point 2, but both
|   points have to be addressed before SGML stands a chance to displace
|   LaTeX.

At the time when troff was king, there were several communities that used
different macro packages.  LaTeX is not at all the be-all and end-all of
document systems, it just happens to be more widely used than others.  The
same can be said of WordPerfect in the general market, yet Emacs is a much
more powerful editor than this puny little word processor.

I disagree strongly with the requirement that things be "standard" and
"free" in order to succeed.  MS Windows is neither.  Windows is technically
inferior to an ant hill, and almost as full of bugs, yet it is out-selling
the proverbial hotcakes, even in academic circles.

I think your work with qwertz is a good idea, and I applaud it, but I am
not so happy with the arguments that you propose for it.  SGML cannot
compete with LaTeX on LaTeX's terms and expect to win.  For SGML to win,
_more_ than what LaTeX provides must be accessible, and ready to use.  The
University of Oslo is now working on a project "UNIPUB" to aid production
and demand-printing of academic literature using SGML, and LaTeX would not
stand a chance competing on this ground.  I think it is things like this
that will bring SGML to the academic community, not LaTeX-vs-SGML fights.

|   Therefore, as an SGML fan, I hope that Erik is wrong about ISO 12083.

But, don't you see?  There is nothing in ISO 12083 that will magically turn
it into a success.  There is nothing in being an international standard
that guarantees success, either.  There is a continuous fight for SGML
going on all over the place, and some places SGML wins, and others SGML
loses to some proprietary format.  I want to clip from an article posted to
the ISO 10646 mailing list by Alain LaBonte (who will be angry that I
omitted the acute accent in his name) to show you just how sad the case is.
Alain has an idea about the portability of source code using rich character
sets, and here is how he argues:

    This measure is not so revolutionary when one thinks that for text,
    street people are used to exchange WP documents in an user-unreadable
    format such as WordPerfect format, MS-Word format and so on.  So why
    not for program texts?

(Note: "street people" means "the man in the street", not the homeless.)
No comment required.

What I am most concerned about is that ISO 12083 will be viewed as _the_
SGML application, that everybody should use, and the demerits of having
_one_ SGML application be an international standard are numerous and very
significant.  If ISO 12083 is any good, it should catch on without the ISO
imprimatur.

|   One more thing: I implied above that there were indeed a few LaTeX
|   constructs I needed with no equivalent in ISO 12083.  These are the
|   various kinds of "theorem environments" for definitions, theorems,
|   corollaries, etc.  In such cases, my suggestion is to *extend* ISO
|   12083 with custom elements for your application.  This is much easier
|   than starting from scratch, and your documents will be fail to conform
|   to the standard only in minor and well-documented ways.

... which will break the blind interchange principle, and will force small
communities to come up with their own conventions for breaking the
standard.  Presto!  You have community standards for each community, and
the value of ISO 12083 has been reduced to a resource from which people can
pick and choose, rather than an actual standard.  Quad erat demonstrandum.

Best regards,
</Erik>

Just a couple of follow-up points to Erik Naggum's reply to my message
about the ISO 12083 DTDs:

1. I actually agree strongly that one should be very careful before
enacting an international standard.  My main experience here is in the
area of programming languages.  The lesson I've learned is that a
language is ripe for de jur standarization by some standards body like
ISO *only* after it has become a widely accepted de facto standard.
Positive examples include ANSI C and IEEE Scheme.  Negative examples
include, above all, Ada.   A de facto standard has proven its value.
A de jur standard which is *only* a de jur standard has been approved
only by the standards committee, but not yet by its intended users
community.

2. Erik writes that "SGML cannot compete with LaTeX on LaTeX's terms
and expect to win."  Agreed, but we can agree, I suppose, that an
SGML-based functional equivalent to LaTeX offers significant
advantages, such as (text) editor, company and formatter independence.
I do not really care so much whether the DTD which competes
successfully with LaTeX is an ISO standard or not.  I would just like
to see *some* DTD comparable to LaTeX become widely used by *my*
community.  The question is, which one?  How does one get the ball
rolling?  Starting at the top, with an ISO standard, instead of from
the initiative of users, is surely not the *best* way to proceed, but
it is arguably better than not proceeding at all.

3. I am not in favor of the "blind interchange principle".  Again, I would
encourage users, or groups of users, to modify and extend ISO 12083
to suit their needs.   There are precedents for this.  Just about
every producer of a compiler for some standard programming language or
operating system offers their own extensions.  

That's it for now.

Tom Gordon

    What very well could happen is what's happening in the US with the CALS
    standards (MIL-M-28001). It's being cited all over the place and in many
    cases it doesn't apply. I can see this happening with ISO 12083.

    I wonder if there's a way to get an ISO standard withdrawn??

I think a far more productive activity would be to work on standardising
a few general purpose DTD fragments. For example, I have a chunk o' DTD
for references which is more or less a naive encoding of MARC. Something
like that would be very useful to standardise; processing it nicely takes
some fairly complicated code; however that code can be reused whenever I
use it with a different DTD.

The chances of me getting my personal report DTD accepted by NISO are
not too good, and quite rightly so. However a standard way of handling
bibliographies would be much more useful and generally applicable.

Hmmm -
 Quick question; could a parameter entity reference expanding into
several ELEMENT defines occur in a DOCTYPE ?

I'm not sure that I have completely recovered yet from the last contact
I had with MARC, but I would still be interested in taking a look at
how such an implementation would look (posting a chunk o' DTD to
comp.text.sgml should be seen as the first step on the road to becoming
one of the standards :-)

A big problem with DTD fragments is going to be name-space collisions. I
find this with the TEI fragment: my DTD already has <title> and <author>
elements, and a system of phrases etc. The way in which bibliographic
information is to be encoded can have ramifications elsewhere in the
DTD. I hereby reserve the name "Snafu" for my DTD, unless a popular
movement arises to make it a synonym for DTD in general :-)

A question about the TEI DTD, just in case the relevant authorities are
reading:

Why is the biblScope attribute not permitted at the analytic level?
It seems like a logical place to put the page numbers of an extract,
rather than at the monograph level as suggested.

Also a note, in case this has not already been fixed (is there a recent
DTD fragment around somewhere? I am still missing several parameter
entities):

There are a few typos in the P2CO document. <publ> is used instead of
<publisher> in one example. <individual> is used in several places
where <indiv> is presumably intended. The content model for monogr has
(edition, (editor | resp)*) where (edition, (editor | resp))* would be
more reasonable (cf. examples which otherwise fail).

  <!ENTITY % n.title 'bibTitle' >
  <!ENTITY % n.author 'bibAuthor' >

(This also makes it easier to use DTDs in a language other than
English; unfortunately, you have to put up with English in the
attribute names or make somewhat more serious changes.)

I should note in passing that the choice to use the shorter GI 'title'
for bibliographic references rather than for the title of the document
is an instance of a general design principle which the TEI attempts to
apply, and which I recommend to everyone:  use the shorter name for
the more common element, and qualify the other one.  Hence in the
TEI DTD, 'title' refers to the title of a work being mentioned, and
'docTitle' to the title of the work being created, because the
latter typically appears just once, on the title page, while there
may be *lots* of bibligraphic references.

|  A big problem with DTD fragments is going to be name-space collisions. I
|  find this with the TEI fragment: my DTD already has <title> and <author>
|  elements, and a system of phrases etc. The way in which bibliographic
|  information is to be encoded can have ramifications elsewhere in the
|  DTD. I hereby reserve the name "Snafu" for my DTD, unless a popular
|  movement arises to make it a synonym for DTD in general :-)

The way to avoid name space collisions is to define architectures, not
DTDs (although the architecture may be defined as a DTD "template", as
HyTime is).  In the simple case, the concrete instantiation of the
architecture uses the architectural form names as the actual element
GIs (e.g., nameloc GI used for instances of Nameloc architectural
forms), but concrete DTDs can also uses other GIs as well, since the
architecture-level processing only looks at the architectural form
name.  Name conflict between archtectures is avoided because each
architecture has its own "architectural form declaration attribute",
e.g. HyTime= for the HyTime architecture, InfoMaster= for the
InfoMaster Architecture, TEI= for the TEI architecture.

--
Eliot Kimber                      Internet:  drma...@vnet.ibm.com
Dept E14/B500                     IBMMAIL:   USIB2DK9@IBMMAIL
Network Programs Information Development     Phone: 1-919-254-5160
IBM Corporation
Research Triangle Park, NC 27709