xmltools

[tomh]

I have been experimenting with Glen's XMLTOOLS (they
are working very well...) and the question
has arisen, "How should the XML doc be formatted, with
respect to values, and subvalues?" Glen has suggested I
post this question to the group.

Below are 2 examples of what COULD be done, but I "troll" here
to see what SHOULD be done.

First example: DTD with XML doc. This is unmapped data, no
dictionaries are being opened yet, QPROC is doing all the work.
The VALUES tag has (xml) attributes describing where the
values and subvalues reside in the (items) attribute. :/

Second example: XML doc. (ver.4 has no dtd maker)
This is unmapped data also. We would depend on
levels in this example, for values and subvalues.

These are merely examples and serve as a jumping off point,
its not like you have to choose between these 2. All suggestions
and/or comments requested.

To begin, I have no opinion yet. Glens programs will parse both
examples equally well. Although, if dictionaries are used,
and/or if a 'mapper' is in place... I guess the question is
"How do you want values and subvalues to look?"

Thanks, Tom H

:RUN XBP XTEST6
input sentence: ?LIST STUFF 13 16 COLOR CAR VALUES FRUIT
<?xml version="1.0" encoding="UTF-8"?>
<!ELEMENT STUFF (item)*>
<!ELEMENT item (ID, COLOR, CAR, VALUES, FRUIT)>
<!ELEMENT ID (#PCDATA)>
<!ELEMENT COLOR (#PCDATA)>
<!ELEMENT CAR (#PCDATA)>
<!ELEMENT VALUES ANY>
<!ATTLIST VALUES vmc CDATA #IMPLIED
smc CDATA #IMPLIED>
<!ELEMENT FRUIT (#PCDATA)>

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE STUFF SYSTEM "XTEST6.DTD">
<STUFF>
<item>
<ID>13</ID>
<COLOR>GREEN</COLOR>
<CAR>ESCALADE</CAR>
<VALUES>
<VALUES>
<VALUES vmc="1" smc="1">FOUR</VALUES>
<VALUES vmc="1" smc="2">four</VALUES>
<VALUES vmc="1" smc="3">4</VALUES>
</VALUES>
<VALUES vmc="2">CCC</VALUES>
<VALUES vmc="3">u</VALUES>
<VALUES vmc="4">Glass</VALUES>
</VALUES>
<FRUIT>PEACH</FRUIT>
</item>
<item>
<ID>16</ID>
<COLOR>YELLOW</COLOR>
<CAR>CORVETTE</CAR>
<VALUES>
<VALUES vmc="1">SIX</VALUES>
<VALUES vmc="2">AAA</VALUES>
<VALUES vmc="3">y</VALUES>
<VALUES vmc="4">Glass</VALUES>
</VALUES>
<FRUIT>PEAR</FRUIT>
</item>
</STUFF>
:

:RUN XBP XTEST4
input sentence: ?LIST STUFF 13 14 COLOR CAR VALUES FRUIT
<?xml version="1.0" encoding="UTF-8" ?>
<STUFF>
<item>
<ID>13</ID>
<COLOR>GREEN</COLOR>
<CAR>ESCALADE</CAR>
<VALUES>
<VALUES>
<VALUES>FOUR</VALUES>
<VALUES>four</VALUES>
<VALUES>4</VALUES>
</VALUES>
<VALUES>CCC</VALUES>
<VALUES>u</VALUES>
<VALUES>Glass</VALUES>
</VALUES>
<FRUIT>PEACH</FRUIT>
</item>
<item>
<ID>14</ID>
<COLOR>GREEN</COLOR>
<CAR>MUSTANG</CAR>
<VALUES/>
<FRUIT>PEACH</FRUIT>
</item>
</STUFF>
:

[Pickie]

Hi Tom,

I don't think the vmc and smc counters should be in by default -
personally, I see this as a hang-over from the MV side. However, I
think that people might like to be able to flag that they want them.
If you do have them, shouldn't you also have the amc numbers? Mmmm,
maybe not...

In the case where we don't have the counters, I think it would be nice
to have single-valued attributes default to come out as XMLvalues
rather than XMLattributes. Example below (assuming that the dictionary
definitions of @ID, COLOR, etc. have "S" in the correct d-type
position).

<item @ID="13" COLOR="GREEN" CAR="ESCALADE" FRUIT="PEACH">

<VALUES>
<VALUES>
<VALUES>FOUR</VALUES>
<VALUES>four</VALUES>
<VALUES>4</VALUES>
</VALUES>
<VALUES>CCC</VALUES>
<VALUES>u</VALUES>
<VALUES>Glass</VALUES>
</VALUES>

</item>

A keyword (like ATTRIBUTE?) would force the more verbose output.

Another wrinkle you might want to consider is when using a phrase for
associated multivalues. Let's assume a file INVOICE with a dictionary
phrase called LINE.ITEMS which has in it the three multivalues CODE
QUANTITY PRICE. I would like to see a command like

LIST INVOICE XML Customer Delivery.Date LINE.ITEMS

come out in XML something like

<INVOICE @ID="13" Customer = "123456" Delivery.Date="12 Jan 2006">
<LINE.ITEMS CODE="AS781287" QUANTITY="3" PRICE="42.34" />
<LINE.ITEMS CODE="BZ8934" QUANTITY="2" PRICE="12.99" />
</INVOICE>

and

LIST INVOICE XML Customer Delivery.Date LINE.ITEMS ATTRIBUTE

would come out

<INVOICE>
<@ID>13</@ID>
<Customer>123456</Customer>
<Delivery.Date>12 Jan 2006</Delivery.Date>
<LINE.ITEMS
<CODE>AS781287</CODE>
<QUANTITY>3</QUANTITY>
<PRICE>42.34</PRICE>
<LINE.ITEMS>
<CODE>BZ8934</CODE>
<QUANTITY>2</QUANTITY>
<PRICE>12.9</PRICE>
</INVOICE>

"TO pathname" allows you to send the output to a file just like it does
for CSVand DELIMITER.

[Pickie]

Oh - another point. You organised the output like this

<STUFF>
<item>...</item>
<item>...</item>
<item>...</item>
</STUFF>

But I would prefer it like this

<file>
<STUFF>...</STUFF>
<STUFF>...</STUFF>
<STUFF>...</STUFF>
</file>

There IS a keyword "FILE" - can we overload it to allow me to choose
this display option?

[GlenB]

So you are going to rely on DOM node to determine the MV index? What
does VALUES point to in the real record? Here's what I would know:

ID is "13"
Dictionary COLOR is "GREEN"
Dictionary CAR is "ESCALADE"
Dictionary FRUIT is "Peach"

Where does "VALUES" exist and where do I put them back? If I go by DOM
node, it looks like:

RECORD<1,1,1> = "FOUR"
RECORD<1,1,2> = "four"
RECORD<1,1,3> = "4"
RECORD<1,2> = "CCC"
RECORD<1,3> = "u"
RECORD<1,4> = "Glass"

This would be easy to accomplish.

>
> "TO pathname" allows you to send the output to a file just like it does
> for CSVand DELIMITER.

More comments to come, but I gotta go for now....

Glen

[Pickie]

Maybe I got a bit orthogonal to Tom's original post. I was thinking
of XML output using the dictionary entries, rather than what to do when
you don't (or can't) use them.

As I recall, Pick Systems had a way to output XML that had value and
subvalue numbers embedded in it. At the time I thought it was arguable
as to whether they should appear or not. I had a quick prowl of their
site, but couldn't find it.

If you want a general-purpose mechanism that is ignorant of _any_
dictionary information, Wouldn't it have to be able to handle
attributes as well?

Here's another suggestion. If we had data like the following,

0001: AAA
0002:
0003: BBB1ÿBBB2.1üBBB2.2üBBB2.3ÿBBB3
0004: CCC

then this could be represented as

<a i='1,1,1'>AAA</a>
<a i='3,1',1>BBB1</a>
<a i='3,2,1'>BBB2.1</a>
<a i='3,2,2'>BBB2.2</a>
<a i='3,2,3'>BBB2.3</a>
<a i='3,3',1>BBB3</a>
<a i='4,1,1'>CCC</a>

"a" and "i" could be VALUES and POSITION, or whatever you think
appropriate. I used "a" and "i" because I'm lazy, but I don't see why
we would want to waste bandwidth either. I didn't see there was any
real need to break out the seperate parts of the position (like am='1'
vm='1' sm='1').

This is about as minimalist as you can get apart from just sending the
pick item as a string. It should build the same Pick item no matter
what order it is processed.

Regards, KeithJ

[Homer Hazel]

Keith,

I took your XML like stuff and modified it so that it
was a program to see what would happen.

As you knew, it created your data perfectly.

There is something I do not understand the
significance of and that is your use of the
apostrophe in your xml like string.

In the first line you use i='1,1,1' and in
the second line, you use i='3,1',1

I think I know why you did it, but I wonder
if you actually need the apostrophe at all?

Larry Hazel

Pickie wrote:

> <><snip>

[GlenB]

Pickie wrote:
> Maybe I got a bit orthogonal to Tom's original post. I was thinking
> of XML output using the dictionary entries, rather than what to do when
> you don't (or can't) use them.
>

The question is: "how should attributes be represented when there is
no dictionary specified for them?"

> As I recall, Pick Systems had a way to output XML that had value and
> subvalue numbers embedded in it. At the time I thought it was arguable
> as to whether they should appear or not. I had a quick prowl of their
> site, but couldn't find it.
>

There are several ways to embed the relationships. One way is to
specify order as element attributes for the field in question. Another
way is to nest the same tag name in hashing order of the data, which is
what I thought you were doing. The order of tag presence and nesting
specifies the depth and order of each piece of information.

<A>
<A>1</A>
<A>2</A>
</A>

001: 1]2

<A>1</A>
<A>2</A>

001: 1
002: 2

> If you want a general-purpose mechanism that is ignorant of _any_
> dictionary information, Wouldn't it have to be able to handle
> attributes as well?
>
> Here's another suggestion. If we had data like the following,
>
> 0001: AAA
> 0002:
> 0003: BBB1ÿBBB2.1üBBB2.2üBBB2.3ÿBBB3
> 0004: CCC
>
> then this could be represented as
>
> <a i='1,1,1'>AAA</a>
> <a i='3,1',1>BBB1</a>
> <a i='3,2,1'>BBB2.1</a>
> <a i='3,2,2'>BBB2.2</a>
> <a i='3,2,3'>BBB2.3</a>
> <a i='3,3',1>BBB3</a>
> <a i='4,1,1'>CCC</a>
>

That format is what we need to vote on. If everyone agrees that
specifying the index is better than order nesting a special tag name,
then we can move on to specifications of the element index
attribute(s).

Glen

[Steve Bush]

The order nesting and indexing forms are not mutually incompatible. How
about a combination of the two forms to take advantage of the space saving
effect of the indexing form for sparse arrays AND the space saving effect of
the non indexed form for well filled dynamic arrays?

0001: AAA
0002:
0003: BBB1ÿüüBBB2.3ÿBBB3
0004: CCC

Could be represented as

<a>AAA</a>
<a i='3'> <a>BBB1</a> <a><a i='3'>BBB2.3</a></a> <a>BBB3</a>
<a>CCC</a>

I note that the indexed form (particularly with multilevel indexing) will
require more advanced parsers whereas the dumb xml nesting will be more
universally understood. Using only a single level of indexing (to indicate
the field, value or subvalue of whatever level you happen to be at) would be
easier for universal parsing on other systems.

Cheers,
Steve Bush

[GlenB]

Steve Bush wrote:
> The order nesting and indexing forms are not mutually incompatible. How
> about a combination of the two forms to take advantage of the space saving
> effect of the indexing form for sparse arrays AND the space saving effect of
> the non indexed form for well filled dynamic arrays?
>

Are there really any space-savings aspects of either? It's messy
either way you set it up. Putting the two formats together just makes
things even more complicated to decipher. That's true visually and
programmatically.

> 0001: AAA
> 0002:
> 0003: BBB1ÿüüBBB2.3ÿBBB3
> 0004: CCC
>
> Could be represented as
>
> <a>AAA</a>
> <a i='3'> <a>BBB1</a> <a><a i='3'>BBB2.3</a></a> <a>BBB3</a>
> <a>CCC</a>
>
> I note that the indexed form (particularly with multilevel indexing) will
> require more advanced parsers whereas the dumb xml nesting will be more
> universally understood. Using only a single level of indexing (to indicate
> the field, value or subvalue of whatever level you happen to be at) would be
> easier for universal parsing on other systems.
>

Thus my question as to which one of the two is more preferable. I
really don't think that combining the two is a good idea for unknown
data. Since there won't be a DTD, it needs to be a simple format that
anyone can implement with any DOM or SAX tool. I, personally, like the
idea of assigning a specific tag name and an MV-specific index
attribute. Using native XML nesting is the other option, but I have a
feeling that using the raw structure in combination with dictionary
elements will confuse a lot of people.

If we normally say:

<COLOR>blue</COLOR>
<CAR>Escalade</CAR>

Then I think it's more comfortable to say:

<MVFIELD mvref="2,1,3">1234</MVFIELD>

Instead of:

<MVFIELD/>
<MVFIELD>
<MVFIELD>
<MVFIELD/>
<MVFIELD/>
<MVFIELD>1234</MVFIELD>
</MVFIELD>
</MVFIELD>

> Cheers,
> Steve Bush
>

Glen

[Steve Bush]

> -----Original Message-----
> From: Ope...@googlegroups.com
> [mailto:Ope...@googlegroups.com] On Behalf Of GlenB
> Sent: 03 August 2006 22:28
> To: OpenQM
> Subject: Re: xmltools

<snip>

> Then I think it's more comfortable to say:
>
> <MVFIELD mvref="2,1,3">1234</MVFIELD>
>
> Instead of:
>
> <MVFIELD/>
> <MVFIELD>
> <MVFIELD>
> <MVFIELD/>
> <MVFIELD/>
> <MVFIELD>1234</MVFIELD>
> </MVFIELD>
> </MVFIELD>
>

I was thinking of something like this which will be briefer for large
dynamic arrays with many repeating multivalues and seems more in tune with
XML and PICK spirit than flat lists.

<MVFIELD i=2>
<MVFIELD>
<MVFIELD i=3>1234</MVFIELD>
</MVFIELD>
</MVFIELD>

On your other subject about whether to include field names perhaps they
could be optionally in like this? This wouldn't be so easy in the multilevel
indexing scheme which is perhaps a hint that flattening the format has some
disadvantages.

<MVFIELD i=2 name=TEL_NOS>
<MVFIELD>
<MVFIELD i=3>1234</MVFIELD>

[Pickie]

Homer Hazel wrote:

<snip>

> In the first line you use i='1,1,1' and in
> the second line, you use i='3,1',1
>
> I think I know why you did it, but I wonder
> if you actually need the apostrophe at all?

<snip>

Hi Larry,

just a typo - the second line should have been i='3,1,1' - that's cut
and patse for you!

Regards, Keith

[Pickie]

GlenB wrote:

<snip>>

> Then I think it's more comfortable to say:
>
> <MVFIELD mvref="2,1,3">1234</MVFIELD>
>
> Instead of:
>
> <MVFIELD/>
> <MVFIELD>
> <MVFIELD>
> <MVFIELD/>
> <MVFIELD/>
> <MVFIELD>1234</MVFIELD>
> </MVFIELD>
> </MVFIELD>

<snip>

It's better to use the MV-specific index attribute - (in my opinion).
For a start, it's more concise (in most cases); also, it's easy to code
the 'unbundler" program in Pick Basic.

I think we have to remember that the target of the XML transfer is a
Pick machine/system, not a general-purpose "thang". Within the Pick
world, we understand completely the <attribute#,value#,subvalue#>
mechanism, so why shouldn't we have XML data that closely follows that?

The whole idea of XML, as I see it, is a general-purpose mechanism for
data transfer that allows for specialised "handling" of the data at the
sending and receiving ends. I think that trying to hint at the data
structure by the repeated <MVFIELD> indentation is a losing
proposition.

Regards, KeithJ

[Steve Bush]

Hi Keith,

If the aim is a Pick machine and ease of unbundling over an ASCII/XML link
then why bother with the whole pseudo XML indexing thing in the first place?
Appearances? Appearances can be objectives too.

<RECORD>
field1%FE%FEfield3.1%FD%FD%FDfield3.3%FE%FE%FE%FE%FE%FEfield9
</RECORD>

This is a very widely accepted way of transporting non-ASCII data (Pick
field marks aren't ASCII of course) over ASCII/xml links and many languages
have special functions to do this trivial conversion. Characters 0-31, "%"
and characters 128-255 are converted to "%" followed by the hex code.

This is much more compact, much faster to convert to and from and much
easier to write the conversion routine than even the multi-level indexing
XML proposed.

But if the aim really is XML then perhaps the objective should be fairly
standard XML that is readily understandable by widely available XML parsers
... which I don't think the multilevel indexing scheme is.

Personally, I would hate to see a PICK-only flavour of XML cutting us off
from the rest of the world especially in such an area as XML which is
superbly capable of making us look "normal" to the other guys.

Cheers,
Steve

> -----Original Message-----
> From: Ope...@googlegroups.com
> [mailto:Ope...@googlegroups.com] On Behalf Of Pickie
> Sent: 04 August 2006 09:29
> To: OpenQM
> Subject: Re: xmltools
>
<snip>

[GlenB]

Steve Bush wrote:
> Hi Keith,
>
> If the aim is a Pick machine and ease of unbundling over an ASCII/XML link
> then why bother with the whole pseudo XML indexing thing in the first place?
> Appearances? Appearances can be objectives too.
>
> <RECORD>
> field1%FE%FEfield3.1%FD%FD%FDfield3.3%FE%FE%FE%FE%FE%FEfield9
> </RECORD>
>
> This is a very widely accepted way of transporting non-ASCII data (Pick
> field marks aren't ASCII of course) over ASCII/xml links and many languages
> have special functions to do this trivial conversion. Characters 0-31, "%"
> and characters 128-255 are converted to "%" followed by the hex code.
>

Yes. This would work just fine. It's also quite tough to visually
debug, if there's a transactional error.

> This is much more compact, much faster to convert to and from and much
> easier to write the conversion routine than even the multi-level indexing
> XML proposed.
>

Yes and I would implement it in my own in-house tools. I wouldn't
publish it as a QM format, though. Which, afterall, is what this first
project is aimed at.

> But if the aim really is XML then perhaps the objective should be fairly
> standard XML that is readily understandable by widely available XML parsers
> ... which I don't think the multilevel indexing scheme is.
>
> Personally, I would hate to see a PICK-only flavour of XML cutting us off
> from the rest of the world especially in such an area as XML which is
> superbly capable of making us look "normal" to the other guys.
>

Right now, I don't care about "the other guys". I just want to get a
query tool working and have the facilities to do I/O using some kind of
structured XML. We can move on to complicated schemas and integration
later. Whatever the group decides on, will need to be 100% backwards
compatible in nature. If changes are made to the DTD or schema, then
they will be additions - not re-writes. Your suggestion of embedding
the entire record as PCDATA is definately doable and 100% backwards
compatible in nature. However, I _really_ don't want to debug something
that ugly.

> Cheers,
> Steve

Glen

[Pickie]

Hi Steve,

You make a good point about a Pick-only flavour of XML cutting us off
from the rest of the world, but I don't think this is actually the case
here. If the attribute, value, subvalue positions are given in the
XML, then the data is inherently "Pickish".

I don't think my example of

0001: AAA
0002:
0003: BBB1ÿBBB2.1üBBB2.2üBBB2.3ÿBBB3
0004: CCC

would be better represented as

<item a='1' v='1' s='1'>AAA</item>
<item a='3' v='1' s='1'>BBB1</item>
<item a='3' v='2' s='1'>BBB2.1</item>
<item a='3' v='2' s='2'>BBB2.2</item>
<item a='3' v='2' s='3'>BBB2.3</item>
<item a='3' v='3' s='1'>BBB3</item>
<item a='4' v='1' s='1'>CCC</item>

or as

<item> <a id='1'> <v id='1'> <s id='1'>AAA</s></v></a></item>
<item> <a id='3'> <v id='1'> <s id='1'>BBB1</s></v></a></item>
<item> <a id='3'> <v id='2'> <s id='1'>BBB2.1</s></v></a></item>
<item> <a id='3'> <v id='2'> <s id='2'>BBB2.2</s></v></a></item>
<item> <a id='3'> <v id='2'> <s id='3'>BBB2.3</s></v></a></item>
<item> <a id='3'> <v id='3'> <s id='1'>BBB3</s></v></a></item>
<item> <a id='4'> <v id='1'> <s id='1'>CCC</s></v></a></item>

even though they _look_ more like general-purpose XML.

Given the problem is to represent the data in XML without having any
dictionaries, I think there are only two real alternatives.

1 - Represent it as a string - the "raw Pick" option.
2 - Represent it as Pick-specific XML.

In the latter case, I'd argue that my mechanism (using i='3,2,3') is
concise. I did make the comment "This is about as minimalist as you
can get apart from just sending the pick item as a string". A possible
benefit is that whatever the order of the XML lines, it would rebuild
to Pick data that is identical to the original.

On the other hand, If one uses dictionaries, in my opinion the XML
output would be logically equivalent to the original Pick, but would
not necessarily re-build the same Pick data. This is the idea I was
originally pursueing, but not the one Tom was talking about.

Regards, KeithJ

[Steve Bush]

Hi KeithJ,

Regarding your point about the data being inherently pickish:

I think that the attribute, value and subvalue numbers are quite happily
understood in the XML world as indexes into repeating data just like they
are in the Pick world. <line lineno=10> ... </line>, which is common in XML
data, seems correlated to <value valueno=10> ... <value> to me so I think
Pick data is inherently unified with the rest of the world once expressed in
XML.

Regarding multi-level versus single-level indexing:

The multilevel indexed thing loses the inherent association of multivalues
which is a hallmark of both Pick and XML. For me, pick to pick raw record
transfer (or serialisation to use the jargon) using single level indexing
still seems to be more XML compatible and more compact while remaining fast
and easy to create and parse.

<a>AAA</a>
<a i=3>
<v>BBB</v>
<v>
<s>BBB2.1</s>
<s>BBB2.2</s>
<s>BBB2.3</s>
</v>
</a>
<a>CCC</a>

If fully indexed like the following, even generic XML parsers will be able
to reconstruct something akin to a Pick record with little or no extra help.

<a i=1>AAA</a>
<a i=3>
<v i=1>BBB</v>
<v i=2>
<s i=1>BBB2.1</s>
<s i=2>BBB2.2</s>
<s i=3>BBB2.3</s>
</v>
</a>
<a>CCC</a>

It occurs to me that whatever becomes the internal raw record transfer
format is likely to end up being the defacto format for communication with
external systems, hence my interest.

Just my thoughts ...

Cheers,
Steve

[GlenB]

Since nothing can be decided on, here's something to vote on. If you
don't like this, vote against it. If you vote against it, be prepared
to fully support your negative vote.

<?xml version="1.0">
<FILE filename="myfile" account="myaccount">
<ITEM id="1234">
<ATTRIBUTE index="1">
<DATA index="1,1,1">ABC1</DATA>
<DATA index="1,1,2">ABC2</DATA>
<DATA index="1,1,3">ABC3</DATA>
<DATA index="2,1,1">DEF1</DATA>
<DATA index="2,2,1">DEF2</DATA>
<DATA index="2,3,1">DEF3</DATA>
</ATTRIBUTE>
</ITEM>
</FILE>

Glen

[Simon Verona]

I don't like this too much...

I'd vote for :

<?xml version="1.0">
<FILE filename="myfile" account="myaccount">
<ITEM id="1234">
<ATTRIBUTE index="1">

<Value index="1">
<SubValue index="1">ABC1</SubValue>
<SubValue index="2">ABC2</SubValue>
<SubValue index="3">ABC3</SubValue>
</Value>
</ATTRIBUTE>
<ATTRIBUTE index="2">
<Value index="1">
<SubValue index="1">DEF1</SubValue>
</Value>
<Value index="2">
<SubValue> index="1">DEF2</SubValue>
</Value>
<Value index="3">
<SubValue> index="1">DEF3</SubValue>
</Value>
</ATTRIBUTE>
</ITEM>
</FILE>

I would have thought that this would be easier to parse using more generic
Tools, plus would also support a "Dictionary" convention by the use of a
"name" to supplement the "index" at the Attribute level. Or have I missed
the mark?

For example, I don't think it would be easy to locate all the Attributes
date extracting just Value 1 using something like Xquery in the example
you've given?

I know that this is more bandwidth intensive, but XML is generically a
bandwidth intensive medium because of the repetitiveness of descriptions.

My 2 pence worth (believe me it's worth no more than that!)

Regards
Simon

-----Original Message-----
From: Ope...@googlegroups.com [mailto:Ope...@googlegroups.com] On Behalf Of
GlenB
Sent: 07 August 2006 16:46
To: OpenQM
Subject: Re: xmltools

[Steve Bush]

I vote for this, but I think the first line should be <?xml version="1.0" ?>
instead of <?xml version="1.0">

[GlenB]

We aren't looking at dictionary typed data. This is specifically for
unknown data transmission. You limit your levels of MV representation
when you specify a mixture of XML tag nesting and hash indexing.
Having <value> and <subvalue> means I can only put 2 levels of MV in
any XML document. I'd much rather have one attribute that defines the
entire hash index as a string. I can be conviced otherwise, but I
haven't heard any reasonable counter arguments.

> For example, I don't think it would be easy to locate all the Attributes
> date extracting just Value 1 using something like Xquery in the example
> you've given?
>

XQuery is best for elements that have meaning. In this specific
scenario the element names that contain the data don't mean anything.
You can still use XQuery in both of our examples, but your example
limits hashing depth.

> I know that this is more bandwidth intensive, but XML is generically a
> bandwidth intensive medium because of the repetitiveness of descriptions.
>

It doesn't have to be. I could tar an entire Pick record and then
base64 encode it. The resulting XML would be potentially 200%+ smaller
than verbose text XML.

> My 2 pence worth (believe me it's worth no more than that!)
>
> Regards
> Simon

Glen

[GlenB]

Steve Bush wrote:
> I vote for this, but I think the first line should be <?xml version="1.0" ?>
> instead of <?xml version="1.0">
>

Yes. There should be an ending question mark.

Glen

[patric...@yahoo.com]

It would seem what you are really trying to do is represent a
3-dimension array as xml. If we keep that focus then the below design
makes sense. As soon as we start adding dictionary items back in it
gets confusing.

[tomh]

GlenB wrote:

> Glen's stuff.....

I like this best.

However, in this particular experiment... I came from a Query processor
'list' statement because other posts (from way back) sounded like this
was the method of choice.
But we wont know which [mv] attribute mark count it is, because qproc
will be doing all the work. In addition, the only way this experiment
even knows there are values and subvalues is because they are 'stumbled
upon' when processing the qproc output. So...

> <DATA index="1,1,1">ABC1</DATA>

should be...
> <DATA index="1,1">ABC1</DATA>
and only when there are values and/or subvalues.

Oh, I don't like putting data in xml.attribute spots.
But that is for another thread.

> <FILE filename="myfile" account="myaccount">
> <ITEM id="1234">

- (yet another) Tom H (not 'D', nor 'K', or....)

[GlenB]

tomh wrote:
>
> I like this best.
>
> However, in this particular experiment... I came from a Query processor
> 'list' statement because other posts (from way back) sounded like this
> was the method of choice.
> But we wont know which [mv] attribute mark count it is, because qproc
> will be doing all the work. In addition, the only way this experiment
> even knows there are values and subvalues is because they are 'stumbled
> upon' when processing the qproc output. So...

Ignore what QPROC is doing right now. We need to determine what
information we need, in order to build proper XML. Martin originally
suggested that we determine what we need from within QPROC. I don't
know how far the rewrite has gotten, but I'm sure he will either make
the XML hook fairly flexible or ask the group for guidance regarding
it. I'd personally suggest a class, since it would be easier to
maintain and enhance. If we all can't decide on a standard
non-dictionary XML format then Martin & Doug may need to make the
decision from the examples in this thread. The dictionary based stuff
is easier to nail down, since there is some structure there already.

>
> > <DATA index="1,1,1">ABC1</DATA>
> should be...
> > <DATA index="1,1">ABC1</DATA>
> and only when there are values and/or subvalues.
>
> Oh, I don't like putting data in xml.attribute spots.
> But that is for another thread.
>

Do you mean <xml> attributes or <element attribute="value">?

> > <FILE filename="myfile" account="myaccount">
> > <ITEM id="1234">
>
> - (yet another) Tom H (not 'D', nor 'K', or....)

Glen

[tomh]

GlenB wrote:
> tomh wrote:
> >
> > I like this best.
> >
>

> Ignore what QPROC is doing right now. We need to determine what
> information we need, in order to build proper XML. Martin originally
> suggested that we determine what we need from within QPROC. I don't
> know how far the rewrite has gotten, but I'm sure he will either make
> the XML hook fairly flexible or ask the group for guidance regarding
> it. I'd personally suggest a class, since it would be easier to
> maintain and enhance. If we all can't decide on a standard
> non-dictionary XML format then Martin & Doug may need to make the
> decision from the examples in this thread. The dictionary based stuff
> is easier to nail down, since there is some structure there already.

>glen's stuff above

ok, you are right. Then my vote still stands in the affirmative for
your latest example

>
>
> Do you mean <xml> attributes or <element attribute="value">?
>

I mean data. I will probably get flamed for this but...

If the [mv] color = red then
<color>red</color>
not
<sometag color="red">

- tom h

[Simon Verona]

How about just using the same Tag <VALUE> but with nesting:

Eg:
<?xml version="1.0"?>

<FILE filename="myfile" account="myaccount">
<ITEM id="1234">

<VALUE index="1">
<VALUE index="1">
<VALUE index="1">ABC1</VALUE>
<VALUE index="2">ABC2</VALUE>
<VALUE index="3">ABC3</VALUE>
</VALUE>
</VALUE>
<VALUE index="2">
<VALUE index="1">
<VALUE index="1">DEF1</VALUE>
</VALUE>
<VALUE index="2">
<VALUE> index="1">DEF2</VALUE>
</VALUE>
<VALUE index="3">
<VALUE> index="1">DEF3</VALUE>
</VALUE>
</VALUE>
</ITEM>
</FILE>

Though having said the above, I'm not convinced I know what we are trying to
achieve.... I thought the idea was to generically represent MultiValue
data - ie data that OpenQM could easily import and export. Surely if you
use more than 3 levels, you are building a schema for data that OpenQM can't
itself handle??

I'm not convinced by the nesting of the same element-tag, but I think it is
useable still and allows unlimited levels.

I'm late on in this thread, but is the idea that OpenQM could convert to and
from this XML format "on-the-fly" ? I apologise if this has been posted
before, but I have read the thread now and I'm still slightly confused as to
the original aim....

Regards
Simon

-----Original Message-----
From: Ope...@googlegroups.com [mailto:Ope...@googlegroups.com] On Behalf Of
GlenB
Sent: 07 August 2006 17:59
To: OpenQM
Subject: Re: xmltools

[GlenB]

Nesting requires a lot more overhead to achieve the same result as
using an element attribute to specify a single MV attribute data's
index. Both designs are unlimited in terms of dimensional references.
Nesting makes troubleshooting and internal coding more difficult, but
it allows outside developers to dummy-down their parsing tools. My
opinion, overall, is that the correlation between index attribute and
BASIC array reference is 1:1 so there will be no confusion for output
or input documents. Anyone with MV knowledge will be able to quickly
refer to the XML document and instantly know where that data should be
in an MV file. Those who don't have MV knowledge, but understand we are
using lists of lists, should easily understand that an "index"
reference of 1,2,4 will mean 3-dimensions with indexes of 1, 2, and 4
respectively.

<DATA index="1,2,4">ABC</DATA>

RECORD<1,2,4> = "ABC"

> I'm not convinced by the nesting of the same element-tag, but I think it is
> useable still and allows unlimited levels.
>
> I'm late on in this thread, but is the idea that OpenQM could convert to and
> from this XML format "on-the-fly" ? I apologise if this has been posted
> before, but I have read the thread now and I'm still slightly confused as to
> the original aim....
>

QPROC will eventually have a TOXML keyword. That will fire off the XML
generator, based on the QPROC sentence you issue.

LIST CUSTOMER BY NAME NAME ADDRESS LASTPAYMENT TOXML

> Regards
> Simon
>

Glen

[Pickie]

Hi Tom,

Regarding your comment about data in XML attribute spots.

It seemed to make sense to me to use XML elements
<color>red</color>
for multivalued data, and to use XML attributes
<sometag color="red">
for single valued data.

So a simplified invoice might look like

<invoice customer.id="123456">
<product.code>ABC</product.code><quantity>5</quantity><price>12.93</price>
<product.code>DEF</product.code><quantity>10</quantity><price>200.00</price>
<product.code>GHI</product.code><quantity>20</quantity><price>44.10</price>
</invoice>

However, I do seem to have given contrary examples on the odd occasion
- typical Pick person!

Regards, KeithJ