New topic: Should Jangle connectors use Badgerfish or other conventions?

[Ross Singer]

One of the things I just ran into while writing the article is that
the Jangle connectors don't have a very robust system for dealing with
Atom extensions. They get by with OpenSearch, but that's only because
OpenSearch elements don't use attributes (or, in the case of the
description document, it's known what those attributes would be).

So should the connectors use a formalized syntax for sending their
data? It seems like it would be _much_ easier to serialize a JSON
object using something like Badgerfish (http://badgerfish.ning.com/)
than something that is currently rather arbitrary and ad-hoc.

Does anybody know of other conventions for dealing with XML as hashes?

-Ross.

[Mike Rylander]

There's the XML::Simple format:

http://search.cpan.org/~grantm/XML-Simple-2.18/lib/XML/Simple.pm

Some main differences I see:

* the default textContent "name" is not "$" (easily changed at xml parse time)
* XML::Simple throws away the element name of the root node (IOW,
there's no wrapping, one-key hash that just contains the root node
element name as the lone key)
* attributes are prefixed with @ in BadgerFish

Also, the BadgerFish php and ruby modules may not have robust array
handling for normalizing a single element that /can/ be repeated even
when they're not -- very handy for processing well-defined structures
with N>=1 , but not a deal breaker by any means.

--
Mike Rylander
| VP, Research and Design
| Equinox Software, Inc. / The Evergreen Experts
| phone: 1-877-OPEN-ILS (673-6457)
| email: mi...@esilibrary.com
| web: http://www.esilibrary.com

[townxelliot]

Hi Ross,

On Wed, 2008-07-16 at 12:03 -0400, Ross Singer wrote:
> One of the things I just ran into while writing the article is that
> the Jangle connectors don't have a very robust system for dealing with
> Atom extensions. They get by with OpenSearch, but that's only because
> OpenSearch elements don't use attributes (or, in the case of the
> description document, it's known what those attributes would be).
>
> So should the connectors use a formalized syntax for sending their
> data? It seems like it would be _much_ easier to serialize a JSON
> object using something like Badgerfish (http://badgerfish.ning.com/)
> than something that is currently rather arbitrary and ad-hoc.
>

When we first talked about Jangle, my vision of it was that connectors
would send object data, serialised to JSON. Then the Jangle bit in front
would deal with the difficulties of serialising to XML etc.. Because you
just have to deal with objects in the connector, it should be easier to
develop an object model which connectors adhere to, rather than an XML
schema which you embed in or convert to JSON (which I think is how
things have gone so far).

Then I'd use some kind of automated serialisation in the connector to
convert those objects into JSON: there are libraries for practically
every programming language. Connectors don't need to know anything about
Atom or DLF schemas or even XML in that case: they just have to build
object models. The Jangle layer takes objects in JSON and does the
translation to XML (rather than the connector working with XML and
translating it to JSON).

Elliot

> Does anybody know of other conventions for dealing with XML as hashes?
>
> -Ross.
>
> >

--
Elliot Smith
Developer
Talis Information Ltd.
6190 Knights Court, Solihull Parkway
Birmingham Business Park, B37 7YB
+44 (0)870 400 5000
elliot...@talis.com

[Ross Singer]

Yeah, I'm not married to Badgerfish, but offer it as an example.

I note that they liken it to XML::Simple, and would be perfectly to
use its notation instead (if people have strong preferences).

It would be nice to have something quasi-standard.

I have actually run into the issue you mention with N>=1 elements when
using the CobraVsMongoose Ruby implementation of Badgerfish. Every
element really needed to be checked to see if it was an array or a
hash. I found this exceptionally annoying (although dealing with it
in Atom will be far less painful than it was with EAD). IIRC, there
may have been some issues with preserving order, as well (EAD nests
elements within text nodes, for example), but that may have been with
standard hashes instead of JSON.

How does XML::Simple deal with the repeatable fields issue?

-Ross.

[Ross Singer]

Elliot, I agree with what you're saying, in principle. And I think
the "core" aspects of Jangle (i.e. the metadata needed to create a
vanilla Atom feed) can be serialized in any capacity as long as they
follow a standard convention (which is, in fact, what they're doing
now).

I'm not sure Jangle can or should have to be aware of every extension
a connector may wish to use, though. If a standardized notation of
how the "extra" elements should be serialized can be agreed upon, the
Jangle core can just add them without not needing to know what they
are or what they do.

So, yes, perhaps I am laying this out as a way to extend the API, if you will.

I agree, the core JSON responses should not have to know whether they
will be attributes or values or what their end result tag name will
necessarily be.

-Ross.

[Richard Wallis]

On the subject of getting JSON out of XML, I have been, for another
project, banging my head against a brick wall for the last couple of
days trying to get a JSON version of the XML output from the Amazon
Associate Web Services.

Which looks like this:
http://xml-uk.amznxslt.com/onca/xml?Service=AWSECommerceService&Subscrip
tionId=0HYDKXSNE8KQZCFNGQR2&AssociateTag=taliscom-21&Operation=ItemSearc
h&SearchIndex=Books&ResponseGroup=Medium&Power=isbn%3A0007107927


I have finally achieved this using an xslt transform by fixing a defect
in the xml2json.xslt published by Doeke Zanstra.
http://talis-rjw.s3.amazonaws.com/PrismDev/rjw-xml2json.xsl

The result is this, using Amazon's built in xslt transform service:
http://xml-uk.amznxslt.com/onca/xml?Service=AWSECommerceService&Subscrip
tionId=0HYDKXSNE8KQZCFNGQR2&AssociateTag=taliscom-21&Operation=ItemSearc
h&SearchIndex=Books&ResponseGroup=Medium&Power=isbn%3A0007107927&Style=h
ttp://talis-rjw.s3.amazonaws.com/PrismDev/rjw-xml2json.xsl

As most environments have xslt transform capability available, this
might be an easy way to get from XML to JSON in a connector.

~Richard.

[Mike Rylander]

On Wed, Jul 16, 2008 at 12:33 PM, Ross Singer <rossf...@gmail.com> wrote:
>
> Yeah, I'm not married to Badgerfish, but offer it as an example.
>
> I note that they liken it to XML::Simple, and would be perfectly to
> use its notation instead (if people have strong preferences).
>
> It would be nice to have something quasi-standard.
>
> I have actually run into the issue you mention with N>=1 elements when
> using the CobraVsMongoose Ruby implementation of Badgerfish. Every
> element really needed to be checked to see if it was an array or a
> hash. I found this exceptionally annoying (although dealing with it
> in Atom will be far less painful than it was with EAD). IIRC, there
> may have been some issues with preserving order, as well (EAD nests
> elements within text nodes, for example), but that may have been with
> standard hashes instead of JSON.
>
> How does XML::Simple deal with the repeatable fields issue?

For XML::Simple proper, you tell it what elements should always be
treated as arrays, and it does that. Obviously, this requires a prior
knowledge of the data structure, but ... you kinda need that anyway.
:)