Exporting authority records, etc.

[Krista Ferrante]

Hi,

The group I work with at Tufts University would like to use ICA-AtoM
and we have taken a serious look at the demo. I noticed that there is
an export mechanism for archival descriptions. However, we need to be
able to export the authority records and taxonomies even if it were
only as text delimited files. Without this export function, the
authority records and taxonomies would be trapped in ICA-AtoM. Being
able to export all aspects of the data input is essential to our
operations. Is there a way to export the authority records that I'm
not seeing in the demo or documentation? If not, are there plans to
add this functionality?

Thank you,
Krista Ferrante
Project Archivist
Tufts University Digital Collections and Archives

[Evelyn McLellan]

Hi Krista,

Hi Krista,

The EAD xml export feature in ICA-AtoM exports the authority records
and taxonomy terms along with the archival descriptions. However, we
also plan to add EAC-CPF XML import/export support in version 1.1,
scheduled for release next April.This will allow for the exchange of
ICA-ISAAR(CPF) authority files via EAC-CPF XML import/export and then,
hopefully, via OAI harvesting as well.

Evelyn McLellan
ICA-AtoM Community Manager

[PATRICIA VIVECA ORTIZ CASTRO]

Evelyn: In 1.0.7 how we can export all the database? Thanks en advance, Patricia

-----Mensaje original-----
De: Evelyn McLellan [mailto:epmcl...@gmail.com]
Enviado el: viernes, 20 de noviembre de 2009 16:21
Para: ICA-AtoM Users
Asunto: [ica-atom-users] Re: Exporting authority records, etc.

--

You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To post to this group, send email to ica-ato...@googlegroups.com.
To unsubscribe from this group, send email to ica-atom-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ica-atom-users?hl=.

[Krista Ferrante]

Thank you for your response.

Are there any plans to allow for a simple export of the complete data set in something like a delimited file? EAD won't serve our purposes because it doesn't export all of the authority record fields and we need to have it all.   While having the ability to export to EAC would be nice, we are often required to put our descriptive information into other XML schemas for use in our digital repository. The schemas we currently use are TEI, FOXML, and others.  We know it is unrealistic to expect the ICA-AtoM working group make a special case for these. However, we would need to have the flexibility to use our descriptions in these other types of XML and a delimited file export of the full data set, in all of the modules (description, authority, archival institution, and taxonomy) would do that.  

Ultimately, this would free the working group from having to build an export mechanism for every new encoding standard out there by giving the user all of their data to parse themselves. 

Best,
Krista Ferrante

[Evelyn McLellan]

In general we support standardized export formats because everyone can
use them. Exporting all fields in a delimited format and having
different users parse them in their own way would lead to numerous
silos of non-interchangeable data, which is what the development of
international standards is designed to avoid. That being said,
however, it would help to know what you would want the structure of a
delimited format to be. Maybe we can assist your developers to write a
program that would help you to import the data into repositories that
use TEI and FOXML (and whatever else you're using).

Evelyn

On Nov 20, 12:34 pm, Krista Ferrante <ksferra...@gmail.com> wrote:
> Thank you for your response.
>
> Are there any plans to allow for a simple export of the complete data set in
> something like a delimited file? EAD won't serve our purposes because it
> doesn't export all of the authority record fields and we need to have it
> all.   While having the ability to export to EAC would be nice, we are often
> required to put our descriptive information into other XML schemas for use
> in our digital repository. The schemas we currently use are TEI, FOXML, and
> others.  We know it is unrealistic to expect the ICA-AtoM working group make
> a special case for these. However, we would need to have the flexibility to
> use our descriptions in these other types of XML and a delimited file export
> of the full data set, in all of the modules (description, authority,
> archival institution, and taxonomy) would do that.
>
> Ultimately, this would free the working group from having to build an export
> mechanism for every new encoding standard out there by giving the user all
> of their data to parse themselves.
>
> Best,
> Krista Ferrante
>

> > ica-atom-user...@googlegroups.com<ica-atom-users%2Bunsu...@googlegroups.com>
> > .

[Evelyn McLellan]

Hi Patricia,

Do you want to migrate all your data to ICA-AtoM 1.0.8? Or are you
looking to export the data to another system?

Evelyn

On Nov 20, 12:12 pm, "PATRICIA VIVECA ORTIZ CASTRO"

<port...@uahurtado.cl> wrote:
> Evelyn: In 1.0.7 how we can export all the database? Thanks en advance, Patricia
>
> -----Mensaje original-----

> De: Evelyn McLellan [mailto:epmclel...@gmail.com]

[David Juhasz]

Hi Krista,

You can export a generic delimited text file or XML file from MySQL
using phpMyAdmin or a similar tool [1]. This is not ideal for ICA-
AtoM data because relational data is tricky to represent in a flat
file format and some data (e.g. repository, publication status) is
inherited and not stored for each description, but it is one option.

[1] => http://www.phpmyadmin.net/home_page/index.php

David Juhasz,
Software Engineer, Artefactual Systems Inc.
http://www.artefactual.com | P: 604.527.2056 | F: 604.521.2059

[deborah kaplan]

Evelyn,

I'm hoping I can add some more clarity to our needs as a developer for
the team.

We aren't particularly picky about what format the data are exported
in, as long as it is parsable text. XML, comma delimited, etc.: these
are all perfectly adequate for our needs. Once we get parsable text,
we can put it into any of our appropriate formats (e.g. import to our
collection management database, import to our digital library).

The reality is that we are going to have numerous silos. We have other
tools we need to work with: our digital library, our collection
management database, our university databases such as PeopleSoft. The
development of ICA-ATOM is not going to make the need for those tools
go away. Our goal is to make sure that the data elements themselves
are interchangeable within and among those individual software silos,
however. As far as we can see, the ICA-ATOM is perfect for allowing us
to use international standards to have consistent data elements across
all of our description. However, we can't use the tool if it doesn't
allow us to export that data for import into our institutional silos.
In other words, disallowing us from exporting the data prevents us
from using the international standards and forces us to manage our
data elements in the silos instead of mastering those data elements in
the standardized tool.

Additionally, while we understand your concern for designing for
standardization, we have practical requirements having to do with the
ability and persistence of data. Part of those practical requirements
is that we simply can't put data into any tool which ties us to that
tool. From a preservation and sustainability standpoint, that's simply
a poor idea. We can't put data into a tool that doesn't allow a full
export of every piece of information we store in that tool.

Therefore, our concern is primarily that we can get it every single
piece of data that we put into ICA-ATOM. If allowing that kind of
export is not going to be something ICA-ATOM allows, then it is not
something which suits our needs. This would be unfortunate for us, but
I understand that our use case might not be what you are coding for.

-Deborah
--
Deborah Kaplan
Digital Resources Archivist
Digital Collections and Archives
Tufts University

[Evelyn McLellan]

Hi Deborah,

We do not foresee adding some kind of end-user export functionality
that exports all fields in a generic delimited format because that
would overstress our resources, and because we feel that most ICA-AtoM
users would not find it useful. Unfortunately, the best we can offer
is David's suggestion of exporting a generic delimited text file or
XML file from MySQL using phpMyAdmin or a similar tool. Since MySQL is
open-source, there are always tools to get the data out. However the
data structure in ICA-AtoM is very complex and any tool you choose is
not going to be straightforward. If you do try any of them at some
point, we would certainly appreciate hearing back from you about your
experiences.

Evelyn

[Peter Van Garderen]

Great discussion. Thanks for the detailed and pragmatic description of your
technical requirements Deborah. Of course an application like ICA-AtoM is not
going to be used in isolation and the data must be portable.

As Evelyn mentioned earlier, our design goal is to use applicable professional
standards to eventually export all our data entities. We don't want to introduce
a new defacto, system-specific ICA-AtoM export standard when overlapping and
extending existing standards should do the trick (although that will take a few
more releases to accomplish fully).

That said, there will always likely be some system-specific metadata (i.e.
menus, settings, etc) which might never be properly covered by a metadata
standard. However, as David points out, ICA-AtoM is a fully open-source
application and you will always have the ability to access the data directly in
the database and use a tool like phpMyAdmin to dump the entire database in SQL,
CSV, or XML format. It sounds to me that is what you are asking for and that you
have the capacity to analyze and parse the dumped data for your own custom
imports into other systems. Please let us know if that will work for your needs.
Otherwise I hope you'll keep brainstorming alternatives with us. These types of
discussions help to inform our ongoing design and requirements planning.

Cheers,

--peter

Peter Van Garderen
ICA-AtoM Software Release Manager

P.S. I should note also that over the coming year we hope to investigate the use
of RDF triple-stores as a next-generation (i.e. 2.0) data store for ICA-AtoM.
That would open the re-use of all ICA-AtoM data to SPARQL queries and the
ability to repurpose the application data in all sorts of ways.

> --
>
> You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
> To post to this group, send email to ica-ato...@googlegroups.com.

> To unsubscribe from this group, send email to ica-atom-user...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/ica-atom-users?hl=en.
>
>
>