Integrating pXRF data into a collections Database

[Nina]

Dear All,

We use KE EMu at the Penn Museum for our collections database. Now that we have a portable XRF unit, we are trying to develop standards for tracking the data collected and ensuring that the results remain associated with the object information. To do this, we'd like to attach the actual spectra, not just a summary report. Given that the information will be made available to the museum as a whole, and potentially to an even wider audience, what file format do people find most useful for sharing data. We currently have .pdz, .csv, .txt and .xls as options. Considerations include maintaining the native data, using file formats that are accessible without proprietary software, and general ease of use.

Has anyone else dealt with this? Would you mind sharing your thoughts or experience for managing your pXRF data?

Thanks in advance.

Best,
Nina

Nina Owczarek ~ Assistant Conservator ~ Penn Museum ~ 215.898.4018

[Arlen Heginbotham]

Hi Nina,

We store all our spectra for the long term as .csv files.  These are readable by many kinds of software and will be far into the future.  The .txt and .xls files should contain essentially the same string of numbers and so should be fine too, but we've settled on .csvs.  

Best,

Arlen


Arlen Heginbotham
Associate Conservator
Decorative Arts and Sculpture Conservation Department
J. Paul Getty Museum
1200 Getty Center Drive, Suite 1000
Los Angeles, CA 90049-1687
310.440.7178
fax 310.440.7745


>>>

From: Nina <nina...@gmail.com> To: <px...@googlegroups.com> Date: 5/1/2012 09:05 AM Subject: [pxrf] Integrating pXRF data into a collections Database

-- This message comes from the pXRF for Cultural Heritage group. Access all group activities through its website. BOOKMARK: https://sites.google.com/site/pxrfgroup/

[Nina Owczarek]

Thanks so much Arlen for letting me know how you are storing your files. .csv is certainly attractive, but we haven't written off .xls or .txt yet. It looks like .pdz might get pushed off the list first though.

We are still soliciting feedback on the subject, so please feel free to weigh in on this if you have thoughts to share too.

Best,
Nina

[Bruce Kaiser]

Nina

Note pdz are the RAW data and contain vital details 

CSV and TXT contain less info and are not raw and easily altered and can be created any time using the S1PXRF software

I would strongly suggest archiving the pdz files.

:)

Thanks

Dr Bruce Kaiser

509 528 7608

[Arlen Heginbotham]

Agreed, we definitely archive both.  

Arlen

>>>

From: Bruce Kaiser <pant...@aol.com> To: "px...@googlegroups.com" <px...@googlegroups.com> CC: "px...@googlegroups.com" <px...@googlegroups.com> Date: 5/2/2012 05:29 PM Subject: Re: [pxrf] Integrating pXRF data into a collections Database

[gus]

Hi Nina:

If it of any interest, we have a Bruker hand held XRF analyser in the
Art Conservation Program at Queen's University. I save the spectra
as .pdz and .csv files. I then export the .csv files the program
GRAMS, where they become .spc files. I find the GRAMS program very
useful. I can make montages of spectra anotate them and send the
results to students and colleagues.

If you would like to know more about GRAMS please contact me.

Gus.

[Dennis Piechota]

Gus,

If you wouldn't mind I think a lot of us would be interested in hearing more on how the widely used ThermoScientific GRAMS suite of software works with non-Niton generated files. Your experience with the Innox-XT and the Tracer analyzers makes you a great reviewer!

Nina,

I would think one consideration is how essential the general KE EMu database (which I have no experience with) will be to accessing and maintaining the chain of evidence stream. Analyses could be recorded as a one-off justification for a particular conservation treatment decision on a single object or the record could include one reading of a systematic series of analyses for research purposes unrelated to conservation. While both purposes would benefit from referencing the raw captures, I think the immediate need to critically examine capture methods would vary in these two contexts.

I know in my case if as a conservator I say "XRF analysis showed the presence of x, therefore we treated the object in this way", the focus moves off the analysis and onto the treatment method and outcome for that artifact. If the analyses are meant to characterize and interpret a group or class of artifacts then more critical examination of the analytical methods is desired.

If I could hazard an analogy sometimes we just want to know the temperature; other times we need to know the make and model of the thermometer. A central artifact db might contain the visualized analytical results only with a link referring to a separate db of downloadable source documentsin the manufacturers' original formats (.pdz, .ndt, etc.).

Dennis

Dennis Piechota

Fiske Center for Archaeological Research
UMass Boston
Office: 617-287-6829

XRF Group Admin

pXRF for Cultural Heritage

RTI Group Admin

RTI_help

"The best way to learn is to teach." Frank Oppenheimer

[Nina Owczarek]

Dear Bruce, Arlen, Gus and Dennis,

Thanks for your replies!

Bruce: I would *love* it if you had the time to spell out some of the details that the .pdz files contain that are lost in the .csv and .txt formats. While it's true that they are easy to convert to the more accessible formats, I'm thinking about what happens to the data down the road, or if a user of the data is not necessarily a user of the actual XRF equipment and might not have the software.

Gus: I ditto what Dennis said.

Dennis: EMu allows for flexibility to track the information in various ways. The system is set up in modules that can be interconnected for any given set of circumstances. We also have places to enter analytical information to associate it with the conservation record or the catalog record depending on they nature of the information. What you've described is exactly why this is so useful. We can also set various levels of permission to access the information in the database.

Here's how it works: Conservation information is entered in the conservation module, and then attached to the catalog record. Each image or document on a given object is entered separately in the multimedia module and then attached to the catalog record or the conservation record. This allows us to control where the information is accessed. For example, we attach after treatment images to both records, but before treatment images only to the conservation record. We use this system to attach bibliographic, exhibit, loan, donor, location information and more, so everything is interconnected. We can do the same with our XRF data. If we've collected spectra on restoration materials, or specifically to inform treatment, we can attach it to the conservation record only. We can also create a scientific event for a project and attach it to several catalog records, but attach only the relevant spectra for any one object.

In general, because one needs to understand the XRF data in order to use it in a meaningful and responsible way, we are not planning on making the raw data generally available to the public at this point. We are thinking of including more generalized summary data in the catalog record so people can know what data already exists and can know a little more about our collections. If a serious scholar wants to access the raw data, we can provide access on an individual basis. 

So... to get to the real point of all that: EMu will be the central storage database for the spectra and XRF reports. This will also be where we will store articles and bibliographic references that use the data from our artifacts. We'd really like to set a standard on our XRF data management using EMu for storage.

All: This discussion has been really helpful for me so far. I really appreciate the feedback!

Best,
Nina

[Aaron Shugar]

Just on a side note,  The Tracer (maybe all manufacturers) does not record the filter that you are potentially using (yes the newer software does have some pre-set ideas of filter use, but their use is not required, and you can make your own mind up about which one to use).

I recommend recording your run setting in your file name, something like this

NAME 40kV 20uA Red Filter NV (NV=no vacuum V=vacuum).

In addition, I always back up my original proprietary file with a .csv which is a universally recognized file format.

Good luck all

[gus]

Dennis:

The program GRAMS allows the capture, storage, processing and
reporting of data from almost any make and model of
instrument or system for spectroscopy or chromatography. I have used
GRAMS for many years to analyse and process infrared and Raman
spectra.

In the Art Conservation Program at Queen's, I also use GRAMS for
storing and handling XRF spectra. I export the XRF spectra as .csv
files to a desk top computer and save them. To modify the .csv file
for importation into GRAMS, I have to remove the header information to
get XY data only. The X column (the energies) have to be multiplied
by .02 to obtain keV. I have no idea why this has to be done, but when
I first imported the spectra into GRAMS, I noticed that the energies
were 50 times too large. Once the spectra are in .csv format they can
be processed using the GRAMS software. One of our students wrote a
macro to make the conversion simple.

The only problem with GRAMS is the cost of the latest release of the
software.

> pXRF for Cultural Heritage <http://groups.google.com/group/pxrf>
> RTI Group Admin
> RTI_help <http://groups.google.com/group/rti_help/>

>
> "The best way to learn is to teach." Frank Oppenheimer
>
>
>
> On Wed, May 2, 2012 at 9:19 PM, gus <shurv...@queensu.ca> wrote:
> > Hi Nina:
>
> > If it of any interest, we have a Bruker hand held XRF analyser in the
> > Art Conservation Program at Queen's University. I save the spectra
> > as .pdz and .csv files. I then export the .csv files the program
> > GRAMS, where they become .spc files. I find the GRAMS program very
> > useful. I can make montages of spectra anotate them and send the
> > results to students and colleagues.
>
> > If you would like to know more about GRAMS please contact me.
>
> > Gus.
>
> > --
> > This message comes from the pXRF for Cultural Heritage group.
> > Access all group activities through its website.

> > BOOKMARK:  https://sites.google.com/site/pxrfgroup/- Hide quoted text -
>
> - Show quoted text -

[Dennis Piechota]

Has anyone investigated the related topic of how to implement the US National Science Foundation data sharing requirement for grant applicants? It's my understanding that, though it is a secondary consideration, an applicant can get on or off the bubble (the discretionary funding group) depending on the presence or absence of a well thought out data /artifact access plan. One could do this a number of ways using the museum/institution's internal database or handing off all info to be curated by a third party archiving agency. Either way reference to the NSF-funded data would need to be included in the internal db.

Below is the 2011 section describing this NSF requirement. I've highlighted the section requiring access to primary data ( eg XRF readings) and physical collections

NSF Policy on Dissemination and Sharing of Research Results 

(Excerpt from NSF Award and Administration Guide Chapter VI.D.) 

4.  Dissemination and Sharing of Research Results 

a. Investigators are expected to promptly prepare and submit for publication, with 

authorship that accurately reflects the contributions of those involved, all significant 

findings from work conducted under NSF grants. Grantees are expected to permit and 

encourage such publication by those actually performing that work, unless a grantee 

intends to publish or disseminate such findings itself. 

b. Investigators are expected to share with other researchers, at no more than 

incremental cost and within a reasonable time, the primary data, samples, physical 

collections and other supporting materials created or gathered in the course of work 

under NSF grants. Grantees are expected to encourage and facilitate such sharing. 

Privileged or confidential information should be released only in a form that protects the 

privacy of individuals and subjects involved. General adjustments and, where essential, 

exceptions to this sharing expectation may be specified by the funding NSF Program or 

Division/Office for a particular field or discipline to safeguard the rights of individuals and 

subjects, the validity of results, or the integrity of collections or to accommodate the 

legitimate interest of investigators. A grantee or investigator also may request a 

particular adjustment or exception from the cognizant NSF Program Officer. 

c. Investigators and grantees are encouraged to share software and inventions created 

under the grant or otherwise make them or their products widely available and usable. 

d. NSF normally allows grantees to retain principal legal rights to intellectual property 

developed under NSF grants to provide incentives for development and dissemination 

of inventions, software and publications that can enhance their usefulness, accessibility 

and upkeep. Such incentives do not, however, reduce the responsibility that 

investigators and organizations have as members of the scientific and engineering 

community, to make results, data and collections available to other researchers. 

Denniis

Dennis Piechota

Archaeological conservator and micromorphologist

Fiske Center for Archaeological Research
UMass Boston
Office: 617-287-6829

XRF Group Admin

pXRF for Cultural Heritage

RTI Group Admin

RTI_help