Transforming the EAD with Saxon has failed

[Bruno Freitas]

Hi AtoM Users,

I'm trying generate the finding aid in AtoM 2.2.0, but it return me the error below.

Transforming the EAD with Saxon has failed

I read the Documentation in AtoM website (https://www.accesstomemory.org/en/docs/2.2/user-manual/reports-printing/print-finding-aid/#fa-trouble-ead-export-failed) and the chapter FINDING AID GENERATION ERROR; THE JOBS PAGE SAYS THAT “EXPORTING EAD HAS FAILED”, but I didn't find the problem.

I checked the data in the fields and I didn't see problem with html or characters wrong. It's very strange, because the AtoM export to EAD XML.

Could someone help me about this?

Thanks,

Bruno Freitas

[Dan Gillean]

Hi Bruno,

One of the first things we recommend is making sure that gearman is properly installed and configured. Instructions can be found here:

• https://www.accesstomemory.org/docs/2.2/admin-manual/installation/asynchronous-jobs/

You can also try checking on the status of the AtoM worker:

sudo start atom-worker   # Starts the worker
sudo stop atom-worker    # Stops the worker
sudo restart atom-worker # Restarts the workers
sudo status atom-worker  # Obtain current running status

I might suggest trying to restart the atom-worker, and see if that helps?

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at http://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/eab48f46-40cb-48ec-9205-6be79ecf6d8f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bruno Freitas]

Hi Dan,

I checked the atom-worker and it is runnig.

Anyway, I restarted the service, but still there is the error.

I thought that could been some problem in language, because we are using pt_br. However, I changed to english and no success too.

I have a question about start job. In web application when I run the generator the start hour that I see on the list jobs isn't the same of the my webserver. I attached the image for you.

Thank you.

best regards,

Bruno Freitas

[Mike G]

Hi Bruno,

Can you confirm you have Java installed? I get something like this:

$ java -version

java version "1.8.0_60"

Java(TM) SE Runtime Environment (build 1.8.0_60-b27)

Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

Also ensure that saxon9he.jar is located in your atom directory, for example: /usr/share/nginx/atom/lib/task/pdf/saxon9he.jar

As for your time differences, is it possible your mysql server is on a different machine and has a different time? What happens if you go into mysql and use the following query: SELECT NOW();

Does it print the right time then? It's possible you found a bug.

[Bruno Freitas]

Hi Mike,

Maybe there is problem of the permissions on saxon or another path to save. What do you think about?

How should the permissions be?

You can see my configuration below.

File saxon9he.jar

$ java -version

java version "1.8.0_60"

Java(TM) SE Runtime Environment (build 1.8.0_60-b27)

Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

/usr/share/nginx/atom/lib/task/pdf$ ls

ead-pdf-full-details.xsl  ead-pdf-inventory-summary.xsl  helper-functions.xsl  saxon9he.jar

Permisssions

-rw-rw-r--  1 www-data www-data 10374719 Jul  8 15:20 saxon9he.jar

Times 

• System

$ date

Fri Oct  2 10:09:35 BRT 2015

• Mysql

mysql> select now();

+---------------------+

| now()               |

+---------------------+

| 2015-10-02 10:09:15 |

I don't know where the job get the hour. 

My application version is 2.2.0 -123.

Do you get the file generated in your AtoM? And What display standard do you use? RAD?

[Mike G]

Hi Bruno,

That time difference thing is very perplexing indeed. We aren't using any custom code for that, and it's just the propel ORM that fills in the creation times. To my knowledge we've never seen that bug before, and unfortunately it sounds like something that wouldn't be easy to fix without a developer looking at your particular installation.

I think the permissions look okay for saxon9he.jar--I think all you need is read permissions. What user is your AtoM worker running as? Try: ps aux | grep 'jobs:worker'

My understanding was the finding aid PDFs were designed for RAD, so you may not get great results using ISAD. If the documentation isn't clear on that let me know, and I'll let our docs people know to add a note about that in the next version of AtoM!

Let's see what happens if you manually try to transform using saxon from the command line. I've attached a file ("test1.xml"), can you download that file into your AtoM directory and try this?

java -jar lib/task/pdf/saxon9he.jar -s:'test1.xml' -xsl:'lib/task/pdf/ead-pdf-full-details.xsl' -o:'./test.fo'

Maybe it will spit out a more informative error for us to troubleshoot. Let me know how it goes.

cheers

[filipe...@gmail.com]

hello

We also have the same problem the Report generation is failing with Transforming the EAD with Saxon has failed

Iv tested using the test.xml provide and running the command without sudo renders a permission deny and running with sudo it continues.

[Mike G]

Hi,

So the command works if you use sudo? That definitely sounds like a permissions issue then. Please make sure your AtoM directory can be written to by the appropriate user (the one your web server is running as, typically www-data).

For example, if your AtoM install is /usr/share/nginx/atom, you could issue the following: sudo chown -R www-data:www-data /usr/share/nginx/atom

regards

[filipe...@gmail.com]

Hello

That was done already before i posted here.

didnt work

[Mike G]

Hello,

Can you run `ls -hal` in your AtoM directory and show the output please?

thanks

[filipe...@gmail.com]

Hello

The output of the directory is this:

informatica@atom:/usr/share/nginx/atom$ ls -hal
total 316K
drwxr-xr-x 15 www-data www-data 4.0K Nov  5 13:45 .
drwxr-xr-x  4 root     root     4.0K Nov  5 11:56 ..
drwxr-xr-x  3 www-data www-data 4.0K Aug 27 19:06 apps
drwxr-xr-x  3 www-data www-data 4.0K Nov  5 13:19 cache
drwxr-xr-x  3 www-data www-data 4.0K Nov  5 12:18 config
-rw-rw-r--  1 www-data www-data  485 Jul  8 19:20 CONTRIBUTING
-rw-rw-r--  1 www-data www-data 5.1K Jul  8 19:20 COPYRIGHT
drwxr-xr-x  2 www-data www-data 4.0K Aug 27 19:06 css
drwxr-xr-x  9 www-data www-data 4.0K Nov  5 13:19 data
drwxr-xr-x  2 www-data www-data 4.0K Nov  5 12:05 downloads
-rw-rw-r--  1 www-data www-data 178K Jul  8 19:20 ead.dtd
-rw-rw-r--  1 www-data www-data 1.2K Jul  8 19:20 favicon.ico
drwxr-xr-x  5 www-data www-data 4.0K Aug 27 19:06 images
-rw-rw-r--  1 www-data www-data  229 Jul  8 19:20 index.php
drwxr-xr-x  2 www-data www-data 4.0K Aug 27 19:06 js
drwxr-xr-x 16 www-data www-data 4.0K Aug 27 19:06 lib
-rw-rw-r--  1 www-data www-data  34K Jul  8 19:20 LICENSE
drwxr-xr-x  2 www-data www-data 4.0K Nov  5 12:13 log
drwxr-xr-x 30 www-data www-data 4.0K Aug 27 19:06 plugins
-rw-rw-r--  1 www-data www-data  751 Jul  8 19:20 qubit_dev.php
-rw-rw-r--  1 www-data www-data  334 Jul  8 19:20 README
-rw-rw-r--  1 www-data www-data   26 Jul  8 19:20 robots.txt
-rwxrwxr-x  1 www-data www-data  445 Jul  8 19:20 symfony
-rw-r--r--  1 www-data www-data 1.4K Nov  5 12:49 test1.xml
drwxr-xr-x  5 www-data www-data 4.0K Nov  5 13:56 uploads
drwxr-xr-x 14 www-data www-data 4.0K Aug 27 19:06 vendor
informatica@atom:/usr/share/nginx/atom$

[Mike G]

Hi Filipe,

Admittedly I'm sort of stumped now. Those permissions look OK, although there could be a host of other problems. It's fairly hard to tell without looking at your install personally.

Can you drop this attached file into your AtoM directory, then run: sudo -u www-data java -jar 'lib/task/pdf/saxon9he.jar' -s:'./saxon-test.xml' -xsl:'lib/task/pdf/ead-pdf-inventory-summary.xsl' -o:'./test.fo'

Maybe that will output some more informative errors.

cheers

[Bruno Freitas]

Hi Mike,

I had run the attached file saxon-test.xml manually. My permissions are OK and I'm using the last command that you sent for us (sudo -u www-data...).

The output was

Error 

  I/O error reported by XML parser processing file:/usr/share/nginx/atom/saxon-test.xml:

  Connection timed out

Transformation failed: Run-time errors were reported

Best regards,

Bruno Freitas

[Mike G]

Hey,

I apologize but I'm out of ideas. I'd need access to the server at this point to play around with it, and unfortunately that's a bit beyond forum support.

My only suggestion now is trying a more recent version of the saxon9he.jar (e.g. http://sourceforge.net/projects/saxon/files/Saxon-HE/9.6/SaxonHE9-6-0-7J.zip/download), and see if that bug is fixed.

I should also mention that unless you're using RAD as your default template in AtoM, the finding aids generated will probably not be very presentable anyway; that feature was developed with RAD heavily in mind.

Regards

[andybe...@gmail.com]

Mike,

this happens to me - but only when not connected to the internet.

Atom is on ubuntu 14.04 server
Access from either my desktop (Debian8) or laptop (Windows7)  using Icedove/Thunderbird.

This is a problem in the short term.  I discovered this when moving to the setup that must prevail at the office whilst AtoM is being assessed by the Archivists. - isolated network not connected to the internet.

Hope this will help you toward a solution.

Regards,
Andy
PS I have eliminated the computer and the router by swapping cables around.

[andybe...@gmail.com]

Oops - that should read Icewease/Firefox

On Sunday, 10 January 2016 19:00:39 UTC, andybe...@gmail.com wrote:

Mike,

Atom is on ubuntu 14.04 server
Access from either my desktop (Debian8) or laptop (Windows7)  using Icedove/Thunderbird.

Also......    the date of the generated file was several hours out from my time.  Perhaps because there is no time server available?

[Dan Gillean]

Hi Andy,

Thanks for sharing this. I find this surprising, but as I don't have any further information about the data you are working with, or what your installation is like, I won't speculate further at the moment than one theory: AtoM 2.2's validation was moved to using an XSD - in this case, the source provided in the EAD header is used to fetch the original DTD for validation:

• https://projects.artefactual.com/issues/2787

It's possible that the EAD generation is failing because it can't access the EAD DTD, and therefore proceed with the EAD generation?

I did want to mention another PDF generation issue that we have discovered, and fixed for the upcoming 2.2.1 release:

• https://projects.artefactual.com/issues/9254

This is related to the use of the "public" option when generating finding aids. Essentially, if this option is selected, AtoM will not include physical storage locations or draft records in the Finding aid produced. It does this by first generating the EAD without these elements - but in some cases, it was failing to properly close an EAD tag after these removals, causing the Finding aid generation to fail.

So: if these issues are the case, I might recommend trying it without the public option setting for now... and with an internet connection if possible.

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--

You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0cfc68ec-6976-4d20-98a5-6daa27f88072%40googlegroups.com.

[Johan Pieterse]

Hi All

I found the problem/solution... A hack below...

In the xml generated:

<?xml version="1.0" encoding="utf-8"?>

<!DOCTYPE ead PUBLIC "+//ISBN 1-931666-00-8//DTD ead.dtd (Encoded Archival Description (EAD) Version 2002)//EN" "http://lcweb2.loc.gov/xmlcommon/dtds/ead2002/ead.dtd">

For some reason the http://lcweb2.loc.gov/xmlcommon/dtds/ead2002/ does not find the ead.dtd. I had to point to my local server. Something like http://localhost/atom/data/dtd/

Also had to copy /atom/data/ent folder content to /atom/data/dtd. Stil trying to find out why... 

Edited ./atom/plugins/sfEadPlugin/modules/sfEadPlugin/templates/indexSuccessHeader.xml.php and "fix" to point to local

Regards

Johan

[Dan Gillean]

Hi Johan,

Very strange! I have yet to be able to reproduce this issue, but I thank you very much for sharing your solution. Hopefully it will help others, until we can isolate and reproduce the cause and add a permanent fix.

Kind regards,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/21dbd65c-3a87-429b-aae4-108a3276a7cc%40googlegroups.com.

[Rob Stevens]

I stumbled across this post today when I was looking into the same issue with AtoM 2.3.0.  The 'hack' works with this version as well.

The ultimate cause seems to be that the box which has gearman installed (be it local, or remote) is attempting to make an outbound connection to retrieve http://lcweb2.loc.gov/xmlcommon/dtds/ead2002/ead.dtd.  If it fails (due to, say, a firewall in the way), you will receive the error message described.  By including ead.dtd in the webroot, and updating indexSuccessHeader.xml.php, it works.

-- 

Rob Stevens

[Dan Gillean]

Hi Rob,

Thanks so much for searching the forum for an answer, coming up with a workaround, and updating this thread with the solution. I've made a note for us to update the Troubleshooting section of the Finding aid generation documentation (here) with a description of the issue, the suggestion to check firewall settings and internet connectivity, and a link to your workaround and Johan's above.

We're a bit swamped here, so it might take me a bit to get to it, but it's on my radar now. Thanks again for sharing!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/ffa991c3-f40e-4d91-9135-632ba5eab770%40googlegroups.com.

[Dan Field]

Hi guys. We've run into the same issue with 2.3 here at NLW.

We've modified the code to set the details of our web proxy in order for Java to be able to retrieve external XSD/DTDs

    putenv("JAVA_FLAGS=-Dhttp.proxyHost=cache.llgc.org.uk -Dhttp.proxyPort=80 -Dhttp.nonProxyHosts=\"localhost|127.0.0.1|*.llgc.org.uk\"");

    $cmd = sprintf("java \${JAVA_FLAGS} -jar '%s' -s:'%s' -xsl:'%s' -o:'%s' 2>&1", $saxonPath, $eadFilePath, $eadXslFilePath, $foFilePath);

Hopefully that helps someone else with proxy issues. This could be better set from a config variable but I don;t believe that the Atom installer currently configures a web proxy anywhere in it's workflow. 

On Friday, 11 November 2016 18:28:49 UTC, Dan Gillean wrote:

Hi Rob,

Thanks so much for searching the forum for an answer, coming up with a workaround, and updating this thread with the solution. I've made a note for us to update the Troubleshooting section of the Finding aid generation documentation (here) with a description of the issue, the suggestion to check firewall settings and internet connectivity, and a link to your workaround and Johan's above.

We're a bit swamped here, so it might take me a bit to get to it, but it's on my radar now. Thanks again for sharing!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

On Thu, Nov 10, 2016 at 4:25 PM, Rob Stevens <rostev...@gmail.com> wrote:

I stumbled across this post today when I was looking into the same issue with AtoM 2.3.0.  The 'hack' works with this version as well.

The ultimate cause seems to be that the box which has gearman installed (be it local, or remote) is attempting to make an outbound connection to retrieve http://lcweb2.loc.gov/xmlcommon/dtds/ead2002/ead.dtd.  If it fails (due to, say, a firewall in the way), you will receive the error message described.  By including ead.dtd in the webroot, and updating indexSuccessHeader.xml.php, it works.

-- 

Rob Stevens

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

[Dan Gillean]

Hi Dan,

Thanks for sharing what worked for you! It would definitely useful to see this moved to a config variable in the future.

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/576ff5a9-ce4e-471e-8172-6a2df7362832%40googlegroups.com.

[raul.t...@gmail.com]

This is an old post, but I came across this problem when installing an instance of Atom 2.4 and your solution worked for me, but in case anyone else is having this issue and does not want to alter the code, you can use the  JAVA_TOOL_OPTIONS environment variable. In my case, I just included  the following in the atom-worker's init script 

JAVA_TOOL_OPTIONS="-Dhttp.proxyHost=proxyserver.whatever -Dhttp.proxyPort=8080"

[Dan Gillean]

Thank you for the tip, Raul! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/53eee9e4-60a3-478d-bbef-03ee5e6b887d%40googlegroups.com.

[Cindy]

We ran into the same issue on 2.4, the error was:

I/O error reported by XML parser processing file:/tmp/phpgKzJVQ: Connection refused

  (Connection refused)

Transformation failed: Run-time errors were reported 

Johan's workaround works for us.

Thanks,

Cindy