Import SKOS-XML (Plugin?)

Bernhard

unread,

Aug 15, 2016, 6:17:46 AM8/15/16

to AtoM Users

Hallo!
I can not interpret this message below! Is it possible that anything went wrong during installation (AtoM 2.3)?
- It was possible to import the xml-file (SN_Bz1-3_skos.xml) in version 2.1.
- the sfSkosPlugin (checkbox) is enabled.

_______

root@bm-VirtualBox:/usr/share/nginx/atom# php symfony import:bulk --taxonomy="42" /home/bm/Downloads/SN_Bz1-3_skos.xml
Importing 1 files from /home/bm/Downloads/SN_Bz1-3_skos.xml (indexing is DISABLED) ...

PHP Fatal error: Uncaught Error: Class 'sfSkosPlugin' not found in /usr/share/nginx/atom/lib/QubitXmlImport.class.php:222
Stack trace:
#0 /usr/share/nginx/atom/lib/task/import/importBulkTask.class.php(102): QubitXmlImport->import('/home/bm/Downlo...', Array)
#1 /usr/share/nginx/atom/vendor/symfony/lib/task/sfBaseTask.class.php(68): importBulkTask->execute(Array, Array)
#2 /usr/share/nginx/atom/vendor/symfony/lib/task/sfTask.class.php(97): sfBaseTask->doRun(Object(sfCommandManager), NULL)
#3 /usr/share/nginx/atom/vendor/symfony/lib/command/sfSymfonyCommandApplication.class.php(76): sfTask->runFromCLI(Object(sfCommandManager), NULL)
#4 /usr/share/nginx/atom/vendor/symfony/lib/command/cli.php(20): sfSymfonyCommandApplication->run()
#5 /usr/share/nginx/atom/symfony(14): include('/usr/share/ngin...')
#6 {main}
thrown in /usr/share/nginx/atom/lib/QubitXmlImport.class.php on line 222
root@bm-VirtualBox:/usr/share/nginx/atom#
________

I am pleased about information! What should I do? >New Installation?
Best regards
Bernhard

Dan Gillean

unread,

Aug 17, 2016, 2:49:56 PM8/17/16

to ICA-AtoM Users

Hi Bernhard,

I haven't seen this error before. Can you try clearing the application cache, restarting PHP-FPM, and see if it works?

php symfony cc
sudo service php5-fpm restart

If not, I would like to do a test locally with your SKOS file, if you are willing to share it with me - please feel free to send me a copy on or off-list, and I will attempt to import it into 2.3 on my local test machine, to see if I can reproduce your issue.

Thanks,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/d9acdbd9-f739-452c-915d-a4639782a392%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Dan Gillean

unread,

Aug 17, 2016, 3:12:29 PM8/17/16

to ICA-AtoM Users

As a quick follow up, you could also try restarting nginx and memcached:

sudo service nginx restart
sudo service memcached restart

Finally, you can check to see that the SKOS plugin files are actually present - e.g. from your root AtoM directory, confirm that plugins/sfSkosPlugin/lib/sfSkosPlugin.class.php is in place. Depending on how you've installed or upgraded, it's possible something has gone wrong and the file is missing? You could compare the contents with our version on GitHub, linked above.

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

On Wed, Aug 17, 2016 at 11:49 AM, Dan Gillean <d...@artefactual.com> wrote:

Hi Bernhard,

I haven't seen this error before. Can you try clearing the application cache, restarting PHP-FPM, and see if it works?
php symfony cc
sudo service php5-fpm restart

If not, I would like to do a test locally with your SKOS file, if you are willing to share it with me - please feel free to send me a copy on or off-list, and I will attempt to import it into 2.3 on my local test machine, to see if I can reproduce your issue.

Thanks,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

To post to this group, send email to ica-ato...@googlegroups.com.

Bernhard

unread,

Aug 18, 2016, 11:35:35 AM8/18/16

to AtoM Users

Hi Dan!

Thanks for your help! A week ago I installed AtoM 2.3. on Ubuntu 16.04 LTS (nginx). I'm doing as your instructions say (php7.0). I repeat all today (=new installation). Only one problem I had during installation:

####
root@bm-VirtualBox:/# sudo pecl install apcu_bc-beta
pecl/apcu_bc is already installed and is the same as the released version 1.0.3
install failed
####

But AtoM2.3. is running. All seems to be perfect: It's possible to "add", "edit" and "set" anything per web interface.

When I try to import the skos-file (command-line), I get now the following message:

#####################################################################
root@bm-VirtualBox:/usr/share/nginx/atom# php symfony import:bulk --taxonomy="42" /usr/share/nginx/atom/uploads/SN_skos.xml
PHP Warning: Module 'apc' already loaded in Unknown on line 0
Importing 1 files from /usr/share/nginx/atom/uploads/SN_skos.xml (indexing is DISABLED) ...

PHP Fatal error: Uncaught Error: Class 'sfSkosPlugin' not found in /usr/share/nginx/atom/lib/QubitXmlImport.class.php:222
Stack trace:

#0 /usr/share/nginx/atom/lib/task/import/importBulkTask.class.php(102): QubitXmlImport->import('/usr/share/ngin...', Array)

#1 /usr/share/nginx/atom/vendor/symfony/lib/task/sfBaseTask.class.php(68): importBulkTask->execute(Array, Array)
#2 /usr/share/nginx/atom/vendor/symfony/lib/task/sfTask.class.php(97): sfBaseTask->doRun(Object(sfCommandManager), NULL)
#3 /usr/share/nginx/atom/vendor/symfony/lib/command/sfSymfonyCommandApplication.class.php(76): sfTask->runFromCLI(Object(sfCommandManager), NULL)
#4 /usr/share/nginx/atom/vendor/symfony/lib/command/cli.php(20): sfSymfonyCommandApplication->run()
#5 /usr/share/nginx/atom/symfony(14): include('/usr/share/ngin...')
#6 {main}
thrown in /usr/share/nginx/atom/lib/QubitXmlImport.class.php on line 222
root@bm-VirtualBox:/usr/share/nginx/atom#

######################################################################

"sfSkosPlugin" is on its place. I checked this: plugins/sfSkosPlugin/lib/sfSkosPlugin.class.php
"QubitXmlImport.class.php" is the same like this: https://github.com/artefactual/atom/blob/qa/2.4.x/lib/QubitXmlImport.class.php

Before I import, I do this:

sudo php symfony tools:purge
sudo php symfony cc
sudo php symfony search:populate
sudo systemctl restart php7.0-fpm.service
sudo systemctl restart nginx.service

I read something about problems with "php-apc" and "php7.0". Unfortunately, I do not understand anything in this matter.

best regrads

Bernhard

Dan Gillean

unread,

Aug 18, 2016, 3:02:08 PM8/18/16

to ICA-AtoM Users

Hi Berhard,

I am a bit mystified! Did you follow all the other steps mentioned in the documentation for apcu_bc-beta? The full set of commands is:

sudo apt install php-dev
sudo pecl install apcu_bc-beta
echo "extension=apc.so" >> /etc/php/7.0/mods-available/apcu-bc.ini
ln -sf /etc/php/7.0/mods-available/apcu-bc.ini /etc/php/7.0/fpm/conf.d/30-apcu-bc.ini
ln -sf /etc/php/7.0/mods-available/apcu-bc.ini /etc/php/7.0/cli/conf.d/30-apcu-bc.ini
sudo systemctl restart php7.0-fpm

Docs: https://www.accesstomemory.org/docs/2.3/admin-manual/installation/linux/ubuntu-xenial/#php

I am still using 14.04 and PHP 5.5 in my local test environment on a daily basis, but I will start by trying to import your SKOS file when I have time, to see if it causes any issues. If not, then you are probably correct that it is related to PHP extensions in Ubuntu 16.04.

Regards,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/b0a37ac0-256f-436a-bd03-087282c886c0%40googlegroups.com.

Bernhard Mertelseder

unread,

Aug 18, 2016, 4:21:38 PM8/18/16

to ica-ato...@googlegroups.com

Hi Dan!

Many thanks for the answer! I installed everthing according the proposals from your page (copy&paste):

sudo apt install php-dev

yes, done!

sudo pecl install apcu_bc-beta

yes, done, but it causes the message below

echo "extension=apc.so" >> /etc/php/7.0/mods-available/apcu-bc.ini

yes, done. The file "apcu-bc.ini" contains only one line: extension=apc.so

ln -sf /etc/php/7.0/mods-available/apcu-bc.ini /etc/php/7.0/fpm/conf.d/30-apcu-bc.ini

yes, done. It's a link. in both "files" are the same text (extension=apc.so)

ln -sf /etc/php/7.0/mods-available/apcu-bc.ini /etc/php/7.0/cli/conf.d/30-apcu-bc.ini

yes, done. Also this link is correct. If I open the files, I see the same text.

sudo systemctl restart php7.0-fpm

yes, done. The "service --status-all" says:

[ + ] acpid
[ - ] alsa-utils
[ - ] anacron
[ + ] apparmor
[ + ] apport
[ + ] avahi-daemon
[ - ] bluetooth
[ - ] bootmisc.sh
[ - ] brltty
[ - ] checkfs.sh
[ - ] checkroot-bootclean.sh
[ - ] checkroot.sh
[ + ] console-setup
[ + ] cron
[ + ] cups
[ + ] cups-browsed
[ + ] dbus
[ - ] dns-clean
[ - ] elasticsearch
[ + ] gearman-job-server
[ + ] grub-common
[ - ] hostname.sh
[ - ] hwclock.sh
[ + ] irqbalance
[ - ] kerneloops
[ + ] keyboard-setup
[ - ] killprocs
[ + ] kmod
[ + ] lightdm
[ + ] memcached
[ - ] mountall-bootclean.sh
[ - ] mountall.sh
[ - ] mountdevsubfs.sh
[ - ] mountkernfs.sh
[ - ] mountnfs-bootclean.sh
[ - ] mountnfs.sh
[ + ] mysql
[ + ] network-manager
[ + ] networking
[ + ] nginx
[ + ] ondemand
[ + ] php7.0-fpm
[ - ] plymouth
[ - ] plymouth-log
[ - ] pppd-dns
[ + ] procps
[ + ] rc.local
[ + ] resolvconf
[ - ] rsync
[ + ] rsyslog
[ - ] saned
[ - ] sendsigs
[ + ] speech-dispatcher
[ - ] thermald
[ + ] udev
[ + ] ufw
[ - ] umountfs
[ - ] umountnfs.sh
[ - ] umountroot
[ - ] unattended-upgrades
[ + ] urandom
[ - ] uuidd
[ + ] whoopsie
[ - ] x11-common

I see in this moment that elasticserach is not running. I don't know. Could be this cause the problem?

I try:
sudo systemctl start elasticsearch.service
sudo systemctl start elasticsearch

or

sudo service elasticsearch restart

sudo /etc/init.d/elasticsearch start

To post to this group, send email to ica-ato...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/b0a37ac0-256f-436a-bd03-087282c886c0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/drytfrO9PC0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZKaCdEqc-xeBA25ygjce9E5VVC0VT62LnF00dVCF3B4wg%40mail.gmail.com.

Dan Gillean

unread,

Aug 18, 2016, 4:29:52 PM8/18/16

to ICA-AtoM Users

Hi Bernard,

In 16.04, the correct command to start ES should be:

systemctl start elasticsearch

You can also try:

systemctl status elasticsearch
systemctl restart elasticsearch

I'll try to test with your SKOS file soon and report back!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAA55CmvbtB796o-AbM8F_XJy65mTVbm16mpQEQonPN94tTVbZg%40mail.gmail.com.

Bernhard Mertelseder

unread,

Aug 18, 2016, 4:45:06 PM8/18/16

to ica-ato...@googlegroups.com

Hi Dan!

Sorry! The message disappeared while typing and without end.

But can not start elasticsearch. Should I switch to Ubuntu 14.04?

I will waiting for your response (no hurry!)

Best regards!!

Bernhard

2016-08-18 22:29 GMT+02:00 Dan Gillean <d...@artefactual.com>:

Hi Bernard,

In 16.04, the correct command to start ES should be:
systemctl start elasticsearch
You can also try:
systemctl status elasticsearch
systemctl restart elasticsearch
I'll try to test with your SKOS file soon and report back!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAA55CmvbtB796o-AbM8F_XJy65mTVbm16mpQEQonPN94tTVbZg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/drytfrO9PC0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZKc0hho1wQsf2p5zjK1SAcG6LuRjzhQyT0mDFfoxRnp7g%40mail.gmail.com.

Dan Gillean

unread,

Aug 18, 2016, 6:04:50 PM8/18/16

to ICA-AtoM Users

Hi Bernhard,

One thing to check would be your memory allocation - it's possible Elasticsearch has run out of memory? It's true this could be affecting other things in the VM.

You could certainly try installing using Ubuntu 14.04 - it is more rigorously tested at this point and we have not encountered these problems so far - but I would like to know what the issue is!

Another thing worth mentioning: I noticed you are using Virtualbox. Are you deploying for production, or is this just for local testing? If you are just doing local testing, have you seen our Vagrant box? It might be an easier way for you to make use of Virtualbox. Note that we do not allocate enough resources for these images to be considered ready for production - the Vagrant box is intended for local developers and testers. See:

https://www.accesstomemory.org/docs/2.3/dev-manual/env/vagrant/

See also the first 3 videos in this series if you are trying to install on a local desktop or laptop.

https://www.youtube.com/playlist?list=PLZiwlG5eSMeyeETe15EsEBSu5htPLK-wm

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

On Thu, Aug 18, 2016 at 1:45 PM, Bernhard Mertelseder <bernhard.m...@gmail.com> wrote:

Hi Dan!
Sorry! The message disappeared while typing and without end.
But can not start elasticsearch. Should I switch to Ubuntu 14.04?
I will waiting for your response (no hurry!)
Best regards!!
Bernhard

2016-08-18 22:29 GMT+02:00 Dan Gillean <d...@artefactual.com>:

Hi Bernard,

In 16.04, the correct command to start ES should be:
systemctl start elasticsearch
You can also try:
systemctl status elasticsearch
systemctl restart elasticsearch
I'll try to test with your SKOS file soon and report back!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZKc0hho1wQsf2p5zjK1SAcG6LuRjzhQyT0mDFfoxRnp7g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAA55CmsR7kn5OcSWVAyFVSDPkWeePsaYm0qeyD4wG9zn5mNhMA%40mail.gmail.com.

Dan Gillean

unread,

Aug 24, 2016, 5:07:12 PM8/24/16

to ICA-AtoM Users

Hi Bernhard,

Interestingly, I have tried importing your SKOS file sample... and I got the same error! And yet, I tried with a different SKOS file I made in an earlier version of AtoM (attached here for testing, though it is very silly), and it imported without any difficulty. So the module does not seem to be broken - instead, it seems there might be something about this file that is throwing the error? I'm not sure what - at a glance, I can see that the AtoM version includes URLs as the rdf:resource, but otherwise they seem much the same. Your sample file is also muuuuuch bigger (2.6 MB vs my test file which is 21KB).

If you still have your 2.1 instance and you know the terms import there, I wonder if you can try to import the terms into 2.1, export from 2.1, and try importing that file into AtoM 2.3? My only other suggestion at this time would be to try breaking up the file into smaller pieces if at all possible. It is unfortunate that the error message is extremely unhelpful.

Have you made any progress on your own?

Regards,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

On Thu, Aug 18, 2016 at 3:04 PM, Dan Gillean <d...@artefactual.com> wrote:

Hi Bernhard,

One thing to check would be your memory allocation - it's possible Elasticsearch has run out of memory? It's true this could be affecting other things in the VM.

You could certainly try installing using Ubuntu 14.04 - it is more rigorously tested at this point and we have not encountered these problems so far - but I would like to know what the issue is!

Another thing worth mentioning: I noticed you are using Virtualbox. Are you deploying for production, or is this just for local testing? If you are just doing local testing, have you seen our Vagrant box? It might be an easier way for you to make use of Virtualbox. Note that we do not allocate enough resources for these images to be considered ready for production - the Vagrant box is intended for local developers and testers. See:
https://www.accesstomemory.org/docs/2.3/dev-manual/env/vagrant/
See also the first 3 videos in this series if you are trying to install on a local desktop or laptop.
https://www.youtube.com/playlist?list=PLZiwlG5eSMeyeETe15EsEBSu5htPLK-wm
Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

everything-ever;skos.xml

Bernhard Mertelseder

unread,

Jan 6, 2017, 6:40:28 AM1/6/17

to ica-ato...@googlegroups.com

Hi Dan!

Now i am using der Version 2.4. (Vagrant on my Windwos PC) and it was possible to import also my SKOS-file (about 7500 terms) without problems. I could not solve the problem in August with my (broken?) Ubunutu or AtoM installation, but I learned basics about work with the Command Line at that time.

Regards,

Bernhard

To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZKV5XUQV5R%2B29Y7FUFohjo1OAHvsFCf0ete3rE8g00QnQ%40mail.gmail.com.

Dan Gillean

unread,

Jan 6, 2017, 1:34:17 PM1/6/17

to ICA-AtoM Users

Hi Bernhard,

That's great! Thanks for the update. If you've just downloaded 2.4 recently, then it likely worked because of some of the recent development we've done to better support SKOS imports - see:

https://projects.artefactual.com/issues/10621

Now AtoM will use an external library called EasyRDF to parse SKOS files, and will import them asynchronously using the job scheduler, avoiding browser timeout issues for large documents. EasyRDF supports more than just RDF/XML serializations too - so you should be able to import SKOS files serialized as Turtle or N3 as well, for example. I'm glad to see that our recent development has also solved the issue you were encountering!

We're also finalizing a 2.3.1 release that includes a number of bug fixes for Ubuntu 16.04 installations. Naturally, all of these fixes will be in the 2.4 release as well. You can see the details of 2.3.1 here:

https://wiki.accesstomemory.org/Releases/Release_announcements/Release_2.3.1

We hope to have the bug fix release ready very soon!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZKV5XUQV5R%2B29Y7FUFohjo1OAHvsFCf0ete3rE8g00QnQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.

To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAA55CmuVuGvYk1%3DE-RHSNFz%2B85eHDJne7EHg1Xhr73FBMF1ysA%40mail.gmail.com.

Reply all

Reply to author

Forward