Three Issues with SPARQL update to delete rdf properties in Fedora 4.7.1

[Steve Gu]

I was trying to ingest documents into Fedora 4.7.1 and found three issues with SPARQL when I was trying to update the rdf resource of the object.

How I implemented it:

In a similar way to fedora 4 migration utility.  RDF properties were grabbed from the resource and created SPARQL update statement which is passed to fedora 4 client to update.  The SPARQL statements include DELETE and INSERT.  All properties will be first deleted before they are inserted.

Issues I found:

All issues are related to DELETE.

1. if I create multiple DELETE for the same predicate, such as  <> <predicate1> ?o0 .  <> <predicate1> ?o1 .,  then the system will hang until the error of OutOfMemory.  This has been posted as another topic

2. If there is a predicate that the current object does not have a value, then the block of DELETE statements were not executed.  In the example below, predicate2, 3 are not deleted.

<> <predicate_with_no_value_in_the_object> ?o0 .

          <> <predicate2> ?o1 .

<> <predicate3> ?o2. 

3. The DELETE block can only contain a sinlge one predicate that currently have multiple values.  If there are more than one, the statement won't be executed.  If the statements were executed on fedora 4.7 admin, page, a error popup without any message shows up if there are more than three.

<> <predicate1_with_multiple_values_in_the_object> ?o0 .

          <> <predicate2_with multiple_values_in the object> ?o1 .

<> <predicate3> ?o2. 

Has anybody noticed the same issue before?  This seems to me a bug that needs to be fixed.   

I do not want to query the object to first and issue separate statements for properties with multiple values.  Any other work around?  Or I totally get the sparql statement wrong?

thanks for your help.

Steve

[aj...@virginia.edu]

Can you give us complete examples instead of brief excerpts? Please include the data against which you are working and complete SPARQL forms.

---

A. Sororal

the University of Virginia Library 

--
You received this message because you are subscribed to the Google Groups "Fedora Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fedora-communi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Steve Gu]

I attached for files, and here is how to reproduce:

1. curl -X PATCH -H "Content-Type: application/sparql-update" --data-binary "@rdf_insert" "http://localhost:9030/fcrepo/rest/object/to/insert" to add rdf resources.  There will be four properties with multiple values, dc:subject, fedora3model:datastream, fedora3model:keywords, and fedora3model:mods.subject_topic

2. run patch with @rdf_delete.txt.  The file contains all the four properties with multiple values.  They are at the end of the file. -----Fail  No properties deleted.

3. run patch with @rdf_delete_one_multiple.txt.  The file contains only one property with multiple values.  -----  Success.

4. Run 1 to insert the properties back

5. Run patch with @rdf_delete_one_multiple_one_non_existing.txt.  The file contains one property that does not currently exist.  ------Fail.  No properties deleted.

6.  Run path with @rdf_delete_duplicates.txt.  The file contains multiple entries for the same property.   With HikCP 2.4.3, the system was hung until the error of OutOfMemory (about 2 hours).   With HikCP2.5.1, the system was hung but for much less time.

thanks
Steve

[Kevin Ford]

Thanks, Steve, for the extra info.

Using your files, I can't quite reproduce the problem you are
experiencing - that is, HikCP hanging until an OutOfMemory error is
reached - but the results are problematic nonetheless. In my case, the
PATCH delete requests almost always resulted in no triples being deleted
and sometimes a 400 Bad Request response. I will say that the last
delete statement caused my fcrepo to hang for a minute or two before it
dumped out a 400 Bad Request.

I started to whittle the queries down so I could see at which point
something was a problem. I filed this issue -
https://jira.duraspace.org/browse/FCREPO-2383 - as a result. That's not
to say this is the cause of the issue you are experiencing, but it is a
problem nonetheless.

I included a couple of test scripts in the report. If you have a spare
moment, I'd be interested to know how your system responds.

Yours,
Kevin

> --
> You received this message because you are subscribed to the Google
> Groups "Fedora Community" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to fedora-communi...@googlegroups.com

> <mailto:fedora-communi...@googlegroups.com>.

[Bethany Seeger]

I looked into this a little today and have this to share. This is with
the fcrepo-webapp RC 2 & tomcat.

With these two files, I see something akin to recursion happening in
fedora.

Insert.txt:
INSERT {
<> <http://purl.org/dc/elements/1.1/title> "The title1".
<> <http://purl.org/dc/elements/1.1/title> "The title2".
<> <http://purl.org/dc/elements/1.1/title> "The title3".
<> <http://purl.org/dc/elements/1.1/title> "The title4".
<> <http://purl.org/dc/elements/1.1/title> "The title5".
} WHERE {}

Delete.txt:

DELETE WHERE {

<> <http://purl.org/dc/elements/1.1/title> ?o2 .
<> <http://purl.org/dc/elements/1.1/title> ?o40 .
<> <http://purl.org/dc/elements/1.1/title> ?o41 .
<> <http://purl.org/dc/elements/1.1/title> ?o42 .
<> <http://purl.org/dc/elements/1.1/title> ?o43 .

}

Running the two seems to send fedora into a nose dive. Eventually it runs
into an exception and returns this info to my curl command:

curl -X PATCH -H "Content-Type:application/sparql-update" --data-binary

"@rdf_del_dups_title.txt" "http://localhost:8080/fcrepo/rest/test2"
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'
Property 'dc:title': Unable to remove value 'The
title5^^http://www.w3.org/2001/XMLSchema#str...'

You can see a snippet of my log file here; it has debug messages turned
on: https://gist.github.com/bseeger/af167065e1b611d1d8f099c0d56462f6
The section that you see repeating repeats many times before it gets to
this point in the log.


If I run the above delete file with only one triple total (<>
<http://purl.org/dc/elements/1.1/title> ?o2 .) it deletes the
dcterms:title triples just fine. When I add a second:

DELETE WHERE {

<> <http://purl.org/dc/elements/1.1/title> ?o2 .
<> <http://purl.org/dc/elements/1.1/subject> ?o3 .
}


That’s when this odd behavior starts to happen and nothing gets deleted.

-Bethany



On 1/13/17, 6:41 PM, "fedora-c...@googlegroups.com on behalf of Kevin
Ford" <fedora-c...@googlegroups.com on behalf of kfo...@artic.edu>
wrote:

>email to fedora-communi...@googlegroups.com.

[Andrew Woods]

Hello Steve,

Thank you very much for the scripted test scenarios. They are extremely helpful towards pinpointing the issues.

You may have seen that the ticket Kevin created has been resolved:

* https://jira.duraspace.org/browse/FCREPO-2383

My testing indicates that the fix to that ticket resolves your issue #2. It would be great if you could also verify that.

For your issues #5 and #6, I have created the following two tickets:

* https://jira.duraspace.org/browse/FCREPO-2390

* https://jira.duraspace.org/browse/FCREPO-2391

With the help of your scripts, both of those issues are easy to reproduce. If you have an interest in digging more deeply into the Fedora code base, these tickets represent excellent opportunities.

Additionally, you are encouraged to attend the weekly Tech Meeting on Thursdays @11am ET.

https://wiki.duraspace.org/display/FF/2017-01-26+-+Fedora+Tech+Meeting

Regards,

Andrew