Re: [icatgroup:2155] Digest for icatgroup@googlegroups.com - 6 updates in 1 topic

[William Taylor - STFC UKRI]

Hi both, I don't think this is directly related but just wanted to flag https://github.com/icatproject/icat.server/issues/276 as another issue arising from differences between the code behind the SOAP and REST APIs. Having both use the same implementation would probably be ideal, but I think it'd be a fairly significant undertaking.

Will

________________________________________
From: icat...@googlegroups.com <icat...@googlegroups.com>
Sent: 18 June 2026 10:32
To: Digest recipients
Subject: [icatgroup:2155] Digest for icat...@googlegroups.com - 6 updates in 1 topic

icat...@googlegroups.com<https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!forum/icatgroup/topics> [Google Groups Logo] <https://groups.google.com/forum/?utm_source=digest&utm_medium=email/#!overview> Google Groups<https://groups.google.com/forum/?utm_source=digest&utm_medium=email/#!overview>
Topic digest
View all topics<https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!forum/icatgroup/topics>

* ICAT update entities - 6 Updates

ICAT update entities <http://groups.google.com/group/icatgroup/t/cb6b3ef5a9065b1c?utm_source=digest&utm_medium=email>
Marjolaine Bodin <esrf.marjo...@gmail.com>: Jun 17 05:31PM +0200

Dear all,

I have a question regarding the update of entities in ICAT via the REST
entityManager endpoint.

I'm trying to update the title of an Investigation using a user with CRU
flags on the Investigation table, and I'm running into two issues depending
on what I send.

*Case 1: partial payload*

-> 412 error, OBJECT_ALREADY_EXISTS

I guess there is a conflict with the unique constraint on (FACILITY_ID,
NAME, VISIT_ID).


*Case 2: full payload*

-> 403 error INSUFFICIENT_PRIVILEGES

ICAT treats the update as a delete+recreate of the relationship, which
requires DELETE permission. Is it expected?


*Case 3: Updating it via soap works.*


See attached a script , and the results:

=== Test 1: without facility => 412 OBJECT_ALREADY_EXISTS ===
EXPECTED FAILURE: 412 -
{"code":"OBJECT_ALREADY_EXISTS","message":"Investigation exists with
facility = 'Facility:305', name = 'ID002606', visitId = 'ID00'","offset":0}

=== Test 2: with facility => 403 INSUFFICIENT_PRIVILEGES ===
EXPECTED FAILURE: 403 - {"code":"INSUFFICIENT_PRIVILEGES","message":"DELETE
access implied by UPDATE to this Investigation is not allowed.","offset":0}

=== Test 3: SOAP update => succeeds ===
SUCCESS: SOAP update worked

Am I doing something wrong? Is this a known limitation of the REST
entityManager?

Thanks a lot for your help,

All the best,

Marjolaine
Patrick Austin - STFC UKRI <patrick...@stfc.ac.uk>: Jun 17 04:07PM

Hi Marjolaine,

Unfortunately I cannot read the attachment as due to the extension .py, it gets blocked.

From having briefly looked at the code, it looks like there are the following caveats:

*
If the body contains the id field, then it is treated as an update. If not, it is treated as a create.
*
There is a check to see if during the update, the ids of certain related entities are changed. This is then treated as an "implied" deletion, and if you don't have delete permissions will throw with DELETE access implied by UPDATE to this " + klass.getSimpleName() + " is not allowed.
*
I'm also not 100% convinced this code will function as intended as to me it looks like it will check the wrong ids (i.e. in this case bean is the Investigation but if arg is a related Entity like Facility then its id would always be different)? I don't suppose you have debug logging to see what it says about the update?<https://github.com/icatproject/icat.server/blob/192497568549f74164aad74a1f8bd2a2836578fc/src/main/java/org/icatproject/core/manager/EntityBeanManager.java#L1759-L1774>

If I had to speculate, I'd say you might be hitting these in each of your first two cases respectively? And that to be successful you should include the Investigation id in the body, but not any related entities like Facility.

This is all just based on the icat.server code, so I have not tried this myself and don't have any first hand experience with the endpoint. If you send your Python in the email body rather than as an attachment that might help diagnose the problem.

Thanks,

Patrick

________________________________
From: icat...@googlegroups.com <icat...@googlegroups.com> on behalf of Marjolaine Bodin <esrf.marjo...@gmail.com>
Sent: 17 June 2026 16:31
To: icatgroup <icat...@googlegroups.com>
Subject: [icatgroup:2149] ICAT update entities


Dear all,

I have a question regarding the update of entities in ICAT via the REST entityManager endpoint.

I'm trying to update the title of an Investigation using a user with CRU flags on the Investigation table, and I'm running into two issues depending on what I send.

Case 1: partial payload

-> 412 error, OBJECT_ALREADY_EXISTS

I guess there is a conflict with the unique constraint on (FACILITY_ID, NAME, VISIT_ID).


Case 2: full payload

-> 403 error INSUFFICIENT_PRIVILEGES

ICAT treats the update as a delete+recreate of the relationship, which requires DELETE permission. Is it expected?


Case 3: Updating it via soap works.


See attached a script , and the results:

=== Test 1: without facility => 412 OBJECT_ALREADY_EXISTS ===
EXPECTED FAILURE: 412 - {"code":"OBJECT_ALREADY_EXISTS","message":"Investigation exists with facility = 'Facility:305', name = 'ID002606', visitId = 'ID00'","offset":0}

=== Test 2: with facility => 403 INSUFFICIENT_PRIVILEGES ===
EXPECTED FAILURE: 403 - {"code":"INSUFFICIENT_PRIVILEGES","message":"DELETE access implied by UPDATE to this Investigation is not allowed.","offset":0}

=== Test 3: SOAP update => succeeds ===
SUCCESS: SOAP update worked


Am I doing something wrong? Is this a known limitation of the REST entityManager?

Thanks a lot for your help,

All the best,

Marjolaine


--
You received this message because you are subscribed to the Google Groups "icatgroup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to icatgroup+...@googlegroups.com<mailto:icatgroup+unsubscribe<mailto:icatgroup%2Bunsubscribe>@googlegroups.com>.
To view this discussion visit https://groups.google.com/d/msgid/icatgroup/CAHTuXn5cSt91TG9w7QNhKcDAAy%3DSws%2BG7ske7qgDky8_knpDuQ%40mail.gmail.com<https://groups.google.com/d/msgid/icatgroup/CAHTuXn5cSt91TG9w7QNhKcDAAy%3DSws%2BG7ske7qgDky8_knpDuQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Marjolaine Bodin <esrf.marjo...@gmail.com>: Jun 17 06:09PM +0200

Hi Patrick, of course sorry.

here is the example:

import requests

ICAT_URL = "http://icat-test:8080/icat"
ICAT_SOAP_URL = "http://icat-test:8080/ICATService/ICAT"
SESSION_ID = "valid_session_id" # can update investigation entity
INVESTIGATION_ID = 130847627


def test_update_partial():
print("=== Test 1: without facility => 412 OBJECT_ALREADY_EXISTS ===")
import json
payload = {
"sessionId": SESSION_ID,
"entities": json.dumps([{
"Investigation": {
"id": INVESTIGATION_ID,
"title": "test",
}
}])
}
response = requests.post(f"{ICAT_URL}/entityManager", data=payload)
if response.status_code == 412:
print(f"EXPECTED FAILURE: {response.status_code} - {response.text}")
else:
print(f"UNEXPECTED: {response.status_code} - {response.text}")


def test_update_full_object():
print("\n=== Test 2: with facility => 403 INSUFFICIENT_PRIVILEGES ===")
import json
payload = {
"sessionId": SESSION_ID,
"entities": json.dumps([{
"Investigation": {
"id": INVESTIGATION_ID,
"name": "ID002606",
"visitId": "ID00",
"title": "test",
"facility": {"id": 305},
"type": {"id": 325},
}
}])
}
response = requests.post(f"{ICAT_URL}/entityManager", data=payload)
if response.status_code == 403:
print(f"EXPECTED FAILURE: {response.status_code} - {response.text}")
else:
print(f"UNEXPECTED: {response.status_code} - {response.text}")


def test_soap():
print("\n=== Test 3: SOAP update => succeeds ===")
soap_body = f"""<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:tns="http://icatproject.org">
<soap:Body>
<tns:update>
<sessionId>{SESSION_ID}</sessionId>
<bean xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:type="tns:investigation">
<id>{INVESTIGATION_ID}</id>
<name>ID002606</name>
<visitId>ID00</visitId>
<title>test</title>
<startDate>2026-06-17T08:00:00.000+00:00</startDate>
<facility><id>305</id></facility>
<type><id>325</id></type>
</bean>
</tns:update>
</soap:Body>
</soap:Envelope>"""

response = requests.post(
ICAT_SOAP_URL,
data=soap_body.encode("utf-8"),
headers={"Content-Type": "text/xml; charset=utf-8", "SOAPAction":
""}
)
if response.status_code == 200:
print("SUCCESS: SOAP update worked")
else:
print(f"UNEXPECTED FAILURE: {response.status_code} -
{response.text}")


if __name__ == "__main__":
test_update_partial()
test_update_full_object()
test_soap()

On Wed, Jun 17, 2026 at 6:07 PM Patrick Austin - STFC UKRI <
Marjolaine Bodin <esrf.marjo...@gmail.com>: Jun 17 06:12PM +0200

I’m sending the investigation ID, so this is treated as an update (as shown
in the ICAT logs). However, when I include the facility ID and type, I get
a PERMISSIONS error, which I don’t understand.
If I don't include them (first case), I have this other 412 error.

Thanks a lot for your feedback!
Marjolaine

On Wed, Jun 17, 2026 at 6:09 PM Marjolaine Bodin <
Patrick Austin - STFC UKRI <patrick...@stfc.ac.uk>: Jun 17 04:30PM

Hi Marjolaine,

Regarding the first error (partial update), I can't see an obvious place where the problem is - I would have expected that to work, but it must be going wrong somewhere.

Regarding the second, comparing what the REST vs. SOAP versions do I'm fairly confident that the former is wrong, and will always erroneously think you're changing the id of the related item (probably Facility in this case), leading to it checking DELETE permissions, which you don't have.

While I think that could be patched, I think the more general question would be why do the two interfaces not share the same underlying code? And rather than patching the one that is seemingly less reliable, can/should we use the same underlying functions on EntityBeanManager?

Potentially questions for a collaboration meeting?

Patrick

________________________________
From: Marjolaine Bodin <esrf.marjo...@gmail.com>
Sent: 17 June 2026 17:12
To: Austin, Patrick (STFC,RAL,SC) <patrick...@stfc.ac.uk>
Cc: icatgroup <icat...@googlegroups.com>
Subject: Re: [icatgroup:2149] ICAT update entities

I’m sending the investigation ID, so this is treated as an update (as shown in the ICAT logs). However, when I include the facility ID and type, I get a PERMISSIONS error, which I don’t understand.
If I don't include them (first case), I have this other 412 error.

Thanks a lot for your feedback!
Marjolaine

On Wed, Jun 17, 2026 at 6:09 PM Marjolaine Bodin <esrf.marjo...@gmail.com<mailto:esrf.marjolaine.bodin<mailto:esrf.marjolaine.bodin>@gmail.com>> wrote:
Hi Patrick, of course sorry.

here is the example:

import requests

ICAT_URL = "http://icat-test:8080/icat"
ICAT_SOAP_URL = "http://icat-test:8080/ICATService/ICAT"
SESSION_ID = "valid_session_id" # can update investigation entity
INVESTIGATION_ID = 130847627


def test_update_partial():
print("=== Test 1: without facility => 412 OBJECT_ALREADY_EXISTS ===")
import json
payload = {
"sessionId": SESSION_ID,
"entities": json.dumps([{
"Investigation": {
"id": INVESTIGATION_ID,
"title": "test",
}
}])
}
response = requests.post<http://requests.post>(f"{ICAT_URL}/entityManager", data=payload)
if response.status_code == 412:
print(f"EXPECTED FAILURE: {response.status_code} - {response.text}")
else:
print(f"UNEXPECTED: {response.status_code} - {response.text}")


def test_update_full_object():
print("\n=== Test 2: with facility => 403 INSUFFICIENT_PRIVILEGES ===")
import json
payload = {
"sessionId": SESSION_ID,
"entities": json.dumps([{
"Investigation": {
"id": INVESTIGATION_ID,
"name": "ID002606",
"visitId": "ID00",
"title": "test",
"facility": {"id": 305},
"type": {"id": 325},
}
}])
}
response = requests.post<http://requests.post>(f"{ICAT_URL}/entityManager", data=payload)
if response.status_code == 403:
print(f"EXPECTED FAILURE: {response.status_code} - {response.text}")
else:
print(f"UNEXPECTED: {response.status_code} - {response.text}")


def test_soap():
print("\n=== Test 3: SOAP update => succeeds ===")
soap_body = f"""<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tns="http://icatproject.org">
<soap:Body>
<tns:update>
<sessionId>{SESSION_ID}</sessionId>
<bean xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="tns:investigation">
<id>{INVESTIGATION_ID}</id>
<name>ID002606</name>
<visitId>ID00</visitId>
<title>test</title>
<startDate>2026-06-17T08:00:00.000+00:00</startDate>
<facility><id>305</id></facility>
<type><id>325</id></type>
</bean>
</tns:update>
</soap:Body>
</soap:Envelope>"""

response = requests.post<http://requests.post>(
ICAT_SOAP_URL,
data=soap_body.encode("utf-8"),
headers={"Content-Type": "text/xml; charset=utf-8", "SOAPAction": ""}
)
if response.status_code == 200:
print("SUCCESS: SOAP update worked")
else:
print(f"UNEXPECTED FAILURE: {response.status_code} - {response.text}")


if __name__ == "__main__":
test_update_partial()
test_update_full_object()
test_soap()

On Wed, Jun 17, 2026 at 6:07 PM Patrick Austin - STFC UKRI <patrick...@stfc.ac.uk<mailto:patrick.austin<mailto:patrick.austin>@stfc.ac.uk>> wrote:
Hi Marjolaine,

Unfortunately I cannot read the attachment as due to the extension .py, it gets blocked.

From having briefly looked at the code, it looks like there are the following caveats:

*
If the body contains the id field, then it is treated as an update. If not, it is treated as a create.
*
There is a check to see if during the update, the ids of certain related entities are changed. This is then treated as an "implied" deletion, and if you don't have delete permissions will throw with DELETE access implied by UPDATE to this " + klass.getSimpleName() + " is not allowed.
*
I'm also not 100% convinced this code will function as intended as to me it looks like it will check the wrong ids (i.e. in this case bean is the Investigation but if arg is a related Entity like Facility then its id would always be different)? I don't suppose you have debug logging to see what it says about the update?<https://github.com/icatproject/icat.server/blob/192497568549f74164aad74a1f8bd2a2836578fc/src/main/java/org/icatproject/core/manager/EntityBeanManager.java#L1759-L1774>

If I had to speculate, I'd say you might be hitting these in each of your first two cases respectively? And that to be successful you should include the Investigation id in the body, but not any related entities like Facility.

This is all just based on the icat.server code, so I have not tried this myself and don't have any first hand experience with the endpoint. If you send your Python in the email body rather than as an attachment that might help diagnose the problem.

Thanks,

Patrick

________________________________
From: icat...@googlegroups.com<mailto:icatgroup<mailto:icatgroup>@googlegroups.com> <icat...@googlegroups.com<mailto:icatgroup<mailto:icatgroup>@googlegroups.com>> on behalf of Marjolaine Bodin <esrf.marjo...@gmail.com<mailto:esrf.marjolaine.bodin<mailto:esrf.marjolaine.bodin>@gmail.com>>
Sent: 17 June 2026 16:31
To: icatgroup <icat...@googlegroups.com<mailto:icatgroup<mailto:icatgroup>@googlegroups.com>>
Subject: [icatgroup:2149] ICAT update entities


Dear all,

I have a question regarding the update of entities in ICAT via the REST entityManager endpoint.

I'm trying to update the title of an Investigation using a user with CRU flags on the Investigation table, and I'm running into two issues depending on what I send.

Case 1: partial payload

-> 412 error, OBJECT_ALREADY_EXISTS

I guess there is a conflict with the unique constraint on (FACILITY_ID, NAME, VISIT_ID).


Case 2: full payload

-> 403 error INSUFFICIENT_PRIVILEGES

ICAT treats the update as a delete+recreate of the relationship, which requires DELETE permission. Is it expected?


Case 3: Updating it via soap works.


See attached a script , and the results:

=== Test 1: without facility => 412 OBJECT_ALREADY_EXISTS ===
EXPECTED FAILURE: 412 - {"code":"OBJECT_ALREADY_EXISTS","message":"Investigation exists with facility = 'Facility:305', name = 'ID002606', visitId = 'ID00'","offset":0}

=== Test 2: with facility => 403 INSUFFICIENT_PRIVILEGES ===
EXPECTED FAILURE: 403 - {"code":"INSUFFICIENT_PRIVILEGES","message":"DELETE access implied by UPDATE to this Investigation is not allowed.","offset":0}

=== Test 3: SOAP update => succeeds ===
SUCCESS: SOAP update worked


Am I doing something wrong? Is this a known limitation of the REST entityManager?

Thanks a lot for your help,

All the best,

Marjolaine


--
You received this message because you are subscribed to the Google Groups "icatgroup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to icatgroup+...@googlegroups.com<mailto:icatgroup+unsubscribe<mailto:icatgroup%2Bunsubscribe>@googlegroups.com>.
To view this discussion visit https://groups.google.com/d/msgid/icatgroup/CAHTuXn5cSt91TG9w7QNhKcDAAy%3DSws%2BG7ske7qgDky8_knpDuQ%40mail.gmail.com<https://groups.google.com/d/msgid/icatgroup/CAHTuXn5cSt91TG9w7QNhKcDAAy%3DSws%2BG7ske7qgDky8_knpDuQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Marjolaine Bodin <esrf.marjo...@gmail.com>: Jun 18 09:25AM +0200

Hi Patrick,
Thanks a lot for your feedback!
Happy to discuss further in a collaboration meeting if that makes sense!
All the best,
Marjolaine

On Wed, Jun 17, 2026 at 6:30 PM Patrick Austin - STFC UKRI <
Back to top<./#digest_top>
You received this digest because you're subscribed to updates for this group. You can change your settings on the group membership page<https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!forum/icatgroup/join>.
To unsubscribe from this group and stop receiving emails from it send an email to icatgroup+...@googlegroups.com<mailto:icatgroup+...@googlegroups.com>.

[Marjolaine Bodin]

Hi William,

That's very interesting, thank you. So I run a test to update (partial update) an instrument, and it works well. So it may indeed be linked to the investigation.

{
            "Instrument": {
                "id": 106066527,
                "description": "test",
            }
 }

All the best,

  Marjolaine

To unsubscribe from this group and stop receiving emails from it, send an email to icatgroup+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/icatgroup/CWLP265MB4436D2F277418902605D8BACEAE32%40CWLP265MB4436.GBRP265.PROD.OUTLOOK.COM.