Issue 71 in contentmirror: Plone throws IntegrityCheck Error when Creating/Modifying Content

[conten...@googlecode.com]

Status: New
Owner: ----
Labels: Type-Defect Priority-Medium

New issue 71 by davydany: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

What steps will reproduce the problem?
1. Install ContentMirror to Plone using the latest version from SVN.
Install with SQLAlchemy 0.5.8.
2. Configure it to run with PostgreSQL 8.3, and create the tables in
Postgres generated by mirror-ddl's SQL (also happens after doing a bulk
import, using mirror-bulk)
3. Create Content. Change the state from Private to Published. Or just
modify content.

What is the expected output? What do you see instead?
The expected output is for the content to be created, and mirrored into the
Postgres DB.

However, I get an exception. I posted the exception below (for readability
purposes)

What version of the product are you using? On what operating system?
Plone 3.3.5
ContentMirror from SVN (r226)
Ubuntu 8.04.4 LTS

Please provide any additional information below.
Traceback (innermost last):
Module ZPublisher.Publish, line 125, in publish
Module Zope2.App.startup, line 238, in commit
Module transaction._manager, line 89, in commit
Module transaction._transaction, line 323, in commit
Module transaction._transaction, line 373, in _callBeforeCommitHooks
Module ore.contentmirror.operation, line 205, in flush
Module sqlalchemy.orm.session, line 1354, in flush
Module sqlalchemy.orm.session, line 1432, in _flush
Module sqlalchemy.orm.unitofwork, line 261, in execute
Module sqlalchemy.orm.unitofwork, line 753, in execute
Module sqlalchemy.orm.unitofwork, line 773, in execute_save_steps
Module sqlalchemy.orm.unitofwork, line 789, in
execute_cyclical_dependencies
Module sqlalchemy.orm.unitofwork, line 753, in execute
Module sqlalchemy.orm.unitofwork, line 773, in execute_save_steps
Module sqlalchemy.orm.unitofwork, line 789, in
execute_cyclical_dependencies
Module sqlalchemy.orm.unitofwork, line 753, in execute
Module sqlalchemy.orm.unitofwork, line 768, in execute_save_steps
Module sqlalchemy.orm.unitofwork, line 759, in save_objects
Module sqlalchemy.orm.mapper, line 1428, in _save_obj
Module sqlalchemy.engine.base, line 824, in execute
Module sqlalchemy.engine.base, line 874, in _execute_clauseelement
Module sqlalchemy.engine.base, line 896, in __execute_context
Module sqlalchemy.engine.base, line 950, in _cursor_execute
Module sqlalchemy.engine.base, line 931, in _handle_dbapi_exception
IntegrityError: (IntegrityError) duplicate key value violates unique
constraint "content_content_uid_idx"
'INSERT INTO content (content_id, id, content_uid, object_type, status,
portal_type, folder_position, container_id, path, relative_path, title,
description, subject, location, contributors, creators, creation_date,
modification_date, effectivedate, expirationdate, language, rights,
excludefromnav) VALUES
(%(content_id)s, %(id)s, %(content_uid)s, %(object_type)s, %(status)s, %(portal_type)s, %(folder_position)s, %(container_id)s, %(path)s, %(relative_path)s, %(title)s, %(description)s, %(subject)s, %(location)s, %(contributors)s, %(creators)s, %(creation_date)s, %(modification_date)s, %(effectivedate)s, %(expirationdate)s, %(language)s, %(rights)s, %(excludefromnav)s)'
{'contributors': '', 'object_type': 'ATFolderPeer', 'creation_date':
datetime.datetime(2010, 7, 12, 19, 42, 54,
999000), 'id': 'news', 'subject': '', 'modification_date':
datetime.datetime(2010, 7, 13, 17, 46, 30,
353000), 'title': 'News', 'folder_position':
0, 'location': '', 'excludefromnav':
False, 'status': 'published', 'description': 'This folder contains only the
News Stories.', 'effectivedate': datetime.datetime(2010, 7, 13, 17, 46, 30,
352000), 'content_uid': 'ec1b8571e7fa619b9d0a8778ade2a02d', 'portal_type': 'Folder', 'container_id':
1, 'relative_path': 'news-events/news', 'path': '/crest/news-events/news', 'creators': 'admin', 'language': '', 'rights': '', 'expirationdate':
None, 'content_id': 6L}

[conten...@googlecode.com]

Comment #1 on issue 71 by davydany: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

I also noticed that this doesn't come initially. Try creating about 3-4
Contents and it gets thrown.

I mentioned that this error comes with Postgres, but it also comes with
MySQL.

[conten...@googlecode.com]

Comment #2 on issue 71 by davydany: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

Any updates? It would save me a lot of trouble if this issue would be
resolved.

[conten...@googlecode.com]

Comment #3 on issue 71 by kapilt: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

Offhand it looks like you have multiple content objects with the same
archetypes UID in plone, which seems strange. If you query the archtypes
UID from the database that's given in the traceback it would be interesting
to validate that what is serialized is indeed the news event path that's in
the traceback.

Looking at the content type in the traceback, it seems to be some sort of
peer content type, which might the issue, namely that its ATPeerFolder.. I
haven't see this before, but if it behaves as though it has the same
archetypes UID as another content object then that would explain the
constraint violation. Its not clear to me off hand what use such a peer
would have, but i can say that's its violating a basic tenent of
contentmirror and plone, namely that a content object's archetypes UID is
unique, and that's its not supported. If you wanted to you could setup a
serializer for this content type that either didn't insert anything into
the database or inserted something with a unique UID that's specific to the
peer instance (path hash would work).

[conten...@googlecode.com]

Comment #4 on issue 71 by kapilt: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

doh.. sorry i totally missed that atfolderpeer is the contentmirror
generated peer. I'm still uncertain why there would be an integrity error
on UID unless there is a duplicate content UID in plone. It would be good
to have a sanity check that the uid that matches in the database has the
same path as the object being referenced here.

[conten...@googlecode.com]

Comment #5 on issue 71 by davydany: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

I've been playing with it a little more. Here is what I did, and the errors
I got:

1. Created new folder called "Test Folder"
2. Renamed test folder as "Dummy Folder" and got error message for
IntegrityCheck Error. Folder still remained as "Test Folder"
Inspecting it further, I saw that it was using this UID:
9e159e6ccec77133ddebfc2e3e9953fe

3. Tried creating an image content inside "Test Folder". Same IntegritCheck
Error was thrown with the same UID: 9e159e6ccec77133ddebfc2e3e9953fe

So the UID isn't being newly generated.

[conten...@googlecode.com]

Updates:
Status: Fixed

Comment #6 on issue 71 by kapilt: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

So it looks like the event subscriber was generating an add operation in
response to modified/workflow events instead of an update operation. Looks
like the regression was from mid-april 2010. I fixed this in Revision 228,
and added several tests to verify the operations generated for each event.

[conten...@googlecode.com]

Updates:
Labels: Milestone-Release0.6

Comment #7 on issue 71 by kapilt: Plone throws IntegrityCheck Error when
Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

(No comment was entered for this change.)

[conten...@googlecode.com]

Comment #8 on issue 71 by d...@tonimueller.org: Plone throws IntegrityCheck

Error when Creating/Modifying Content
http://code.google.com/p/contentmirror/issues/detail?id=71

I'm using 0.7, and get:

2011-07-29T17:14:24 ERROR Zope.SiteErrorLog 1311952464.650.737942450127
https://www.example.com/portal_factory/Event/event.2011-07-29.2293407898/atct_edit
Traceback (innermost last):
Module ZPublisher.Publish, line 135, in publish
Module Zope2.App.startup, line 291, in commit
Module transaction._manager, line 93, in commit
Module transaction._transaction, line 316, in commit
Module transaction._transaction, line 366, in _callBeforeCommitHooks
Module ore.contentmirror.operation, line 212, in flush
Module sqlalchemy.orm.session, line 1388, in flush
Module sqlalchemy.orm.session, line 1469, in _flush
Module sqlalchemy.orm.unitofwork, line 302, in execute
Module sqlalchemy.orm.unitofwork, line 446, in execute
Module sqlalchemy.orm.mapper, line 1878, in _save_obj
Module sqlalchemy.engine.base, line 1191, in execute
Module sqlalchemy.engine.base, line 1271, in _execute_clauseelement
Module sqlalchemy.engine.base, line 1302, in __execute_context
Module sqlalchemy.engine.base, line 1401, in _cursor_execute
Module sqlalchemy.engine.base, line 1394, in _cursor_execute
Module sqlalchemy.engine.default, line 299, in do_execute

IntegrityError: (IntegrityError) duplicate key value violates unique
constraint "content_content_uid_idx"
"INSERT INTO content (content_id, id, content_uid, object_type, status,
portal_type, folder_position, container_id, path, relative_path, title,
description, subject, location, contributors, creators, creation_date,
modification_date, effectivedate, expirationdate, language, rights,
excludefromnav) VALUES

(nextval('content_sequence'), %(id)s, %(content_uid)s, %(object_type)s, %(status)s, %(portal_type)s, %(folder_position)s, %(container_id)s, %(path)s, %(relative_path)s, %(title)s, %(description)s, %(subject)s, %(location)s, %(contributors)s, %(creators)s, %(creation_date)s, %(modification_date)s, %(effectivedate)s, %(expirationdate)s, %(language)s, %(rights)s, %(excludefromnav)s)
RETURNING content.content_id"
{'contributors': '', 'object_type': 'ATEventPeer', 'creation_date':
datetime.datetime(2011, 7, 29, 17, 14, 24,
255571), 'id': '2018the-new-energy-era2019', 'subject': '', 'modification_date':
datetime.datetime(2011, 7, 29, 17, 14, 24,
568288), 'title': '\xe2\x80\x98The New Energy
Era\xe2\x80\x99 ', 'folder_position': 249, 'location': 'West-Cork,
Ireland', 'excludefromnav':
False, 'status': 'private', 'description': '', 'effectivedate':
None, 'content_uid': '03f7602407a7077a32d09390b2c8a091', 'portal_type': 'Event', 'container_id':
None, 'relative_path': '2018the-new-energy-era2019', 'path': '/01/mnt/Plone/2018the-new-energy-era2019', 'language': 'en', 'rights': '', 'expirationdate':
None, 'creators': 'woestmann'}

Searching the database does not reveal an object with this UID, however:

contentmirror=# select * from content where content_uid
= '03f7602407a7077a32d09390b2c8a091';
content_id | id | content_uid | object_type | status | portal_type |
folder_position | container_id | path | relative_path | title | description
| subject | location | contributors | creators | creation_date |
modification_date | effectivedate | expirationdate | language | rights |
excludefromnav
------------+----+-------------+-------------+--------+-------------+-----------------+--------------+------+---------------+-------+-------------+---------+----------+--------------+----------+---------------+-------------------+---------------+----------------+----------+--------+----------------
(0 rows)