Stable 4 - Sample migration error

[Steve Castellarin]

I've been working with our 3.1 CRITs database and have been cleaning up old data, etc.  When it comes to my sample migration - I get about half way through the samples and get a rather generic error:

An error occurred during migration!

Migrated: 0

Error: AttributeError

Document ID: XXX

I'll go to that document in Mongo, get it's information then view the sample in CRITs 3.1 - and then delete that sample from CRITs.  When I try my next mongodump/mongorestore, relationship vocabulary changes and then run the manage.py upgrade  script I get another error, with a different Document ID.  I've done this a handful of times and it's getting frustrating.

How can I go about debugging what sort of error is causing the upgrade/migration to fail?

Thanks!

[Michael Goffin]

You have an attribute of the sample that isn't correct and needs to be fixed. It's just bad data. Your best bet instead of just nuking samples, is to find out what the issue is with one sample, and then blanket verify it's not an issue with any others, and move on to the next problem.

Unfortunately lots of people write scripts or directly modify the database and do things we cannot account for. And for people with very old versions of CRITs (3.1 for example) trying to upgrade to the latest version which is a year or more newer, this is going to happen. This is also why we have advocated for years that people track the stable branch for their installation, not base it off of a zip file, and pull changes often to keep up-to-date.

--
You received this message because you are subscribed to the Google Groups "crits-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crits-users+unsubscribe@googlegroups.com.
To post to this group, send email to crits...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/crits-users/CACKvhwE%3DJOaD8ygF1WavWDp9tswrV9NRuvR425X317664DNzMw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Steve Castellarin]

Hey Mike,

Delete the two samples was a stab in the dark - not the way I want to go through migrating.

In my last upgrade attempt I have one sample that I've checked through and I'm not seeing what attribute could be causing the issue.  I've run db.sample.distinct("relationships.relationship") and that shows everything matching the vocabulary.  And there are no object attributes for this sample.

Any idea on where I can look next?

On Wednesday, February 15, 2017 at 4:20:46 PM UTC-5, Michael Goffin wrote:

You have an attribute of the sample that isn't correct and needs to be fixed. It's just bad data. Your best bet instead of just nuking samples, is to find out what the issue is with one sample, and then blanket verify it's not an issue with any others, and move on to the next problem.

Unfortunately lots of people write scripts or directly modify the database and do things we cannot account for. And for people with very old versions of CRITs (3.1 for example) trying to upgrade to the latest version which is a year or more newer, this is going to happen. This is also why we have advocated for years that people track the stable branch for their installation, not base it off of a zip file, and pull changes often to keep up-to-date.

On Wed, Feb 15, 2017 at 3:34 PM, Steve Castellarin <steve.ca...@gmail.com> wrote:

I've been working with our 3.1 CRITs database and have been cleaning up old data, etc.  When it comes to my sample migration - I get about half way through the samples and get a rather generic error:

An error occurred during migration!

Migrated: 0

Error: AttributeError

Document ID: XXX

I'll go to that document in Mongo, get it's information then view the sample in CRITs 3.1 - and then delete that sample from CRITs.  When I try my next mongodump/mongorestore, relationship vocabulary changes and then run the manage.py upgrade  script I get another error, with a different Document ID.  I've done this a handful of times and it's getting frustrating.

How can I go about debugging what sort of error is causing the upgrade/migration to fail?

Thanks!

--
You received this message because you are subscribed to the Google Groups "crits-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to crits-users...@googlegroups.com.

[Michael Goffin]

Without having direct access to your data (which I don't want), I can't tell you what the issue is. The "new vocabulary" won't cause this issue because it's just string changes. Having improperly formatted sub-documents or values which don't match the field type are the main issue.

To unsubscribe from this group and stop receiving emails from it, send an email to crits-users+unsubscribe@googlegroups.com.

To post to this group, send email to crits...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/crits-users/2ba0a380-d3e9-4cee-ac60-ab5cadde279d%40googlegroups.com.

[Bradley Logan]

I might know what your problem is.

Try changing line 138 of crits/crits/samples/sample.py from:

"        if self.filedata.grid_id:"

to:

"        if self.filedata.grid_id and hasattr(self.filedata, 'seek'):"

If this fixes it, the problem is likely that CRITs thinks you have a binary for the current Sample, but there isn't actually any binary data in the DB, so the 'seek' attribute is invalid.

[Steve Castellarin]

Hey Bradley,

Thanks for the response.  I made the change and that did the trick.

Thanks!
Steve

--
You received this message because you are subscribed to a topic in the Google Groups "crits-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/crits-users/hcV2Hcr09GY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to crits-users+unsubscribe@googlegroups.com.

To post to this group, send email to crits...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/crits-users/1ddabb46-51bf-41c8-8df5-3347c0c51ad0%40googlegroups.com.