NER annotations disappeared (multi-user)

[Kamran Ibiyev]

Hi there,

I created 2 new accounts and granted them the annotator role. They annotated almost 20k sentences and on the next day when we check we see that approximately 90% of the annotations disappeared. 

What can be the reason? They annotate and on the same day they see their changes, but after a while, it disappears.

I use the university server and reach the UI via Putty SSH Tunneling.

I am using INCEpTION -- 0.17.1.

Your suggestions or any info are highly appreciated.

Thanks,

Kamran

[Richard Eckart de Castilho]

Hi Kamran,

such a thing should not happen. I am not aware of a bug that could cause
such a problem. That said, software is never free of bugs and safeguards
on the side of the user are always well advised.

Did you set up the internal backup functionality in INCEpTION?

Do you perform an external backup of exception or regularly export your project?

Do you keep the server logs somewhere save and do you possibly see anything that
might look remotely relevant in the server logs?

Does the problem affect only the two new accounts or are all your users affected?

Best regards,

-- Richard

[Kamran Ibiyev]

Hi Richard,

We used the embedded database driver and we don't have an automatic/regular backup.

In the server logs of activities of the users, everything seems fine.  But we don't have the error logs. And the machine didn't crash for a long time.

Both users were affected.

If we set up a backup interval, would it take a backup if there are no changes in an annotation?

If it is possible to import the annotations and do the curation in Inception?

Thanks,

Kamran

[Kamran Ibiyev]

Dear Richard,

In the logs, we can see that those users worked, but we cannot find the work they did (90%).

Is it possible that those changes are stored somewhere and can be reachable?

Thank you for your time.
Kamran

[Richard Eckart de Castilho]

Hi,

> On 16. Feb 2021, at 14:37, Kamran Ibiyev <kamran...@gmail.com> wrote:
>
> We used the embedded database driver and we don't have an automatic/regular backup.

Please do not use the embedded DB driver except when toying around. There should be warning shown in the browser on the bottom-right side if the embedded DB driver is used. We have had reports from people whose server had crashed e.g. due to power loss and they ended up with a corrupted embedded DB and lost all their metadata (user logins, project setups, etc.).

That said, you seem to be having a different issue because the annotations themselves are not stored in the SQL database.

> If we set up a backup interval, would it take a backup if there are no changes in an annotation?

The backup is made if at the time of saving/updating the annotations the interval conditions are met.
The internal backup is an imperfect solution but a practical one.

To ensure your data is safe, it is a good idea to regularly export the data and / or to keep system-level backups of the INCEpTION data folder and the contents of the SQL database.

> If it is possible to import the annotations and do the curation in Inception?

In principle, you can import annotations to individual user accounts via the remote API.

Did you lose curation data or data from the two individual annotators?

Best,

-- Richard

[Richard Eckart de Castilho]

On 16. Feb 2021, at 14:40, Kamran Ibiyev <kamran...@gmail.com> wrote:
>
> In the logs, we can see that those users worked, but we cannot find the work they did (90%).
> Is it possible that those changes are stored somewhere and can be reachable?

The annotations are stored in files within the directory hierarchy of the INCEpTION data folder under "repository/project/<PROJECT-ID>/document/<DOC-ID>/annotation/<USERNAME>.ser".
The format is a binary Apache UIMA data format.

If you have the internal backup enabled, then several snapshots of these files are kept - so if you should identify data loss, you may be able to replace the <USERNAME>.ser file with a previous revision.

If you export the project, there is an event log file in archive which contains information about the
actions that the users did. In *theory*, this file should contain sufficient information to re-create
most of the data. However, reconstruction is not the purpose for keeping this event log and we do not
actually have any tooling that would read the event log and try using this information to re-create lost
annotations.

Best,

-- Richard

[Jan-Christoph Klie]

Hi,

we are sorry for your loss. Can you maybe share an export of the project as XMI and the .inception folder with us? Then I will have a look. You can send a link per mail to klie...@gmail.com . I do not have that much hope though. INCEpTION 0.17.1 is also not the newest version, so maybe you want to upgrade.

Best,

Jan

[Attila Novák]

Dear Richard and Jan,

I'm Kamran's colleague. We are new to INCEpTION.

The two users worked for 4-5 days. The funny thing is that we can see some of what they did in the middle, but the parts of the corpus they annotated at the beginning and the at end of their work seem to be untouched.

Unfortunately, we do not have backups of the .ser files. Now I have turned on the automatic backup feature. 

Neither do we have the stderr output of the application. 

However, reconstructing what they did from the logfiles does not seem to be impossible...

Jan, thanks for looking at the issue. I created an XMI export. I'll mail the link to you.

Thanks again,

Attila