Implement LLM pre-annotations

[Jacob Beck]

Dear all,

we are currently designing a study in which humans are validating a set of LLM pre-annotations in PDF documents. Is there any possibility in INCEpTION to "upload" these pre-annotations so that the human annotator can adjudicate them?

Grateful for any response!

Jacob

[Richard Eckart de Castilho]

Hi Jacob,

> On 28. Mar 2024, at 16:52, Jacob Beck <jacob...@stat.uni-muenchen.de> wrote:
>
> we are currently designing a study in which humans are validating a set of LLM pre-annotations in PDF documents. Is there any possibility in INCEpTION to "upload" these pre-annotations so that the human annotator can adjudicate them?

Here is what you can do:

* Enable the remote API
* Create a project
* Add annotators to the project
* Define your annotation scheme
* Import a PDF document into the project (via UI or remote API)
* Export the document as JSON CAS (or XMI CAS XML 1.0) (via UI or remote API)
* Use DKPro Cassis to load the CAS file, to add your annotations to it and then save it again
* Use the remote API to upload the processed CAS file as annotations of the individual annotators

If you access a document as one of the annotators now, you should be able to see your annotations.

Note that if an annotator "resets" a document, it will not reset to the pre-processed state but rather to the state without any annotations as all.

Cheers,

-- Richard

[Jacob Beck]

Thank you very much for your response!