running the annif models developed for the GermEval LLMs4Subjects shared task

[Jennifer D'Souza]

Dear annif community,

I have plans to test the models released on huggingface here https://huggingface.co/NatLibFi/Annif-LLMs4Subjects-GermEval2025-data . Some of these model produced results that got Annif the top place of the GermEval LLMs4Subjects leaderboard.

I want to test these models out myself. However, I realize that the backends used are newly developed(?). I was browsing through the project config to see the entire list https://github.com/NatLibFi/Annif-LLMs4Subjects-GermEval2025/blob/main/projects.toml . I am assuming they are new because I could not find installation instructions for them except one i.e. omikuji. I did see this documentation already https://github.com/NatLibFi/Annif/wiki/Optional-features-and-dependencies. So I opened three related issues on the repo https://github.com/NatLibFi/Annif-LLMs4Subjects-GermEval2025/issues . My main ask here is how to install the relevant backends to be able to run these new SOTA models from huggingface? Also what's the best way to go on windows... docker?

Thanks in advance for any support that can be provided here.

Regards,

Jennifer

[Osma Suominen]

Hi Jennifer!

Thanks for testing our system and sorry that it's not that well
documented - we focused on the experiment and the competition, not so
much on documenting the code.

I already responded to each of the three issues you opened, so it's
probably easiest to continue discussion about the specifics over there.

Annif doesn't work that well on Windows (as you've found out) so I think
your best bet is either to use Linux (perhaps in a VM) or to try the
Docker images. However, it might be a bit tricky to include the
experimental backends (XTransformer, LLM-ensemble) in a Docker image as
you will probably have to modify the Dockerfile to include them as well
and build your own image.

Hope this helps!

-Osma

On 07/01/2026 13:49, Jennifer D'Souza wrote:
> Dear annif community,
>
> I have plans to test the models released on huggingface here https://

> huggingface.co/NatLibFi/Annif-LLMs4Subjects-GermEval2025-data <https://
> huggingface.co/NatLibFi/Annif-LLMs4Subjects-GermEval2025-data> . Some of

> these model produced results that got Annif the top place of the
> GermEval LLMs4Subjects leaderboard.
>
> I want to test these models out myself. However, I realize that the
> backends used are newly developed(?). I was browsing through the project
> config to see the entire list https://github.com/NatLibFi/Annif-

> LLMs4Subjects-GermEval2025/blob/main/projects.toml <https://github.com/
> NatLibFi/Annif-LLMs4Subjects-GermEval2025/blob/main/projects.toml> . I

> am assuming they are new because I could not find installation
> instructions for them except one i.e. omikuji. I did see this
> documentation already https://github.com/NatLibFi/Annif/wiki/Optional-

> features-and-dependencies <https://github.com/NatLibFi/Annif/wiki/
> Optional-features-and-dependencies>. So I opened three related issues on
> the repo https://github.com/NatLibFi/Annif-LLMs4Subjects-GermEval2025/
> issues <https://github.com/NatLibFi/Annif-LLMs4Subjects-GermEval2025/
> issues> . My main ask here is how to install the relevant backends to be

> able to run these new SOTA models from huggingface? Also what's the best
> way to go on windows... docker?
>
> Thanks in advance for any support that can be provided here.
>
> Regards,
> Jennifer
>

> --
> You received this message because you are subscribed to the Google
> Groups "Annif Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to annif-users...@googlegroups.com <mailto:annif-
> users+un...@googlegroups.com>.
> To view this discussion visit https://groups.google.com/d/msgid/annif-
> users/62c966c8-e526-4412-9db7-56ef4d722147n%40googlegroups.com <https://
> groups.google.com/d/msgid/annif-users/62c966c8-
> e526-4412-9db7-56ef4d722147n%40googlegroups.com?
> utm_medium=email&utm_source=footer>.

--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 15 (Unioninkatu 36)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.s...@helsinki.fi
http://www.nationallibrary.fi

[Jennifer D'Souza]

Hi Osma,

thanks a lot for the instructions within the individual issues. I will try them out in the next days and hope to document my experience as I go along.

In the meantime, I documented the working out-of-box annif backends for the SOTA models at LLMs4Subjects@GermEval here https://github.com/jd-coderepos/annif-llms4subjects-germEval2025-docker . This specifically for Windows using the annif docker containers. Please let me know if you have any feedback.

Best regards,

Jennifer

You received this message because you are subscribed to a topic in the Google Groups "Annif Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/annif-users/h078BPDMALg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to annif-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/annif-users/8b37d4d3-cc4b-45cb-a0f8-f13ea7e5ecc9%40helsinki.fi.