optimize and hyperparameters

[jo...@hassenbach.com]

Hi!

I'm training, evaluating and optimizing my own models using Annif 1.4.1 and I have observed the following two things, which I find peculiar:

1. when running the Annif command "optimize" it seems the analysis is influenced by the hyperparameters (specifically limit) set in my projects.cfg file. If I already set a limit there, the optimize command tells me it goes through all limits 1 to 15, but in reality it doesnt. It gets capped at the limit I set. Is this intended to happen?

2. after running the optimize command, I adjusted my projects.cfg config file so it has the optimal limit and threshold settings that where recommended to me. However, when now training and evaluating my new and "better" models, Annif seems to ignore the threshold I set in the config file. I can set it manually by giving it to the eval command directly, but ideally I would want Annif to read it in via the config file instead of silently ignoring it.

Maybe someone else has some thoughts on this?
Best,
Jona

[Osma Suominen]

Hi Jona!

On 20/05/2026 12:22, jo...@hassenbach.com wrote:
> Hi!
>
> I'm training, evaluating and optimizing my own models using Annif 1.4.1
> and I have observed the following two things, which I find peculiar:
>
> 1. when running the Annif command "optimize" it seems the analysis is
> influenced by the hyperparameters (specifically limit) set in my
> projects.cfg file. If I already set a limit there, the optimize command
> tells me it goes through all limits 1 to 15, but in reality it doesnt.
> It gets capped at the limit I set. Is this intended to happen?

Thanks for pointing out this. It could be seen as a bug. The projects
will internally apply the configured limit in any case, and the "annif
optimize" command can apply another (stricter/smaller) limit on top of
that but the configured limit will still have an effect.

Would you mind reporting this as an issue on GitHub? I.e. what you
observed and what you would prefer instead?

Issues can be reported here: https://github.com/NatLibFi/Annif/issues

> 2. after running the optimize command, I adjusted my projects.cfg config
> file so it has the optimal limit and threshold settings that where
> recommended to me. However, when now training and evaluating my new and
> "better" models, Annif seems to ignore the threshold I set in the config
> file. I can set it manually by giving it to the eval command directly,
> but ideally I would want Annif to read it in via the config file instead
> of silently ignoring it.

There is no configuration setting for threshold, so even if you set
something like "threshold=0.1" in projects.cfg, it won't do anything.

If you would like to have this, please open a feature request issue on
GitHub proposing such a configuration setting.

Best,
Osma


--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 15 (Unioninkatu 36)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.s...@helsinki.fi
http://www.nationallibrary.fi