NN Ensemble - Wraning

[Parthasarathi Mukhopadhyay]

Dear all

I was trying to understand the differences in results in TFIDF, Omikuji & NN_Ensemble (sources are TFIDF & Omikuji) on the basis of the same vocabulary backend (LCSH) and same training & test datasets.

In case of NN_ensemble, the train and learn command for a small training data set of 5000 records always giving me this warning :

/annif/annif-venv/lib/python3.8/site-packages/keras/engine/functional.py:1410: CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
  layer_config = serialize_layer_fn(layer)

I then added the  line layer_config = serialize_layer_fn(layer) in the NN_ensemble section of the project configuration (desperately :))  but with no effect.

But this is a harmless waring as the call for suggestion from the nn_ensemble project is working (but again with a few warnings) -

Loaded model with 3 trees; it took 5.12s
2022-08-14 12:04:59.413076: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-08-14 12:04:59.414800: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 402488000 exceeds 10% of free system memory.
2022-08-14 12:04:59.520588: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 402488000 exceeds 10% of free system memory.
2022-08-14 12:04:59.546821: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 402488000 exceeds 10% of free system memory.
2022-08-14 12:04:59.741520: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 402488000 exceeds 10% of free system memory.
2022-08-14 12:04:59.920718: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 402488000 exceeds 10% of free system memory.
<http://id.loc.gov/authorities/subjects/sh85138279> Truth 0.3579663038253784
<http://id.loc.gov/authorities/subjects/sh85138284> Truth--Religious aspects--Christianity 0.34840771555900574
<http://id.loc.gov/authorities/subjects/sh85105557> Postmodernism 0.34473705291748047
<http://id.loc.gov/authorities/subjects/sh2010117193> Truth--Congresses 0.3438728153705597
<http://id.loc.gov/authorities/subjects/sh2010117194> Truth--History 0.3374820053577423
<http://id.loc.gov/authorities/subjects/sh85067647> Introspection--Religious aspects 0.3357619643211365
<http://id.loc.gov/authorities/subjects/sh2010115712> Truthfulness and falsehood--Fiction 0.33320915699005127
<http://id.loc.gov/authorities/subjects/sh2009008144> British newspapers--Headlines 0.3263297379016876
<http://id.loc.gov/authorities/subjects/sh2001002255> Truth commissions 0.3221234381198883
<http://id.loc.gov/authorities/subjects/sh85108424> Psychoanalytic interpretation 0.31999868154525757

My NN_ensemble config is -

[lcsh-nn-ensemble-en]
name=LCSH NN ensemble English
language=en
backend=nn_ensemble
sources=lcsh-tfidf-en,lcsh-omikuji-parabel-en
limit=100
vocab=lcsh-en
nodes=100
dropout_rate=0.2
epochs=10
lmdb_map_size=2147483648

How to stop these warnings? Nobody likes warnings :)

And if anyone here could point me to some published works on comparative studies of results from different language models and backend algorithms, I'd be grateful.

Best regards

Parthasarathi Mukhopadhyay Professor, Department of Library and Information Science, University of Kalyani, Kalyani - 741 235 (WB), India

[Osma Suominen]

Hi Parthasarathi!

I think we've seen this CustomMaskWarning before. In my understanding,
it was a bug in TensorFlow, fixed in TF 2.8. Which version of Annif are
you using? The most recent release Annif 0.58 upgraded TensorFlow to
version 2.9, which should have fixed this warning as well.

Here are some related links:

The warning was noted in this Annif PR which fixed other warnings:
https://github.com/NatLibFi/Annif/pull/575

It's been discussed on StackOverflow:
https://stackoverflow.com/questions/69590274/warning-custom-mask-layers-require-a-config-and-must-override-when-saving-the/70268806#70268806

Upgrading Annif and TensorFlow with it should get rid of the warning.
But in my understanding it is pretty harmless.

Best,
Osma

Parthasarathi Mukhopadhyay kirjoitti 14.8.2022 klo 9.48:
> Dear all
>
> I was trying to understand the differences in results in TFIDF, Omikuji
> & NN_Ensemble (sources are TFIDF & Omikuji) on the basis of the
> same vocabulary backend (LCSH) and same training & test datasets.
>
> In case of NN_ensemble, the train and learn command for a small
> training data set of 5000 records always giving me this warning :
>
> /annif/annif-venv/lib/python3.8/site-packages/keras/engine/functional.py:1410:
> CustomMaskWarning: Custom mask layers require a config and must override
> get_config. When loading, the custom mask layer must be passed to the
> custom_objects argument.
>   layer_config = serialize_layer_fn(layer)
>

> I then added the  line /layer_config = serialize_layer_fn(layer) /in the

> Parthasarathi Mukhopadhyay
>
> Professor, Department of Library and Information Science,
>
> University of Kalyani, Kalyani - 741 235 (WB), India
>

> --
> You received this message because you are subscribed to the Google
> Groups "Annif Users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to annif-users...@googlegroups.com
> <mailto:annif-users...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/annif-users/CAGM_5ubesS-4qnZTguM3GLaqQybsoFDK7sp7mGjbVsqkpLv4Kw%40mail.gmail.com
> <https://groups.google.com/d/msgid/annif-users/CAGM_5ubesS-4qnZTguM3GLaqQybsoFDK7sp7mGjbVsqkpLv4Kw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
Osma Suominen
D.Sc. (Tech), Information Systems Specialist
National Library of Finland
P.O. Box 15 (Unioninkatu 36)
00014 HELSINGIN YLIOPISTO
Tel. +358 50 3199529
osma.s...@helsinki.fi
http://www.nationallibrary.fi

[Parthasarathi Mukhopadhyay]

Thanks Osma.

I'm presently using Annif version 0.57.  I'm glad to know that it will not be there with the upgrade to 0.58.

Regards

To unsubscribe from this group and stop receiving emails from it, send an email to annif-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/annif-users/ee0e56c8-e2ed-1d10-4489-94b15e2d82a0%40helsinki.fi.