Serving a string based model via Rust. How do I call the tf.keras.preprocessing.text.Tokenizer?

[MTRNord]

Hi :) I am currently building a small Model to predict some text if it is spam or not and want to build a high level api in rust with it.

In python to invoke/predict things I called via this piece of code: https://github.com/MTRNord/matrix-spam-ml/blob/main/model_v2.py#L241-L247 however in Rust I can't seem to find how I am supposed to tokenize my strings to get the data I can pass into my pretrained model.

What am I missing? :) Or is there a better way of solving this?

----
MTRNord

[MTRNord]

Alternatively the TextVectorization would exist but that leads for me to the same question of how I am supposed to use it from within rust to give my model the correct input data.

[MTRNord]

I solved it now by using the TextVectorizationlayer and using strings on the rust and python side as inputs :)