Serving a string based model via Rust. How do I call the tf.keras.preprocessing.text.Tokenizer?

56 views
Skip to first unread message

MTRNord

unread,
Sep 26, 2022, 3:44:03 PMSep 26
to Rust for TensorFlow
Hi :) I am currently building a small Model to predict some text if it is spam or not and want to build a high level api in rust with it.

In python to invoke/predict things I called via this piece of code: https://github.com/MTRNord/matrix-spam-ml/blob/main/model_v2.py#L241-L247 however in Rust I can't seem to find how I am supposed to tokenize my strings to get the data I can pass into my pretrained model.

What am I missing? :) Or is there a better way of solving this?

----
MTRNord

MTRNord

unread,
Sep 26, 2022, 4:04:58 PMSep 26
to Rust for TensorFlow, MTRNord
Alternatively the TextVectorization would exist but that leads for me to the same question of how I am supposed to use it from within rust to give my model the correct input data.

MTRNord

unread,
Sep 27, 2022, 2:22:29 PMSep 27
to Rust for TensorFlow, MTRNord

I solved it now by using the TextVectorizationlayer and using strings on the rust and python side as inputs :)
Reply all
Reply to author
Forward
0 new messages