Using Anomaly Detector for other type of text

[OC]

Hi,

Great work here, I was wondering if Anomaly Detector could be used in my case.

I have a system where people write a message of 10 to 80 characters daily.

I would like to be able to test the anomaly in their message based on their previous messages.

Is this something Anomaly Detector is suited for ? Assuming I have up to 100k users or more, what will the performance look like?

[Daniel Yavorovych]

Hi!

Using text-to-vector, you can work with data from your case in tools like Keras and Tensorflow.

Given the low traffic of your users ’messages, you can train staff models in the background each time such messages appear, but before that use the previous model to predict the results for the anomaly detector (considering the user model, time of day and day of the week, and other dependent factors that can be represented by separate vectors).

In other words, an anomaly detector may be suitable for your task, and the performance will be high enough to perform predict realt-time operations.

четверг, 20 февраля 2020 г., 11:00:31 UTC+2 пользователь OC написал:

[OC]

Thank you Daniel,

I was really hoping for a usable example :D

If you could point me to anything similar, it would be of great help.

[Daniel Yavorovych]

I could not find a ready-made example for your case, but here is the material that may be useful:

For make matrix from text you can use https://towardsdatascience.com/learn-word2vec-by-implementing-it-in-tensorflow-45641adaf2ac

Find anomalies using TF & Keras: https://www.curiousily.com/posts/anomaly-detection-in-time-series-with-lstms-using-keras-in-python/#sp-500-index-data

Your task is to carry out the normalization stage and obtain from the set of texts, time and other parameters - winds for the operation of the anomaly detector mechanism.

четверг, 20 февраля 2020 г., 11:57:24 UTC+2 пользователь OC написал:

[OC]

The LSTM looks quite useful.

Thank you