Using fast text to compute word similarities on the fly

[Pranav Lal]

  Hi all,

I want to use fast text in my chat bot. I have a list of questions and answers from stackoverflow. These are already in a csv file.

Here is the algorithm I plan to use for the bot.

1. I store the tokenized questions.

2. I capture the question from the user and tokenize it.

3. I hen use key words to narrow the list of questions I need to search.

4. I then use text similarity to find all the questions similar to the question entered by the user.

5. I assemble the answer.

I have already implemented this in spacy but want to try fast text because I suspect it may be a faster library. How do I calculate text similarity? I understand how to do the rest.

Pranav

[avi...@mavenick.co]

For text similarity you can use fuzzy match there is one python library called "fuzzywuzzy", using that you can get similarity between 2 string. It uses Levenshtein distance .

[eduard...@yahoo.com.br]

When you say text similarity you means sentence similarity? With fastText you can tokenize, classify (including language detection), find word/Ngram similarities, but you need more to answer questions. I don`t know if we can indicate a link here.