[Python] how do I optimise my semantic similarity check to detect bot like responses to an...

Currently I'm using tf-idf vectoriser to check for semantic similarity, but this is not working too well because responses like 'sssss' 'hkfjhwekhwke' are not being detected as bot responses.

# Semantic Similarity Check
try:
tfidf = TfidfVectorizer().fit([text, question_text])
vecs = tfidf.transform([text, question_text])
similarity = cosine_similarity(vecs[0], vecs[1])[0][0]
if similarity < 0.1:
result["similarity_flag"] = 1
except:
pass

return result

Continue reading...