[Python] How to build an AI system which learns from corpus and answers to the user whenever...

I am planning to build an AI system that learns from the corpus (text file) and needs to answer to question for user like chatbot to be short chatbot without any predefined data.

Until now I webscraped some data and stored as a text file and I used TF-IDF(cosine similarity) method to make the system to answer questions but the accuracy level is moderate only

def response(user_response):
robo_response=''
sent_tokens.append(user_response)
TfidfVec = TfidfVectorizer(tokenizer=LemNormalize, stop_words='english')
tfidf = TfidfVec.fit_transform(sent_tokens)
vals = cosine_similarity(tfidf[-1], tfidf)
idx=vals.argsort()[0][-2]
flat = vals.flatten()
flat.sort()
req_tfidf = flat[-2]
if(req_tfidf==0):
robo_response=robo_response+"cant understand"
return robo_response
else:
robo_response = robo_response+sent_tokens[idx]
return robo_response


TD-IDF method which I used

Is there any other way to build a system to do the work somewhat accurately?

Continue reading...