[Python] Extracting Synonyms for a List of Terms from a Set of Natural Language Questions

I am currently working on a natural language processing (NLP) task where I have two distinct datasets: - A list of key terms (e.g., ["price", "delivery", "support"])

• A list of user-generated questions or sentences (e.g., ["What is the cost of the product?", "When will it be shipped?", "Can I get help with setup?"])

My goal is to identify synonyms or semantically similar words/phrases for each term by analyzing how related concepts are expressed in the questions.

I want to automatically extract context-specific synonyms or similar expressions for each term based solely on the usage in the questions.

For example:

For the term "price", I’d expect synonyms like "cost" or "charge" if those appear in relevant questions.

For "delivery", words like "shipped" or "arrive" might be suitable.

For "support", words such as "help", "assistance", or "setup" could be good matches.

However, traditional static synonym lists or dictionaries (like WordNet) are not sufficient because they don’t account for contextual usage in the actual questions.

Challenges Some challenges I’m facing:

• Contextual Variance: The same word can mean different things in different questions (e.g., “charge” can refer to money or electricity).
• Non-literal Synonyms: Some expressions (e.g., “get help”) are not direct synonyms but are conceptually similar.
• Multi-word Phrases: Many relevant terms are multi-word expressions, not just single tokens.
• Scalability: I want a solution that could scale across a large number of terms and questions.

I’m looking for suggestions, methodologies, or code-level examples that can help me:

• Automatically extract synonyms or related phrases for each term from the question corpus
• Ensure the results are context-aware, not just based on static dictionary lookups

Any guidance or ideas would be greatly appreciated! If you've tackled something similar or can suggest an efficient approach (rule-based, ML-based, or statistical techniques) I’d love to hear your thoughts.

Thanks in advance!

I have tried to identify the semantic words via embedding. But this is not working as expected.

Continue reading...