I am working with a dataset of about ~900 datapoints w.r.t survey responses to an open ended satisfaction survey. I have some background in java, but after doing some research, I have found that using python and nlp/topic modelling would be the way to proceed if I want to categorise this data. A big hurdle I am facing is that the data is extremely heterogeneous being collected and transcribed by multiple different interviewers who have different methods of noting down what the respondents wrote. Some have the survey identity number mentioned in the transcript, some do not, some are in first person and some in third person, multiple responses have spelling mistakes etc. In such a situation, if I run the usual Bert classifier and get the vector embeddings, would it even be meaningful? I'm assuming I have to convert the data into some homogenous form following which I can do any analysis. My ultimate goal is to find some common topics among the suggestions given by the respondents and classify them into clusters which I will manually look at. After obtaining these standout topic, I assume I will have to manually label some data and do some testing/training but I am a little fuzzy on the exact process and would appreciate any help. Thanks! Continue reading...
