Processing open ended consumer responses used to be time consuming. The usual process involved categorizing the response into several categories and then summarize the themes that emerge. Natural language processing (NLP) libraries can help with the task of analyzing open ended responses to assess consumer sentiment, aspects of the experience consumers are talking about the most etc. The graph below shows one example of such an analysis that was done with reviews of a retailer’s store in the United States. The reviews span the years \(2005 – 2018\) and were extracted using Python’s BeautifulSoup package and processed with TextBlob (a NLP Python package).
The sentiment score returned by TextBlob ranges from \(-1\) to \(1\) with positive values representing positive sentiment and negative values representing negative sentiment. The graph shows that at this particular store consumer sentiment is positive and has been increasing over the past 8 years. The grey part of the graph represents \(95\% \) confidence intervals which shows the range of plausible values for the true value of sentiment scores.
We can analyze the data further to see if the trend is real by an analysis of word counts, adjectives and nouns used in the reviews and by comparing the sentiment scores of a few sample reviews with human generated sentiment scores. In any case, machine learning techniques such as NLP are powerful and firms should consider how to incorporate such techniques in their insights workflow.