Inside the Mind of a Twitter Reviewer
Using big data to understand what consumers are saying about a large telecommunications firm
Marketing as a business practice has been taught in higher education since at least the early 1900s. Even before then, companies had been interested in presenting themselves to consumers in a positive light, and that interest continues today as the global marketing industry is now worth an estimated $1.7 trillion. Companies around the world often ask, “How can we appeal to consumers?”, but perhaps less often, they ask, “What do consumers think of us?”
Surely companies have surveyed their consumer bases and have asked this question before, but survey data is typically structured and might not necessarily reflect consumers’ opinions entirely. There is a real difference between consumers’ ratings of a company on a scale of one to five and consumers’ responses to open-ended questions. The sample size of a survey also calls into question the efficacy of the survey itself – does a survey of a few thousand really reflect the viewpoints of hundreds of thousands, or even millions? Most would be inclined to answer “no”, which means the question moves from “What do consumers think of us?” to “How do we find out what consumers think of us?”.
A large U.S. telecommunications company (telco) has been wondering exactly that, and OGC is providing the answer. As the world becomes increasingly interconnected, opportunities for companies to leverage that interconnectedness have risen, with Twitter’s presenting itself as one such opportunity for OGC and this telco. Part of what makes Twitter a popular social media platform is the largely unstructured way in which people can express themselves about quite nearly anything, including this telco. Rather than having to prompt consumers for their opinions, the company can simply turn to Twitter for them.
To enable this, OGC has built a data pipeline that originates with Twitter, listens in on what users are saying, and filters on specific Tweets that include the name of the telco. These Tweets are then transferred from the Twitter end of the pipeline to an internal OGC database. The challenge for OGC at this point is no longer in obtaining individual opinions about the telco; instead, we now have tens of thousands of individual opinions that need to be aggregated and understood efficiently.
This process of aggregation and digestion starts with an exploratory data analysis. Although exploratory data analyses are generally thought of as a primer for more sophisticated modeling techniques, these analyses still yield a decent amount of information about the underlying structure and patterning in text data. The format in which Twitter supplies its data is partly what makes this possible. Each Tweet that has been pulled from the site and resides in OGC’s database comes with an accompanying username, a timestamp, and if provided by the user, a geolocation. To take things a step further, OGC has also meticulously labeled these Tweets as expressing positive, neutral, or negative sentiment towards the telco. Since consumer sentiment is really the crux of the overall analysis, the majority of the exploratory portion focused on better understanding the nature of these three different types of sentiments.
First, we took a look at how sentiment towards this telco changed over the course of a day and over the course of a week. With respect to intraday sentiment, we found that while negative and neutral sentiment did not change dramatically, positive sentiment increased by about 17.2% when moving from the morning to the afternoon. We found similar patterning in the rates of positive sentiment throughout the course of the week. As can be seen in the chart below, daily positive sentiment frequency decreases steadily throughout the workweek but then spikes dramatically come the weekend.
These temporal insights are valuable from both an operations perspective and a marketing perspective. On the operations side, this telco can staff the customer service parts of its business depending upon shifts in sentiment. For example, positive sentiment on Monday afternoons is markedly higher than it is on Friday mornings, so customer service staffing could be reduced on Monday afternoons and increased on Friday mornings. When it comes to marketing, these insights can help guide the timing and content of social media posts. During the workweek, social media initiatives can focus on increasing positive sentiment, whereas on the weekend, the uptick in positive sentiment can be funneled towards new initiatives.
We also examined the word lengths of the Tweets, which revealed that there are two distinct types of positive sentiment expressed towards this telco. Shorter positive Tweets are often enterprise-level and tend to reflect excitement about a new product offering or satisfaction with COVID assistance. Longer positive Tweets, on the other hand, are mainly consumer-level and involve interactions with technicians or customer service representatives. The former serve as immediate feedback about recent business decisions that can be used to inform upon future decisions, and the latter represent a sort of gold standard for this telco’s customer relations.
Unique terms were counted, too, and the most notable takeaway from these counts was the way in which COVID-19 was framed in positive and negative Tweets. Positive Tweets used the term “COVID”, which carries a more scientific connotation than “pandemic”, the term used in negative Tweets. This disparity in term choice suggests that there is an opportunity for this telco to better connect with its consumer base by using language that closely aligns the business with the plight of its customers.
The last part of exploratory data analysis looked at state-level sentiment towards the telco. The map of the contiguous United States below depicts that sentiment with a color scale ranging from -1, represented by dark red and indicating entirely negative sentiment, to +1, represented by dark green and indicating entirely positive sentiment (States from which no Tweets came were colored gray). There are clearly pockets of positive, neutral, and negative sentiment dispersed throughout the country. These pockets can be used to understand regional variances in this telco’s service, whether they be in the form of the telco’s infrastructure, pricing, or customer service quality.
While these insights on their own have considerable value, there is still plenty more to come. We at OGC are currently implementing artificial intelligence algorithms to predict the sentiment of Tweets in real time. Once that implementation is complete, this telco will have an ideal and immediate look into consumer opinion that can be used to guide future business decisions in marketing, strategy, and operations.
Kevin Schattin, Data Scientist