Text Annotation
Social Media Categorization for Supermajor Oil Company
The Challenge
Our client, one of the seven "supermajor" oil companies in the world, was looking for a scalable partner to review and classify nearly 2,000 tweets to diagnose the public perception of the company and utilize the insights to tailor internal and external communications. The project covered seven reputational drivers including products and services, innovation, workplace, integrity and transparency, citizenship, leadership, and performance. The main challenge was handling highly subjective content and displaying the findings to ensure our client could make actionable decisions based on the uncovered opinion-based insights.
• • • •The Solution• • • •
Utilizing our global community database of more than 1.3 million contributors, DataForce quickly sourced and trained a team of 11 with the relevant skills and background in a two-week timeframe. A dedicated project manager and coordinator were assigned to ensure the timeline, training, budget, and all other particulars were met along with seven annotators and two reviewers for the classification portion. Each tweet was assigned through our proprietary annotation platform to two different annotators to ensure there were multiple perceptions per tweet. In the case of a disagreement, the results were then allocated to a QA reviewer. The QA reviewer would disambiguate the initial categorizations and decipher the tweet alongside the annotator’s viewpoint to determine categorical placement based on guidelines received by our client.
Given the high subjectivity of the task, we made sure to be on target by categorizing and delivering a fraction of the dataset first, which included 250 tweets. After completing the pilot and discussing the outcome with the client, we further fine-tuned the guidelines and specified training instructions before going into the full production of 1,926 tweets. It is important to note that this review of the first 250 tweets provided a mutual opportunity to dissect the guidelines and reassess the categorization breakdown, providing an opportunity to alter the project early on so our client could obtain the results they desired in the end.
Our client not only received the final data annotated within eight weeks of project kickoff, but they also had the ability to view the results in a simple way, thanks to a customized dashboard we built catered to their preferences.
Following the final delivery, we received excellent feedback from our client noting the measurable shift in the performance of their model thanks to the labeled and classified dataset. We were proud to report a 100% acceptance score, indicating our client was fully satisfied with the quality of the dataset results.
DataForce has a global community of over 1,000,000 members from around the globe and linguistic experts in over 250 languages. DataForce is its own platform but can also use client or third-party tools. This way, your data is always under control.