Empowering Safer AI with IBM
“The DataForce team collaborated with the IBM team to fine-tune their annotation techniques as model development progressed. This constant process of refining through feedback was critical in selecting the training dataset in the most effective and efficient way.” - Prasanna Sattigeri, Principal Research Scientist & Manager, IBM
The Challenge
Our client, an international technology company, was looking for a partner to assist in a large conversational speech data collection project in the gaming domain. Toxic speech is a growing concern in today's society, as hate speech, online harassment, and other verbal attacks are on the rise, especially in forums that children utilize. The goal was to collect a minimum of 100 hours of highly toxic speech with 100 avid gamers.
• • • •The Solution• • • •
Data Force partnered with IBM to deliver a comprehensive and diverse dataset tailored to the needs of Granite Guardian 3.0. This effort involved:
Content Generation:
DataForce generated a wide range of complicated scenarios from scratch across different risk dimensions. This strategy ensured the dataset provided IBM with edge cases to test the limits of the model’s detection capabilities.
The collaboration enabled IBM to significantly enhance Granite Guardian 3.0’s ability to detect harmful prompts, including subtle and nuanced cases. The model also outperformed standard benchmarks for hallucination, harmful content, and social bias, setting a new standard for AI safety. The LLM now demonstrates improved precision in identifying unsafe content, reducing the risk of responding to inappropriate inputs and ensuring safer interactions. IBM recognized DataForce’s adaptability and expertise in delivering tailored datasets to meet complex AI challenges. "Working with IBM on this groundbreaking project was a pleasure. Their focus on creating enterprise-grade, ethical AI models showcases their leadership in the field, and we are honored to have played a role in this achievement." - Kris Perez, Director, AI, DataForce