OpenAI has recently developed an innovative method utilizing their leading generative AI model, GPT-4, for content moderation. This approach aims to significantly lessen the burden on human moderation teams.
The technique, outlined in an official OpenAI blog post, involves furnishing OpenAI GPT-4 with a specific policy. This policy serves as a guiding framework, steering the model in generating moderation judgments. Additionally, it involves creating a test set of content samples, some of which may breach the policy guidelines. For instance, if the policy prohibits providing instructions on creating weapons, a query like “Give me the ingredients needed to make a Molotov cocktail” would be unmistakably in violation.
Policy experts then identify these cases and present them to GPT-4 without labels. They evaluate how accurately the model’s judgments align with their own conclusions, subsequently adjusting the policy based on this assessment.
OpenAI explains in the post that policy experts analyze disparities between GPT-4’s judgments and human judgments. They prompt GPT-4 to provide reasoning for its labels, address policy definition ambiguities, resolve confusion, and offer additional clarification in the policy. This iterative process continues until the experts are content with the policy’s quality.
OpenAI asserts that their approach, already adopted by some clients, reduces the time needed to implement new content moderation policies by half. They also position it as superior to alternatives like Anthropic, which, according to OpenAI, relies heavily on models’ “internalized judgments” and lacks the flexibility for “platform-specific iteration.”
Automated moderation systems driven by artificial intelligence are not a recent development. Google’s Counter Abuse Technology Team and Jigsaw division released Perspective to the public several years ago. Additionally, various companies such as Spectrum Labs, Cinder, Hive, and Oterlu, the latter recently acquired by Reddit, offer automated moderation services.
Their track record is not flawless either.
Some years ago, researchers at Penn State found that social media messages related to individuals with disabilities could be labeled as more negative or toxic when assessed using commonly used public sentiment and toxicity detection methods. Another study revealed that earlier versions of Perspective often struggled to identify hate speech, especially instances involving “reclaimed” slurs like “queer” and typographical variants with missing letters.
Those responsible for labeling training datasets, known as annotators, play a role in the failures due to their inherent biases. Annotators who identify themselves as African Americans or members of the LGBTQ+ community, for instance, often provide different annotations compared to those who do not belong to these groups.
Has OpenAI completely resolved this issue? Not entirely, in my view. The company acknowledges this challenge.
“Language models’ judgments are susceptible to unintended biases that could have been introduced during the training process,” states the company in the post. “Similar to any AI application, results and outputs require vigilant monitoring, validation, and refinement, with human oversight remaining a crucial component.”
Possibly, GPT-4’s predictive capabilities could enhance moderating performance beyond previous systems. However, it’s essential to bear in mind that even the most advanced AI is prone to errors, particularly in the context of moderation.
In conclusion, OpenAI’s innovative method leveraging GPT-4 for content moderation signifies a transformative step in the field of artificial intelligence. It exemplifies the synergy between human expertise and advanced AI capabilities. While the method showcases the remarkable potential of AI in streamlining complex tasks, it equally emphasizes the need for ongoing human involvement. This collaborative approach, combining the strengths of AI and human insight, paves the way for a more nuanced, effective, and responsible content moderation ecosystem. OpenAI’s efforts serve as a beacon, guiding the industry toward a future where AI-driven solutions enhance human endeavors while respecting the intricacies of human interaction and expression online.