From the source
OpenAI announced a classifier it says can better detect abusive imagery of children, including whether it is AI-generated, and is offering vetted organizations free access to it.
- The tool is meant to help investigators cut through a flood of reports that has threatened to overwhelm systems like NCMEC's CyberTipline.
- OpenAI says the classifier flags 48 percent fewer images as false positives, while the images it did identify were twice as likely to contain abusive material.
- It is the first broadly available CSAM classifier with a specific flag for AI-generated material, according to OpenAI's head of child safety public policy, John Buckley.
- ChatGPT users who attempt to produce CSAM will now see a warning, and a first offense triggers a 24-hour pause in access to ChatGPT with a notice that the account could be permanently closed.
- OpenAI says it reports all instances of CSAM to NCMEC, and North Carolina Attorney General Jeff Jackson called the partnership with a tech company to deter creation of synthetic CSAM "still basically an experiment."