Discord bans teen relationship servers and the sharing of AI-generated CSAM

Discord has updated its coverage meant to guard youngsters and youths on its platform after reports got here out that predators have been using the app to create and unfold baby sexual abuse supplies (CSAM), in addition to to groom younger teenagers. The platform now explicitly prohibits AI-generated photorealistic CSAM. As The Washington Post not too long ago reported, the rise in generative AI has additionally led to the explosion of lifelike photographs with sexual depictions of kids. The publication had seen conversations about the usage of Midjourney — a text-to-image generative AI on Discord — to create inappropriate photographs of kids.

Along with banning AI-generated CSAM, Discord now additionally explicitly prohibits every other form of textual content or media content material that sexualizes youngsters. The platform has banned teen relationship servers, as nicely, and has vowed to take motion towards customers participating on this habits. A earlier NBC News investigation discovered Discord servers marketed as teen relationship servers with contributors that solicited nude photographs from minors. 

Grownup customers had beforehand been prosecuted for grooming youngsters on Discord, and there are even crime rings extorting underage customers to ship sexual photographs of themselves. Banning teen relationship servers fully may assist mitigate the problem. Discord has additionally included a line in its coverage, which states that older teenagers discovered to be grooming youthful teenagers will probably be “reviewed and actioned underneath [its] Inappropriate Sexual Conduct with Kids and Grooming Coverage.”

Other than updating its guidelines, Discord not too long ago launched a Family Center tool that folks can use to regulate their youngsters’ exercise on the chat service. Whereas dad and mom will not be capable to see the precise contents of their youngsters’ message, the opt-in software permits them to see who their youngsters are mates with and who they discuss to on the platform. Discord is hoping that these new measures and instruments may also help preserve its underage customers protected together with its outdated measures, which embrace proactively scanning photographs uploaded to its platform utilizing PhotoDNA.