Moderation Gap: How 'Harmless' Images Become Deadly for Pakistani Women

A Chayn report says moderation systems trained to detect nudity miss non-explicit images used to extort and endanger Pakistani women, with sometimes fatal results.
By Tom Whitfield
4 min readAt a glance
- Chayn's report, Explicit Harms of Non-Explicit Images, calls for consent-based moderation instead of nudity detection.
- Pakistan's DRF helpline receives approximately 250 complaints per month about online abuse.
- Two-thirds of Instagram users in Pakistan are male; over 70% of its estimated 100 million TikTok users are male.
- 14-year-old Hira Anwar was killed by her father last year over her TikTok videos.
- Meta did not respond to a request for comment.
A charity report released by Chayn, "Explicit Harms of Non-Explicit Images," argues that social media moderation systems trained to detect nudity systematically miss image-based abuse that can end in suicide, murder and femicide. One platform decision cited in the material illustrates the failure precisely: a Pakistani teenager reported a fake account created with her photo, captioned "love of my life," and the platform replied that it broke no rules.
The complainant, Maniza, a teenager from a conservative Pakistani town, had shared a photo with a man she hoped to marry. She wore jeans and a vest top, clothing she would never wear outside the family home, where she always covered her hair.
"He started getting pushy about meeting up. He threatened to send the photo to my family," she said. "When I reported it, they said it didn't break any rules. They didn't understand my family would kill me if they saw it."
Why does the content stay up?
Hera Hussain, chief executive of Chayn, a charity tackling gender-based violence, draws a direct line between platform policy and harm. "Automated content moderation systems are trained to spot nudity, which is an important aspect of tackling image-based abuse, but misses that images can be deadly without nudity," she said.
The report frames the core defect as a definitional one: platforms judge images by what they show, not by how they are used. Chayn urges a shift from content-based rules to consent-based reporting pathways, with cultural context built into moderation decisions.
The consequences extend beyond distress. Hussain lists documented outcomes: women forced out of education and jobs, pressured or forced into marriage, subjected to verbal and physical assault and divorce, and in extreme cases suicide, murder and femicide.
One case from the report involves Zarghuna, a 15-year-old from Balochistan. A male student photographed her wading in a lake during a school picnic and later projected the image onto a large screen at a school event.
"My wet clothes were clinging to my body. Even if you're wearing a burqa, if it outlines your body, people will say you are making an exhibition," she said. She added that classmates regularly receive WhatsApp messages demanding sexual favours, with threats to publish images in hand.
How large is the exposure?
The gender divide in Pakistan is measurable. Two-thirds of Instagram users in the country are male. Of Pakistan's estimated 100 million TikTok users, more than 70 percent are also believed to be male.
The harm has already produced fatalities. In the past year, 14-year-old Hira Anwar was lured from the US to Pakistan and killed by her father over her TikTok videos. Sana Yousaf, another teenage TikTok user, was murdered last year by a man who had become obsessed with her account.
Pakistan's Digital Rights Foundation operates a helpline that receives approximately 250 complaints a month. In one recent case, the organisation secured the removal of AI-generated videos showing a teacher kissing a man on Instagram.
Nighat Dad, a lawyer who founded the DRF, said platform policies "are often designed without sufficient understanding of cultural contexts and the ways gendered harm manifests differently across communities." She added: "We repeatedly see cases where something may not fit a platform's narrow definition of intimate or sexual content, but in the survivor's context it is clearly being used to sexualise, shame or endanger her."
What does this change for moderation teams?
For platforms, the report is a demand to rebuild reporting flows around consent rather than nudity detection. For advocacy groups, it confirms that western-centric definitions of intimate imagery exclude women in the global south, where family honour tied to perceived chastity turns an ordinary selfie into leverage for extortion.
Maniza's case shows the endpoint. Her mother, knowing the reaction her father would have, pulled her from school. "I have to get married as soon as a proposal is found. Every day I am living in fear. That picture is still out there and there's nothing I can do," she said.
Hussain summarised the policy argument in one line: "Otherwise it's like someone throwing a bomb and focusing on what type of bomb it is instead of the damage it can do." Meta did not respond to a request for comment.
Whether platforms adopt consent-based reporting in the coming policy cycles will determine whether non-explicit imagery gains the same enforcement weight as nudity itself.
Source: theguardian.com
Tom Whitfield
Show full bio
News editor covering regulation, policy and their impact on Social platform news and policy at Reach Brief.
Related articles
Pakistan Rolls Out Tax Rules for Social Media Creators
Molly Russell's father to tell UK cinema audiences: 'Instagram helped kill my daughter'
Faceless Content Is Trending Globally, But Gulf Women Pioneered It
Report: Instagram Algorithm Surfaces Antisemitic Content to Millions
UK lawmakers call for criminal liability for camming platforms