The European Commission has launched a new formal investigation into X under the Digital Services Act (DSA), intensifying regulatory scrutiny over the platform’s use of its AI chatbot, Grok. Announced on January 26, the move follows mounting concerns that Grok AI image-generation and recommender functionalities may have exposed users in the EU to illegal and harmful content, including manipulated sexually explicit images and material that could amount to child sexual abuse material (CSAM). This latest European Commission investigation into X runs in parallel with an extension of an ongoing probe first opened in December 2023. The Commission will now examine whether X properly assessed and mitigated the systemic risks associated with deploying Grok’s functionalities into its platform in the EU, as required under the Digital Services Act (DSA).

Focus on Grok AI and Illegal Content Risks

At the core of the new proceedings is whether X fulfilled its obligations to assess and reduce risks stemming from Grok AI. The Commission said the risks appear to have already materialised, exposing EU citizens to serious harm. Regulators will investigate whether X:

• Diligently assessed and mitigated systemic risks, including the dissemination of illegal content, negative effects related to gender-based violence, and serious consequences for users’ physical and mental well-being.
• Conducted and submitted an ad hoc risk assessment report to the Commission for Grok’s functionalities before deploying them, given their critical impact on X’s overall risk profile.

If proven, these failures would constitute infringements of Articles 34(1) and (2), 35(1), and 42(2) of the Digital Services Act. The Commission stressed that the opening of formal proceedings does not prejudge the outcome but confirmed that an in-depth investigation will now proceed as a matter of priority.

Recommender Systems Also Under Expanded Scrutiny

In a related step, the European Commission has extended its December 2023 investigation into X’s recommender systems. This expanded review will assess whether X properly evaluated and mitigated all systemic risks linked to how its algorithms promote content, including the impact of its recently announced switch to a Grok-based recommender system. As a designated very large online platform (VLOP) under the DSA, X is legally required to identify, assess, and reduce systemic risks arising from its services in the EU. These risks include the spread of illegal content and threats to fundamental rights, particularly those affecting minors. Henna Virkkunen, Executive Vice-President for Tech Sovereignty, Security and Democracy, underlined the seriousness of the case in a statement: “Sexual deepfakes of women and children are a violent, unacceptable form of degradation. With this investigation, we will determine whether X has met its legal obligations under the DSA, or whether it treated rights of European citizens - including those of women and children - as collateral damage of its service.” Earlier this month, a European Commission spokesperson had also addressed the issue while speaking to journalists in Brussels, calling the matter urgent and unacceptable. “I can confirm from this podium that the Commission is also very seriously looking into this matter,” the spokesperson said, adding: “This is not ‘spicy’. This is illegal. This is appalling. This is disgusting. This has no place in Europe.”

International Pressure Builds Around Grok AI

The investigation comes against a backdrop of rising regulatory pressure worldwide over Grok AI’s image-generation capabilities. On January 16, X announced changes to Grok aimed at preventing the creation of nonconsensual sexualised images, including content that critics say amounts to CSAM. The update followed weeks of scrutiny and reports of explicit material generated using Grok. In the United States, California Attorney General Rob Bonta confirmed on January 14 that his office had opened an investigation into xAI, the company behind Grok, over reports describing the depiction of women and children in explicit situations. Bonta called the reports “shocking” and urged immediate action, saying his office is examining whether the company may have violated the law. U.S. lawmakers have also stepped in. On January 12, three senators urged Apple and Google to remove X and Grok from their app stores, arguing that the chatbot had repeatedly violated app store policies related to abusive and exploitative content.

Next Steps in the European Commission Investigation Into X

As part of the Digital Services Act (DSA) enforcement process, the Commission will continue gathering evidence by sending additional requests for information, conducting interviews, or carrying out inspections. Interim measures could be imposed if X fails to make meaningful adjustments to its service. The Commission is also empowered to adopt a non-compliance decision or accept commitments from X to remedy the issues under investigation. Notably, the opening of formal proceedings shifts enforcement authority to the Commission, relieving national Digital Services Coordinators of their supervisory powers for the suspected infringements. The investigation complements earlier DSA proceedings that resulted in a €120 million fine against X in December 2025 for deceptive design, lack of advertising transparency, and insufficient data access for researchers. With Grok AI now firmly in regulators’ sights, the outcome of this probe could have major implications for how AI-driven features are governed on large online platforms across the EU.

Elon Musk’s social media platform X has announced a series of changes to its AI chatbot Grok, aiming to prevent the creation of nonconsensual sexualized images, including content that critics and authorities say amounts to child sexual abuse material (CSAM). The announcement was made Wednesday via X’s official Safety account, following weeks of growing scrutiny over Grok AI’s image-generation capabilities and reports of nonconsensual sexualized content.

X Reiterates Zero Tolerance Policy on CSAM and Nonconsensual Content

In its statement, X emphasized that it maintains “zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content.” The platform said it continues to remove high-priority violative content, including CSAM, and to take enforcement action against accounts that violate X’s rules. Where required, accounts seeking child sexual exploitation material are reported to law enforcement authorities. The company acknowledged that the rapid evolution of generative AI presents industry-wide challenges and said it is actively working with users, partners, governing bodies, and other platforms to respond more quickly as new risks emerge.

Grok AI Image Generation Restrictions Expanded

As part of the update, X said it has implemented technological measures to restrict Grok AI from editing images of real people into revealing clothing, such as bikinis. These restrictions apply globally and affect all users, including paid subscribers. In a further change, image creation and image editing through the @Grok account are now limited to paid subscribers worldwide. X said this step adds an additional layer of accountability by helping ensure that users who attempt to abuse Grok in violation of laws or platform policies can be identified. X also confirmed the introduction of geoblocking measures in certain jurisdictions. In regions where such content is illegal, users will no longer be able to generate images of real people in bikinis, underwear, or similar attire using Grok AI. Similar geoblocking controls are being rolled out for the standalone Grok app by xAI.

Announcement Follows Widespread Abuse Reports

The update comes amid a growing scandal involving Grok AI, after thousands of users were reported to have generated sexualized images of women and children using the tool. Numerous reports documented how users took publicly available images and used Grok to depict individuals in explicit or suggestive scenarios without their consent. Particular concern has centered on a feature known as “Spicy Mode,” which xAI developed as part of Grok’s image-generation system and promoted as a differentiator. Critics say the feature enabled large-scale abuse and contributed to the spread of nonconsensual intimate imagery. According to one analysis cited in media reports, more than half of the approximately 20,000 images generated by Grok over a recent holiday period depicted people in minimal clothing, with some images appearing to involve children.

U.S. and European Authorities Escalate Scrutiny

On January 14, 2026, ahead of X’s announcement, California Attorney General Rob Bonta confirmed that his office had opened an investigation into xAI over the proliferation of nonconsensual sexually explicit material produced using Grok. In a statement, Bonta said reports describing the depiction of women and children in explicit situations were “shocking” and urged xAI to take immediate action. His office is examining whether and how xAI may have violated the law. Regulatory pressure has also intensified internationally. The European Commission confirmed earlier this month that it is examining Grok’s image-generation capabilities, particularly the creation of sexually explicit images involving minors. European officials have signaled that enforcement action is being considered.

App Store Pressure Adds to Challenges

On January 12, 2026, three U.S. senators urged Apple and Google to remove X and Grok from their app stores, arguing that Grok AI has repeatedly violated app store policies related to abusive and exploitative content. The lawmakers warned that app distribution platforms may also bear responsibility if such content continues.

Ongoing Oversight and Industry Implications

X said the latest changes do not alter its existing safety rules, which apply to all AI prompts and generated content, regardless of whether users are free or paid subscribers. The platform stated that its safety teams are working continuously to add safeguards, remove illegal content, suspend accounts where appropriate, and cooperate with authorities. As investigations continue across multiple jurisdictions, the Grok controversy is becoming a defining case in the broader debate over AI safety, accountability, and the protection of children and vulnerable individuals in the age of generative AI.

Concerns surrounding Grok AI are escalating rapidly, with pressure now mounting in the United States after ongoing scrutiny in Europe. Three U.S. senators have urged Apple and Google to remove the X app and Grok AI from the Apple App Store and Google Play Store, citing the large-scale creation of nonconsensual sexualized images of real people, including children. The move comes as a direct follow-up to the European Commission’s investigation into Grok AI’s image-generation capabilities, marking a significant expansion of regulatory attention beyond the EU. While European regulators have openly weighed enforcement actions, U.S. authorities are now signaling that app distribution platforms may also bear responsibility.

U.S. senators Cite App Store Policy Violations by Grok AI

In a letter dated January 9, 2026, Senators Ron Wyden, Ed Markey, and Ben Ray Luján formally asked Apple CEO Tim Cook and Google CEO Sundar Pichai to enforce their app store policies against X Corp. The lawmakers argue that Grok AI, which operates within the X app, has repeatedly violated rules governing abusive and exploitative content. According to the senators, users have leveraged Grok AI to generate nonconsensual sexualized images of women, depicting abuse, humiliation, torture, and even death. More alarmingly, the letter states that Grok AI has also been used to create sexualized images of children, content the senators described as both harmful and potentially illegal. The lawmakers emphasized that such activity directly conflicts with policies enforced by both the Apple App Store and Google Play Store, which prohibit content involving sexual exploitation, especially material involving minors.

Researchers Flag Potential Child Abuse Material Linked to Grok AI

The letter also references findings by independent researchers who identified an archive connected to Grok AI containing nearly 100 images flagged as potential child sexual abuse material. These images were reportedly generated over several months, raising questions about X Corp’s oversight and response mechanisms. The senators stated that X appeared fully aware of the issue, pointing to public reactions by Elon Musk, who acknowledged reports of Grok-generated images with emoji responses. In their view, this signaled a lack of seriousness in addressing the misuse of Grok AI.

Premium Restrictions Fail to Calm Controversy

In response to the backlash, X recently limited Grok AI’s image-generation feature to premium subscribers. However, the senators dismissed this move as inadequate. Sen. Wyden said the change merely placed a paywall around harmful behavior rather than stopping it, arguing that it allowed the production of abusive content to continue while generating revenue. The lawmakers stressed that restricting access does not absolve X of responsibility, particularly when nonconsensual sexualized images remain possible through the platform.

Pressure Mounts on Apple App Store and Google Play Store

The senators warned that allowing the X app and Grok AI to remain available on the Apple App Store and Google Play Store would undermine both companies’ claims that their platforms offer safer environments than alternative app distribution methods. They also pointed to recent instances where Apple and Google acted swiftly to remove other controversial apps under government pressure, arguing that similar urgency should apply in the case of Grok AI. At minimum, the lawmakers said, temporary removal of the apps would be appropriate while a full investigation is conducted. They requested a written response from both companies by January 23, 2026, outlining how Grok AI and the X app are being assessed under existing policies. Apple and Google have not publicly commented on the letter, and X has yet to issue a formal response. The latest development adds momentum to global scrutiny of Grok AI, reinforcing concerns already raised by the European Commission. Together, actions in the U.S. and Europe signal a broader shift toward holding AI platforms, and the app ecosystems that distribute them, accountable for how generative technologies are deployed and controlled at scale.

The Grok AI investigation has intensified after the European Commission confirmed it is examining the creation of sexually explicit and suggestive images of girls, including minors, generated by Grok, the artificial intelligence chatbot integrated into social media platform X. The scrutiny follows widespread outrage linked to a paid feature known as “Spicy Mode,” introduced last summer, which critics say enabled the generation and manipulation of sexualised imagery. Speaking to journalists in Brussels on Monday, a spokesperson for the European Commission said the matter was being treated with urgency. “I can confirm from this podium that the Commission is also very seriously looking into this matter,” the spokesperson said, adding: “This is not 'spicy'. This is illegal. This is appalling. This is disgusting. This has no place in Europe.”

European Commission Examines Grok’s Compliance With EU Law

The European Commission Grok probe places renewed focus on the responsibilities of AI developers and social media platforms under the EU’s Digital Services Act (DSA). The European Commission, which acts as the EU’s digital watchdog, said it is assessing whether X and its AI systems are meeting their legal obligations to prevent the dissemination of illegal content, particularly material involving minors. The inquiry comes after reports that Grok was used to generate sexually explicit images of young girls, including through prompts that altered existing images. The controversy escalated following the rollout of an “edit image” feature that allowed users to modify photos with instructions such as “put her in a bikini” or “remove her clothes.” On Sunday, X said it had removed the images in question and banned the users involved. “We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary,” the company’s X Safety account posted. [caption id="attachment_108277" align="aligncenter" width="370"] Source: X[/caption]

International Backlash and Parallel Investigations

The X AI chatbot Grok is now facing regulatory pressure beyond the European Commission. Authorities in France, Malaysia, and India have launched or expanded investigations into the platform’s handling of explicit and sexualised content generated by the AI tool. In France, prosecutors last week expanded an existing investigation into X to include allegations that Grok was being used to generate and distribute child sexual abuse material. The original probe, opened in July, focused on claims that X’s algorithms were being manipulated for foreign interference. India has also taken a firm stance. Last week, Indian authorities reportedly ordered X to remove sexualised content, curb offending accounts, and submit an “Action Taken Report” within 72 hours or face legal consequences. As of Monday, there was no public confirmation on whether X had complied. [caption id="attachment_108281" align="aligncenter" width="1024"] Source: India's Ministry of Electronics and Information Technology[/caption] Malaysia’s Communications and Multimedia Commission said it had received public complaints about “indecent, grossly offensive” content on X and confirmed it was investigating the matter. The regulator added that X’s representatives would be summoned.

DSA enforcement and Grok’s previous controversies

The current Grok AI investigation is not the first time the European Commission has taken action related to the chatbot. Last November, the Commission requested information from X after Grok generated Holocaust denial content. That request was issued under the DSA, and the Commission said it is still analysing the company’s response. In December, X was fined €120 million under the DSA over its handling of account verification check marks and advertising practices. “I think X is very well aware that we are very serious about DSA enforcement. They will remember the fine that they have received from us,” the Commission spokesperson said.

Public reaction and growing concerns over AI misuse

The controversy has prompted intense discussion across online platforms, particularly Reddit, where users have raised alarms about the potential misuse of generative AI tools to create non-consensual and abusive content. Many posts focused on how easily Grok could be prompted to alter real images, transforming ordinary photographs of women and children into sexualised or explicit content. Some Reddit users referenced reporting by the BBC, which said it had observed multiple examples on X of users asking the chatbot to manipulate real images—such as making women appear in bikinis or placing them in sexualised scenarios—without consent. These examples, shared widely online, have fuelled broader concerns about the adequacy of content safeguards. Separately, the UK’s media regulator Ofcom said it had made “urgent contact” with Elon Musk’s company xAI following reports that Grok could be used to generate “sexualised images of children” and produce “undressed images” of individuals. Ofcom said it was seeking information on the steps taken by X and xAI to comply with their legal duties to protect users in the UK and would assess whether the matter warrants further investigation. Across Reddit and other forums, users have questioned why such image-editing capabilities were available at all, with some arguing that the episode exposes gaps in oversight around AI systems deployed at scale. Others expressed scepticism about enforcement outcomes, warning that regulatory responses often come only after harm has already occurred. Although X has reportedly restricted visibility of Grok’s media features, users continue to flag instances of image manipulation and redistribution. Digital rights advocates note that once explicit content is created and shared, removing individual posts does not fully address the broader risk to those affected. Grok has acknowledged shortcomings in its safeguards, stating it had identified lapses and was “urgently fixing them.” The AI tool has also issued an apology for generating an image of two young girls in sexualised attire based on a user prompt. As scrutiny intensifies, the episode is emerging as a key test of how AI-generated content is regulated—and how accountability is enforced—when powerful tools enable harm at scale.

By Gauravdeep Singh, Head – State e-Mission Team (SeMT), Ministry of Electronics and Information Technology The Digital Personal Data Protection (DPDP) Act has fundamentally altered the risk landscape for Indian organisations. Data breaches now trigger mandatory compliance obligations regardless of their origin, transforming incidents that were once purely operational concerns into regulatory events with significant financial and legal implications.

Case Study 1: Cloud Misconfiguration in a Consumer Platform

A prominent consumer-facing platform experienced a data exposure incident when a misconfigured storage bucket on its public cloud infrastructure inadvertently made customer data publicly accessible. While no malicious actor was involved, the incident still constituted a reportable data breach under the DPDP Act framework. The organisation faced several immediate obligations:

• Notification to affected individuals within prescribed timelines
• Formal reporting to the Data Protection Board
• Comprehensive internal investigation and remediation measures
• Potential penalties for failure to implement reasonable security safeguards as mandated under the Act

Such incidents highlight a critical gap in traditional risk management approaches. The financial exposure—encompassing regulatory penalties, legal costs, remediation expenses, and reputational damage—frequently exceeds conventional cyber insurance coverage limits, particularly when compliance failures are implicated.

Case Study 2: Ransomware Attack on Healthcare and EdTech Infrastructure

A mid-sized healthcare and education technology provider fell victim to a ransomware attack that encrypted sensitive personal records. Despite successful restoration from backup systems, the organisation confronted extensive regulatory and operational obligations:

• Forensic assessment to determine whether data confidentiality was compromised
• Mandatory notification to regulatory authorities and affected data principals
• Ongoing legal and compliance proceedings

The total cost extended far beyond any ransom demand. Forensic investigations, legal advisory services, public communications, regulatory compliance activities, and operational disruption collectively created substantial financial strain, costs that would have been mitigated with appropriate insurance coverage.

Case Study 3: AI-Enabled Fraud and Social Engineering

The emergence of AI-driven attack vectors has introduced new dimensions of cyber risk. Deepfake technology and sophisticated phishing campaigns now enable threat actors to impersonate senior leadership with unprecedented authenticity, compelling finance teams to authorise fraudulent fund transfers or inappropriate data disclosures. These attacks often circumvent traditional technical security controls because they exploit human trust rather than system vulnerabilities. As a result, organisations are increasingly seeking insurance coverage for social engineering and cyber fraud events, particularly those involving personal data or financial information, that fall outside conventional cybersecurity threat models.

The Evolution of Cyber Insurance in India

The Indian cyber insurance market is undergoing significant transformation in response to the DPDP Act and evolving threat landscape. Modern policies now extend beyond traditional hacking incidents to address:

• Data breaches resulting from human error or operational failures
• Third-party vendor and SaaS provider security failures
• Cloud service disruptions and availability incidents
• Regulatory investigation costs and legal defense expenses
• Incident response, crisis management, and public relations support

Organisations are reassessing their coverage adequacy as they recognise that historical policy limits of Rs. 10–20 crore may prove insufficient when regulatory penalties, legal costs, business interruption losses, and remediation expenses are aggregated under the DPDP compliance framework.

The SME and MSME Vulnerability

Small and medium enterprises represent the most vulnerable segment of the market. While many SMEs and MSMEs regularly process personal data, they frequently lack:

• Mature information security controls and governance frameworks
• Dedicated compliance and data protection teams
• Financial reserves to absorb penalties, legal costs, or operational disruption

For organisations in this segment, even a relatively minor cyber incident can trigger prolonged operational shutdowns or, in severe cases, permanent closure. Despite this heightened vulnerability, cyber insurance adoption among SMEs remains disproportionately low, driven primarily by awareness gaps and perceived cost barriers.

Implications for the Cyber Insurance Ecosystem

The Indian cyber insurance market is entering a period of accelerated growth and structural evolution. Several key trends are emerging:

• Higher policy limits becoming standard practice across industries
• Enhanced underwriting processes emphasising compliance readiness and data governance maturity
• Comprehensive coverage integrating legal advisory, forensic investigation, and regulatory support
• Risk-based pricing models that reward robust data protection practices

Looking ahead, cyber insurance will increasingly be evaluated not merely as a risk-transfer mechanism, but as an indicator of an organisation's overall data protection posture and regulatory preparedness.

DPDP Act and the End of Optional Cyber Insurance

The DPDP Act has fundamentally redefined cyber risk in the Indian context. Data breaches are no longer isolated IT failures; they are regulatory events carrying substantial financial, legal, and reputational consequences. In this environment, cyber insurance is transitioning from a discretionary safeguard to a strategic imperative. Organisations that integrate cyber insurance into a comprehensive data governance and enterprise risk management strategy will be better positioned to navigate the evolving regulatory landscape. Conversely, those that remain uninsured or underinsured may discover that the cost of inadequate preparation far exceeds the investment required for robust protection. (This article reflects the author’s analysis and personal viewpoints and is intended for informational purposes only. It should not be construed as legal or regulatory advice.)

Nikhil Jhanji, Principal Product Manager, Privy by IDfy The DPDP rules have finally given enterprises a clear structure for how personal data enters and moves through their systems. What has not been discussed enough is that this same structure also reduces the space in which deepfakes and synthetic identities can slip through. For months the Act lived in broad conversation without detail. Now enterprises have to translate the rules into real action. As they do that work, a practical advantage becomes visible. The discipline required around consent, accuracy, and provenance creates an environment where false personas cannot blend in as easily. This was not the intention of the framework, but it is an important consequence.

DPDP Rules Bring Structure to Enterprise Data Intake

The first shift happens at data entry. The rules require clear consent, proof of lawful purpose, and timely correction of errors. This forces organisations to examine the origin of the data they collect and to maintain records that confirm why the data exists. Better visibility into the source and purpose of data makes it harder for synthetic identities to enter the system through weak or careless intake flows. This matters because the word synthetic now carries two very different meanings. One meaning refers to responsible synthetic data used in privacy enhancing technologies. This type is created intentionally, documented carefully, and used to train models or test systems without revealing personal information. It supports the goals of privacy regulation and does not imitate real individuals.

Synthetic Data vs Synthetic Identity: A Critical Difference

The other meaning refers to deceptive synthetic identities, false personas deliberately created to exploit weak verification processes. These may include deepfake facial images, manipulated voice samples, and fabricated documents or profiles that appear legitimate enough to pass routine checks.

This form of synthetic identity thrives in environments with poor data discipline and is designed specifically to mislead systems and people.

The DPDP rules help enterprises tell the difference with more clarity. Responsible synthetic data has provenance and purposeful creation. Deceptive synthetic identity has neither. Once intake and governance become more structured, the distinction becomes easier to detect through both human review and automated systems.

Cleaner Data Improves Fraud and Risk Detection

As organisations rewrite consent journeys and strengthen provenance under the DPDP rules, the second advantage becomes clear. Cleaner input improves downstream behaviour. Fraud engines perform better with consistent signals. Risk decisions become clearer. Customer support teams gain more dependable records. When data is scattered and unchecked, synthetic personas move more freely. When data is organised and verified, they become more visible. This is where the influence of DPDP rules becomes subtle. Deepfake content succeeds by matching familiar patterns. It blends into weak systems that cannot challenge continuity. Structured data environments limit these opportunities. They reduce ambiguity and shrink the number of places where a false identity can hide. This gives enterprises a stronger base for every detection capability they depend on. There is also a behavioural shift introduced by the DPDP rules. Once teams begin managing data with more discipline, their instinct around authenticity improves. Consent is checked properly. Accuracy is taken seriously. Records are maintained rather than ignored. This change in everyday behaviour strengthens identity awareness across the organisation. Deepfake risk is not only technical. It is also operational, and disciplined teams recognise anomalies faster.

DPDP Rules Do Not Stop Deepfakes—but They Shrink the Attack Surface

None of this means that DPDP rules stop deepfakes. They do not. Deepfake quality is rising and will continue to challenge even mature systems. What the rules offer is a necessary foundation. They push organisations to adopt habits of verification, documentation, and controlled intake. Those habits shrink the attack surface for synthetic identities and improve the effectiveness of whatever detection tools a company chooses to use. As enterprises interpret the rules, many will see the work as procedural. New notices. Updated consent. Retention plans. But the real strength will emerge in the functions that depend on reliable identity and reliable records. Credit decisions. Access management. Customer onboarding. Dispute resolution. Identity verification. These areas become more stable when the data that supports them is consistent and traceable. The rise of deepfakes makes this stability essential. False personas are cheap to create and increasingly convincing. They will exploit gaps wherever they exist. Strong tools matter, but so does the quality of the data that flows into those tools. Without clean and verified data, even advanced detection systems struggle. The DPDP rules arrive at a moment when enterprises need stronger foundations. By demanding better intake discipline and clearer data pathways, they reduce the natural openings that deceptive synthetic content relies on. In a world where authentic and synthetic individuals now compete for space inside enterprise systems, this shift may become one of the most practical outcomes of the entire compliance effort. (This article reflects the author’s analysis and personal viewpoints and is intended for informational purposes only. It should not be construed as legal or regulatory advice.)

The FBI is warning of AI-assisted fake kidnapping scams:

Criminal actors typically will contact their victims through text message claiming they have kidnapped their loved one and demand a ransom be paid for their release. Oftentimes, the criminal actor will express significant claims of violence towards the loved one if the ransom is not paid immediately. The criminal actor will then send what appears to be a genuine photo or video of the victim’s loved one, which upon close inspection often reveals inaccuracies when compared to confirmed photos of the loved one. Examples of these inaccuracies include missing tattoos or scars and inaccurate body proportions. Criminal actors will sometimes purposefully send these photos using timed message features to limit the amount of time victims have to analyze the images.

Images, videos, audio: It can all be faked with AI. My guess is that this scam has a low probability of success, so criminals will be figuring out how to automate it.