AI data and algorithmic classification
A deleted chat leaves the servers in thirty days. The risk score that chat generated stays for seven years. How AI labs built an intelligence apparatus aimed at their own users, and why the conversation dies long before its classification does.
AI data – On Friday afternoon, 14 August 2026, someone tells Claude they have bought an AR-15 semi-automatic rifle and have Anthropic’s founder in their sights. The following Tuesday, at nine in the morning, the company calls the San Francisco police. Officers arrive at 500 Howard Street, establish that the author of the message is not in the building, and ask to read the conversations.
The employee who receives them refuses, citing company policy. In the police report obtained by the San Francisco Standard the officer puts both things on record: the report of a murder threat, and the denial of the only documentary evidence of that threat. Reached by phone by the paper, the man says he was joking and admits he is embarrassed. He was neither arrested nor charged. His account was closed.
Almost everywhere the episode has been read as a question of proportionality, whether a tasteless joke deserves a call to law enforcement. Underneath that single case, though, sits an architecture that concerns everyone else, and that none of the parties involved has any interest in describing in full: the automatic classification system that intercepted the message, the register that preserved its outcome, and the startling gap between how long a conversation lives and how long the judgment passed on it lives.
How long do AI companies keep your chats and risk scores?
An ordinary conversation on an Anthropic consumer account disappears from the history the moment the user deletes it and is removed from back-end systems within thirty days. With consent to use the data for model improvement, retention rises to five years in de-identified form. If instead the automated trust and safety systems flag the conversation as a usage policy violation, inputs and outputs are kept for up to two years and classification scores for up to seven. On Google Gemini, activity is kept for eighteen months by default, while chats sampled for human review stay three years on a separate path. OpenAI, under a court order, holds logs that users had already deleted.
Seven years against thirty days
The conversation that concerns you as a customer has a short life. The verdict that concerns you as a suspect has one eighty-four times longer. The delete button governs the most ephemeral copy in the archive, and not the one that counts. Anthropic’s privacy center states these terms with a precision worth reading twice, because the hierarchy that emerges overturns the common idea of what deleting means.
What happens to a chat the moment it is flagged
The text a user sends to a language model does not reach the model directly. It passes through a chain of classifiers that turn it into numerical vectors, assign it scores on predefined risk dimensions and decide, above a certain threshold, to block generation or to route the request to a model with tighter constraints. The same happens on the way out, on the response, while it is being produced. Anyone familiar with the probabilistic regime LLMs run on will recognise the mechanism: it is public in its broad outlines and shared by every provider. What changes from company to company, and what almost no user knows, is what remains afterwards.
Why a score is not a text
This is where the vocabulary of traditional privacy analysis stops working. Talking about confidentiality of content implies that the object in dispute is the conversation, a text you can read, delete, export. A classification score is instead an operational abstraction: a position on a scale, separable from the material that generated it, aggregable with other scores, comparable over time.
Oscar Gandy anticipated this in 1993 when he described the panoptic sort as a system that does not watch individuals in order to know them but in order to sort them into classes of differential treatment, and whose product is a position in a ranking rather than a dossier. Gilles Deleuze, three years earlier, had called dividual what remains of a person inside a control system that breaks them into modulable variables. The conversation of the man of 14 August was wiped from the servers. The dividual that conversation generated has six and a half years of documentary existence ahead of it.
The conversation dies in thirty days. The judgment on whoever wrote it survives seven years.
How AI labs moved from monitoring to prediction
On 9 September 2026 the journalist Daniel Boguslaw published in the American Prospect a reconstruction built on job listings and a podcast interview with Anthropic security executives. The material is ordinary in form and remarkable in content, and it breaks down into three documented steps.
The tools: real-time OSINT alerting
Anthropic holds a contract with Samdesk, a real-time risk detection company that scans open sources to flag events as they form. In a webinar devoted to security operations centres, the head of the company’s global security operations centre described a concrete case: an executive travelling in a major city, an authorised protest whose start time shifts for permit reasons, an alert arriving sixty minutes ahead of the change, an alternative route taking the executive to the hotel service entrance instead of through the march. A lawful demonstration handled with the operational grammar of severe weather, tracked and navigated around.
The org chart: when activism joins the threat list
The team that handles this is called Global Safety, Intelligence, and Security, and in August it opened a position for an enterprise intelligence specialist paying between 180 and 230 thousand dollars. The profile asks the hire to identify, assess, track and investigate global threats. In the list of those threats, activism appears in the same sequence as geopolitical instability, terrorism, crime and state-actor targeting of the AI industry. The listing explicitly includes OSINT collection on specific threats, actors and events.
The direction: proactive engagement and persons of interest
In the podcast the head of the security programme describes the goal as a shift from reactive information gathering to proactive, predictive and preventive threat engagement. Speaking to the Wall Street Journal in July, the company had already explained that it tracks concerning behaviour over time through a person of interest process, in order to catch escalation patterns early. According to the same paper, several individuals involved in incidents later reported to police were already being monitored by corporate security.
Lucia Zedner coined the formula pre-crime for this configuration in 2007: a regime in which risk calculation replaces establishing the fact, intervention precedes the event, and the operative category becomes the probable rather than the guilty. The theoretical point that makes the formula useful here is procedural before it is moral. In criminal law the prosecution produces a case file and the defendant can contest it. In a person of interest process run by a private company there is no notification, no adversarial hearing, no limitation period other than the one the company has set for itself in its own policy. The same asymmetry we examined in algorithmic surveillance systems applied to US prisons here operates on paying customers.
Anthropic, OpenAI and Google: how long they really keep chat data
A great deal of wrong information circulates on this terrain, including the wrong information produced by conversational assistants themselves when you ask them about it. The figures verified as of September 2026 are these.
Anthropic: from thirty days to five years
The position changed in August 2025. Until then, consumer user data was not used for training; the updated terms introduced a consent switch with a deadline of 8 October, beyond which the choice became mandatory in order to keep using the service. Those who consent move retention from thirty days to five years, those who refuse stay at thirty days. The interface through which the choice was presented has been challenged by legal analysts as a possible dark pattern under the GDPR, with no enforcement action by supervisory authorities on record so far. Incognito chats remain excluded from training in every case, and the commercial APIs follow a stricter regime than the web interface, with standard retention cut to seven days and Zero Data Retention available to qualified enterprise customers.
OpenAI: when a court suspends deletion
The most instructive case is OpenAI’s, because it shows what happens when a corporate policy meets a court. On 13 May 2025 Judge Ona Wang ordered the company to preserve and segregate all output logs that would otherwise have been deleted, including conversations users had already erased, as part of the copyright litigation brought by the New York Times and other publishers. The order was lifted on 9 October 2025, but the logs accumulated in the meantime remain available to the proceedings.
In November the same judge ordered twenty million de-identified logs handed over to the plaintiffs. OpenAI objected that 99.99 per cent of the transcripts were irrelevant and that production would expose private conversations, and on 5 January 2026 District Judge Sidney Stein upheld the order. The most telling figure in the whole affair emerges from that same decision: twenty million conversations amount to half a percentage point of the retained logs, which the ruling puts in the order of tens of billions.
Google Gemini: eighteen months of activity, three years of human review
The least legible configuration is Google’s, because it is the most integrated. Gemini activity is kept for eighteen months by default, with auto-delete options at three or thirty-six months and the ability to switch recording off. Switching it off zeroes nothing: conversations still sit in the account for up to seventy-two hours to deliver the service, and above all the chats sampled for human review are detached from the account and stored separately for up to three years, untouched by activity deletion and with no user control to remove them. Google itself advises against entering confidential information into Gemini, because a flesh-and-blood reviewer might read it.
The retention path no setting governs
The comparative picture that emerges inverts the intuitive hierarchy. The decisive difference between the three providers does not run through the generosity of the settings offered to users. It runs through the number of retention paths those settings do not govern, and every company has at least one: the human review pipeline for Google, the litigation file for OpenAI, the trust and safety score register for Anthropic. The question of who actually holds sovereignty over data is decided on these paths, not in the preference panels.
Why aggregation is not the problem, and where it becomes one
A substantial share of the economic value of these archives requires identifying no one. It is worth saying this plainly, because the most common argument against AI companies on this terrain is also the weakest.
Clio and the statistical map of what we ask machines
Anthropic built and publicly documented a dedicated tool, introduced in December 2024 under the name Clio and now rebranded Anthropic Insights, which uses the model itself to extract attributes from conversations, group them by theme, describe the clusters while excluding private information, and submit the result to an audit pass that checks no cluster identifies fewer than roughly a thousand individuals. No human analyst reads the raw material. The comparison the company uses is Google Trends, and for once the comparison holds: the product is a distribution, not a profile.
On 26 August 2026 the same tool was opened to external researchers from Stanford, Oxford and METR, with roughly 250 thousand privacy-protected conversations and access limited to aggregated outputs. It is the first time a frontier lab has exposed its usage data to independent study. On transparency this is a precedent pointing in the opposite direction from everything else in this reconstruction.
Where the same infrastructure produces a person of interest
Shoshana Zuboff described the mechanism by which behavioural surplus generated as a by-product of a service becomes the raw material of a predictive product, and identified in the prediction imperative the push that leads companies to extend collection from the past to the present to intervention. The Anthropic Insights case shows where that description needs calibrating. Statistical aggregation of millions of conversations produces enormous strategic knowledge, a real-time map of what a substantial slice of humanity is asking a machine, and it produces it without violating anyone’s confidentiality in particular.
The harm does not sit in the cluster. It sits in the parallel path where the same analytical infrastructure, applied to a single account rather than to a distribution, produces a person of interest. The two functions run on the same technology stack and are separated only by a configuration choice and an internal policy, which is exactly the dynamic described in the economy of behavioural futures. This, and not aggregation, is what an analysis of safeguards should be looking at.
What the EU AI Act bans on individual risk scoring
The prohibitions in Article 5 of the European regulation on artificial intelligence have been in force since 2 February 2025, and two of those letters bear directly on this matter.
The social scoring ban
Letter c prohibits social scoring systems, meaning the evaluation or classification of natural persons or groups over a certain period of time on the basis of social behaviour or known, inferred or predicted personal characteristics, where this results in detrimental treatment that is disproportionate or unrelated to the context in which the data was collected. It is a rule that applies to private actors too, not only to public authorities.
The predictive policing ban and its two exits
Letter d prohibits AI systems that assess or predict the risk of a natural person committing a criminal offence based solely on profiling or on assessing personality traits and characteristics. Recital 42 explains the rationale by invoking the presumption of innocence: a person should be judged on actual behaviour, not on behaviour predicted by a machine. The ban has a precise perimeter, however, and two exits. It does not cover systems supporting a human assessment of involvement in criminal activity that is already grounded in objective and verifiable facts directly linked to that activity, and it does not cover predictions that are not based solely on profiling, including predictions based on location rather than on the person.
What is left uncovered
A person of interest process starting from the content of a specific message most likely falls inside the first exit, because the message of 14 August is an objective and verifiable fact. The question neither letter resolves concerns the six and a half years that follow, when the fact has been deleted and only the score remains, when the score is combined with other scores to detect escalation patterns, and when tracking concerning behaviour over time starts to look a great deal like evaluating a person over a certain period on the basis of inferred characteristics. The European text addresses whoever places on the market or uses an AI system for these purposes, and an internal classification register presents itself as neither product nor service. It presents itself as corporate security, with the same enforcement asymmetry that makes platform compliance theatre so often a formal exercise.
Anthropic’s contradiction between public policy and internal security
On the level of public documents the company’s position is the most restrictive in the sector. The usage policy bars customers from deploying its products for criminal justice determinations, for tracking a person’s physical location without consent, for collecting information on an individual or group in order to track, target or flag their identity, and it explicitly names predictive policing among prohibited uses.
The standoff with the Pentagon
The company defended that policy at considerable cost. On 24 February 2026 the Secretary of Defense gave Anthropic three days to grant unlimited use of its models for any lawful purpose. The company refused, holding on to two reservations only, domestic mass surveillance and fully autonomous weapons. On 27 February the president ordered all federal agencies to cease using Anthropic technology immediately and the Pentagon designated the company a supply chain risk, the first such case for an American firm. Anthropic sued on 9 March; on 26 March Judge Rita Lin granted a preliminary injunction, finding the government action was probably retaliatory against a position protected by the First Amendment; in late August 2026 a second judge blocked the designation. The ACLU and the Center for Democracy and Technology intervened in support of the company, and the full reconstruction of that standoff remains the necessary context for what follows.
Both things are true at once. A company that accepted exclusion from federal contracts rather than grant mass surveillance of American citizens is at the same time building an OSINT intelligence capability that lists activism among global threats, and reporting users to the police on the basis of on-platform content. The Prospect also notes that the company has resumed recruiting for sales roles in the national security sector.
What the sources do not prove
What the reporting does not establish has to be added, because an investigation that omits the limits of its own sources is worth less. None of the available evidence shows that Claude was used to monitor the protest described in the podcast: the alert came from Samdesk, a third-party vendor, and the usage policy binds customers, not internal security operations carried out with someone else’s tools. No evidence shows a system capable of predicting who will commit a crime, nor that individual participants in that demonstration were investigated or reported.
The threats this apparatus responds to are, moreover, documented: a Molotov cocktail against the home of OpenAI’s CEO in April, a man stopped in Anthropic’s lobby after announcing an executive would be killed, a sevenfold rise in threats against AI leaders and data centres between February and May according to threat intelligence firm Liferaft. Eric Goldman, a legal scholar at Santa Clara University, has identified the dynamic driving the system: whoever fails to report is exposed to enormous liability if the crime then happens, and the more unpopular a company becomes the stronger the pressure to over-report, in the knowledge that some of the people named to law enforcement should never have been.
What knowing you are read does to you: dataveillance chilling effects
In 2016 Jonathon Penney measured traffic to Wikipedia entries on privacy-sensitive topics before and after the June 2013 revelations about the PRISM programme, finding an immediate statistically significant drop and a change in the long-term trend. Parallel studies detected the same effect on Google search queries and on willingness to express minority opinions on social platforms. Moritz Büchi, Noemi Festic and Michael Latzer later generalised the mechanism under the name chilling effects of digital dataveillance: the perception of being subject to continuous, non-specific collection of digital traces reduces communicative behaviour, regardless of whether actual surveillance is taking place.
Why the harm changes nature on conversational assistants
A drop in visits to an encyclopedia page reduces access to information that exists elsewhere. A chat with a language model, by contrast, is used by millions of people as a confessional, a diary, an approximate medical consultation, a vent, a room in which to try out thoughts not yet presentable. Its use value depends entirely on the candour of whoever is writing. Once it becomes widely understood that a classification threshold can convert an ill-judged sentence into a phone call to the police and a score kept for seven years, self-censorship does not degrade access to an external archive: it degrades the only input the service has.
The more cautious users stop writing certain things. The genuinely dangerous ones stop first, because they have the strongest motive. The classifier goes on intercepting the middle band, made up in large part of people joking badly.
An attribute nobody can appeal
Dario Amodei wrote in August that public hostility towards artificial intelligence does not stem from the warnings of its executives but from a deeper crisis of trust, from the widespread suspicion that companies, governments and the tech industry are preparing a new way to rip people off. As a diagnosis it is not wrong. As a self-portrait it stays incomplete, because trust in a classification system does not depend on the intentions of whoever administers it but on the procedures that can be brought against it.
The man of 14 August said he was joking, was never charged, and lost access to the service. Somewhere, in a register no deletion touches and no policy requires him to know about, the score that classified him as a threat will go on existing until August 2033. There is no procedure to contest it, because formally it is not a decision. It is an attribute.
- Daniel Boguslaw, Anthropic Is Building a Predictive Surveillance System to Monitor Activists, The American Prospect, 9 September 2026
- Rachyl Jones, Anthropic called SFPD over threat against CEO, The San Francisco Standard, 4 September 2026
- Anthropic Privacy Center, How long do you store my data?, updated 1 July 2026
- Anthropic, Updates to Consumer Terms and Privacy Policy, August 2025
- Anthropic, Clio: Privacy-preserving insights into real-world AI use, December 2024
- Anthropic, Enabling independent research on how people use Claude, 26 August 2026
- Anthropic, Usage Policy and related updates
- Bloomberg Law, OpenAI Must Turn Over 20 Million ChatGPT Logs, Judge Affirms, January 2026
- Ars Technica and Engadget on the lifting of the preservation order, October 2025
- Google, Gemini Apps Privacy Hub, updated August 2026
- Regulation (EU) 2024/1689, art. 5(1)(c) and 5(1)(d), recital 42, and European Commission Guidelines on prohibited practices
- Congressional Research Service, Pentagon-Anthropic Dispute over Autonomous Weapon Systems, IN12669
- TechPolicy.Press, A Timeline of the Anthropic-Pentagon Dispute
- Jonathon W. Penney, Chilling Effects: Online Surveillance and Wikipedia Use, Berkeley Technology Law Journal, 31(1), 2016
- Moritz Büchi, Noemi Festic, Michael Latzer, The Chilling Effects of Digital Dataveillance, Big Data & Society, 2022
Post scriptum
The source material for this piece contained three substantial errors, corrected before writing. The claim that Anthropic does not use chat data to train its models absent explicit consent was true until August 2025 and no longer holds in the terms stated, because the choice was made mandatory in order to keep using the service. The retention figure of 28 days is wrong: the term is thirty days for ordinary conversations, five years where consent to training is given, two years for flagged inputs and outputs, and seven years for classification scores. That last figure, absent from the original material, is the central data point of this investigation.
The Gemini retention figure of three years is imprecise: default activity retention is eighteen months, while the three years apply to conversations sampled for human review, stored on a separate path and not removed by activity deletion. The original material also credited OpenAI with the ability to delete history permanently, without mentioning the May 2025 preservation order or the twenty million logs ordered produced in November and confirmed in January.
Similar Posts
data sovereigniy: the ecology of new digital powers
Data sovereignty is an entity’s capacity to control what information it produces, where that information is processed and who extracts value from it. In 2026 it is contested by sellers of surveillance and sellers of independence from Silicon Valley.
the new era of digital surveillance: the futurable control
What is surveillance capitalism? From Zuboff’s behavioral surplus to Wiener’s cybernetics and Merleau-Ponty: how data, prediction and algorithms turn behavior and intention into a commodity. They don’t sell what you were — they sell the probability of what you’ll do. And it isn’t fate.
chat control: the new european digital surveillance system
A Machine That Learns to Read You: How Chat Control Reshapes the Structure of Power in the Digital Environment Chat control – Digital surveillance – European Union dossier On July 9, 2026, the European Parliament let the extension of voluntary scanning of unencrypted messages pass by procedural inertia, in force until 2028. The real test, mandatory scanning…
brain computer interface: who contorl your data?
In March 2026 China’s NMPA approved NEO, the world’s first commercially cleared invasive brain implant. Dong Hui, paralyzed for six years, wrote his name again. His motor intention now passes through a proprietary decoder and a 345-million-dollar IPO. Who owns the gesture when it crosses a server?
scriptwriting, ai and knowledge: is the end of stories?
A badly written video costs more than a badly shot one: footage is fixed in post, words are not. Scriptwriting for video governs attention, working memory and rhythm before editing. When information is abundant, a story’s value shifts from the news it carries to its form, from Simon to McKee.
you need creative campaign? digital world and ai content
A digital creative campaign is not won on the volume of content it produces. It is won on the trace it leaves in memory. Encoding, dual coding, narrative transport, and the reason three out of four award-winning works fail to move market share.
