Deleted, But It Remembers
A client asks you to delete their files. The file is gone from the folder, but the bot still quotes it. One version deletes the file. The other deletes everything the bot kept from it too.
01Try it
A client asked for its files to be deleted. You work at the studio and ask the company bot about that client. Ask first, delete the files, then ask again and see which bot still remembers.
- Audit write-up (internal)
- Proposal: design system audit
- Email: can you do the fixes?
- Email: delete our fileskept as proof
The chat history shows up here.
The chat history shows up here.
Real: the files, the search and the AI. The deletion only happens for you, so other visitors still see Halden's files. Reload the page to start again.
What happened every time
Each test: ask, delete, ask again, then ask in other words. Pink means the bot quoted a deleted file after the deletion. Click a question to read the answers.
What did the Halden audit find?How many duplicate components were in Halden's Figma library?
Halden's audit found 38 duplicates among the 210 components reviewed [1]. These were components that differed in name, or in a value that should have been a variant, rather than in purpose [1].
I don't know based on the Northwind documents.
Did Halden ask us to do the fixes after the audit?Who was going to fix the problems the Halden audit found?
Halden's own team was going to fix the problems the audit found, using the written review as their guide [1][4]. The audit itself did not include the fixes, and any fixes Northwind did would have needed a separate retainer engagement [3][4].
The sources don't say who was going to fix the problems the Halden audit found. Halden's files were scheduled for deletion [1], and the Design System Audit is described as a written review that does not include doing the fixes [3]. Beyond that, the documents do not identify who would address the audit's findings.
What was in the Halden Health proposal?What did we quote Halden for the design system audit?
The design system audit quoted to Halden Health was €3,800 excluding VAT for two weeks of written review, code NW-AUD-DS [1][2]. The fee covers the written review and prioritised list only, not the fixes, which would need Halden's own team or a separate retainer [2][3].
I don't know based on the Northwind documents.
What actually happened
Deleted from the folder, still in the bot
After the deletion, the careless bot quoted a deleted file in 9 of 9 answers to the same question, and 9 of 9 when it was asked in other words. The folder was empty.
Saved answers don't check
Asked the same question again, the bot simply repeated its saved answer, deleted file and all. It never looked anything up, so cleaning the bot's copies wouldn't have helped. Saved answers have to be deleted too.
Its copies were still there
Asked in other words, the bot searched again and found the copies it had made of the files when they were added. Deleting the file didn't delete those copies.
The careful bot really forgot
The careful bot didn't quote a deleted file once in 18 answers. Sometimes it mentioned the deletion instead, from the one email kept as proof.
Old links still opened
Click a [1] in an old answer after the deletion. The careless one still opens the file. The careful one checks first and says it was deleted.
The code
What the two approaches do, in one sentence. The code itself is one click away.
The careless version deletes the file and stops. The careful one also deletes the bot's copies and its saved answers, and old links check before they open anything.
Show the code anyway
// "Please delete our files."await sql("DELETE FROM harbor.files WHERE project_id = $1", [project]);await confirmInWriting(client);await withTx(async (q) => { await q("DELETE FROM harbor.files WHERE project_id = $1", [project]); await q("DELETE FROM rag.chunks WHERE doc_slug = ANY($1)", [docs]); await q(`DELETE FROM answer_cache WHERE EXISTS ( SELECT 1 FROM jsonb_array_elements(sources) s WHERE s->>'doc' = ANY($1))`, [docs]); await q("INSERT INTO deletions (client, docs) VALUES ($1, $2)", [client, docs]);});// an old citation, clicked laterif (await isDeleted(doc)) return new Response("Gone", { status: 410 });Still curious?
A few more questions people ask. Open what you need.
Where else could a deleted file be hiding?
In the bot's copies, saved answers, logs, backups and on the laptops of people who downloaded it. Make the list when you build the feature. Sofie's confirmation says what stays: invoices, which the law says to keep, and copies the client downloaded.
Is every trace of Halden gone?
No. Some of the studio's own notes still mention Halden by name. They aren't Halden's files. Whether to delete those is a decision for a person, not the computer.
Is this what privacy law asks for?
It's the same work privacy laws ask for: find every copy, delete it, confirm it. Doing it in this lab doesn't make anyone compliant. Halden asked under its contract, which allows it.
What if someone just loses access?
Same idea: every lookup and every link has to check at that moment, not trust what was true when it was made. Chapter 6 is about exactly that.
Who decides it remembered?
A simple check in code: did an answer after the deletion point to a deleted file? One run of the test, not repeated until it looked good.
Not sure this could happen to you?
That's what the call is for. Tell me how your product works, and I'll tell you what applies to you and what I'd check first.