Delete a source document from every retrieval index

Removing a file from source storage does not remove its chunks from lexical search, vector collections, caches or evaluation snapshots. A deletion workflow needs lineage and proof that active retrieval can no longer return the source.

Delete by stable source identity

The RAG ingestion lab removes synthetic refund-v1 from four stores.

from ingestion_cases import deletion_case

result = deletion_case()
assert result["source"] == "refund-v1"
assert result["complete"] is True
assert all(values == [] for values in result["residual"].values())
assert result["before_counts"] == {"source": 4, "chunks": 4, "lexical": 4, "embeddings": 4}

Source, chunks, lexical entry and embedding record all disappear in the fixture. This local dictionary operation does not prove deletion from a live service, backup or provider.

Maintain a deletion map

Every derived record should carry source ID and version. Inventory all stores: raw/processed objects, metadata database, lexical/vector indexes, caches, logs, offline exports and queued jobs. Define which copies are deleted, access-restricted or retained under policy.

Use idempotent deletion so a retry produces the same final state. Stop new ingestion for the source, remove derived records, invalidate caches and record status per store. Verify with exact identifiers and test retrieval queries that previously returned the document.

Handle active readers and backups

Versioned indexes complicate deletion: an inactive old index can reintroduce the source during rollback. Apply tombstones or rebuild every eligible index before it can become active. Document backup retention and restoration controls instead of claiming immediate physical erasure where it does not occur.

Authorization changes may require faster removal than ordinary refresh. Define service levels and incident ownership. Protect deletion endpoints with authenticated authority; retrieved text cannot request its own deletion.

The Generative & Agentic AI course includes deletion evidence in RAG data contracts and operational runbooks.

Exercise

Create a source with five chunks in two index versions and a cache. Execute deletion twice, prove no active or rollback path returns it, and save a per-store manifest without exposing the deleted content.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Track document freshness in a RAG pipeline.
• Continue with Version embeddings alongside the source text.

Reference: Qdrant point-management documentation.