Version embeddings alongside the source text

A vector without provenance cannot be reproduced, compared or deleted reliably. The same chunk encoded by a different model or normalization may have a different dimension and meaning, even when its ID appears unchanged.

Record a complete embedding identity

The RAG ingestion lab creates two metadata records for one synthetic chunk.

from ingestion_cases import embedding_version_case

result = embedding_version_case()
assert result["versions"] == 2
assert result["distinct_vector_digests"] is True
assert result["embedding_model_executed"] is False
print(result["records"])

Both records link to the same source digest and chunk ID, but use different fictional model names and dimensions. Their vector digests differ. No embedding is calculated; the case verifies the version contract.

Include all transformation inputs

Store source ID/version/digest, page/span, normalized chunk text digest, chunker version, embedding model snapshot, dimensions, normalization, instruction prefix and generation date. If preprocessing or language routing changes, it belongs in the identity too.

Do not overwrite vectors in place during an upgrade. Build a versioned collection or named vector, evaluate it against the current index and switch an approved pointer. Keep old evidence long enough for rollback under policy.

Test compatibility and deletion

Reject a query vector with incompatible dimensions or model space. Ensure every source deletion finds records across all embedding versions. Retrieval logs and eval runs should identify which index version served them.

Re-embedding can change nearest neighbours even when the source does not. Freeze labelled queries, compare per-case transitions and measure cost/storage plus quality. A model release note is not task-specific evidence.

The Generative & Agentic AI course connects embedding provenance to model selection, reindexing and retrieval evaluation.

Exercise

Create two embedding-version manifests for the same corpus. Make one dimension incompatible, verify the client refuses cross-space queries, and demonstrate a version switch plus rollback without losing source lineage.

Continue learning

This article is part of the RAG ingestion and document preparation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Delete a source document from every retrieval index.
• Continue with Choose an embedding model with task-specific evidence.

Reference: Qdrant vector documentation.