Your Vector Database Is Not an Anonymous Copy of Your Documents

A vector database can look safer than a document store. Instead of readable emails, support tickets, contracts, or notes, it contains long rows of numbers called embeddings.

That does not make the underlying information anonymous.

A research team from Cornell Tech and the University of Pennsylvania has shown a new way to translate embeddings from an unknown model into the space of a known model. Their method does not need the original text, access to the original encoder, or a list pairing the unknown vectors with known examples.[1]

The practical lesson is simple: treat embeddings as sensitive derivatives of the documents that created them.

First, what is an embedding?

An embedding is a numerical representation of meaning.

Imagine a map where nearby places are related. On an embedding map, a customer complaint about a late delivery may sit near other delivery complaints. A renewal email may sit near other commercial conversations. The coordinates are not readable sentences, but their position still carries information about the source.

This is why embeddings are useful. They help systems find related material even when two passages do not use the same words. They power semantic search, recommendations, classification, clustering, and many retrieval-augmented generation systems.[1]

The same property creates the privacy problem. An embedding is valuable precisely because it preserves meaning.

What the researchers built

The researchers call their method vec2vec.

Different embedding models normally produce incompatible coordinates. It is like having two maps with different scales, orientations, and coordinate systems. A point from one map cannot simply be dropped onto the other.

Vec2vec learns a translation between those maps. It studies the overall geometry of two collections of embeddings and learns a shared representation between them. Crucially, the method does not require the same documents to appear in both collections.[1]

In the paper's experiments, some translations reached cosine similarity as high as 0.96 against the target vectors. The method also achieved perfect matching on a test involving more than 8,000 shuffled embeddings in some model pairings.[1]

Those results do not mean every vector database can be perfectly decoded. They do show that changing the embedding model, hiding the encoder, or removing the original text may provide less protection than teams assume.

Why translation matters

Many techniques for extracting information from embeddings are designed for a known model. If someone only has a database of vectors from an unknown model, those techniques may not work directly.

Vec2vec changes that situation. It can translate unknown embeddings into the space of a model the researcher understands. Existing classification or inversion methods can then be applied to the translated vectors.[1]

The paper demonstrates two kinds of leakage:

1. Attribute inference. The translated embedding can reveal categories or attributes associated with the original text. The experiments included topics from tweets and disease descriptions associated with medical records.[1]
2. Text inversion. A model attempts to reconstruct information from the source text. In selected model pairings, the researchers' off-the-shelf inversion method extracted information from as many as 80% of the tested emails and 67% of the tested tweets. The reconstructions were imperfect, but they sometimes exposed names, dates, financial details, outages, promotions, and other sensitive context.[1]

This was a controlled research setting. The medical dataset was pseudo re-identified, the email corpus came from the defunct Enron organization, and the paper describes the results as a lower bound on what improved translation and inversion methods may eventually achieve.[1]

The easy mistake: treating vectors as anonymized data

Teams often apply one of these mental shortcuts:

• “The database only contains numbers.”
• “An attacker would need to know which embedding model we used.”
• “The original documents live somewhere else.”
• “We can safely give a vendor the vectors without giving them the text.”

The research weakens all four assumptions.

Embeddings are not copies of the source documents, but they are also not random identifiers. They are structured representations designed to preserve relationships and meaning. The model name and original encoder may be unknown, yet the geometry across a large collection can still help reveal what the vectors represent.[1]

A better rule is: if the source text would be sensitive, the embedding should inherit the same sensitivity until a real risk assessment proves otherwise.

A practical vector database security checklist

You do not need to stop using vector search. You do need to include it in the same security program as the source data.

1. Inventory what becomes an embedding

List every source that feeds your vector database. Include inboxes, call transcripts, support tickets, contracts, internal documents, health information, customer records, and generated summaries.

Do not describe the source as “knowledge.” Name the actual records. If your team is still deciding where AI belongs, review the available agent features before mapping the data each workflow may touch.

2. Keep source-level access boundaries

Do not combine documents with different permission levels into one unrestricted vector collection merely because the records are now embeddings.

If a salesperson cannot read a legal file in the source system, semantic search should not quietly make that file discoverable. Preserve tenant, workspace, department, and document-level controls through ingestion, retrieval, and deletion.

3. Minimize before embedding

Remove information the search task does not require. A support-answer system may not need full payment details, personal identifiers, or an entire email thread.

Choose the smallest useful unit of text. Store references back to the controlled source when possible instead of duplicating every sensitive field in the retrieval layer.

4. Protect the vector database like the source database

Use encryption in transit and at rest, narrow service credentials, private network access where appropriate, tenant isolation, audit logs, and tested backup controls.

Review every application or vendor that can reach the data. A convenient integration expands the trust boundary, so keep an inventory of connected systems through a controlled connector catalog rather than relying on memory.

5. Log retrieval, not just document access

A system can leak information through repeated searches even when no one downloads a complete file. Record which identity searched, which collection was queried, which records were retrieved, and what downstream system received the result.

For agent-run workflows, inspect the execution path instead of trusting the final answer alone. A readable agent trace helps show which tools and records were involved.

6. Test deletion all the way through

Deleting a source document should remove its chunks, embeddings, caches, replicas, and derived indexes. Verify the full path rather than assuming a connector handled it.

Run a sample deletion and confirm that the content no longer appears in retrieval results or backups beyond the documented retention period.

7. Reassess vendor and sharing decisions

Ask whether a vendor can export embeddings, combine them across customers, retain them after termination, or use them to improve another model. Treat an embedding export as a data export, not harmless analytics.

For high-risk collections, get a security review before sharing vectors outside the original trust boundary.

What this paper does not prove

The research is an arXiv preprint, not a claim that every embedding can be reconstructed perfectly.

The authors assume access to a second embedding model and high-level knowledge such as the modality and language of the hidden documents. Performance varies by model pairing and dataset. The paper's reconstructed text is often partial or inaccurate, and the reported percentages come from specific experiments rather than a universal success rate.[1]

These limits matter. The right response is not panic. It is to stop treating the absence of readable text as a security control.

The safer mental model

Think of an embedding as a compressed semantic fingerprint.

It is different from the original document, but it still exists to preserve useful information about that document. Better tools may recover more of that information over time.

A secure AI workflow therefore asks three questions before storing vectors:

1. What sensitive meaning could this representation retain?
2. Who can access the collection and the systems connected to it?
3. Can we prove that access, retention, and deletion controls work end to end?

If your team needs help mapping the documents, tools, permissions, and review points in an AI workflow, book an AI Readiness Audit.

Sources

[1] https://arxiv.org/abs/2505.12540v4