Does Retrieval Make Generated Copy Sound Like the Client?

The claim that retrieval-augmented generation makes a model write more like a particular brand is easy to assert and rarely measured. This study measured it, on a corpus of 1,091 genuinely published items (932 social captions and 159 web passages across eight clients) split into a 772-item retrieval pool and a 160-item held-out test set.

On retrieval quality, dense embedding search scored 0.765 with a 95% bootstrap confidence interval of [0.756, 0.774], against 0.730 for BM25 and 0.754 for a hybrid. On the generation side, retrieval raised brand-voice similarity for every one of the eight clients, from a 0.700 baseline to between 0.737 and 0.759 depending on the variant, and the effect survived Holm correction for multiple comparisons at p < 0.001. A plagiarism guard confirmed that none of 96 generations was a near-copy of any source caption.

The result I am most willing to be judged on is a negative one. A style filter applied at retrieval time drove exemplar style violations to exactly 0.000 in all 72 brief-by-strategy cells, at a relevance cost of 0.021 to 0.025. The same filter applied at generation time did nothing whatsoever, an adjusted p-value of 1.0. Two placements of the same idea, one worth keeping and one worth deleting.

The awkward part is the correction. An earlier version of this work reported significance across three comparisons at p-values of 0.029, 0.0383 and 0.1303. Applying the same Holm correction to those figures gives 0.087, 0.087 and 0.130, which means the earlier claim did not hold and the report now retracts it explicitly rather than quietly restating the number. Rerunning the analysis surfaced six defects in my own pipeline, four of which had put a wrong number in a document. The worst was that the real published posts were sitting inside the retrieval pool they were being scored against, twenty of twenty-one at a cosine above 0.99: the naive figure read 0.8140 and the corrected one 0.7652, so the error was larger than the entire effect being measured.

The system went into daily use with a team of five at the end of July 2026.