Updating a search index: compare old and new generations and retain cutover evidence

THE TAKEAWAYS

What to take away

Check ingestion acceptance, search availability, and retrieval of required documents separately.
Fix the comparison questions and source versions so differences can be explained.
Include post-cutover source updates and rollback conditions in the operational handover.

EDITORIAL PRACTICE

Describe what changed in this generation

An index update can combine added source material, different chunking, retrieval fields, and embedding settings. We recommend documenting what changes and what stays fixed before starting comparisons. Calling an index merely 'new' makes it difficult to explain whether different results come from additional material or altered configuration.

Record the scope and reference date of source capture, the transformation version, and the index identifier. A checklist does not need copies of confidential source documents. Provide references that support reconciliation within the existing permission boundary. Likewise, do not count an index recreated under a different name as evidence that the underlying documents have become newer.

EDITORIAL PRACTICE

Separate documented update methods from your business cutover procedure

For production schema changes, Microsoft's Azure AI Search documentation discusses creating and testing another index alongside the existing one, then switching through an alias.[1] Google Cloud's documentation describes complete replacement using isCompleteOverwrite.[2] These describe product capabilities; they do not mean that complete replacement removes the need for business verification.

An implementer must check whether the chosen product can retain parallel indexes, how references are switched, and what permissions and cost that requires. Do not guarantee uninterrupted service or immediate propagation when the documentation does not establish it. If the current contract or capacity does not support parallel retention, decide an acceptable interruption and an approved procedure before adding another environment.

EDITORIAL PRACTICE

Use the same questions and inspect more than counts

Choose comparison questions from actual uses. For internal policies, examples might include frequently used procedures, alternate terminology, recently revised policies, and documents with different access rights. These are illustrative examples from the editors. One approach is to begin with a small comparison set that links expected source documents to reviewers, rather than submitting a large collection of questions with unknown answers.

For each question, record the source identifiers returned by each generation, whether required material is present, and the reason for differences. Do not classify every ranking change as a defect: establish whether source revisions, chunking, or retrieval conditions explain it. If answer generation is included, evaluate retrieved evidence separately from generated prose so that generation issues are not attributed to the index.

EDITORIAL PRACTICE

Account for source changes made during comparison

Sources continue to be added, corrected, and removed while the new index is built and tested. Without accounting for the difference between the comparison date and cutover, an index that passed testing may already be stale when activated. Our design proposal is to track changes after the reference date and record the point through which they were applied before cutover.

Check deletions and permission changes separately from additions. A document absent from one result view might remain visible through another interface or cache, so verify the paths users actually follow. Preserve the reasons for changes whose propagation is still unconfirmed. Do not interpret missing data as zero results, or substitute an observation timestamp for the source's actual modification time.

EDITORIAL PRACTICE

Define the conditions that would require a rollback

Agree in advance on the cutover decision owner, observed indicators, rollback conditions, and old-generation retention. Possible conditions include missing critical material, results outside a user's permissions, or continuing latency beyond the accepted range. These are not numeric recommendations. Set your own thresholds from business importance and measurements.

In an offshore handover, share inspection of production results and source documents, not just the screen confirming the cutover operation. Explicitly state if rollback has not actually been tested. Even a retained old index may no longer be suitable for a simple return if later source updates were not applied to it. Distinguish having a stored copy from being able to resume business use.

WITH OPERUN.

Delegate everyday AI management.

Judge a search update complete after confirming that required material can be retrieved under the appropriate conditions, rather than merely after creating a new index. Check the product's update method and align comparison questions, source deltas, cutover, and rollback conditions. Alongside ingestion counts, evidence from inspecting real search results supports decisions after a change of personnel. Record the index identifier currently used by production, and read back that the actual search target matches the reference you intended to switch.

Discuss your product and AI work

Primary sources

Sources checked on 8 November 2026. The workflows and checklists are editorial proposals.