Improving Model Performance

• Published on May 6, 2026

• 5 minute(s) read

After identifying a drop in the model’s performance, the next step is to determine its root cause and address it accordingly. This article explains how to use Training Data Management (TDM) to investigate performance issues and improve your model through retraining. To detect performance reductions, follow the steps described in Monitoring Model Performance.

The approach described in this article is designed for cases where the issue is related to new or underrepresented documents with different layout patterns (e.g., different field positions), but the logic behind the identification and annotation remains consistent.

Identification logic and annotation rules

For training to be effective, both identification logic (e.g., which field is captured and how) and annotation rules (how data is labeled) need to remain logically consistent, even if the document pattern varies.

Learn more about the annotation process in Text Segmentation and Training a Semi-structured Model.

When to use this approach

Use this approach when:

• You’ve identified a performance drop through reporting (e.g., reduced machine accuracy or increased manual supervision, as described in Monitoring Model Performance).

• Documents with new layout patterns are submitted to production.

You should not use this approach for use cases involving different annotation rules across documents. If your business logic varies by document type or layout, contact your Hyperscience representative to discuss alternative solutions.

Step 1 - Select and analyze low-performance documents

a. Select representative documents

Select documents from submissions that show low prediction quality, increased Supervision, or repeated QA corrections. These documents often have new or unseen layout patterns that may not be well represented in your training set. We recommend selecting at least 20 samples of each type of document with low performance. To learn how to download documents from the system, see the Document Renderer section in Flow Blocks.

b. Run Training Data Analysis

Upload the samples in Training Data Management (TDM), and use Training Data Analysis to assess whether the documents contributing to performance issues follow a consistent pattern or introduce too many variations for a single model to handle.

Training Data Analysis groups documents based on text and location. It does not explain why a document underperforms, but it helps you assess whether retraining with more examples may improve model performance.

Note that new groups will appear every time you run the Training Data Analysis. To learn more, see Step 4 of our Training a Semi-structured Model article.

After running the analysis, review how the new documents are distributed across the groups. This distribution can help you decide whether to annotate some of them and whether more samples are needed to improve model performance.

c. Review the analysis results

Knowing the number, size, and content of the document groups can help you understand why your model performs the way it does.

• Several large groups suggest consistency in the production data. Adding more annotated examples to the training set and then retraining is likely to help improve performance.

• Many small groups indicate high document diversity or documents that differ in patterns and annotations. Having many small groups may lead to lower automation.

• The same type of document across multiple groups suggests different layout patterns. Revisit those documents and ensure they follow the same annotation logic. Avoid introducing ambiguity in the annotation rules.

As a general rule, we recommend annotating the newly added documents.

• If they belong to an existing large group, no additional action is required.

• If they form a new group, aim to annotate more documents to reach at least 15–20 examples. Annotating this set of documents ensures that the group contributes effectively to model training.

Step 2 - Review annotations

After confirming that the layout pattern of your documents is consistent, the next step is to review the quality of the annotations in your training set. Even when documents are well represented, inconsistent annotations can lead to performance drops. In this step, you’ll evaluate whether the model was trained with complete and representative labels.

a. Review annotations by group

• Go through the groups you’ve selected and review how the target fields are annotated.

• Look for documents where these fields are missing, formatted, or annotated differently.

b. Correct annotations

If you find inconsistencies in the annotations of your training set:

• Annotate missing fields.

• Correct existing annotations if needed.

• Make sure the annotation rules are applied consistently across groups.

• Exclude documents from training if they are not represented consistently across the dataset.

c. Confirm coverage and re-analyze your training set

Make sure that each field is consistently represented in at least 15-20 documents. This representation ensures the model has enough examples to learn from. After making changes to the training set, re-analyze your data to get up-to-date information.

Step 3 - Retrain and evaluate the model

Once you’ve improved your training set by correcting annotations and adding representative examples, the next step is to retrain the model. This retraining generates a new model version that reflects the updated training data.

a. Initiate model retraining

• Make sure your training set now includes consistently annotated, well-represented documents that reflect the observed performance issues.

• In TDM, click Actions > Train new model… to start training a new version of your model.

b. Test the new model version

After retraining, evaluate the model’s performance by using it to process a new set of 15-20 documents that match the structure of the original problematic ones.

c. Interpret the results

If the model shows better accuracy and automation, based on the steps described in Monitoring Model Performance, you can:

• Promote the model to production.

• Continue monitoring performance over time for future shifts.

If performance doesn’t improve:

• Recheck annotations for inconsistencies.

• Confirm you’ve added enough diverse, high-quality examples.

• Contact your Hyperscience representative for support in handling more complex scenarios.