Model Mitigation and Saturation

• Published on Sep 24, 2025
• 9 minute(s) read

This article is intended for users who continue to experience poor model performance even after completing the initial monitoring and retraining steps. Before using the guidance in this article, make sure you’ve completed the following:

• You’ve identified a reduction in performance using the steps described in Monitoring Model Performance.
• You’ve addressed common model performance issues by applying the steps outlined in Improving Model Performance.

If you’ve completed all of the above and the model still underperforms, you’re likely facing one of two advanced challenges:

• Overcomplicated model— You’re trying to apply your business rules through the Identification model.
• Model saturation—Your dataset is too diverse for the model to generalize effectively.

In this article, you’ll learn how to:

• Distinguish between business logic and identification logic.
• Recognize signs of saturation and complexity.
• Apply strategies such as defining the ingestion of your documents and utilizing a Classification model to meet your needs.

Business logic vs identification logic

To improve performance, it’s critical to separate business logic from identification logic.

• Business logic defines how extracted data should be interpreted or utilized, based on external contexts such as Vendor rules, system processes, or downstream conditions.

• Example: The same Field “Reference Number” may mean “Customer Number” for one vendor and “Account Number” for another. The model cannot determine the meaning; it depends on who sent the Document.

• Identification logic defines what the model should extract and where to find it on the Page, based on relative position to other text.

• Example: Extract the “Reference Number” from a specific zone, regardless of what the number represents.

By keeping business rules out of the model and focusing only on consistent text and patterns, you ensure more reliable results. The next sections explain how to achieve this in practice and how to handle complex datasets.

How to achieve reliable results

To obtain reliable results, it's essential to apply consistent rules when annotating:

• Choose one way of identifying a field across documents and don’t deviate from it, even if the label or exact placement varies slightly.

• Use the same approach across documents that belong to the same group, such as different vendors with similar layouts.

• Avoid encoding business context or subjective decisions in your annotations.

• If a value can only be identified by who sent the document or by other outside knowledge, this is business logic and should be handled outside the model. Learn more about the Annotation process in Training a Semi-structured Model.

Understanding the root cause

Based on the business and identification logic, you can determine the root cause of poor model performance. Most common scenarios are:

• Overcomplicating the model— You’re trying to apply business logic in the model instead of identification logic.
• Model saturation— You’re training the model on a dataset that’s too diverse or complex to generalize from.

The next sections will help you recognize which of these scenarios applies to your case and what to do about it.

Overcomplicating the model

In some cases, model underperformance is caused by how it’s being used. This scenario often arises when the model is expected to perform tasks beyond its design, such as applying business logic or making decisions that should occur outside the platform. The meaning of the extracted data depends on the business context and information outside of the documents. The model can’t learn these kinds of rules.

Mitigating model underperformance

Focus identification logic on what's explicitly on the page (e.g., text and the location) instead of assumptions that require outside context or knowledge. Based on your use case, we recommend the following steps:

• Understand the business goal

• Start by clarifying what you're trying to achieve. Ensure you’ve defined:

• the business rules,

  • the data you need to extract, and
  • the conditions on how the extracted data should be used.

• Move business rules outside of the model

• Apply business conditions, such as “only if status is Delivered” or “use this value only for Vendor X,” in post-processing or using Custom Code Blocks. Learn more in Modifying Custom Code Blocks.

• Review your annotation rules

• Ensure that each rule is based on text and location — not on business conditions or context that is not explicit in the document.

  • Annotate based on consistent dependencies between fields when they’re visible on the page.

• Example: “Always annotate the ‘Bill to’ address as the one below the invoice address.” The model can learn and generalize from such relative positioning. To learn more, see Text Segmentation.

  • Avoid rules that require external context or knowledge outside the document.

• Example: “If the invoice comes from Wales, annotate the address to the right of the invoice address; otherwise, annotate the one below.” This rule depends on information the model cannot see and will lead to poor performance.

• Prioritize Average Handling Time (AHT) over Automation

• Not all documents need to be fully automated. We recommend prioritizing stability and accuracy for high-volume cases. In many cases, it’s more efficient to focus on a portion of the data that covers a large part of your volume, even if it doesn’t completely represent the diversity across documents.

Model saturation

Even if your field definitions are clear and your annotation strategy is consistent, the model may still underperform. In these cases, the issue is often related to dataset complexity, not to logic misuse.

Model saturation happens when a model is trained on documents that are too diverse. There’s so much variation that the model can’t find stable patterns to learn from. Even with correct annotations, the model performs poorly because the task is too broad or too complex.

Handling complex tasks

When a model is trained on too many diverse documents, performance becomes unstable. In these situations, it’s better to reduce the scope or simplify the task. This means breaking it down into several smaller tasks that are easier to manage and train.

• Separate the dataset into smaller groups

• Break down large, diverse datasets into smaller groups of documents that follow a similar pattern. Each group should represent a clearly defined use case with consistent field positioning and formatting.

• Route documents to separate flows using multiple input connections

• When possible, configure multiple folder or queue listeners to route different documents (e.g., documents with different patterns) into dedicated flows.

• Use a Classification model to route documents into layout-based groups.

• Hyperscience provides a Classification model that helps you organize your documents into logical subgroups before training.

Next steps

After you’ve grouped the dataset into documents with similar patterns, you can train a dedicated model for each group following the guidelines described in Training a Semi-structured Model.

To ensure stability and reliable results, keep your dataset within the following platform limits:

• Training Set: up to ~5,000 pages
• Training Data Analysis: up to ~2,000 documents
• Document groups: avoid training one model on more than ~100 groups.

Monitor the performance of your models, based on the steps outlined in Monitoring Model Performance.

Definitions

• Vendor: A third-party entity that conducts business with your company and sends you documents.
• Field: A labeled piece of information you want to capture from a document, like “Name,” “Date of Birth,” or “Total Amount.”
• Document: A group of one or more pages processed as a single unit in Hyperscience.
• Submission: A logical entity in Hyperscience, consisting of one or more pages.
• Annotation: Refers to a user-provided input that defines the correct prediction for a machine learning task.
• Average Handling Time (AHT): A metric that represents the average time it takes to process a submission.
• Automation: Refers to the processing of data without the need for human intervention.
• Accuracy: Measures the effectiveness of the models based on the proportion of correct predictions out of all predictions made.
• Training Data Analysis: A tool that analyzes your training data to compute the importance of each training document.