Document Drift Management (Layout Triage)

Document Drift Management (Layout Triage)

Document Drift Management (also known as Layout Triage), is a post-processing feature that helps you manage documents that don't match layouts during Classification. When submissions don't meet the Structured Layout Match Threshold or are manually flagged as having incorrect or missing layouts, their pages are marked as unmatched.

This feature replaces the previous "Find Potential Layouts" and "No Layout Found" processes, offering a more streamlined and effective way to handle unmatched pages.

Document Drift Management allows you to determine and create the necessary layouts for these pages, ensuring that similar documents can be accurately classified and processed. In this article, you'll learn how to use Document Drift Management to handle unmatched pages efficiently.

Misclassified vs. Unmatched pages
Misclassified pages are those that the system recognizes but assigns to the wrong layout. You can manually reassign these pages to the correct one. Unmatched pages don't fit any existing layout, so they remain unclassified.

Grouping pages

Document Drift Management works by grouping unmatched pages based on their similarities. It uses a pixel-level analysis to identify pages that share common visual templates. These pages are then automatically clustered into groups, making it easier to identify potential new layouts.

Hyperscience recommends using this feature for Structured use cases.
Document Drift Management effectively handles Structured documents, where consistent formatting is essential for accurate classification.

After the pages are grouped, you can review and adjust them as needed, then create new layouts from these groups.

Using Document Drift Management

To access Document Drift Management, click on Submissions in the left-hand sidebar, and then click the Unmatched tab.

The Unmatched tab displays submissions with pages that did not match any existing layouts during the Classification process. This tab is the starting point for managing and organizing unmatched pages using Document Drift Management.

Step 1 - Unmatched Submissions

The Unmatched Submissions tab displays a table of all submissions with unmatched pages.

Navigating the Unmatched Submissions tab

Filters

Define a specific range of submission IDs by entering values for its beginning and end:

Managing Columns

You can add and remove columns by clicking on the table’s menu.

Columns

The Unmatched Submissions table displays the following columns:

Grouping Pages

Follow the steps below to run a grouping job:

  1. Select the submissions containing the pages you want to group.
  2. After you’ve made your selections, click Group Pages. The system initiates a visual clustering process, grouping pages based on pixel-level similarities.
  3. After the grouping job is complete, click on the Page Groups tab.

Step 2 - Group Pages

The Page Groups tab displays a list of all clusters of similar pages that have been grouped. It helps you manage these groups, review ungrouped pages, and refine them before creating new layouts.

You need a minimum of 5 similar pages to create a group. If you have less than 5 similar pages, they will go to the Ungrouped category. However, you can manually create groups with fewer pages.

Navigating the Page Groups tab

The Page Groups table contains the following columns:

Layout Triage

Organize unmatched pages into layouts based on visual similarity using the triage experience.

Best for Structured documents. Similar pages of Semi-structured documents may be grouped together. Extra pages, like fax cover sheets, and those with fewer than 5 similar pages will go to the Ungrouped category.

To start the triage process, select the groups you want to manage, then click Triage Groups.

Follow the steps provided in the in-product guidance to begin the triage process.

Step 1: Review group

While pages are automatically grouped based on visual similarities, it's important to manually review and adjust these groups to ensure accuracy. To create a group:

  1. Click on a group in the To Do list on the left to view its contents and start the triage process.
  2. Click on a page to display it in the preview section.
  3. Select multiple pages and click New Group from Selection to create a more precise grouping.

Step 2: Enter layout metadata
You can define the following properties of the layout:

Step 3: Provide blank When creating a new Structured layout, provide a blank version of each page.

To set a blank form:

  1. Select the page you want to define as blank form.
  2. Click Set as blank.

OR

  1. Upload a blank from your local machine by clicking Browse.
  2. Select your file.
  3. Click Upload.

Triage Ungrouped

The number of ungrouped pages will increase over time. You can triage them when you have enough similar examples.

  1. Review pages — Review the pages in the Ungrouped folder and delete any unnecessary pages.
  2. Manually create page groups — Select sets of pages in the Ungrouped folder to manually create additional groups as needed.
  3. Follow the steps above for manually created page groups.

Similar Layout Variation Detection - Beta Feature

Structured layouts evolve as document versions change, often resulting in unmatched submissions and reduced automation rates.

Example use case A common example is the U.S. tax form W-8, which undergoes periodic updates while maintaining its overall structure.

With Similar Layout Variation Detection:

Using Similar Layout Variation Detection

If enabled, Similar Layout Variation Detection will be available on the Page Groups tab. You’ll see an additional column called Similar variation that indicates whether a group resembles an existing layout or no similar variations were found.

Fields Overlay Use the Fields Overlay feature in v41.1 and later to verify that the document is a variation of an existing layout.

To compare the fields of the selected variation with those in your document:

  1. Enter the triage experience for a group that the system has detected a similar layout variation for.
  2. Click on 2 - Enter layout metadata.
  3. Click the Toggle field overlay button to review the fields.
  4. Continue the triage process.

Working across environments

To ensure smooth layout management and prevent data loss or overwrites, it's important to follow a structured workflow when working with layout variations across lower (e.g., development, UAT) and production environments.

Limitations