Document Drift Management (Layout Triage)

• Updated on Nov 7, 2024
• Published on Oct 10, 2024
• 9 minute(s) read

Document Drift Management (also known as Layout Triage), is a post-processing feature that helps you manage documents that don't match layouts during Classification. When submissions don't meet the Structured Layout Match Threshold or are manually flagged as having incorrect or missing layouts, their pages are marked as unmatched.

This feature replaces the previous "Find Potential Layouts" and "No Layout Found" processes, offering a more streamlined and effective way to handle unmatched pages.

Document Drift Management allows you to determine and create the necessary layouts for these pages, ensuring that similar documents can be accurately classified and processed. In this article, you'll learn how to use Document Drift Management to handle unmatched pages efficiently.

Grouping pages

Document Drift Management works by grouping unmatched pages based on their similarities. It uses a pixel-level analysis to identify pages that share common visual templates. These pages are then automatically clustered into groups, making it easier to identify potential new layouts.

Hyperscience recommends using this feature for Structured use cases. Document Drift Management effectively handles Structured documents, where consistent formatting is essential for accurate classification.

After the pages are grouped, you can review and adjust them as needed, then create new layouts from these groups.

Using Document Drift Management

To access Document Drift Management, click on Submissions in the left-hand sidebar, and then click the Unmatched tab.

The Unmatched tab displays submissions with pages that did not match any existing layouts during the Classification process. This tab is the starting point for managing and organizing unmatched pages using Document Drift Management.

Step 1: Unmatched Submissions

The Unmatched Submissions tab displays a table of all submissions with unmatched pages.

Filters

Define a specific range of submission IDs by entering values for its beginning and end:

• Submission ID Start
• Submission ID End
• Submission Date— Choose a date range you want to see results for.

Managing Columns

Manage the table’s columns from its menu.

Columns

The Unmatched Submissions table displays the following columns:

• Submission ID — The specific ID of the submission
• Submission Date— The date the submission was uploaded
• Unmatched/Total Pages— The number of unmatched pages and the total pages in the submission
• Unmatched Page IDs— Displays the specific ID of the unmatched pages within the submission. You can also see each unmatched page with the preview button. Click it to lock the preview, and use keyboard arrows to go through pages.
• Grouping Job — The consecutive ID number of the clustering job. Also indicates the job’s status:
  • Queued — The Grouping job is pending.
  • Running — The Grouping job is currently processing.
  • Failed — The Grouping job failed.
  • Canceled — The Grouping job was canceled by the user.

Grouping Pages

Follow the steps below to run a grouping job:

1. Select the submissions containing the pages you want to group. Check the pages by using the preview button.
2. After you’ve made your selections, click Group Pages. The system initiates a visual clustering process, grouping pages based on pixel-level similarities.
3. After the grouping job is complete, click on the Page Groups tab.

Step 2: Group Pages

The Page Groups tab displays a list of all clusters of similar pages that have been grouped together. It helps you manage these groups, review ungrouped pages, and refine them before creating new layouts.

You need a minimum of 5 similar pages to create a group. If you have less than 5 similar pages, they will go to the Ungrouped category. However, you can manually create groups with fewer pages.

Navigating the Page Groups tab

The Page Groups table contains the following columns:

• Group Name— The name of the Clustering.
• Updated — The date the group was updated and the number of additional pages added after the last update.
• Pages — The number of pages in the group.
• Actions — A menu with options to download pages or to archive or delete the group.

Triage Groups

Organize unmatched pages into layouts based on visual similarity using the triage experience.

To start the triage process, select the groups you want to manage, then click Triage Groups.

Follow the steps provided in the in-product guidance to begin the triage process.

Triage similar page groups

While pages are automatically grouped based on visual similarities, it's important to manually review and adjust these groups to ensure accuracy. To create a group:

1. Click on a group in the list on the left to view its contents and start the triage process.
2. Click on a page to display it in the preview section.
3. Select multiple pages and click New Group from Selection to create a more precise grouping of those pages.
4. Send the selected pages to the Unmatched group by clicking the button under the page preview.

Define layout properties

You can define the following properties of the layout:

• Layout Name — Typically the name of the form
• Layout Code — Serves as a "note" that helps identify the layout during triage.
• Layout Type — Choose the layout type based on your documents.
• Index — The page index can help identify if any pages are missing from a group.

Click Save

Ensure all properties are filled out, as they are required to proceed with the triage process and save the changes.

Define Blank form

When creating a new Structured layout, provide a blank version of each page. For best results, use an original form that hasn't been filled in. Clustering is most effective with Structured documents.

Triage Ungrouped

The number of ungrouped pages will increase over time. You can triage them when you have enough similar examples.

1. Review pages - Review the pages in the Ungrouped folder and delete any unnecessary pages, like fax cover sheets or blank sheets.
2. Manually create page groups.

Step 3 - Create Potential Layouts

Once you have triaged your groups, they will be ready for layout creation. Find them in the Potential Layouts tab.

• Create a layout based on a group by clicking the Create Layout button.
• From the Actions menu, you can edit the name of your layout, or you can archive or delete it.

Creating blank forms

To create a blank manually:

1. Open the Layout Editor.
2. Click Edit Layout variation to access the blanking tool.
3. Draw redaction boxes over areas of the image you would like to remove from the layout variation.
4. Click Preview to see the form with the blank fields.
5. Click Save Changes.

Limitations

This section outlines the known limitations of the Document Drift Management feature:

• Uploading blanks— Only the first page of a multi-page blank PDF will be uploaded.
• Layout variations — Not directly supported within Document Drift Management.
• Clustering Job minimum — Clustering works with a minimum of 5 pages per group.
• Clustering Job Limit: A maximum of 10,000 pages at a time.
• Triage Group Limit: A maximum of 500 groups can be triaged at once.