TDM for Classification Models

Classification models are a crucial part of document processing as they help the system determine which layout should be used to process each page you upload. Training Data Management for Classification allows you to add, remove, and update training pages for Semi-structured Classification (also known as NLC) models to achieve more accurate classification results. In this way, TDM helps you maximize the performance of your Classification models.

NLC (Non-structured layout classifier) finds the correct Semi-structured or Additional layout for a given set of submission pages, based on the words in the submitted documents. Note that NLC works on a page level. Learn more in Semi-structured Document Classification.

Each release contains a set of layouts. The creation of a release generates a single Classification model. For example, if you create a release with two layouts, then one Classification model will be generated. It will be trained to identify the document pages submitted through the release’s flow.

• If you create a new layout and create a new release for it, then a new Classification model will be created. Note that you need to add your new layout to a new release or a copy of an existing release for document pages to be matched to that layout. Learn more in Adding a New Release.

TDM for Classification logic

TDM for Classification operates on a document level. However, the Training Data tab displays the number of uploaded documents and the required and recommended number of pages per layout. Learn more in the Training Data Tab section of this article.

TDM for Classification allows you to manage example documents that should be included in or excluded from your model’s training:

• Layouts eligible to train — These are the layouts that meet the minimum number of pages required for training. To ensure this requirement is met, upload documents that:
  • have pages that match your layout and
  • are diverse, but still represent your layout.

Limits, requirements, and recommendations:

• Maximum Number of Training Pages - 100,000 pages
• Maximum Pages per layout - 5,000 pages

You need at least 10 page examples to meet the minimum requirements for model training. Our recommendation for a robust model is 120 page examples per layout. Do not upload the same document multiple times.

• Excluded documents - TDM uses these as examples of documents that you expect to process but don't want to match. They serve as counter-examples of the documents that your model should not classify.

Access TDM for Classification

To access TDM for Classification, go to the Models section. The Classification tab appears by default.

A table with all Classification models appears:

• Filter the Classification Models table by release with the Filter by release drop-down list, which is located on the right-hand side of the page.

• Access the model management page for a particular model by clicking on its name in the table or

  • Access TDM features from the specific release for this model:
    • Go to Library > Releases.
    • Click on the release's name for the model you want to manage training data for.
    • Click View Model in the Automatic Document Classification card.

The Classification models table contains the following columns:

• Model shows the name of your Classification model.
• Compatible Releases indicate the number of releases the Classification model can predict.
• Status displays the model's current state (e.g., Needs Training or Live).
• Date Deployed shows the date the model was deployed.

Using TDM for Classification

Using submission data in TDM. The Send documents to Training Data Management setting for Identification and Classification models allows you to control whether submission data is used for model training. It is disabled by default and can be managed from the System Settings (Administration > System Settings).

Overview tab

Projected Automation Chart

The Projected Automation chart displays the performance of the model that’s currently live. Learn more about these metrics in our Accuracy article.

• Expand it by clicking the arrow button.

The chart displays how the target accuracy affects the automation. The lower the accuracy, the higher the automation, and vice versa. To learn more, see Automation.

Note that projected model performance (i.e., accuracy and automation) can increase by adding more QA records. You can also see the margin of error (MoE) for this model.

Model History table

The Model History table provides a comprehensive overview of your model’s lifecycle. It displays the following columns:

• Name — The name of the last available model for this layout.
• Date Created— Date and time the model was created. Helps in tracking the model’s version history and ensures you’re working with the most recent model version.
• Version — The specific version of the model that was trained on.
• Source — Indicates where the model was trained—either within the current instance or externally and then uploaded to this instance.
• Proj auto — Displays the predicted automation based on the Test Target Accuracy. Learn more in our Evaluating Model Training Results article.
• Docs Trained — The total number of documents used for training the model.
• Last Deploy — The last date and hour the model was deployed.
• Actions — The options in this menu allow you to take the following actions on a model version:
  • Deploy
  • Undeploy
  • Reject Candidate
  • Download

You can find specific records in the table in the following ways:

• Filtering— Filter the contents of the Model History table by creation date, last-deploy date, source, and trainer version. Click Filter and select the criteria that match what you’re looking for.
• Searching — Enter the name of a model version in the search box.
• Sorting — Sort the table's contents by clicking on the names of the following columns:
  • Name
  • Date created
  • Version
  • Source
  • Last deploy

Additionally, you can choose which columns are included in the table by clicking the menu next to the Filter drop-down list and clicking the Manage columns… option. You can also adjust the target accuracy by clicking the up and down arrows located next to the Manage Columns option.

Model Compatibility table

The Model Compatibility table indicates the releases that your model is compatible with and contains the columns described below.

• Release name displays the names of the releases to which your layouts are added. Click on the column header to sort its contents by name.
• Created On shows the creation date of the release. Sort its contents chronologically by clicking on the column header.
• Status indicates whether your release is live or locked. Learn more in What is a Release?. Sort this column’s contents by clicking its header.

You can use the pagination options at the bottom of the table to display all releases that are compatible with your model.

Training Data tab

Training Data Summary card

The Training Data Summary card displays insights on the status of your training dataset.

• Training Data Status indicates your training data's health based on the number of pages uploaded for each layout:
  • Requirements Not Met — The minimum number of required pages uploaded for each layout is 10. If you’ve added less than 10, you won’t be able to run a training.
  • Not Optimized — Hyperscience recommends uploading at least 120 pages to build a robust classification model. If you upload more than 10 but fewer than 120 pages, the status will indicate that your training data can be optimized. However, you’ll still be able to proceed with training.
  • Ready To Train— This status will be displayed after you’ve reached the minimum required and the recommended number of uploaded pages to start a model training.

You need to upload at least 10 examples per layout for the model to learn what documents should be considered as part of your training set. Note that you can run a training without the Excluded examples if you have more than two layouts. However, we recommend adding documents in the Excluded section as well, as they serve as counter-examples.

• Layouts Eligible to train— indicates the number of layouts that meet the minimum requirements for training.
• Pages / Req. / Recomm—displays the number of uploaded pages, the number of required pages, and the number of recommended pages for your model.
• Documents - shows the number of documents uploaded for this model.
• Excluded Pages / Req.— number of pages used as counter-examples. The excluded pages train the model on what should not be matched. Note that they are recommended but not required.
• Excluded Documents— the number of excluded documents.

Training Data Health card

The Training Data Health card displays a breakdown of your dataset. It shows all layouts included in the Classification model, as well as bars next to each layout indicating the number of uploaded pages. Note that you’ll have the required and recommended number of pages for each layout.

Follow the steps below to add training data to your model:

1. Click the Add Training Data button.
2. Select the layout you want to add data to from the Upload To Layout drop-down.
3. Drag and drop your files into the dialog box or click Browse.
4. Once you’ve uploaded your files, click Continue.

Training Data table

The Training Data table displays all documents that can be used as training data for the model.

It contains the following columns:

• Document ID displays the unique ID number of the document.

  • Hover over the preview icon to see the pages of the uploaded document.
  • Freeze the preview by clicking on the preview icon.
  • Page through the document using the arrow keys on your keyboard or the arrows in the preview dialog.

• File Name shows the original document names to help you manage training data more quickly.

  • Search for a document by its original name in the search bar.
  • Document names can come from submissions or from TDM uploads.
  • When importing older training data, the document name field will appear as “N/A,” since document names aren’t included in training data exports.

• Pages indicates the number of pages in the document.

• Layout

• Usage Rule indicates the way the system will use the specific document for training:

  • Always— The document will always be used in future model trainings. It will never be deleted, regardless of the system’s data-deletion settings.
  • Auto— The document may be used in future model trainings until it is automatically deleted according to the system’s data-deletion settings.
  • Never— The document will never be used in future model trainings and will be automatically deleted according to the system’s data-deletion settings. Documents processed through Supervision or QA will always display a status of 'Never' in TDM.
  • Loading— The document has just been uploaded and is going through pre-processing. Once they load, the status will change to 'Auto'.

• Source

  • Upload— The model was uploaded from TDM.
  • Processing—The document is currently being processed.
  • QA— Indicates the legacy documents that were processed through QA in v37 or earlier.

• Scheduled Deletion (Scheduled Del.)—displays the scheduled data deletion, depending on your PII settings.

• [v42.2 and later] Tags — You can organize the training documents by adding labels. Tags can be added manually and used to filter the Training Data table.

Excluded Training Data table

The Excluded Training Data table displays the documents used as counter-examples for your Classification model. The columns are the same as those described above for the Training Data table.

• Excluded documents help train the model on what should not be matched. They serve as counter-examples and are only required if your release contains a single layout.

Training a Classification Model

Follow the steps described below to learn how to train a classification model using TDM.

Upload your documents

To upload documents to TDM for Classification:

1. Click the Add Training Data button on the right-hand side of the Training Data Health card.
2. Choose Upload Files or Import Training Data from the dialog box.

Review your documents

Review your documents using the Training Document View. It helps you match each document to a specific layout.

• Assign a layout to the training document from the Layout drop-down menu on the right-hand side of the page.
• Click Save Changes after you’ve classified your document.