Training a Specialized Model

Training a Specialized Model

ORCA is a Vision Language Model (VLM) that extracts information from documents. To learn more, see ORCA (Optical Reasoning and Cognition Agent) VLMs.

While the base model works out of the box, you can improve its performance by training it on your specific data using annotated documents. ORCA VLMs can be managed and trained directly in Training Data Management (TDM). This article explains how to create a specialized model on top of the ORCA base model for your use case.

ORCA VLM specialization

The ORCA base model provides general-purpose extraction capabilities, but every customer’s documents are different.

Training the model on your documents allows you to:

• Improve extraction Accuracy for your specific layouts. Learn more in Accuracy.
• Increase Automation rates. To learn more, see Automation.
• Reduce the number of fields that require human review.
• Specialize the model to your organization’s document formats.

Training creates a model tailored to your use case while still leveraging the capabilities of the ORCA base model.

Steps to specialize an ORCA model:

1. Upload and annotate training documents.
2. Train the model.
3. Review the candidate model.
4. Deploy the candidate model.
5. Evaluate the candidate model.
6. Retrain if needed or promote the model to production.

Before specializing an ORCA model, ensure that:

• An ORCA base model is installed. To learn more, see Installing ORCA VLMs.
• You’ve configured a Semi-structured layout with the fields you need for your use case. ORCA VLMs extract fields from Semi-structured layouts only.
• The latest Layout Version is locked.
• A model definition exists for the layout. Learn more in Model Definitions.

Upload and annotate training documents

Before specializing an ORCA base model, you must upload and annotate training documents for your layout. Doing so provides the ground-truth values the model will learn from during training.

Dataset requirements

• The uploaded documents should not exceed 5 pages. Larger documents may lead to an Out-of-Memory (OOM) Error. Supporting larger documents requires increasing GPU memory.
• Ensure that all training documents are unique. Duplicate annotated documents are excluded from training.
• If too many documents are excluded, the training process may fail due to insufficient data.
• Include documents that represent the different patterns in your Dataset.
• Annotate enough documents to capture these patterns before training the model.

Annotating documents

To access the VLM Annotations experience, click the Document ID link in the Training Data table for each file you want to annotate.

When annotating documents for adapting the ORCA base model, follow these guidelines:

• Enter text exactly as it appears in the document.

• Avoid formatting or input validation.

• Review pre-populated values carefully.

• Add missing text manually when necessary.

• When a field appears multiple times in a document, annotate the values in natural reading order:

• Top to bottom

  • Left to right
  • First page to last page

Make sure to click Save on each annotated document.

Train the model

During training, the system uses the annotated document values as ground truth and trains a model tailored to your document format.

Number of required documents

• Ensure you have at least 120 annotated documents to train the model.
• For more stable and reliable performance, we recommend using 200+ annotated documents, covering the variety of document patterns in your dataset. Using a larger and more diverse dataset generally improves model performance.

To initiate training:

• Click on Train Model in the Actions drop-down menu.

Training results

Training produces a candidate model that you need to deploy and evaluate against production data. To deploy the candidate:

• On the Model Details page, click History.
• Find your candidate model and click Deploy.

The candidate model will be Live and you’ll be able to submit documents to be processed by the adapted ORCA base model.

Evaluate the candidate model

To evaluate the candidate model, you must first deploy it and then process documents through the system. Follow the steps below:

Using testing documents

We recommend setting aside 50-100 representative documents for testing your model’s performance. Doing so allows you to evaluate how the model performs on realistic data.

Evaluating

1. Deploy the candidate model
2. Run your testing documents through the system with a 100% QA sample rate.
3. Evaluate the model’s performance.

ORCA VLM transcriptions

Unlike traditional Identification models, ORCA VLMs directly generate transcriptions for each field.