ORCA (Optical Reasoning and Cognition Agent) VLMs

• Updated on Apr 24, 2026
• Published on Jul 9, 2025
• 6 minute(s) read

Our ORCA (Optical Reasoning and Cognition Agent) Vision Language Models (VLMs) leverage the power of GPUs to find and extract data in documents. In this article, you'll learn how to implement these VLMs and use the VLM features available in v41.2.

Benefits and considerations

In Hyperscience, ORCA VLMs work "out of the box" to extract data from documents. Because they do not require training to extract data, the use of VLMs can reduce implementation times, making them a valuable option for use cases that need to be set up as quickly as possible or that prevent models from being trained. This flexibility, along with their ability to detect visual elements (e.g., stamps, signatures), expands the data-extraction capabilities of the Hyperscience Platform.

All VLMs, ORCA included, require GPU hardware. This hardware may need to be purchased for on-premise implementations or provisioned through a cloud provider. Note that ORCA requires a minimum GPU of 24GB of VRAM for inference and a minimum GPU of 24GB (48GB recommended) for fine-tuning. Use of GPUs with VRAM under these respective amounts are unsupported by ORCA.

In v41.2, Hyperscience provides assistance with the implementation of VLMs. If they've determined that using ORCA VLMs is the best option for your specific use case, your Hyperscience representative will give you the flow required to incorporate VLMs in the processing of your submissions.

Infrastructure requirements

An application machine with a GPU is required in order to use ORCA VLMs. If you need to fine-tune ORCA model definitions, you can do so with a trainer machine that has a GPU. To learn more about the technical requirements for GPUs, see Infrastructure Requirements (for Docker and Podman) or Kubernetes Installation Overview.

Features available in v41.2

You can take advantage of the following features when using ORCA VLMs in v41.2.

Fine-tuning

With the sample documents you provide, the Hyperscience team will perform annotations, which are then used to generate an archive of use-case-specific weights. The weights are ingested by the layout's VLM flow in your instance, helping the model to detect and transcribe data more accurately than it would have otherwise.

Requirements

• Sample documents — The team needs at least 40 representative documents (1-3 pages each) for each layout you intend to use VLMs with. Providing additional documents is recommended but not required to complete fine-tuning. While you can use VLMs to process documents containing more than 3 pages, those documents cannot be used for fine-tuning.
• A list of fields you want to extract from the documents — This list of fields informs the layout-creation process.
• A list of business- or use-case-specific rules used to process the documents — These rules help the team to make relevant and accurate annotations.

Thresholding and Quality Assurance

Thresholding allows you to set a target accuracy for the VLM's output. This target determines the volume of Supervision tasks the system generates, as well as which fields are sent to Supervision and which can be processed automatically. In order to find the threshold for a given target accuracy, the Hyperscience team completes a set of Vision Language Model Quality Assurance (VLM QA) tasks in your instance.

Keyers at your organization can also complete VLM QA tasks after the VLM has been implemented and used to process submissions (see Vision Language Model Quality Assurance for more information). The results of these tasks are used to determine the accuracy of the VLM's layout, which can be found in the Manual Accuracy vs. Machine Accuracy Report ( Reporting > Accuracy). To learn more about this report, see Manual Accuracy vs. Machine Accuracy.

Requirements

• VLM flows — A separate copy of the VLM flow ("Vision Language Model Flow via GPU') for each layout that will be used with the VLM. You will use these flows to process documents that have those layouts.

Supervision

The system generates Flexible Extraction tasks for fields that it makes low-confidence predictions for. To learn more about Flexible Extraction tasks, see Transcription.

Requirements

• Completion of thresholding — Because thresholds help determine which fields are sent to Supervision, thresholding work needs to be completed before the system can generate Supervision tasks.

Setting up ORCA

In order to use ORCA in the processing of your submissions, you first need to configure your system’s infrastructure and install the VLM.

1. Configure your infrastructure for ORCA.

• If you have an on-premise deployment of Hyperscience, follow the instructions in our "Enabling Application Machines with GPUs" article for Docker, Podman, or Kubernetes.
• If you have a SaaS deployment of Hyperscience, contact your Hyperscience representative to have them make any necessary changes to your infrastructure.

2. Install ORCA with a flow that contains the Install LLM/VLM Block.

The ORCA VLM is installed when a submission is processed through the "Vision Language Model Flow via GPU," the “LLM Install Flow,” or through a custom flow that includes the Install LLM/VLM Block. You can obtain the "Vision Language Model Flow via GPU" or “LLM Install Flow” from your Hyperscience representative.

The installation process typically takes about 10-20 minutes and needs to be completed only once. For more information on the installation process, see Flow Blocks.

Note that, if you used a VLM installed with the Install LLM/VLM Block in v41.0.1-41.0.9 or v41.1.0-v41.1.3, you need to change the model_name in your flow’s code from HSVLM to ORCA and the model_uuid to 14554188-cf8e-4f10-9057-d1df2f710072 after upgrading to a later version.

Using ORCA to process submissions

Unless you are using ORCA through only the ORCA Composite Block, your flow needs to have a release with at least one layout in order to use ORCA.

Note that ORCA can extract data from fields only; it cannot be used to extract data from tables.

“Vision Language Model Flow via GPU” settings

The settings available in the “Vision Language Model Flow via GPU” flow are listed below according to their type. To view settings of a particular type, select that type from the Settings Type drop-down list in the flow’s settings.

LLM Install

Vision Language Model

Quality Assurance