Helm Chart

Helm Chart

Prerequisites

Before attempting to install Hyperscience, please be sure to follow the infrastructure requirements and guidelines in the Kubernetes Installation Overview to ensure that your cluster is compliant with Hyperscience's requirements.

Then, follow the hsk8s (Hyperscience Kubernetes CLI) instructions to install hsk8s and Helm repo.

Make sure that you have imported environment variables from the previous step:

source hs_env.bash

Create a values.yaml file

Use the examples below to create a values.yaml file for AWS or GCP.

Minimal values.yaml file for AWS

secrets:
  platform: "hyperscience-platform" # required: platform secret

app:
  repository: "0123456789.dkr.ecr.us-east-1.amazonaws.com/forms" # required
  tag: "42.0.14"
  dotenv:
    FORMS_DB_TYPE: postgres
    FORMS_DB_HOST: hyperscience.xxxxxxxx.us-east-1.rds.amazonaws.com  # your RDS database's endpoint
  storage_mode:
    s3:
      bucket: my-hyperscience-bucket # your S3 bucket, if using S3 or GCS as Object storage
      prefix: my-hyperscience-prefix # Optional
  secret_env_vars:
  - name: AWS_ACCESS_KEY_ID
    valueFrom:
      secretKeyRef:
        key: AWS_ACCESS_KEY_ID
        name: hyperscience-platform
  - name: AWS_SECRET_ACCESS_KEY
    valueFrom:
      secretKeyRef:
        key: AWS_SECRET_ACCESS_KEY
        name: hyperscience-platform

serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: arn:aws:iam::0123456789:role/my-hypescience-role

blocks:
  repository: "0123456789.dkr.ecr.us-east-1.amazonaws.com/sdm_blocks"

operator:
  repository: "0123456789.dkr.ecr.us-east-1.amazonaws.com/hyperoperator" # required

trainer:
  repository: 0123456789.dkr.ecr.us-east-1.amazonaws.com/trainer # required
  tags:
  - 42.0.14

Minimal values.yaml file for GCP

secrets:
  platform: hyperscience-platform # required: "platform secret"

app:
  repository: us-central1-docker.pkg.dev/gcp-project-name/hyperscience/forms # required
  tag: 42.0.14
  dotenv:
    FORMS_DB_TYPE: postgres
    FORMS_DB_HOST: xxxxxxxx.xxxxxxxx.us-central1.sql.goog.
  storage_mode:
    gcs:
      bucket: my-hyperscience-bucket
      prefix: my-hyperscience-prefix
  secret_env_vars:
  - name: FILE_STORE_GOOGLE_CLOUD_KEY
    valueFrom:
      secretKeyRef:
        key: FILE_STORE_GOOGLE_CLOUD_KEY
        name: hyperscience-platform
  serviceAccount:
    annotations:
      iam.gke.io/gcp-service-account: <service-account-id>@<gcp-project-id>.iam.gserviceaccount.com

blocks:
  repository: us-central1-docker.pkg.dev/gcp-project-name/hyperscience/sdm_blocks

operator:
  repository: us-central1-docker.pkg.dev/gcp-project-name/hyperscience/hyperoperator # required

trainer:
  repository: us-central1-docker.pkg.dev/gcp-project-name/hyperscience/trainer # required
  tags:
  - 42.0.14
cloud:
  aws:
    includeRdsCerts: false

Advanced values.yaml

The following command can be used to retrieve all the possible options of the Helm chart:

helm show values $HS_HELM_CHART

It will return the template for the latest Helm chart version. Save the file as values-full.yaml. It's best practice to only add the options you want to change from values-full.yaml in your values.yaml.

Allocating tasks to trainers with GPUs

In v42.3 and later, if you are using trainers that have GPUs, you can allocate training tasks to nodes that are adequately sized to complete those tasks.

gpuTiers:
  small:
    minGPUMemory: 0
    maxGPUMemory: 16384
    nodeSelector:
      kubernetes.io/os: linux
  medium:
    minGPUMemory: 16385
    maxGPUMemory: 22888
    nodeSelector:
      kubernetes.io/os: linux
  large:
    minGPUMemory: 22889
    maxGPUMemory: 45776
    nodeSelector:
      kubernetes.io/os: linux

Kubernetes Secrets

Platform Secret

We require a kubernetes native secret to store database credentials and shared tokens that allow intra-app communication. This secret needs to contain at least the following keys:

You should obtain the FORMS_DB_NAME, FORMS_DB_USER, and FORMS_DB_PASS from your database configuration.

apiVersion: v1
kind: Secret
metadata:
  name: hyperscience-platform
stringData:
  FORMS_DB_NAME: my-postgres-db
  FORMS_DB_USER: my-db-role
  FORMS_DB_PASS: my-postgres-password
  BLOCK_ORCHESTRATOR_TOKEN: e271dc47fa80ddc9e6590042ad9ed2b7

Application Configuration

There are two ways to configure the Hyperscience application environment. The first and recommended way is to create environment key-value pairs in the values.yaml path app.dotenv. Optionally, you can create your own ConfigMap with your desired configuration and pass the ConfigMap name to the app.dotenv_configmap_name setting in values.yaml.

Installation

Make sure you followed the hsk8s (Hyperscience Kubernetes CLI) instructions to install hsk8s and Helm Repo.

Run the helm install command

helm install $HS_HELM_RELEASE -f values.yaml $HS_HELM_CHART --create-namespace

Updating values.yaml

In order to apply a change made to values.yaml, you should run helm upgrade. Make sure to specify a chart version with --version, otherwise the latest chart version will be used.

helm upgrade $HS_HELM_RELEASE -f values.yaml $HS_HELM_CHART --version X.Y.Z

Scaling

By default, only one instance of each block type will be run. For more information on how to scale the system, refer to the Scaling article.