# Managing Third-Party Python Packages The Hyperscience Platform supports a special type of block called _PythonBlock_ that allows users to run custom Python code. By default, users can only use packages that are part of the Python Standard Library. To be able to use third-party Python packages, you will need to have them installed on the Hyperscience Platform. If you no longer need packages you’ve installed, you can uninstall them. The tutorials below will guide you through the installation and removal of third-party Python packages. ## Managing packages in v36 and later Starting with v36, third-party Python package management is available directly in the Hyperscience application UI. The Python version used by `PythonBlock` (and `CodeBlock`) has evolved across platform versions. You should download packages targeting the Python version that matches your platform version: | Platform version | Supported Python versions | Default | |----------------------|-----------------------------------|----------| | v36–v39 | 3.9 | 3.9 | | v39.1–v39.x | 3.9, 3.11 | 3.9 | | v40–v40.x | 3.9 _(deprecated)_, 3.11 | 3.11 | | v41–v42.1 | 3.9 _(deprecated)_, 3.11, 3.12 | 3.12 | | v42.2–v42.x | 3.9 _(deprecated)_, 3.11, 3.12, 3.13 | 3.13 | | v43+ | 3.11 _(deprecated)_, 3.12, 3.13 | 3.13 | For details on selecting a Python version per block and controlling the default, see the [`PythonBlock` documentation](https://flows-sdk.hyperscience.ai/pages/source-docs.html#flows_sdk.blocks.PythonBlock). On platform versions that support multiple Python versions, packages are tracked **separately per Python version**. A package installed for Python 3.12 is not automatically available to a block running Python 3.13, and vice versa. You must install it once for each Python version you intend to use it with. Warning If you are migrating from an older Hyperscience version, some of your packages might be incompatible with the Python version used on the new version. You should test to ensure the packages still work on the updated application version (e.g., by creating a submission for a flow that uses code from those packages). If your testing shows that a package is incompatible with your target Python version, you can update it in the application. To do so: 1. Uninstall the package by following the steps in [Uninstalling third-party Python packages](https://flows-sdk.hyperscience.ai/pages/python_packages.html#uninstalling-third-party-python-packages). 2. Download the package again, explicitly targeting the correct `--python-version` for your platform (see the table above). 3. Install the downloaded package by following the steps in [Installing third-party Python packages](https://flows-sdk.hyperscience.ai/pages/python_packages.html#installing-third-party-python-packages). ### Installing third-party Python packages The steps below will walk you through downloading an example package called `regex` and installing it on a running Hyperscience instance. 1. Download the `regex` package using `pip download`, targeting the Python version you want to install it for (see the table above): ``` pip download --python-version --platform=manylinux2014_x86_64 --only-binary=:all: regex ``` For example, for Python 3.13: ``` pip download --python-version 3.13 --platform=manylinux2014_x86_64 --only-binary=:all: regex ``` Note that this command requires access to PyPi repositories and can be run on your local machine. **Do not omit `--platform=manylinux2014_x86_64`**, especially if you are using Mac or Windows. Otherwise, you will download the incorrect package. **NOTE:** When working with your own packages, make sure that any packages you download for installation do not require GPU acceleration. Doing so ensures compatibility with any machine running the packages. 2. In the application, go to **Flows**  **Python Packages**, navigate to the section for the relevant Python version, and click **Add Package**. 3. Click **Choose Files**/ **Browse**, and find and open the package’s wheel file (e.g., `regex-2022.10.31-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`) on your machine. **TIP:** You can upload multiple packages at once by selecting their wheel files (`.whl`) in this step. 4. Click **Upload**. The package appears in the Python Packages table for that Python version. Repeat for any other Python versions you need it available on. ### Uninstalling third-party Python packages Now we will uninstall the `regex` package we just installed. 1. In the application, go to **Flows**  **Python Packages**, navigate to the section for the relevant Python version, and find the `regex` package in the Python Packages table. 2. Click the **Uninstall Package** button in the package’s row in the table. The package is removed from that Python version’s package list. ### More information To learn more about managing Python packages—including updating and downloading installed packages—see [Developing Flows](https://help.hyperscience.ai/deployment/docs/developing-flows) in our application documentation. ## Managing packages in v35 In v35, we added a number of package-management features to the Hyperscience application. These features do not require you to run commands in your virtual environment. ### Installing third-party Python packages in v35 The steps below will walk you through downloading an example package called `regex` and installing it on a running Hyperscience instance. 1. Download the `regex` package using `pip download`: ``` pip download --python-version 3.7 --platform=manylinux2014_x86_64 --only-binary=:all: regex ``` Note that this command requires access to PyPi repositories and can be run on your local machine. **Do not omit `--platform=manylinux2014_x86_64`**, especially if you are using Mac or Windows. Otherwise, you will download the incorrect package. 2. In the application, go to **Flows**  **Python Packages**, and click **Add Packages**. 3. Click **Choose Files**, and find and open the package’s wheel file (`.whl`, e.g., `regex-2022.10.31-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`) on your machine. **TIP:** You can upload multiple packages at once by selecting their wheel files in this step. 4. Click **Upload**. The package appears in the Python Packages table. ### More information on v35 To learn more about managing Python packages in v35—including updating and downloading installed packages—see [Developing Flows](https://help.hyperscience.ai/deployment/docs/developing-flows) in our application documentation. ## Managing packages in v34 Docker-based deployments In v34, you need to SSH into your application machine to install and manage Python packages. The steps below will walk you through downloading an example package called `regex` and installing it on a running Hyperscience instance. This process requires SSH access to a machine running the Hyperscience Platform. 1. Download the wheel file using `pip download`: **Do not omit `--platform=manylinux2014_x86_64`**, especially if you are using Mac or Windows. Otherwise, you will download the incorrect package. 2. Upload the wheel file (e.g., `regex-2022.3.15-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`) to an application machine. Note that an application machine is a machine that is running the Hyperscience Platform, not a machine that is running the trainer. ``` scp regex-2022.3.15-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl root@app_machine.hyperscience.com:/mnt/hs/media/ ``` Always upload the wheel files to `/mnt/hs/media` so that the Docker containers on the application machine will have access to them. `/mnt/hs/media` is a shared volume that is mapped to a directory inside the containers. 3. `ssh` to the application machine. ``` ssh root@app_machine.hyperscience.com ``` 4. `cd` to the directory where Hyperscience has been installed. This directory is usually `/opt/hs/bundle_install/`, but may vary depending on your installation. ``` cd ``` 5. Execute the `install` command: ``` run.sh manage python_packages install /var/www/forms/forms/media/regex-2022.4.24-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ``` `/var/www/forms/forms/media/` is the directory inside the container that `/mnt/hs/media` maps to. The `install` command takes in either a wheel file or a directory that contains wheel files. Look for the following output for a successful installation: ``` [2022-04-26 03:53:33.220 | INFO | MainThread | sdm.management.commands.python_packages:179] regex 2022.4.24 has been uploaded for installation. [2022-04-26 03:53:33.223 | INFO | MainThread | sdm.management.commands.python_packages:295] Installation has been triggered. Please note: It may take up to several minutes for changes to packages to be picked up by the flow execution processes on all machines ``` Users can go to **Flows**  **Python Packages** in the Hyperscience Platform to see the list of third-party Python packages that have been installed. If there is an existing version of the `regex` package available in the system, your install command will be rejected. To overwrite the existing version, use the `-f`/`--force` option. You may find this option useful when upgrading packages. ``` run.sh manage python_packages install -f /var/www/forms/forms/media/regex-2022.4.24-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ``` When you have more than one wheel file that you wish to install, you should supply the directory that contains the wheel files instead of the wheel files themselves as the argument. The program will take in all files with the `.whl` extension. ``` run.sh manage python_packages install -f /var/www/forms/forms/media/ ``` ## Managing packages in v34 on-premise Kubernetes deployments In v34, you need to use `kubectl` to install and manage Python packages for on-premise Kubernetes deployments. The steps below will walk you through downloading an example package called `regex` and installing it on a running Hyperscience instance. 1. Download the package’s wheel file using `pip download`: **Do not omit `--platform=manylinux2014_x86_64`**, especially if you are using Mac or Windows. Otherwise, you will download the incorrect package. 2. Upload the wheel file (e.g., `regex-2022.3.15-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl`) to a Kubernetes pod via `kubectl`: ``` KUBERNETES_NAMESPACE= WHEEL_FILEPATH= CONTAINER=gunicorn POD=$(kubectl -n $KUBERNETES_NAMESPACE get pods -l app.kubernetes.io/component=frontend \ --field-selector=status.phase=Running \ -o=jsonpath="{.items[0].metadata.name}") kubectl cp -n $KUBERNETES_NAMESPACE -c $CONTAINER $WHEEL_FILEPATH $POD:/tmp/$WHEEL_FILEPATH ``` 3. Install the wheel file: This command installs the package in the Hyperscience Platform and removes the wheel file previously uploaded to the pod. ``` kubectl exec $POD -c $CONTAINER -- bash -c "source /var/www/entrypoint.sh; /var/www/forms/forms/manage.py python_packages install /tmp/$WHEEL_FILEPATH; rm /tmp/$WHEEL_FILEPATH" ``` **NOTE:** Make sure to run step 3 in the same session as step 2, as it requires the previously set environment variables. If there is an existing version of the `regex` package, the `install` command will be rejected. To overwrite the existing version, use the `-f`/`--force` option. You may find this option useful when upgrading packages. ``` kubectl exec $POD -c $CONTAINER -- bash -c "source /var/www/entrypoint.sh; /var/www/forms/forms/manage.py python_packages install -f /tmp/$WHEEL_FILEPATH; rm /tmp/$WHEEL_FILEPATH" ``` After step 3 is completed, the `regex` package should be listed in the Hyperscience Platform under **Flows**  **Python Packages**.