# Submissions halt when the database connection is lost

- Published on Sep 24, 2024
- 1 minute(s) read

## Issue

In some cases, the database connection may be lost, causing submissions to halt.

The error looks similar to the one shown below.

```plaintext
{

"failure": "Exception",

"stacktrace": "Traceback (most recent call last):

File \"/var/www/venv/lib/python3.7/site-packages/django/db/utils.py\", line 96, in inner

return func(*args, **kwargs)

File \"/var/www/forms/forms/forms/db/backends/mssql/pyodbc/base.py\", line 477, in fetchmany

return list(map(tuple, self.cursor.fetchmany(chunk)))

pyodbc.OperationalError: ('08S01', '[08S01] [Microsoft][ODBC Driver 17 for SQL Server]
Session Provider: Connection has been closed by peer [xFFFFFFFF].  (-1) (SQLGetData)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File \"/var/www/forms/forms/job_queue/management/commands/worker.py\", line 236, in _process_thread

_process_job(worker, job)

File \"/var/www/venv/lib/python3.7/site-packages/tenacity/__init__.py\", line 292, in wrapped_f

return self.call(f, *args, **kwargs)

File \"/var/www/venv/lib/python3.7/site-packages/tenacity/__init__.py\", line 358, in call

do = self.iter(retry_state=retry_state)

File \"/var/www/venv/lib/python3.7/site-packages/tenacity/__init__.py\", line 319, in iter

return fut.result()

File \"/opt/hs/python/lib/python3.7/concurrent/futures/_base.py\", line 428, in result

return self.__get_result()

File \"/opt/hs/python/lib/python3.7/concurrent/futures/_base.py\", line 384, in __get_result

raise self._exception

File \"/var/www/venv/lib/python3.7/site-packages/tenacity/__init__.py\", line 361, in call

result = fn(*args, **kwargs)

File \"/var/www/forms/forms/job_queue/management/commands/worker.py\", line 93, in _process_job

worker.process(job)

File \"/var/www/forms/forms/form_extraction/workers/oicr.py\", line 662, in process

return self.process_page(job)

File \"/var/www/forms/forms/form_extraction/workers/oicr.py\", line 419, in process_page

template_binary_image=template_binary_image,

File \"/var/www/forms/forms/form_extraction/workers/oicr.py\", line 178, in _predict_form_field

field, form_image, template_image, fetch_debug_data=fetch_debug_data

File \"/var/www/forms/forms/form_extraction/oicr.py\", line 41, in predict_field

all_bboxes = self.get_all_bboxes(field)

File \"/var/www/forms/forms/form_extraction/oicr.py\", line 77, in get_all_bboxes

return extract_all_bboxes(field.template_field)

File \"/var/www/forms/forms/form_extraction/oicr_utils.py\", line 96, in extract_all_bboxes

for template_field in template_field.template_page.fields.all()

File \"/var/www/venv/lib/python3.7/site-packages/django/db/models/query.py\", line 274, in __iter__

self._fetch_all()

File \"/var/www/venv/lib/python3.7/site-packages/django/db/models/query.py\", line 1244, in _fetch_all

self._result_cache = list(self._iterable_class(self))

File \"/var/www/venv/lib/python3.7/site-packages/django/db/models/query.py\", line 55, in __iter__

results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)

File \"/var/www/forms/forms/forms/db/backends/mssql/compiler.py\", line 134, in execute_sql

ret = super().execute_sql(*args, **kwargs)

File \"/var/www/venv/lib/python3.7/site-packages/django/db/models/sql/compiler.py\", line 1173, in execute_sql

return list(result)

File \"/var/www/venv/lib/python3.7/site-packages/django/db/models/sql/compiler.py\", line 1552, in cursor_iter

for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):

File \"/var/www/venv/lib/python3.7/site-packages/django/db/models/sql/compiler.py\", line 1552, in

for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):

File \"/var/www/venv/lib/python3.7/site-packages/django/db/utils.py\", line 96, in inner

return func(*args, **kwargs)

File \"/var/www/venv/lib/python3.7/site-packages/django/db/utils.py\", line 89, in __exit__

raise dj_exc_value.with_traceback(traceback) from exc_value

File \"/var/www/venv/lib/python3.7/site-packages/django/db/utils.py\", line 96, in inner

return func(*args, **kwargs)

File \"/var/www/forms/forms/forms/db/backends/mssql/pyodbc/base.py\", line 477, in fetchmany

return list(map(tuple, self.cursor.fetchmany(chunk)))

django.db.utils.OperationalError: ('08S01', '[08S01] [Microsoft][ODBC Driver 17 for SQL Server]
Session Provider: Connection has been closed by peer [xFFFFFFFF].  (-1) (SQLGetData)')"

}
```

## Affected versions

All versions

## Affected configurations

All configurations

## Cause

The cause is not clear.

## Solution / Workaround

Check the state of the database. If the database connection appears to be healthy, simply retrying the halted jobs should solve the problem.