Handle 500 Retry Error

[krowvin]

Currently if you get a 500 or if CDA does a retry on an error it silences the underlying error and shows the "retry failed" error.

i.e.
(this is the full output)

urllib3.exceptions.ResponseError: too many 500 error responses

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\requests\adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\urllib3\connectionpool.py", line 942, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\urllib3\connectionpool.py", line 942, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\urllib3\connectionpool.py", line 942, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  [Previous line repeated 3 more times]
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\urllib3\connectionpool.py", line 932, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\urllib3\util\retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='water.dev.cwbi.us', port=443): Max retries exceeded with url: /cwms-data/location/group (Caused by ResponseError('too many 500 error responses'))        

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "\Projects\SWT\swt-wm-cwbi-jobs\scripts\create_ldm_location_groups.py", line 126, in <module>
    print(cwms.store_location_groups(location_groups))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\cwms\locations\location_groups.py", line 106, in store_location_groups
    return api.post(endpoint=endpoint, data=data, api_version=1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\cwms\api.py", line 335, in post
    with SESSION.post(endpoint, params=params, headers=headers, data=data) as response:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\requests\sessions.py", line 637, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\requests_toolbelt\sessions.py", line 76, in request
    return super(BaseUrlSession, self).request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "\Projects\SWT\swt-wm-cwbi-jobs\venv\Lib\site-packages\requests\adapters.py", line 691, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='water.dev.cwbi.us', port=443): Max retries exceeded with url: /cwms-data/location/group (Caused by ResponseError('too many 500 error responses'))