feature: More granular exception control

[hansonmcoombs]

Is your feature request related to a problem? Please describe.

I often find debugging errors and edge cases to be quite challenging as the issues that cause the exception are often hidden beneath a layer of other general exceptions, which obfuscate the actual source of the error. As and example: when debugging the edge case raised in: #2855 The actual exception I got was: "unable to read budget information from first entry in list file". This is useful for providing a high level explanation of the issue, but makes debugging quite difficult. Additionally many of the exceptions raised like this are the base class Exception, which means that any try/except clause will include all Exceptions that could be raised throughout the case. As an example a try/except clause for the budget file issue would capture both FileNotFoundExceptions as well as the general parsing exception. I could always test whether the file exists first, but here I would have to test all possible exceptions rather than just being able to handle a specific exception appropriately.

Describe the solution you'd like

I would suggest two simple solutions:

1. Create one or more deticated "FlopyParsingException" or simlilar, so that people could catch just that exception.
2. Add from context to the raised exception so that those of us who end up debugging can get some additional clarity on the error.

Implementing this in the ListBudget._set_entries method would transform

    def _set_entries(self):
        if len(self.idx_map) < 1:
            return None, None
        if len(self.entries) > 0:
            raise Exception(f"entries already set:{self.entries}")
        if not self.idx_map:
            raise Exception("must call build_index before call set_entries")
        try:
            incdict, cumdict = self._get_sp(
                self.idx_map[0][0], self.idx_map[0][1], self.idx_map[0][2]
            )
        except Exception
            raise Exception(
                "unable to read budget information from first entry in list file"
            )

to

    def _set_entries(self):
        if len(self.idx_map) < 1:
            return None, None
        if len(self.entries) > 0:
            raise Exception(f"entries already set:{self.entries}")
        if not self.idx_map:
            raise Exception("must call build_index before call set_entries")
        try:
            incdict, cumdict = self._get_sp(
                self.idx_map[0][0], self.idx_map[0][1], self.idx_map[0][2]
            )
        except Exception as e:
            raise FlopyParsingException(
                "unable to read budget information from first entry in list file"
            ) from e

Where FlopyParsingException is defined as:

class  FlopyParsingException(Exception):
    """An exception to catch errors when parsing modflow text files"""
    pass

This would transfrom the traceback from:

File ".../python3.14/site-packages/flopy/utils/mflistfile.py", line 679, in _set_entries
    raise Exception(
        "unable to read budget information from first entry in list file"
    )
Exception: unable to read budget information from first entry in list file

to

Traceback (most recent call last):
  File ".../python3.14/site-packages/flopy/utils/mflistfile.py", line 675, in _set_entries
    incdict, cumdict = self._get_sp(
    ^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 0)

The above exception was the direct cause of the following exception:

File ".../python3.14/site-packages/flopy/utils/mflistfile.py", line 679, in _set_entries
    raise FlopyParsingException(
        "unable to read budget information from first entry in list file"
    )
FlopyParsingException: unable to read budget information from first entry in list file

I would be happy to help implement this with a pull request, but I would want some consensus that this is the way to go.

Describe alternatives you've considered
None

[wpbonelli]

Thanks for this. Opaque exceptions are a frequent complaint. I spun off #2857 to track the topic at a high level and switched this issue to a bug. It would take a pretty substantial effort to rework error handling in 3.x (hundreds of bare except and generic Exception in the codebase) so IMO it makes sense to address specific cases like this one ad hoc in 3.x, and overhaul the general approach in 4.x. I think both of your proposals, specific exception types and chaining, make sense.

In this specific case, though, I don't think adding from e would help much. When I run

import sys
import tempfile
from pathlib import Path

import flopy

LST = """\
  MASS BUDGET FOR ENTIRE MODEL AT END OF TIME STEP    1, STRESS PERIOD   1
  ---------------------------------------------------------------------------------------------------

     CUMULATIVE MASS      M       RATES FOR THIS TIME STEP      M/T          PACKAGE NAME    
     ------------------                 ------------------------                     ----------------

           IN:                                      IN:
           ---                                      ---
     STORAGE-AQUEOUS =       1.6172E-08       STORAGE-AQUEOUS =       1.6172E-08     MST                              
                 SRC =          60.3962                   SRC =          60.3962     SRC_0                            
        FLOW-JA-FACE =      3.9204E-103          FLOW-JA-FACE =      3.9204E-103     GWT-GWT_9                        

            TOTAL IN =          60.3962              TOTAL IN =          60.3962

          OUT:                                     OUT:
          ----                                     ----
     STORAGE-AQUEOUS =          60.3063       STORAGE-AQUEOUS =          60.3063     MST                              
        FLOW-JA-FACE =       8.5159-100          FLOW-JA-FACE =       8.5159-100     GWT-GWT_9                        

           TOTAL OUT =          60.3962             TOTAL OUT =          60.3962

            IN - OUT =      -3.1974E-13              IN - OUT =      -3.1974E-13

 PERCENT DISCREPANCY =          -0.00     PERCENT DISCREPANCY =          -0.00
"""

path = Path(tempfile.mkdtemp()) / "repro.lst"
path.write_text(LST)
flopy.utils.Mf6ListBudget(str(path), budgetkey="MASS BUDGET")

I see

error casting in flux for FLOW-JA-FACE  to float in ts,sp 1 1
Traceback (most recent call last):
  File "/Users/wbonelli/dev/flopy/flopy/utils/mflistfile.py", line 670, in _set_entries
    incdict, cumdict = self._get_sp(
    ^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/wbonelli/dev/flopy/reproduce_issue_2856.py", line 43, in <module>
    flopy.utils.Mf6ListBudget(str(path), budgetkey="MASS BUDGET")
  File "/Users/wbonelli/dev/flopy/flopy/utils/mflistfile.py", line 924, in __init__
    super().__init__(file_name, budgetkey=budgetkey, timeunit=timeunit, **kwargs)
  File "/Users/wbonelli/dev/flopy/flopy/utils/mflistfile.py", line 82, in __init__
    self._load()
  File "/Users/wbonelli/dev/flopy/flopy/utils/mflistfile.py", line 690, in _load
    incdict, cumdict = self._set_entries()
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/wbonelli/dev/flopy/flopy/utils/mflistfile.py", line 674, in _set_entries
    raise Exception(
Exception: unable to read budget information from first entry in list file

I think adding from e would just change the connecting text from

During handling of the above exception, another exception occurred:

to

The above exception was the direct cause of the following exception:

Are you using flopy in a notebook or IDE? I think jupyter and/or vscode truncate error traces sometimes.

Really the unpacking error is not much better than the generic one. I think we can come up with something better by disentangling the logic for parsing table headers from parsing values.

[hansonmcoombs]

Yep that all makes sense. I don't have a huge amount of clarity on the plan going forward, but yes overhauling exception handling generally in a major version update makes a lot of sense.

You are correct regarding the from e in this instance. My appologies I must have missed the traceback... it was late and I was tired and a bad example. But I have certainly had to dig quite deep to find the root cause of exceptions.

[wpbonelli]

sorry, I didn't mean to phrased anything in any kind of negative sense.. I completely agree, I get frustrated with the exceptions a lot too.

The main reason I don't feel great about doing it near-term (besides the amount of effort- maybe overestimating) is it seems like there may be a lot of situations like the one in the #2859 description, non-ideal behaviors that people could have come to expect, that would make it hard to decide how to get from the current behavior to the optimal without breaking a whole bunch of scripts out there.. I didn't mean to imply there was a clear plan, these issues you made are a good help to form one.

[wpbonelli]

I should have made a separate issue to track the specific bug you mentioned here, or just not made another one. I associated this one with it specifically for some reason. my bad. this one should stand as the discussion point for exceptions in general

[hansonmcoombs]

I didn't see any negative intentions :) I made the issue when I was tired and frustrated and I want to be clear that I really value the work that you have done.

I do agree that a new version release (either 4. 3.#.) is the time to overhaul the the exception handling. Otherwise you will just make someone else grumpy!

Let me know if you need any help with this.

[wpbonelli]

I can't speak for anyone else but I like the idea of a specific exception type for parsing errors (and other things). There is no risk to that in 3.x. How far were you thinking of going, just this case, or more widely?

also if you have any thoughts on #2859 in particular, or know of other similar cases, or more general ideas on how error handling should work?