[Python] generate enums as IntEnum or IntFlag type when requested

[jtdavis777]

Fixes #8617 and addresses #5083

This PR has enum types inherit from IntEnum or (IntFlag for bit_flags) directly, instead of just providing this as type hinting, and changes the setter function signatures to require enum types. This code should still be backward and forward compatible on receipt, as ints are still returned by the getters.

This code, chiefly, allows users to go to and from string string values to the enum values, but also provides some runtime type safety (over just type hint safety).

[fliiiix]

Some docs at least in docs/source/flatc.md would be nice for this change

[fliiiix]

I played around a bit and it seemed fine, i couldn't integrated it into our main product because this crate was not updated yet rdelfin/flatbuffers-build#30

[fliiiix]

LGTM 👍

[jtdavis777]

You said it was compatible.. do we need this flag at all? Can't we always do the better thing?
so, I think it depends on what we mean by python 3 support. Now that we have dropped python 2 I think this is safe, but I believe technically this is python 3.4. Though TBF the current implementation is not consistent here so I think just doing this as part of --python-typing is fine. Will change to that!

[fliiiix]

Will you still updated this MR @jtdavis777 ?

[jtdavis777]

@fliiix yes it is on my list! Thank you for the reminder.

[fliiiix]

No hurry, I just finally got some time to look at changes and im running this right now on my version and am still interested to have this at some point on master :)

[fliiiix]

Im integrating the latest version right now any updates on this MR? I would still like to use that at some point :)

[jtdavis777]

No updates yet, I want to just fold it into the typing flag. Apologies for the delays!

[jtdavis777]

okay @fliiiix and @aardappel I think this one is ready for another pass - I think we have some work to do to make all the type hints actually consistent, but I don't want to tackle that here. this should generate compliant pyi files and type hints for where they are generated.

[fliiiix]

Looks mostly good one small blocking issue everything else seems fine to me and worked in my testing ✨

[fliiiix]

i don't think this is valid see:

error: Detected enum "EnumType" in a type stub with zero members. There is a chance this is due to a recent change in the semantics of enum membership. If so, use `member = value` to mark an enum member, instead of `member: type`  [misc]

https://typing.python.org/en/latest/spec/enums.html#defining-members

I think the previous:

 stub << "  " << namer_.Variant(*val) << " = cast(" << ScalarType(enum_def->underlying_type.base_type) << ", ...)\n";

is still how we should do it

[jtdavis777]

I think with the change over to enums inheriting from Enum types, it should be

stub << "  " << namer_.Variant(*val) << namer_.Type(*enum_def) << "\n";

to embrace the stronger typing. thoughts?

[fliiiix]

TL;DR lets try to use the strong types as it seems to be ok for mypy 👍

I think thats not how it works but im struggling to find a good source proving me right or wrong, how i think about it is that the IntEnum is a strong type containing ints.
https://mypy.readthedocs.io/en/stable/literal_types.html#enums

Using the Type of the Enum inside the Enum itself is recursive, but i just tried it on a small scale example:

==> TestEnum.py <==
from enum import IntEnum

class TestEnum(IntEnum):
    A = 1
    B = 2

==> TestEnum.pyi <==
from enum import IntEnum

class TestEnum(IntEnum):
    A = TestEnum
    B = TestEnum

==> Test.py <==
import TestEnum

def function(e: TestEnum.TestEnum):
    ...

And mypy is happy about it 🤷

uv run mypy Test.py                        
Success: no issues found in 1 source file

[jtdavis777]

awesome! I left the old way for when typing is off, but I think the file won't generate at all in that case? haha

[fliiiix]

idk seems in my real project mypy is not happy with it im still getting these errors now ->

FluidType.pyi:10: error: Detected enum "FluidType.FluidType" in a type stub with zero members. There is a chance this is due to a recent change in the semantics of enum membership. If so, use `member = value` to mark an enum member, instead of `member: type`  [misc]
FluidType.pyi:10: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members

[fliiiix]

i guess we should switch back to cast(int, ...) at least that seemed to work everywhere :)

[jtdavis777]

yea you are probably right. I can do that

[jtdavis777]

done!

[fliiiix]

looking at the examples and the code this seems now fine to me 👍 (i also started a build to see but i assume it should be good like this)

[fliiiix]

i think you forgot to rerun the generation thats why this job still fails -> https://github.com/google/flatbuffers/actions/runs/23049893282/job/66948134837?pr=8782

I just figured out something broke in my setup so it might be monday until i can give it another try :/

Manually inspecting the .pyi files this should be fine :)