[WIP] feat: build FunctionGraph from functions

[zilto]

• Relates to Build DAG from functions directly #1268

Changes

• The changes affect core code path, but is generally safe. Only outdated tests using graph.create_function_graph() instead of graph.FunctionGraph.from_modules() needed to be updated.
• The function signature of graph.create_function_graph() was modified, but it was never advertised as a public API. Further rewiring could provide full backwards compatibility (could do after validating the general solution)

TODO

• update documentation
• change create_function_graph() for fully backwards-compatible APi

How I tested this

• all tests pass after slight edits

Notes

• graph.create_function_graph() should really return a FunctionGraph instead of a dict

[elijahbenizzy]

Hey -- sorry, it's been a bit, will be looking over soon!

[skrawcz]

The type signature of this should be updated.

[skrawcz]

also 🤦 (at myself I think) that the name of this function and what it returns - a dict, not a function graph...

[skrawcz]

Yeah so what's left? This ?

• add in ability to pass functions and update code to handle that?
• add tests for new functionality
• add example

[skrawcz]

My expectation

# module Z
# some funcs

# module 1
def func1( ...) -> ...:
  ...

# module 1, or perhaps another module 2
def func2(...)->...:
  ...

# driver code somewhere else or in one of the modules - importing things appropriately
funcs = [func1, func2] 

driver.Builder().with_modules(module_z, *funcs)... # this should work

it should also work with .allow_module_overrides() on the driver Builder, thus allowing one to inject a new function implementation.

[leblancfg]

I saw the note in #1271:

Benefits
facilitate marimo integration

FYI I attempted this with this branch, but it returns TypeError: unhashable type: 'Cell'.

[zilto]

I saw the note in #1271:

Benefits
facilitate marimo integration

FYI I attempted this with this branch, but it returns TypeError: unhashable type: 'Cell'.

@leblancfg You're ahead of us! Will open a separate PR and share my marimo development pattern once this is merged. Feel free to discuss here #1274

[elijahbenizzy]

OK, with_modules(...) shouldn't accept functions. with_function(...) should, IMO.

That said -- in order to have function overrides, we should probably want to combine from_fucntions and from_modules into construct(...) or compile(...) that takes in a list of functions/modules. Happy to take a stab! Should be straightforward -- we can even leave around the old one for backwards compatibility.

[skrawcz]

OK, with_modules(...) shouldn't accept functions. with_function(...) should, IMO.

Disagree. I think this will make things more confusing, as it will:

1. create two ways to create nodes in the graph.
2. provide no clear precedence when say someone wants to override a function in a module.

I think extending with_modules is fine. It also makes it very clear the order of application.

[zilto]

closing in favor of #1275