Use ogsi model by AlexAxthelm · Pull Request #32 · RMI/stitch

AlexAxthelm · 2026-03-04T20:06:10Z

Big PR, co-written with @mbarlow12:

Extract OG specific domain fields and logic from the API deployment, and use the equivalents from the stitch.ogsi package instead.

Includes pretty big refactors on API/DB layers, including using OGFieldResources instead of generic Resources, and consequent changes to how memberships attach to them. Biggest change along those lines is consolidating all the indivudal OG source tables into a single oil_gas_field_sources table in the DB (and storing provenence source info there).

Big user facing changes:

removing the /resources endpoint, and replacing it with /oil-gas-fields, which behaves similarly
Querying a single Oil-gas-field (GET /oil-gas-fields/{id}) now returns a computed "flattened/coalesced" view (still primitive logic), extracted from source_data rather than the raw resource object.

Copilot

Pull request overview

Adds the OGSI domain model to Stitch, refactors the generic Resource API to be domain-agnostic, and introduces a new domain-specific Oil & Gas Fields API + frontend wiring.

Changes:

Add stitch-ogsi/stitch-models workspace dependencies and use OGSI models for Oil & Gas Fields.
Introduce /api/v1/oil-gas-fields endpoints backed by a new oil_gas_fields table (1:1 with resources).
Update the frontend MVP to query and display OG fields instead of generic resources.

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
uv.lock	Adds workspace deps for `stitch-models` and `stitch-ogsi`.
packages/stitch-models/src/stitch/models/init.py	Introduces `EmptySourcePayload` and a `BaseResource` alias for domain-agnostic use.
deployments/stitch-frontend/src/queries/ogfields.js	Adds React Query key/query factory for OG fields.
deployments/stitch-frontend/src/queries/api.js	Adds `getOGFields` / `getOGField` API helpers.
deployments/stitch-frontend/src/hooks/useOGFields.js	Adds hooks wrapping authenticated queries for OG fields.
deployments/stitch-frontend/src/components/OGFieldsView.jsx	Adds list view UI for OG fields + cache controls.
deployments/stitch-frontend/src/components/OGFieldsList.jsx	Adds list rendering and JSON debug output for OG fields.
deployments/stitch-frontend/src/components/OGFieldView.jsx	Adds single-OG-field fetch view by ID.
deployments/stitch-frontend/src/App.test.jsx	Updates app smoke tests to expect OG field headings.
deployments/stitch-frontend/src/App.jsx	Switches app to OGFields views instead of Resources views.
deployments/api/tests/utils.py	Simplifies test factories to a minimal, domain-agnostic `CreateResource`.
deployments/api/tests/routers/test_resources_unit.py	Updates resources router unit tests for the new entities/factories.
deployments/api/tests/routers/test_resources_integration.py	Updates resources integration tests to match simplified resource model.
deployments/api/tests/db/test_resource_actions.py	Updates DB integration tests for simplified resource creation/listing.
deployments/api/tests/conftest.py	Removes fixtures tied to removed source tables/models.
deployments/api/src/stitch/api/routers/resources.py	Switches router models to domain-agnostic `resources.entities`.
deployments/api/src/stitch/api/routers/oil_gas_fields.py	Adds new `/oil-gas-fields` router using OGSI models + new DB model.
deployments/api/src/stitch/api/resources/entities.py	Introduces domain-agnostic `CreateResource`/`Resource` API models.
deployments/api/src/stitch/api/ogsi/entities.py	Adds OGSI-ish API models (currently not referenced elsewhere).
deployments/api/src/stitch/api/main.py	Registers the new oil-gas-fields router.
deployments/api/src/stitch/api/entities.py	Removes legacy source/resource entities; keeps `User`.
deployments/api/src/stitch/api/db/resource_actions.py	Refactors resource actions to be domain-agnostic (no sources/memberships).
deployments/api/src/stitch/api/db/model/sources.py	Deletes source table models and related helpers.
deployments/api/src/stitch/api/db/model/resource.py	Removes `country`, loosens membership `source` typing, adjusts `source_pk` handling.
deployments/api/src/stitch/api/db/model/oil_gas_field.py	Adds `OilGasFieldModel` with JSON payload + FK to resources.
deployments/api/src/stitch/api/db/model/mixins.py	Makes `PayloadMixin` accept any `BaseModel` payload (not only `SourceBase`).
deployments/api/src/stitch/api/db/model/init.py	Exports `OilGasFieldModel`; removes source model exports.
deployments/api/src/stitch/api/db/init_job.py	Updates dev seeding to create resources + example OG fields (no source tables).
deployments/api/pyproject.toml	Adds `stitch-models` and `stitch-ogsi` as dependencies.

Comments suppressed due to low confidence (2)

deployments/stitch-frontend/src/components/OGFieldView.jsx:59

Grammar: the empty-state message says “fetch a ogfield”, which should be “fetch an ogfield” (or “fetch an OG field”).

        message={`No ogfield loaded. Click the button above to fetch a ogfield.`}
      />

deployments/api/src/stitch/api/ogsi/entities.py:18

CreateOilGasField/OilGasField models are introduced here but aren’t referenced anywhere in the API code (the router uses stitch.ogsi.model.OGFieldView / OilGasFieldBase instead). If these types are not part of the intended API surface, remove the file; otherwise, update the router to use them to avoid dead code.

from __future__ import annotations

from pydantic import BaseModel, ConfigDict, Field
from stitch.api.resources.entities import CreateResource, Resource as ResourceView


class CreateOilGasField(BaseModel):
    model_config = ConfigDict(extra="forbid")
    resource: CreateResource
    owner: str | None = Field(default=None)
    operator: str | None = Field(default=None)


class OilGasField(BaseModel):
    id: int
    resource: ResourceView
    owner: str | None = None
    operator: str | None = None

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

deployments/api/src/stitch/api/routers/oil_gas_fields.py

deployments/api/tests/db/test_resource_actions.py

deployments/stitch-frontend/src/components/OGFieldsList.jsx

deployments/api/src/stitch/api/db/model/resource.py

deployments/api/src/stitch/api/routers/oil_gas_fields.py

deployments/api/src/stitch/api/db/resource_actions.py

deployments/stitch-frontend/src/queries/api.js

Tweak member pkgs

feat: db models and entities

…ource_data in create

fix: address failing python tests

feat: rename actions files, split source/resource calls, add utils

Copilot

Pull request overview

Copilot reviewed 43 out of 44 changed files in this pull request and generated 11 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

deployments/stitch-frontend/src/components/OGFieldView.jsx

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py

deployments/api/src/stitch/api/db/utils.py

deployments/api/src/stitch/api/routers/oil_gas_field_sources.py

deployments/api/src/stitch/api/db/model/oil_gas_field_source.py

deployments/api/src/stitch/api/db/init_job.py

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py

deployments/api/src/stitch/api/routers/oil_gas_field_sources.py

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py

deployments/api/src/stitch/api/routers/oil_gas_fields.py

`-> Self` was causing FastAPI to be unable to generate the openAPI spec (and therefore unable to serve `/docs`).

mbarlow12

The only larger item is to try to split out the front end work, perhaps?

That'd involve reverting back to /resources temporarily (see https://github.com/RMI/stitch/pull/32/changes#r2907068697).

Other than that, the only piece is a bigger separation of the coalescing logic and updating the data within provenance structures.

mbarlow12 · 2026-03-09T16:57:12Z

deployments/api/src/stitch/api/db/model/oil_gas_field_source.py

+        return self.__class__.type_adapter.validate_python(self)
+
+    @classmethod
+    def from_entity(cls, entity: OGFieldSource):


NB
We can keep Self here (and in other class methods). These shouldn't ever reach the actual API layer, and for the most part, type checkers & LSPs will still register it as returning Self.

mbarlow12 · 2026-03-09T17:04:50Z

packages/stitch-models/tests/test_resource.py

+        res = ResourceWithSrcUnion.model_validate(
+            {"id": 1, "res_b": 4.5, "res_c": "hi", "source_data": data}
+        )
+        assert len(res.source_data) == 2


Suggested change

assert len(res.source_data) == 2

assert len(res.source_data) == 2

assert isinstance(res.source_data[0], FooSource)

assert isinstance(res.source_data[1], BarSource)

While I trust the validation in general, this is more concrete check that we actually have the objects we want.

mbarlow12 · 2026-03-09T17:32:27Z

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py

+                prov[fname] = picked_src
+
+        return OGFieldView(
+            id=int(self.id) if self.id is not None else 0,


NB
Since we're throwing an exception if self.id is None, we know that if we're at this point self.id is definitely not None, so the int cast and else 0 aren't necessary.

mbarlow12 · 2026-03-09T17:40:01Z

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py

+
+
+class OGFieldResource(OilGasFieldBase, Resource[int, OGFieldSource]):
+    def to_view(self) -> "OGFieldView":


Since we know that we'll have some kind of service/component that handles the coalescing of source_data. I think having this as a separate function will make it easier to integrate the coalesce features this/next sprint.

def coalesce_og_field_resource(resource: OGFieldResource) -> OGFieldView: ...

to_view can also accept a Callable as an argument:

def to_view(self, coalesce_fn: Callable[[Self], OGFieldView]) -> OGFieldView: return coalesesce_fn(self)

Then, somewhere else (probably stitch-api):

def coalesce_og_field_resource(...): # same logic as to_view # .... og_field_resource = get_og_field_resource(id=resource_id) view = og_field_resource.to_view(coalesce_og_field_resource) # pass the function itself # or og_view = coalesce_og_field_resource(og_field_resource) # call the function directly

This frees up the logic within the coalesce operation to perform whatever actions it may need (additional db fetches, permissions/auth checks, aggregating over multiple Source rows, etc...), and as long as it returns an OGFieldView, everything works correctly.

mbarlow12 · 2026-03-09T17:49:01Z

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py


 class OGFieldView(OilGasFieldBase):
    id: int
+    provenance: OGFieldProvenance


I'm hesitant to place provenance on the View. I'll think about it.

The original idea with provenance being on the Resource was that with a larger source_data object, provenance gave the necessary data for the UI to independently attribute certain data to its respective Resource(s) & Source(s).

But this is bringing up a distinction worth noting. We probably want to audit both:

what source contributed to this specific attribute

which resource(s) contributed to this source_data through merging

The question is then, do we want to capture all that in a single provenance class/object or split into 2?

mbarlow12 · 2026-03-09T17:53:32Z

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py

+    """Which source "won" for each coalesced field."""
+
+    # Keys are OilGasFieldBase field names, values are the `source` discriminator.
+    by_field: dict[str, OGSISrcKey] = Field(default_factory=dict)


I think we'll want a tuple of (OGSISrcKey, id). If multiple Source objects contribute to a single Resource, we'll lose some information if Owner comes from woodmac id 115 and Operator comes from woodmac id 77.

Suggested change

by_field: dict[str, OGSISrcKey] = Field(default_factory=dict)

by_field: Mapping[str, tuple[OGSISrcKey, int]] = Field(default_factory=dict)

mbarlow12 · 2026-03-09T17:56:58Z

packages/stitch-ogsi/src/stitch/ogsi/model/__init__.py

-
-
-class OGFieldResource(OilGasFieldBase, Resource[int, OGSourcePayload]): ...
+class OGFieldProvenance(BaseModel):


NB
We don't necessarily need a complete class here. Provenance could be an alias.

type OGFieldProvenance = dict[str, tuple[OGSISrcKey, int]] # or Mapping/MutableMapping

Really just eliminates the need to grab prov.by_field.

mbarlow12 · 2026-03-09T18:00:32Z

deployments/api/src/stitch/api/routers/oil_gas_fields.py

+router = APIRouter(
+    prefix="/oil-gas-fields",
+    tags=["oil_gas_fields"],
+    responses={404: {"description": "Not found"}},
+)


I'm realizing that we may want to temporarily revert back to /resources so the existing frontend won't break.

AlexAxthelm added 12 commits March 3, 2026 14:02

Add dependency on stitch models for API

2fa6e45

remove previous source model in favor of generic

2d08ac8

Remove domain model from init_job seed data

5ef1c25

linting: unused object

e68fcd7

update tests

6ec0a17

Merge branch 'main' into use-ogsi-model

0b55dd9

remove country from data model

5a7b844

Add OilGasField endpoint

4d67828

Get OGfields in frontend

85fddad

add OGfields in init_job

c47fc77

match path between frontend and api

7fd4a71

update path to kebab case

5bfbb24

AlexAxthelm changed the base branch from use-stitch-models to main March 4, 2026 22:10

AlexAxthelm requested review from Copilot, jdhoffa and mbarlow12 March 4, 2026 22:15

Copilot started reviewing on behalf of AlexAxthelm March 4, 2026 22:16 View session

Trigger CI

e904786

AlexAxthelm mentioned this pull request Mar 4, 2026

DNM: Placeholder data model #20

Closed

style: ruff

344e99b

Copilot AI reviewed Mar 4, 2026

View reviewed changes

AlexAxthelm added 4 commits March 5, 2026 10:35

add domain model to DB

da251d8

require user for fetching

6113520

rename source field file

faaf88f

use OilGasFieldSourceModel

2c2de18

AlexAxthelm self-assigned this Mar 5, 2026

AlexAxthelm added 4 commits March 5, 2026 13:57

seed ogfield source table

87b30fe

wip: fetching resources, no constituents

cbe3f12

style: format

ac8781b

wip; adding in og_field memberships

da6b8a0

mbarlow12 and others added 19 commits March 6, 2026 01:14

feat: rename actions files, split source/resource calls, add utils

e9d7607

update routers

30ae2cd

Merge pull request #34 from RMI/tweak-member-pkgs

90e6a35

Tweak member pkgs

Merge pull request #35 from RMI/ogsi-db-models

f6872fc

feat: db models and entities

fix(ogsi): flatten OGSISrcKey Literal union for SQLAlchemy compatibility

b91c6ed

fix(api): use resource_id param instead of builtin id, handle empty s…

41fed36

…ource_data in create

fix(tests): update imports and route paths to match restructured modules

85850ad

Merge pull request #37 from RMI/tests/integrate-models-test-fixes

f67a975

fix: address failing python tests

Merge pull request #36 from RMI/ogsi-db-rest

bfbbfa2

feat: rename actions files, split source/resource calls, add utils

linting: unused imports

0d5220b

style: format

2831af7

update args for entity creation to match mixin

1964611

iselect sources in load with resource

5320ea0

Explicitly query source members

53d09d3

use correct type

4e447f6

style: format

d7e3541

Cast single GET as OGFieldView

3ae0694

linting/formatting

91f8e9f

Add provenance data to model output

f0ad029

AlexAxthelm requested a review from Copilot March 6, 2026 11:51

Copilot started reviewing on behalf of AlexAxthelm March 6, 2026 11:51 View session

style: format

02cf08b

Copilot AI reviewed Mar 6, 2026

View reviewed changes

require user to get source data

bf430f1

AlexAxthelm marked this pull request as ready for review March 6, 2026 12:08

Merge branch 'main' into use-ogsi-model

8af20d9

AlexAxthelm marked this pull request as draft March 9, 2026 16:01

Remove Self from project

b420bae

`-> Self` was causing FastAPI to be unable to generate the openAPI spec (and therefore unable to serve `/docs`).

AlexAxthelm marked this pull request as ready for review March 9, 2026 16:15

mbarlow12 requested changes Mar 9, 2026

View reviewed changes



		class OGFieldResource(OilGasFieldBase, Resource[int, OGFieldSource]):
		def to_view(self) -> "OGFieldView":

	by_field: dict[str, OGSISrcKey] = Field(default_factory=dict)
	by_field: Mapping[str, tuple[OGSISrcKey, int]] = Field(default_factory=dict)



		class OGFieldResource(OilGasFieldBase, Resource[int, OGSourcePayload]): ...
		class OGFieldProvenance(BaseModel):

Conversation

AlexAxthelm commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mbarlow12 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlexAxthelm commented Mar 4, 2026 •

edited

Loading