Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,24 @@ covering only what changed for that package:
[PostgreSQL](src/EFCore.ComplexIndexes.PostgreSQL/CHANGELOG.md),
[SQL Server](src/EFCore.ComplexIndexes.SqlServer/CHANGELOG.md).

## 5.2.0

The features AuditOffice's review of its own workarounds asked for, in the order they pay off:
a validation for a failure that reports nothing, a read model so an application can check its
own obligations, filters that resolve property paths the way index parts already do, an amend
API, and typed filter predicates.

- **Changed:** an index or constraint name longer than the provider's identifier limit is rejected at `dotnet ef migrations add`. PostgreSQL truncates a name past 63 bytes with a NOTICE and applies the migration cleanly, so the index exists under a name that neither the declaration nor a later constraint-violation error reports — a slice dispatching on the constraint name falls through in silence; SQL Server rejects the statement at apply time instead. Explicit and default names alike are checked, measured the way the provider measures them (bytes on PostgreSQL, characters on SQL Server), on the target model only. The names this package derives are never truncated, unlike EF Core's own default names, so a long table name plus a long column path reaches the limit quietly, and a default temporal foreign key name, built from two table names, is the first to. Give the declaration a name; a name-only change renames in place.

- **New:** a read model. `GetComplexIndexes()` on an entity type or the model returns every declaration this package holds — property-level, entity-level, composite and expression indexes alike — as `ComplexIndexDeclaration`s carrying the parts as property paths, `IsUnique`, `Filter`, the explicit `Name` and, for entity-level declarations, the provider options; `FindComplexIndex(name)` looks one up by explicit name. The PostgreSQL package adds `GetExclusionConstraints()` and `FindExclusionConstraint(name)` for exclusion constraints, whose element type `ExclusionPartDefinition` and annotation key `NpgsqlExclusionAnnotations.Constraints` are now public. Both work on the mutable model inside `OnModelCreating`, so an application can check a convention such as "every unique index and exclusion constraint on a withdrawable aggregate is filtered to live rows" while the model is built. The differ builds its own descriptors from the same readers, so what the read model reports is what the migration is scaffolded from.

- **New:** filters resolve `{Property.Path}` placeholders, the way expression parts already did — on complex, composite and expression indexes, and on PostgreSQL exclusion constraints. `filter: "{RevokedAt} IS NULL"` becomes `"revoked_at" IS NULL` (`[revoked_at]` on SQL Server) at `migrations add`, honouring `HasColumnName` and, on PostgreSQL, `ToJson()` members; the resolved text is what the migration carries, rendered by the stock generator with no runtime wiring, and what the snapshot is compared on. Only a brace pair holding a dotted identifier path outside a single-quoted literal is a placeholder, so existing filters with array or JSON literals (`'{urgent}'`, `'{"a": 1}'`) are unaffected, and a placeholder naming no property fails loudly. Template resolution now lives in the core differ over a new `QuoteIdentifier` seam, which is also why the SQL Server satellite resolves them.
- **New:** property paths see through a value converter. For a value object mapped as one column, `x => x.Email.Value` — in `HasComplexIndex`, a typed `HasExpressionIndex`, an exclusion element, or a filter placeholder — resolves to that column, provided the member's type is the converter's provider type. Without that guard `x.CreatedAt.Year` would have indexed the whole column; it still fails, and says so.

- **New:** a mutable API. On an `IMutableEntityType`, `AddComplexIndexFilter(predicate, where)` ANDs a predicate onto the filter of every selected complex index — property-level and entity-level alike — so a shared convention can install a live-rows filter the way it installs the query filter, instead of every configuration repeating it; `AddComplexIndex(definition)` adds an entity-level declaration with the fluent API's identity and name rules. The PostgreSQL package adds `AddExclusionConstraintFilter`. An unfiltered declaration gets the predicate, a filtered one `(existing) AND (predicate)`, one that already carries it is left alone, so the calls are safe to repeat. They amend what is declared at the time of the call, so they belong after the configurations.

- **New:** typed filter predicates on PostgreSQL. Every `filter:` string has a lambda form — `HasComplexIndex(x => x.Email, x => x.RevokedAt == null)`, `HasComplexCompositeIndex(…, x => …)`, `HasExpressionIndex(…, x => …)`, `HasExclusionConstraint(…, x => …)` — and the index and constraint builders take `HasFilter(x => …)` (`HasFilter<TEntity>` on the non-generic index builders). The predicate is translated at the declaration into a filter template, so the column names come from the model: `==`/`!=` (`IS NULL`/`IS NOT NULL` against null), `<`, `<=`, `>`, `>=`, `&&`, `||`, `!`, boolean properties, and on the operands the string operations and constants a typed expression index accepts. Enums are refused — how one is stored depends on a value conversion the filter cannot see, so `Status == Status.Active` would compare a text column against `0` and fail at apply time — as are values with no portable SQL spelling (`DateTime`, `Guid`), which the typed expression translator used to render as bare text. The amend calls have typed forms too.

## 5.1.0

Small enhancements around the two seams, plus the silent failures found while planning the next
Expand Down
48 changes: 46 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,29 @@ revoked rows" collapsed to one constraint. Name collisions matter more here than
every ADD is preceded by `DROP CONSTRAINT IF EXISTS`, so a duplicate name does not fail at apply
time — the second constraint silently replaces the first.

### The read model is the differ's reader

`ComplexIndexModelExtensions.GetDeclaredComplexIndexes` (core) and
`NpgsqlExclusionModelExtensions.GetDeclaredExclusionConstraints` (PostgreSQL) turn the annotations
into `ComplexIndexDeclaration` / `ExclusionConstraintDeclaration` objects — the public surface an
application uses to check its own conventions ("every unique index and exclusion constraint on a
withdrawable aggregate is filtered"). Since 5.2.0 the differs build their descriptors from exactly
these readers and resolve columns on top; there is no second parser. Keep it that way: a read model
that disagrees with the differ silently checks something other than what the migration enforces,
which is the outcome the guard exists to prevent. Declarations are reported unresolved — parts as
property paths, `Name` null for a default-named one — because resolution needs the relational model
and, for JSON members and templates, the satellite; `FindComplexIndex` therefore matches explicit
names only.

The mutable side (`ComplexIndexMutableExtensions`, `NpgsqlExclusionMutableExtensions`) works on
`IMutableEntityType`: `AddComplexIndex` is `ComplexIndexStorage.AddOrReplace` retyped, so it
carries the same identity and name rules; `AddComplexIndexFilter` / `AddExclusionConstraintFilter`
AND a predicate onto selected declarations through `ComplexIndexStorage.Conjoin`, whose
idempotence rule is deliberately narrow — the predicate is "already there" only when it is the
whole filter or the exact conjunct this method appends. They amend what is declared *at the time
of the call*: from `OnModelCreating` after the configurations, never from a model-finalizing
convention, since a convention-source annotation write cannot overwrite the explicit blob.

### Two integration seams: design-time vs. runtime

There are two distinct hook points, and it matters which one a feature uses:
Expand Down Expand Up @@ -340,7 +363,28 @@ differ let satellites resolve what the core cannot:
nested inside the document resolves to a `->` extraction yielding `jsonb`. A table-split complex
property stays unresolved: there is no single column to stand for it.
- `ResolveTemplatePart` — substitutes template placeholders with quoted columns or parenthesized
JSON extractions; core throws (identifier quoting is provider-specific).
JSON extractions. Since 5.2.0 the core implements it, quoting through the `QuoteIdentifier`
virtual (ANSI by default; the SQL Server satellite brackets), so satellites override neither.

Filters go through the same placeholder resolver (`ResolveFilter`, called while building
descriptors for indexes and, in the Npgsql differ, exclusion constraints), with a narrower rule
because filters are pre-existing SQL: only a brace pair holding a dotted identifier path *outside a
single-quoted literal* is a placeholder, there is no `{{` escape (`'{{1,2},{3,4}}'` is an array
literal), and an unresolvable placeholder throws. The resolved filter is what the descriptor
carries on both sides of the diff, so placeholder filters never churn and need no runtime wiring.
`ResolveProperty` (core, shared by every path walk) also unwraps one member of a converter-mapped
value object — `Email.Value` resolves to the `Email` column only when the property has a converter
and the member's type equals the converter's provider type; without that check `CreatedAt.Year`
would silently index the whole column.

Typed filters (`NpgsqlTypedFilterExtensions`, PostgreSQL only) run
`NpgsqlLinqIndexTranslator.TranslatePredicate` at the declaration and store the resulting
placeholder template as the filter string — nothing downstream knows the filter was typed. The
predicate subset is boolean structure over the expression translator's operands, with two
deliberate refusals: enums, because the stored form depends on a value conversion the translator
cannot see (`Status == Status.Active` would compare `text` against `0` and fail at apply), and
literals without a portable SQL spelling (`DateTime`, `Guid`), which `IFormattable` used to render
as bare text. Both throw at the declaration.

`NULLS FIRST/LAST` (`DbOrder.NullsFirst/NullsLast`, `ExpressionIndexBuilder.NullsFirst()/NullsLast()`)
rides on the parts as `NullSort`. EF's native `CreateIndexOperation` has no slot for it, so any
Expand Down Expand Up @@ -404,7 +448,7 @@ still skipped silently — an index on those is nothing this package could creat

### Key extension points

- **Adding a new provider**: Subclass `CustomMigrationsModelDiffer` (override `IsForwardedIndexAnnotation`, optionally `ValidateCreateIndexOperation`/`ResolveUnmappedPart`/`ResolveTemplatePart`), implement `IDesignTimeServices` to replace the differ, and ship a `.targets` file that injects the attribute (with `ForProvider` set). The PostgreSQL project is the full-featured reference; the SQL Server project is the minimal one (whitelist + validation, no custom SQL generator).
- **Adding a new provider**: Subclass `CustomMigrationsModelDiffer` (override `IsForwardedIndexAnnotation`, optionally `ValidateCreateIndexOperation`/`ResolveUnmappedPart`/`QuoteIdentifier`), implement `IDesignTimeServices` to replace the differ, and ship a `.targets` file that injects the attribute (with `ForProvider` set). The PostgreSQL project is the full-featured reference; the SQL Server project is the minimal one (whitelist + validation, no custom SQL generator).
- **New index options**: Add constants to `ComplexIndexAnnotations.cs` (or `NpgsqlAnnotations.cs`), expose them via `ComplexIndexBuilder`, and read them in the differ when constructing `CreateIndexOperation`.

### Expression path extraction
Expand Down
4 changes: 2 additions & 2 deletions Directory.Build.props
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<Project>
<PropertyGroup>
<Version>5.1.0</Version>
<Version>5.2.0</Version>
<Authors>CaffeinatedCoder</Authors>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
Expand Down Expand Up @@ -63,7 +63,7 @@
-->
<PropertyGroup>
<EnablePackageValidation>true</EnablePackageValidation>
<PackageValidationBaselineVersion>5.0.3</PackageValidationBaselineVersion>
<PackageValidationBaselineVersion>5.1.0</PackageValidationBaselineVersion>
</PropertyGroup>

<!--
Expand Down
72 changes: 69 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,19 @@ builder.ComplexProperty(x => x.EmailAddress, c =>
The same overloads exist on the non-generic builder, so a property configured by name works too:
`c.Property("Value").HasComplexIndex()`.

A filter may name properties instead of columns: `{Property.Path}` placeholders resolve to the
mapped column at `migrations add` — `HasColumnName`, complex members and (on PostgreSQL) `ToJson()`
members included — so the filter and the column mapping cannot drift apart:

```csharp
c.Property(x => x.Value).HasComplexIndex(isUnique: true, filter: "{DeletedAt} IS NULL");
// WHERE "deleted_at" IS NULL (PostgreSQL) WHERE [deleted_at] IS NULL (SQL Server)
```

The resolved text is baked into the migration, so no runtime wiring is involved. Only a brace pair
holding a dotted identifier path, outside a single-quoted literal, is a placeholder — `'{urgent}'`
and `'{"a": 1}'` stay what they are — and one that names no property fails loudly.

A property-level declaration holds **one** index per property. To give the same column several
differently-filtered indexes (the classic soft-delete pattern), declare them at the **entity level**
— the selector reaches into complex properties, and each index needs its own explicit name:
Expand All @@ -128,10 +141,18 @@ builder.HasComplexIndex(x => x.EmailAddress.Value,
indexName: "ix_person_email_all");
```

Selectors also see through a value converter: for a value object mapped as one column
(`HasConversion(e => e.Value, v => new(v))`), `x => x.Email.Value` resolves to that column,
provided the member's type is the converter's provider type. `x => x.CreatedAt.Year` does not
resolve, and says so.

Index names must be unique per table, and the package enforces it rather than letting the database
reject the migration: reusing a name throws at the declaration, and two declarations that resolve to
the same name — including a property-level and an entity-level index over one column, which share a
default name — throw during `dotnet ef migrations add`.
default name — throw during `dotnet ef migrations add`. So does a name longer than the provider's
identifier limit: PostgreSQL would otherwise truncate it to 63 bytes with a NOTICE and apply the
migration cleanly, leaving the index under a name that no declaration and no constraint-violation
error ever reports. Default names are checked too, since this package never truncates them.

### Composite index across scalar and nested properties

Expand All @@ -156,6 +177,51 @@ Direction maps to EF Core's native `CreateIndexOperation.IsDescending`, so it is

Markers of different kinds compose in any order; markers of the same kind do not — `DbOrder.Asc(DbOrder.Desc(x.A))` is a contradiction and throws. To control where nulls sort, see [null ordering](docs/postgresql-indexes.md#per-column-null-ordering) (PostgreSQL only).

### Reading declarations back

Every index declared through this package can be read back from the model — the finalized
`context.Model`, or the mutable one inside `OnModelCreating` — so an application can enforce its
own conventions instead of trusting each configuration to remember them:

```csharp
// "Every unique index on a withdrawable aggregate is filtered to live rows."
var unfiltered = modelBuilder.Model.GetEntityTypes()
.Where(IsWithdrawable)
.SelectMany(e => e.GetComplexIndexes())
.Where(ix => ix.IsUnique && ix.Filter is null)
.ToList();

var byName = modelBuilder.Model.FindComplexIndex("ux_person_email_active");
```

`GetComplexIndexes()` unifies property-level, entity-level, composite and expression indexes as
`ComplexIndexDeclaration`s: parts as property paths, `IsUnique`, `Filter`, the explicit `Name`
(null when the differ derives the default from resolved column names — those are not matched by
`FindComplexIndex`), and for entity-level declarations the provider options. It reports what was
*declared*; column names are resolved by the differ only. `GetDeclaredComplexIndexes()` leaves
inherited declarations to the type that declares them. The differ reads the model through the
same code, so the read model and the migration cannot disagree. PostgreSQL exclusion constraints
have the same surface — see [reading constraints back](docs/postgresql-constraints.md#reading-constraints-back).

### Amending declarations

The same convention can be *installed* rather than checked. On the mutable model,
`AddComplexIndexFilter` ANDs a predicate onto the filter of every selected declaration —
property-level and entity-level alike — and `AddComplexIndex` adds an entity-level declaration
with the fluent API's identity rules:

```csharp
// At the end of OnModelCreating, after the configurations have been applied:
foreach (var entityType in modelBuilder.Model.GetEntityTypes().Where(IsWithdrawable))
entityType.AddComplexIndexFilter("{RevokedAt} IS NULL", ix => ix.IsUnique);
```

An unfiltered index gets the predicate; a filtered one gets `(existing) AND (predicate)`; one that
already carries it is left alone, so the call is safe to repeat. It amends what is declared *at the
time of the call*, which is why it belongs after the configurations — and why the read model is
still worth a check for what a later declaration might add. Exclusion constraints have
`AddExclusionConstraintFilter` — see [amending constraints](docs/postgresql-constraints.md#amending-constraints).

---

## Documentation
Expand All @@ -164,8 +230,8 @@ Provider-specific features live in their own pages:

| Page | Covers |
|---|---|
| **[PostgreSQL — indexes](docs/postgresql-indexes.md)** | Index methods (GIN, GiST, BRIN, SP-GiST, Hash), operator classes, `INCLUDE`, expression (functional) indexes in raw SQL and typed LINQ, JSON member indexes, `NULLS FIRST/LAST` |
| **[PostgreSQL — temporal and exclusion constraints](docs/postgresql-constraints.md)** | `UNIQUE … WITHOUT OVERLAPS`, temporal foreign keys (`PERIOD`), `EXCLUDE` constraints with `WHERE` predicates, the `btree_gist` extension |
| **[PostgreSQL — indexes](docs/postgresql-indexes.md)** | Index methods (GIN, GiST, BRIN, SP-GiST, Hash), operator classes, `INCLUDE`, expression (functional) indexes in raw SQL and typed LINQ, typed filter predicates, JSON member indexes, `NULLS FIRST/LAST` |
| **[PostgreSQL — temporal and exclusion constraints](docs/postgresql-constraints.md)** | `UNIQUE … WITHOUT OVERLAPS`, temporal foreign keys (`PERIOD`), `EXCLUDE` constraints with `WHERE` predicates, reading and amending them, the `btree_gist` extension |
| **[SQL Server](docs/sqlserver.md)** | Clustered/nonclustered, covering (`INCLUDE`), online builds, fill factor, sort-in-tempdb, data compression — and the declarations SQL Server rejects outright |

Working on the package itself: [CONTRIBUTING.md](CONTRIBUTING.md) covers the setup and the quality
Expand Down
4 changes: 2 additions & 2 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ remedy for those is to upgrade.

| Version | Supported |
|---|---|
| 5.1.x | ✅ |
| < 5.1 | ❌ |
| 5.2.x | ✅ |
| < 5.2 | ❌ |

### For how long

Expand Down
Loading
Loading