Skip to content

Fix PendingModelChangesWarning for Unicode diacritics (NFC/NFD) on Linux#38197

Open
TomasMoralesBarr wants to merge 7 commits intodotnet:mainfrom
TomasMoralesBarr:fix/unicode-normalization-seed-diff
Open

Fix PendingModelChangesWarning for Unicode diacritics (NFC/NFD) on Linux#38197
TomasMoralesBarr wants to merge 7 commits intodotnet:mainfrom
TomasMoralesBarr:fix/unicode-normalization-seed-diff

Conversation

@TomasMoralesBarr
Copy link
Copy Markdown
Contributor

@TomasMoralesBarr TomasMoralesBarr commented Apr 30, 2026

Fixes #38191

Problem

On Linux/WSL2, string values containing diacritic characters (e.g. Café)
can be saved in NFD Unicode normalization form, while on Windows they are
typically NFC. EF Core's MigrationsModelDiffer used ordinal comparison
which treated NFC and NFD as different values, causing a false
PendingModelChangesWarning.

Fix

Normalize string values to NFC before comparison in DiffData().

Test

Added Seed_data_with_NFD_vs_NFC_unicode_string_is_considered_unchanged to
MigrationsModelDifferTest which confirms NFC and NFD strings representing
the same character produce no migration diff.

  • I've read the guidelines for contributing and seen the walkthrough
  • I've posted a comment on an issue with a detailed description of how I am planning to contribute and got approval from a member of the team
  • The code builds and tests pass locally (also verified by our automated build checks)
  • Commit messages follow this format:
        Summary of the changes
        - Detail 1
        - Detail 2

        Fixes #bugnumber
  • Tests for the changes have been added (for bug fixes / features)
  • Code follows the same patterns and style as existing code in this repo

Fixes dotnet#38191

On Linux/WSL2, string values with diacritic characters (e.g. 'Café') can be stored in NFD Unicode form, while on Windows they are typically NFC. EF Core's migration differ used ordinal string comparison which considered NFC and NFD as
different values, causing a false PendingModelChangesWarning.

Fix by normalizing string seed data values to NFC before comparison in DiffData."
Copilot AI review requested due to automatic review settings April 30, 2026 05:50
@TomasMoralesBarr TomasMoralesBarr requested a review from a team as a code owner April 30, 2026 05:50
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes false PendingModelChangesWarning on Linux/WSL2 caused by NFC vs NFD Unicode normalization differences when diffing seed data in migrations.

Changes:

  • Normalize string seed-data values prior to comparison during MigrationsModelDiffer.DiffData().
  • Add a regression test asserting that NFC and NFD equivalents in seed data produce no migration operations.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs Normalizes string values before comparing seed-data column modifications during diffing.
test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest.cs Adds a test covering NFC vs NFD equivalence for seed-data string values.

Comment thread src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs Outdated
Comment thread src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs Outdated
Comment thread test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest.cs Outdated
Comment thread test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest.cs Outdated
Comment thread test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest.cs Outdated
Use NormalizationForm.FormC explicitly instead of relying on the default
to make the intent clear to future readers of the code.
Copilot AI review requested due to automatic review settings April 30, 2026 06:14
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Addresses false PendingModelChangesWarning on Linux/WSL2 when seed data strings differ only by Unicode normalization form (NFC vs NFD), by normalizing string values prior to comparison in the seed data diff logic.

Changes:

  • Normalize string seed values to NFC before equality comparison in MigrationsModelDiffer.DiffData().
  • Add a regression test asserting that canonically equivalent NFC/NFD seed strings produce no migration operations.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs Normalizes string values (NFC) before comparing source/target seed data values.
test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest.cs Adds a regression test covering NFC vs NFD equivalence for seeded string values.

Comment thread src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs Outdated
Comment thread test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest.cs Outdated
Comment thread test/EFCore.Relational.Tests/Migrations/Internal/MigrationsModelDifferTest.cs Outdated
Comment thread src/EFCore.Relational/Migrations/Internal/MigrationsModelDiffer.cs Outdated
Copy link
Copy Markdown
Member

@AndriySvyryd AndriySvyryd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @roji that a more robust way of fixing this is by using StringComparison.InvariantCulture in a new comparer returned from

public static ValueComparer CreateDefault(

That way it will also benefit the update pipeline.

TomasMoralesBarr and others added 2 commits May 2, 2026 19:02
…alence

- Add DefaultStringValueComparer in ValueComparer using StringComparison.InvariantCulture
- Route string type through DefaultStringValueComparer in CreateDefault<T>()
- Remove NFC normalization workaround from MigrationsModelDiffer.DiffData()
- Restore regression test for NFC/NFD seed data in MigrationsModelDifferTest
- Add unit tests for DefaultStringValueComparer in ValueComparerTest

Fixes dotnet#38191"
Copilot AI review requested due to automatic review settings May 4, 2026 03:42
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

// produce the same hash code.
public override int GetHashCode(string instance)
=> StringComparer.InvariantCulture.GetHashCode(instance);

Comment on lines +307 to +310
if (nonNullableType == typeof(string))
{
return new DefaultStringValueComparer(favorStructuralComparisons);
}
var comparer = ValueComparer.CreateDefault<string>(false);

Assert.True(comparer.Equals(nfcString, nfdString));
Assert.Equal(comparer.GetHashCode(nfcString), comparer.GetHashCode(nfdString));
…ueComparer

- Update ValueComparerExtensions.IsDefault() to walk the base type chain,
  so subclasses of DefaultValueComparer<T> (e.g. DefaultStringValueComparer)
  are correctly recognized as default comparers
- Update CosmosTypeMappingSourceTest assertions from DefaultValueComparer<string>
  to DefaultStringValueComparer
- Regenerate compiled model baselines to reflect new string equality expression
  (string.Equals with InvariantCulture instead of ==)"
Replace 'v1 == v2' with 'string.Equals(v1, v2, StringComparison.InvariantCulture)'
in all Scaffolding/Baselines across SqlServer and Cosmos test projects,
reflecting the new DefaultStringValueComparer expression tree output."
Copilot AI review requested due to automatic review settings May 4, 2026 06:28
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 57 out of 62 changed files in this pull request and generated 1 comment.

Comment on lines +384 to +399
#pragma warning disable CA1309 // Use ordinal StringComparison - InvariantCulture is intentional to handle Unicode canonical equivalence (NFC/NFD)
internal sealed class DefaultStringValueComparer(bool favorStructuralComparisons)
: DefaultValueComparer<string>((v1, v2) => string.Equals(v1, v2, StringComparison.InvariantCulture), favorStructuralComparisons)
{
private static readonly MethodInfo StringEqualsMethodInfo
= typeof(string).GetMethod(nameof(string.Equals), [typeof(string), typeof(string), typeof(StringComparison)])!;

// String canonical equivalence (e.g. NFC vs NFD Unicode normalization) is handled by InvariantCulture comparison.
// Override hash code to be consistent with InvariantCulture equality so that canonically equivalent strings
// produce the same hash code.
public override int GetHashCode(string instance)
=> StringComparer.InvariantCulture.GetHashCode(instance);

public override Expression ExtractEqualsBody(Expression leftExpression, Expression rightExpression)
=> Expression.Call(StringEqualsMethodInfo, leftExpression, rightExpression, Expression.Constant(StringComparison.InvariantCulture));
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PendingModelChangesWarning Diacritic Café

3 participants