File-level directives: allow quoting and additional properties - #55592
File-level directives: allow quoting and additional properties#55592jjonescz wants to merge 22 commits into
Conversation
|
Azure Pipelines: Successfully started running 2 pipeline(s). 1 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
This PR updates file-based program #: directive parsing/emission to support double-quoted values (for whitespace) and additional MSBuild item metadata on #:package, #:project, and #:ref, and adds a new analyzer (CA2267) to help migrate legacy unquoted-whitespace directives.
Changes:
- Extend file-level directive parsing to support quoted values and trailing
Name=Valuemetadata, and emit metadata into generated MSBuild items. - Add/expand CLI and conversion tests for quoting and metadata round-tripping.
- Introduce CA2267 (“Prefer quoted file-based program directive values”) with code fix, plus documentation/resource updates.
Reviewed changes
Copilot reviewed 46 out of 46 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/dotnet.Tests/CommandTests/Run/RunFileTests_Directives.cs | Adds E2E coverage for #:ref metadata emitted into <ProjectReference>. |
| test/dotnet.Tests/CommandTests/Run/FileBasedAppSourceEditorTests.cs | Tests directive editing/round-tripping with metadata and quoting. |
| test/dotnet.Tests/CommandTests/Project/Convert/DotnetProjectConvertTests.cs | Adds conversion tests for quoting/metadata and updates escaping/whitespace expectations. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/tests/Microsoft.CodeAnalysis.NetAnalyzers.UnitTests/Microsoft.NetCore.Analyzers/Usage/PreferQuotedFileBasedProgramDirectiveTests.cs | New unit tests for CA2267 analyzer + fixer. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Utilities/Compiler/DiagnosticCategoryAndIdRanges.txt | Extends Usage ID range to include CA2267. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.zh-Hant.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.zh-Hans.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.tr.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.ru.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.pt-BR.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.pl.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.ko.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.ja.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.it.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.fr.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.es.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.de.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/xlf/MicrosoftNetCoreAnalyzersResources.cs.xlf | Adds localized entries for new CA2267 strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/Usage/PreferQuotedFileBasedProgramDirective.Fixer.cs | Adds shared fixer base for CA2267. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/Usage/PreferQuotedFileBasedProgramDirective.cs | Adds shared diagnostic descriptor for CA2267. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/MicrosoftNetCoreAnalyzersResources.resx | Adds CA2267 resource strings. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/AnalyzerReleases.Unshipped.md | Registers CA2267 in unshipped analyzer list. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers.sarif.template | Adds CA2267 to SARIF template metadata. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers.md | Adds generated documentation entry for CA2267. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.CSharp.NetAnalyzers/Microsoft.NetCore.Analyzers/Usage/FileBasedProgramDirectiveQuoting.cs | Adds shared quoting detection/rewrite logic for analyzer/fixer. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.CSharp.NetAnalyzers/Microsoft.NetCore.Analyzers/Usage/CSharpPreferQuotedFileBasedProgramDirective.Fixer.cs | Implements C# code fix for CA2267. |
| src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.CSharp.NetAnalyzers/Microsoft.NetCore.Analyzers/Usage/CSharpPreferQuotedFileBasedProgramDirective.cs | Implements C# analyzer for CA2267. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.zh-Hant.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.zh-Hans.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.tr.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.ru.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.pt-BR.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.pl.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.ko.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.ja.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.it.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.fr.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.es.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.de.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/xlf/FileBasedProgramsResources.cs.xlf | Adds localized CLI strings for new directive parsing errors. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/VirtualProjectBuilder.cs | Emits directive metadata as child elements on generated MSBuild items. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/InternalAPI.Unshipped.txt | Tracks new internal API surface for directive metadata properties. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs | Implements tokenization, quoted values, and metadata parsing for directives. |
| src/Cli/Microsoft.DotNet.FileBasedPrograms/FileBasedProgramsResources.resx | Adds/updates directive parsing error messages. |
| src/Cli/dotnet/Commands/Project/Convert/ProjectConvertCommand.cs | Preserves #:ref metadata when converting to a project. |
| documentation/general/dotnet-run-file.md | Documents quoting rules, legacy behavior, and new metadata support. |
Suppressed comments (1)
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:565
- Metadata element names are emitted directly into the generated project file. XmlConvert.VerifyName allows ':' (QName) which can yield invalid MSBuild XML without an xmlns declaration. Use VerifyNCName here to ensure generated elements are always safe.
try
{
name = XmlConvert.VerifyName(name);
}
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 46 out of 46 changed files in this pull request and generated 1 comment.
Suppressed comments (5)
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:366
- Token.ContainsDiagnostics is not specific to unterminated quotes; it will also be true for invalid escape sequences (e.g. "\q"). Reporting UnterminatedQuoteInDirective for all diagnostics can mislead users. Distinguish the unterminated-quote case (missing closing ") from other invalid string literal diagnostics and use InvalidQuoteInDirective for the latter.
var token = SyntaxFactory.ParseToken(text, offset: i);
if (token.ContainsDiagnostics)
{
context.ReportError(FileBasedProgramsResources.UnterminatedQuoteInDirective);
return null;
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:572
- Metadata names are emitted as XML element names. XmlConvert.VerifyName permits ':' (QName), which can yield invalid MSBuild XML when written as <a:b> without a namespace binding. Prefer XmlConvert.VerifyNCName here to prevent generating malformed project files.
try
{
name = XmlConvert.VerifyName(name);
}
catch (XmlException ex)
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:563
- ParseMetadata treats tokens like "=value" as valid because it only checks separatorIndex < 0. XmlConvert.VerifyName("") will throw, which can surface as an unhandled exception instead of a directive parse error. Treat separatorIndex <= 0 as invalid metadata and report InvalidDirectiveMetadata.
var separatorIndex = token.IndexOf('=');
if (separatorIndex < 0)
{
context.ReportError(string.Format(FileBasedProgramsResources.InvalidDirectiveMetadata, token));
return null;
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:984
- Ref.Metadata is left uninitialized (default ImmutableArray), unlike Package/Project which default to Empty. Default ImmutableArray can throw on common operations and forces callers to handle IsDefault. Initialize to ImmutableArray.Empty for consistency and safer consumption.
public ImmutableArray<(string Name, string Value)> Metadata { get; init; }
test/dotnet.Tests/CommandTests/Run/RunFileTests_Directives.cs:350
- The new RefDirective_Metadata test summary claims it validates that trailing #:ref metadata is emitted into the generated , but the test only asserts the app output. If metadata emission regresses (metadata dropped), the build/run would likely still succeed and this test would not fail. Add an assertion that inspects the generated virtual project and verifies test and a b c are present under the ProjectReference.
|
Azure Pipelines: Successfully started running 2 pipeline(s). 1 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
| async ct => | ||
| { | ||
| var text = await context.Document.GetTextAsync(ct).ConfigureAwait(false); | ||
| return context.Document.WithText(text.Replace(triviaSpan, newDirectiveText)); |
There was a problem hiding this comment.
This is going to reload the document for each individual fix in a multi-fix scenario, right?
That's why we have SyntaxEditorBasedCodeFixProvider instead to help force down the path that allows actually batching it all in a single pass?
There was a problem hiding this comment.
Thanks, I'm not that familiar with NetAnalyzers.
There was a problem hiding this comment.
With the recent changes I've tried to generally set it up to generally be inline with what Roslyn is internally doing. There's probably a few nuances, but Roslyn is the "gold star" here so we want to follow it where feasible.
3d3c0d2 to
a7d2d57
Compare
|
@333fred @RikkiGibson for reviews, thanks |
|
@333fred @RikkiGibson for reviews, thanks |
| /// (<c>FileBasedProgramDirectiveQuoting</c>), so both agree on quoting, name validity, and metadata | ||
| /// detection instead of each duplicating the logic. |
There was a problem hiding this comment.
Redunant comment elements.
| /// (<c>FileBasedProgramDirectiveQuoting</c>), so both agree on quoting, name validity, and metadata | |
| /// detection instead of each duplicating the logic. | |
| /// (<c>FileBasedProgramDirectiveQuoting</c>). |
| { | ||
| // Characters that are not allowed in a directive or metadata name because they would be confused | ||
| // with a separator: whitespace, '@', '=', '/'. | ||
| private static readonly Regex s_disallowedNameCharacters = new("""[\s@=/]"""); |
There was a problem hiding this comment.
Surely we should at least be using a generated regex here, or just string.IndexOfAny?
There was a problem hiding this comment.
This started as generated regex but was previously switched to normal regex since the code was switched to target netstandard2.0. I guess I can simplify via manual checks, but not really via IndexOfAny due to usage of \s.
| /// Returns whether every token from <paramref name="start"/> onwards is a valid <c>Name=Value</c> | ||
| /// item-metadata pair (a valid MSBuild name, then <c>'='</c>, then any value). | ||
| /// </summary> | ||
| public static bool AllValidMetadata(IReadOnlyList<string> tokens, int start) |
There was a problem hiding this comment.
Can we use something more concrete than IReadOnlyList?
There was a problem hiding this comment.
I guess we can switch to ImmutableArray.
| { | ||
| // FormatLiteral produces a properly escaped C# string literal (e.g. "a\"b", "a\tb") that | ||
| // the parser decodes back to the original value. | ||
| return SymbolDisplay.FormatLiteral(value, quote: true); |
There was a problem hiding this comment.
We should probably always be calling FormatLiteral, and just changing whether to quote or not.
There was a problem hiding this comment.
FormatLiteral(value, quote: false) still escapes backslashes and other characters, but bare directive tokens are not decoded as C# literals (and I think it should stay that way for backwards compatibility). See also FileBasedAppSourceEditorTests.AddWithSpecialCharactersEscapes.
| // escape sequences. Verbatim (@"...") literals can't start here (the '@' would precede the | ||
| // quote and fail the check above), and raw ("""...""") literals lex to a different token kind | ||
| // and are rejected below. | ||
| var token = SyntaxFactory.ParseToken(text, offset: i); |
There was a problem hiding this comment.
I'm a tad concerned that this is an expensive operation, but I don't think that the tokenizer API we added for Razor would actually work here (at least, not without constant resets to position 0 and skipping, which may or may not work). Open to other ideas on how to avoid allocating a new source text and lexer every time we need to look at a token.
There was a problem hiding this comment.
Good question, thanks. I think we can share the tokenizer, let me try it.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 48 out of 48 changed files in this pull request and generated no new comments.
Suppressed comments (6)
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:487
- Issue #49367 explicitly requires
#:package "Humanizer"@2.0, but after lexing the quoted name, the following@enters this branch and reportsInvalidQuoteInDirective. Permit the package/SDK separator after a quoted name (and add the issue's exact example as a parser test) so the PR actually supports the syntax it resolves.
if (quoteClosed)
{
context.ReportError(FileBasedProgramsResources.InvalidQuoteInDirective);
return null;
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:775
- Issue #49367 also requires
#:property Xyz = "abc ". Strict tokenization producesXyz,=, andabc, so this branch rejects that required example instead of preserving the quoted value's trailing space. Parse optional whitespace around the property separator before enforcing the single logical name/value token, and cover the exact issue example.
if (tokens.Length > 1)
{
context.ReportError(string.Format(FileBasedProgramsResources.UnexpectedDirectiveText, context.DirectiveKind));
return null;
src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.NetAnalyzers/Microsoft.NetCore.Analyzers/MicrosoftNetCoreAnalyzersResources.resx:2313
- These messages promise that the diagnostic and fix will add quotes, but the new tests intentionally fix
#:property Prop = ValuetoProp=Valueand#:package First @ 1.0toFirst@1.0without quotes. Use wording such as “rewrite using unambiguous syntax” so the warning and code-action title accurately describe every offered fix.
<data name="PreferQuotedFileBasedProgramDirectiveMessage" xml:space="preserve">
<value>Wrap the value of the '#:{0}' directive in quotes; the unquoted-whitespace form is deprecated</value>
</data>
<data name="PreferQuotedFileBasedProgramDirectiveCodeFixTitle" xml:space="preserve">
<value>Add quotes around the directive value</value>
src/Cli/Microsoft.DotNet.FileBasedPrograms/FileLevelDirectiveHelpers.cs:1362
- Metadata names are parsed case-insensitively (
ParseMetadatauses anOrdinalIgnoreCaseset), but tupleSequenceEqualcompares them case-sensitively. Consequently, otherwise identical duplicate package directives usingPrivateAssetsversusprivateassetsare incorrectly reported as conflicting. Compare metadata names withOrdinalIgnoreCasewhile retaining ordinal value comparison.
(CSharpDirective.Package existing, CSharpDirective.Package current) =>
string.Equals(existing.Version, current.Version, StringComparison.Ordinal) &&
existing.Metadata.SequenceEqual(current.Metadata),
documentation/general/dotnet-run-file.md:213
- This overstates the compatibility fallback: for
package,project, andref, an unquoted remainder such aspath A=Bis interpreted as metadata rather than as one legacy value. Qualify the statement so users understand that fallback happens only when the trailing tokens cannot be parsed as the new metadata form.
For backward compatibility, a directive whose value contains no double quotes is still accepted in a
*legacy mode*: the entire remainder after the name and separator is taken verbatim as a single value
(including any internal whitespace), matching how these directives behaved before quoting and metadata
were supported. Analyzer [CA2267](https://learn.microsoft.com/dotnet/fundamentals/code-analysis/quality-rules/ca2267)
flags such legacy directives and offers a code fix to rewrite them into the quoted form.
src/Microsoft.CodeAnalysis.NetAnalyzers/src/Microsoft.CodeAnalysis.CSharp.NetAnalyzers/Microsoft.NetCore.Analyzers/Usage/FileBasedProgramDirectiveQuoting.cs:97
- This marks every multi-token
propertyorsdkdirective as a deprecated legacy form even when the CLI parser never accepted it—for example,#:property Name Value(missing=) and#:sdk Name Value(no@, so the name contains forbidden whitespace). Those malformed directives receive a misleading CA2267 warning with no fix. Mirror the parser's required separator/name validation before reporting these cases.
case "property":
case "sdk":
case "include":
case "exclude":
return true;
Resolves #52399.
Resolves #49367.
Docs: dotnet/docs#55352