feat: Set User-agent header when requesting schema over HTTP(S)

[svifred]

What does this PR do?

Some CDNs block requests, when User-agent header is unset. This PR adds the User-agent header til HTTP(S) requests for schemas.

What issues does this PR fix or reference?

No issue in GitHub, but problem has been spotted "in the wild", where CloudFlare challenges requests for schemas in VS Code.

Is it tested? How?

Only tested in unit tests.

Note on AI assistance

Extensive AI assistance was used to create this PR, specifically Claude Sonnet 4.6 using OpenCode.

[datho7561]

This seems like it would be pretty helpful. Do you have an example of a schema behind a CDN that rejects requests without a User-Agent to test against?

[svifred]

This seems like it would be pretty helpful. Do you have an example of a schema behind a CDN that rejects requests without a User-Agent to test against?

I'll see what I can dig up without referencing enterprise internal infrastructure. Stay tuned!

[svifred]

This seems like it would be pretty helpful. Do you have an example of a schema behind a CDN that rejects requests without a User-Agent to test against?

@datho7561 You may use https://schema.bankdata.dk/schemas/automation-highway-yaml/1/schema.json

• curl -v https://schema.bankdata.dk/schemas/automation-highway-yaml/1/schema.json results in 403 Forbidden with a CloudFlare error page.
• curl -v -H "User-Agent: yaml-language-server/dev" https://schema.bankdata.dk/schemas/automation-highway-yaml/1/schema.json results in 200 OK and the schema.

Please note, that not all values of user agent is treated equally, meaning yaml-language-server is accepted, other values may not be.

[datho7561]

Verified it works with that schema you linked. Thanks, Steffen!

I'm going to replicate this on the vscode-yaml side, since on VS Code, the language server forwards the request to the client to resolve, and the code on the client that does the resolving also doesn't set a User-Agent.