Make CRLF required by default for HTTP/1.x request lines, headers, and trailers

[DeagleGross]

This is a behavioral change, but users will still be able to configure the server behavior to their desire. Per RFC CRLF is sequence of characters included in the grammar, and there is a note that LF MAY recognize a single LF:

Although the line terminator for the start-line and fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR.

Closes #66724

[Copilot]

Pull request overview

This PR changes Kestrel’s HTTP/1.x parsing behavior to require CRLF line terminators by default (aligning with RFC 9112), while preserving an AppContext switch to restore the legacy behavior that accepts bare LF terminators.

Changes:

• Flip the default value of Microsoft.AspNetCore.Server.Kestrel.DisableHttp1LineFeedTerminators so CRLF-only is the default when the switch is not set.
• Update HttpParser’s public constructor to match the new default and document how to restore LF-accepting behavior.
• Add tests validating default/switch-driven behavior using isolated remote execution.

Show a summary per file

File	Description
src/Servers/Kestrel/Core/test/HttpParserTests.cs	Adds RemoteExecutor-based tests for default/switch behavior around LF vs CRLF header terminators.
src/Servers/Kestrel/Core/src/KestrelServerOptions.cs	Changes the switch default logic (unset => CRLF required) and updates inline documentation to reflect .NET 11 behavior.
src/Servers/Kestrel/Core/src/Internal/Http/HttpParser.cs	Updates the public ctor default behavior and adds remarks documenting the new default and opt-out switch.

Copilot's findings

Comments suppressed due to low confidence (1)

src/Servers/Kestrel/Core/test/HttpParserTests.cs:1019

• Same as above: use KestrelServerOptions.DisableHttp1LineFeedTerminatorsSwitchKey instead of duplicating the AppContext switch key string literal.

        using var remoteHandle = RemoteExecutor.Invoke(static () =>
        {
            AppContext.SetSwitch("Microsoft.AspNetCore.Server.Kestrel.DisableHttp1LineFeedTerminators", false);

            var parser = new HttpParser<RequestHandler>(showErrorDetails: true);
            var buffer = new ReadOnlySequence<byte>(Encoding.ASCII.GetBytes("a:b\n\r\n"));

• Files reviewed: 3/3 changed files
• Comments generated: 1