Diagnostic infra doesn't respect `CLICOLOR_FORCE=1`

[fmease]

We've recently migrated to anstream/anstyle in #147207.

When lowering rustc's ColorConfig to anstream's/colorchoice's ColorChoice, we utilize a method called to_color_choice which interestingly isn't a straightforward 1:1 mapping as you might've thought:

rust/compiler/rustc_errors/src/emitter.rs

Lines 591 to 604 in 95aeb46

	pub fn to_color_choice(self) -> ColorChoice {
	match self {
	ColorConfig::Always => {
	if io::stderr().is_terminal() {
	ColorChoice::Always
	} else {
	ColorChoice::AlwaysAnsi
	}
	}
	ColorConfig::Never => ColorChoice::Never,
	ColorConfig::Auto if io::stderr().is_terminal() => ColorChoice::Auto,
	ColorConfig::Auto => ColorChoice::Never,
	}
	}

Most importantly, instead of mapping Auto to Auto, it maps Auto to Never if STDERR isn't a terminal. I'm not sure why this choice was made. Regardless, it leads to us not fully utilizing anstream's "color capability detection".

For instance, it means that CLICOLOR_FORCE=1 (cc https://bixense.com/clicolors/) doesn't work: CLICOLOR_FORCE=1 rustc erroneous.rs 2>&1 >/dev/null | cat¹ or CLICOLOR_FORCE=1 rustc erroneous.rs 2>/dev/null doesn't force color as cat / the file is a "non-terminal" and thus to_color_choice maps it to Never instead of asking anstyle-query.

Note that most if not all other detection mechanisms of anstyle-query do work for rustc/rustdoc. E.g., NO_COLOR=1 suppresses colors, TERM=dumb suppresses colors, in CI= TERM=dumb CI trumps TERM, in CLICOLOR= TERM=dumb CLICOLOR trumps TERM and so on.

Footnotes

1. This bash (or zsh with option multios disabled) snippet redirects the stderr to a "non-terminal" (cat). ↩

[fmease]

There's one (probably unrelated) detail of to_color_choice that I've omitted so far: It branches on stderr().is_terminal() to decide between Always and AlwaysAnsi. Sadly, I'm too unfamiliar with the matter, so I don't know whether this is necessary and if so why that would be.

I'm mentioning that because I find it a bit odd that such a general conversion function fixates on STDERR. I don't know if there are places in rustc where we use colored output when printing to STDOUT. If so, the behavior would probably be buggy or at least inconsistent with STDERR.

I know that rustdoc sometimes prints colors to STDOUT, see #148834 (comment). Ignoring the fact that rustdoc only queries STDERR for that even if !no_capture which is buggy in and of itself, to_color_choice only considering STDERR is likely problematic, too.

[epage]

It branches on stderr().is_terminal() to decide between Always and AlwaysAnsi. Sadly, I'm too unfamiliar with the matter, so I don't know whether this is necessary and if so why that would be.

The distinction between Always and AlwaysAnsi is mostly to do with old versions of Windows / Windows terminals running in compatibility mode.