__PSDumpAMSILogContent debug logging sent to stdout about powershell HOT 13 OPEN

The WG reviewed this and agreed that this is a bucket 3 breaking change (thanks @mklement0 for the code search) and accept making the change to write to STDERR. We believe the env var is a carryover from WinPS5.1 and used as a test hook.

from powershell.

@rhubarb-geek-nz your usage is dumping to the console interactively. My question is if there's any tools that rely on it going to stdout which would break if we change it to stderr.

from powershell.

The WG re-discussed this based on discovery of the behavior. As this was originally intended as test code and not intended to be used by users, we recommend that this code simply be removed.

AMSI on Linux doesn't exist until there is a malware scanner that supports something like the AMSI interface. If that were to exist, this specific code doesn't play a role in that anyways as the AMSI logging would simply need to be updated to call a Linux dll instead of just the Windows one.

from powershell.

I agree that Console.Error.WriteLine() is the better choice.

As an aside: an outside caller - unfortunately - sees all of PowerShell's output streams including Console.WriteLine() and PowerShell error output via stdout; see:

• #7989 (comment)

Console.Error.WriteLine() unconditionally writes to stderr, while PowerShell error output only goes to stderr if the caller uses a 2> redirection.

from powershell.

... while PowerShell error output only goes to stderr if the caller uses a 2> redirection.

Of course it would do that™ .....

$ pwsh -c "Write-Error 'foo'" >out.1 2>out.2
$ ls -l
total 4
-rw-r--r-- 1 onlyme users  0 Apr 18 19:18 out.1
-rw-r--r-- 1 onlyme users 35 Apr 18 19:18 out.2
$ rm out.*
$ pwsh -c "Write-Error 'foo'" >out.1
$ ls -l
-rw-r--r-- 1 onlyme users 35 Apr 18 19:19 out.1

from powershell.

while PowerShell error output only goes to stderr if the caller uses a 2> redirection.

I don't think that is true. A process has no idea if any redirection has occurred because at the process level there is no such thing as redirection. All a process gets is stdin, stdout and stderr file-handles/descriptors, it has no idea if the parent did any "redirection".

My investigation suggests that always goes to stderr unless the process thinks it is somehow interactive, it looks like it determines that by seeing if stdin is a virtual terminal or not, perhaps isatty.

#!/bin/sh -x

ssh -T </dev/null >stdout 2>stderr localhost pwsh -c "Write-Error 'foo'"

cat stdout

cat stderr

ssh -t </dev/null >stdout 2>stderr localhost pwsh -c "Write-Error 'foo'"

cat stdout

cat stderr

ssh -T >stdout 2>stderr localhost pwsh -c "Write-Error 'foo'"

cat stdout

cat stderr

ssh -t >stdout 2>stderr localhost pwsh -c "Write-Error 'foo'"

cat stdout

cat stderr

Gives the result

+ ssh -T localhost pwsh -c Write-Error 'foo'
+ cat stdout
+ cat stderr
Write-Error: foo
+ ssh -t localhost pwsh -c Write-Error 'foo'
+ cat stdout
+ cat stderr
Pseudo-terminal will not be allocated because stdin is not a terminal.
Write-Error: foo
+ ssh -T localhost pwsh -c Write-Error 'foo'
+ cat stdout
+ cat stderr
Write-Error: foo
+ ssh -t localhost pwsh -c Write-Error 'foo'
+ cat stdout
Write-Error: foo
+ cat stderr
Connection to localhost closed.

The only case where the error message went to stdout was when stdin was a virtual terminal.

The reason I am using ssh is to control the virtual terminal and demonstrate that as far as pwsh is concerned, there is no redirection going on because the shell redirection occurred on the client, not on the server. The pwsh process was just told this is your stdin, stdout and stderr.

from powershell.

This is indeterminate because nohup set stdin and stdout to the same file, and changed stdin to /dev/null

$ nohup pwsh -c "Write-Error 'foo'" > stdout
nohup: ignoring input and redirecting stderr to stdout
$ cat stdout
Write-Error: foo

But given that stdin is not a terminal I expect that pwsh was writing to stderr

$ tty
/dev/pts/1
$ nohup tty > stdout
nohup: ignoring input and redirecting stderr to stdout
$ cat stdout
not a tty

from powershell.

Yes, redirected means: a given standard stream targets something other than a terminal (which, from a shell, can be achieved via a redirection, e.g., > or 2> or, for stdout only, via a pipeline, |, and, in PowerShell, with direct variable assignment (vs. command substitution in POSIX-like shells)).

That the caller may merge standard streams is incidental to this discussion, and, from what I can tell, so is whether the redirection is mediated via another utility, such as ssh.

It is a condition utilities can easily check for and it is common for utilities to modify their behavior based on this distinction (e.g. ls (multi-column format) vs. ls | cat (line-oriented format)), though what PowerShell does is certainly unusual:

I haven't looked into the implementation, but PowerShell clearly detects when stderr isn't connected to a terminal and only then sends its error output there; a simpler demo (works analogously with /dev/null from a POSIX-compatible shell):
pwsh -noprofile -c 'Write-Output stdout; Write-Error error' >$null (no output)
vs.
pwsh -noprofile -c 'Write-Output stdout; Write-Error error' 2>$null (prints 'stdout'))

All of this is a tangent, however, which detracts from the original issue, so I'm hiding this comment, and I suggest you hide yours too.

from powershell.

Agree that the logging should be going to stderr, but not clear to me how this might affect existing tools relying on this. I don't see any tests using this special env var.

from powershell.

I don't see any tests using this special env var.

There is a whole world out there! It is built into every production release, and anyone can use it for whatever purposes.

#21473 (comment)

I was using it to track down a performance issue. It is quite 'illuminating' to see the volume of traffic sent to AMSI during what one might consider are normal script operations.

from powershell.

A global GitHub code search for __PSDumpAMSILogContent shows only its use in the PowerShell repo itself (and forks of it): https://github.com/search?q=__PSDumpAMSILogContent&type=code

Also, the variable isn't documented.

from powershell.

I made the change, but now noticed an unexpected behavior. With the Console.WriteLine() calls now Console.Error.WriteLine(), you don't get ANY output in the interactive shell. STDERR is only written if you call pwsh itself. It might make more sense to just remove this env var altogether as it's not useful expect as a test hook.

from powershell.

It was useful to demonstrate that AMSI logging was implemented on Linux,

I think that writing a real AMSI module to demonstrate the volume and verbosity of the PowerShell logging would be an alternative approach. ( outside of the scope of this PowerShell project ). While not so easy to demonstrate with a simple script it would show what PowerShell is leaking from the process.

from powershell.