What's the equivalent of xargs in PowerShell? - powershell

The POSIX-defined xargs command takes all of the items it receives from standard input and passes them as command-line arguments to the command it receives on it's own command line. E.g: grep -rn "String" | xargs rm.
What's the equivalent in PowerShell?
The following questions all ask this:
Convert xargs Bash command to PowerShell?
What is the PowerShell equivalent to this Bash command?
but there is no correct answer because all the answers either use ForEach-Object, which will process items one-at-a-time (like xargs -n1) which gets the desired result for the examples given, or store the intermediate result in a variable, which offends my functional commandline-fu.

There are two ways that I've found. The first is probably more idiomatic PowerShell, and the second is more true to the pipe-based spirit of xargs.
As an example, let's say we want to pass all our cat pics to myapp.exe.
Method #1: Command substitution
You can do something similar to using $(command substitution) in sh by embedding your pipeline in the command string:
&"myapp.exe" #(Get-ChildItem -Recurse -Filter *.jpg | Another-Step)
The #(...) creates an array from the command inside it, and PowerShell automatically expands arrays passed to & into seperate command-line parameters.
However, this does not really answer the question, because it will only work if you have control over the command you want to pass to, which may not be the case.
Method #2: True piping
You can also construct a "double pipeline" by having a sub-expression to pipe your objects, collecting them to an array, and then piping the array to your final command.
,#(Get-ChildItem -Recurse -Filter *.jpg | Another-Step) | %{&"myapp.exe" $_}
The #(...) as before collects the items into an array, and the array is then piped to the final command which is invoked using % (ForEach-Object). Ordinarily, this would then loop over each item individually, because PowerShell will automatically flatten the array when it's piped, but this can be avoided by prepending the , operator. The $_ special variable is then used as normal to embed the passed array.
So the key is to wrap the pipeline you want to collect in ,#(...), and then pipe that to something in %{...}.

I've been using this filter for the basic xargs execution.
filter xargs { ($h,$t) = $args; & $h ($t + $_) }
which is roughly equivalent to:
filter xargs { & $args[0] ($args[1..$args.length] + $_) }
Examples
docker ps -q | xargs docker stop
gem list | % { $_.split()[0] } | xargs gem uninstall -aIx

I think the closest you can get to is to use Splatting. In this example I use foreach loop to collect all pipelined values to extend unnamed positional parameters:
function Xargs {
param(
$Cmd
)
process {
$args += ,$_
}
end {
& $Cmd #args
}
}
Now test:
PS> '-c','"print(123)"' | Xargs python
123
Note that this doesn't work with named parameters nor flags -- it only passes pure positional parameters, so it should work best with non-cmdlets.
That also means I highly discourage you from trying '-WhatIf' | Xargs Remove-WholeSystem.

I hade issue like this before
In PowerShell you can use:
docker ps -aq | ForEach-Object { docker rm $_ }

Related

Adding Arguments to PowerShell Alias Functions

I just installed grep on PowerShell through chocolatey and using it, I could search for a string I want from any text as shown below:
However, I don't want to always have to type grep --color, hence I tried to make an alias on my $profile like this:
Function grep($a1){
grep --color $a1
}
but then I would get an error:
So I tried:
Set-Alias grep "grep --color"
but then I get yet another error:
I can't think of anything else to make this work, hence I'd really appreciate any help.
Aliases in PowerShell are mere alternative names for other commands, so you cannot use them to include arguments to pass to these other commands.
You therefore indeed need a function, but since you're naming it the same as the external program you're wrapping, you need to disambiguate so as to avoid an infinite recursion:
function grep {
$externalGrep = Get-Command -Type Application grep
if ($MyInvocation.ExpectingInput) { # pipeline (stdin) input present
# $args passes all arguments through.
$input | & $externalGrep --color $args
} else {
& $externalGrep --color $args
}
}
Note:
Use of the automatic $input variable to relay pipeline (stdin) input means that this input is collected (buffered) in full first. More effort is needed to implement a true streaming solution - see this answer for an example.
Alternatively - at least on Unix-like platforms - you can set an environment variable to control grep's coloring behavior, which may obviate the need for a function wrapper; the equivalent of the --color parameter is to set $env:GREP_COLOR='always' (the other supported values are 'never' and 'auto').

Awk command to Powershell equivalent

I hope can you help me, essentially, I'm looking for the Powershell equivalent of the awk command:
awk '/"Box11"/ { print $0 }' test.txt|awk '{ SUM += $4} END { print SUM} '
What his does is print lines that contain the string Box11, then piping it to another awk that prints the total of the 4th column (delimited by spaces).
Multiple ways of doing it but this would do the trick:
Get-Content c:\temp\test.txt | Where-Object{$_ -match '"Box11"'} |
ForEach-Object{($_ -split "\s+")[3]} | Measure-Object -Sum |
Select-Object -ExpandProperty Sum
Get a string array of the file. For each line that contains the string "Box11" we split the line on each group of spaces. Then pass the 4 element of each match to Measure-Object.
A short hand, if you value that, would look like this:
gc c:\temp\test.txt | ?{$_ -match '"Box11"'} | %{($_ -split "\s+")[3]} |
Measure -Sum | Select -Exp Sum
If this file/string input had header this would be a good start as well. Assuming of course that your file is delimited with one space exactly.
Get-Content c:\temp\test.txt | ConvertFrom-Csv -Delimiter " "
I know this post is old but i thought I'd add to this. Currently if yo have WSL (windows sub system for Linux) enabled, (windows 10 all version on systems that support virtualization, in the turn windows features on) with a distribution installed in the subsystem. You can can call Linux commands directly from windows
wsl -e awk '/"Box11"/{sum += $4} END{print sum}' test.txt
(borrowed from #Ed Moritn)
( or any awk command of your choice. )
Basically cmd or PowerShell takes the command and pipes it into the subsystem and the results are returned (bit of an over simplification but in effect accurate). But the -e flag allows you to execute the command without opening an instance.
edit
Since writing this initial response I have found two answers which are better solutions. The first is GNUwin32 This is a collection of Gnutils which have been ported to windows standalone .exe files including sed, awk, grep and many more, allowing you to call get-childitem | awk.exe '{print $1}' directly. These tools are fully portable with no installation required. The second option is Msys32, a platform that grew out of chocolatey (though it is almost fully code in dependant now) designed for cross compiling binaries. Once installed in the /bin folder, are many Linux utilities as exe files. most of these executable can be pulled from the bin and are portable with no required installation of dependencies. The reason msys32 is preferred (in my books) over the gnuwin32 is the fact that that gnuwin32 has gawk version 3.1 and msys32 has nawk and gawk vs 5.1.
You can get get awk for Windows now. I have been using it as a direct replacement and haven't had any problems yet.
It can be easily installed via Chocolatey

Compressing to tar.xz using 7-zip through a pipe on windows

My command line is this (powershell):
$7z ="`"c:\Program Files\7-Zip\7z.exe`""
&$7z a -r -ttar -bd -so . | &$7z a -r -txz -bd $archive -si
The produced archive file indeed contains a tar file, but that tar file is corrupt.
Note, that breaking the pipe into two commands works correctly:
&$7z a -r -ttar -bd ${archive}.tmp .
&$7z a -r -txz -bd $archive ${archive}.tmp
The produced archive is perfectly valid.
So, what is wrong with my pipeline?
(I am using Powershell)
Nothing is wrong with your pipeline it is the way that the pipeline works that's causing the error.
PowerShell pipe works in an asynchronous way. Meaning that output of the first command is available to the second command immediately one object at the time even if the first one has not finished executing, See here.
Both Unix and PowerShell pipes operate in the same way. The reason why you might be seeing a difference from Unix to PowerShell is the way in which they go about it is different.
Unix passes Strings between the commands. Where as a Powershell pipe will pass full-fledged .net object between commands. This difference in the data type being past between command will be why it works on unix and not in PowerShell. If 7z.exe can not huddle these .net objects correctly the files will be come corrupt, See here.
Try adding | %{ "$_" } in between the pipes like
&$7z a -r -ttar -bd -so . | %{ "$_" } | &$7z a -r -txz -bd $archive -si
The point is that the second call to 7z expects unmodified data on STDIN, but PowerShell is converting the output from the first call to 7z to (multiple) (string) objects. % is an alias for foreach-object, so what the additional command does is to loop over each object and convert it to a plain string before passing it on to the second call to 7z.
Edit: Reading through PowerShell’s Object Pipeline Corrupts Piped Binary Data it looks to me now as if my suggestion would not work, and there's also no way to fix it. Well, other than wrapping the whole pipeline into a cmd /c "..." call to make cmd and not PowerShell handle the pipeline.
Edit2: I also was trying this solution from the PowerShell Cookbook, but it was very slow.
In the end, I created a .cmd script with the 7z pipes that I'm calling from my PowerShell script.

How do I reference variables when executing a shell command in PowerShell?

I'm a newbie to PowerShell. What's wrong with my script below? It's not wanting to emit the value of $config. However, when I wrap that command in double quotes, everything looks okay.
param($config, $logfolder)
# Must run log analysis in chronological order.
ls $logfolder | Sort-Object LastWriteTime | % {
perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile="$($_.FullName)" -config=$config update
}
# Execute with - .\regen-logs.ps1 webgenesis "C:\inetpub\logs\LogFiles\W3SVC5"
# Returns for each file - Error: Couldn't open config file "awstats.config.conf" nor "awstats.conf" after searching in path "D:\Websites\_awstats\wwwroot\cgi-bin,/etc/awstats,/usr/local/etc/awstats,/etc,/etc/opt/awstats": No such file or directory
As-is, what gets emitted and executed seems to have "-config=$config" passed as an argument. At least, that's my best guess. I don't know if $_ is working correctly either.
If I put quotes around the perl command like so, I get the command I do want to execute.
ls $logfolder | Sort-Object LastWriteTime | % {
"perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile=`"$($_.FullName)`" -config=$config update"
}
# Outputs for each log file something like - perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile="C:\inetpub\logs\LogFiles\W3SVC5\u_ex110602.log" -config=webgenesis update
If putting quotes around it produces the correct commandline, one way to execute the contents of a string is with Invoke-Expression (alias iex):
$v = "myexe -myarg1 -myarg2=$someVar"
iex $v
Put double quotes around "-config=$config". Without this, PowerShell will interpret -config=$config as one string argument that just happens to contain a $ sign in it.
I think you need to start your perl command out with & so that PowerShell interprets things as a command and not a string.
& perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile=`"$($_.FullName)`" -config=$config update
Also, see: Run a program in a foreach

Translate a Unix1Liner to PowerShell

I would like to translate the following Unix 1 Liner to PowerShell.
Synopsis of the command:
This command will search recursively form the PWD (pressent working directory) for any file with the extenstion .jsp, and look inside the file for a simple string match of 'logoutButtonForm'. If it finds a match, it will print the file name and the text that it matched.
find . -name "*.jsp" -exec grep -aH "logoutButtonForm" {}\;
I am new to power shell and have done some googling/binging but have not found a good answer yet.
ls . -r *.jsp | Select-String logoutButtonForm -case
I tend to prefer -Filter over -Include. Guess I never trusted the -Exclude/-Include parameters after observing buggy behavior in PowerShell 1.0. Also, -Filter is significantly faster than using -Include.