I hope can you help me, essentially, I'm looking for the Powershell equivalent of the awk command:
awk '/"Box11"/ { print $0 }' test.txt|awk '{ SUM += $4} END { print SUM} '
What his does is print lines that contain the string Box11, then piping it to another awk that prints the total of the 4th column (delimited by spaces).
Multiple ways of doing it but this would do the trick:
Get-Content c:\temp\test.txt | Where-Object{$_ -match '"Box11"'} |
ForEach-Object{($_ -split "\s+")[3]} | Measure-Object -Sum |
Select-Object -ExpandProperty Sum
Get a string array of the file. For each line that contains the string "Box11" we split the line on each group of spaces. Then pass the 4 element of each match to Measure-Object.
A short hand, if you value that, would look like this:
gc c:\temp\test.txt | ?{$_ -match '"Box11"'} | %{($_ -split "\s+")[3]} |
Measure -Sum | Select -Exp Sum
If this file/string input had header this would be a good start as well. Assuming of course that your file is delimited with one space exactly.
Get-Content c:\temp\test.txt | ConvertFrom-Csv -Delimiter " "
I know this post is old but i thought I'd add to this. Currently if yo have WSL (windows sub system for Linux) enabled, (windows 10 all version on systems that support virtualization, in the turn windows features on) with a distribution installed in the subsystem. You can can call Linux commands directly from windows
wsl -e awk '/"Box11"/{sum += $4} END{print sum}' test.txt
(borrowed from #Ed Moritn)
( or any awk command of your choice. )
Basically cmd or PowerShell takes the command and pipes it into the subsystem and the results are returned (bit of an over simplification but in effect accurate). But the -e flag allows you to execute the command without opening an instance.
edit
Since writing this initial response I have found two answers which are better solutions. The first is GNUwin32 This is a collection of Gnutils which have been ported to windows standalone .exe files including sed, awk, grep and many more, allowing you to call get-childitem | awk.exe '{print $1}' directly. These tools are fully portable with no installation required. The second option is Msys32, a platform that grew out of chocolatey (though it is almost fully code in dependant now) designed for cross compiling binaries. Once installed in the /bin folder, are many Linux utilities as exe files. most of these executable can be pulled from the bin and are portable with no required installation of dependencies. The reason msys32 is preferred (in my books) over the gnuwin32 is the fact that that gnuwin32 has gawk version 3.1 and msys32 has nawk and gawk vs 5.1.
You can get get awk for Windows now. I have been using it as a direct replacement and haven't had any problems yet.
It can be easily installed via Chocolatey
Related
I am creating a PowerShell script just to backup my WSL distros, but when I try to run the command with variables it's not working, it displays the usage text as though I provided the incorrect arguments.
$DistrosArray1 = (wsl -l --quiet) | where {$_ -ne ""}
$DistrosArray2 = 'Arch', 'Ubuntu-22.04', 'docker-desktop-data', 'docker-desktop'
$CheckArrayDifference = Compare-Object -ReferenceObject $DistrosArray1 -DifferenceObject $DistrosArray2 -PassThru
echo $CheckArrayDifference
# Does not return anything (there is no difference)
foreach ($Distro in $DistrosArray1) {
wsl --export $Distro "$Distro.tar"
# This method is not working
}
foreach ($Distro in $DistrosArray2) {
wsl --export $Distro "$Distro.tar"
# This method is working
}
It sounds like you are running into complications from issue #4607 -- The wsl.exe command outputs some oddly mangled UTF16 encoding that creates issues when attempting to process it from PowerShell (or even from inside WSL).
This is now fixed in the latest WSL Preview release 0.64.0, but you do have to "opt-in" to the fix so that older workarounds (like the ones #Bender and I provided) don't inadvertently break.
Simply set:
$env:WSL_UTF8=1
... before your code, and WSL will no longer spit out the "mangled UTF16."
Other examples in my answers to:
Why cannot I match for strings from wsl.exe output?
Powershell - Strange WSL output string encoding
How to ask WSL to check if the distribution exists, using Bash and wsl.exe?
Older solution:
Let's simplify the problem and make a "safe" example that doesn't attempt to export:
$DistrosArray1 = (wsl -l --quiet) | where {$_ -ne ""}
wsl -d $DistrosArray1[0]
Results in:
There is no distribution with the supplied name.
I've successfully used the method in this comment to handle it. For this particular example:
$console = ([console]::OutputEncoding)
[console]::OutputEncoding = New-Object System.Text.UnicodeEncoding
$DistrosArray1 = (wsl -l --quiet) | where {$_ -ne ""}
wsl -d $DistrosArray1[0]
This will run the first distribution in the list correctly.
Reset the encoding after with:
[console]::OutputEncoding = $console
That shouldn't be a problem for most non-interactive scripts since it will just be the final line of the "wrapper", but as #BendertheGreatest pointed out in the comments, it's a critical step.
This is part of a known issue with wsl.exe output. Here is what I put together from workarounds provided on that issue:
$DistrosArray1 = wsl -l --quiet | wsl iconv -c -f utf16 -t ascii
foreach ($Distro in $DistrosArray1) {
wsl --export $Distro "$Distro.tar"
}
Unfortunately I could not get this working with a conversion to UTF8 (changing ascii to utf8 produces additional garbage characters although they are consistent and detectable in my limited testing), so only any characters outside of the ASCII range will likely cause problems for you.
I need help with a very specific question. I need to replace the following UNIX-line to a windows equivalent without installation of third-party software.
awk -F ";" '$6 ~/2019-03-11/ {print}' myInputFile.csv >> myOutputFile.csv
What the line does:
It scans myInputFile.csv, where the column delimiter is a ;, for the text 2019-03-11 in the 6th column and appends the line to myOutputFile.csv
Unfortunately, I cannot install gawk or anything like it on the client's machine.
I haven't used a windows machine for many years and am totally lost. :-(
I believe what you are after is the following in Powershell:
$ Get-Content .\myInputFile.csv | %{ if ($_.Split(';')[5] -match "2019-03-11") { $_; } } >> .\myOutputFile.csv
I will not put my hands in fire for the outcome of this.
If you want to call this from a bat-file, you have to do some ugly-ness.
findstr.exe has rather limited RegEx capabilities, but this should be sufficient:
findstr "^[^;][^;]*;[^;][^;]*;[^;][^;]*;[^;][^;]*;[^;][^;]*;2019-03-11" <myInputFile.csv >>myOutputFile.csv
Another pure cmdline way (provided there are at least 7 columns):
For /F "tokens=1-6* delims=;" %A in ('findstr "2019-03-11" ^<myInputFile.csv') do #if "%F"=="2019-03-11" (>>myOutputFile.csv Echo %A;%B;%C;%D;%E;%F;%G)
In a batch file you'll have to double all percent signs.
In Linux we can:
grep 'MemTotal' /proc/meminfo | sed -e 's/MemTotal://' -e 's/ kB//'
What are the alternatives in Windows command prompt and PowerShell? I will run this command from ruby.
I have tried using:
wmic memorychip get /VALUE | findstr "Capacity"
to get the prefixed output of available bytes. But I need to get only numeric value out of it. And I could not find anything like sed in Windows.
For a simple PowerShell answer this simple approach is brought to you by "Deranged PowerShell Zealot" from BrianDesmond.com. Look at the comments and not the blog post.
(Get-WmiObject Win32_PhysicalMemory | measure-object Capacity -sum).sum/1gb
That will return the GB value of memory installed on the computer. You can substitute 1gb for 1mb depending on your output needs.
If you are looking for a batch type solution consider adding that tag but understand that those solutions will most likely require for loops to parse the string tokens (I dont know for sure) so getting a one liner might not be as clean as you expect.
I would like to translate the following Unix 1 Liner to PowerShell.
Synopsis of the command:
This command will search recursively form the PWD (pressent working directory) for any file with the extenstion .jsp, and look inside the file for a simple string match of 'logoutButtonForm'. If it finds a match, it will print the file name and the text that it matched.
find . -name "*.jsp" -exec grep -aH "logoutButtonForm" {}\;
I am new to power shell and have done some googling/binging but have not found a good answer yet.
ls . -r *.jsp | Select-String logoutButtonForm -case
I tend to prefer -Filter over -Include. Guess I never trusted the -Exclude/-Include parameters after observing buggy behavior in PowerShell 1.0. Also, -Filter is significantly faster than using -Include.
I'm trying to dig through some logs and need information before and after the line I can match on. How would I do this in PowerShell ala "grep -C 2"?
In version 1, I can't wait for r2, then I get to put it on production machines :)
The PowerShell equivalent of grep is select-string. You can use the following.
cat file | select-string "pattern" -context 2
Note: this works in PowerShell v2.0 only.
Instead of using (gc $bigfile) again, which will cause PowerShell to read in $bigfile to memory on every object piped to it by the ForEach-Object cmdlet, you should probably read the file into a variable and then array index from that, like so:
$bigfile = gc 'c:\scripts\bigfile.txt'
$bigfile | Select-String "melissao" | % {$bigfile[($_.LineNumber -3)..($_.LineNumber +1)]}
Also, since the line numbering starts at 1 and array indexing starts at 0 you'll have to go backwards by 3, not 2, to get the line two spaces above "melissao", and go forwards by 1, not 2, to get the line two spaces below "melissao." Doing this will net you the 5 lines you want, "melissao" flanked by the two lines above and below it.
I'm not super familiar with grep -C 2, so I don't know if this replicates that functionality exactly.
Alas, it looks like you will have to build it yourself or wait for the next version of powershell, as it seems to have been implemented there. You can download a pre-release version of Powershell 2.0 from here.
Getting closer here- because Select-String returns MatchInfo objects which I can pull a line number out of (39017), now I just need to pull the line surrounding... so:
gc $bigfile | Select-String melissao |
%{(gc $bigfile)[($_.LineNumber -2)..($_.LineNumber +2)]}
If anyone could clean this up a bit to make it less slow, you may have the answer. Otherwise, this solution works but obviously not quickly.
Download grep for Windows, and call grep from PowerShell?