I'm trying to dig through some logs and need information before and after the line I can match on. How would I do this in PowerShell ala "grep -C 2"?
In version 1, I can't wait for r2, then I get to put it on production machines :)
The PowerShell equivalent of grep is select-string. You can use the following.
cat file | select-string "pattern" -context 2
Note: this works in PowerShell v2.0 only.
Instead of using (gc $bigfile) again, which will cause PowerShell to read in $bigfile to memory on every object piped to it by the ForEach-Object cmdlet, you should probably read the file into a variable and then array index from that, like so:
$bigfile = gc 'c:\scripts\bigfile.txt'
$bigfile | Select-String "melissao" | % {$bigfile[($_.LineNumber -3)..($_.LineNumber +1)]}
Also, since the line numbering starts at 1 and array indexing starts at 0 you'll have to go backwards by 3, not 2, to get the line two spaces above "melissao", and go forwards by 1, not 2, to get the line two spaces below "melissao." Doing this will net you the 5 lines you want, "melissao" flanked by the two lines above and below it.
I'm not super familiar with grep -C 2, so I don't know if this replicates that functionality exactly.
Alas, it looks like you will have to build it yourself or wait for the next version of powershell, as it seems to have been implemented there. You can download a pre-release version of Powershell 2.0 from here.
Getting closer here- because Select-String returns MatchInfo objects which I can pull a line number out of (39017), now I just need to pull the line surrounding... so:
gc $bigfile | Select-String melissao |
%{(gc $bigfile)[($_.LineNumber -2)..($_.LineNumber +2)]}
If anyone could clean this up a bit to make it less slow, you may have the answer. Otherwise, this solution works but obviously not quickly.
Download grep for Windows, and call grep from PowerShell?
Related
I need help with a very specific question. I need to replace the following UNIX-line to a windows equivalent without installation of third-party software.
awk -F ";" '$6 ~/2019-03-11/ {print}' myInputFile.csv >> myOutputFile.csv
What the line does:
It scans myInputFile.csv, where the column delimiter is a ;, for the text 2019-03-11 in the 6th column and appends the line to myOutputFile.csv
Unfortunately, I cannot install gawk or anything like it on the client's machine.
I haven't used a windows machine for many years and am totally lost. :-(
I believe what you are after is the following in Powershell:
$ Get-Content .\myInputFile.csv | %{ if ($_.Split(';')[5] -match "2019-03-11") { $_; } } >> .\myOutputFile.csv
I will not put my hands in fire for the outcome of this.
If you want to call this from a bat-file, you have to do some ugly-ness.
findstr.exe has rather limited RegEx capabilities, but this should be sufficient:
findstr "^[^;][^;]*;[^;][^;]*;[^;][^;]*;[^;][^;]*;[^;][^;]*;2019-03-11" <myInputFile.csv >>myOutputFile.csv
Another pure cmdline way (provided there are at least 7 columns):
For /F "tokens=1-6* delims=;" %A in ('findstr "2019-03-11" ^<myInputFile.csv') do #if "%F"=="2019-03-11" (>>myOutputFile.csv Echo %A;%B;%C;%D;%E;%F;%G)
In a batch file you'll have to double all percent signs.
I'm wanting to pass arbitrary scripts to Powershell via stdin.
(In practice, I'd like to avoid having to put the script into a temporary file, but for the purposes of this question I will pipe the contents of a file to powershell.)
So I'm doing something like so (in this example, from a Windows cmd shell):
type myfile.txt | powershell -
It works if myfile.txt contains something like this:
1..3 | % { $_ *2 }
echo done
(It outputs 2\n4\n6\ndone.)
However, if I split this first statement across multiple lines like so, then Powershell simply exists without generating any output at all:
1..3 |
% { $_ *2 }
echo done
This seems to fail for any multiline statement. For example, this also fails to produce output:
1..3 | % {
$_ *2 }
echo done
I'm surprised by this since each are legal Powershell scripts that would work normally if placed into a .ps1 file and run as normal.
I've tried various things including escaping the EOL using line continuation chars, to no avail. The same effect occurs if the parent shell is Powershell, or even Python (using subprocess.Popen with stdin=PIPE). In each case, Powershell exits without any error, and the exit code is 0.
Interestingly, if I run the following, only "before.txt" gets created.
"before" | out-file before.txt
1..3 |
% { $_ *2 }
"after" | out-file after.txt
echo done
Any ideas why Powershell would have trouble reading a multi-line command, if read from stdin?
I'm going to consider this answered by this:
How to end a multi-line command in PowerShell since it shows that an extra newline is required.
However, I'm going to raise this to MS as a bug since this should not be required when reading from a non-tty, or when -NonInteractive switch is specified.
Please vote on my bug report to the Powershell team.
This is not a complete answer, but from what I can tell, the problem has to do with the input being sent in line by line.
To demonstrate the line-by-line issue, I invoke powershell this way:
powershell.exe -command "gc myfile.txt" | powershell.exe -
vs
powershell.exe -command "gc myfile.txt -raw" | powershell.exe -
The first example replicates what you see with type, the second reads the entire contents of the file, and it works as expected.
It also works from within PowerShell if you put the script contents in a string and pipe it into powershell.exe -.
I had a theory that it had to do with line-by-line input lacking line breaks, but it's not so clear cut. If that were the case, why would the first option work but not the second (removing the line break splitting the single pipeline should have no effect, while removing the line break between the pipeline and the echo should make it fail). Maybe there's something unclear about the way powershell is handling the input with or without line breaks.
I hope can you help me, essentially, I'm looking for the Powershell equivalent of the awk command:
awk '/"Box11"/ { print $0 }' test.txt|awk '{ SUM += $4} END { print SUM} '
What his does is print lines that contain the string Box11, then piping it to another awk that prints the total of the 4th column (delimited by spaces).
Multiple ways of doing it but this would do the trick:
Get-Content c:\temp\test.txt | Where-Object{$_ -match '"Box11"'} |
ForEach-Object{($_ -split "\s+")[3]} | Measure-Object -Sum |
Select-Object -ExpandProperty Sum
Get a string array of the file. For each line that contains the string "Box11" we split the line on each group of spaces. Then pass the 4 element of each match to Measure-Object.
A short hand, if you value that, would look like this:
gc c:\temp\test.txt | ?{$_ -match '"Box11"'} | %{($_ -split "\s+")[3]} |
Measure -Sum | Select -Exp Sum
If this file/string input had header this would be a good start as well. Assuming of course that your file is delimited with one space exactly.
Get-Content c:\temp\test.txt | ConvertFrom-Csv -Delimiter " "
I know this post is old but i thought I'd add to this. Currently if yo have WSL (windows sub system for Linux) enabled, (windows 10 all version on systems that support virtualization, in the turn windows features on) with a distribution installed in the subsystem. You can can call Linux commands directly from windows
wsl -e awk '/"Box11"/{sum += $4} END{print sum}' test.txt
(borrowed from #Ed Moritn)
( or any awk command of your choice. )
Basically cmd or PowerShell takes the command and pipes it into the subsystem and the results are returned (bit of an over simplification but in effect accurate). But the -e flag allows you to execute the command without opening an instance.
edit
Since writing this initial response I have found two answers which are better solutions. The first is GNUwin32 This is a collection of Gnutils which have been ported to windows standalone .exe files including sed, awk, grep and many more, allowing you to call get-childitem | awk.exe '{print $1}' directly. These tools are fully portable with no installation required. The second option is Msys32, a platform that grew out of chocolatey (though it is almost fully code in dependant now) designed for cross compiling binaries. Once installed in the /bin folder, are many Linux utilities as exe files. most of these executable can be pulled from the bin and are portable with no required installation of dependencies. The reason msys32 is preferred (in my books) over the gnuwin32 is the fact that that gnuwin32 has gawk version 3.1 and msys32 has nawk and gawk vs 5.1.
You can get get awk for Windows now. I have been using it as a direct replacement and haven't had any problems yet.
It can be easily installed via Chocolatey
In Linux we can:
grep 'MemTotal' /proc/meminfo | sed -e 's/MemTotal://' -e 's/ kB//'
What are the alternatives in Windows command prompt and PowerShell? I will run this command from ruby.
I have tried using:
wmic memorychip get /VALUE | findstr "Capacity"
to get the prefixed output of available bytes. But I need to get only numeric value out of it. And I could not find anything like sed in Windows.
For a simple PowerShell answer this simple approach is brought to you by "Deranged PowerShell Zealot" from BrianDesmond.com. Look at the comments and not the blog post.
(Get-WmiObject Win32_PhysicalMemory | measure-object Capacity -sum).sum/1gb
That will return the GB value of memory installed on the computer. You can substitute 1gb for 1mb depending on your output needs.
If you are looking for a batch type solution consider adding that tag but understand that those solutions will most likely require for loops to parse the string tokens (I dont know for sure) so getting a one liner might not be as clean as you expect.
I'm a newbie to PowerShell. What's wrong with my script below? It's not wanting to emit the value of $config. However, when I wrap that command in double quotes, everything looks okay.
param($config, $logfolder)
# Must run log analysis in chronological order.
ls $logfolder | Sort-Object LastWriteTime | % {
perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile="$($_.FullName)" -config=$config update
}
# Execute with - .\regen-logs.ps1 webgenesis "C:\inetpub\logs\LogFiles\W3SVC5"
# Returns for each file - Error: Couldn't open config file "awstats.config.conf" nor "awstats.conf" after searching in path "D:\Websites\_awstats\wwwroot\cgi-bin,/etc/awstats,/usr/local/etc/awstats,/etc,/etc/opt/awstats": No such file or directory
As-is, what gets emitted and executed seems to have "-config=$config" passed as an argument. At least, that's my best guess. I don't know if $_ is working correctly either.
If I put quotes around the perl command like so, I get the command I do want to execute.
ls $logfolder | Sort-Object LastWriteTime | % {
"perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile=`"$($_.FullName)`" -config=$config update"
}
# Outputs for each log file something like - perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile="C:\inetpub\logs\LogFiles\W3SVC5\u_ex110602.log" -config=webgenesis update
If putting quotes around it produces the correct commandline, one way to execute the contents of a string is with Invoke-Expression (alias iex):
$v = "myexe -myarg1 -myarg2=$someVar"
iex $v
Put double quotes around "-config=$config". Without this, PowerShell will interpret -config=$config as one string argument that just happens to contain a $ sign in it.
I think you need to start your perl command out with & so that PowerShell interprets things as a command and not a string.
& perl D:\Websites\_awstats\wwwroot\cgi-bin\awstats.pl -LogFile=`"$($_.FullName)`" -config=$config update
Also, see: Run a program in a foreach